Subject: [Moq] Agenda topics for side meeting on Tuesday

From: Spencer Dawkins at IETF <[email-masked]>
Date: Sun, 7 Nov 2021 12:36:38 -0600

Dear MOQ fans,

It's time to put together an agenda for the Tuesday meeting. I'd like for
this meeting to be a working session, not a forum for presentations, so
please keep that in mind.

As a high-order bit, I think we've spent a lot of time thinking about use
cases here, and I'd like to push down another level, and make sure we
understand what the work for the IETF (and possibly for the IRTF) is.

I know what the protocol work is for my use cases:

   - Mapping RTP onto QUIC (many options exist, so picking one or more
   would be helpful) This is squarely in scope for AVTCORE
   - Mapping RTCP onto QUIC (which may be the same approach as used for
   RTP, or may include things like using QUIC facilities instead of RTCP
   facilities, where those QUIC facilities exist).. Again, this is squarely in
   scope for AVTCORE
   - SDP description for RTP/RTCP over QUIC. This is squarely in scope for
   MMUSIC, as soon as we figure out the details in AVTCORE.

What is the IETF/IRTF work for your use cases?

Best,

Spencer

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 8 Nov 2021 06:13:12 -0600

So, three things I should say before the Tuesday side meeting.

First - this side meeting will be under Notewell, because it's focused on
input to the IETF.

Second - I'm planning to record the session, just so I can verify what we
said, so that I can produce decent minutes. If anyone objects, please also
volunteer to take excellent notes!

Third - James and I are most familiar with the RTP over QUIC proposals, but
we recognize that there will be commonalities between various proposals,
so  we're perfectly happy to talk about other use cases and requirement
sets. So, letting us know what you're thinking about, as requested below,
will be super useful.

Please Do The Right Thing.

Best,

Spencer

On Sun, Nov 7, 2021 at 12:36 PM Spencer Dawkins at IETF <
[email-masked]> wrote:

>    - SDP description for RTP/RTCP over QUIC. This is squarely in scope
>    for MMUSIC, as soon as we figure out the details in AVTCORE.
>

---

From: Ian Swett <[email-masked]>
Date: Mon, 8 Nov 2021 12:38:10 -0500

Hi,

I've looked at the following uses cases fairly closely for QUIC in the past.

   - Live Video Ingestion - Existing protocols don't fully meet our needs
   (ie: performance vs reliability, standardization, etc), and this is
   becoming a real problem.
   - Live Video Delivery - Currently we're pushing the edge of what HTTP
   can do efficiently, including some non-standard extensions.  This is also
   not optimized for QUIC or HTTP/3, which could improve performance.
   - WebRTC over QUIC, ie: RTP over QUIC + Datachannels over QUIC - There
   could be a number of benefits of WebRTC over QUIC.  A number of people who
   were interested in this are now focused on WebTransport, but there's still
   the matter of what you send over WebTransport, so I think there's some work
   to be done in addition to finishing WebTransport.

As with QUIC, I see we have a number of different working groups different
parts or applications could land in.  I worry that could cause
fragmentation if each sub-domain solves their problem without collaborating
closely with others.  I'm not sure if a new WG(ie: QUIC-style) is necessary
here, but I do think caution should be taken to ensure we don't reinvent
the wheel 3+ different ways.

Thanks, Ian

On Mon, Nov 8, 2021 at 7:13 AM Spencer Dawkins at IETF <
[email-masked]> wrote:

> Third - James and I are most familiar with the RTP over QUIC proposals,
> but we recognize that there will be commonalities between various
> proposals, so  we're perfectly happy to talk about other use cases and
> requirement sets. So, letting us know what you're thinking about, as
> requested below, will be super useful.
>
> --
> Moq mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/moq
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 8 Nov 2021 12:10:43 -0600

Hi, Ian,

>    to be done in addition to finishing WebTransport.
Thanks for this - it's very helpful, and it will definitely be on the list
for tomorrow's meeting.

> the wheel 3+ different ways.

Yes, exactly. It's sad when three people working in isolation solve a
similar problem in three different ways that have to be implemented three
times, but it's doubly sad when they all show up in QUIC either asking for
advice on three similar solutions, or even proposing three similar but
incompatible extensions! 😈

Best,

Spencer, who should be apologizing to Lucas for even thinking such a thing!

>>> It's time to put together an agenda for the Tuesday meeting. I'd like
>>> for this meeting to be a working session, not a forum for presentations, so
>>> As a high-order bit, I think we've spent a lot of time thinking about
>>> use cases here, and I'd like to push down another level, and make sure we

---

From: Suhas Nandakumar <[email-masked]>
Date: Mon, 8 Nov 2021 10:52:34 -0800

I would like to add one more flavor of the media over quic usecase, "Non
RTP based Interactive media delivery over quic or h3".

Thanks
Suhas

On Mon, Nov 8, 2021 at 10:11 AM Spencer Dawkins at IETF <
[email-masked]> wrote:

>>    - Live Video Ingestion - Existing protocols don't fully meet our
>>    needs (ie: performance vs reliability, standardization, etc), and this is
>>    becoming a real problem.
>>    - WebRTC over QUIC, ie: RTP over QUIC + Datachannels over QUIC -
>>    There could be a number of benefits of WebRTC over QUIC.  A number of
>>    people who were interested in this are now focused on WebTransport, but
>>    there's still the matter of what you send over WebTransport, so I think
>>    there's some work to be done in addition to finishing WebTransport.
>>
>>
>> As with QUIC, I see we have a number of different working groups
>> different parts or applications could land in.  I worry that could cause
>>> First - this side meeting will be under Notewell, because it's focused
>>> on input to the IETF.
>>>
>>> Second - I'm planning to record the session, just so I can verify what
>>> we said, so that I can produce decent minutes. If anyone objects, please
>>> also volunteer to take excellent notes!
>>>
>>>>    - Mapping RTCP onto QUIC (which may be the same approach as used
>>>>    for RTP, or may include things like using QUIC facilities instead of RTCP

---

From: Nils Ohlmeier <[email-masked]>
Date: Mon, 8 Nov 2021 10:55:37 -0800

Hi,

> On Nov 8, 2021, at 09:38, Ian Swett <ianswett=[email-masked]> wrote:
> Live Video Ingestion - Existing protocols don't fully meet our needs (ie: performance vs reliability, standardization, etc), and this is becoming a real problem.
> Live Video Delivery - Currently we're pushing the edge of what HTTP can do efficiently, including some non-standard extensions.  This is also not optimized for QUIC or HTTP/3, which could improve performance.
Yes fully agreed. I think that is why lots of people are interested in WebTransport, because it could result in some improvements in these areas.
> WebRTC over QUIC, ie: RTP over QUIC + Datachannels over QUIC - There could be a number of benefits of WebRTC over QUIC.  A number of people who were interested in this are now focused on WebTransport, but there's still the matter of what you send over WebTransport, so I think there's some work to be done in addition to finishing WebTransport.
This one keeps popping up. But so far all I have heard were some hand wavy hopes for improvements. It would be really helpful to have a list of things which don’t work in WebRTC to see if switching things over to QUIC could really fix/improve these.

One aspect is that WebRTC was specifically designed to be a P2P solution. Which I understand is a disadvantage for certain use cases.
Maybe all of this boils down to not calling this WebRTC over QUIC, which brings a lot of expectations with it, but name it different like QUIC RTC or something like that.

> As with QUIC, I see we have a number of different working groups different parts or applications could land in.  I worry that could cause fragmentation if each sub-domain solves their problem without collaborating closely with others.  I'm not sure if a new WG(ie: QUIC-style) is necessary here, but I do think caution should be taken to ensure we don't reinvent the wheel 3+ different ways.

Agreed.

Cheers
  Nils

> 
> Thanks, Ian
> 
> On Mon, Nov 8, 2021 at 7:13 AM Spencer Dawkins at IETF <[email-masked] <mailto:[email-masked]>> wrote:
> First - this side meeting will be under Notewell, because it's focused on input to the IETF. 
> 
> Second - I'm planning to record the session, just so I can verify what we said, so that I can produce decent minutes. If anyone objects, please also volunteer to take excellent notes!
> 
> Third - James and I are most familiar with the RTP over QUIC proposals, but we recognize that there will be commonalities between various proposals, so  we're perfectly happy to talk about other use cases and requirement sets. So, letting us know what you're thinking about, as requested below, will be super useful. 
> 
> On Sun, Nov 7, 2021 at 12:36 PM Spencer Dawkins at IETF <[email-masked] <mailto:[email-masked]>> wrote:
> Dear MOQ fans,
> 
> It's time to put together an agenda for the Tuesday meeting. I'd like for this meeting to be a working session, not a forum for presentations, so please keep that in mind. 
> 
> As a high-order bit, I think we've spent a lot of time thinking about use cases here, and I'd like to push down another level, and make sure we understand what the work for the IETF (and possibly for the IRTF) is.
> 
> Mapping RTP onto QUIC (many options exist, so picking one or more would be helpful) This is squarely in scope for AVTCORE
> Mapping RTCP onto QUIC (which may be the same approach as used for RTP, or may include things like using QUIC facilities instead of RTCP facilities, where those QUIC facilities exist).. Again, this is squarely in scope for AVTCORE
> SDP description for RTP/RTCP over QUIC. This is squarely in scope for MMUSIC, as soon as we figure out the details in AVTCORE.
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/moq <https://www.ietf.org/mailman/listinfo/moq>
> -- 
> Moq mailing list
> [email-masked]

---

From: Suhas Nandakumar <[email-masked]>
Date: Mon, 8 Nov 2021 11:23:03 -0800

> incompatible extensions! 😈

One common theme that might needs addressing is : what sort of API knobs
can be exposed by transport (QUIC) to help apps know about losses/recovery
and influence the choice of congestion control . If some aspects of this
can be done in a standard way, it would definitely help
streaming/interactive use-cases.

Best,

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 8 Nov 2021 15:04:25 -0600

So, full disclosure -

On Mon, Nov 8, 2021 at 1:23 PM Suhas Nandakumar <[email-masked]> wrote:

>>>    - Live Video Delivery - Currently we're pushing the edge of what
>>>    HTTP can do efficiently, including some non-standard extensions.  This is
>>>    also not optimized for QUIC or HTTP/3, which could improve performance.
>> Thanks for this - it's very helpful, and it will definitely be on the
>> list for tomorrow's meeting.
>>
>>

Yes, exactly (it's listed as a consideration in
https://www.rfc-editor.org/rfc/rfc9049.html#name-support-in-endpoint-protoco
.

Having said that, I fully expect that we're going to end up with at least
some pain points that are outside the typical IETF working group's charter.
The ones I'm especially aware of, are

   - The kind of tight integration between codec operation and awareness of
   path characteristics we've been talking about at length, in the thread
   Christian started at
   https://mailarchive.ietf.org/arch/msg/moq/jk9eDELqs_gyg1kSGAmhhzJ-_ts/.
   What I think is normal, is that we tend to have ICCRG looking at congestion
   control mechanisms (quoting today's ICCRG session) "until they're ready to
   come to the IETF. So I think this is worth pursuing, but chartering it in
   the IETF will probably be more complicated than adopting a draft i(say)
   encapsulating RTP in QUIC.
   - I'll say more about this in a reply to another note in this
   (increasingly awesome) thread, but there are things we've been talking
   about that are likely required in order to build a product, but aren't
   candidates for standardization, for various reasons.

But I still think they're worth talking about during our meeting tomorrow,
because if we can agree on a strategy for them, we can take them off the
"media over QUIC" table without dropping them on the floor.

Best,

Spencer, who isn't even the MOQ list admin, and so is definitely not
running anything for the rest of the MOQ fans!

>>>>>    - Mapping RTP onto QUIC (many options exist, so picking one or
>>>>>    more would be helpful) This is squarely in scope for AVTCORE
>>>>>    - SDP description for RTP/RTCP over QUIC. This is squarely in
>>>>>    scope for MMUSIC, as soon as we figure out the details in AVTCORE.
>>>>>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 8 Nov 2021 15:53:03 -0600

Nils,

On Mon, Nov 8, 2021 at 12:55 PM Nils Ohlmeier <[email-masked]> wrote:

> Yes fully agreed. I think that is why lots of people are interested in
> WebTransport, because it could result in some improvements in these areas.
>
>
> This one keeps popping up. But so far all I have heard were some hand wavy
> hopes for improvements. It would be really helpful to have a list of things
> which don’t work in WebRTC to see if switching things over to QUIC could
> really fix/improve these.
>
> One aspect is that WebRTC was specifically designed to be a P2P solution.
> Which I understand is a disadvantage for certain use cases.
> Maybe all of this boils down to not calling this WebRTC over QUIC, which
> brings a lot of expectations with it, but name it different like QUIC RTC
> or something like that.
>

It's my opinion that part of what we're doing is taking terms that don't
map directly to work in the IETF, and translating them into words that do.

You have this (IMO) EXACTLY right. I want to understand if there is a
difference between the way (for instance) SIP uses RTP, and the way WebRTC
uses RTP, that would have an effect on what we write in an "RTP
encapsulated in QUIC" draft(*). We'll get to that point faster if we don't
talk about "WebRTC over QUIC".

Best,

Spencer

 (*) I do understand that WebRTC does have specific requirements for AVP
profiles, congestion behavior, etc. But talking about "RTP over QUIC" would
still help us understand what impact those requirements would have on our
work.

---

From: Luke Curley <[email-masked]>
Date: Mon, 8 Nov 2021 14:17:08 -0800

Twitch is using WebTransport and QUIC streams for live video distribution
(Warp). I gave a presentation at Demuxed about it recently (including
plugging this ML) and would love to start the conversation here too. I've
got a draft and I'm just working on getting approval to publish it now.

I don't think there's a good way to map RTP to QUIC. It's tempting because
there's a lot of standards behind RTP, but any approach involves disabling
huge swaths of functionality on either front. QUIC and RTP have
incompatible ideas on how to perform fragmentation, congestion control,
flow control, reliability, feedback, etc. My project briefly used RTP over
QUIC but we abandoned it when it became too cumbersome.

There are some compelling ways to map live media to QUIC and fully
utilizing QUIC's functionality. For example, sending each frame as its own
stream, which is how Facebook's RUSH works. I think we have enough
non-video people on this list that we should discuss how video works and
what we're trying to accomplish, instead of focusing on how we can port RTP.

On Mon, Nov 8, 2021, 1:05 PM Spencer Dawkins at IETF <
[email-masked]> wrote:

>    - The kind of tight integration between codec operation and awareness
>    of path characteristics we've been talking about at length, in the thread
>    Christian started at
>>>>> Third - James and I are most familiar with the RTP over QUIC
>>>>> proposals, but we recognize that there will be commonalities between
>>>>> various proposals, so  we're perfectly happy to talk about other use cases
>>>>> and requirement sets. So, letting us know what you're thinking about, as

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 8 Nov 2021 16:45:10 -0600

Hi, Luke,

> got a draft and I'm just working on getting approval to publish it now.

Excellent!

> QUIC but we abandoned it when it became too cumbersome.

Understood.

> what we're trying to accomplish, instead of focusing on how we can port RTP.

I'm trying not to prejudge where "Media Over QUIC" is headed, but I suspect
there may be more than one "we" in these discussions, bound by more than
one set of constraints.

But let's see who is trying to do what, more clearly, and then see where
that takes us.

Best,

Spencer

---

From: Justin Uberti <[email-masked]>
Date: Mon, 8 Nov 2021 15:15:59 -0800

> QUIC but we abandoned it when it became too cumbersome.
Yeah, this is the real meat of the discussion. At Google we built something
called QRTP that basically used QUIC as a secure RTP packet transport, and
I think this sort of thing would have a lot of value in terms of making RTP
apps easier to deploy. But you're not getting any new functionality in the
protocol, and you're still mostly in RTP-land (which can be a positive or a
negative depending on where you are coming from).

IOW, we may need to choose between:
1) Adapting RTP to QUIC so that RTP can be sent to 'normal' QUIC servers,
making it easier to deploy RTP-based applications
2) Building RTP-like things on top of QUIC (e.g., RUSH) so that you can get
basic realtime functionality without having to use RTP

My own sentiment is that both may have value, especially if 2) is focused
on a different use case than traditional RTP. We tried a couple attempts at
2) and eventually concluded we were simply rebuilding RTP with a new name,
which led to the final QRTP direction.

>> But I still think they're worth talking about during our meeting
>> tomorrow, because if we can agree on a strategy for them, we can take them
>> off the "media over QUIC" table without dropping them on the floor.
>>
>> Best,
>>
>>>>>> Second - I'm planning to record the session, just so I can verify
>>>>>> what we said, so that I can produce decent minutes. If anyone objects,
>>>>>> please also volunteer to take excellent notes!
>>>>>>
>>>>>>> It's time to put together an agenda for the Tuesday meeting. I'd
>>>>>>> like for this meeting to be a working session, not a forum for
>>>>>>> presentations, so please keep that in mind.
>>>>>>>
>>>>>>> As a high-order bit, I think we've spent a lot of time thinking
>>>>>>> about use cases here, and I'd like to push down another level, and make
>>>>>>> sure we understand what the work for the IETF (and possibly for the IRTF)
>>>>>>> is.
>>>>>>>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 8 Nov 2021 21:20:09 -0600

Hi, Justin,

On Mon, Nov 8, 2021 at 5:16 PM Justin Uberti <[email-masked]>
wrote:

>> I don't think there's a good way to map RTP to QUIC. It's tempting
>> because there's a lot of standards behind RTP, but any approach involves
>> disabling huge swaths of functionality on either front. QUIC and RTP have
> Yeah, this is the real meat of the discussion. At Google we built
> something called QRTP that basically used QUIC as a secure RTP packet
> transport, and I think this sort of thing would have a lot of value in
> terms of making RTP apps easier to deploy. But you're not getting any new
> functionality in the protocol, and you're still mostly in RTP-land (which
> can be a positive or a negative depending on where you are coming from).
>
> 2) Building RTP-like things on top of QUIC (e.g., RUSH) so that you can
> get basic realtime functionality without having to use RTP
>

I'm really glad that you're contributing to this discussion.

I'm remembering the "media over HTTP" aspects of the RIPT BOF (
https://datatracker.ietf.org/group/ript/documents/) that seemed to
complicate life enormously for RIPT. I know you presented at that BOF - do
you think doing media over QUIC is going to avoid most of the problems Mark
Nottingham was worried about with RIPT?

Best,

Spencer

---

From: Justin Uberti <[email-masked]>
Date: Mon, 8 Nov 2021 20:35:12 -0800

On Mon, Nov 8, 2021 at 7:20 PM Spencer Dawkins at IETF <
[email-masked]> wrote:

> Hi, Justin,
>
> On Mon, Nov 8, 2021 at 5:16 PM Justin Uberti <
> [email-masked]> wrote:
>
>>
>>
>>> Twitch is using WebTransport and QUIC streams for live video
>>> distribution (Warp). I gave a presentation at Demuxed about it recently
>>> (including plugging this ML) and would love to start the conversation here
>>> too. I've got a draft and I'm just working on getting approval to publish
>>> it now.
>>>

I think two things are different this time around.
1) RIPT was trying to replace a lot of things - SIP, SDP, RTP (and possibly
more). Ultimately Jonathan was more interested in replacing the call setup
side of things (i.e., trunking) and I was more interested in replacing the
wire protocol, and I came to the understanding that we should tackle these
efforts separately. IOW, the wire protocol will be a real challenge, but I
think we can take this on if we don't let our scope go much beyond that.
2) QUIC was less mature then. Mark was reluctant to start looking at lossy
QUIC, IIRC, while QUIC itself was still a moving target. This is no longer
the case.

So I'm still bullish here. But before we jump into things it would be good
to make sure we're aligned on goals, since I'm not sure we all mean the
same thing when we say media-over-QUIC.

Justin

---

From: Suhas Nandakumar <[email-masked]>
Date: Mon, 8 Nov 2021 21:02:17 -0800

On Mon, Nov 8, 2021 at 8:35 PM Justin Uberti <[email-masked]>
wrote:

>>> 1) Adapting RTP to QUIC so that RTP can be sent to 'normal'
>>> QUIC servers, making it easier to deploy RTP-based applications
>>> My own sentiment is that both may have value, especially if 2) is
>>> focused on a different use case than traditional RTP. We tried a couple
>>> attempts at 2) and eventually concluded we were simply rebuilding RTP with
>>> a new name, which led to the final QRTP direction.
>>>
>>
> 1) RIPT was trying to replace a lot of things - SIP, SDP, RTP (and
> possibly more). Ultimately Jonathan was more interested in replacing the
> call setup side of things (i.e., trunking) and I was more interested in
> replacing the wire protocol, and I came to the understanding that we should
> tackle these efforts separately. IOW, the wire protocol will be a real
> challenge, but I think we can take this on if we don't let our scope go
> much beyond that.

+1 on QUIC's state back then. As a person who attempted to prototype the
RIPT over  QUIC, work back then, I wish there was DATAGRAM support in the
stack (and more matured) and also a way to bring in media recovery aspects
at the application space. We are far off from where we were back then.

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 9 Nov 2021 05:18:58 -0600

Hi, Justin and Suhas,

On Mon, Nov 8, 2021 at 11:02 PM Suhas Nandakumar <[email-masked]>
wrote:

>
>
> On Mon, Nov 8, 2021 at 8:35 PM Justin Uberti <
>> 2) QUIC was less mature then. Mark was reluctant to start looking at
>> lossy QUIC, IIRC, while QUIC itself was still a moving target. This is no
>> longer the case.
>>
>

Thank you both for this explanation (and especially for explaining
on-list!) That's very helpful (and a good reminder that people have done a
lot of work on "Media over QUIC" before the datagram specification

>> same thing when we say media-over-QUIC.
Based on what we've said on this list in the past couple of weeks, I'm sure
we *don't *all mean the same thing! But I'd like to let maybe three flowers
bloom, instead of a hundred flowers that can't grow to maturity.

Thanks to everyone, who is helping us get organized.

Best,

Spencer

---

From: Bernard Aboba <[email-masked]>
Date: Tue, 9 Nov 2021 09:42:13 -0500

"One aspect is that WebRTC was specifically designed to be a P2P solution.
Which I understand is a disadvantage for certain use cases."

[BA] It turns out that several use cases (including Remote Desktop and Game
Streaming) aren't just client/server.  For example, while at home you may
want to remote the desktop of a machine at work in addition to potentially
being interested in a desktop in the cloud.  Similarly, you may be
interested in playing a game on a phone from your game console, not just
playing it from the cloud.

So it turns out that many developers of these use cases actually prefer P2P
APIs to client/server APIs, because the former allows them to handle both
P2P and client/server use cases with a single code base.  These are expert
developers, so they do not fear ICE.  They are now using the RTCDataChannel
API, and while they are potentially interested in migrating to P2P QUIC,
WebTransport is a difficult sell, because they'd still need RTCDataChannel
for P2P scenarios.

Luke said:

"For example, sending each frame as its own stream, which is how Facebook's
RUSH works."

[BA]  This is certainly a lot easier to implement with WebCodecs +
WebTransport.  Now that WebCodecs supports scalable video coding, you can
even vary transport reliability (e.g. maximum transmission time) based on
the type of frame being transmitted.  So far, people playing with this seem
to be getting good results, but I haven't seen a detailed performance
evaluation in challenging scenarios (e.g. high loss or variable RTT).