Subject: [Wish] New draft version draft-murillo-whip-01

From: Sergio Garcia Murillo <[email-masked]>
Date: Wed, 19 May 2021 21:43:11 +0200

Hi all,

I have just published a new version of the whip draft (
https://datatracker.ietf.org/doc/html/draft-murillo-whip-01) with the
updated I outlined in a previous email a while back:

- Return a resource URL in the response to the HTTP POST with the SDP O/A.
- Allow trickle ICE and ICE restarts via HTTP PATCH with an
application/trickle-ice-sdpfrag
<https://www.iana.org/assignments/media-types/application/trickle-ice-sdpfrag>
body
to the resource URL.
- Allow to explicitly terminate the session by the WHIP client by sending a
HTTP DELETE  to the resource URL.

Best regards
Sergio

---

From: Adam Roach <[email-masked]>
Date: Thu, 27 May 2021 16:37:29 -0500

Thanks for the new version, Sergio!

I apologize for being somewhat absent since our March meeting. I had a 
partially-typed-up summary of issues and resolutions started, but had to 
work on some different priorities for a couple of months. I'll go over 
the items that I had notes for from our last set of conversations below, 
along with comments and proposals. In general, I really like all of the 
changes to the draft.

----

Item 1: SVC and Simulcast. There are basically three questions here: (1) 
does the protocol need to support SVC and simulcast; and (2) are clients 
required to support SVC and simulcast, and (3) are servers required to 
support SC and simulcast.

My suggestion for a path forward is:

- Yes, the protocol needs to support these features, and will 
informatively cite RFC 8853 and RFC 6190 as the means for doing so.

- No, clients are not required to support these mechanisms, and will use 
the procedures in RFC 8853 §5.3 and RFC 6190 §7.2.2 to negotiate whether 
they will do so.

- No, servers are not required to support these mechanisms, and will use 
the same procedures as clients to negotiate their support.

----

Item 2: Signaling within a session

The new proposal in the document sends the initial offer in a POST, and 
the server replies with a 201 that contains an answer in the body, and 
contains a link corresponding to the created session in a "Location" 
header field. I've poked around and convinced myself that this is a 
perfectly good use of 201, and that it gets the O/A exchange completed 
in a single transaction, and so I support the approach.

----

Item 3: ICE Trickle & Restart

The new proposal in the document specifies the use of PATCH with 
application/sdpfrag for both trickle and restart.

I really like the ability for the client to initiate restart, BTW, as it 
allows for transition between, e.g., mobile networks and WiFi for mobile 
clients, so this is a very important addition for me.

This proposed mechanism looks perfectly fine to me, although it comes 
with the caveat that this approach only allows the client to trickle 
candidates, and only allows the client to request an ICE restart.

For the trickle case, I think this is okay: in the vast majority of the 
cases, the server will not need to gather candidates and will simply be 
using host candidates. Even in the unlikely case that the server needs 
to gather candidates, this approach at least allows client and server 
collection to take place in parallel.

For the restart case, this requires the client to be completely 
responsible for detecting and repairing failed ICE connections. Since 
there is no media flowing from the server to the client, this means that 
the client will need to wait some multiple of the RTCP transmission 
interval before it can determine that the connection is no longer 
working -- in practice, this means it will take on the order of 5 to 20 
seconds to perform such detection and trigger a restart. This opens up a 
number of options:

(a) Live with this situation
(b) Specify something at the media level, such as a faster cadence for 
RTCP RR packet transmission from the network to the client so detection 
can take place faster
(c) Introduce a two-way channel for server-to-client communications

I gather that (c) remains unpopular due to its complexity. I'm okay with 
both (a) and (b), with a slight preference for something along the lines 
of (b) -- largely because users will consider the system to be 
completely failed long before client detection if we go with (a).

----

Item 4: Session Termination

The current draft supports this through the use of a DELETE message sent 
to the session endpoint. This approach seems quite sensible, and it 
allows the signaling layer to perform swift resource cleanup in the 
sunny-day case.

As with the ICE signaling above, this approach means that servers cannot 
initiate session termination on the signaling layer. I would propose 
that we add explicit text that specifies that network-initiated teardown 
can be performed by the network sending RTCP BYE packet(s) to the 
client. The client would be responsible for then initiating a DELETE 
operation on the signaling layer (which gives the network confirmation 
that the client is aware of the session termination, and has had the 
opportunity to notify the user).

----

Item 5: Heartbeating

This proposal does not include any means of signaling-level detection of 
client-side session failure (e.g., the client loses a network connection 
or crashes). Based on previous conversations about the intention of 
media channels potentially surviving a signaling-level failure, I gather 
that this is considered a feature rather than an omission.

After some internal discussion, I'm okay with this, even if it isn't my 
preferred approach. However, if the idea here is that media can survive 
independently of session state, then I'd like to also see a (required) 
client-to-network RTCP BYE packet sent whenever a session is terminated. 
I propose that this would happen in addition to the DELETE operation, 
not instead of it.

----

Item 6: Extensibility

This proposal does not have any defined extension points. The operations 
performed on the session resource are mapped onto HTTP methods, and the 
currently proposed methods have a good semantic match to the operations 
that they correspond to. What I would propose is:

(1) The base WHIP document does not address extensibility except for a 
statement that GET operations on the created resource URL (the one that 
we get in the "Location" header field) are reserved for future 
extensibility (this means that unextended servers will need to return a 
405 response to GET on the resource);

(2) Separate from the specification of the base WHIP document, we create 
a new document that discusses how to use GET to bootstrap protocol 
extensions. I have ideas about how to do this, but would like to make 
progress on the core protocol before we start those discussions.

/a

On 5/19/21 14:43, Sergio Garcia Murillo wrote:
> <https://www.iana.org/assignments/media-types/application/trickle-ice-sdpfrag> body 
> to the resource URL.
> - Allow to explicitly terminate the session by the WHIP client by 
> sending a HTTP DELETE to the resource URL.
>
> Best regards
> Sergio
>

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 28 May 2021 11:31:30 +0200

Hi Adam, welcome back!

Answers inline below:

> the same procedures as clients to negotiate their support.
My intention was that the protocol must support the negotiation of both svc
and simulcast, and both clients and publishers may support them, so I agree
with your suggestion. Opened an issue for clarifying it:

https://github.com/murillo128/webrtc-http-ingest-protocol/issues/9

> contains a link corresponding to the created session in a "Location" header
> field. I've poked around and convinced myself that this is a perfectly good
> use of 201, and that it gets the O/A exchange completed in a single
> transaction, and so I support the approach.
>
> ----
>
> This proposed mechanism looks perfectly fine to me, although it comes with
> the caveat that this approach only allows the client to trickle candidates,
> and only allows the client to request an ICE restart.
>
> using host candidates. Even in the unlikely case that the server needs to
> gather candidates, this approach at least allows client and server
> responsible for detecting and repairing failed ICE connections. Since there
> is no media flowing from the server to the client, this means that the
> client will need to wait some multiple of the RTCP transmission interval
> before it can determine that the connection is no longer working -- in
> practice, this means it will take on the order of 5 to 20 seconds to
> perform such detection and trigger a restart. This opens up a number of
> options:
>
> RTCP RR packet transmission from the network to the client so detection can
> take place faster
> both (a) and (b), with a slight preference for something along the lines of
> (b) -- largely because users will consider the system to be completely
> failed long before client detection if we go with (a).
>

I am fine with recommending having  RTCP SR/RR reports sent more often, but
I don't think it is something we can mandate. Also, the client can use the
STUN consent fresnesh to detect the network interruption. However, reducing
the timeouts to be able to react faster will likely cause a lot of false
positives (even if we do c), so I think we should not do it and for being
able to detect network changes, like wifi to 4g, it is best that the
clients rely on the OS provided APIs rather than on any media timeout:

https://developer.android.com/training/monitoring-device-state/connectivity-status-type
https://wicg.github.io/netinfo/
https://developer.apple.com/library/archive/samplecode/Reachability/Introduction/Intro.html

----
> to the session endpoint. This approach seems quite sensible, and it allows
> the signaling layer to perform swift resource cleanup in the sunny-day case.
>
> initiate session termination on the signaling layer. I would propose that
> we add explicit text that specifies that network-initiated teardown can be
> performed by the network sending RTCP BYE packet(s) to the client. The
> client would be responsible for then initiating a DELETE operation on the
> signaling layer (which gives the network confirmation that the client is
> aware of the session termination, and has had the opportunity to notify the
> user).
>

I am fine with the RTCP BYE approach, the draft also includes using the
revocation of consent as per RFC7675

The media server may terminate the session by using the Immediate
Revocation of Consent as defined in {{!RFC7675}} section 5.2.

> client-side session failure (e.g., the client loses a network connection or
> crashes). Based on previous conversations about the intention of media
> channels potentially surviving a signaling-level failure, I gather that
> this is considered a feature rather than an omission.
>
> client-to-network RTCP BYE packet sent whenever a session is terminated. I
> propose that this would happen in addition to the DELETE operation, not
> instead of it.
>

I like the idea.

> statement that GET operations on the created resource URL (the one that we
> get in the "Location" header field) are reserved for future extensibility
> (this means that unextended servers will need to return a 405 response to
> GET on the resource);
>
> (2) Separate from the specification of the base WHIP document, we create a
> new document that discusses how to use GET to bootstrap protocol

I am not sure if using GET on the resource URI to return something
different than the SDP representation is covered under the HTTP best
practices, we should double check that.

For extensibility I was thinking that each extension we could introduce a
Link header with the extension rel type and the uri for the HTTP resource
of the extension on the 201 created, similar to what web push is doing:

https://datatracker.ietf.org/doc/html/rfc8030#section-9.2

So for example, taking a potential extension of server to client
communication using server sent events as specified in
https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events

The url for connecting to the server side event resource for the published
stream will be returned in the initial HTTP "201 Created" response with a
"Link" header an a "rel" attribute of
 "urn:ietf:params:whip:server-side-events",  the http 201 response to the
creation POST would look like

HTTP/1.1 201 Created
Content-Type: application/sdp
Location: https://watever.com/publications/213786HF
Link: <https://watever.com/publications/213786HF/sse>;rel="urn:ietf:params:whip:server-side-events
"

Other extensions could be added by registering new "rel" values on IANA.

Best regards
Sergio

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 28 May 2021 14:42:58 +0200

> https://datatracker.ietf.org/doc/html/draft-murillo-whip-01

Thanks.  Is anyone implementing a client to test against?

I'm a little confused about how multiple flows are handled.  Is a single
WHIP endpoint able to multiplex multiple flows, or does one WHIP endpoint
correspond to exactly one flow?  If the former, how are ICE restarts and
ICE requests correlated with the flow?  (If the latter, some use cases will
require a dynamic allocation subprotocol.)

> The initial offer by the WHIP client MAY [...] only contain local
> candidates or even an empty list of candidates.

> The WHIP endpoint SDP answer SHALL contain the full list of ICE
> candidates publicly accessible of the media server.

If I read this correctly, you're defining a new, asymmetric form of ICE:
the server collects all local candidates, then allows the client to do
Trickle ICE.  I think it's a good idea, but unless I'm mistaken, that's
a new mode of operation for ICE, one that I haven't seen described in any
other document.

Have you actually implemented that?  How much surgery to your ICE stack
was required?  In short, how painful was it?

On a related note, what happens if the client is on a restrictive network
that doesn't allow UDP?  Are you assuming it has learned the address of
a TURN server out of band, or are you relying on passive TCP candidates?
(Either would be fine, but it needs to be spelled out.)

> The WHIP client MAY perform trickle ICE or an ICE restarts [RFC8863] by
> sending a HTTP PATCH request

Why the uncommon request method?  I find the use of PATCH here suprising.

> A WHIP resource MAY not support either trickle ICE (i.e. ICE lite media
> servers) or ICE restart,

Does the client know beforehand whether that's the case?  For ICE
restarts, that's not a problem (you just attempt an ICE restart, and if
that fails, you tear down the flow and start a new one).

For Trickle ICE, on the other hand, the client needs to know beforehand
whether it can send an incomplete offer and then collect candidates or
whether it must send an offer with a full set of candidates.

Perhaps we could require that a server should reject an offer with
ice-options:trickle unless it can accept further ICE candidates?  That way
a client can fallback to non-trickle operation straight away.

> Unlike [RFC5763] a WHIP client MAY use a setup attribute value of
> setup:active in the SDP offer, in which case the WHIP endpoint MUST
> use a setup attribute value of setup:passive in the SDP answer.

I assume that is to simplify client implementations.  A short rationale
would be welcome.

> Authentication and authorization is supported by the Authorization HTTP
> header with a bearer token as per [RFC6750].

Is that a MUST?  What if my server doesn't do OAuth2, do I need to
implement it just to do WISH?  Or can I stick to HTTP Basic?

Thanks again,

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 28 May 2021 14:54:13 +0200

> a client can fallback to non-trickle operation straight away.

On further reflection, let's just make server-side support for Trickle
mandatory and be done with it.

-- Juliusz

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 28 May 2021 15:34:20 +0200

> Thanks.  Is anyone implementing a client to test against?

Implementing a WHIP client is trivially simple in web, you can check an
example here for the previous version:

https://medooze.medium.com/whip-webrtc-meets-the-broadcasting-world-86772eba8ae7

> require a dynamic allocation subprotocol.)

A WHIP endpoint is an entity able to receive the HTTP POST request from a
client. The URL may be a "factory" one returning an unique url on the
Location header or have an unique URL for each stream. It is up to the
server implementation.

In any way, the WHIP resource URL must uniquely identify the flow.

> a new mode of operation for ICE, one that I haven't seen described in any

That would be the same as having a normal webrtc client with no STUN/TURN
servers configured against one that have them, right?

> was required?  In short, how painful was it?
I do ICE lite on servers, so I had to do nothing.. :)

In case you need to do full ice on the server and you don't have a way of
statically specifying the public ip address or requires collecting TURN
relay candidates, your server will have to delay sending the 201 response
with the SDP answer when all the candidates are collected.

> (Either would be fine, but it needs to be spelled out.)

Clients can trickle ICE normally. Anyway, on my implementation, I don't
even require to exchange candidates with the server in that case as the
server will learn the new candidates from the STUN requests sent by the
client.

> Why the uncommon request method?  I find the use of PATCH here suprising.

That is the semantically correct method for updating a resource in HTTP:
https://datatracker.ietf.org/doc/html/rfc5789#page-4

2 <https://datatracker.ietf.org/doc/html/rfc5789#section-2>.  The PATCH Method

   The PATCH method requests that a set of changes described in the
   request entity be applied to the resource identified by the Request-
   URI.  The set of changes is represented in a format called a "patch
   document" identified by a media type

> a client can fallback to non-trickle operation straight away.
 I would say that a server should be either able to work in full ice, or
not require the trickle candidate exchange at all to not have to ask the
client to implement a fallback mode. In this case, the server may just
reject the PATCH request without any consecuence.

> I assume that is to simplify client implementations.  A short rationale

Yes, that's the rationale.

> implement it just to do WISH?  Or can I stick to HTTP Basic?

I don't think we should allow HTTP basic usage. Anyway, I am open to
discuss it.

Best regards
Sergio

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 28 May 2021 16:18:39 +0200

> client. The URL may be a "factory" one returning an unique url on the Location
> header or have an unique URL for each stream. It is up to the server
> implementation.

Perfect.

Could you please spell out the exact protocol you envision?  The server
replies 200 with the SDP answer and a Location header, or the server
replies 308 which causes the client to send the SDP offer again?

This requires special-case handling in the client (candidates and restarts
need to be sent to the right endpoint, not to the multiplexer), so please
spell it out in the draft.

> Anyway, on my implementation, I don't even require to exchange
> candidates with the server in that case as the server will learn the new
> candidates from the STUN requests sent by the client.

What happens if the client is on a restrictive network that doesn't allow UDP?

>> Does the client know beforehand whether that's the case?

> I would say that a server should be either able to work in full ice, or not
> require the trickle candidate exchange at all to not have to ask the client to
> implement a fallback mode. In this case, the server may just reject the PATCH
> request without any consecuence.

Sorry if I'm being obtuse.  Client does trickle, so it sends an SDP offer
with incomplete candidates.  After contacting its STUN server, it attempts
to PATCH its offer, and receives 405.

What should the client do in that situation?

-- Juliusz

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 28 May 2021 16:32:31 +0200

El vie, 28 may 2021 a las 16:18, Juliusz Chroboczek (<[email-masked]>)
escribió:

> Location

It is explicitly stated that the HTTP PATCH request with the sdp fragment
body containing the ICE candidates/restart info must be sent to the WHIP
Resource URI returned on the Location header of the 201 create response to
the original HTTP PUT request.

How do you think it should be further explained?

> What happens if the client is on a restrictive network that doesn't allow

When the client allocates a relay candidate on the TURN server to connect
on a restricted network, it will send a STUN request via it to the server
host candidates. The server will receive the STUN request and get the relay
candidate info from the remote ip address/port.

> not

It is stated on the draft:

   A WHIP client receiving a 405 response for an HTTP PATCH request
   SHALL not send further request for ICE trickle or restart.  If the
   WHIP client gathers additional candidates (via STUN/TURN) after the
   SDP offer is sent, it MUST send STUN request to the ICE candidates
   received from the media server as per [RFC8838
<https://datatracker.ietf.org/doc/html/rfc8838>] regardless if the

Murillo & Gouaillard    Expires November 20, 2021               [Page 5]

------------------------------

Internet-Draft                    whip                          May 2021

   HTTP PATCH is supported by either the WHIP client or the WHIP
   resource.

 Best regards
Sergio

---

From: Juliusz Chroboczek <[email-masked]>
Date: Fri, 28 May 2021 17:15:23 +0200

> It is explicitly stated that the HTTP PATCH request with the sdp fragment body
> containing the ICE candidates/restart info must be sent to the WHIP Resource
> URI returned on the Location header of the 201 create response to the original
> HTTP PUT request.

It was not clear to me that there are two URLs, the "WHIP Endpoint URL"
and the "WHIP Resource URL".  Everything is clear now.

> How do you think it should be further explained?

Perhaps the fact that there are two distinct URLs could be stated in
Section 3?

>>>> On a related note, what happens if the client is on a restrictive
>>>> network that doesn't allow UDP?  Are you assuming it has learned
>>>> the address of a TURN server out of band, or are you relying on
>>>> passive TCP candidates?  (Either would be fine, but it needs to
>>>> be spelled out.)

> When the client allocates a relay candidate on the TURN server to connect on a
> restricted network, it will send a STUN request via it to the server host
> candidates.

How did the client acquire the TURN server's address and credentials in
the first place?  You're assuming that this has happened out of band
before the first WHIP interaction, by some means not specified by this
protocol, right?

If so, that's fine with me, I just think that it should be stated explicitly.

>    received from the media server as per [RFC8838] regardless if the

Aha, I see.  You're assuming that either the server is accessible from the
Internet, or it implements trickling of client candidates.

Makes perfect sense, thanks.

-- Juliusz

---

From: Sergio Garcia Murillo <[email-masked]>
Date: Fri, 28 May 2021 21:40:07 +0200

> Perhaps the fact that there are two distinct URLs could be stated in

I have opened a new issue for handling that:
https://github.com/murillo128/webrtc-http-ingest-protocol/issues/10

> If so, that's fine with me, I just think that it should be stated

Initially my idea was to not include stun/turn configuration on WHIP/WISH
and let the end user configure it manually. However, I see the value of
trying to cover that part too.

Providing the ice servers urls on the WHIP request is not possible, as it
would have to already have created the peerconnection to retrieve the SDP
offer, so maybe it is time to bring back to live Justin's proposal:
https://datatracker.ietf.org/doc/html/draft-uberti-behave-turn-rest-00

> Internet, or it implements trickling of client candidates.

Yes.

Best regards
Sergio