Subject: "Timeout" request header to tell server to wait for resource to become available

From: Brendan Long <[email-masked]>
Date: Fri, 27 Mar 2015 19:31:15 -0500

Resending this as an unsigned email since someone had trouble
reading it.

---

I'm hoping this is the right group to address this to. Please let me
know if I should take it somewhere else, or if I should do an
independent submission instead. The use-cases I have in mind are
MPEG-DASH and HLS, but I feel like this header isn't really specific
to either of them, so an RFC would be a good place to define it.

Use Case: MPEG-DASH

When live streaming using MPEG-DASH, clients need to poll the server
for manifest changes and new media segments. Due to fluctuations in
encoding time and network latency, the client can't know exactly
when a segment will be available, so it tries to request the new
resource a short time after should become available (added latency).
If the client requests the new resource too soon, it will get a 404
response and need to retry (introducing an RTT of latency).

It would be nice if we could just request a new media segment or
updated MPD and have the server send it when it's ready.

Solution

Add a header telling the server to wait for a resource to become
available before giving up. I chose to name this header "Timeout"
with a value as a timeout in milliseconds. The actual timeout would
be the value of the smaller of the value of the timeout header or
the server's own configured timeout. If the client also sends an
"If-None-Match" header, the server should also calculate etags
(however it chooses to do so) and wait to send to a 200 or 304
response until the etag of the resource changes.

Header Examples

Basic:

GET /rep1/segment-1.mp4 HTTP/1.0

Timeout: 5000

Meaning: Give me /rep1/segment-1.mp4 as soon as you see it, but you
can give up after 5000 ms.

With ETag:

GET /manifest.mpd HTTP/1.0

Timeout: 5000

If-None-Match: W/"5f0-1193253510"

Meaning: Give me /manifest.mpd as soon as you see it and the etag
doesn't match W/"5f0-1193253510", but you can give up after 5000 ms.

Software Examples

I wrote some Express middleware to implement this behavior here:

https://github.com/brendanlong/express-timeout-header

And an example demonstrating the usage with media segments MPEG-DASH
in Google Chrome:

https://github.com/brendanlong/dash-http-timeout

https://www.youtube.com/watch?v=YUcfNzPaqf0

Conclusion

Does this seem like a good idea? Are there other use-cases? Should I
write some sort of RFC?

PS -

As I write this, I'm starting to wonder if we just need headers like
"Wait-Until: available" or "Wait-Until: etag-changes", since you can
always close the HTTP connection, but the timeout could be useful
for HTTP/1.1 pipelining. Either way works for me though.

See here if that's your preference:

https://github.com/brendanlong/express-timeout-header/tree/wait-until

---

From: Julian Reschke <[email-masked]>
Date: Sat, 28 Mar 2015 01:50:57 +0100

On 2015-03-28 01:31, Brendan Long wrote:
> ...

Nit: "Timeout" is already defined by RFC 4918.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 1 Apr 2015 12:02:25 +1100

Hi Brendan,

I guess I’m wondering why this needs to be coordinated between the client and server; couldn’t the DASH server just pause before determining that it really is a 404?

I.e., are there other clients that need to know it’s a 404 right away?

Cheers,

> On 28 Mar 2015, at 11:31 am, Brendan Long <[email-masked]> wrote:
> 
> Resending this as an unsigned email since someone had trouble reading it.
> 
> ---
> 
> I'm hoping this is the right group to address this to. Please let me know if I should take it somewhere else, or if I should do an independent submission instead. The use-cases I have in mind are MPEG-DASH and HLS, but I feel like this header isn't really specific to either of them, so an RFC would be a good place to define it.
> 
> Use Case: MPEG-DASH
> 
> When live streaming using MPEG-DASH, clients need to poll the server for manifest changes and new media segments. Due to fluctuations in encoding time and network latency, the client can't know exactly when a segment will be available, so it tries to request the new resource a short time after should become available (added latency). If the client requests the new resource too soon, it will get a 404 response and need to retry (introducing an RTT of latency).
> 
> It would be nice if we could just request a new media segment or updated MPD and have the server send it when it's ready.
> 
> Solution
> 
> Add a header telling the server to wait for a resource to become available before giving up. I chose to name this header "Timeout" with a value as a timeout in milliseconds. The actual timeout would be the value of the smaller of the value of the timeout header or the server's own configured timeout. If the client also sends an "If-None-Match" header, the server should also calculate etags (however it chooses to do so) and wait to send to a 200 or 304 response until the etag of the resource changes.
> 
> Header Examples
> 
> Basic:
> Meaning: Give me /rep1/segment-1.mp4 as soon as you see it, but you can give up after 5000 ms.
> 
> With ETag:
> 
> Meaning: Give me /manifest.mpd as soon as you see it and the etag doesn't match W/"5f0-1193253510", but you can give up after 5000 ms.
> 
> Software Examples
> 
> And an example demonstrating the usage with media segments MPEG-DASH in Google Chrome:
> 
> Does this seem like a good idea? Are there other use-cases? Should I write some sort of RFC?
> 
> 
> PS -
> 
> As I write this, I'm starting to wonder if we just need headers like "Wait-Until: available" or "Wait-Until: etag-changes", since you can always close the HTTP connection, but the timeout could be useful for HTTP/1.1 pipelining. Either way works for me though.
> 

--
Mark Nottingham   https://www.mnot.net/