Subject: question/comment on draft-ietf-httpbis-p5-range-20.txt

From: "Fall, Kevin" <[email-masked]>
Date: Thu, 13 Sep 2012 15:48:19 +0000

Hi all.
I originally posted the question below to the authors of the range-related ID, but they suggested I post to the entire list.  I just joined and am now doing so.  Please see the questions below.

To provide a bit more background, these questions are coming up in the context of MPEG-DASH, where (potentially real-time generated) media segments are being fetched by a client via HTTP, possibly with Range requests.  As a possible alternative to what I mentioned below in the second half, would it be possible to clarify the use of "Range: 0-" as a request for 'whatever you have available for me right now'.

Thanks,
- Kevin

From: <Fall>, Kevin Fall <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, September 12, 2012 3:51 AM PDT
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: question/comment on draft-ietf-httpbis-p5-range-20.txt

Hi folks-

I was just reading over draft-ietf-httpbis-p5-range-20 and have two small comments/questions.
First, I see there are no examples where the multipart/byteranges response contains multiple ranges that are out of order.
As far as I can tell this is ok, but not explicitly mentioned or provided in an example.  If, indeed, this is ok, I believe it would be helpful to mention or illustrate.

Perhaps more controversial, Section 3.1 requires a Partial Content request to only be permitted if the corresponding request had a Range header field.
We at least have one case where it would be rather useful to allow a Partial Content / Content-Range response even if the request didn't include the Range header.
Would it be possible to consider replacing MUST with SHOULD in line #2 of Section 3.1? (along with maybe replacing 'partial GET' with simply 'GET' in line #1)?
[or similar wording such as Content-Range responses SHOULD only be present in responses if partial GETs were the request types].  I can explain the use case
in more detail if interested.

Thanks,
- Kevin

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 15 Sep 2012 10:07:48 +1000

Comments below.

On 14/09/2012, at 1:48 AM, "Fall, Kevin" <[email-masked]> wrote:

> From: <Fall>, Kevin Fall <[email-masked]>
> To: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>

Section 4.1 <https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/latest/p5-range.html#rfc.section.4.1> says:

"""When a client asks for multiple ranges in one request, the server should return them in the order that they appeared in the request.""" 

> in more detail if interested.

This can't be changed; it would change the semantics of a 200 response for clients that don't understand partial content, which is OPTIONAL.

Regards,

--
Mark Nottingham   http://www.mnot.net/

---

From: "Fall, Kevin" <[email-masked]>
Date: Sat, 15 Sep 2012 00:53:14 +0000

Thanks.  Reponses inline.

On 9/14/12 5:07 PM PDT, "Mark Nottingham" <[email-masked]> wrote:

>> I originally posted the question below to the authors of the
>>range-related ID, but they suggested I post to the entire list.  I just
>>joined and am now doing so.  Please see the questions below.
>> 
>> To provide a bit more background, these questions are coming up in the
>>context of MPEG-DASH, where (potentially real-time generated) media
>>segments are being fetched by a client via HTTP, possibly with Range
>>requests.  As a possible alternative to what I mentioned below in the
>>second half, would it be possible to clarify the use of "Range: 0-" as a
>>request for 'whatever you have available for me right now'.
>> 
>> Thanks,
>> - Kevin
>> 
>> To: "[email-masked]" <[email-masked]>, "[email-masked]"
>><[email-masked]>, "[email-masked]"
>><[email-masked]>
>> First, I see there are no examples where the multipart/byteranges
>>response contains multiple ranges that are out of order.
>> As far as I can tell this is ok, but not explicitly mentioned or
>>provided in an example.  If, indeed, this is ok, I believe it would be
>>helpful to mention or illustrate.
>
>Section 4.1 
><https://svn.tools.ietf.org/svn/wg/httpbis/draft-ietf-httpbis/latest/p5-ra
>nge.html#rfc.section.4.1> says:
>
>"""When a client asks for multiple ranges in one request, the server
>should return them in the order that they appeared in the request."""

Yes, but this isn't quite sufficient for what I am asking.
If I ask for the range (0-99) and the server wishes to return to me the
ranges (0-50) and (51-99) separately, this isn't a request for multiple
ranges in one request but does represent multiple range responses.  If
this situation isn't allowed at all (which may be the case given my
reading of the language in 4.1), then I'd like to understand if it could
be.
Why would it be ok to coalesce multiple range requests into a single range
response, but not vice versa?
If the answer is 'because some implementations can't decode
multipart/byteranges but can do single range requests', then see below...

> 
>
>
>> Perhaps more controversial, Section 3.1 requires a Partial Content
>>request to only be permitted if the corresponding request had a Range
>>header field.
>> We at least have one case where it would be rather useful to allow a
>>Partial Content / Content-Range response even if the request didn't
>>include the Range header.
>> Would it be possible to consider replacing MUST with SHOULD in line #2
>>of Section 3.1? (along with maybe replacing 'partial GET' with simply
>>'GET' in line #1)?
>> [or similar wording such as Content-Range responses SHOULD only be
>>present in responses if partial GETs were the request types].  I can
>>explain the use case
>This can't be changed; it would change the semantics of a 200 response
>for clients that don't understand partial content, which is OPTIONAL.

I understand that partial content is intended to be OPTIONAL, so let me
ask if this works instead:

Client requests range (0-) [or maybe (0-, 0-) to induce multipart
behavior] with Range header.
Server responds with some byte range(s), hopefully in order but not
necessarily so, and uses Status code 206 to indicate not everything is
necessarily there.

Is this use case to be prohibited?

Thx
- K

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 15 Sep 2012 11:04:17 +1000

> reading of the language in 4.1), then I'd like to understand if it could

Ah, I see.

> response, but not vice versa?

Coalescing is primarily to avoid overhead (possibly from a DoS).

> multipart/byteranges but can do single range requests', then see below...

Yes, that too...

> Is this use case to be prohibited?

Imagine that you have two clients, and one of them makes that request a minute before the other does.

The first client will see the stream from the beginning.

If the second client goes through the same cache that the first one does, it will also see the stream from the beginning (because, from the cache's standpoint it's requesting the first *byte* that's been seen).

Is that your intent? Or do you want the second client to see the stream "live"? I suspect it's the latter...

Range really a mechanism to allow the *client* to request partial content; you're wanting to have partial content semantics driven by the server. As such, I don't think Range is what you're looking for (unless I misunderstand your use case).

Cheers,

---

From: "Fall, Kevin" <[email-masked]>
Date: Tue, 18 Sep 2012 17:05:51 +0000

Comments inline.

>Coalescing is primarily to avoid overhead (possibly from a DoS).

I guess that's ok, although if you were really out be annoying it seems
like you could construct requests with many tiny ranges.

>If the second client goes through the same cache that the first one does,
>it will also see the stream from the beginning (because, from the cache's
>standpoint it's requesting the first *byte* that's been seen).
>

Yeah, its the latter case where the issue at hand comes up most in focus.

>
>Range really a mechanism to allow the *client* to request partial
>content; you're wanting to have partial content semantics driven by the
>server. As such, I don't think Range is what you're looking for (unless I
>misunderstand your use case).

I think you do understand the use case.  However, while I believe I
understand the original intent of Range, it seems to me it is a reasonable
if not elegant way to do "server driven" partial content.  Indeed, would
you agree that the method I suggested above (client suggests multiple
ranges including (0-), (0-) would be "legal" to indicate to the server
that it is permitted to repond with multiple ranges of sizes chosen by the
server?

Another angle on the issue:  if Range winds up not being accepted as the
basis for the mechanism of "server driven" partial content, then
undoubtedly somebody is going to suggest some other orthogonal thing for
it (as I think there do exist legitimate use cases for live-style HTTP
streaming).  Personally I'd rather see whatever mechanism it is to be
harmonious with client-side partial content.  (my $0.02).

{FWIW, I'm willing to take a stab at a couple of paragraphs to describe
this 'server driven' case, if the authors of httpbis-p5-range are
amenable.}

Regards,
- Kevin

---

From: Julian Reschke <[email-masked]>
Date: Tue, 18 Sep 2012 19:51:21 +0200

On 2012-09-18 19:05, Fall, Kevin wrote:
> ...
> ...

The authors aren't relevant here; the Working Group is.

That being said, our current charter (for HTTP/1.1) essentially forbids 
to do anything that could break existing clients, and also doesn't 
really allow us to invent anything new.

(The situation for HTTP/2.0 is slightly different)

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Sep 2012 13:46:31 -0700

> that it is permitted to repond with multiple ranges of sizes chosen by the

The problem is that existing implementations in caches will interpret it in a different way, resulting in them serving a non-live stream (which is NOT what you want).

Cheers,

--
Mark Nottingham
http://www.mnot.net/

---

From: "Fall, Kevin" <[email-masked]>
Date: Thu, 27 Sep 2012 16:58:20 +0000

[KF] comments inline...

>The authors aren't relevant here; the Working Group is.

[KF] ok.

>really allow us to invent anything new.

[KF] fair enough.

>(The situation for HTTP/2.0 is slightly different)

[KF]  Right.  So, is the group comfortable with considering this issue in
the context of HTTP/2.0 discussions?

thx
- Kevin

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 27 Sep 2012 13:06:08 -0400

> the context of HTTP/2.0 discussions?

I think what you really want is to define a new mechanism within HTTP. We can't retroactively change the semantics of existing mechanisms, even in the 1->2 transition, so something new is required.

The proposed (and hopefully, about to be approved any moment) charter does allow us to define new mechanisms, if there's good consensus to do so, and it doesn't distract from our "core" work. 

Cheers,

--
Mark Nottingham
http://www.mnot.net/