Subject: Re: Working Group Last Call for HTTP Random Access and Live Content

From: Mark Nottingham <[email-masked]>
Date: Fri, 9 Feb 2018 13:38:24 +1100

Hi Folks,

We didn't receive any feedback during the WGLC period, which is causing your chairs a bit of concern.

If you have read this document and believe it should progress as Experimental, please say so on-list (or privately to us, if necessary).

Thanks,

> On 22 Nov 2017, at 6:21 pm, Mark Nottingham <[email-masked]> wrote:
> 
> Everyone,
> 
> We discussed this document at the Singapore meeting, where we agreed that it's ready to progress. There are no outstanding design issues on it.
> 
> Please have a look at:
>  https://tools.ietf.org/html/draft-ietf-httpbis-rand-access-live-02
> 
> and bring up any issues either on the mailing list or in the issues list. Statements of support to the list would also be helpful.
> 
> As discussed, we'd like to have a longer-than-typical WGLC for this document, to allow some experimentation to take place, account for the holiday period, and to assure that there aren't bad interactions with deployed code.  So, WGLC will end on 2018-01-20.

--
Mark Nottingham   https://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Fri, 9 Feb 2018 07:55:56 +0100

On 2018-02-09 03:38, Mark Nottingham wrote:
> ...

I actually read it and submitted a few editorial PRs.

I'll try to do another read on the weekend and provide more feedback.

Best regards, Julian

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 9 Feb 2018 17:48:12 +1000

Er, I forgot the "sorry this is 3 weeks late" bit.

On 9 February 2018 at 17:46, Matthew Kerwin <[email-masked]> wrote:
> On 9 February 2018 at 12:38, Mark Nottingham <[email-masked]> wrote:
>> Hi Folks,
>>
> For the most part, I can think of a couple of ways it could go wrong,
> but they're all pretty screwy to start with, and the worst that
> happens is a dodgy client finds that its "very large" resource has
> been truncated mid-download.  So I don't think it really needs to be
> Experimental just for that.
>
> That said, I don't feel comfortable about the Shift Buffer
> Representations section.  AFAIK this document is introducing this
> concept, at least as far as breaking the relationship between resource
> representation size and byte range, so it should spend more effort
> describing/standardising some of the other behaviours.  (Such as: What
> happens if I try to resume an interrupted non-range request using a
> range request (assuming I didn't make the first request when the
> original first byte was still available)? Whose byte "0" is the real
> byte "0"?)  It seems like we've opened a little Pandora's box without
> providing any advice or guidance.
>
> Editorial:
>
> * No references until Section 2.1? I would have expected an earlier
> pointer to RFC7233 (at the least), maybe even in the introduction.
>
> * Throughout, "indeterminate-length" (or "indeterminate length") is
> used as an adjective, as in "indeterminate length resource".
> Technically is *is* an attributive noun, but it's awkward, especially
> when not hyphenated. What about "resource of indeterminate length"?
>
> * From Section 2.1 on there's a bunch of ABNF, but there's no
> fore-warning of that anywhere in the document. Is it okay to defer
> that entirely to the normative reference [RFC7233]?
>
> * In 2.2: "Very Large Value", with caps, looks like it should have a
> definition somewhere ;)  (Also it's not consistently capitalised
> throughout the document, one way or the other.)
>
> * "2^^63" is new notation to me. Usually it's either "2^63" or "2**63".
>
> Cheers
>

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 9 Feb 2018 17:46:53 +1000

Editorial:

Cheers

---

From: Julian Reschke <[email-masked]>
Date: Mon, 12 Feb 2018 07:44:09 +0100

> pointer to RFC7233 (at the least), maybe even in the introduction.

Please see the changes in 
<https://www.greenbytes.de/tech/webdav/draft-ietf-httpbis-rand-access-live-latest-from-previous.diff.html>.

> that entirely to the normative reference [RFC7233]?

I stumbled across that as well. However, the ABNF is just cited from the 
spec, it doesn't have a normative role here.

> throughout the document, one way or the other.)

Yep.

---

From: Craig Pratt <[email-masked]>
Date: Thu, 15 Feb 2018 23:27:49 -0800

Yeah - and much thanks Julian.

I've merged your changes into the working copy on github last
month along with some additional edits of my own. I've just been
waiting on some feedback from my co-authors. I'll poke them and
get a new version posted asap...

cp

On 2/8/18 10:55 PM, Julian Reschke wrote:

Hi Folks,

We didn't receive any feedback during the WGLC period, which is
causing your chairs a bit of concern.

If you have read this document and believe it should progress as
Experimental, please say so on-list (or privately to us, if
necessary).

...

I'll try to do another read on the weekend and provide more
feedback.

Best regards, Julian

--

craig pratt

Caspia Consulting

[email-masked]

503.746.8008

---

From: Craig Pratt <[email-masked]>
Date: Fri, 16 Feb 2018 00:59:55 -0800

Thanks Matthew. Reply in-line.

On 2/8/18 11:46 PM, Matthew Kerwin wrote:

Hi Folks,

Thanks,

I'm surprised there haven't been more questions on this section. I
have a couple clarifications drafted. But nothing on this specific
issue - which I hadn't considered.

For Range requests, Content-Range allows the server to communicate
what byte of the representation is being returned as byte 0 of the
response body - so there's no ambiguity. But as you point out, there
*is* ambiguity for non-Range GET requests. Byte 0 of the response
body will be presumed to be byte 0 of the representation - for both
clients and caching proxies.

Now if Content-Range was allowed (and understood) for non-Range
request responses, or perhaps a well-defined response was defined
for GET requests to indicate that non-Range requests were not
supported for such a resource, the ambiguity could be resolved.
There are an uncomfortable number of "if"s there...

As shift-buffer representation are not a critical part of this draft
- and have non-Range aspects that need to be clarified - I'm
thinking it's just better to remove this section. Defining all
aspects of shift buffers (including Range requests and live content)
can be left for a separate draft.

Editorial:

Cheers

Many of these have been addressed in the soon-to-be-published draft.
But I'll make sure we have these covered. Watch for a new draft
revision in the next week.

Thanks much!

cp

--

craig pratt

Caspia Consulting

[email-masked]

503.746.8008

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 16 Feb 2018 20:44:12 +0000

I wonder whether what is being proposed to be handled by this is a 
philosophical change for what is meant by a representation.

I guess in the end, my only concerns are that

1. We don't create unsolvable problems for proxies, or require them to 
change behaviour around byte-ranges (e.g. when it comes to caching or 
framing etc.).
2. We have a clear picture of what we are even doing here.  It seems 
like over-loading byte ranges is a hack (large value????).  There should 
be no room for hacks in http.
3. It's not clear to me why byte ranges are needed to be used for this 
at all.

In the case of the appended content, I'm not convinced the argument in 
the introduction isn't a straw man.  2 options are given

1. Read the whole thing from the beginning.
2. multiple byte-range requests.  A latency problem is identified.

what about a 3rd option

3. Ask the server to send you from a certain timestamp, or "now" not 
using byte ranges, but with some other specific attribute of the request 
that is deterministic.  Does it even need to be a new header, or could 
it just be a part of a querystring?  Definitely never use Range in any 
other dimension than bytes.

So I guess I'm overall not convinced there's a need to alter range 
request handling.  The whole spec seems to be built on an assumption 
that Range requests are needed to solve this problem.  I would question 
that.  Given that Range, C-L etc are framing headers, and breaking 
framing is something that should not even be considered, due to all the 
intermediaries out there that will make this problematic.

Adrien

------ Original Message ------
From: "Mark Nottingham" <[email-masked]>
To: "HTTP Working Group" <[email-masked]>
Cc: "Patrick McManus" <[email-masked]>
Sent: 9/02/2018 3:38:24 PM
Subject: Re: Working Group Last Call for HTTP Random Access and Live 
Content

>Experimental, please say so on-list (or privately to us, if necessary).
>
>Thanks,
>
>
>>We discussed this document at the Singapore meeting, where we agreed 
>>that it's ready to progress. There are no outstanding design issues on 
>>it.
>>
>>Please have a look at:
>>and bring up any issues either on the mailing list or in the issues 
>>list. Statements of support to the list would also be helpful.
>>
>>As discussed, we'd like to have a longer-than-typical WGLC for this 
>>document, to allow some experimentation to take place, account for the 
>>holiday period, and to assure that there aren't bad interactions with 
>>deployed code.  So, WGLC will end on 2018-01-20.
>
>
>--

---

From: Matthew Kerwin <[email-masked]>
Date: Sat, 17 Feb 2018 10:46:02 +1000

On 17 February 2018 at 06:44, Adrien de Croy <[email-masked]> wrote:
> 2. We have a clear picture of what we are even doing here.  It seems like
> over-loading byte ranges is a hack (large value????).  There should be no
> room for hacks in http.
> 3. It's not clear to me why byte ranges are needed to be used for this at
> all.
>

The question is how much of a hack it is.  What will happen to
existing intermediaries if apparent petty-jiggle-bytes of data start
streaming through them?  Using my crystal ball, I don't see many
serious issues except in pathological cases.  Surely most middleboxes
can already deal, one way or another, with streams getting cut off
before the originally-signalled EOF?  Or with attempting to
pre-allocate too much space for a giant representation?

I like this hack because I can't see it failing too badly if the
resource definitively stop growing before it reaches the Very Large
Number of bytes (certainly not in a way that can't be recovered); and
if it grows beyond that size it's pretty easy to request an Even
Larger Number.

A conservative implementation could even replace the Very Large Number
with a Pretty Big Number, so the overheads of hitting end-of-range and
initiating a new request are made less common.  It's all tuneable.

The *only* change to Range semantics is having the server assume that
an absurdly large range request was intentional, and respond in kind.
And that would only be implemented by servers that are designed to
serve these indeterminate responses in the first place, so it's not a
terrible assumption.

>
> In the case of the appended content, I'm not convinced the argument in the
> introduction isn't a straw man.  2 options are given
>
> 3. Ask the server to send you from a certain timestamp, or "now" not using
> byte ranges, but with some other specific attribute of the request that is
> deterministic.  Does it even need to be a new header, or could it just be a
> part of a querystring?  Definitely never use Range in any other dimension
> than bytes.
>

This is the the point where you change what is meant by a
representation.  If you change the URI, then it's a different
resource, so all existing caching is out the window.  Intermediaries
now need to understand your specific REST architecture to optimise
delivery.  And if you chuck in a different header you have to put it
in Vary, so it's just as bad.

>
> So I guess I'm overall not convinced there's a need to alter range request
> handling.  The whole spec seems to be built on an assumption that Range
> requests are needed to solve this problem.  I would question that.  Given
> that Range, C-L etc are framing headers, and breaking framing is something
> that should not even be considered, due to all the intermediaries out there
> that will make this problematic.
>
> Adrien
>

HTTP is a packet protocol.  The only place you can modify a packet
protocol to enable streaming is in the framing.  (Something something
encapsulation doesn't seem to be working too well...)

Cheers