Subject: END_SEGMENT?

From: <[email-masked]>
Date: Thu, 17 Apr 2014 07:52:22 +0000

What is the purpose of END_SEGMENT?

Section 6.1 of the editor's copy says: "[END_SEGMENT] indicates that this frame is the last for the current segment. Intermediaries MUST NOT coalesce frames across a segment boundary and MUST preserve segment boundaries when forwarding frames."

Segment of what?

Are there recommended uses or can it be used for whatever an application wants?

Should intermediaries immediately flush the frames when END_SEGMENT is set?

I accidentally discovered the draft "Protocol Layering in HTTP/2" which suggests a potential use case.  In Section 3.1 it says: "... the DATA frames of HTTP/2 SHOULD include a flag to mark the message boundary, for example END_SEGMENT"

http://www.ietf.org/id/draft-zhu-httpbis-http2-protocol-layering-00.txt

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Jeff Pinner <[email-masked]>
Date: Thu, 17 Apr 2014 09:25:31 -0700

> What is the purpose of END_SEGMENT?
END_SEGMENTS allows the layering of record-based semantics onto the framing
layer.

Consider this use case in HTTP/1.1. An API provides a stream of data using
HTTP/1.1 chunked encoding. Each chunk contains a single record, and the
length of this record is indicated by the chunk length. In HTTP/1.1 the
length of these chunks was unconstrained, but when translating into HTTP/2,
these chunks must be segmented into multiple data frames to fit within the
frame size limit.

If the chunk delineation was meaningful, then END_SEGMENT allows this
meaning to be preserved.

The flag is also useful for layering other record-based protocols on top of
HTTP.

---

From: <[email-masked]>
Date: Thu, 17 Apr 2014 20:00:59 +0000

I think the first paragraph from Jeff's explanation below should be added to the spec as an explanation of the purpose of END_SEGMENT. Perhaps more.

On Apr 17, 2014, at 18:29, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>> wrote:

On Thu, Apr 17, 2014 at 12:52 AM, <[email-masked]<mailto:[email-masked]>> wrote:

END_SEGMENTS allows the layering of record-based semantics onto the framing layer.

Consider this use case in HTTP/1.1. An API provides a stream of data using HTTP/1.1 chunked encoding. Each chunk contains a single record, and the length of this record is indicated by the chunk length. In HTTP/1.1 the length of these chunks was unconstrained, but when translating into HTTP/2, these chunks must be segmented into multiple data frames to fit within the frame size limit.

If the chunk delineation was meaningful, then END_SEGMENT allows this meaning to be preserved.

The flag is also useful for layering other record-based protocols on top of HTTP.

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 18 Apr 2014 08:33:23 +1200

On 18/04/2014 4:25 a.m., Jeff Pinner wrote:
> On Thu, Apr 17, 2014 at 12:52 AM, K.Morgan wrote:
>>

To which the question must be asked:
  do we really want to retain and encourage this broken use of HTTP?

Broken because chunked encoding is hop-by-hop. Any recipient is apt to
re-size the chunks (or even strip them) completely transparently
underneath that application.

It seems the the border is an application specific detail which should
be embeded in the DATA frames payload not forced into HTTP/2.

Amos

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 18 Apr 2014 08:11:33 +1000

> be embeded in the DATA frames payload not forced into HTTP/2.

... could be useful in T-E for ensuring intermediaries don't coalesce
public and private data into a single frame/compression context.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 18 Apr 2014 11:30:35 +1000

Note that we have a related issue here (albeit editorial):
  https://github.com/http2/http2-spec/issues/397

Cheers,

On 17 Apr 2014, at 5:52 pm, [email-masked] wrote:

> Section 6.1 of the editor’s copy says: "[END_SEGMENT] indicates that this frame is the last for the current segment. Intermediaries MUST NOT coalesce frames across a segment boundary and MUST preserve segment boundaries when forwarding frames."
>  
> Segment of what?
>  
> I accidentally discovered the draft “Protocol Layering in HTTP/2” which suggests a potential use case.  In Section 3.1 it says: “… the DATA frames of HTTP/2 SHOULD include a flag to mark the message boundary, for example END_SEGMENT”

--
Mark Nottingham   http://www.mnot.net/

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 17 Apr 2014 22:35:21 -0700

> If the chunk delineation was meaningful, then END_SEGMENT allows this meaning to be preserved.

FWIW, that won't work in HTTP/1.1 (intermediaries and network libraries
will consume and coalesce chunks before any application gets to see them).

....Roy

---

From: Roberto Peon <[email-masked]>
Date: Fri, 18 Apr 2014 00:33:33 -0700

True, but sometimes we've run into hardware that just can't cope with
certain chunking, and nonetheless need to support it.
We ran into more of this quite recently as a matter of fact :(

More interesting to me, in any case, is that END_SEGMENT allows for
effective layering.
-=R

On Thu, Apr 17, 2014 at 10:35 PM, Roy T. Fielding <[email-masked]> wrote:

> > Consider this use case in HTTP/1.1. An API provides a stream of data
> using HTTP/1.1 chunked encoding. Each chunk contains a single record, and
> the length of this record is indicated by the chunk length. In HTTP/1.1 the

---

From: David Krauss <[email-masked]>
Date: Sat, 19 Apr 2014 03:41:58 +0800

> Should intermediaries immediately flush the frames when END_SEGMENT is set?

Although it doesn’t say so, there seems to be no reason not to flush once coalescing is disabled. It doesn’t look like a bad idea to sprinkle these bits liberally on an interactive streaming protocol.

---

From: Jeff Pinner <[email-masked]>
Date: Mon, 21 Apr 2014 08:50:00 -0700

> will consume and coalesce chunks before any application gets to see them).
It depends -- I agree generically it isn't universally true that chunked
encoding is preserved. That being said, parts of stream.twitter.com work
precisely this way (it helps that the stream is encrypted and the API
clients do not use generic network libraries).