Subject: Fw: New Version Notification for draft-bishop-http2-extension-frames-01.txt

From: Mike Bishop <[email-masked]>
Date: Thu, 22 May 2014 17:34:33 +0000

When we decided in Zurich to foreclose the option to extend HTTP/2, we thought that we were close to done with the basic protocol.  And it's true -- the fundamentals of HTTP/2 haven't changed all that much since then.  But various cases keep coming up where certain parties need to add one more feature.  While these aren't core to the HTTP/2 protocol, they're also not worth versioning the protocol for later, and so we've added them to the spec for experimentation as optional components.

This draft proposes that many of these would be perfectly valid use-cases for extensions, and that we might make progress more quickly if we add a simple extension model.  I believe that the time we take in getting the extension model right will be more than offset by the ability to unblock the protocol and still handle new situations as they arise, even though we're late in the process.

I want to emphasize that the goal of the extension model is simplicity in the core protocol, and I'd welcome feedback on how to simplify it further.

Sent from Windows Mail

From: [email-masked]<mailto:[email-masked]>
Sent: ?Thursday?, ?May? ?22?, ?2014 ?10?:?24? ?AM
To: Mike Bishop<mailto:[email-masked]>, Mike Bishop<mailto:[email-masked]>

A new version of I-D, draft-bishop-http2-extension-frames-01.txt
has been successfully submitted by Mike Bishop and posted to the
IETF repository.

Name:  draft-bishop-http2-extension-frames
Revision: 01
Title:  Extension Frames in HTTP/2
Document date: 2014-05-22
Group:  Individual Submission
Pages:  18
URL:            http://www.ietf.org/internet-drafts/draft-bishop-http2-extension-frames-01.txt
Status:         https://datatracker.ietf.org/doc/draft-bishop-http2-extension-frames/
Htmlized:       http://tools.ietf.org/html/draft-bishop-http2-extension-frames-01
Diff:           http://www.ietf.org/rfcdiff?url2=draft-bishop-http2-extension-frames-01

Abstract:
   This document describes a proposed modification to the HTTP/2
   specification to better support the creation of extensions without
   the need to version the core protocol or invoke additional protocol
   identifiers.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: James M Snell <[email-masked]>
Date: Thu, 22 May 2014 10:41:44 -0700

+1... many times over.

On Thu, May 22, 2014 at 10:34 AM, Mike Bishop
<[email-masked]> wrote:
> When we decided in Zurich to foreclose the option to extend HTTP/2, we
> thought that we were close to done with the basic protocol.  And it’s true
> -- the fundamentals of HTTP/2 haven’t changed all that much since then.  But
> various cases keep coming up where certain parties need to add one more
> feature.  While these aren’t core to the HTTP/2 protocol, they’re also not
> worth versioning the protocol for later, and so we’ve added them to the spec
> for experimentation as optional components.
>
> This draft proposes that many of these would be perfectly valid use-cases
> for extensions, and that we might make progress more quickly if we add a
> simple extension model.  I believe that the time we take in getting the
> extension model right will be more than offset by the ability to unblock the
> protocol and still handle new situations as they arise, even though we’re
> late in the process.
>
> I want to emphasize that the goal of the extension model is simplicity in
> the core protocol, and I’d welcome feedback on how to simplify it further.
>
> Sent from Windows Mail
>
> From: [email-masked]
> Sent: ‎Thursday‎, ‎May‎ ‎22‎, ‎2014 ‎10‎:‎24‎ ‎AM
> To: Mike Bishop, Mike Bishop
>
>
> URL:
> http://www.ietf.org/internet-drafts/draft-bishop-http2-extension-frames-01.txt
> Status:
> https://datatracker.ietf.org/doc/draft-bishop-http2-extension-frames/
> Htmlized:
> http://tools.ietf.org/html/draft-bishop-http2-extension-frames-01
> Diff:
> http://www.ietf.org/rfcdiff?url2=draft-bishop-http2-extension-frames-01
>
> Abstract:

---

From: Roberto Peon <[email-masked]>
Date: Thu, 22 May 2014 10:57:55 -0700

Yup.

---

From: Michael Sweet <[email-masked]>
Date: Thu, 22 May 2014 13:57:58 -0400

+1

On May 22, 2014, at 1:34 PM, Mike Bishop <[email-masked]> wrote:

> When we decided in Zurich to foreclose the option to extend HTTP/2, we thought that we were close to done with the basic protocol.  And it’s true -- the fundamentals of HTTP/2 haven’t changed all that much since then.  But various cases keep coming up where certain parties need to add one more feature.  While these aren’t core to the HTTP/2 protocol, they’re also not worth versioning the protocol for later, and so we’ve added them to the spec for experimentation as optional components.
> 
> This draft proposes that many of these would be perfectly valid use-cases for extensions, and that we might make progress more quickly if we add a simple extension model.  I believe that the time we take in getting the extension model right will be more than offset by the ability to unblock the protocol and still handle new situations as they arise, even though we’re late in the process.
> 
> I want to emphasize that the goal of the extension model is simplicity in the core protocol, and I’d welcome feedback on how to simplify it further.
> Sent from Windows Mail
> 

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 23 May 2014 09:48:33 +1000

I like this. Is an unwritten implication of the proposal that an extension
can redefine existing frame types?

I'm thinking of DATA compression as an extension, adding settings
parameters/flags/frame headers/etc. The current compression mechanism is
hop-by-hop, and it would be negotiated as a hop-by-hop extension, so it
works there; but I'll have to think more about the implications for
introducing end-to-end compression. My initial reaction is that we'd have
to introduce an extension frame type that duplicates DATA, like
COMPRESSED_DATA. This could have far reaching consequences, especially for
things like caching or transforming proxies.

Also: there's a lot of "it's safe it ignore these" things in there. I'd
feel much more comfortable if there were some more hard, loud failures.

3.3.4.1.  Handling by Intermediaries

   Intermediaries MUST forward all end-to-end frames regardless of
   whether they recognize the frame type.  Endpoints (user agents and
   origin servers) MUST discard any frame types which they do not
   recognize.  Such frames are, by definition, informational and can be
   safely ignored without affecting the shared state with the sender.

   All hop-by-hop extension-defined frames MUST be dropped by
   intermediaries which do not support the extension.  However, each
   extension SHOULD specify how an intermediary translates the frames
   defined by the extension toward other peers which might or might not
   support the same extension.  When an intermediary advertises support
   for an extension, it MUST abide by the extension-defined intermediary
   behavior.

3.4:

   [...] Implementations MUST ignore unknown settings and
   MUST NOT emit settings defined by an extension which has not been
   announced in an EXTENSIONS (Section 3.3.2) frame.

If extensions are negotiated, there should never be a case where you
receive an extension-defined frame type or setting that you don't
recognise, so it should be a hard error. At the least you'd be flagging a
broken implementation.

On 23 May 2014 03:34, Mike Bishop <[email-masked]> wrote:

> -- the fundamentals of HTTP/2 haven’t changed all that much since
> then.  But various cases keep coming up where certain parties need to add
> one more feature.  While these aren’t core to the HTTP/2 protocol, they’re
> also not worth versioning the protocol for later, and so we’ve added them
> to the spec for experimentation as optional components.
>
> extension model right will be more than offset by the ability to unblock
> the protocol and still handle new situations as they arise, even though
> we’re late in the process.
>
>   *From:* [email-masked]
> *Sent:* ‎Thursday‎, ‎May‎ ‎22‎, ‎2014 ‎10‎:‎24‎ ‎AM
> *To:* Mike Bishop <[email-masked]>, Mike Bishop<[email-masked]>
>
>

-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Roberto Peon <[email-masked]>
Date: Thu, 22 May 2014 17:24:21 -0700

Extensions cannot safely modify the semantics of defined frames. This is
implied from:

Such frames are, by definition, informational and can be
   safely ignored without affecting the shared state with the sender.

Anything that does modify defined-frame semantics isn't guaranteed to
interoperate, and is more than simply an extension.

-=R

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 May 2014 11:13:08 +1000

Hi Mike,

Thanks for that. I’ve heard from a number of people that they want to re-visit the extensibility discussion, and so I’ve re-opened ticket #95:
 <https://github.com/http2/http2-spec/issues/95>

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 23 May 2014 12:04:06 +1000

> interoperate, and is more than simply an extension.
Sorry, I should have looked at the appendices a bit better.

So, an extension can't extend DATA frames, nor replace them without going
hop-by-hop. The only end-to-end extensions we can introduce must be
informational.

Presumably this stems from the fact that we can't negotiate end-to-end
extensions. I guess a justification here is that you can use
application-level extensions instead? (I.e. new HTTP headers, etc., which
*are* end-to-end, and can be used for negotiation.)

I'd probably prefer the proposal if it just eliminated end-to-end
extensions altogether. That would harden up a lot of the wishy-washy
"ignore without error" cases, and I can't think of an end-to-end extension
(especially an informational one) that can't equally be served by a new
HTTP header*.  As a result extensions are less rich than I could have
hoped, but they're simple enough to actually potentially get implemented.

* Particularly since "An end-to-end frame on stream zero is meaningless,
and MUST be discarded upon receipt." If it's on a stream, it might as well
be in a header. The only difference is that extension frames are flow
controlled and headers (currently) aren't.

Incidentally, the compressed data frames in Appendix 3 are pretty wild; it
might be less controversial to reign them in a bit, to more closely match
what's currently in the draft (and align with the discussion that lead
thereto).

---

From: James M Snell <[email-masked]>
Date: Thu, 22 May 2014 20:17:00 -0700

I have two real world use cases for extension end to end frames that cannot
be served by new http headers. Using headers to negotiate their use works
perfectly well. So I'd be -1 on removing that aspect.

---

From: James M Snell <[email-masked]>
Date: Thu, 22 May 2014 20:18:32 -0700

I will say, however, that I'd prefer that unknown end to end extensions at
the endpoints result in hard exceptions rather than must ignore.

---

From: Mike Bishop <[email-masked]>
Date: Fri, 23 May 2014 05:26:57 +0000

My intermediate version said all extensions were hop-by-hop and negotiated; end-to-end was added in response to James' feedback.  The requirement that extensions define how, if at all, they get translated to back-end links that do/do not support the extension was the only end-to-end version I had prior to that change.  I'm still ambivalent about that, but if there's a use-case, I'm not opposed.

If we do allow end-to-end, though, I think they *have* to be silently discardable -- because, as Matthew observes, there's no chance to negotiate.  If I can't know what you'll recognize, I can't avoid sending you something you won't.  Sure, there are headers and history, but you can't know what rules the intermediary is using to route -- the selection of the back-end server may depend on the twenty-seventh character of the path and the tide level in Pamlico Sound, for all you know.

The only end-to-end extension I've currently thought of is the one that was a sample in the previous version, Server Hint.  (Forget if that's what we actually called it.)  Basically, rather than actively pushing, it's a server-driven "You'll want to check that these resources are in your cache and current." that works for when you suspect a client may already have a resource, or the resource is hosted on another server.  There's no harm if an intermediary observes it going by and also freshens its cache, but the intended recipient is the user agent.  But like Push, if the UA just drops it, it will discover that it wants the resource a little later.

As for hardening, Gabriel and I discussed that some.  Let's say we're just talking about hop-by-hop:

  *
If we say you can't send any extension frames until you've seen the other party's EXTENSIONS, then you can't use any in your first request.  That may be acceptable, but seems sub-optimal.
  *
If we say you MUST NOT send even informative extensions once you've seen the other party's EXTENSIONS and you know they haven't requested it, how do you know when to enforce the MUST NOT?  Now we need an ACK for this too, or every connection begins with SETTINGS-EXTENSIONS-PING to simulate one.  Plus, that means the server has to maintain two different states -- one in which it permits unknown frames, and one in which it doesn't.

Sent from Windows Mail

From: James M Snell<mailto:[email-masked]>
Sent: ?Thursday?, ?May? ?22?, ?2014 ?8?:?18? ?PM
To: Matthew Kerwin<mailto:[email-masked]>
Cc: Mike Bishop<mailto:[email-masked]>, HTTP Working Group<mailto:[email-masked]>, Roberto Peon<mailto:[email-masked]>

I will say, however, that I'd prefer that unknown end to end extensions at the endpoints result in hard exceptions rather than must ignore.

On May 22, 2014 7:08 PM, "Matthew Kerwin" <[email-masked]<mailto:[email-masked]>> wrote:
On 23 May 2014 10:24, Roberto Peon <[email-masked]<mailto:[email-masked]>> wrote:
Extensions cannot safely modify the semantics of defined frames. This is implied from:

Anything that does modify defined-frame semantics isn't guaranteed to interoperate, and is more than simply an extension.

So, an extension can't extend DATA frames, nor replace them without going hop-by-hop. The only end-to-end extensions we can introduce must be informational.

Presumably this stems from the fact that we can't negotiate end-to-end extensions. I guess a justification here is that you can use application-level extensions instead? (I.e. new HTTP headers, etc., which *are* end-to-end, and can be used for negotiation.)

I'd probably prefer the proposal if it just eliminated end-to-end extensions altogether. That would harden up a lot of the wishy-washy "ignore without error" cases, and I can't think of an end-to-end extension (especially an informational one) that can't equally be served by a new HTTP header*.  As a result extensions are less rich than I could have hoped, but they're simple enough to actually potentially get implemented.

* Particularly since "An end-to-end frame on stream zero is meaningless, and MUST be discarded upon receipt." If it's on a stream, it might as well be in a header. The only difference is that extension frames are flow controlled and headers (currently) aren't.

Incidentally, the compressed data frames in Appendix 3 are pretty wild; it might be less controversial to reign them in a bit, to more closely match what's currently in the draft (and align with the discussion that lead thereto).

---

From: Matthew Kerwin <[email-masked]>
Date: Fri, 23 May 2014 19:15:26 +1000

On 23 May 2014 15:26, Mike Bishop <[email-masked]> wrote:

>  My intermediate version said all extensions were hop-by-hop and
> negotiated; end-to-end was added in response to James’ feedback.  The
> requirement that extensions define how, if at all, they get translated to
> back-end links that do/do not support the extension was the only end-to-end
> version I had prior to that change.  I’m still ambivalent about that, but
> if there’s a use-case, I’m not opposed.
>
>
​I'd be interested to know what James's extensions are.
​

>  If we do allow end-to-end, though, I think they *have* to be silently
> discardable -- because, as Matthew observes, there’s no chance to
> negotiate.  If I can’t know what you’ll recognize, I can’t avoid sending
> you something you won’t.
>
>
​Yeah, I agree, if they are allowed.​

>  The only end-to-end extension I’ve currently thought of is the one that
> was a sample in the previous version, Server Hint.  (Forget if that’s what
> we actually called it.)  Basically, rather than actively pushing, it’s a
> server-driven “You’ll want to check that these resources are in your cache
> and current.” that works for when you suspect a client may already have a
> resource, or the resource is hosted on another server.  There’s no harm if
> an intermediary observes it going by and also freshens its cache, but the
> intended recipient is the user agent.  But like Push, if the UA just drops
> it, it will discover that it wants the resource a little later.
>
>
​Sounds to me like a HTTP header. In fact, it sounds like something in the
Cache-Control family. Just because HTTP/2 does server push doesn't mean
sending unsolicited information (call it hints, whatever) is limited ​to
HTTP/2. HTTP/1.1 could definitely benefit from such an extension, if it was
supported.

>  As for hardening, Gabriel and I discussed that some.  Let’s say we’re
> just talking about hop-by-hop:
>
>    - If we say you can’t send *any* extension frames until you’ve seen
>    the other party’s EXTENSIONS, then you can’t use any in your first
>    request.  That may be acceptable, but seems sub-optimal.
>
>
​​Could you move the EXTENSIONS frame to a settings parameter? A single
settings frame could contain multiple such parameters, and settings are
already mandatorily sent as soon as the connection is established, and
they're acked.​

>    - If we say you MUST NOT send even informative extensions once you’ve
>    seen the other party’s EXTENSIONS and you know they haven’t requested it,
>    how do you know when to
>    ​
>    ​
>    enforce the MUST NOT?
>
>
​Start at time 0. I don't see why it should be a problem for an extension
frame that may have caused a protocol error earlier to suddenly become
legal.​

Now we need an ACK for this too, or every connection begins ​with
> SETTINGS-EXTENSIONS-PING to simulate one.  Plus, that means the server has
> to maintain two different states -- one in which it permits unknown frames,
> and one in which it doesn’t.
>
>
Another reason to use a setting?

---

From: James M Snell <[email-masked]>
Date: Fri, 23 May 2014 05:58:19 -0700

I've discussed some of this before. One case involves more efficient
handling of multipart data as an alternative to MIME; another deals with
inserting incremental checksums into a long lived stream. I am also
exploring whether or not it will be possible to implement things like mqtt
efficiently and visibly as a sub protocol of http2 framing without being
forced to push it all into opaque data frames. I'm quite certain that these
cases aren't interesting to everyone, nor are they critical to the core of
http2, a good end to end extensibility model makes it possible for me to
experiment independently without having to worry so much about the
capabilities of the intermediary infrastructure.

---

From: <[email-masked]>
Date: Sat, 31 May 2014 08:02:15 +0000

+1 from me too

Mark-

Is this on the agenda for discussion next week in New York?

-Keith

On May 22, 2014, at 20:02, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>> wrote:

Yup.

On Thu, May 22, 2014 at 10:41 AM, James M Snell <[email-masked]<mailto:[email-masked]>> wrote:
+1... many times over.

On Thu, May 22, 2014 at 10:34 AM, Mike Bishop
<[email-masked]<mailto:[email-masked]>> wrote:
> until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.
>
> The IETF Secretariat
>

This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 31 May 2014 18:05:49 +1000

Yes, we reopened the extensibility issue.