Subject: #555: frame synchronization

From: Mark Nottingham <[email-masked]>
Date: Sun, 20 Jul 2014 15:52:17 -0400

<https://github.com/http2/http2-spec/issues/555>

Anyone with thoughts on this? 

On 12 Jul 2014, at 4:30 am, Poul-Henning Kamp <[email-masked]> wrote:

> Another issue I don't want to raise in the current melee, but I
> think we should take it as an isssue, discuss it and close it,
> so we can claim that we thought about it:
> 
> 
> Has anybody considered frame-desynchronization ?
> 
> The framing layer has neither pattern, checksum or sequence numbers
> to help us tell if the sender and receiver have become mis-synchronized,
> 
> If we wanted to be strictly compliant with "end-to-end argument" we
> should have an integrity check, but given that we have a CRC and
> sums in the layers below I don't think that's really an overhead
> which would justify itself.
> 
> However, it would make good sense to add a pattern or sequence
> number to the frame header to give us a "must match field" to detect
> frame misalignment.  (I looked at the current frames and there
> are no fields which lend themselves naturally to this).
> 
> If a frame header gets misaligned one byte either way, there are
> pretty good chances that it will still look legit enough for
> processing to commence.
> 
> Given that many people talk about processing frames while they are
> still being received, adding a more robust misalignment detection
> sounds like a good idea to me.
> 
> The easiest is probably a single byte sequence number which just
> increments from frame to frame.  That number could be included
> in RST frames to indicate which frame caused the error.
> 
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> [email-masked]         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe    
> Never attribute to malice what can adequately be explained by incompetence.

--
Mark Nottingham   http://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Sun, 20 Jul 2014 16:56:08 -0700

As I already said, I don't see any need. Desync ends pretty quickly with a
GoAway.

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 21 Jul 2014 08:07:08 +0000

In message <CABkgnnVbn7LYmbB3y=[email-masked]>

>As I already said, I don't see any need. Desync ends pretty quickly with a

The question is if we should do something to make sure that it happens
faster and with more certainty than "pretty quickly" ?

Has anybody tried to do a fuzzer-test to see how robust their
implementation is for an almost correctly working peer ?

---

From: Kinkie <[email-masked]>
Date: Mon, 21 Jul 2014 11:27:27 +0200

> faster and with more certainty than "pretty quickly" ?

I agree: Having some more deterministic method to determine an error
condition would be useful.

---

From: Michael Sweet <[email-masked]>
Date: Mon, 21 Jul 2014 09:27:34 -0400

Generally I think this is unnecessary - HTTP depends on "reliable" transport (TCP as defined) so we probably don't need this for frames generally.

It could be useful for HEADERS/PUSH_PROMISE for tracking which version of the header table is in use (see my recent posting on the Cost Analysis thread), but that all depends on how hard we want to avoid dropping connections when things go wrong...

>> in RST frames to indicate which frame caused the error.

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Kinkie <[email-masked]>
Date: Mon, 21 Jul 2014 15:37:52 +0200

> Generally I think this is unnecessary - HTTP depends on "reliable" transport (TCP as defined) so we probably don't need this for frames generally.

To me the most obvious advantage is not about data corrupted/lost by
transport, but in rapid detection of buggy http2 implementations in
peers.

> It could be useful for HEADERS/PUSH_PROMISE for tracking which version of the header table is in use (see my recent posting on the Cost Analysis thread), but that all depends on how hard we want to…

I don't think we want to avoid it at all; in case of a frame desync
due to a buggy peer the most reliable recovery mechanism is reset and
retry, isn't it?

  Kinkie

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 21 Jul 2014 14:19:45 +0000

>transport, but in rapid detection of buggy http2 implementations in

+1

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jul 2014 10:35:48 -0400

I’m only hearing PHK and Kinkie argue for this, and the people I’m seeing F2F in Toronto are against it. 

Considering how big of a change this is, I don’t think there’s enough support to get it over the wall.

Anyone else want to stand up for it?

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 21 Jul 2014 15:32:05 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>I'm only hearing PHK and Kinkie argue for this, and the people I'm 
>seeing F2F in Toronto are against it. 

Could you get them to say why ?

>Considering how big of a change this is, I don't think there's 
>enough support to get it over the wall.

This is not necessarily a big change, it may be as little as 
1-N bit wide sequence number which increments in every frame.

I raised this issue because I have seen any mention of
frame-desynchronization or detection probabilities thereof anywhere
at all ?

I agree that sooner or later something will blow up, my question is
if "later" is soon enough or of we should do something to increase
probability of detection.

---

From: Cory Benfield <[email-masked]>
Date: Mon, 21 Jul 2014 17:31:03 +0100

> probability of detection.

Based on no better reasoning than my own implementation experience
(and the interop bugs _I_ hit), I don't think frame synchronization is
the bit of the spec that will catch people out. I don't expect people
to utterly lose a frame and I don't expect them to 'drift' on the
frames (that is, where they think the frame borders are). More likely
they will get confused on some of the implicit (or less obvious) state
tracking. These are things like:

- Not counting the optional padding flags in the flow control window
(a mistake I made). This will lead to either the endpoint being
blocked but me believing it still has room in the window (plausibly
detected by BLOCKED), or to me sending against a technically closed
window (getting an inevitable GOAWAY).
- Desyncing on HPACK state (very very hard to detect except by human
examination or bizarre statistical analysis on headers).
- Desyncing on SETTINGS values caused by the SETTINGS-SETTINGS/ACK flow.
- Desyncing on next stream IDs caused by not pumping the receive queue
hard enough.

Those are the ideas that came to me in 30 seconds of thought. I see no
way to make those easier to catch.

Am I missing something? Is it easier to frame desync than I thought?
It's possible that my lack of concern is a side-effect of my
implementation language, so please do say if you think I'm missing
something, but otherwise I'm really not that worried about frame
desync.

---

From: Patrick McManus <[email-masked]>
Date: Mon, 21 Jul 2014 12:43:47 -0400

On Mon, Jul 21, 2014 at 11:32 AM, Poul-Henning Kamp <[email-masked]>
wrote:

> it may be as little as

an implementation can always assert on those 10 new 0 bits that we've
effectively added to every frame in draft -14 :)

-P

(or whatever fraction of them they have not explicitly allowed by settings,
of course.)

---

From: Yoav Nir <[email-masked]>
Date: Mon, 21 Jul 2014 18:28:20 -0400

I’ll stand up against it. 

It’s added complexity and added thing to check, and this is not to recover from a network mis-hap, but from an implementation bug. And such implementation bugs are bound to be discovered in stress testing (everybody stress-tests their implementations, right?)  These bugs can happen not only at the edges, but also in intermediaries that mess with the streams, but still, this is something that is bound to happen in testing.

If a frame gets unread, a stream will get pre-maturely ended. This will lead to a protocol error and a GO-AWAY. So Martin’s “pretty quickly” is bounded by the lifetime of a single stream, which should be short enough.

If the stream does not include the content-length header, then the stream will appear to end correctly, but the resource will be missing a piece. That will not be detected, but hopefully this will get detected by the application. I guess an HTTP/1.x implementation could “miss” a TCP segment and get a bad resource. I don’t think this justifies a new integrity mechanism. If anything we might want a resource hash/integrity/checksum field rather than a per-frame field. But that is an idea for another thread.

Yoav