Subject: Clarification for RFC7231 CONNECT (section 4.3.6)

From: Nico Williams <[email-masked]>
Date: Tue, 28 Apr 2015 15:38:58 -0500

Section 4.3.6 of RFC7231 says:

   A server MUST NOT send any Transfer-Encoding or Content-Length header
   fields in a 2xx (Successful) response to CONNECT.  A client MUST
   ignore any Content-Length or Transfer-Encoding header fields received
   in a successful response to CONNECT.

I find the second sentence strange: if a proxy sends a body in a 2xx
response to a CONNECT and the client ignores it, the client will... find
the response body and pass it along to the application as application
protocol data!

Surely that's not really the intent when a proxy violates the first
sentence of the quoted paragraph by sending a response body.

Even if no known proxies send a response body in 2xx CONNECT responses,
it seems more plausible that proxies do just that than that they include
non-zero-length Content-Length but no response body.

Also, it's not stated explicitly, but 2xx responses to CONNECT should
have no body.

Two possible errata spring to mind:

a) explicitly state that 2xx responses to CONNECTs may have no body,

b) relax the requirement for clients to be that they must ignore
response bodies in 2xx CONNECT responses, but that they must still skip
to the end of the response, body included when the proxy sends one.

Before I submit them, I thought I'd ask on the list.  (No such errata
have been filed yet.)

This might seem academic, but I'm fixing a bug in a popular client that
affects protocols tunneled over CONNECT where the server speaks first or
doesn't wait to hear from the client before speaking (e.g, SSH).  In the
process of fixing it I ran into the text quoted above, and it would be
harder to remove handling of Content-Length / Transfer-Encoding for 2xx
responses than to leave it in but ignore the body.  Obviously I can make
this client implement the quoted requirement, but as I said, it seems
quite wrong.

Nico
--

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 07:01:20 +0200

> non-zero-length Content-Length but no response body.

In fact it's worse. I've long seen proxies sending various content-length
values in response to CONNECT. Some send content-length:0, others send
1 million etc... I'm pretty sure they used to do that to workaround broken
intermediaries which ignore the method and only follow content-length, and
used to let a lot of data pass through. I think the text above is well
suited to cover that case. Don't forget that RFC723x aim at describing
what *is* deployed and how to best interoperate.

Regards,
Willy

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 29 Apr 2015 05:09:11 +0000

I've also seen clients send bodies on CONNECT requests...

It's not clear what is supposed to happen, but IME if you pass it 
through, then things don't work, so currently we silently swallow it.

Adrien

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 07:13:25 +0200

> I've also seen clients send bodies on CONNECT requests...

Seen it as well.

> through, then things don't work, so currently we silently swallow it.

In the case I've seen, the body in question was in fact the contents
supposed to be sent. Similarly I think they were advertising a body
to ensure broken intermediaries would correctly transfer the data.
Just like the good old time where it was easy to SSH home via a proxy
by sending a POST with a large body :-)

Cheers,
Willy

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 29 Apr 2015 18:27:17 +1200

> by sending a POST with a large body :-)

Squid used to have that silent-drop behaviour. Then we got complaints
about VoIP audio codecs being screwed up. Nowdays it appears to be SPDY
or WebSockets handshakes occuring in the first packet.

Based on that experience I expect people to start seeing CONNECT +
HTTP/2 SETTINGS frame in one packet shortly (if not already).

Amos

---

From: Zhong Yu <[email-masked]>
Date: Wed, 29 Apr 2015 10:37:55 -0500

> a) explicitly state that 2xx responses to CONNECTs may have no body,

that part is already in the (other) rfc
http://tools.ietf.org/html/rfc7230#section-3.3.3

Zhong Yu
bayou.io

---

From: Nico Williams <[email-masked]>
Date: Wed, 29 Apr 2015 11:40:55 -0500

> used to let a lot of data pass through. I think the text above is well

It sounds like the desired effect is to have any response body in a 2xx
response get treated as application data to be prepended to the
subsequent server->client stream.

If this was the desired effect, then the RFC should have said so.

Ignoring Content-Length has that effect for identity Transfer-Encodings,
but it does NOT for non-identity Transfer-Encodings!

Ignoring Content-Length is only one way to have that effect anyways.
The other is to properly parse the 2xx response, including the body, and
then prepend the body's contents to the server->client stream in the
tunnel.  I don't see why the RFC should describe the implementation
design when there is more than one possible design -- the RFC should
describe behavior.

(Ironically, ignoring Content-Length is probably the easiest way to
implement this behavior in the client I mentioned earlier, because after
reading and parsing the header one byte at a time, the client can leave
the body in the lower layer's buffers to be found by the post-CONNECT
code.  The alternative is to extract the body and copy it to the buffer
used by the post-CONNECT code, which turns out to be a bit more
complicated.)

Presumably the same sort of considerations go for CONNECT request
bodies.  Perhaps the client is sending its first flight of application
data in the CONNECT request body, so the proxy should immediately send
it to the target when the CONNECT succeeds.  This really needs to be
described if it is intended!

> what *is* deployed and how to best interoperate.

Yes, sure, but then it helps to more than prescribe behavior: describing
what deployed implementations do is quite helpful.

Suppose there are proxies that behave in conflicting ways, then that
would mean that clients would need to be configurable as to what they do
with 2xx CONNECT responses.  E.g., one kind of proxy might send policy
information in the 2xx CONNECT response body, while another might send
application data -- the client couldn't tell which behavior the proxy is
using without response headers to tell it, or out-of-band configuration.

ISTM that this section of the RFC needs work, and I may write up an
erratum about it.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 29 Apr 2015 11:44:43 -0500

> HTTP/2 SETTINGS frame in one packet shortly (if not already).

Thanks folks, this is very useful information.  It seems the client I
was talking about then has two bugs, not one, and I'll be fixing one of
them independently of the other.

(The bug I'll be fixing is that the client throws away any data read
along with the 2xx CONNECT response, which breaks SSH in a racy way.
The other bug is that it throws away the 2xx CONNECT response instead of
prepending it to the server->client tunneled stream.  These two bugs are
independent of each other.  The first bug is the urgent one for me.)

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Wed, 29 Apr 2015 15:26:12 -0500

Thanks folks.  I've filed an erratum:

http://www.rfc-editor.org/errata_search.php?rfc=7231&eid=4351

We'll see what happens.

Nico
--

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 29 Apr 2015 21:46:57 +0000

One other thing could use clarification.

We find many browsers do not show the content associated with a 403 
response to CONNECT, and instead show some generic "unable to connect" 
page which is most unhelpful (even when the content is over the size 
threshold for showing "friendly" aka unfriendly errors).  Is there some 
way we can make this more forceful?

Adrien