Subject: Question about proxies forwarding HTTP/1.0 responses

From: Jamie Lokier <[email-masked]>
Date: Thu, 27 May 2004 03:25:57 +0100

In RFC 2616 it says:

   Proxy and gateway applications need to be careful when forwarding
   messages in protocol versions different from that of the application.
   Since the protocol version indicates the protocol capability of the
   sender, a proxy/gateway MUST NOT send a message with a version
   indicator which is greater than its actual version. If a higher
                                   ^^^^^^^^^^^^^^^^^^
   version request is received, the proxy/gateway MUST either downgrade
   the request version, or respond with an error, or switch to tunnel
   behavior.

I'm having trouble parsing the highlighted sentence part.  Does "its
actual version" refer to the version of the message, or the version of
the proxy/gateway?  I think the intention is the version of the
proxy/gateway, but the text's use of "its" is linguistically
ambiguous.

This has come up while deciding whether a proxy which is
HTTP/1.1-capable, which receives a HTTP/1.0 response, should indicate
a HTTP/1.0 version in its response to the client.

To be more precise, I'm fairly sure the proxy should indicate it is
HTTP/1.1 capable, because that's the capability of the message sender,
which is the proxy.  (It's not a tunnel).  My question really is about
the observed behaviour of widely deployed proxies over the past few
years.  Does anyone know what the behaviour of deployed proxies is?

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 26 May 2004 22:42:09 -0600 (MDT)

On Thu, 27 May 2004, Jamie Lokier wrote:

> the proxy/gateway?

The version of the proxy/gateway.

> a HTTP/1.0 version in its response to the client.

It seems to me that this case is not related to the rule you cited
above. In your case, the message has a lower, not higher version
compared to the proxy version, and the above downgrading rule is
specific to requests, not responses. The proxy should indicate its own
version in the response unless it suspects the client cannot handle
proxy's version. Versions are supposed to be a hop-by-hop mechanism.

> HTTP/1.1 capable, because that's the capability of the message
> sender, which is the proxy.  (It's not a tunnel).  My question
> really is about the observed behaviour of widely deployed proxies
> over the past few years.  Does anyone know what the behaviour of
> deployed proxies is?

Proxies that claim HTTP/1.1 support usually send HTTP/1.1 messages. A
few proxies have an option to configure the protocol version in the
messages they sent. There are a few proxies that support many HTTP/1.1
features but do not use HTTP/1.1 versions. Squid is one of them.

BTW, RFC 2145 has more specific rules regarding HTTP versions.

HTH,

Alex.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 27 May 2004 14:41:01 +0100

Alex Rousskov wrote:
> messages they sent.

Thanks Alex.  That answers my question.

I've been investigating the various workarounds in popular HTTP
servers and proxies, to see what folklore a new internet-facing server
should include for robustness.

Just fyi, I was looking at this part of Apache's default config:

  BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0

The "nokeepalive" directive disables persistence if a request comes
from a user-agent containing "MSIE 4.0b2;" _and_ if there's no Via header.

The "downgrade-1.0" makes the server pretend it's a HTTP/1.0 request.

The "force-response-1.0" makes the server send "HTTP/1.0" in the
response.

I wanted to decide whether the latter two can be ignored if there's a
Via header -- so that the user-agent pattern matching may be skipped
altogether.  Same applies to the other BrowserMatch directives.

Of course the only way to be sure is testing, but you have given me
confidence that it is fine to omit all the user-agent checks which
affect hop-by-hop if there's a Via header.

> Versions are supposed to be a hop-by-hop mechanism.

Yes, but because of some broken HTTP implementations there are
workarounds deployed.  I wondered if a proxy would forward a HTTP/1.0
response and label it as if the sender was HTTP/1.0 as part of the
workarounds which kept the net working in 1997 or so.

> There are a few proxies that support many HTTP/1.1 features but do
> not use HTTP/1.1 versions. Squid is one of them.

Do you know of any practical compatibility reason why Squid must
report itself as HTTP/1.0 even if it supported all HTTP/1.1 proxy
requirements?  Or is it simply that it doesn't implement the
requirements yet?

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 09:09:35 -0600 (MDT)

On Thu, 27 May 2004, Jamie Lokier wrote:

> ... you have given me confidence that it is fine to omit all the
> user-agent checks which affect hop-by-hop if there's a Via header.

Please note that I said "usually send". All products I know have
agent-specific workarounds. I do not know whether it is OK to do no
agent-specific workarounds if there is a Via header. I suspect it may
NOT be OK in some cases since Via header does not necessarily indicate
that the message will be "fixed" by another proxy on the way to the
broken agent.

Many things in the path may add Via headers, some of which are not
even HTTP proxies. Consider ICAP servers, for example (RFC 3507).

> workarounds deployed.  I wondered if a proxy would forward a
> HTTP/1.0 response and label it as if the sender was HTTP/1.0 as part
> of the workarounds which kept the net working in 1997 or so.

IMHO, the Internet still works on workarounds, to a large extent. This
sad situation is not going to change until we aim higher than just
"running code". I am not sure we ever will.

> requirements?  Or is it simply that it doesn't implement the

Last time I checked, Squid did not support all RFC 2616 MUSTs and so
Squid developers [correctly] decided not to use HTTP/1.1 version in
the headers. Here is a probably outdated HTTP/1.1 checklist for Squid:
	http://devel.squid-cache.org/squidhttp1.1.htm

Note that announcing HTTP/1.1 support buys you little. For good or
bad, you can do persistent connections, ranges, Vary, and other key
HTTP/1.1 features while announcing HTTP/1.0 compliance only.

Alex.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 27 May 2004 16:39:33 +0100

> that the message will be "fixed" by another proxy on the way to the

That's true, but the workarounds I wish to omit if Via is present are
specifically only the hop-by-hop ones, equivalent to Apache's
downgrade-1.0 and force-response-1.0.

The nokeepalive workaround is already only done by Apache if Via isn't
present.  Although it isn't documented, Squid also has a workaround
which disables keep-alives with Netscape 3.x -- one which Apache
curiously doesn't have in its default configuration, although Apache's
documentation mentions it.  Squid's workaround is also only done if
Via isn't present.

So that's fine.

force-response-1.0 means the server announces HTTP/1.0 instead of
HTTP/1.1.  If it's true that real HTTP/1.1 proxies do _not_ pass along
the HTTP version announcment of the response (hence the subject of
this thread), and HTTP/1.0 proxies always announce HTTP/1.0, then
force-response-1.0 would have no effect on the broken clients when the
request is via a proxy.

So that's fine if it's disabled by Via.

downgrade-1.0 is really there to disable chunked responses.  If real
HTTP/1.1 proxies _do_ convert non-chunked responses from the server to
chunked HTTP/1.1 responses to the client, assuming the client
announced HTTP/1.1, then downgrade-1.0 would not be effective in this
case anyway.

That means the only functional place for that workaround is at the
nearest proxy to the client, if the proxy is HTTP/1.1.  It is not
effective at the server if there's a proxy and it converts a
non-chunked response to HTTP/1.1 chunked.

I guess that one depends on the behaviour of real proxies.  So that's
banother question: do real deployed HTTP/1.1 proxies convert
non-chunked responses to chunked?

> even HTTP proxies. Consider ICAP servers, for example (RFC 3507).

The question is whether those things modify the hop-by-hop
characteristics of the message -- i.e. HTTP version, keep-alive, and
chunkedness.  Hmm.  I'd guess ICAP servers would have to, simply
because deployed origin servers already check Via for determining
whether to apply a nokeepalive rule.

> "running code". I am not sure we ever will.

Indeed, I was not too impressed at the code in _any_ of the servers
and proxies I looked at for parsing HTTP headers.  They work as long
as their peers send politely formatted messages.  Message headers with
embedded CRs and spaces in the field names and continuations are
interpreted in lots of different ways.

> HTTP/1.1 features while announcing HTTP/1.0 compliance only.

Not to mention that there are plenty of server implementations which
are far from RFC 2616 compliant that announce themselves as HTTP/1.1,
so it cannot be taken too seriously.

Does announcing HTTP/1.1 actually buy you anything at all?

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 10:00:54 -0600 (MDT)

On Thu, 27 May 2004, Jamie Lokier wrote:

> do real deployed HTTP/1.1 proxies convert

Yes, I think so (their actions may depend on HTTP transaction in
question, of course).

> Does announcing HTTP/1.1 actually buy you anything at all?

Your marketing materials can mention HTTP/1.1 "support".

You can also save a few bytes by not transmitting Connection:
keep-alive headers (assuming that's more common than closing the
connection).

Alex.

---

From: patrick mcmanus <[email-masked]>
Date: Thu, 27 May 2004 12:38:54 -0400

> Does announcing HTTP/1.1 actually buy you anything at all?

It can get you chunked responses.. if the server is dynamically
generating the response your alternatives under HTTP/1.0 cost you either
  a] latency while the server buffers or
  b] the loss of a persistent connection

it's not much, but it is something.

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 10:49:40 -0600 (MDT)

On Thu, 27 May 2004, patrick mcmanus wrote:

> It can get you chunked responses..

... and can messages with HTTP/1.0 version strings and TE: chunked.
Recall that the context here is not HTTP/1.1 features, but HTTP/1.1
version string in the message. That alone does not seem to buy you
much. Cache-control support is a similar example; you do not need an
HTTP/1.1 message to support Cache-controls.

Alex.

---

From: patrick mcmanus <[email-masked]>
Date: Thu, 27 May 2004 12:51:40 -0400

> non-chunked responses to chunked?

I have done this for two reasons:

 * the server's response didn't have a content-length, but my proxy
wanted to have a persistent connection with the client.. The proxy can
chunk and stream the response simultaneously. 

 * to add some other kind of transfer-encoding in addition to chunked.
Commonly, this would be gzip. 

-Pat

---

From: patrick mcmanus <[email-masked]>
Date: Thu, 27 May 2004 13:01:44 -0400

> ... and can messages with HTTP/1.0 version strings and TE: chunked.

HTTP/1.0 with TE: chunked cannot safely cause a chunked response. That
combination is much more likely to be a HTTP/1.0 proxy that propogated a
TE header it didn't know was hop-to-hop. There is a real chance that
proxy won't be able to interpret the response correctly.

> HTTP/1.1 message to support Cache-controls.

cache-control is end-to-end, iirc.. Totally different, right?

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 27 May 2004 11:18:03 -0600 (MDT)

> proxy won't be able to interpret the response correctly.

Oof. I should have been more careful when writing HTTP header names.
Sorry. What I meant to say is that it is safe to respond with chunked
HTTP/1.0 response to an HTTP/1.1 request:

	GET / HTTP/1.1
	...

	HTTP/1.0 200 OK
	Transfer-Encoding: chunked
	...

You are absolutely right that TE request-header has nothing to do with
the ability to use chunking with HTTP/1.0 messages.

Alex.

---

From: patrick mcmanus <[email-masked]>
Date: Thu, 27 May 2004 13:32:05 -0400

> HTTP/1.0 response to an HTTP/1.1 request:

mea culpa too. I had read this thread as being about request-side
'announcements' of version level when it was about response-side.. sorry
bout that.