Subject: Is forwarding hop-by-hop headers a MUST-level violation?

From: Alex Rousskov <[email-masked]>
Date: Tue, 6 Jul 2004 11:25:09 -0600 (MDT)

Hi,

	Some proxies forward some hop-by-hop headers. While discussing
a related Apache bugzilla entry[1], Nick Kew suggested that forwarding
a hop-by-hop header by a proxy is only a SHOULD-level violation, if it
is a violation at all.

Section 13.5.1 of RFC 2616 says:

   For the purpose of defining the behavior of caches and non-caching
   proxies, we divide HTTP headers into two categories:

      - End-to-end headers, which are  transmitted to the ultimate
        recipient of a request or response. End-to-end headers in
        responses MUST be stored as part of a cache entry and MUST be
        transmitted in any response formed from a cache entry.

      - Hop-by-hop headers, which are meaningful only for a single
        transport-level connection, and are not stored by caches or
        forwarded by proxies.

The "Hop-by-hop headers ... are not stored or forwarded" claim is not
an explicit MUST- or SHOULD-level requirement. It looks like an
assertion or definition. Given the negative impact forwarding some
hop-by-hop headers would have, I always assumed that proxies MUST NOT
forward any hop-by-hop headers. Was I wrong?

Is it worth changing the above language to a more consistent and
explicit:

      - Hop-by-hop headers, which are meaningful only for a single
        transport-level connection. Hop-by-hop headers MUST NOT be
        stored by caches or forwarded by proxies.

BTW, the above "End-to-end headers" rules do not say anything about
forwarding End-to-end request headers. Perhaps the wording should be
polished to include request headers as well?

Thanks,

Alex.

[1] http://issues.apache.org/bugzilla/show_bug.cgi?id=15861

P.S. There are other MUSTs specific to some hop-by-hop headers,
including those in Connection header list. Here, we are talking about
the general rule for all hop-by-hop headers.

---

From: Jeffrey Mogul <[email-masked]>
Date: Fri, 09 Jul 2004 17:41:35 -0700

I suspect this is indeed a bug in the spec.  However, it's
possible (my memory is useless on this) that we decided that it
was not necessary to specify this because other parts of the
document were supposed to do it.  (Some of my co-authors had a
strong aversion to making the same normative requirement in two
places.)

For example, TE is listed here as hop-by-hop, and later (14.39)
the spec says "the keyword MUST be supplied within a Connection
header" (although now I'm not whether we want the keyword to be
covered by Connection, or the field name itself).  However,
13.5.1 also lists "Trailers" (sic!) and the spec for "Trailer"
doesn't include this prohibition.  So I guess we botched it.

    

Yeah, although it (in effect) says that they have to be forwarded
on cache hits, so it would be odd not to forward them on cache
misses (or other non-cache-hit situations).

-Jeff

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 9 Jul 2004 22:41:02 -0600 (MDT)

On Fri, 9 Jul 2004, Jeffrey Mogul wrote:

> I suspect this is indeed a bug in the spec.

If there are no objections, could this be added to the errata list
after discussing the exact fix? The text below contains a possible
solution, but there are probably better ways to fix the bug.

> However, it's possible (my memory is useless on this) that we
> decided that it was not necessary to specify this because other
> parts of the document were supposed to do it.  (Some of my
> co-authors had a strong aversion to making the same normative
> requirement in two places.)

Checking:

      - Connection:
	"MUST NOT be communicated by proxies over further connections"

      - Keep-Alive
	RFC 2068 does not seem to have an explicit MUST for it

      - Proxy-Authenticate
	"SHOULD NOT be passed on to downstream clients"

      - Proxy-Authorization
	Nothing explicit except for "MAY relay" for cooperative auth.

      - TE
	"MUST be supplied within a Connection header"

      - Trailer(s)
	No explicit rules

      - Transfer-Encoding
	No explicit rules

      - Upgrade
	"MUST be supplied within a Connection header"

So we have MUSTs, a SHOULD, and "nothing" dancing all over the place.
To make this consistent, I would suggest either

	- make sure that each header has an explicit "MUST NOT
	  forward" and "MUST supply within a Connection header" rules.
	  Add/polish current header sections as needed.

	- or remove existing rules from individual header descriptions
	  and add an "explicit "MUST NOT forward" and "MUST supply
	  within a Connection header" rules to Section 13.5.1

The latter sounds like a better approach, especially since Keep-Alive
does not have a dedicated section in RFC 2616.

Other opinions?

> For example, TE is listed here as hop-by-hop, and later (14.39) the
> spec says "the keyword MUST be supplied within a Connection header"
> (although now I'm not whether we want the keyword to be covered by
> Connection, or the field name itself).

I paused at that as well. We seem to have no choice now. Some
implementation may already foolishly rely on ho-by-hop headers always
being in the Connection header. We do not want to worsen the situation
for them when newer (post-errata) implementations stop listing
hop-by-hop headers in the Connection.

> Yeah, although it (in effect) says that they have to be forwarded on
> cache hits, so it would be odd not to forward them on cache misses
> (or other non-cache-hit situations).

I am sure some implementor will think (or, at least, argue) that
current end-to-end definition applies to caches only, not proxies.
IIRC, there have been arguments like that in the past, in contexts
like the Age header. It is better to be explicit. How about:

----------------------
13.5.1 End-to-end and Hop-by-hop Headers

   For the purpose of defining the behavior of proxies (including
   caches), we divide HTTP headers of a given message into two
   categories: hop-by-hop and end-to-end.

   Hop-by-hop headers are meaningful only for a single
   transport-level connection. Hop-by-hop headers are:

          - Connection
          - Keep-Alive
          - Proxy-Authenticate
          - Proxy-Authorization
          - TE
          - Trailer
          - Transfer-Encoding
          - Upgrade

   and all other headers not defined in this specification
   but listed in a Connection header. All hop-by-hop headers
   (other than Connection) MUST be listed in the Connection header.
   Proxies MUST NOT forward hop-by-hop headers.

   All other headers are end-to-end headers. They are
   intended for the ultimate recipient of a message and
   can be meaningful for intermediaries. Proxies MUST forward
   end-to-end headers.
--------------------------

And then remove per-header MUSTs and a SHOULD mentioned above?

Thanks,

Alex.

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 10 Jul 2004 13:22:38 +0100

> 	RFC 2068 does not seem to have an explicit MUST for it

However RFC 2068 does imply that Keep-Alive should appear in the
Connection header -- at least when the client knows its talking to a
proxy.

   The problem was that some existing 1.0 clients may be sending
   Keep-Alive to a proxy server that doesn't understand Connection,
   which would then erroneously forward it to the next inbound server,

>    transport-level connection. Hop-by-hop headers are:
...
>           - Transfer-Encoding
...
> 
>    All hop-by-hop headers (other than Connection) MUST be listed in
>    the Connection header.

I haven't noticed Transfer-Encoding being listed in the Connection
header to date.  All proxies need to understand Transfer-Encoding to
determine the message boundary, so I see no need for it to appear in
Connection.  Can't the above text safely say "(other than Connection
and Transfer-Encoding)"?

-- Jamie

---

From: Jeffrey Mogul <[email-masked]>
Date: Mon, 12 Jul 2004 11:36:02 -0700

How about:
    
    ----------------------

    
	      - Connection
	      - Keep-Alive
	      - Proxy-Authenticate
	      - Proxy-Authorization
	      - TE
	      - Trailer
	      - Transfer-Encoding
	      - Upgrade

    
I'd note that "Keep-Alive" is not actually specified as an
HTTP/1.1 header, but is included in this list to prevent
compatibility problems with older code.

I would also suggest adding something like:

    A proxy MUST NOT forward one of the hop-by-hop headers listed
    above even if it is NOT listed in a Connection header of the
    received message.

lest some implementor believe that it is not necessary to check
for errors in the use of Connection.

Alex suggests

Hmm.  I'm not sure this is wise, given the possibility that
an implementor who sees this change between versions of the
spec will notice that the new version of 13.5.1 still enforces
the rule.  In other words, someone might misunderstand a local
wording change as a global requirements change.  I would not
change language in the spec, at this point, if the actual
requirement hasn't changed.

-Jeff

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 12 Jul 2004 19:49:15 +0100

Jeffrey Mogul wrote:
>        All hop-by-hop headers (other than Connection) MUST be listed
>        in the Connection header.

Again, existing implementations don't list Transfer-Encoding in the
Connection header.  This is very common and conforming behaviour.

Is this proposal meant to imply that all such existing implementations
are no longer conforming?

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 12 Jul 2004 12:59:45 -0600 (MDT)

> Connection header.  This is very common and conforming behaviour.

Understood. Most existing implementations do not list most of the
explicitly documented hop-by-hop headers in the Connection header.

> Is this proposal meant to imply that all such existing
> implementations are no longer conforming?

Yes. Why is that a problem? We are changing the old protocol (RFC
2616) to fix a bug. Old implementations might be compliant with RFC
2616 (the vast majority is not). New or updated implementations may be
compliant with RFC XXXX. The change does not break old
implementations, and might actually help some. I do not see a
compelling reason to make an exception for Transfer-Encoding. Do you?

Note that several HTTP errata entries would have a similar effect of
making implementations based on old/buggy RFC 2616 incompliant with
the updated HTTP/1.1 standard. That fact alone does not prove me
right, of course.

Thanks,

Alex.

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 12 Jul 2004 21:42:01 +0100

Alex Rousskov wrote:
> I do not see a compelling reason to make an exception for
> Transfer-Encoding. Do you?

Yes.  The reason is similar to making an exception for Connection.
Any agent which parses and understands Connection (i.e. to filter out
named headers) _must_ also understand and parse Transfer-Encoding, as
it determines the message boundary.

Therefore there is nothing to be gained by putting Transfer-Encoding
in Connection.  Mentioning it is always redundant.

This isn't true of the other hop-by-hop headers.

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 12 Jul 2004 14:52:30 -0600 (MDT)

> it determines the message boundary.

Understanding message boundary is often independent of forwarding
headers. An implementation may have no problem isolating the message,
but may still forward the Transfer-Encoding hop-by-hop header (which
may or may not represent encoding for the next hop). Caching proxies
are especially vulnerable to this because their client-side code is
often rather different/isolated from the server-side code.

Alex.

---

From: David Morris <[email-masked]>
Date: Mon, 12 Jul 2004 14:32:20 -0700 (PDT)

There should be an exception for ALL headers declared in the
specification. It is absurd to bloat the size of HTTP headers to declare
information contained in the RFC.

It is a long time since I spent time thinking about the specification but
my recollection is that intent is to provide extensibility by providing a
mechanism for declaring new hop-hop headers.

Dave Morris

---

From: David Morris <[email-masked]>
Date: Mon, 12 Jul 2004 14:34:03 -0700 (PDT)

If the proxy implementor can't read and understand the RFC notion of
hop-hop headers, how can you have confidence they'll even look at the
content header and use it to control filtering?

---

From: Robert Collins <[email-masked]>
Date: Tue, 13 Jul 2004 07:48:59 +1000

> information contained in the RFC.

The corollary to this is that new editions of HTTP/1.1 that add
hop-by-hop headers will either:
* break existing conformant applications if the *don't* list the
headers.
* add just those headers, leading to more special cases.

Its absurd to optimise out ~30 chars like this.

Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 13 Jul 2004 03:11:18 +0100

> content header and use it to control filtering?

I think David means that if a proxy doesn't understand the notion of
HTTP/1.1 hop-by-hop headers which must be removed, how can you have
confidence that it'll look at the Connection header at all?

But, I agree with this plan to put nearly all hop-by-hop headers into
Connection, for the following reasons.

There are (rather absurdly) *two* versions of HTTP/1.1 (RFC 2068 and
RFC 2616).  The list of hop-by-hop headers to ignore is different in
each version.  Neither standard lists a superset of the other.

That means to be compatible with proxies which may be based on either
standard, you MUST put TE, Public, Proxy-Authorization and Trailers
into the Connection header.  Those are the headers which appear in one
but not the other.

It isn't stated like that in RFC 2616, but that's what you must do to
be safe as there are quite a few implementations which were written
following RFC 2068.

So for example you may see a proxy forwarding the TE request header,
and that proxy is conforming with RFC 2068 (which doesn't mention TE),
but a client will only get the desired result if it puts TE in the
Connection header.

So I personally do agree with putting hop-by-hop headers into
Connection, and specifying that clearly.

Of all the hop-by-hop headers listed in RFC 2068 and RFC 2616, the two
that I think don't ever need to go in the list are Connection itself,
and Transfer-Encoding for the reason I gave.  All the others have good
reasons to be in Connection.

> often rather different/isolated from the server-side code.

Well, if a proxy is so broken that it blindly forwards
Transfer-Encoding (and it isn't just tunnelling), then (a) it's going
to fail very quickly in the real world - we really don't have to worry
about a proxy doing this, and (b) as David said: a proxy coded like
that isn't going to filter headers based on Connection anyway.

[ More dangerous IMHO is the rule that "Content-Type:
multipart/byteranges" does funny things with the message boundary, and
a proxy needs to be aware of that.  I have seen implementations which
will do the multipart parsing for any content type matching
multipart/*, which is not conformant with RFC 2616 (see the section on
Message Length).  A real proxy needs to at least match
multipart/x-byteranges; I'm not sure if in the real world it needs to
match multipart/* as well.  IMHO the multipart message length rule is
broken as chunked would have been fine for it. ]

-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 12 Jul 2004 22:27:47 -0600 (MDT)

On Mon, 12 Jul 2004, Jeffrey Mogul wrote:

> I'd note that "Keep-Alive" is not actually specified as an HTTP/1.1
> header, but is included in this list to prevent compatibility
> problems with older code.

Agreed.

> for errors in the use of Connection.

Agreed, I would polish the wording a little more:

	A proxy MUST NOT forward any hop-by-hop header, regardless
	of whether the header is listed in a Connection header of
	the received message.

> Hmm.  I'm not sure this is wise, given the possibility that an
> implementor who sees this change between versions of the spec will
> notice that the new version of 13.5.1 still enforces the rule.  In
> other words, someone might misunderstand a local wording change as a
> global requirements change.  I would not change language in the
> spec, at this point, if the actual requirement hasn't changed.

Good point. I was mostly worried about the existing SHOULD requirement
for Proxy-Authenticate that would contradict the newly added MUST.
Should we replace that SHOULD with a reference to the corresponding
new MUST in 13.5.1? And leave other header-specific sections in peace?

Thanks,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 12 Jul 2004 22:53:12 -0600 (MDT)

On Mon, 12 Jul 2004, David Morris wrote:

> hop-hop headers,

RFC notion is defined by normative rules in the RFC. Please note that
current normative rules are incomplete and inconsistent.

> how can you have confidence they'll even look at the content header
> and use it to control filtering?

I am confident that most implementations will remain incompliant even
if we fix the spec (compliance is not driven by correct specs).
However, leaving the spec buggy, despite real implementors
questions/concerns, is bad for everybody involved (compliance with
incorrect specs is impossible or meaningless).

On Tue, 2004-07-13 at 07:32, David Morris wrote:

> specification. It is absurd to bloat the size of HTTP headers to
> declare information contained in the RFC.

I would agree 100% if we were talking about changing an Internet Draft.
Unfortunately, we are working in a context of an existing IETF
Standard that needs to be fixed. As developer questions and actual
implementations indicate the "information contained in the RFC [2616]"
is imprecise, inconsistent, or missing, depending on the hop-by-hop
header. Existing implementations may honestly forward some hop-by-hop
headers (while also honoring the Connection header logic). Thus, we
cannot simply reply on [hop-by-hop header] information contained in
the RFC".

If we add explicit hop-by-hop headers to Connection, the above
mentioned implementations will be automatically "fixed" when they
receive messages from newer HTTP agents. I suspect that chances of
breaking an old implementation by listing explicit hop-by-hop headers
are non-zero, but hope that they are negligible.

The proposed new wording does two things:

	1) documents the intent of the original wording in a normative
	and consistent way (proxies MUST NOT forward)

	2) attempts to increase the probability that old and new
	implementations will do the right thing (by listing all
	hop-by-hop headers in the Connection header)

I do not have strong feelings about (2). Adding a few bytes to a few
messages does not bother me much, but I am worried that, in some
corner cases, listing more headers in Connection might expose
currently undetected vulnerabilities in old products.

I did not hear any objections to (1). Can we declare consensus on that
part of the fix?

Thank you,

Alex.

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 13 Jul 2004 12:12:14 +0100

>From my corner, hand raised in a yes-from-me way.

Note that the semantics of the hop-by-hop header Proxy-Authorization
are that it MAY be forwarded.  So wording of the hop-by-hop section
should perhaps not say that Proxy-Authorization MUST be removed, as it
would be a contradiction.

> currently undetected vulnerabilities in old products.

I wouldn't be surprised to find some old products check for Connection
== "close", or !strncmp(connection, "close") if you see what I mean.
I've never investigated it though; it may just be over-carefulness on
my part that I insist on making close the first token when it's
present.  (And when it's not, if Keep-Alive is present I make that the
first token for the same reason).

I am not sure if it makes sense to require TE, Trailers and
Proxy-Authorization are listed in Connection because those weren't in
the hop-by-hop list of RFC 2068.  Perhaps they are all sufficiently
harmless in the cases where they might be forwarded by an RFC 2068 proxy.

-- Jamie

---

From: David Morris <[email-masked]>
Date: Wed, 14 Jul 2004 11:32:33 -0700 (PDT)

> currently undetected vulnerabilities in old products.

The issue of a few bytes is when the additional constant payload results
in an extra MTU on the network for a significant percentage of requests.
Worse when the window size is exceeded. About 50 bytes would be a 10%
probability that a dial-up user required an additional packet (if my
memory of MTUs is correct).

Since we know there are many broken implementations, no matter what we
specify, that will continue. Better that the result is a precise
definition of what should be rather than attempts to fix broken code.

It will be easier to fix broken code if the specification is correct and
makes sense rather than including kludges.

The fundamental issue is that there are origin server headers (in the
request) and there are headers which apply to the gray area in-between.
Proxy authentication can only apply to a single node because we weren't
wise enough (or motivated) to design an extensible architecture. But I
have designed software which appears at the HTTP protocol level as a proxy
but which would forward the authentication to the next proxy in line. I
can believe that my proxy would not remove chunked encoding because of
buffering issues end-end response time, etc. So to mandate removal of that
header doesn't really apply. Hop-hop headers MAY convey information for an
intermediate recipient. If that recipient understands the header and
complys with its implications, it MUST remove the header. Hop-hop headers
are the pre-defined set in the RFC and headers specified in the
Connection: header.

A proxy which doesn't understand a hop-hop header could reject the request
and perhaps MUST not cache the response but removal of the header could do
as much damage as forwarding it.

In the end, either forwarding a hop-hop header will be relatively benign
OR it will break the transfer. Broken transfers will result in the
potential for repair. Not a bad thing.

Dave Morris

---

From: Alex Rousskov <[email-masked]>
Date: Thu, 15 Jul 2004 23:54:55 -0600 (MDT)

On Wed, 14 Jul 2004, David Morris wrote:

> Better that the result is a precise definition of what should be
> rather than attempts to fix broken code.

I think the proposed definition is precise. The fact that it may fix
some broken code is a side-effect.

> It will be easier to fix broken code if the specification is correct
> and makes sense rather than including kludges.

I do not think the proposed rules are kludges.

> The fundamental issue is that there are origin server headers (in
> the request) and there are headers which apply to the gray area
> in-between. Proxy authentication can only apply to a single node
> because we weren't wise enough (or motivated) to design an
> extensible architecture. But I have designed software which appears
> at the HTTP protocol level as a proxy but which would forward the
> authentication to the next proxy in line. I can believe that my
> proxy would not remove chunked encoding because of buffering issues
> end-end response time, etc. So to mandate removal of that header
> doesn't really apply.

A proxy is free to add the same header if that does not violate the
protocol. Recall that we only define proxy input and output. If you
want to optimize and keep the header, instead of removing and then
adding it back, it is perfectly fine in this context.

> Hop-hop headers MAY convey information for an intermediate
> recipient. If that recipient understands the header and

Agreed, except there is no "If that recipient..." part. The recipient
MUST remove (or, in HTTP/1.1 terminology, MUST NOT forward). Is some
cases, the recipient MAY add. In those cases, it may be impossible to
test whether the recipient is internally compliant or buggy, but that
is not important.

> A proxy which doesn't understand a hop-hop header could reject the
> request and perhaps MUST not cache the response but removal of the
> header could do as much damage as forwarding it.

Not if we are still talking about HTTP. Removal of a hop-by-hop header
should do no damage. If there is damage from doing that, the header
was incorrectly designed (and that problem should be fixed in that
header specification, not in the general case).

The bottom line: do you have any objections to fixing the RFC by
saying "proxies MUST NOT forward hop-by-hop headers" explicitly? If
yes, what are those objections?

Let's leave the second change (i.e., listing hop-by-hop headers in the
Connection header) aside, for now.

Thank you,

Alex.

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 16 Jul 2004 00:02:55 -0600 (MDT)

On Tue, 13 Jul 2004, Jamie Lokier wrote:

> should perhaps not say that Proxy-Authorization MUST be removed, as
> it would be a contradiction.

I hesitate opening another debate around this. We simply lack the
proxy terminology (client-side actions versus server-side actions,
etc.) to perfectly express what we want. I would just use the existing
language, but make it normative.

> == "close", or !strncmp(connection, "close") if you see what I mean.

I saw some _new_ products that do that. That is one reason why I am
not pushing for (2).

Thanks,

Alex.

---

From: David Morris <[email-masked]>
Date: Fri, 16 Jul 2004 17:31:00 -0700 (PDT)

> yes, what are those objections?

Sorry, I wasn't clear ... I've no issue with that change ...

> Connection header) aside, for now.

Yes, my issue is with specifications around clarification of what a
"hop-by-hop" header is.

Dave Morris

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 23 Jul 2004 22:30:35 +0100

> I saw some _new_ products that do that. That is one reason why I am

Ouch.  Do the products you've seen check for (effectively)
!strcmp (connection, "close") or !strncmp (connection, "close", 5)?

If the former, is transmitting _two_ Connection headers, one of which
is literally "Connection: close", an adequate workaround for those products?

Thanks,
-- Jamie

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 23 Jul 2004 16:18:00 -0600 (MDT)

> is literally "Connection: close", an adequate workaround for those products?

Implementations and their bugs vary, of course. As you may suspect,
some cannot handle multiple Connection: headers (some will only see
the first "line", some only the last one, and some will see none).

It is probably impossible to change any RFC 2616 functionality without
breaking some buggy implementations. I am not sure whether we should
take those implementations into account. If we should, then we should
not require listing known hop-by-hop headers in the Connection header.

Alex.