Subject: Relative ordering of fields in HTTP request

From: Srikanth Kandula <[email-masked]>
Date: Wed, 5 Nov 2003 01:31:55 -0500 (EST)

Hi,

My current understanding is that header fields (say Accept: and
Connection: ) can fit inside a request header in any order.  Is this
correct?  If yes, is there a method by which one can enforce that a
certain field appear in certain order.

More generally can the HTTP Server have a say in the ordering of
request header fields?

Thanks,
Srikanth

---

From: Scott Lawrence <[email-masked]>
Date: Wed, 5 Nov 2003 09:53:39 -0500 (EST)

Srikanth Kandula <[email-masked]> writes:

> correct?

Yes.

> If yes, is there a method by which one can enforce that a

No.

-- 
Scott Lawrence        
  http://skrb.org/scott/

---

From: "Joris Dobbelsteen" <[email-masked]>
Date: Wed, 5 Nov 2003 17:04:19 +0100

There is no way to enforce an order of the headers.

The server has no say over the data it receives, except that it could deny
or accept the request. One should look at the entire request as a whole and
not scan through it expecting ordering.

Next a proxy is allowed to relocate headers or compact them if it wants to.
For example, multiple 'Accept' headers could be compacted into a single
'Accept' header.

- Joris

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Srikanth Kandula
> Sent: Wednesday, 5 November 2003 7:32
> To: [email-masked]
> Subject: Relative ordering of fields in HTTP request
> 
> 
> Hi,
> 
> Connection: ) can fit inside a request header in any order.  
> Is this correct?  If yes, is there a method by which one can 
> enforce that a certain field appear in certain order.
> 
> More generally can the HTTP Server have a say in the ordering 
> of request header fields?
> 
> Thanks,
> Srikanth
>

---

From: Jeffrey Mogul <[email-masked]>
Date: Wed, 05 Nov 2003 10:55:14 -0800

Scott Lawrence <[email-masked]> writes:

    
    Yes.

    No.
    
Scott is basically correct, but if someone out there misreads
this as "HTTP header field order does not matter at all" then
this is a mistake.

RFC2616, section 4.2 says:

   Multiple message-header fields with the same field-name MAY be
   present in a message if and only if the entire field-value for that
   header field is defined as a comma-separated list [i.e., #(values)].
   It MUST be possible to combine the multiple header fields into one
   "field-name: field-value" pair, without changing the semantics of the
   message, by appending each subsequent field-value to the first, each
   separated by a comma. The order in which header fields with the same
   field-name are received is therefore significant to the
   interpretation of the combined field value, and thus a proxy MUST NOT
   change the order of these field values when a message is forwarded.

Note the "MUST NOT" in the final sentence.

For example, on a path with a proxy the client sends a request with:

	HeaderX: A, B, C, D

or it could send the same request with:

	HeaderX: A, B
	HeaderX: C, D

It is legal for the proxy to convert that second form to

	HeaderX: A, B, C, D

but it is NOT legal for the proxy to convert it to

	HeaderX: C, D
	HeaderX: A, B

because then the ultimate recipient would see "C, D, A, B"
instead of "A, B, C, D".

This only applies to "message-header fields with the same
field-name"; section 4.2 also says:

   The order in which header fields with differing field names are
   received is not significant. 

and so it is definitely legal to reorder (for example)

	Accept: text/plain
	Connection: close

to become

	Connection: close
	Accept: text/plain

-Jeff

P.S.: Alex: does your proxy validation suite check for that
MUST NOT from section 4.2?

---

From: Alex Rousskov <[email-masked]>
Date: Wed, 5 Nov 2003 12:51:57 -0700 (MST)

> MUST NOT from section 4.2?

Yes; the RFC text[1] is cross-referenced with test cases[2] so you can
see what is covered. The test suite checks for order preservation
under two basic scenarios:

	* a proxy MUST NOT change the order of multiple header
	  field-values with one field-name (8 test cases)

	* a proxy MUST NOT change the order of cached multiple header
	  field-values with one field-name (4 test cases)

The first test clause contains test cases for both requests and
responses. There are implementations that violate some of the above
test cases, IIRC.

Note that HTTP does have a [theoretical] problem in this area. From
RFC 2616 point of view, extension headers are not #value headers.
However, in the specification that defines a given extension header
(RFC X), the header may be declared using a #value construct. This
means that X-aware intermediaries might split the header while
X-unaware intermediaries might reorder or drop some of the splitted
values. This probably means that HTTP extensions must not declare
#value headers, but they do.

The reason for this [theoretical] problem is that HTTP relies on
header definition in some spec to determine header syntax (e.g., "is
it a list?")  instead of having a fixed set of syntax constructs that
can be recognized without knowing the header semantics. This is not
something one can fix today, of course.

Alex.

[1] http://coad.measurement-factory.com/cgi-bin/coad/SpecCgi?spec_id=rfc2616#excerpt/rfc2616/e3cfaf4252ec84762a2b6860fd58b599
[2] http://coad.measurement-factory.com/cgi-bin/coad/GraseIndexCgi?index_ids=test_clause/rfc2616/endHdr-fwd-order,test_clause/rfc2616/endHdr-hit-order

---

From: Robert Collins <[email-masked]>
Date: Thu, 06 Nov 2003 07:37:39 +1100

> and so it is definitely legal to reorder (for example)

While it's legal, we've found that some firewalls expect certain headers
within the first X Kb, or the first tcp data payload, and reject or drop
requests if they aren't there. (I don't recall the specific headers
offhand - it's in the squid bugzilla if you're interested). We recently
fixed squid to not reorder headers that we received, to keep the early
headers at the start of the request transmission as the client hands to
us - allowing debugging of clients and more predictable behaviour.

Rob

-- 
GPG key available at: <http://members.aardvark.net.au/lifeless/keys.txt>.