Subject: #282: Recommend minimum sizes for protocol elements

From: Mark Nottingham <[email-masked]>
Date: Sun, 26 Jun 2011 17:03:33 +1000

<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/282>

Revised proposal:

For HTTP headers, insert at end of p1 3.2:

"""
HTTP does not place a pre-defined limit on the length of header fields, either in isolation or as a set. A server MUST be prepared to receive request headers of unbounded length and respond with the 413 (Request Entity Too Large) status code if the received header(s) would be longer than the server wishes to handle (see Section 8.4.14 of [Part2]).

A client that receives response headers that are longer than it wishes to handle can only treat it as a server error.

Various ad-hoc limitations on header length are found in practice. It is RECOMMENDED that all HTTP senders and recipients support messages whose combined header block (i.e., from the start of the request-line or response-line until the CRLF CLRF that terminates the headers) have 4,000 or more octets.
"""

For max-age and other cache-control directives, move the definition of delta-seconds from Age to a new subsection of p6 1.4 Syntax Notation:

"""
1.4.x Delta Seconds

The delta-seconds rule specifies a non-negative integer, representing time in seconds.

delta-seconds = 1*DIGIT

If an implementation receives a delta-seconds value larger than the largest positive integer it can represent, or if any of its subsequent calculations overflows, it MUST consider the value to be 2147483648 (2^31). Recipients parsing a delta-seconds value SHOULD use an arithmetic type of at least 31 bits of range, and senders MUST NOT send delta-seconds with a value greater than 2147483648.
"""

For other situations, add section to p1 Security Considerations:

"""
11.5 Protocol Element Size Overflows

Because HTTP uses mostly textual, character-delimited fields, attackers can overflow buffers in implementations, and/or perform a Denial of Service against implementations that accept fields with unlimited lengths.

To promote interoperability, this specification makes specific recommendations for size limits on request-targets [ref] and blocks of header fields [ref]. These are minimum recommendations, chosen to be supportable even by implementations with limited resources; it is expected that most implementations will choose substantially higher limits.

This specification also provides a way for servers to reject messages that have request-targets that are too long [ref] or request entities that are too large [ref].

Other fields (including but not limited to request methods, response status phrases, header field-names, and body chunks) SHOULD be limited by implementations carefully, so as to not impede interoperability.
"""

--
Mark Nottingham   http://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 26 Jun 2011 08:19:30 +0000

In message <[email-masked]>, Mark Nottingham wri
tes:

>1.4.x Delta Seconds
>
>The delta-seconds rule specifies a non-negative integer, representing =
>time in seconds.

Some high volume sites cache content for only a couple of seconds,
but derive 20-30% cache hit ratios from it.  For these sites the
1 second granularity of age related values are a serious limitation
on caching policy.

Does anybody know of anything that would break if we allowed real
numbers for these fields ?

ie:
	delta-seconds = 1*DIGIT [ '.' *DIGIT]

Specifying truncation rather than rounding for implementations that
do not support real numbers, would ensure compliance for existing
implementations.  The 2^31 should remain unaltered.

>For other situations, add section to p1 Security Considerations:

Do we need to caution against dropping headers selectively due
to size ?

I know of at least one filtering device which can be circumvented
by making a particular header longer than a certain length.  This
causes the device to drop just this one header, which in turn
bypasses filter features.  (The vendor has been notified, but
emits few if any clue-o-trons)

I'm not quite sure how to express this, something like "don't drop
headers unless you know what they mean".

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Brian Pane <[email-masked]>
Date: Sun, 26 Jun 2011 05:44:21 -0700

On Sun, Jun 26, 2011 at 1:19 AM, Poul-Henning Kamp <[email-masked]> wrote:
> In message <[email-masked]>, Mark Nottingham writes:
>
>>1.4.x Delta Seconds
>>

I fear that changing the grammar in HTTP/1.1 would break a lot of
caching intermediaries.  Thinking ahead to future HTTP versions,
though, it seems reasonable to allow fractional seconds.

-Brian

---

From: Brian Pane <[email-masked]>
Date: Sun, 26 Jun 2011 06:37:42 -0700

> Various ad-hoc limitations on header length are found in practice. It is RECOMMENDED that all HTTP senders and recipients support messages whose combined header block (i.e., from the start of the r…

This looks good.  The language currently in Section 8.4.14 allows the
server to send the 413 and drop the connection as soon as it sees the
4001st byte of the request header, thus thwarting malicious clients
who try to keep the server busy by sending an infinite stream of
headers.

In the case of HTTP requests arriving via SPDY, short-circuiting the
413 response will be a little more complicated: in order to keep the
server's session-scoped gzip state from falling out of sync with the
client, the server will have to either: 1) send the entirety of the
request headers through its gzip engine before discarding them or 2)
drop the entire session, rather than just the affected stream, when it
sends the 413, and thus drop any other requests that might have been
in line behind the too-large request.  I don't see any good way to
solve this in the HTTP spec, though; it's really an inherent tradeoff
between efficiency (session-scoped compression state) and resiliency
(request/stream-scoped compression state) at a lower layer.

-Brian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 27 Jun 2011 18:55:56 +1000

> implementations.  The 2^31 should remain unaltered.

I've found such low TTLs useful as well, but only in controlled deployments (i.e., reverse proxies). For forward proxies and browser caches, you can't assume clock sync that tight (IME clock sync is a problem even in reverse proxies).

Nothing stopping you from specifying a CC extension with higher precision, of course, but realise that Date only has one second resolution.

> headers unless you know what they mean".

I *think* Roy intends to expand upon this in "Use of HTTP for proxy communication" in p1.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 28 Jun 2011 15:08:28 +1000

OK, I've set the milestone for this to -15.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 28 Jun 2011 15:07:39 +1000

> (request/stream-scoped compression state) at a lower layer.

Yep. We're aiming to make it possible for something like SPDY to overtake what p1 does without worrying about the other parts, but it'll still need to address the issues brought up in p1.

Cheers,