Subject: Leading zeroes in 1*DIGIT productions

From: Jeff Walden <[email-masked]>
Date: Thu, 18 Dec 2008 11:41:21 -0800

Many places in RFC2616 which specify decimal numbers do so using 1*DIGIT, e.g.

first-byte-pos  = 1*DIGIT

This production admits the possibility of leading zeroes in a non-zero number, so, for example, you might have this header:

Content-Length: 017

I'm not sure whether this is intentional or not (HTTP-Version makes it explicit, but e.g. Content-Length does not), but it's slightly confusing if the number (as above) matches the common format for octal numbers, not to mention a little bit nonsensical to include gratuitous leading zeroes.  I would prefer if something like the following were used instead:

nonzero-decimal-digit  = "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9"
decimal-digit   = "0" / nonzero-decimal-digit
decimal-number  = "0" / nonzero-decimal-digit*decimal-digit
first-byte-pos  = decimal-number

Alternately, making it clear that leading zeroes are allowed but (perhaps) MUST NOT be sent  would be acceptable.

Jeff

---

From: Julian Reschke <[email-masked]>
Date: Thu, 18 Dec 2008 20:49:13 +0100

Jeff Walden wrote:
> This production admits the possibility of leading zeroes in a non-zero 
> number, so, for example, you might have this header:
> 
> Content-Length: 017

Right.

> I'm not sure whether this is intentional or not (HTTP-Version makes it 
> explicit, but e.g. Content-Length does not), but it's slightly confusing 
> if the number (as above) matches the common format for octal numbers, 
> not to mention a little bit nonsensical to include gratuitous leading 
> zeroes.  I would prefer if something like the following were used instead:
> 

That would be an incompatible change, something we can't make with this 
revision.

> Alternately, making it clear that leading zeroes are allowed but 
> (perhaps) MUST NOT be sent  would be acceptable.

How would that help? As far as I can tell, leading zeros are totally 
harmless. Do you have evidence of problems caused by them?

BR, Julian

---

From: David Morris <[email-masked]>
Date: Thu, 18 Dec 2008 12:16:21 -0800 (PST)

Leading zeros should be a non-problem. No where do we suggest the value is
anything but a decimal integer. Parsing leading zeros should be easier
than having to reject leading zeros. A few leading zeros sent by a sloppy
program won't impact the network.

That is a strong vote for nothing to change.

Dave Morris

---

From: Geoffrey Sneddon <[email-masked]>
Date: Thu, 18 Dec 2008 21:55:11 +0000

On 18 Dec 2008, at 19:49, Julian Reschke wrote:

> harmless.

If that's the case, why are they MUST NOT in the case of HTTP-Version?

--
Geoffrey Sneddon
<http://gsnedders.com/>

---

From: Julian Reschke <[email-masked]>
Date: Thu, 18 Dec 2008 23:05:00 +0100

> If that's the case, why are they MUST NOT in the case of HTTP-Version?

I can only guess it's because recipients have treated the version as 
string in the past, and tripped over things like 01/01.

So yes, if there's a demonstrated interop problem with leading zeros 
somewhere else, that may change the picture.

So again: any evidence of that?

BR, Julian

---

From: Jeff Walden <[email-masked]>
Date: Thu, 18 Dec 2008 14:15:42 -0800

> harmless. Do you have evidence of problems caused by them?

No; it's merely that they're only implicitly allowed by the production, not explicitly noted in the accompanying prose.  A note like "leading zeroes are ignored" would be fine (and note that I did say "perhaps"); I just don't see evidence that this oddity was intentional, and if it can't be removed I'd rather it be explicitly allowed.

Jeff

---

From: Julian Reschke <[email-masked]>
Date: Thu, 18 Dec 2008 23:21:50 +0100

Jeff Walden wrote:
> No; it's merely that they're only implicitly allowed by the production, 
> not explicitly noted in the accompanying prose.  A note like "leading 
> zeroes are ignored" would be fine (and note that I did say "perhaps"); I 
> just don't see evidence that this oddity was intentional, and if it 
> can't be removed I'd rather it be explicitly allowed.

Ahem, it *is* explicitly allowed by the grammar.

And code that properly processes the value will need no special 
treatment of leading zeros, it just follows from the way decimal numbers 
work.

BR, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 21 Dec 2008 04:20:51 +0000

David Morris wrote:
> A few leading zeros sent by a sloppy program won't impact the network.

What about a million leading zeros in a chunk header from a malicious
program?  Should implementations be expected to accept that?

Also, is it permitted for a proxy to change Content-Length by removing
leading zeros?

-- Jamie

---

From: David Morris <[email-masked]>
Date: Sat, 20 Dec 2008 20:49:50 -0800 (PST)

> program?  Should implementations be expected to accept that?

An implementation can always set limits to protect itself. That is just
good programming. Certainly the HTTP specification doesn't have to
describe common sense.

> Also, is it permitted for a proxy to change Content-Length by removing

Of course, just like a proxy is free to convert chunked encoding to
explict content length. Or in the right circumstances, the converse.

---

From: Jeff Walden <[email-masked]>
Date: Mon, 22 Dec 2008 18:35:01 -0800

> Ahem, it *is* explicitly allowed by the grammar.

Yes; that doesn't mean that it doesn't look plausibly unintentional.

> treatment of leading zeros, it just follows from the way decimal numbers

So actually, I do know of a server that had a problem specifically with leading zeros because it was using an API which would treat such numbers as octal, absent the use-base-10 hint I promptly added when I discovered the problem:

http://hg.mozilla.org/mozilla-central/file/87267825681b/netwerk/test/httpserver/httpd.js#l1520

I didn't bring it up initially because it's more a matter of the person who wrote that line not being familiar with how parseInt works in JavaScript than of a deficiency in the specification, but code which properly processes /most/ Content-Length values may not necessarily properly process /all/ such values, in particular numbers which have the format of octal numbers with leading zeroes.  I think explicitly noting leading zeroes in accompanying prose makes it more likely that care will be taken in processing such values.

Jeff

P.S. -- Yes, that's an HTTP/1.1 server written in JavaScript.  No, it's not RFC2616-compliant (yet?), but it does enough to be used in fairly extensive browser testing.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 24 Dec 2008 10:02:15 +0100

Jeff Walden wrote:
> ...
> So actually, I do know of a server that had a problem specifically with 
> leading zeros because it was using an API which would treat such numbers 
> as octal, absent the use-base-10 hint I promptly added when I discovered 
> the problem:
> 
> I didn't bring it up initially because it's more a matter of the person 
> who wrote that line not being familiar with how parseInt works in 

Right. It is a documentation/education problem, but not with HTTP/1.1 
but with Javascript.

> JavaScript than of a deficiency in the specification, but code which 
> properly processes /most/ Content-Length values may not necessarily 
> properly process /all/ such values, in particular numbers which have the 
> format of octal numbers with leading zeroes.  I think explicitly noting 
> leading zeroes in accompanying prose makes it more likely that care will 
> be taken in processing such values.

Disagreed.

First of all, anybody who makes this mistake is likely to find out 
*very* soon.

Furthermore, the spec already says "decimal". That implies leading 
zeros. Repeating this doesn't make the spec any clearer, just longer. 
So, in general, I'm in favor of removing redundant text such as this 
would be.

 > ...

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 24 Dec 2008 10:43:15 +0100

> First of all, anybody who makes this mistake is likely to find out 

OK, that's obviously not correct, unless leading zeros would be widespread.

I still believe that adding more prose in *HTTP/1.1* won't help with 
this issue.

BR, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 26 Dec 2008 22:35:23 +0000

> First of all, anybody who makes this mistake is likely to find out 

Why do you say "very soon"?  Content-Length with leading zeros is not
seen much if at all in the wild, is it?

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 26 Dec 2008 22:45:56 +0000

> seen much if at all in the wild, is it?

(Ignore this; Julian already replied).

-- Jamie