Subject: Linear whitespace

From: Marc Schneider <[email-masked]>
Date: Wed, 04 May 2005 12:50:48 -0400

At the end of section 2.1 of RFC 2616 there is a rule about implied linear 
whitespace

implied *LWS
The grammar described by this specification is word-based. Except where 
noted otherwise, linear white space (LWS) can be included between any two 
adjacent words (token or quoted-string), and between adjacent words and 
separators, without changing the interpretation of a field. At least one 
delimiter (LWS and/or separators) MUST exist between any two tokens (for 
the definition of "token" below), since they would otherwise be interpreted 
as a single token.
And in section 2.2 a definition of LWS is given

HTTP/1.1 header field values can be folded onto multiple lines if the 
continuation line begins with a space or horizontal tab. All linear white 
space, including folding, has the same semantics as SP. A recipient MAY 
replace any linear white space with a single SP before interpreting the 
field value or forwarding the message downstream.
        LWS            = [CRLF] 1*( SP | HT )

Does the optional carriage return linefeed at the beginning of the LWS only 
apply to headers? Or can this CRLF appear between any two words or between 
a word an separator?

For example, is the following valid?

GET<CR><LF><SP>/<CR><LF><SP>HTTP/1.1<CR><LF>host:<SP>blah...
Thanks.

Marc Schneider
Senior Software Engineer

OPNET Technologies, Inc. (NASDAQ: OPNT)
tel: +1.240.497.3000
fax: +1.240.497.3001
<http://www.opnet.com/>http://www.opnet.com

====================================================
Register for OPNET's Online Technology Workshops
<http://www.opnet.com/TechWorkshops/>http://www.opnet.com/TechWorkshops/
====================================================
Register for OPNETWORK 2005 (August 22-26, Washington DC)
<http://www.opnet.com/opnetwork2005/>http://www.opnet.com/opnetwork2005/
====================================================

---

From: Marc Schneider <[email-masked]>
Date: Wed, 04 May 2005 13:01:49 -0400

Looking further in the RFC, I see that extra <CR> and <LF> characters are 
explicitly prohibited from the request line in section 5.1. So ignore the 
example.

At 12:50 PM 5/4/2005, Marc Schneider wrote:
>the definition of "token" below), since they would otherwise be 
>interpreted as a single token.
>Does the optional carriage return linefeed at the beginning of the LWS 
>only apply to headers? Or can this CRLF appear between any two words or 
>between a word an separator?
>

Marc Schneider
Senior Software Engineer

---

From: Marc Schneider <[email-masked]>
Date: Wed, 04 May 2005 13:13:55 -0400

For example, would the following be valid?

GET<SP>/<SP>HTTP/1.1<CR><LF>Host<CR><LF><SP>:<CR><LF><SP>blah...

Thanks,
Marc

At 01:01 PM 5/4/2005, Marc Schneider wrote:
>>At the end of section 2.1 of RFC 2616 there is a rule about implied 
>>linear whitespace
>>
>>implied *LWS

Marc Schneider
Senior Software Engineer

---

From: "Alex Rousskov" <[email-masked]>
Date: Fri, 06 May 2005 08:58:40 -0600

> GET<SP>/<SP>HTTP/1.1<CR><LF>Host<CR><LF><SP>:<CR><LF><SP>blah...

Yes, this is valid. LWS can be used anywhere where it is permitted
and not explicitly prohibited. Unfortunately, this causes security
problems such as cache poisoning because different implementations
have different bugs in supporting LWS syntax. Most implementations
we have tested could not handle some variations of LWS inclusion
so it is a real problem.

IIRC, there have been attempts to explicitly prohibit more places
 from LWS inclusion, including the first part of the example you
give above. Adding new restrictions to a widely deployed protocol
is difficult and is not going to have noticeable short-term effects,
of course. I do not see those restrictions in the RFC 26161 errata,
so I am guessing those attempts have failed.

HTH,

Alex.