Subject: #288: Considering messages in isolation

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Jun 2011 11:43:08 +1000

Proposal for <http://trac.tools.ietf.org/wg/httpbis/trac/ticket/288>:

Add to new message orientation section (see #283) or just to 2.2 Client/Server? messaging:

Recipients MUST consider every message in a connection in isolation; because HTTP is a stateless protocol, it cannot be assumed that two requests on the same connection are from the same client or share any other common attributes.

--
Mark Nottingham   http://www.mnot.net/

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 22 Jun 2011 21:14:51 +1200

On 22/06/2011 1:43 p.m., Mark Nottingham wrote:
> Proposal for<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/288>:
>
over TCP/IP this isn't true, since by definition being on the same 
connection means having the same peer, and therefore sharing the same 
client IP:port, and (in practice) credentials.

Or do you mean by common attributes only aspects encapsulated in an HTTP 
request message itself (e.g. things learned from headers etc)?

Regards

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Jun 2011 19:19:34 +1000

On 22/06/2011, at 7:07 PM, Adrien de Croy wrote:

> over TCP/IP this isn't true, since by definition being on the same connection means having the same peer, and therefore sharing the same client IP:port, and (in practice) credentials.
> 
> Or do you mean by common attributes only aspects encapsulated in an HTTP request message itself (e.g. things learned from headers etc)?

Yes, I was speaking of the HTTP-specific aspects of things.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 28 Jun 2011 15:15:06 +1000

Milestone set for -15.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Jun 2011 14:01:16 +0200

On 2011-06-28 07:15, Mark Nottingham wrote:
> ...

Applied with <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1317>.

I added it to the new section, which now reads:

2.2.  Message Orientation and Buffering

    Fundamentally, HTTP is a message-based protocol.  Although message
    bodies can be chunked (Section 6.2.1) and implementations often make
    parts of a message available progressively, this is not required, and
    some widely-used implementations only make a message available when
    it is complete.  Furthermore, while most proxies will progressively
    stream messages, some amount of buffering will take place, and some
    proxies might buffer messages to perform transformations, check
    content or provide other services.

    Therefore, extensions to and uses of HTTP cannot rely on the
    availability of a partial message, or assume that messages will not
    be buffered.  There are strategies that can be used to test for
    buffering in a given connection, but it should be understood that
    behaviors can differ across connections, and between requests and
    responses.

    Recipients MUST consider every message in a connection in isolation;
    because HTTP is a stateless protocol, it cannot be assumed that two
    requests on the same connection are from the same client or share any
    other common attributes.

Best regards, Julian

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 30 Jun 2011 12:52:51 +1200

How does auth fit in with this, esp any challenge-response based 
authentication or connection-oriented auth.

NTLM clearly requires an assumption that authentication state across 
multiple requests is associated with the connection the requests are 
received on.

Is Digest also a problem with this?

I realise there's not really anything an O-S can do, since a connection 
might have come from a proxy that aggregates clients into the same 
connection.

Does the "Proxy-Support: session-based-authentication" header mess with 
this, IOW do we have a collision here with RFC4559

Regards
Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
WinGate 7 beta out now - http://www.wingate.com/getlatest/

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 30 Jun 2011 11:02:40 +1000

On 30/06/2011, at 10:52 AM, Adrien de Croy wrote:

> 
> How does auth fit in with this, esp any challenge-response based authentication or connection-oriented auth.

I think it's well-established that "connection-oriented" authentication is fundamentally incompatible with HTTP, and shouldn't be attempted. Yes, NTLM does it, and that causes *significant* problems in all of the implementations I'm aware of.

> NTLM clearly requires an assumption that authentication state across multiple requests is associated with the connection the requests are received on.
> 

No.

> I realise there's not really anything an O-S can do, since a connection might have come from a proxy that aggregates clients into the same connection.
> 
> Does the "Proxy-Support: session-based-authentication" header mess with this, IOW do we have a collision here with RFC4559

That's Informational, although I'm a little surprised it was let through without a note to the effect that it breaks HTTP. Might be worth raising an errata to that effect.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 30 Jun 2011 06:54:15 +0200

> I think it's well-established that "connection-oriented" authentication is fundamentally incompatible with HTTP, and shouldn't be attempted. Yes, NTLM does it, and that causes *significant* problem…

+1. Some proxies reduce their keep-alive timeout when overloaded,
resulting in NTLM auth not working over them under high loads ! Also,
any "connection-oriented" authentication results in wrong authentication
when a proxy is present before the authenticating component. I remember
a customer where the outgoing proxy used to work that way, and browsing
accounts were very expensive so very few people had them. A few people
I was working with decided to install a proxy which used only one
persistent connection, and provided free access to a number of
coworkers :-)

And now with more and more components able to multiplex requests over
connection pools, connection-oriented authentication is quite dangerous.

NTLM could fix this design issue by making the client present sort of
a cookie to the server during and after auth.

Regards,
Willy

---

From: Julian Reschke <[email-masked]>
Date: Thu, 30 Jun 2011 09:17:24 +0200

> I think it's well-established that "connection-oriented" authentication is fundamentally incompatible with HTTP, and shouldn't be attempted. Yes, NTLM does it, and that causes *significant* problem…

And it's good that we now have spec text we can people point to when the 
question comes up (for instance, when looking at 
draft-ietf-httpbis-authscheme-registrations).

Best regards, Julian

---

From: Adrien de Croy <[email-masked]>
Date: Thu, 30 Jun 2011 19:54:42 +1200

> I think it's well-established that "connection-oriented" authentication is fundamentally incompatible with HTTP, and shouldn't be attempted. Yes, NTLM does it, and that causes *significant* problem…

well the problem with "shouldn't be attempted" is that NTLM is one of 
the most popular auth mechanisms in use.

On this note, one reason it's so popular is because there's no other 
"secure" mechanism that can (in the default case) use the accounts from 
a Windows or Active Directory user database.  This is because by default 
the passwords are stored as MD4 hashes, and so any auth method that 
requires access to the plaintext password cannot be used unless the 
administrator changes settings to allow use of reversible encryption for 
password storage.

So, until MS makes the default password storage mechanism reversible, no 
mechanisms using plaintext passwords (such as CRAM-MD5, Digest et al) 
will get any traction in a windows environment.  Which leaves us stuck 
with NTLM.

Furthermore in a corporate environment, integrated authentication (which 
is supported by all popular windows-compatible browsers) is based on 
Negotiate, which usually boils down to NTLM or Kerberos.

Designers of new auth mechanisms (since there have been several recently 
discussed on this list) should therefore keep this in mind. I'm not 
saying we should all fall back to MD4 hashes, but it's an issue.

So IMO much as NTLM may be quite problematic, I believe it will be with 
us for a long time to come.

What action if any that leaves us with now is another matter.  Perhaps 
we should make some note somewhere, or explicitly deal with the case.  
For instance state somewhere that the assumption that requests are 
unrelated no longer holds if a particular header is present, indicating 
the use of session-based authentication for instance.

Adrien

>>> Applied with<http://trac.tools.ietf.org/wg/httpbis/trac/changeset/1317>.
>>>

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 30 Jun 2011 11:04:04 +0200

> the use of session-based authentication for instance.

This would be very dangerous, however probably we should document existing
incompatibilities with the rule (eg: NTLM auth) so that implementers are
aware of this and plan on being able to adapt to this mode by configuration,
which implies more than just keeping the 1-to-1 association between client
and server connection, as it also means that connections should not be
dropped too often, and almost never during the challenge.

But I agree with you that stating that this erroneous behaviour should not
be done will not suddenly make NTLM auth disappear with its associated
issues.

Regards,
Willy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jul 2011 14:58:42 +1000

Revised proposal:

"""
Recipients MUST consider every message in a connection in isolation; because HTTP is a stateless protocol, it cannot be assumed that two requests on the same connection are from the same client or share any other common attributes. In particular, intermediaries might mix requests from different clients into a single server connection. Note that some existing HTTP extensions (e.g., [RFC4559]) violate this requirement, thereby potentially causing interoperability and security problems.
"""