Subject: Security Requirements for HTTP, draft -00

From: Paul Hoffman <[email-masked]>
Date: Wed, 23 Jan 2008 13:59:56 -0800

Greetings. Alexey and I have done a small rev on Rob Sayre's earlier 
document describing the security properties of HTTP and how they vary 
from the IETF's "mandatory to implement" policy. We look forward to 
discussion from the WG on how this document should progress (other 
than the obvious places where we have holes...).

--Paul Hoffman, Director
--VPN Consortium

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Mon, 28 Jan 2008 14:52:50 +0100

On Wed, Jan 23, 2008 at 01:59:56PM -0800,
 Paul Hoffman <[email-masked]> wrote 
 a message of 9 lines which said:

> Alexey and I have done a small rev on Rob Sayre's earlier document
> describing the security properties of HTTP and how they vary from
> the IETF's "mandatory to implement" policy.

Some remarks about draft-ietf-httpbis-security-properties-00.txt. 

2.1 says "Almost all HTTP authentication is accomplished through HTML
forms, with session keys stored in cookies." This is clearly false. It
is true only if you say "Web authentication for an human sitting
behind a Web browser". But for HTTP, the protocol, which can be used
by other things than Web graphical browsers, it is not my experience,
I use RFC 2617 Basic Authentication or TLS with certificates a
lot. (2.4 mentions these other uses, such as Web services.)

2.1 says "Many users do not understand the construction of URIs
[RFC3986], or their presentation in common clients [[ CITATION NEEDED
]]." A good bibliography (thanks to Mike Beltzner @ Mozilla) is:

"Decision Strategies and Susceptibility to Phishing", Downs, Holbrook
& Cranor
   http://cups.cs.cmu.edu/soups/2006/proceedings/p79_downs.pdf

"Why Phishing Works", Dhamija, Tygar & Hearst
   http://people.deas.harvard.edu/~rachna/papers/why_phishing_works.pdf

"Do Security Toolbars Actually Prevent Phishing Attacks", Wu, Miller
& Garfinkel
   http://www.simson.net/ref/2006/CHI-security-toolbar-final.pdf

"Phishing Tips and Techniques", Gutmann
   http://www.cs.auckland.ac.nz/~pgut001/pubs/phishing.pdf

2.2.1 says "Since Basic credentials are clear text, they are reusable
by any party." It seems to me that this has nothing to with being in
clear text or not. Basic credentials are dangerous because they are
static, not because they are clear text (for which TLS is a
solution).

2.4 says "These protocols usually don't have much in common with the
Architecture of the World Wide Web. It's not clear why term "Web" is
used to group them," I agree that "Web" is not a good term but it does
not mean they are off-topic for us, far from it, since we work on
HTTP, not on "the Web". Also, some of these, like REST, have "a lot in
common with the Architecture of the World Wide Web".

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 28 Jan 2008 08:47:46 -0800

>lot. (2.4 mentions these other uses, such as Web services.)

I disagree that the statement is "clearly false". I strongly suspect 
that if you add up all the authentications done on every HTTP server 
in the world today, forms+cookies+people would win over 
((nonforms+people) + (nonforms+nonpeople)). The first word in the 
sentence really does apply.

>    http://www.cs.auckland.ac.nz/~pgut001/pubs/phishing.pdf

Thanks. Of these, I tend to lean towards Gutmann's article because it 
is the most forceful. Do others on the list have a preference?

>static, not because they are clear text (for which TLS is a

The sentence is about reusability, not general danger. The 
reusability comes directly from the attacker being able to see the 
Basic credential go by.

>common with the Architecture of the World Wide Web".

Section 2.4 is explicly about "Web Services", not REST and the like. 
Would it suffice to you if we gave a more specific definition of "Web 
Services"?

--Paul Hoffman, Director
--VPN Consortium

---

From: Paul Leach <[email-masked]>
Date: Mon, 28 Jan 2008 12:52:29 -0800

Here are some comments:

WRT: "2.1.  Forms And Cookies

   Almost all HTTP authentication is accomplished through HTML forms,
   with session keys stored in cookies."

I think calling them "session keys" is a little misleading, since they really aren't used to encrypt or integrity protect the session. Calling them "session IDs" would be more appropriate; best practice has them be unforgeable and unguessable (but subject to theft by MITM unless kept secret by other means) -- and almost undoubtedly there are some implementations that use incrementing session ID counters.

WRT: "2.2.2.  Digest Authentication
        ...
   Additionally, implementation experience has shown that
   the message integrity mode is impractical because it requires servers
   to analyze the full request before determining whether the client
   knows the shared secret."

Could you elaborate? The purpose of integrity protection isn't simply to determine if the client knows the shared secret, it is to insure that no MITM can modify the integrity protected data. This intrinsically requires that all integrity protected data be examined. Hence, the above statement seems to really amount to the claim that integrity protection is too expensive to be practical. However, it isn't any more expensive than TLS, and TLS is used pretty widely.

If all the server wants to know is whether the client knows the shared secret, the non-integrity-protected mode does that. The only reason to use message integrity mode is if message integrity is needed to meet security requirements.

WRT: "Many Digest capabilities included to prevent replay attacks expose the server to Denial of Service attacks."

Which capabilities in particular?

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Hoffman
Sent: Wednesday, January 23, 2008 2:00 PM
To: [email-masked]
Subject: Security Requirements for HTTP, draft -00

--Paul Hoffman, Director
--VPN Consortium

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Mon, 28 Jan 2008 22:27:21 +0100

On Mon, Jan 28, 2008 at 08:47:46AM -0800,
 Paul Hoffman <[email-masked]> wrote 
 a message of 59 lines which said:

> I strongly suspect that if you add up all the authentications done
> on every HTTP server in the world today, forms+cookies+people would
> win over ((nonforms+people) + (nonforms+nonpeople)).

May be, it depends on the metrics you use :-) Number of installations,
number of requests per day, number of US $ processed ? :-)

My personal impression is that it is either forms+people or
nonforms+nonpeople and the rest is marginal.

What do other people think? I was extremely surprised that "nonpeople"
uses of HTTP were, it seems, ignored from the version -00.

> reusability comes directly from the attacker being able to see the

No, it comes from the credential being static. If you use OTP, surely
it does not matter if the attacker can see "the credential go by"?
 
> Section 2.4 is explicly about "Web Services", not REST and the like. 

Well, in that case, I have to ask for the I-D to provide a definition
for Web Services (I'm myself lost in the marketing talk). But it would
be a strange definition if it excludes REST.

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 28 Jan 2008 13:46:23 -0800

>number of requests per day, number of US $ processed ? :-)

Number of requests per day.

>nonforms+nonpeople and the rest is marginal.

Fully agree.

>uses of HTTP were, it seems, ignored from the version -00.

Ignored in what way?!?! The security implications of Basic and Digest 
were fully covered.

>No, it comes from the credential being static.

No, it comes from the credential being static *and visible to an attacker*.

>If you use OTP, surely

Correct. Which HTTP authentication standard is for OTP? If we missed 
one, we should certainly cover it.

>be a strange definition if it excludes REST.

We'll see. :-)

--Paul Hoffman, Director
--VPN Consortium

---

From: Paul Leach <[email-masked]>
Date: Mon, 28 Jan 2008 14:08:41 -0800

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul Hoffman
Sent: Monday, January 28, 2008 1:46 PM
To: Stephane Bortzmeyer
Cc: [email-masked]
Subject: Re: Security Requirements for HTTP, draft -00

>nonforms+nonpeople and the rest is marginal.

Fully agree.
[Paul Leach] On the public internet, I agree as well. Within Intranets, a significant proportion of people-auth uses NTLM and/or Kerberos via the Negotiate auth scheme.

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 28 Jan 2008 20:55:29 +0000

Paul Leach wrote:
> Could you elaborate? The purpose of integrity protection isn't
> simply to determine if the client knows the shared secret, it is to
> insure that no MITM can modify the integrity protected data. This
> intrinsically requires that all integrity protected data be
> examined. Hence, the above statement seems to really amount to the
> claim that integrity protection is too expensive to be
> practical. However, it isn't any more expensive than TLS, and TLS is
> used pretty widely.

It is more expensive than TLS, because TLS integrity is checked per
chunk.

TLS allows the message integrity to be checked for each chunk as it is
arrives, so a request can be accepted, then streamed to the server
application as it arrives.  If a check fails, the request is aborted,
and the application never sees any of the failing data, so it cannot
trigger exploits.  Also, a failure is handled the same as a connection
failure in mid request, so well tested code paths are used.

With digest "auth-int", the message integrity cannot be checked until
it has all arrived.  This means either the whole message must be
buffered by a generic server before passing it to the server
application, or unchecked data is streamed to the server application
with the possibility to tell the application at the end to abort work
it has done.  Buffering is arbitrarily expensive in space and time,
and streaming the data (from generic server to server application)
prior to checking it, with rollback, complicates the interface,
complicates the application, and renders the server vulnerable to
unexpected-input exploits in the application.

-- Jamie

---

From: Paul Leach <[email-masked]>
Date: Mon, 28 Jan 2008 14:58:51 -0800

I don't see the distinction you claim. In either case, if you pass any data of a request to the server application before the whole request has been received and verified, then if the verification fails are any point, any processing done up to that point has to be undone. Perhaps most importantly, in both cases no sensitive output can be sent to the client.

Another way of looking at it is that, as far as I can tell, any scheme that works with TLS and allows for a failure on the last chunk of the request would work with auth-int -- the only difference is that auth-int can _only_ fail at the end.

More inline.

-----Original Message-----
From: Jamie Lokier [mailto:[email-masked]]
Sent: Monday, January 28, 2008 12:55 PM
To: Paul Leach
Cc: Paul Hoffman; [email-masked]
Subject: Re: Security Requirements for HTTP, draft -00

trigger exploits.
[Paul Leach] Maybe I am just completely missing something -- first you say that the request is streamed to the server application as it arrives, then you say that if the request is aborted, the application never sees any of the failing data. Which is it? Is there some subtle distinction between "server application" and "application" that I'm missing?

Also, a failure is handled the same as a connection
failure in mid request, so well tested code paths are used.

With digest "auth-int", the message integrity cannot be checked until
it has all arrived.
[Paul Leach] Neither can it with TLS. Chunk integrity is not message integrity.

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 29 Jan 2008 12:18:49 +1300

my 2c

>    with session keys stored in cookies."
I think it does little good to dwell on whether one method of credential
verification is any more prevalent than any other.  In my experience,
all forms are very common.  Session-based auth such as NTLM is very
common for proxy auth, or auth to a corporate intranet webserver.  Forms
+ cookies is prevalent on public webservers.

None of these scenarios can safely be ignored.

> Could you elaborate? The purpose of integrity protection isn't simply to determine if the client knows the shared secret, it is to insure that no MITM can modify the integrity protected data. This…
It's quite different to TLS.  With TLS, you decrypt blocks as they
arrive, and can pass this decrypted data to upstream processes (e.g.
CGI). The raw data is basically available as it is received. With full
message integrity, you need to buffer the entire message (which may not
end, e.g streaming media over HTTP) before you can verify the signature
to ensure it hasn't been tampered with.  Sure, both these may have a
similar crypto computational cost, but the buffering and memory
management cost is wildly different for large content bodies, and in
some cases it's not practical with current methods.

Overall, I think it's quite an interesting discussion document, I would
be interested to hear how people perceive the protocol itself moving
longer term.  Some of the issues HTTP has that other protocols (e.g.
SMTP) don't wrt authentication (and other things) are related to its
fundamental design/structure and how it has evolved.  HTTP was initially
designed to connect, make request, get result and disconnect.  This
doesn't have room in it for a challenge response auth scheme until you
move to persistent connections.  Still however, there is little
per-connection protocol overhead dedicated to setup, except perhaps TLS
certificates etc.  Auth is lumped in with requests rather than being
dealt with by itself.

Some issues may point to a new overall protocol structure (i.e.
pre-negotiation of transfer of large entities in either direction), but
compatibility issues will exert pressure against change.  Is there a
long-term goal for the protocol?

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 29 Jan 2008 15:02:34 +0000

Paul Leach wrote:
> I don't see the distinction you claim. In either case, if you pass
> any data of a request to the server application before the whole
> request has been received and verified, then if the verification fails
> are any point, any processing done up to that point has to be undone.

Yes, but all servers applications must handle that, and it will be
well tested, because it occurs during normal network failures and
client aborts - thousands per day.

> Perhaps most importantly, in both cases no sensitive output can be
> sent to the client.

That's true, but the TLS approach also prevents malicious input
reaching the server application, without needing an unbounded buffer.

> Another way of looking at it is that, as far as I can tell, any scheme that works with TLS and allows for a failure on the last chunk of the request would work with auth-int -- the only difference…

Well, that's right, but that difference is quite significant for apps
which are written to work with known clients which they trust to
provide requests in a certain form, and aren't robust against
malicious input (or aren't thoroughly audited and tested enough to
trust).  It's not so unusual to put authentication/validation in the
generic server.

-- Jamie

---

From: Paul Leach <[email-masked]>
Date: Wed, 30 Jan 2008 11:42:04 -0800

OK, I think I understand. Let me sum up by proposing alternate wording, then do some more analysis (about which no one may care much but me).

Alternate wording:

"Additionally, implementation experience has shown that in some cases, especially those involving large requests or responses such as streams,
the message integrity mode is impractical because it requires servers
to analyze the full request before determining whether message integrity has been violated and hence whether the request can be processed."

Analysis:

In the case where the server knows nothing about the semantics of the request or the capabilities of the server application, I believe that it has no choice but to hold the request until it has determined that nothing has tampered with it -- for both TLS and auth-int.

If something is known about the semantics of the request or of the server application capabilities, optimizations are possible.

For example, if the server app is a database that supports transactions, then the request could be incrementally passed to it, knowing that the transaction can be aborted if and when an integrity violation is detected. Again, this is essentially the same for TLS and auth-int, the only difference being that integrity violation can be detected earlier with TLS.

Similarly, if the request is a file upload, then it can be incrementally processed because the file can simply be deleted if an integrity violation is detected.

Another case is where the semantics of the request are the same as that of some series of smaller pipelined requests, then request processing can be optimized by (logically) transforming it into said smaller requests and performing the smaller requests as they arrive. This describes what happens when the request contains a stream of data, for example. In this case, TLS does have advantages over auth-int, because auth-int prevents such a transformation.

Your previous post mentions some other cases where TLS also has advantages.

Analysis of responses could proceed along similar lines, except that we have a much better idea of the capabilities of most user agents. Hence, we know for example that indefinite streams of output are not practical with auth-int.

I think this reconciles the opinions of both sides -- perhaps in more gory detail than anyone cared to hear, but it satisfied my desire to really understand the issue.

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 31 Jan 2008 09:12:00 +0000

Paul Leach wrote:
> OK, I think I understand. Let me sum up by proposing alternate
> wording, then do some more analysis (about which no one may care much
> but me).

That was an excellent analysis, very clearly put; thank you.

-- Jamie