Subject: Re: security requirements (was: Updating RFC 2617 (HTTP Digest) to use UTF-8)

From: Lisa Dusseault <[email-masked]>
Date: Sat, 4 Nov 2006 10:47:53 -0800

On Oct 17, 2006, at 7:30 PM, Roy T. Fielding wrote:

>
> On Oct 17, 2006, at 5:38 PM, Robert Sayre wrote:
>> Does anyone think mandatory-to-implement authentication schemes or
>> transport-layer security mechanisms will be helpful and realistic?
>
> Not without changing the HTTP version number, but I suppose that
> I shouldn't assume that is obvious.  HTTP/1.1 has already been
> deployed and I have no interest in declaring any of those
> implementations broken just because they failed to anticipate a
> not-yet-specified secure auth mechanism.  That ship has sailed.

Not only is that not obvious, I'm not sure it's true.  Publishing a  
new RFC is not a declaration that implementations of previous RFCs  
are broken.   Nobody expects functioning crystal balls.  Changing a  
version number is a choice that's subject to all kinds of  
consideration and judgement, rather than a choice that's subject to  
hard-and-fast rules. (Even if there were rules/guidelines in a given  
spec about when a version number should change, future protocol  
authors might find good reasons to violate those, and HTTP has little  
or no such guidelines.)

What's the utility of a version number in HTTP?  This is helpful as  
an explanation of the last change in number:
   "the proliferation of incompletely-implemented
    applications calling themselves "HTTP/1.0" has necessitated a
    protocol version change in order for two communicating applications
    to determine each other's true capabilities."

and
   " No change is made to the version
    number for the addition of message components which do not affect
    communication behavior or which only add to extensible field  
values."

So I guess a decision that CLIENTS MUST support Basic and Digest in a  
new HTTP RFC, might be signalled by a minor version bump.  The  
version bump could signal that support along with a bundle of other  
capabilities if necessary.  The server can use that information to  
change its behavior and work better with that client.  My personal  
opinion on that is it's probably not of sufficient usefulness for the  
disruption it would cause -- unless the bundle overall were more than  
just auth improvements -- because servers can already send auth  
challenges prompting the client to use Digest if it can.  But if a WG  
were to decide it was worthwhile, then that would be fine and  
reasonable.

But a decision that SERVERS MUST support Basic and Digest -- well  
that doesn't need a version bump at all to work.  We already have a  
way for servers to advertise support insofar as that's necessary for  
those algorithms.

Lisa

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 14:13:00 -0500

On 11/4/06, Lisa Dusseault <[email-masked]> wrote:
> that doesn't need a version bump at all to work.

I don't understand this point of view at all, and I don't support any
work on HTTP/1.1 where that result is in bounds.

-- 

Robert Sayre

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 04 Nov 2006 20:43:51 +0100

lör 2006-11-04 klockan 10:47 -0800 skrev Lisa Dusseault:

> new HTTP RFC, might be signalled by a minor version bump.

I too don't see thy a version bump would even be remotely needed in this
case. It's already the server who dictates which authentication
protocols is acceptable to the server, the client just selects what it
thinks is best among the available choices. If there is no match
communication is not possible, such as would be the case for a resource
requiring strong secure authentication.

The change to require support for strong authentication is not a
technical change, it's a administrative policy change. The protocol
isn't changed by this, only how the protocol may be applied.

HTTP version numbers do have an implicit defined meaning:

 - Minor numbers signify a change in transport related features, but
keeping the basic message format and meanings of headers intact.

 - Major numbers signify a change in message format, incompatible with
earlier versions. For example if the header format is changed, or if
already well defined headers is redefined to another meaning.

Also remember that HTTP message numbers are hop-by-hop, while most
headers describing capabilities such as authentication requirements is
end-to-end.

Regards
enrik

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 14:59:23 -0500

On 11/4/06, Henrik Nordstrom <[email-masked]> wrote:
> protocols is acceptable to the server,

An HTTP/1.1 message is not a guarantee that the sender supports any
authentication mechanism. Servers receiving a hypothetical HTTP/1.2
message could make that assumption.

> HTTP version numbers do have an implicit defined meaning:

They have an explicit meaning. See RFC 2145. Additionally, RFC 2616
defines the term "conditional compliance".  RFC 2616 section 3 also
defines the term "conditional compliance", which is not compatible
with the addition of a MUST-level security mechanism.

"An HTTP client MUST NOT send a version for which it is not at least
conditionally compliant.'

-- 

Robert Sayre

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 15:16:41 -0500

> conditionally compliant.'

Sorry, that's from RFC 2145. The send button was clicked a bit early. :)

In any case, the requirements and semantics of HTTP version numbers
seem clear as a bell to me. I don't see any interpretation that allows
something as radical as the addition of a mandatory security mechanism
without incrementing the version number.

-- 

Robert Sayre

---

From: Stefan Eissing <[email-masked]>
Date: Sat, 4 Nov 2006 21:41:07 +0100

> without incrementing the version number.

+1.

And besides, I fail to see what shall be accomplished here. I get the  
feeling that people think that waving the magic wand of specification  
revisions will instantaneously change the world around them. It will  
not.

If the spec would be changed in this way, all a reasonable server  
could deduce from a HTTP/1.1 request is that the client *may*  
implement the now mandatory to implement authentication schemes. That  
seems a bit thin a gain compared to the current situation, e.g. the  
server can assume that the client *may* implement basic and digest.

//Stefan

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Sat, 04 Nov 2006 14:41:55 -0600

Lisa Dusseault wrote:
> new HTTP RFC, might be signalled by a minor version bump.[...]

> But a decision that SERVERS MUST support Basic and Digest -- well that
> doesn't need a version bump at all to work.  We already have a way for
> servers to advertise support insofar as that's necessary for those
> algorithms.

This doesn't parse - it would immediately break a massive number of web
applications, much as microsoft recently did in the IE client 'security'
patches through their re-POST of failed POST requests sans-request-body.
Requirements even on the server side can't realistically be altered
within the confines of HTTP/1.0 /1.1.

The only answer is to remove Basic for HTTP/1.2 or /2.0 in the future
revision of the spec as a fundamentally broken mechanism, much as the
HTTP/1.1 spec introduced manditory Host headers to force all browsers
over to mass vhosting by-name.

Bill

---

From: David Morris <[email-masked]>
Date: Sat, 4 Nov 2006 12:44:25 -0800 (PST)

> message could make that assumption.

But in the end it doesn't matter, the server sends the appropriate
challange (assuming acceptable credentials weren't in the request)
and the challange is either understood or it isn't. The outcome
is essentially identical ... the request is authenticated or it isn't.

Yes, it takes an extra set of round trips as the server can't reject
the request out of hand. In terms of %age of total http network traffic,
it will be lost in the noise.

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 15:53:11 -0500

On 11/4/06, David Morris <[email-masked]> wrote:
>
> But in the end it doesn't matter,

I don't think that's an assumption that can be made, if we are to
envision a world where people actually use HTTP authentication instead
of cookies and forms. For instance, servers may wish to redirect
HTTP/1.1 clients.

-- 

Robert Sayre

---

From: [email-masked]
Date: Sat, 4 Nov 2006 22:14:16 +0000

Lisa, Robert,

> without incrementing the version number.
Agreed -- just like indicated in my email from 2006-10-18:
there is no reasonable way to add mandatory requirements
without changing version numbers or breaking conformance
of existing implementations (regardless whether server or client).
imho to drop to require broken legacy stuff (basic auth) seems
feasible, to add to require the impl of any mandatory auth scheme
seems not.
Moreover I would consider the introduction of such a requirement
a regression due to the existence of applications with 
"legitimately anonymous" usage of http (see my aforementioned mail
to the wg list).

Kind regards

Ingo Struck

---

From: David Morris <[email-masked]>
Date: Sat, 4 Nov 2006 13:12:21 -0800 (PST)

I just don't envision such a world.

---

From: Paul Leach <[email-masked]>
Date: Sat, 4 Nov 2006 13:13:41 -0800

Making one or both of the existing auth protocols mandatory-to-implement
does not change the protocol at all, so no version number change is
necessary.

That's because making a protocol feature mandatory-to-implement does NOT
make it mandatory to configure. Hence, for example, one could not
deduce, from either an HTTP/1.1 or a new HTTP/1.2 sent by a client, that
a server can send Basic or Digest challenge and be assured that it will
be understood by the client.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Robert Sayre
Sent: Saturday, November 04, 2006 3:17 PM

[Paul Leach] snip

-- 

Robert Sayre

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 16:21:58 -0500

On 11/4/06, Paul Leach <[email-masked]> wrote:
> make it mandatory to configure.

That's not a meaningful distinction on the Internet.

-- 

Robert Sayre

---

From: Paul Leach <[email-masked]>
Date: Sat, 4 Nov 2006 13:35:16 -0800

It's what those words mean.

To say what you mean, then a protocol spec says that when a client sends
X, it MUST respond with Y -- it does not say that X and Y are
mandatory-to-implement.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Robert Sayre
Sent: Saturday, November 04, 2006 4:22 PM
To: Paul Leach
Cc: Henrik Nordstrom; HTTP Working Group
Subject: Re: security requirements (was: Updating RFC 2617 (HTTP Digest)
to use UTF-8)

-- 

Robert Sayre

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 16:47:54 -0500

> It's what those words mean.

With no malice, I don't think you have good understanding of how the
IESG interprets "mandatory-to-implement". Let's say Basic becomes
mandatory-to-implement. That means FooCorp could not distribute a
FooCorp-branded client that has no way to be configured for Basic
authentication and claim HTTP conformance.

Which is pretty silly given that proprietary Web server applications
exist only as deployed--there is no separate "implementation".

-- 

Robert Sayre

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 04 Nov 2006 22:58:34 +0100

> message could make that assumption.

Neither is a HTTP/1.2 message as it says nothing about what protocol
version the client supports, only the protocol version of the last hop.

Attacking this via protocol version numbers is not the correct approach.
It's done by advertising the support in a header which is the only
end-to-end concept there is in HTTP.

Making the use of a new header mandatory does not require a new protocol
version, just a new standard strack RFC defining the header as
mandatory.

Bumping the protocol version simplifies compliance specifications as it
becomes sufficient to say one is compliant with HTTP/X.Y instead of a
list of RFCs.

Any major changes in the protocol such as moving away from being a
message based protocol to being a connection oriented protocol would
require a major version bump.

Any change in message format on the hop-by-hop level such as the
introduction of something on the same level a Connect header or a
another transfer-encoding model would require a minor version bump. So
does changing the requirements on how trailers may be used to a must.

Regards
Henrik

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 17:07:00 -0500

> version, just a new standard strack RFC defining the header as

Which part of RFC 2145 or RFC 2616 supports this assertion? I think
they both contradict it.

A new RFC can make a header mandatory for RFCNNNN compliance, but not
HTTP/1.1 compliance.

-- 

Robert Sayre

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 04 Nov 2006 23:15:19 +0100

> it will be lost in the noise.

Actually it doesn't even add an extra round trip as the client is
supposed to handle the situation where it does not understand any of the
challenges gracefully and inform the user that it's not capable to
access the resource. Unfortunately many client implementations is broken
and falls back on Basic under such conditions but that is another story
entirely not related to HTTP/1.1 but simply implementation bugs.

Regarding the specs they already say

  "Servers should only include Basic if it is minimally acceptable."

and

  "The user agent MUST choose to use one of the challenges with the
strongest auth-scheme it understands".

and

  "Because Basic authentication involves the cleartext transmission of
   passwords it SHOULD NOT be used (without enhancements) to protect
   sensitive or valuable information."

all quotes from RFC2617 btw.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 04 Nov 2006 23:25:38 +0100

> A new RFC can make a header mandatory for RFCNNNN compliance, but not

Exacly what I said.

Regards
Henrik

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 17:27:37 -0500

> > A new RFC can make a header mandatory for RFCNNNN compliance, but not

OK. Then I submit that such an RFC cannot claim to define HTTP/1.1.

-- 

Robert Sayre

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 04 Nov 2006 23:42:50 +0100

> OK. Then I submit that such an RFC cannot claim to define HTTP/1.1.

Agreed. It's at most an standards track extension to HTTP/1.1.

Also for the record I am against that implementation of strong
authentication should be mandatory for HTTP protocol compliance.

A requirement of implementation of a well defined strong authentication
scheme IF authentication is implemented is fine however.

Regards
Henrik

---

From: Paul Leach <[email-masked]>
Date: Sat, 4 Nov 2006 20:10:09 -0800

-----Original Message-----

With no malice, I don't think you have good understanding of how the
IESG interprets "mandatory-to-implement".
[Paul Leach] I believe I do. I've been through this a couple times
before with other RFCs.

Let's say Basic becomes[Paul Leach] mandatory-to-implement. That means
FooCorp could not distribute a
FooCorp-branded client that has no way to be configured for Basic
authentication and claim HTTP conformance.
[Paul Leach] That's correct. But it can be compliant and be _configured_
to _not_ use Basic, as long as it can also be configured _to_ use Basic
-- i.e., as long as it implements Basic. That's the difference between
MUST and a "mandatory-to-implement" option.

Which is pretty silly given that proprietary Web server applications
exist only as deployed--there is no separate "implementation".
[Paul Leach] I don't understand the above sentence.

---

From: "Robert Sayre" <[email-masked]>
Date: Sat, 4 Nov 2006 23:19:52 -0500

> [Paul Leach] I don't understand the above sentence.

Increasingly, software is written expressly for one website, not
distributed through traditional commercial software channels such as
CD-ROMs or pre-installed on new computers. This style of deployment
has a lot of advantages, and the implement/configure distinction is
meaningless. There is only one copy.

At any rate, I believe other messages have established that the
meaning of the HTTP version number field is pretty clear. I think the
list should revisit this topic when everyone is prepared to accept the
requirements of RFC 2616 and RFC 2145. Is there something unclear
about "conditional conformance"?

-- 

Robert Sayre

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Sun, 05 Nov 2006 00:22:06 -0600

> be understood by the client.

Not if they implemented an RFC 2616 client.

---

From: Paul Leach <[email-masked]>
Date: Sat, 4 Nov 2006 22:51:03 -0800

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of William A. Rowe, Jr.
Sent: Sunday, November 05, 2006 1:22 AM
To: Paul Leach
Cc: Robert Sayre; Henrik Nordstrom; HTTP Working Group
Subject: Re: security requirements (was: Updating RFC 2617 (HTTP Digest)
to use UTF-8)

Not if they implemented an RFC 2616 client.
[Paul Leach] Why do you think that? 2616 says that authentication is
OPTIONAL:

11 Access Authentication

   HTTP provides several OPTIONAL challenge-response authentication
   mechanisms which can be used by a server to challenge a client
   request and by a client to provide authentication information. The
   general framework for access authentication, and the specification of
   "basic" and "digest" authentication, are specified in "HTTP
   Authentication: Basic and Digest Access Authentication" [43]. This
   specification adopts the definitions of "challenge" and "credentials"
   from that specification.

---

From: Paul Leach <[email-masked]>
Date: Sat, 4 Nov 2006 22:58:36 -0800

-----Original Message-----

[Paul Leach] Do you mean "conditional compliance"? As in 2616:

1.2 Requirements

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [34].

   An implementation is not compliant if it fails to satisfy one or more
   of the MUST or REQUIRED level requirements for the protocols it
   implements. An implementation that satisfies all the MUST or REQUIRED
   level and all the SHOULD level requirements for its protocols is said
   to be "unconditionally compliant"; one that satisfies all the MUST
   level requirements but not all the SHOULD level requirements for its
   protocols is said to be "conditionally compliant."

---

From: Julian Reschke <[email-masked]>
Date: Sun, 05 Nov 2006 19:59:38 +0100

> of existing implementations (regardless whether server or client).

...unless it could be demonstrated that in practice all implementation 
already are compliant to that new requirement (which I doubt is going to 
happen :-).

> feasible, to add to require the impl of any mandatory auth scheme

Yep.

HTTP/1.1 is widely deployed. Changing the mandatory requirements so that 
existing compliant implementations become non-compliant just doesn't 
compute.

 > ...

Best regards, Julian

---

From: Lisa Dusseault <[email-masked]>
Date: Sun, 5 Nov 2006 13:23:23 -0800

> Agreed. It's at most an standards track extension to HTTP/1.1.

Slight disagreement here: if RFCNNNN obsoleted RFC2616, without  
bumping the version number, it had better be backwards compatible --  
but it is more than a standards track extension to HTTP/1.1, it  
becomes the new best definition of HTTP/1.1.

> scheme IF authentication is implemented is fine however.

That's not a bad start.  The next thing to think about is to ask in  
what cases authentication implementation IS required.  I certainly  
agree with those who've said that authentication isn't necessary in  
some uses of HTTP.

Lisa

---

From: Lisa Dusseault <[email-masked]>
Date: Sun, 5 Nov 2006 13:26:03 -0800

On Nov 4, 2006, at 12:41 PM, William A. Rowe, Jr. wrote:

>> that

I didn't mean that servers must USE Basic and Digest to replace form- 
based security -- you're right, that would break a bunch of Web  
applications.  IETF specs tend to make requirements of  
implementations and less often make requirements of deployments or  
administrators or sites.  An implementation may SUPPORT Basic and  
Digest as required, and also support form-based auth, and continue to  
use form-based auth to make their WebUIs run.

Lisa

---

From: "Robert Sayre" <[email-masked]>
Date: Sun, 5 Nov 2006 16:34:16 -0500

> becomes the new best definition of HTTP/1.1.

No. Scroll up. Please reconcile your statements with RFC2616 and
RFC2145. If you want a MUST to mean something, you need to take them
seriously.

thanks,

Robert Sayre