Subject: Re: Straw-man charter for http-bis

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 01 Jun 2007 02:01:10 +0200

tor 2007-05-31 klockan 15:59 -0700 skrev Roy T. Fielding:

> If I make the real changes that are needed in draft form and submit
> them to the WG, then I will expect them to be evaluated without bias
> or the WG to be closed.  If the answer is "that's too much
> for me to review, so you aren't allowed to do that in the IETF"
> then I won't.  I will do it elsewhere and the IETF specification
> will become irrelevant.

My requirement is that it's possible to review the changes relative to
2616 without decompiling both documents. This not for the review of the
new document as such, but to review the potential interoperability
issues with previous revisions.

I am fine with quite substantial reordering and restructuring where
motivated, and also some new definitions to better separate the
different aspects & levels of the protocol and partial rewrite to make
use of these. But not a complete rewrite.

> In your opinion.  You chose to ignore mine, in spite of the fact that
> I have a bit of history on the subject, and that is why I have to make
> comments on these proposals.

The scope for this effort has been discussed for quite some time now.
What Mark put down in the proposed WG charter is mainly a summary of
what has been discussed and agreed on this mailinglist for the last year
or so.

It's possible these perhaps isn't the correct goals for the WG, but it's
a distillation what has been discussed and generally agreed on by
consensus on this mailinglist.

> I don't hear anyone else saying that 2616 needs to be revised before
> 2617, yet you continue to take that as an assumption.  2616 doesn't
> *need* to be revised at all.  2617 desperately does need to in order
> to meet the IESG requirements.  Why is that unclear?

2616 at minimum needs editorial updates and clarifications. A fair bit
more is desirable to get a clearer specification and maybe even a better
protocol, but imho not strictly required to finalize HTTP/1.1 as a
STANDARD. I think it's undesirable to move the current document to
STANDARD without cleanup.

2617 at minimum needs quite significant work indeed, and the result will
also take considerably more time to get deployed and tested. The kinds
of problems that need to be addressed is also a bit different from 2616.

Regards
Henrik

---

From: Paul Leach <[email-masked]>
Date: Thu, 31 May 2007 17:30:35 -0700

-----Original Message-----
From: Henrik Nordstrom [mailto:[email-masked]] 
Sent: Thursday, May 31, 2007 4:08 PM

> That was what my second suggestion from the message, part of which you

> quoted above, was about. I guess it wasn't clear enough.

Good. So lets keep that in mind when talking about RFC2617bis.

If Microsoft is willing to rework their authentication schemes so that a
future revision fits the HTTP message model, perhaps by using virtual
sessions instead of transport connections to identify the authentication
session then the need for extending HTTP to officially supporting
transport level authentication disappears.

[Paul Leach] I don't have anything to do with implementations of HTTP by
Microsoft anymore, so I can't say anything about what we might support
in the future. But even if we did what you suggest, it would take a long
time before any significant number of sites used it, because all sites
have to cater to older browsers, from many vendors.

> It would be a better approach, but it would still be pretty helpful to

> tell people how to interop with the existing approach.

The existing RFC is perfectly fine for that purpose.

The question here is if the requirements of RFC4559 "SPNEGO-based
Kerberos and NTLM HTTP Authentication in Microsoft Windows" should be on
the table while revising the HTTP specifications or not. I say firmly
not.
[Paul Leach] What requirements are you referring to?

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jun 2007 10:57:53 +1000

On 01/06/2007, at 8:59 AM, Roy T. Fielding wrote:

> It is not "if I don't get my way".  Who appointed you to be the
> determinant of group consensus in the first place?  I am not going
> to support an IETF working group that says "nobody is allowed
> to do a better job describing HTTP than what is in our charter."
> If you don't allow people to produce drafts, and allow the working
> group to evaluate those drafts on the basis of whether the contents
> are better or not, then all you are doing is dictating the content
> of the specification based upon some imagined position of authority.

You spoke about forking the HTTP spec if the IETF decided to go in a  
direction that you didn't like. Working outside the process is very  
different to working within it, whether that be in the charter  
discussions or the WG. Don't try to make this about me.

> So, you should decide whether your work item is to replace 2616
> or to produce a list of errata to be published in RFC form.  If it
> is the former, than I know a lot more about this subject than you
> and I know for a fact that just making small changes around the
> edges is not sufficient.

That's a naked appeal to authority, and I'll give it as much credence  
as it deserves.

> We already tried that twice.

I believe the circumstances are different; time changes things.

>> Also, on what do you base the accusation that this is being driven  
>> by "short-term corporate agendas?"
>
> On the basis that you presuppose every work item as being limited
> to what you want to do, rather than a task that can be accomplished
> if someone does it, and the continual reference to vendors and
> "developers" that never actually show themselves on this list.

Can you substantiate that?

>> In any case, I don't think re-organising parts of the spec is off  
>> the table; indeed, it's already been discussed on a small scale.  
>> Re-writing the entire spec sentence-for-sentence is, in my opinion.
>

I'm not ignoring you, Roy. However, you're making the same arguments  
repeatedly. If you make arguments that convince me, or if you can get  
other people to agree with you, I'm happy to reconsider my opinion.  
Neither has happened yet.

>>> If an IETF HTTP WG is to be recreated, then its task items should be
>>> to create what the working group believes to be the best documents
>>> to replace 2616 and 2617.  The charter does not need to constrain  
>>> that.
>>
>> That would be disruptive and unproductive. You may be willing to  
>> re-write HTTP from scratch, but the review requirements are much  
>> higher than required for what we're attempting (with step-by-step  
>> diffs, by the way).
>>
>> Somehow, HTTP has been implemented and become one of the most  
>> widely-deployed application protocols today, despite your claims  
>> that the spec needs to be re-written from scratch. I don't hear  
>> *anyone* else saying that this necessary, or a realistic option.
>

Have you been following the same discussion that I have?

Later, you said to Larry:

> I said that I would if it were given a chance to be reviewed.
> I offered as much to Lisa before this effort started, and before
> the revision was public, and again once people started working on it,
> and each of those times I was rebuffed because the people organizing
> this WG don't think a full revision should be in scope.

I'm willing to do a reasonable amount of work to polish the spec, but  
at this point am unwilling to commit to start from scratch, both  
because I don't think it's likely to be successful, or that I'd be  
able to put the time in. My perception is that most people are in the  
same boat. If you're willing to put that kind of effort in, good for  
you, but it would be misleading to say that I'd have the time to see  
it through to the end. If you want to do that, go ahead and find  
other people who are willing to make it work; that's what I've trying  
to do for this effort.

--
Mark Nottingham     http://www.mnot.net/

---

From: "Robert Sayre" <[email-masked]>
Date: Thu, 31 May 2007 21:04:32 -0400

On 5/31/07, Roy T. Fielding <[email-masked]> wrote:
> I am not going
...
> then I won't.  I will do it elsewhere and the IETF specification

I think application of forking pressure is OK. It is always present anyway.

I don't understand why the two approaches are mutually exclusive. I
think the best way to start is by doing exactly what has been done so
far. I agree with Roy that we shouldn't rule out large scale
restructuring at some point, but it might be better to let everyone
look at a few rounds of diffs before any major structural changes are
made. The issues list is already getting a little big.

> 2617, yet you continue to take that as an assumption.

The order is irrelevant, and they don't need to be undertaken by the same group.

> 2616 doesn't *need* to be revised at all.

Disagree. The document is losing usefulness as a reference because it
is poorly structured, crawling with inaccuracies, and the net is full
of things that claim to be HTTP but aren't.

> 2617 desperately does need to in order

The requirements have been revisited.

-- 

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jun 2007 11:12:08 +1000

> made. The issues list is already getting a little big.

Agreed. As I said earlier, we've already talked about rearranging  
sections, and maybe rewriting parts of the caching section.

Perhaps we're just talking past each other -- Roy, are you really  
talking about a wholesale rewrite, or rearranging parts, or what? If  
you gave us an idea of what you want to do, it might help make this  
discussion more productive.

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 01 Jun 2007 03:23:21 +0200

tor 2007-05-31 klockan 17:30 -0700 skrev Paul Leach:

> I don't have anything to do with implementations of HTTP by

Sure. No problem.

> What requirements are you referring to?

In specific the requirements on strict transport connection association
for the authentication chain, requiring responses to challenges to be
transmitted over the same transport connection, and that messages are
authenticated without having any authorization headers simply by being
transmitted on a connection where authentication has completed.

Or in HTTP the required changes in the transport/message layering to
support the above, including forwarding via proxies, plus the inherited
Authorization status from the transport connection..

Regards
Henrik

---

From: "Roy T. Fielding" <[email-masked]>
Date: Thu, 31 May 2007 19:25:12 -0700

On May 31, 2007, at 6:12 PM, Mark Nottingham wrote:
> talking about a wholesale rewrite, or rearranging parts, or what?  
> If you gave us an idea of what you want to do, it might help make  
> this discussion more productive.

I am talking about splitting the document into

    messaging
    conditional requests and range requests
    caching and cache control
    content negotiation
    http and https URIs
    HTTP implementation guide for TCP transports

to go along with

    authentication and access control

and then replacing the BNF with ABNF, removing the (relatively few)
parts of the document that specify server implementation instead of
interface requirements in the method definitions, and fixing the
errata already identified.  I can do that in late July and then
publish one draft each per month until it is done.

Note that there are many other things that *could* be changed,
such as eliminating line-folding from the header definitions for
HTTP over TCP, but that would not be part of making the document
reviewable again.  Other things will probably be revealed once
I am forced to actually read the existing text, line-by-line.
Any changes that I make would be visible because I would use a
public subversion repository to make them.

If, for some unforeseen reason, I am unable to make those drafts
happen, then nothing on your charter schedule needs to change.
If the WG chooses not to adopt the changed drafts after they
have been reviewed, then so be it.  I just don't want to burn
the time editing the drafts only to have someone declare
discussion of them to be out of scope.

BTW, structural changes have to be done first, using 2616 as
the basis.  There is no point of reviewing diffs of 2616 when
nobody bothers to read 2616 as a whole.  Diffs after restructuring
can then be viewed in their proper context, with all of the relevant
requirements in the same comprehensible space.

....Roy

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jun 2007 14:19:25 +1000

On 01/06/2007, at 12:25 PM, Roy T. Fielding wrote:

> I am talking about splitting the document into

4, 5, 6, 7, 9, 10, various parts of 3, parts of the relevant header  
defs in 14

>    conditional requests and range requests

3.12, parts of 10.3.5 and 10.2.7, parts of the relevant header defs  
in 14

>    caching and cache control

13, parts of the relevant header defs in 14

>    content negotiation

12, parts of the relevant header defs in 14

>    http and https URIs

3.2

>    HTTP implementation guide for TCP transports

8, parts of the relevant header defs in 14

>    authentication and access control

11, parts of the relevant header defs in 14

Is that in the ballpark? Couple of questions;

1) If going down this path, my intuition would be to continue having  
sections dedicated to lists of headers, status codes, methods, etc.,  
but to limit them to syntax and base semantics, while pushing the  
protocol-related parts (e.g., the mechanics of caching, conditional  
requests, etc.) into the separate sections. Is that what you had in  
mind?

2) With the exception of the impact of (1), the above seems like  
relatively straightforward rearrangement. Did you just have  
rearrangement in mind (perhaps with new introductory text), or did  
you want deeper changes?

> and then replacing the BNF with ABNF

Already on the table.

> removing the (relatively few)
> interface requirements in the method definitions

Could you please raise this as an issue when you get a chance? I  
think I know what you're talking about, but examples and proposals  
would be helpful.

> , and fixing the

Chicago's cutoff for -00 drafts is July 2; could you have an initial  
draft by then, perhaps just doing the rearrangement and leaving ABNF  
and errata until later? Something to give people even a rough idea of  
what you were talking about would be very helpful.

> discussion of them to be out of scope.

My personal preference would be to make a decision beforehand; a well- 
scoped charter can be frustrating, but it can also prevent many  
ratholes. Knowing what they're getting into raises people's comfort  
level considerably.

If you can get a draft in before Chicago, we could discuss it there  
and either get them built into the WG's scope, or decide that they're  
already within it, or even base the charter on your document.

> requirements in the same comprehensible space.

That makes it seem that you're thinking of something more substantial  
than a straight rearrangement.

One more question (not necessarily for Roy): at what point does  
rewrite/rearrangement affect the transition from Proposed to Draft?

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 1 Jun 2007 06:15:03 +0100

Mark Nottingham wrote:
> I'm very aware of your feelings. I'm also aware of the pain that  
> folks go through when they try to implement the current spec. Yes,  
> some of that is caused by the organisation and committee-speak there,  
> but much more is on very specific points where the spec is silent or  
> misleading -- our issues list now has more than sixty issues. That  
> this effort will help in those cases. No, it's not a magic pill, but  
> a complete rewrite wouldn't be either, and it would have much less  
> chance of success.

Just a little thing.  I think the issues list is very helpful, from an
implementors point of view.  Even where there are unclarified or
unspecified behaviours, an implementor can see at least that there may
be varying interpretations, and make a decision about how to handle
them.

I think that it's _more_ useful than trying to squeeze out every last
typo or grammatical confusion from the standard.  There's only so much
energy; attempted perfection in a standard can take away from
recording practical outside-the-standard issues which have to be dealt
with in practice.

I think a "known common errors in deployed implementations" list would
also be helpful, for the same reasons.  (Like the TCP one).

-- Jamie

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 1 Jun 2007 15:17:04 +1000

> rewrite/rearrangement affect the transition from Proposed to Draft?

*shakes head at self*

Let me re-phrase that: does a rewrite/rearrangement affect the Draft  
Standard status? Considering:

>    A Draft Standard is normally considered to be a final  
> specification,
>    and changes are likely to be made only to solve specific problems
>    encountered.  In most circumstances, it is reasonable for  
> vendors to
>    deploy implementations of Draft Standards into a disruption  
> sensitive
>    environment.

---

From: Leif Johansson <[email-masked]>
Date: Fri, 01 Jun 2007 08:15:28 +0200

Paul Leach wrote:
> Sent: Thursday, May 31, 2007 3:12 PM
> To: Paul Leach
> Cc: Eric Lawrence; Cyrus Daboo; [email-masked]
> Subject: RE: Straw-man charter for http-bis -- call for
> errata/clarifications to 2617
>
> tor 2007-05-31 klockan 14:54 -0700 skrev Paul Leach:
>
>   
>> 1. The requirements (use of connection-keep-alive, proxy issues, etc) 
>> for secure use of per-connection authentication could be described in 
>> 2617bis.  AFAIK, these could reflect some actual implementation 
>> experience.
>>     
>
> Connection oriented authentication requires support in the base HTTP
> specs for such schemes, as it has far going implications on transport
> and message requirements.
> [Paul Leach] Since I think people safely use it today, I don't think any
> additions are needed. At least when no proxy server is involved -- I
> forget the trick used to make sure that proxies preserve connection
> semantics before relying on Kerb/SPNEGO when using a proxy. It may be
> that they won't be used if a proxy is involved.
>
> Would be more fruitful to rework NTLM/Negotiate to fit in the HTTP
> message model I think, operating somewhat similar in principle (but
> obviously not algorithm) to Digest MD5-sess with a virtual session
> identifier separate from the transport connection.
> [Paul Leach] That was what my second suggestion from the message, part
> of which you quoted above, was about. I guess it wasn't clear enough. 
>

I guess I tried to address at least some of these concerns in my
drafts
http://www.ietf.org/internet-drafts/draft-johansson-http-tls-cb-01.txt
and http://www.ietf.org/internet-drafts/draft-johansson-http-gss-01.txt

Based on (and intended to be at least culturally compatible with) Negotiate
this mechanism doesn't alter the fundamental nature of what constitutes
an HTTP authentication mechanism (imho) but still introduces a connection
identifier of sorts based on the notion of channel bindings.

    Cheers Leif

---

From: "Justin Erenkrantz" <[email-masked]>
Date: Thu, 31 May 2007 23:29:35 -0700

> Neither has happened yet.

I feel that artificially restricting the WG's charter to only discuss
'light editorial' rewrites - i.e. typographical and editing mistakes -
would be setting the bar way too low.

Obviously, any edit of 2616 shouldn't add any features to still have
the end result called HTTP/1.1; but, as an implementor of HTTP, it'd
be nice to have a document that was more accessible to people who are
new to learning about HTTP implementations.  For almost everyone on
this list, we've already got our heads wrapped around the current 2616
structure - so I understand your resistance to making anything but
trivial fixes.  Yet, I think aiming that low does a disservice to the
community-at-large and to the future community - as parts of 2616
unintentionally ended up being complete mush.

BTW, one comment that I've seen mentioned here is that we couldn't
begin to touch 2616 unless we get 2617 to be 'secure' first - if
that's true, can someone please explain the rationale behind that and
what specifically the IESG is requiring out of any new WG with respect
to HTTP authentication?

Thanks.  -- justin

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 03:55:55 -0400

> Standard status? 
one of the main purposes of the document status is to show how much
confidence there is in the specification's clarity and accuracy.  if you
substantially rewrite a specification, whatever confidence you had in
the original specification no longer applies to the new specification. 
and you need to reestablish confidence in the rewritten specification. 
that implies a reset to Proposed.

so basically if you argue that the HTTP spec is so bad that a major
rewrite is needed, you're also arguing for
the rewritten HTTP spec to have a Proposed Standard status.  (as was
done for [2]822 and SMTP).

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 10:42:57 -0400

> of things that claim to be HTTP but aren't.
is this because implementors tried to read the existing HTTP spec and
failed to grasp it, or because they didn't even bother trying to read
the spec?  (seriously, at least for SMTP it's pretty obvious that a lot
of authors of bad implementations never bothered to read the spec, they
just copied what they saw someone else do. )

revising 2616 will help the implementors who are actually trying to get
it right, and for that reason it is probably a worthwhile effort.  but
it won't do anything for the remainder of the implementations.

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 10:48:39 -0400

> to do a better job describing HTTP than what is in our charter."
that's your choice, of course.  but the charter wording is up to the
IESG, and they have the authority to make decisions about what kinds of
activities are in scope and which kinds of activities are out of scope. 
(the draft charter being discussed is just someone's idea of a proposal
to be given to IESG and it's reasonable to debate it, or to suggest your
own draft charter to the applications ADs)

for better or worse, there's a lot of investment in the current prose. 
a drastic rewrite might remove some ambiguities but would certainly
create others - and also create questions about exactly what was
changed.  if HTTP is updated by making relatively minor tweaks where
possible, and major changes to text only when necessary, it's much
clearer what was changed than if there's a major restructuring/rewriting
of the document.  that, and rewriting would force a reset to Proposed
Standard and create an ambiguity over whether the Proposed or Draft
version were authoritative.

Keith

---

From: "Robert Sayre" <[email-masked]>
Date: Fri, 1 Jun 2007 11:09:26 -0400

> ratholes. Knowing what they're getting into raises people's comfort

I think Roy's suggestion fits well within the charter you've just sent
out, which doesn't restrict the amount of editing that can be done.
The debate centers on the best way to *describe* HTTP 1.1 in a timely
manner. This is a good argument to be having. Perhaps people should
suggest concrete changes to the charter text instead of merely stating
their opinions. Seems to me like we need some text that states what we
don't have to accomplish regarding authentication, and some text that
explicitly allows large scale editorial changes.

> already within it, or even base the charter on your document.

That would be great, but it's a lot of work, and that's a tight
deadline. An outline sent to the list before the meeting would be
enough, I think.

> than a straight rearrangement.

That's exactly the argument. If a "more substantial" rewrite does a
better job of documenting HTTP, we should consider it. This
possibility shouldn't cause discomfort, because our shared goal is to
accurately document HTTP 1.1, right?

> done for [2]822 and SMTP).

Personally, I don't care what color ribbon our pig wins at the county fair. :)

On 6/1/07, Keith Moore <[email-masked]> wrote:
> the spec?

Here's a concrete example:

<http://skrb.org/ietf/http_errata.html#post>

I am sick of questions about this from people who have only read
RFC2616. It's a waste of my time and theirs.

-- 

Robert Sayre

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 12:55:46 -0400

> accurately document HTTP 1.1, right?
The catch is that HTTP is (currently) specified by RFC 2616, and the
most accurate documentation about HTTP is in RFC 2616.  If you replace
2616 with a completely different specification, you're not "accurately
documenting" HTTP, you're _changing_ the specification.   You will
inevitably create incompatibilities between "old" HTTP and "new" HTTP.

I'm not saying it's inherently a bad idea to do that, I'm saying that a
rewrite is going to cause some interoperability issues even if you end
up with a much clearer and/or more precise specification.

People contemplating a rewrite of HTTP should really look hard that the
experience from the DRUMS working group that updated RFC 822 and SMTP. 
It took a very long time, and while the results are clearer in many ways
than the original specifications, IMHO they are not an improvement in
every respect.  For instance, a lot of effort was spent in  rewriting
the ABNF that describes internet messages.  The new ABNF is perhaps more
precise but it's _much_ harder to write a correct parser for the new
grammar - and there's very little benefit to doing so because new mail
readers still need to be able to parse pre-RFC2822 messages.  So in
practice, to write a good mail reader, you now need to refer to BOTH RFC
822 and RFC 2822 - because the RFC 822 grammar is the best one to write
a parser from, and 2822 has the grammar that you should use when writing
code to generate new messages.   (And they're in different notations!)
This means _more_ work for the implementor, a higher barrier to
producing a correct  implementation, and a greater temptation to not
fully read the specifications and just implement whatever seems to work.

Again, I'm not saying don't rewrite the HTTP spec.  I'm saying don't
underestimate the effort involved, or the harm caused by unintended
changes or the difficulty of comparing old and new specifications with
different structures.  And be really sure you understand how the rewrite
is going to benefit the community, taking into account that a lot of
implementors won't bother reading the spec and a lot of existing
implementations aren't going to get significant updates to conform to
the new spec.

> Personally, I don't care what color ribbon our pig wins at the county
Even if it is cute enough to win a ribbon, do we really need another pig?

Keith

---

From: Julian Reschke <[email-masked]>
Date: Fri, 01 Jun 2007 19:51:23 +0200

Hi,

I'd like to make one small comment with respect to the opinion that 
maintaining an errata list (and potentially handing that to the RFC 
Editor) would be sufficient.

1) Scott Lawrence' original errata list 
(<http://purl.org/NET/http-errata> is excellent, but it hasn't been 
maintained since 2004. So we needed to move somewhere else.

2) Just collecting errata sounds nice in theory, but my experience with 
spec writing is that you can't close a bug until you have applied the 
suggested fix to the spec text. Frequently, something that looks OK in 
isolation doesn't work in the specification context. Thus my preference 
is not only to collect errata and proposed resolutions, but to also have 
them applied to a copy of the original spec (and have that up for review 
for everybody).

3) Finally, looking at the amount of issues we have collected in the 
meantime, I'd be really amazed if the RFC Editor would be willing to 
take over the editorial work for updating the document. I bet the answer 
would be: please submit an Internet Draft.

Best regards, Julian

---

From: Stefan Eissing <[email-masked]>
Date: Fri, 1 Jun 2007 20:50:05 +0200

Am 01.06.2007 um 18:55 schrieb Keith Moore:
>> [Robert]That's exactly the argument. If a "more substantial"  
>> rewrite does a

Taking a step back, what needs attention from the best of minds is  
2617. Let's face it: http authentication is awkward and compared to  
the rest of the protocol it feels like a child's toy, sitting in the  
glove compartment of a BMW.

With this in mind, I think a complete rewrite of 2616 is a waste of  
time and resources.

//Stefan

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 14:54:08 -0400

Julian Reschke wrote:
> 2) Just collecting errata sounds nice in theory, but my experience
> with spec writing is that you can't close a bug until you have applied
> the suggested fix to the spec text. Frequently, something that looks
> OK in isolation doesn't work in the specification context. Thus my
> preference is not only to collect errata and proposed resolutions, but
> to also have them applied to a copy of the original spec (and have
> that up for review for everybody).
>
> take over the editorial work for updating the document. I bet the
> answer would be: please submit an Internet Draft.
>
> Best regards, Julian

My recommendation would be for the group to construct a list of errata
and get consensus on that list.  Each erratum should mention the
specific sections and text of RFC 2616 that it applies to, what the
problem is, and what changes are needed to fix the problem.

By the time the list is nearing completion, it should be apparent
whether it's worth the effort to revise the HTTP specification.  The
original errata list would still be useful, perhaps as an appendix,
because many implementors will just want to know what has changed.

My guess is that if the group sees its task as making a good
errata-and-fix list for 2616,  it will stay focused and finish in a
reasonable amount of time.  If at that point it is seen as appropriate
to actually update 2616, this will be a straightforward task which won't
take a lot of additional time.  (I do not propose that this task be
delegated to the RFC editor - the RFC editor function needs to stay
separate.)

On the other hand, if the working group sees its task as revising 2616,
the chance that it will take several years, dig into a dozen ratholes,
and create even more ambiguity than currently exists, is quite large.

Keith

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 14:55:44 -0400

> glove compartment of a BMW.
very much agree.  HTTP authentication as it currently exists is nearly
useless, and forms-and-cookie authentication (at least as it tends to be
implemented) isn't sufficient.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 01 Jun 2007 21:06:51 +0200

> problem is, and what changes are needed to fix the problem.

Yes, that's what we have been (slowly) doing over the last months. See 
<http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/issues/>.

> because many implementors will just want to know what has changed.

<http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/draft-lafon-rfc2616bis-02.html#changes.from.rfc.2616>

> delegated to the RFC editor - the RFC editor function needs to stay

I personally think that this should be a by-product of collecting and 
resolving the errata.

> and create even more ambiguity than currently exists, is quite large.

I think this is why an attempt was made to restrict the charter as much 
as possible.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 15:12:46 -0400

> <http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/issues/>.
I understand, and this is a useful head start.  But you don't have a
working group yet, and therefore you don't have working group consensus
yet.  The item isn't completed until an open, chartered working group
has actually had time to go over the document.   It's not acceptable to
skip this step.
> I personally think that this should be a by-product of collecting and
I disagree, because I think the temptation to needlessly edit text will
be too large.

Keith

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 1 Jun 2007 12:20:43 -0700

At 2:06 PM -0400 6/1/07, John C Klensin wrote:
>While I'm not sure I'd recommend it (in part for the reasons
>Julian identifies under (2) above), one could, in principle,
>prepare an I-D that was simply a listing of errata and
>discussion of ambiguities in the base spec, get rough consensus
>on that listing, and then ask the the IESG to process it and the
>RFC Editor to publish.  That would be a little bit unusual, and
>would give the reader an extra, and separate, document to read,
>but I don't know of anything that would prohibit it procedurally.

In fact, there is a recent example of this very thing happening: 
"IKEv2 Clarifications and Implementation Guidelines", RFC 4718. It is 
about 50 pages of clarifications on RFC 4306. The process of 
developing the spec went quite smoothly, and it really helped during 
the first f2f interop event.

Based on the feedback we have gotten from developers, we are 
preparing an IKEv2bis which is RFC 4718 mixed into RFC 4306. They 
really wanted it all in one document. The draft for that is currently 
expired, but will be revived in the very near future.

Note that some of the "clarifications" in RFC 4718 are in fact 
technical changes that were realized after RFC 4306 was published. We 
wiggly-worded them to sound like clarifications, but they are fixes 
to things that are broken. I would be quite surprised if the authors 
of an HTTP clarifications effort were not put into the exact same 
situation.

 From my experience with the IKEv2 document, I propose that you do a 
clarifications-and-errata document first, get most of the way 
through, then decide whether or not to do a bis document.

(I was tempted to change my From: line to my @vpnc.org address 
because of the context, but I doubt the message would have gotten to 
this mailing list if I had.)

---

From: Julian Reschke <[email-masked]>
Date: Fri, 01 Jun 2007 21:22:45 +0200

> has actually had time to go over the document.   It's not acceptable to

Right now things that are marked closed are in this state because the 
proposed resolutions got consensus on [email-masked]. I know that's 
not a WG consensus, but it's the closest thing to that that we currently 
have.

Of course, should an HTTP WG be formed, all these proposed resolutions 
can be challenged again (actually, that can happen even in the absence 
of a WG :-).

> I disagree, because I think the temptation to needlessly edit text will

I guess that depends on who's doing the editing. Speaking for myself, I 
have tried to make sure that changes are kept as minimal as possible, 
and that each and every change is linked to the issue it's supposed to 
resolve.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Fri, 01 Jun 2007 15:44:08 -0400

> I guess that depends on who's doing the editing. Speaking for myself,
> I have tried to make sure that changes are kept as minimal as
> possible, and that each and every change is linked to the issue it's
> supposed to resolve.
Maybe I should clarify.  If the working group sees itself as updating
2616, the working group can get into a rathole discussing changes to the
text even if the people participating in that discussion can't actually
change the text and submit a revised internet-draft for that text. But
if it's clear up front that the WG is not chartered to actually change
the document text, the participants are more likely to confine their
discussion to descriptions of specific problems and fixes.

(there's a reason we describe the job of being a WG chair (and for that
matter, an AD) as "herding cats".)

Keith

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 1 Jun 2007 21:25:50 +0100

> Editor) would be sufficient.

Fwiw, I'm not saying that just keeping errata is sufficient.  But I
think that an updated RFC _without_ keeping track of the errata
(i.e. not keeping track of them as they are now) would be a mistake.

Someone else mentioned the importance of being able to see what
changed from one version of the RFC to another, without having to
decompile the text.  I agree.  How the text is written and organised,
and the quality and focus of a "changes" section, are part of that.
Having access to the errata which prompt the new RFC version are also
part of it.

The errata is a list of potential interop and
quality-of-implementation issues (not everything on it, but some
things), which are useful to implementors and people designing similar
protocols.  It's valuable in its own way, but that doesn't mean
editing the RFC is not valuable as well.

-- Jamie

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 02 Jun 2007 13:12:14 +0200

> editing the RFC is not valuable as well.

Also, having the RFC2616bis document WITH the errata corrected greatly
helps verifying the errata.

Additionally it helps to avoid repeated discussions about the same
error, but that's just an positive side effect.

Regards
Henrik

---

From: John C Klensin <[email-masked]>
Date: Fri, 01 Jun 2007 18:06:15 +0000

--On Friday, 01 June, 2007 19:51 +0200 Julian Reschke
<[email-masked]> wrote:

> Hi,
> 
> I'd like to make one small comment with respect to the opinion
> that maintaining an errata list (and potentially handing that
> to the RFC Editor) would be sufficient.
> 
> (<http://purl.org/NET/http-errata> is excellent, but it hasn't
> been maintained since 2004. So we needed to move somewhere
> else.
> 
> 2) Just collecting errata sounds nice in theory, but my
> experience with spec writing is that you can't close a bug
> until you have applied the suggested fix to the spec text.
> Frequently, something that looks OK in isolation doesn't work
> in the specification context. Thus my preference is not only
> to collect errata and proposed resolutions, but to also have
> them applied to a copy of the original spec (and have that up
> for review for everybody).
> 
> 3) Finally, looking at the amount of issues we have collected
> in the meantime, I'd be really amazed if the RFC Editor would
> be willing to take over the editorial work for updating the
> document. I bet the answer would be: please submit an Internet
> Draft.

Let me add one thing to this, while more or less agreeing with
Julian on this particular point (no comment yet about other
issues).

There are two types of errata.  One type just clears up
typographical and similar errors where the intent of the text is
fairly clear.  Those are trivial, the RFC Editor rarely has a
problem with them, but it is not clear that they are worth a lot
of effort.  The second involves a substantive clarification or
correction to the document, especially one that permits one way
to do things and excludes one or more others that some people
might have thought plausible.  

That second type really requires some explicit process of
determining or verifying community consensus; it is not
something the RFC Editor ought to be publishing as authoritative
at the request of any one party, even if that party were an
author of the original document (or the responsible AD at the
time :-( ).

best regards,
    john

---

From: Eliot Lear <[email-masked]>
Date: Fri, 01 Jun 2007 18:31:37 +0000

Keith Moore wrote:
> every respect.  

I think this is a very valid point.  Many of us still have the scars.

Eliot

---

From: Chris Newman <[email-masked]>
Date: Wed, 06 Jun 2007 15:42:54 -0700

Here's my take as AD on some of the interesting topics in this thread:

1. HTTP Digest Authentication

The SASL WG appears to have decided that SASL DIGEST-MD5 is not a useful 
authentication mechanism for a number of technical reasons.  I would be 
uncomfortable having a WG spend a lot of time refining the existing HTTP Digest 
mechanism based on that experience.  However, documenting the i18n behavior of 
deployed implementations sounds like a sensible thing to do.

2. HTTP Security

Phishing demonstrates that HTTP's present security mechanisms are not adequate 
to meet some important requirements of the present users of the protocol.  I 
would be uncomfortable moving HTTP from Draft Standard to Standard given this 
situation.  It's likely that new work on HTTP security mechanisms (as outlined 
by draft-hartman-webauth-phishing) is necessary.  However, even with the 
present security situation, I have no doubt that RFC 2616 is widely useful and 
improving the technical clarity of the base specification is good work that 
would benefit the Internet community.  The minimum work necessary to make a 
draft standard revision of the base specification complete would be to clearly 
document the limitations of the presently deployed HTTP security mechanisms and 
the fact they are not adequate for all situations.  Beyond that I consider it 
inappropriate to hold publication of a useful revision hostage to new security 
engineering work.  That opinion may not be shared by others on the IESG. 
Regardless, I would very much like to see forward progress on the HTTP security 
situation.

3. One vs. Two WGs

I would support the formation of two separate WGs: HTTP and HTTP security as 
the people who have appropriate expertise for those efforts are not identical. 
Indeed I'd be uncomfortable with a single WG that was both revising 2616 and 
designing new HTTP security mechanisms as the latter may be helped by the 
attention of security experts that likely have no interest in the former.

4. Specification Rewrite

Because the IETF process gives quite a bit of control to the document editor 
and design teams, our process allows an alternate editor to produce a competing 
specification and ask for a WG consensus call to adopt that competing 
specification.  This is discussed in the following IESG Note:
   <http://www.ietf.org/IESG/STATEMENTS/Design-Teams.txt>
>From discussions here, I suspect it's unlikely an alternate specification would 
be adopted by the WG in this case, especially because it might drop the target 
status from draft to proposed for the reasons Keith mentioned.  However, this 
is an important mechanism the keep the process open.

                - Chris Newman
                Applications Area Director

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 07 Jun 2007 09:33:13 +0100

Chris Newman wrote:

> The SASL WG appears to have decided that SASL DIGEST-MD5 is not a 
> useful authentication mechanism for a number of technical reasons.  I 
> would be uncomfortable having a WG spend a lot of time refining the 
> existing HTTP Digest mechanism based on that experience.

Indeed.
But I think it might be worth just doing a revision of HTTP Basic, in 
particular to addresas i18n. I haven't yet made up my mind on whether 
this should happen in the HTTPbis or in another HTTPauth WG.

> However, documenting the i18n behavior of deployed implementations 
> sounds like a sensible thing to do.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 07 Jun 2007 10:40:01 +0200

Chris Newman wrote:
> uncomfortable having a WG spend a lot of time refining the existing HTTP 
> Digest mechanism based on that experience.  However, documenting the 
> i18n behavior of deployed implementations sounds like a sensible thing 
> to do.

Agreed.

Q: if we found out that Digest can't be fixed for I18N without a minor 
incompatible change, would it make sense to define a "Digestbis" 
authentication scheme (assuming for a moment that server and UA 
developers would support it)?

> 2. HTTP Security
> 
> Phishing demonstrates that HTTP's present security mechanisms are not 
> adequate to meet some important requirements of the present users of the 
> protocol.  I would be uncomfortable moving HTTP from Draft Standard to 
> Standard given this situation.  It's likely that new work on HTTP 

Q: RFC2616 doesn't define security at all (except for the hooks used by 
RFC2617). Are you saying that RFC2616 can't be advanced to Full Standard 
without fixes to RFC2617? I guess this translates to the question: is 
the reference to RFC2617 normative or informative... (just looking for 
clarification here).

> security mechanisms (as outlined by draft-hartman-webauth-phishing) is 
> necessary.  However, even with the present security situation, I have no 
> doubt that RFC 2616 is widely useful and improving the technical clarity 
> of the base specification is good work that would benefit the Internet 
> community.  The minimum work necessary to make a draft standard revision 
> of the base specification complete would be to clearly document the 
> limitations of the presently deployed HTTP security mechanisms and the 
> fact they are not adequate for all situations.  Beyond that I consider 
> it inappropriate to hold publication of a useful revision hostage to new 
> security engineering work.  That opinion may not be shared by others on 
> the IESG. Regardless, I would very much like to see forward progress on 
> the HTTP security situation.

Agreed.

> 3. One vs. Two WGs
> 
> I would support the formation of two separate WGs: HTTP and HTTP 
> security as the people who have appropriate expertise for those efforts 
> are not identical. Indeed I'd be uncomfortable with a single WG that was 
> both revising 2616 and designing new HTTP security mechanisms as the 
> latter may be helped by the attention of security experts that likely 
> have no interest in the former.

Agreed. There will be some overlap people-wise, but IMHO that's not a 
problem.

Looking at RFC2617, there seem to be several areas that need work:

(1) The framework itself,

(2) Known issues with Basic and Digest (including I18N),

(3) New schemes.

Re (1): in practice, form based authentication is frequently used for 
the simple reason because people feel that the authentication related 
popups in browsers are insufficient/ugly/whatever. It would be really 
great if the people who are going to work on RFC2617bis co-operate with 
the W3C HTML working group on fixing this.

> 4. Specification Rewrite
> 
> Because the IETF process gives quite a bit of control to the document 
> editor and design teams, our process allows an alternate editor to 
> produce a competing specification and ask for a WG consensus call to 
> adopt that competing specification.  This is discussed in the following 
> IESG Note:
> be adopted by the WG in this case, especially because it might drop the 
> target status from draft to proposed for the reasons Keith mentioned.  
> However, this is an important mechanism the keep the process open.

Agreed.

Best regards, Julian

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 7 Jun 2007 08:44:44 -0700

At 3:42 PM -0700 6/6/07, Chris Newman wrote:
>useful authentication mechanism for a number of technical reasons. 
>I would be uncomfortable having a WG spend a lot of time refining 
>the existing HTTP Digest mechanism based on that experience. 

It seems weird to do significant clarification work on 2616 and 
basically ignore 2617, given the normative reference to the latter. A 
better option would be to do full clarifications in 2617, including a 
discussion of the not-clarifiable internationalization issues. One 
such clarification is a list of the problems of HTTP Digest in the 
modern world.

This probably should not take "a lot of time"; if it does, it means 
that the clarifications are all the more valuable. HTTP implementers 
who see a lot of work in 2616bis and nothing in 2617 will not 
necessarily come to the conclusion that the IETF wants; it would be 
better to have a 2617bis that says what we want to say.

>2. HTTP Security
>
>Phishing demonstrates that HTTP's present security mechanisms are 
>not adequate to meet some important requirements of the present 
>users of the protocol.  I would be uncomfortable moving HTTP from 
>Draft Standard to Standard given this situation.  It's likely that 
>new work on HTTP security mechanisms (as outlined by 
>draft-hartman-webauth-phishing) is necessary.  However, even with 
>the present security situation, I have no doubt that RFC 2616 is 
>widely useful and improving the technical clarity of the base 
>specification is good work that would benefit the Internet 
>community.  The minimum work necessary to make a draft standard 
>revision of the base specification complete would be to clearly 
>document the limitations of the presently deployed HTTP security 
>mechanisms and the fact they are not adequate for all situations.

Agree, but...

>Beyond that I consider it inappropriate to hold publication of a 
>useful revision hostage to new security engineering work.  That 
>opinion may not be shared by others on the IESG.

Knowing ahead of time whether or not the work of this proposed WG is 
likely to get smacked down at the end by the IESG would greatly 
affect the people working on HTTPbis.

>Regardless, I would very much like to see forward progress on the 
>HTTP security situation.

draft-hartman-webauth-phishing generated no significant follow-on 
discussion that I can see (I would be happy to be mistaken). There 
are little bits of discussion here and there, but no momentum. 
Without a strong push from the Apps area for this work, I suspect 
that it will not happen or, if it does happen in a limited fashion, 
the results will not be widely adopted in implementations.

>security as the people who have appropriate expertise for those 
>efforts are not identical. Indeed I'd be uncomfortable with a single 
>WG that was both revising 2616 and designing new HTTP security 
>mechanisms as the latter may be helped by the attention of security 
>experts that likely have no interest in the former.

Fair enough.

>4. Specification Rewrite
>
>Because the IETF process gives quite a bit of control to the 
>document editor and design teams, our process allows an alternate 
>editor to produce a competing specification and ask for a WG 
>consensus call to adopt that competing specification.  This is 
>discussed in the following IESG Note:
>be adopted by the WG in this case, especially because it might drop 
>the target status from draft to proposed for the reasons Keith 
>mentioned.  However, this is an important mechanism the keep the 
>process open.

The status of the new document is *much* less important than its 
correctness and usability to HTTP implementers.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 07 Jun 2007 18:01:13 +0200

Paul Hoffman wrote:
>> existing HTTP Digest mechanism based on that experience. However, 
>> documenting the i18n behavior of deployed implementations sounds like 
>> a sensible thing to do.
> 
> discussion of the not-clarifiable internationalization issues. One such 
> clarification is a list of the problems of HTTP Digest in the modern world.
> 
> This probably should not take "a lot of time"; if it does, it means that 
> the clarifications are all the more valuable. HTTP implementers who see 
> a lot of work in 2616bis and nothing in 2617 will not necessarily come 
> to the conclusion that the IETF wants; it would be better to have a 
> 2617bis that says what we want to say.
> ...

Hi,

maybe things become clearer if we consider re-organizing the security stuff?

Currently,

- RFC2616 refers (normatively?) to RFC2617 for authentication, and

- RFC2617 defines a framework (Section 1.2) and two schemes (Basic and 
Digest).

Assuming that there's no immediate need to change the framework defines 
in RCF2617, Section 1.2, wouldn't it make sense to:

- Move the authentication framework itself into RFC2616bis, and

- to then publish stand-alone documents upgrading/fixing both Basic and 
Digest?

The benefits being:

- RFC2616bis doesn't have the dependency on its sister spec anymore, 
which suffers from Basic and Digest problems, and

- Basic, Digest and new schemes could evolve independently.

Best regards, Julian

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Jun 2007 12:09:39 -0400

> better to have a 2617bis that says what we want to say.
2617 doesn't need clarification, it needs to be deprecated and replaced
with not only different schemes but an entirely different framework. 
I18N is the least of its problems.  Maybe it would be useful to have an
informational document that says what's wrong with 2617 and suggests how
to fix the parts that are fixable, for the sake of sites that continue
to use it, but I don't think such work should be critical path for
either htttpbis or httpsec.
> likely to get smacked down at the end by the IESG would greatly affect
> the people working on HTTPbis.
by "at the end" do you mean at RFC publication time, or charter time? 
If the latter, I agree; if the former, I think that's how the process is
supposed to work.   but basically IESG understands that HTTP is
important and that it needs maintenance AND good security, and they're
also going to understand that security work needs to happen in a
separate group, so I think they're going to want to approve charters
that further these ends rather than smack them down .
> correctness and usability to HTTP implementers.
mostly agree, though it would be confusing to "outsiders" to have 2616
be at draft standard and a new, rewritten specification be at
proposed.   as for "correctness" that's fairly arbitrary.  does it mean:
the document that IETF says is correct (say via an applicability
statement), the document that best reflects the "intent" of the protocol
designers, the document that describes what works best in practice with
today's (perhaps not tomorrow's) browsers and servers, or what?

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Jun 2007 12:10:46 -0400

no.  deprecate 2617.  deprecate the framework that is in 2616.  HTTP
security needs a clean slate approach.
> Assuming that there's no immediate need to change the framework
> defines in RCF2617, Section 1.2, wouldn't it make sense to:
>
> - to then publish stand-alone documents upgrading/fixing both Basic
> and Digest?
>
> The benefits being:
>

---

From: Julian Reschke <[email-masked]>
Date: Thu, 07 Jun 2007 18:13:38 +0200

> the people working on HTTPbis.

Definitively.

It seems to me that the requirements to apply errata and clarifications 
to an existing specification (with wide deployment) should be completely 
different from those for new protocols. If they aren't, existing specs 
just won't get revised, because it's either too much work, irrelevant 
(considering running code), or impossible (backwards compatibility).

>> Regardless, I would very much like to see forward progress on the HTTP 
>> security situation.
> 
> discussion that I can see (I would be happy to be mistaken). There are 
> little bits of discussion here and there, but no momentum. Without a 
> strong push from the Apps area for this work, I suspect that it will not 
> happen or, if it does happen in a limited fashion, the results will not 
> be widely adopted in implementations.

I believe that improvements in HTTP authentication require collaboration 
between implementors, namely UAs (Firefox, IE) and servers (httpd, IIS). 
We need to make that happen somehow.

>> adopt that competing specification.  This is discussed in the 
>> following IESG Note:

Correct. Almost nobody cares. Most people don't even understand the 
difference between Informational, Experimental, and Standards Track 
(sometimes I don't as well...).

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 07 Jun 2007 18:18:13 +0200

> security needs a clean slate approach.

I personally have no problem with this. In the wild, most authentication 
isn't using RFC2617 anyway.

However, my understanding is that the IESG doesn't allow RFC2616bis not 
to discuss authentication in *some* manner.

BTW: does the framework really require fixing?

Best regards, Julian

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 7 Jun 2007 10:35:03 -0700

>with not only different schemes but an entirely different framework.

We need to deal with the real world. In the real world, Basic and 
Digest Auth are used. In the real world, the better replacement for 
them is not deployed. It is fine for us to say "please stop doing 
that, use this instead", but it is myopic and unhelpful to deprecate 
something that is in widespread use.

---

From: "Justin Erenkrantz" <[email-masked]>
Date: Thu, 7 Jun 2007 10:57:23 -0700

> something that is in widespread use.

Right - the IETF can wave a magic wand here, but it won't help as
deploying new versions of the servers and the clients take years.

Furthermore, my understanding is that IESG now requires all new
protocols to always be secure.  I think that *mandating* that we use
SSL (or some similar connection-oriented security mechanism) for *all*
Web traffic is going to kill everyone.  As long as authentication
remains optional, I'm okay - but if it's mandatory or required to be
the default behavior, I very likely won't support implementation of
such a short-sighted standard.  -- justin

---

From: Paul Leach <[email-masked]>
Date: Thu, 7 Jun 2007 11:03:59 -0700

For a long time, the IESG has required that all new protocols have a
"security considerations" section. I have not heard that that has
changed to a more stringent mandate. For many protocols, including HTTP,
that section would have to show that they are securable. However, in
addition, IMO it is obvious that for HTTP, that section also says that
anonymous clients and unauthenticated servers are OK in many
circumstances, and here are the mechanisms that can be used when it
isn't OK.

-----Original Message-----
From: Justin Erenkrantz
Sent: Thursday, June 07, 2007 1:57 PM

<snip>

Furthermore, my understanding is that IESG now requires all new
protocols to always be secure.

---

From: Eliot Lear <[email-masked]>
Date: Thu, 07 Jun 2007 18:52:56 +0000

Paul Hoffman wrote:
> strong push from the Apps area for this work, I suspect that it will 
> not happen or, if it does happen in a limited fashion, the results 
> will not be widely adopted in implementations.

I am forced to agree (sadly).  We all need a good kick in the pants on 
this one.  Sam has put together what I think is a fairly provocative 
requirements document (he provoked me to make a comment and a 
contribution or two ;-).  Given the lack luster response, I don't think 
even I can support my early desire to see the security considerations of 
HTTP dealt with, and the situation is truly abysmal.  And so I think we 
need to have two groups, and it's not even clear that we have enough 
support for the 2nd, right now.

I'm CC'ing Sam, by the way, who can perhaps more accurately respond to 
what comments he's gotten.

Eliot

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 07 Jun 2007 23:13:14 +0200

ons 2007-06-06 klockan 15:42 -0700 skrev Chris Newman:

> by draft-hartman-webauth-phishing) is necessary.

Just a reflection on the phishing problem.

IMHO this is more of an UA and education problem, not so much a protocol
problem even if having something more secure than Digest would be a good
thing. But you should also be aware that making HTTP authentication
stronger won't make any of the common forms of phishing much harder.

The most common form of phishing is for collecting non-HTTP details
about the user such as credit card details or bank accounts, or for
phishing of the users login details to some financial service (bank or
similar), but such services tend to stop using static passwords once
hit..

There is imho three primary reasons why Digest has not gained much
foothold, neither being security.

a) Implementation rather complex, with few if any implementation getting
it entirely right..

b) It's inability to integrate well with existing authentication
frameworks on the server side.

c) UA integration, or web site owners requiring control over the login
process beyond a standard "login+password" dialog.

Then there is also the single-sign-on issue, but thats more of an
implementation thing than protocol. Digest fits just as fine in
single-sign-on models as the NTLM or Negotiate schemes widely deployed
for the purpose today, but due to it being a different authentication
mechanism than used for the desktop it's not used in that context.

Even if HTTP had the strongest authentication possible, fulfilling the
goals of draft-hartman-webauth-phishing etc, phishing would still be
equally possible by the exact same means used today. Or as long as the
primary key to the authentication process is a some piece of static
information provided by the user.

The phishing attacks is a soscial enginering attack on the weakness of
any static shared secret authentication mechanism. Works for login
+password, works for credit card details, works for bank account
details, works for very many forms of identity theft based on the user
providing any form of static secret.

Also in coming up with a usable secure authentication scheme to replace
Digest it's important to not underestimate the integration into existing
systems.

Equally important is the complexity of implementation. Very few if any
(certainly none of the commonly used browsers) implement Digest
correctly even if the ambiguous or weak parts of the specification is
taken aside.

Also it's worth noting that TLS + Digest already fulfills most of the
requirements of draft-hartman-webauth-phishing. There is some like the
enrollment criteria it do not address however.

> However, even with the 
> would benefit the Internet community.

It certainly is. RFC2616 provides the message protocol basics and
anonymous access to content, with hooks to hook in pretty much any
message authentication scheme.

> The minimum work necessary to make a 
> engineering work.

Good.

> the people who have appropriate expertise for those efforts are not identical.

The people revising RFC2616 and 2617 to clarify the specifications and
prune out ambiguities or other specification errors could well be the
same group, but the group coming up with a secure message authentication
scheme beyond digest needs a quite different group of people and is not
really HTTP related. Such scheme is needed all over the computing
industry, covering pretty much all protocols not just HTTP. The problems
in coming up with such scheme is by no means HTTP specific.

> attention of security experts that likely have no interest in the former.

Exactly.

> is an important mechanism the keep the process open.

I would probably not be very comfortable with an largely rewritten
alternate specification within the target of specifying HTTP/1.1. If the
goal was to make a revision of HTTP and not a revision of RFC2616 then
yes, or even a requirement for such work. But I do not rule it out
entirely as there is many things about the structure of RFC2616 which
could be done better to not confuse new readers so much..

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 07 Jun 2007 23:26:57 +0200

> BTW: does the framework really require fixing?

Not really imho. It's an extensible negotiable framework for message
based authentication with no dependencies on how such authentication is
implemented besides it being message based to fit in the message
structure of HTTP.

But it might need to be complemented with a framework for message
integrity however. Digest touches this, but not sure having message
integrity at the authentication level is right..

---

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 07 Jun 2007 23:36:13 +0200

> deploying new versions of the servers and the clients take years.

My rough estimate for getting a secure HTTP authentication scheme widely
deployed is about 15 years, assuming that it takes no more than 5 to get
it to draft standard. This assuming no major revisions of HTTP is needed
along the way in which case add another 5 to 10 years..

Regards
Henrik

---

From: [email-masked]
Date: Thu, 7 Jun 2007 23:14:11 +0000

Henrik,

> process beyond a standard "login+password" dialog.
I think that you got that perfectly right.
My opinion is, that a) could be improved by a strictly editorial revision
of RFC2617 while c) requires substantial revision of the contents.
I do not consider b) to be a problem of the spec but of the implementors.
For me it seems that the real obstacle for RFC2617's popularity is c). 

People just seem to insist on having their authentication procedure in the 
"look-and-feel" they want it to be and concrete-cast it into their
HTML/flash/whatever-content (even though that may seem to be a silly idea
from a usability point of view).

Maybe a solution just like rfc2388 could be suitable here.
Needed is a well-defined way to interact between the application
and the protocol -- the only difference here is, that the auth stuff
needs to interact with the HTTP header, while the form submission stuff
needed/needs to interact with the HTTP POST body.
I guess that a similar "bridge" could be a solution for the problem.

Of course that would need interaction with HTML, e.g. it would need
the introduction of 
<input type="authuser" /> together with <input type="authpass" />
or the like, which are defined to have their contents be passed on
to a well-defined HTTP-Auth scheme.

It could even be done without the need to (substantially) change the
HTML spec, e.g. by reserving special form element ids for that purpose,
lets say
<input type="text" id="authuser" />
<input type="password" id="authpass" />
or the like.

Having that said I would tend to use a double tracked approach:
- editorial revision of rfc2617 to make existing impls
  or new impls that have to interact with them happy
- simultaneously develop a "completely new" scheme that does
  not have the fundamental deficiency of being unable to interact
  with the "application layer"

Kind regards

Ingo Struck

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Jun 2007 18:11:53 -0400

Julian Reschke wrote:
> I personally have no problem with this. In the wild, most
> authentication isn't using RFC2617 anyway.
>
> However, my understanding is that the IESG doesn't allow RFC2616bis
> not to discuss authentication in *some* manner.
I'm certain that there will have to be a good answer to the
authentication question before 2616bis will be allowed to get any kind
of standardization status.  (it could probably be in a separate document).
> BTW: does the framework really require fixing?
I am pretty sure that it does.  I think sites will continue to insist on
being in control of the look and feel of the username/password dialog. 
I also think that the phishing concerns have to be dealt with.  The two
of these together make for an interesting set of constraints.

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Jun 2007 18:12:59 -0400

> something that is in widespread use.
basic and digest are used, but not widely.  deprecate means "please stop
doing that, use this instead".

Keith

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Jun 2007 18:23:26 -0400

1. "A Proposed Standard should have no known technical omissions with
respect to the requirements placed upon it."

2. Draft and full Standard documents must also meet the requirements for
Proposed Standard.

3. "Security" is generally accepted as a requirement for all Internet
standards-track protocols, but this is rather vague as the meaning of
"security" varies from one protocol and use case to another.  In the
case of HTTP there is clearly a need for clients to be able to
authenticate to servers, servers to be authenticate to clients, and for
there to be a means to assure the secrecy of data passed between servers
and clients.

> changed to a more stringent mandate. For many protocols, including
> HTTP, that section would have to show that they are securable.
> However, in addition, IMO it is obvious that for HTTP, that section
> also says that anonymous clients and unauthenticated servers are OK
> in many circumstances, and here are the mechanisms that can be used
> when it isn't OK.

---

From: Keith Moore <[email-masked]>
Date: Thu, 07 Jun 2007 18:26:42 -0400

>  I think that *mandating* that we use
> Web traffic is going to kill everyone.  
IETF has no means of doing that anyway.  The most we can do is make
recommendations.  So when we use the word "deprecate" it just means we
are withdrawing our previous recommendation to use something.  We don't
pretend we can stop users from doing something that make sense to them,
or even that we can stop users from doing things that make no sense at
all. 

Keith

---

From: Lisa Dusseault <[email-masked]>
Date: Thu, 7 Jun 2007 16:16:20 -0700

On Jun 7, 2007, at 11:03 AM, Paul Leach wrote:

> changed to a more stringent mandate.

There's a little more, mostly in RFC3552, e.g. "Unprotected (plaintext)
    username/password systems are not acceptable in IETF standards."

> For many protocols, including HTTP,

+1

Lisa

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 8 Jun 2007 10:10:32 +0200

On Thu, Jun 07, 2007 at 06:18:13PM +0200,
 Julian Reschke <[email-masked]> wrote 
 a message of 14 lines which said:

> In the wild, most authentication isn't using RFC2617 anyway.

Any data here? IMHO, this assertion is not true, unless you limit to
big e-commerce Web sites. For instance, HTTP-based Web services use
2617. Also, 2617 is typically the simplest way for a small and rapidly
setup Web site, even if it does not have the visibility of Amazon.

---

From: [email-masked]
Date: Fri, 8 Jun 2007 09:34:04 +0000

> setup Web site, even if it does not have the visibility of Amazon.
Apart from that there is an applications where rfc2617
imho currently is the only widely usable auth scheme:
restricted proxies.
If you want to have a semi-public proxy that needs auth,
anything else but using rfc2617 Proxy-Authentication
is a pain. If you do not want plaintext credentials, rfc2617 digest
currently remains the only working option (at least for me, but admittedly
this doesn't say anything about "widespread-use").

Kind regards

Ingo Struck

---

From: Yves Lafon <[email-masked]>
Date: Fri, 8 Jun 2007 05:56:55 -0400 (EDT)

> along the way in which case add another 5 to 10 years..

Many sites moved away from HTTP authentication, not so much because of the 
technical aspects of basic or digest, but mostly because there was no good 
UI in browser. Of course browsers are not the only consumers of HTTP, far 
from that, but when defining a new authentication scheme, having a way to 
present it nicely in browsers would be part of the adoption path.

Henrik's timeline is describing the positive outcome of creating a new 
HTTP auth scheme.

-- 
Baroula que barouleras, au tiéu toujou t'entourneras.

         ~~Yves

---

From: Joe Orton <[email-masked]>
Date: Fri, 8 Jun 2007 11:38:14 +0100

> being in control of the look and feel of the username/password dialog. 

Fixing that does not require any changes to the HTTP auth framework. Roy 
pointed out a long time ago that this can be done simply by defining an 
extension to HTML which allows an HTML 401/407 response body to contain 
a form which is used to enter credentials.  <form action="authenticate"> 
or something.

(and possibly some method for browsers to advertise support for this)

joe

---

From: Eliot Lear <[email-masked]>
Date: Fri, 08 Jun 2007 14:46:24 +0000

> stronger won't make any of the common forms of phishing much harder.

I disagree in the strongest possible terms.  This *is* a problem we can 
solve technically.  It's just that no one has the will to do it and 
we've organized ourselves so that the work cannot be done in one place.  
If you had a component that was separate from your workstation, that had 
but a single function – authentication – we could write appropriate APIs 
and protocols to access that device such that you would never log in 
without using it.  The riskiest functions would be registration.  In 
that single area would I view this an education problem, but even there, 
if we came up with a standard way to legitimately register individuals 
we could probably make that problem much more solvable.

The problem is this:

    * Registration and authentication occur with the forms interface
      that W3C handles;
    * The APIs are owned in large part by Microsoft and IEEE (POSIX);
    * IETF owns the wire protocol
    * Smartcard design is done by numerous (ISO, IEEE, other)

But to not attempt to solve these problems is dereliction of duty to the 
community we as an organization are supposed to be serving.  The LEAST 
the IETF can do is put forth an authentication mechanism that solves the 
wire protocol problem.  It should jive with the other functions as they 
evolve and provide flexibility to the organizations in question to offer 
opaque communications so we can have better authentication mechanisms as 
time goes on.

It's just shameful.  And yes, I suppose I'm being a bit emotive, but we 
have GOT to get off the dime, and the ONLY work that does so in this 
space right now is Sam's draft and that of Leif Johannson.

> mechanism than used for the desktop it's not used in that context.

I disagree with you on this as well, but then the term "single sign-on" 
is so overloaded we really can't argue the point without debating the 
term first.  So I'll define it as only requiring one password to do 
whatever it is I want to do (what the DIX/WAE BoFs called "Eliot's Dad's 
Problem").

But I also think there's no use in me whining about the lack of this 
stuff, and so I suppose it's time to shut up and either write a draft 
that actually attempts to address Sam's concerns or build some code to 
match some existing drafts, and then we can see how far off we are.

Eliot

---

From: Chris Newman <[email-masked]>
Date: Fri, 08 Jun 2007 11:42:54 -0700

Paul Hoffman wrote on 6/7/07 8:44 -0700:
> Knowing ahead of time whether or not the work of this proposed WG is likely
> to get smacked down at the end by the IESG would greatly affect the people
> working on HTTPbis.

Agreed.  The mechanism to get the knowledge in advance is to be specific in the 
charter about what will and what won't be done.

                - Chris

---

From: Chris Newman <[email-masked]>
Date: Fri, 08 Jun 2007 12:00:26 -0700

Julian Reschke wrote on 6/7/07 18:01 +0200:
> Assuming that there's no immediate need to change the framework defines in
> RCF2617, Section 1.2, wouldn't it make sense to:
>
> - RFC2616bis doesn't have the dependency on its sister spec anymore, which
> suffers from Basic and Digest problems, and
>

Sounds like an idea worth considering to me.  In past cases where Apps has 
bundled authentication mechanisms with general frameworks (e.g. RFC 1731, 
2595), the mechanisms have invariably been split away from the framework for 
one reason or another.

                - Chris

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 09 Jun 2007 00:34:19 +0200

fre 2007-06-08 klockan 15:45 +0100 skrev Eliot Lear:

> without using it.

Note: My comment was regarding seeing stronger authentication in HTTP as
a method to solve phishing. It is not.

Stronger authentication in HTTP is means to secure the wire protocol for
exchanging user credentials, nothing else.

Regarding stronger authentication in general. Sure, and I fully agree.
And what you say is already supported by using smartcards and TLS for
authentication, and is used by quite many today. Here in Sweden I can
get identity cards with a embedded trusted certificate, and use this to
identify myself to the tax authorities, social services etc. (not yet to
the banks who issue these certificates, but that's a completely
different discussion in administrative policies and not wire
protocols..)

But if you read the current discussions regarding phishing it's claimed
that having a hard token requirement is not acceptable and that the
solution must work with just a login+password.

What I am saing is just the simple fact that designing a protocol to
prevent phishing of static secrets of any kind is rather fruitless as
the phishing problem as such is really mainly is an UA and education
problem, not so much a wire protocol problem. As long as the user is
capable of giving out the information and don't need to think twice in
doing so phishing will continue, no matter how strong wire protocols is
designed for exchanging these user known secrets with trusted parties.

> The riskiest functions would be registration.  In 

And we already have that in TLS, with appointed trusted CAs and where
each service provider selects what level of trust he appoints the
different CAs in verifying the user identities.

> whatever it is I want to do (what the DIX/WAE BoFs called "Eliot's Dad's 

In what way do not Digest fit that? (putting aside the security concerns
regarding Digest use of MD5 and how)

What is missing for Digest is some standard means for esablishing the
password without exchaning the password, but it's possible to do such
exchange, at least to a reasonable level.

> match some existing drafts, and then we can see how far off we are.

It's fully possible to come up with an replacement for Digest solving
the wire concerns.

The relevance to RFC2616 in such work is the need for some standard way
of establishing the password in a secure manner. The rest is relevant to
RFC2617 only.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 09 Jun 2007 02:49:17 +0200

fre 2007-06-08 klockan 05:56 -0400 skrev Yves Lafon:

> UI in browser.

Yes, as has been pointed out a number of times.

>  Of course browsers are not the only consumers of HTTP, far 

I would say the two is two quite separate tasks, quite independent of
each other.

- The presentation problem is needed to get solved to get web site
authors to accept HTTP authentication at all, or they will continue to
insist on using forms based authentication using plain-text login
+password just because the other alternatives doesn't "look right", not
caring about security.

- The scheme is needed to add wire security to the exchange. Digest does
a reasonably good job in protecting the wire exchange, but not quite
good enough and has a few other issues as well.

> Henrik's timeline is describing the positive outcome of creating a new 

Correct. Making changes in this area is a slow process, where the spec
writing is only a small part, comparable to getting vendor support. The
hard part is getting it generally accepted and deployed.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Sat, 09 Jun 2007 03:03:15 +0200

> a form which is used to enter credentials.  <form action="authenticate"> 

Preferably with a new header telling the browsers that the response body
contains not only the error message, but also the form(s) which should
be used in the dialog query for the user credentials.

> (and possibly some method for browsers to advertise support for this)

Don't really see the need for the clients to advertise any support here.
They just need to use the information if they know how, or ignore it
otherwise falling back on the current "ugly" auth dialogs..

Regards
Henrik

---

From: Lisa Dusseault <[email-masked]>
Date: Sun, 10 Jun 2007 14:10:12 -0700

> exchange, at least to a reasonable level.

Digest has a bad reputation particularly among Web App developers for  
a number of reasons, some inherent to the design and specification,  
some stemming from implementation and deployment choices.

http://www.xml.com/pub/a/2003/12/17/dive.html:  "most web hosting  
providers don't turn on digest authentication (it requires an Apache  
module that is not on by default). Even if Bob's ISP had  
mod_digest_auth enabled, it wouldn't help Bob, because he has  
no .htaccess rights to configure his passwords; and, because of the  
way Apache works, CGIs can't implement digest authentication on their  
own. (Scripts handled by an Apache module, such as mod_php or  
mod_perl, can implement HTTP digest authentication. But external CGI  
processes can't because Apache does not pass the necessary headers  
along to the CGI script. But that still doesn't help Bob because his  
hosting provider doesn't offer PHP; and, even if they did, his weblog  
software doesn't run on PHP anyway.)"

http://blogs.msdn.com/drnick/archive/2006/05/12/understanding-http- 
authentication.aspx: "Digest authentication requires the use of  
Windows domain accounts.  The digest realm indicates the Windows  
domain name.  Due to this, a server running on an operating system  
that does not support Windows domains, such as Windows XP Home,  
cannot be used with Digest authentication.  When the client is  
running on an operating system that does not support Windows domains,  
a domain account must be explicitly specified during the  
authentication."

http://www.imc.org/atom-syntax/mail-archive/msg06103.html: " (1) Some  
web-servers remove the WWW-Authenticate header before passing it to a  
CGI program."

http://www.imc.org/atom-protocol/mail-archive/msg00836.html: "do all  
digest and WSSE implementations require server-side access to
clear-text passwords or is that just a weakness of the  
implementations I looked at?"

http://www.imc.org/atom-syntax/mail-archive/msg00139.html:  "I'm a  
small site, security is very much a concern and my host does not  
provide Digest and won't do so."

Thus it's hard for an administrator to use today's Web server  
software and Digest authentication, and still have an application- 
specific database of usernames/passwords.  The server software gets  
in the way -- it may even be easier for the Web App developer to  
implement something non-standard like WSSE than have to rely on built- 
in functions.

i18n is also a problem: http://www.agileprogrammer.com/eightytwenty/ 
archive/2006/05/04/14280.aspx

And for humour on the situation: http://bitworking.org/news/ 
Problems_with_HTTP_Authentication_Interop

Lisa

---

From: Martin Duerst <[email-masked]>
Date: Sun, 10 Jun 2007 17:05:44 +0900

At 23:59 07/05/30, Julian Reschke wrote:
>Paul Hoffman wrote:
>> Serious question, not a gratuitous opening of a big can-o-worms: would the WG also consider extensions to HTTP that would go into different documents? That is, is the WG only for the revision of the base spec, or also open to ( animated && lengthy && contentious ) discussion of other documents as well?
>
>I guess the idea was that the more we restrict the scope of what we want to do, the easier it'll be to gather the right group of people to do it.
>
>For instance, RFC2617 needs a revision badly as well (for instance, wrt to I18N of usernames and passwords, and, as far as I can recall, certain problems with the definition of Digest Auth). IMHO; this should occur in a separate working group.

There are some i18n-related fixes needed in RFC 2616 as well.

The two main ones I know of are:

- RFC 2616 prescribes that headers containing non-ASCII have to use
  either iso-8859-1 or RFC 2047. This is unnecessarily complex and
  not necessarily followed. At the least, new extensions should be
  allowed to specify that UTF-8 is used.
- RFC 2616 prescribes a default of iso-8859-1 for the 'charset'
  parameter for text/* mime types. At least in two very important
  cases, this is not followed: For text/html, there is no default,
  or put in other words, the default is whatever your browser is
  set to. For text/xml and related types, the default is US-ASCII.
  At a miminum, the spec should make sure the implementers get
  an idea of this, rather than having to find out the hard way.

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 11 Jun 2007 05:23:44 +0200

> some stemming from implementation and deployment choices.

Nearly all is implementation.

> module that is not on by default). Even if Bob's ISP had

Implementation.

> http://blogs.msdn.com/drnick/archive/2006/05/12/understanding-http-authentication.aspx: "Digest authentication requires the use of Windows domain accounts.

Plain untrue, unless you restrict your view of Digest to the Microsoft
IIS implementation in which case it's implementation.

> web-servers remove the WWW-Authenticate header before passing it to a

Implementation. Well, CGI is not the proper interface to implement
authentication schemes, but it's implementation in the sense that web
servers is a bit poor in allowing applications to set the authentication
requirements in a sensible manner. But it is fully doable with only a
little effort.

> clear-text passwords or is that just a weakness of the implementations
> I looked at?"

No, the realm specific H(A1) is required.

> provide Digest and won't do so." 

Implementation.

> Thus it's hard for an administrator to use today's Web server software
> and Digest authentication, and still have an application-specific
> database of usernames/passwords.  The server software gets in the way
> -- it may even be easier for the Web App developer to implement
> something non-standard like WSSE than have to rely on built-in
> functions.

Thats all implementation. Web servers don't make it easy for
applications to use/control HTTP authentication. So applications don't
use it.

CGI is not the proper interface for HTTP authentication. It's the web
servers responsibility to handle the fine details of HTTP (including
authentication) and the CGIs responsibility to provide content &
applications.

> i18n is also a problem:
> http://www.agileprogrammer.com/eightytwenty/archive/2006/05/04/14280.aspx

Yes, this is a specifications problem inherent to both Basic and Digest,

> And for humour on the situation:
> http://bitworking.org/news/Problems_with_HTTP_Authentication_Interop

Yes, there is lots of broken Digest implementations around either not
reading the specs, not caring to implement anything but the absolute
minimum required for a conditionally compliant implementation, not
testing their implementation, or sticking to old specs considered
obsolete and broken. And pressure from users that things must work even
with the broken implementations as the users consider the broken browser
implementations unfixable. Don't know how many times I have asked users
to file a bug report with vendor X and always receive the same answer
"no, it's of no use. we must work around the bug somehow".

Regards
Henrik

---

From: Keith Moore <[email-masked]>
Date: Sun, 10 Jun 2007 23:33:03 -0400

> Nearly all is implementation.
ah, but what's the reason for all of those implementation-imposed
constraints?

Keith

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2007 05:56:54 +0200

>   allowed to specify that UTF-8 is used.

We have talked about this, but it's currently not on the issues list. I 
think it should be.

>   an idea of this, rather than having to find out the hard way.

Agreed. That issue is 
<http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/issues/#i20>.

Best regards, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 11 Jun 2007 06:17:52 +0200

> ah, but what's the reason for all of those implementation-imposed
[in Digest]

Lack of a test suite allowing implementers to verify their
implementation?

Lack of general interest in having a reasonably secure authentication
mechanism?

Web authors considering look & feel much more important than security,
and not willing to ask for the ability to have both as forms + cookies
accomplishes their goal of getting the look & feel they want?

Digest being different than the other authentication mechanisms, and
therefore a bit of a pain to integrate into existing systems, requiring
a different password store or alternatively access to plaintext?  (a
problem shared with all secure authentication methods)

If not that I am not sure.. the most common implementation bugs are

  - Use of the wrong request method on non-GET requests

  - Omitting the query parameters from the Request-URI

  - Random client nonce-count sequences, including repeated use of the
same nonce-count.

  - md5-sess often broken, for example using wrong nonces.

  - Incorrect escaping (partly a specification issue).

And the most common shortcomings are

  - Not implementing qop at all (i.e. obsolete RFC2069 level of
implementation)

  - Not implementing nonce-count at all, requesting a new challenge from
the server on each request.

  - Not implementing -int qop.

And of HTTP authentication in general:

  - Web authors not given any influence on the user interface for the
login step.

  - Lack of any server controlled session control. Logout or idle
timeouts.

Regards
Henrik

---

From: Keith Moore <[email-masked]>
Date: Mon, 11 Jun 2007 03:16:07 -0400

> accomplishes their goal of getting the look & feel they want?
it's not immediately clear to me that forms+cookies+TLS, when
well-implemented, are worse than digest.  of course, there are lots of
potential pitfalls there.
> a different password store or alternatively access to plaintext? 
I suspect that the tendency to want to share authentication databases
between HTTP and other applications puts any HTTP-specific mechanism at
a disadvantage.

Keith

---

From: Jamie Lokier <[email-masked]>
Date: Mon, 11 Jun 2007 10:38:21 +0100

> accomplishes their goal of getting the look & feel they want?

A few notes on what I think forms + cookies offers that RFC2617 doesn't.

    1. The login cookie (or its validity) can be expired under control
       of the server.  That is, the site can offer a "logout now"
       button, options for "logout at end of session" and "keep me
       logged in between sessions" - and also control the idle expiry
       time.

       This has become more important with the spread of public
       terminals, vs. the convenience of remaining logged in on
       private terminals.

       I've noticed my bank site will expire login credentials after
       just 10 minutes, while other sites which are not so security
       conscious take much longer.  It is difficult for my browser to
       remember my security credentials - the site uses changing form
       field names - so I have to reenter them.  They both seem like
       a good idea to me.

    2. Related to above, different types of access can be maintained
       simultaneously for the same pages.  For example, the server may
       remember who you are between sessions for personalisation - but
       for critical operations require re-authentication between. I
       think Ebay does this - you can remain logged in between
       sessions for convenient browsing, but placing a bid requires a
       password confirmation each session.

    3. Password transfer for authentication can be done over SSL/TLS,
       even if the accessed pages after that are plaintext with the
       cookie as authenticator, for performance or caching reasons.  I
       think in some ways this is stronger than digest authentication,
       and in some ways it is weaker.

    4. Related to 3, a clever site could run client-side ECMAscript to
       apply a crypto operation to the authentication cookie on every
       page fetch, so the shared secret is not revealed on the wire
       and is immune to replay attacks, combining the advantages of
       both methods.  (I'm not aware of any site doing this, but it
       seems plausible, for browser-accessed sites only of course).

    5. Forms + cookies can be implemented by any server-side page
       rendering scheme, e.g. PHP, Perl, CGI, etc. which handles forms
       + cookies for other purposes.  In other words, it's accessible
       to actual site authors, who often can't reconfigure their web
       server but can write those scripts.  Yes, this is an
       implementation detail, but there's practical reasons why it's
       such as _very_ common implementation detail.

> problem shared with all secure authentication methods)

Forms + cookies authentication seems to integrate quite conveniently
into pretty much all existing systems for serving the rest of the page.
Naturally.

If there's a serious wish to define a HTTP authentication scheme which
is better enough that everyone wants to adopt it, I think it's
worth looking into how to improve "forms + cookies" with schemes that
offer all the capabilities of that, but are properly scrutinised as
security schemes.

A synthesis of the best of RFC2617-style, and forms + cookies for
security ought to:

    1. Be better in every respect - security and user interface, than
       forms + cookies _and_ RFC2617 at the same time.  Solicit buy-in
       from big users of forms + cookies at present, in order to
       properly understand and address the reasons they have for using that.

    2. Resemble RFC2617 type schemes in the sense of being accessible
       to non-browser clients, and non-HTML uses.  This surely means
       combining some of the _behaviour_ of cookie schemes, with
       transmission of authentication information over HTTP headers.

    3. Retaining the server control of behaviour that forms + cookies
       offers.  Ability to force "logout now" - not merely request, by
       declaring an authentication token no longer valid.  Ability to
       retain different levels of authentication simultaneously for
       different aspects of a site.  Ability to request that a client
       forgets the user's password.  Time-based expiration.

    4. Describing hooks whereby the UI elements can appear embedded in
       a related HTML page, but aren't required to be.

    5. Ensure that it's possible that site authors who write in
       e.g. PHP, Perl, CGI etc. can easily make use of the scheme.
       Simply because this is an important requirement for practical
       deployment.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2007 09:30:00 +0200

> potential pitfalls there.

You can't do forms with non browser-based applications. WebDAV and APP 
come to mind.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2007 09:32:01 +0200

>   - Web authors not given any influence on the user interface for the

And let's not forgot "logout". A UA should offer a logout button in it's 
chrome when it previously logged in using HTTP auth. And I guess web 
pages need a way to purge authentication information by script (so they 
can keep "logout" as part of the HTML page they send).

Best regards, Julian

---

From: [email-masked]
Date: Mon, 11 Jun 2007 14:41:46 +0000

>        logged in between sessions" - and also control the idle expiry
Again, this is an implementation issue.
Regarding basic auth, this is true -- there is no way to "expire the session".
But using digest you can implement full-fledged session control, because
the server has got the "nonce" values under control and can store
a sessionid in it.
There is a proof of concept for this available at http://auth.ingostruck.de
you can register, login, logout and the session expires, all using plain 
rfc2617 digest auth.

As for "keep me logged in between sessions" I would tend to say that this
is one of the major security risks and would neither incorporate it into a 
potential rfc2617 successor nor recommend to implement it (and it could still 
be done using cookies for non-sensitive parts of a site).

>        cookie as authenticator, for performance or caching reasons.
There is no need for that with digest *and* you do not even need
SSL/TLS for auth purposes.
The only part that needs SSL/TLS is registration (exchange new credentials
between server and client), but this is expressly outside of rfc2617's scope.

Current UA impls do not send auth across schemes (http/https), which makes 
perfectly sense for basic, but for digest this could be done without major
changes to rfc2617, because the scheme is not involved in the hash calculus.

>        seems plausible, for browser-accessed sites only of course).
Replay attacks could already be avoided using digest auth,
if nc would be implemented correctly by UAs. Even with broken ua
impls who mess up nc, a server can implement one-shot nonces at the
cost of another round-trip using the "stale" mechanism
(within my impl a nonce becomes stale after 10 requests and after
every single POST request).

>        such as _very_ common implementation detail.
This one seems to be the crux of the matter, but the 
Authorization, Authenticate and Proxy-Authenticate headers are available
to all those server-side-rendering schemes too (at least through CGI,
via HTTP_AUTHORIZATION etc.).

>        declaring an authentication token no longer valid.
>
>        Time-based expiration. 
Could already be done, just like with cookies.
Maybe an explanation *how* this could be done should be added.

>        Ability to 
>        different aspects of a site.
If different "realms" are properly implemented, this could be done, too.

>        Ability to request that a client 
>        forgets the user's password.  
This is part of "exchange new credentials between server and client"
which has been left out of rfc2617 by purpose.
Maybe this is one of the major deficiencies there (the i18n problem
for passwords is a result of this too).

>        Simply because this is an important requirement for practical
These two seem to be the crucial points to me: 
- hook up to the "application layer" in a well-defined way
  within the spec (like I already wrote in a mail to the wg list)
- create working reference a/o example impls for major web servers

Kind regards

Ingo Struck

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 11 Jun 2007 16:27:54 +0200

> potential pitfalls there.

Issues:
- Server knows the plaintext password
- Client need to support cookies
- Client need to support forms

The first is a security issue, as the user probably is reusing the same
password to multiple services.

The second two is interop issues. The use of forms authentication
assumes there will be an visible user agent and a user manually filling
in the form. This is not available in all clients.

> between HTTP and other applications puts any HTTP-specific mechanism at

Well, it puts any secure authentication scheme different than the
primary authentication scheme of the authentication database used at a
disadvantage, no matter how it's implemented.

Where secure in this context is that the scheme is designed in such
manner that the plain-text password is not exchanged (in plain, or
reversibly encoded / encrypted)

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 11 Jun 2007 23:04:08 +0200

> SSL/TLS for auth purposes.

And more, with Digest you don't need to worry about session theft when
not using SSL/TLS. Unlike cookies digest has built-in replay protection.
Now not every implementation make use of it (partly due to the
implementation bugs mentioned earlier), but it's there.

> changes to rfc2617, because the scheme is not involved in the hash calculus.

Digest has the domain response paramater to control this. Have not
looked into how well supported this is by the current UAs, but it's
there in the specifications.

> via HTTP_AUTHORIZATION etc.).

Some servers filter the Authorization header in CGI for security
reasons, not trusting the CGI with user credentials.

> >        a related HTML page, but aren't required to be.

Yes, and has been on the agenda since at least 1999 as already discussed
here. And it's not a technically hard thing to do, just someone need to
do it and get it accepted, and when done it will significantly shift the
tide in favor for HTTP authentication compared to cookies.

> >        Simply because this is an important requirement for practical

This falls firmly into the land of server configuration. Auth schemes is
best implemented by the server, not each application running on the
server. 

In short applications (using PHP, Perl, CGI etc) need to be able to
configure the protection realms of the server in a consistent manner.
One example is using Apache .htaccess which works quite well for the
purpose (assuming the server admin allows authentication to be used).

The primary thing missing is some well defined way of establishing the
user password without applications having to dive deep into both server
and scheme details..

Note: I do NOT consider it a reasonable task for a potential WG revising
RFC2617 to come up with designs allowing applications to use
authentication on servers where the use of authentication has been
administratively prohibited. Thats an administrative issue, not a
technical one.

Regards
Henrik

---

From: [email-masked]
Date: Mon, 11 Jun 2007 23:15:21 +0000

> implementation bugs mentioned earlier), but it's there.
It's there and it is known how to work around the known implementation
bugs without violating the spec. :-)

> there in the specifications.
I did not try that yet. The spec says
===
 If this directive is omitted or its value is empty, the client should assume
 that the protection space consists of all URIs on the responding server.
====
I would say that this means that for "no value" both
http://foo.com and https://foo.com are the same protection space,
because they inevitably refer to the same server (apart from technical
fancy foods like transparent nat a/o transparent proxying).
However, most UAs consider http://foo.com and https://foo.com to be
different servers.

> reasons, not trusting the CGI with user credentials.
Which is basically a good idea, because the situation where the CGI
scripts really know what they do, is rare.

> tide in favor for HTTP authentication compared to cookies.
If I had some more spare time, then I would have committed to do
that, but I guess that doing it "formally correct" costs lots of
time. However, I posted a coarse draft of how I would put it down:
 
  http://auth.ingostruck.de/20070611-rfc2617-html-bridge.txt

I guess that a spec that fits into rfc requirements does not need far
more words to explain what is needed.

> best implemented by the server, not each application running on the
For sure. The applications only need a simple way to trigger UAs
to use the scheme(s) implemented on the server side.

> purpose (assuming the server admin allows authentication to be used).
.htaccess is ok, but far from being optimal, quite like .htpasswd
And I do not think that this is a thing to be put into a spec.
If a server thinks that it is appropriate to store the
user:realm -> H(A1) mapping in a SQL cluster, then it's a (faulty...) decision
up to the server (and up to the application to store new credentials there).
If the server prefers .htacess/.htpasswd files or a tdb then it's ok too.

> administratively prohibited. Thats an administrative issue, not a
100% agreed. IMHO a WG revising RFC2617 should:
- remove "opaq" (not needed for obvious reasons)
- remove "auth-param" (will never be needed)
- render all fields that are "optional for compatibility reasons"
  mandatory
- move MD5-sess to a separate rfc or drop it (nobody got it right
  and the same could be achieved with MD5)
- make the scheme of the hash calculus mandatory while allowing
  for different hash algorithms (e.g. SHA1, SHA512, RIPEMD-160)
- address the known editorial problems

Kind regards

Ingo Struck

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 12 Jun 2007 00:53:36 +0200

> However, most UAs consider http://foo.com and https://foo.com to be

Most people define server to be port specific. http and https uses
different ports.

>   and the same could be achieved with MD5)

Several got it right. But nearly nobody uses it for various reasons,
where the main one being that very few have authentication backends
capable of providing MD5-sess keying material.

How do you do MD5-sess with MD5 for the target of MD5-sess?

The target of MD5-sess is to allow Digest to operate without requiring
the Digest server to have access to the static H(A1) (somewhat security
sensitive).

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 12 Jun 2007 05:36:56 +0200

> fancy foods like transparent nat a/o transparent proxying).

Do they? Those two URLs refer to different services (ports) on the same
IP.. It's obviously the same IP, but is it the same server?

Yes, it's quite common that the same content is available via both and
that both services is under the same administrative realm or even same
server software instance, but not guaranteed by any means.

Would you also consider http://www.example.com/ and
http://www.example.com:8080/ as the same server?  I don't..  For me the
server is identified by scheme://host:port  (or just host:port, wich
scheme just telling what protocol is run on that port)

Regards
Henrik

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 12 Jun 2007 17:03:17 +1200

with load-balancing and round-robin DNS it's not even necessarily the 
same IP.

---

From: [email-masked]
Date: Tue, 12 Jun 2007 09:18:21 +0000

> Most people define server to be port specific. http and https uses
Ok, maybe another editorial thing to clarify within rfc2617,
but obviously most implementors share that view.

> > - move MD5-sess to a separate rfc or drop it
> Several got it right.
Maybe several server impls, but most UAs that I tested did not.
On the one hand this leads to a performance penalty for the server
side, because the server needs to calculate the correct digest and
then the broken digest if the correct one does not match.
On the other hand it renders what you call the "target of MD5-sess"
defective: the server *needs* to know H(A1) to calculate the broken
digest.

> How do you do MD5-sess with MD5 for the target of MD5-sess?
Only operate with H(A1) all the time. Only calculate H(A1) once
within the UA. At least for mozilla/firefox I know that the UA really
calculates H(A1) for every single request (!), which is simply waste.

> the Digest server to have access to the static H(A1) (somewhat security
No. At least that's not explicated in the spec.
The spec only says that MD5-sess has the following purposes
- allow for "efficient 3rd party authentication servers" (3.2.1)
- prevent that the server needs to know the user's plain text password
  "so that the web server would not need the actual password value" (3.2.2.2)

But it is pointed out, that the server does not need the plain text value
anyway (Note in 3.3, second paragraph).

Of course you are right, that -- if the UAs properly implemented it --
a web server would only need "the session key" passed over from a
"3rd party authentication server".

But the spec says that 
 "The specification of such a protocol is beyond the scope of this
  specification."
so why not just source out the "digest-session" stuff elsewhere
and then specify it properly there?

Kind regards

Ingo

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Jun 2007 21:30:57 +1000

On 08/06/2007, at 6:10 PM, Stephane Bortzmeyer wrote:

> 2617.

My experience is that it isn't adequate for even those purposes, in  
many cases.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 12 Jun 2007 13:56:09 +0200

On Tue, Jun 12, 2007 at 09:30:57PM +1000,
 Mark Nottingham <[email-masked]> wrote 

> My experience is that it isn't adequate for even those purposes, in

More elaborate comments are welcome. Specially if we embark on a
revision of 2617 :-)

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 12 Jun 2007 22:41:18 +0200

> but obviously most implementors share that view.

Agreed, except that would be clarified in 2616 where server is defined..

> defective: the server *needs* to know H(A1) to calculate the broken

Yes, but I don't consider it an argument to drop MD5-sess.

> calculates H(A1) for every single request (!), which is simply waste.

Let me ask the question again: How do you implement the equivalence of
MD5-sess on the server side using just MD5, when the server side account
database do not allow the Digest server direct access to the static
H(A1) hash?

The whole point of MD5-sess is to allow Digest implementations where the
Digest server do not even have access to the static H(A1), only a
session specific hash.  If the server has access to the static H(A1) or
the plaintext password then there is not really much reason to use
MD5-sess.

> anyway (Note in 3.3, second paragraph).

Right. Some of the fine print got lost in the RFC. Should be corrected.

> "3rd party authentication server".

And that is the purpose of MD5-sess.

The difference between MD5 and MD5-sess is that MD5 has a static
password seed, while MD5-sess is using a session unique (per server
nonce) password seed.

>  "The specification of such a protocol is beyond the scope of this

Indeed. It's part of the protocol schemes for authentication servers.

The IETF Digest extension for RADIUS is one such protocol. Unfortunately
"crippled" such that the RADIUS server is only allowed to hand out the
session key if qop -int is used, this because the Radius people do not
agree that it's useful to have a limited session maintained at the web
server based on session-unique MD5 hashed keying material and says every
request should go to the RADIUS server.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 12 Jun 2007 22:54:19 +0200

> with load-balancing and round-robin DNS it's not even necessarily the 

True, but it still has to act as one server, sharing digest nonce
details etc, and should have the same content on all or users will get
quite confused..

Regards
Henrik

---

From: Adrien de Croy <[email-masked]>
Date: Wed, 13 Jun 2007 10:16:27 +1200

my experience also is that it is extremely rare to encounter public web 
servers that use any HTTP auth mechanism.

NTLM and Basic auth is often used for intranets, and proxy access.

I've never seen an instance of Digest auth.

Seems to me that the issue of securing communications and authenticating 
or identifying parties are closely aligned, why not just have some form 
of auth built into TLS, then we could use it for any protocol that can 
use TLS, instead of having to implement separate auth schemes for every 
higher protocol.

---

From: Henrik Nordstrom <[email-masked]>
Date: Wed, 13 Jun 2007 00:56:07 +0200

> I've never seen an instance of Digest auth.

I have set up several servers using Digest auth. There really isn't any
reason not to use Digest over Basic if you anyway have a local password
file.

Yes, some browsers is a bit broken, but most servers know how to work
around them.

Yes, due to the brokenness not all the security features of Digest can
be used (strict replay protection), but it's heaps better than Basic
even without them..

Using TLS is often overkill, and requires much more administration to
get a certificate issued, installed etc.

Regards
Henrik

---

From: [email-masked]
Date: Wed, 13 Jun 2007 09:45:36 +0000

> Yes, but I don't consider it an argument to drop MD5-sess.
Neither do I. Malimplementations sometimes indicate flaws in the spec,
but they are not an argument to drop what has been specified.
I don't want to drop it, I just want to move it out of rfc2617
and then create a proper description of it elsewhere.
The shorter and the more focused a spec, the higher the probability
that people get it right.
My point of having it separated is, that it is far more desirable
to have people at least getting MD5 digest right, than to have a
description of what is possible but rarely ever used.

> > > How do you do MD5-sess with MD5 for the target of MD5-sess?
If we speak of the "true target" (we both agree upon) of having
a digest server ignorant of H(A1), then there clearly is no way
to do it. If you just want to achieve what is the stated target
of MD5-sess outlined in the spec, then MD5 could do the same thing.

> And that is the purpose of MD5-sess.
And the spec should say so or leave session based authentication alone.
If you feel there's need to retain it within rfc2617,
then I would say that a paragraph describing the functionality of
MD5-sess needs to be added.

Kind regards

Ingo

---

From: [email-masked]
Date: Wed, 13 Jun 2007 09:57:42 +0000

On Tuesday 12 June 2007 22:56, Henrik Nordstrom wrote:
> be used (strict replay protection),
Even that could be done at the cost of additional round trips.

> but it's heaps better than Basic even without them..
>
For me this is the principal argument of using digest auth.
a) using TLS needs a cert which costs money and is overkill
   for some applications while on the other hand it is just
   grossly negligent to use basic over unencrypted connections
b) (see my mail from 2007-06-08 to the list), the application of a
   restricted (semi-)public proxy naturally cannot use any sort of
   TLS-auth because it needs to tunnel encrypted connections

Kind regards

Ingo

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 14 Jun 2007 17:00:24 +1000

Since discussion has veered towards "what's wrong with Digest?", a  
few more issues, FWIW;

- Forced timeout: it's important to be able to bound login state by  
time. While you can do this by keeping state on the server, this is  
impractical on massively distributed services (e.g., Yahoo!). While  
the opaque field of Digest can be used to transport a  
cryptographically signed timeout date, client implementation of  
Opaque is spotty AFAIK.

- Dictionary attacks: Because an analogue of the password is on the  
wire, it's possible to make dictionary attacks against poorly chosen  
passwords. Again, this can be mitigated by forcing users to choose  
good passwords, but that is difficult when dealing with large numbers  
of consumers.

- Downgrade attacks: Never mind intermediaries; imagine a pool of  
thousands of servers serving a site which use Digest auth. If one is  
compromised, and the browser has stored credentials, it can perform a  
downgrade attack by requesting Basic authentication.

There's a common thread through these that I don't think has surfaced  
in discussions much yet; the idea that, for some kinds of  
installations, it's very desirable to limit the number of servers  
that need to know the password to a small subset of the total set of  
deployed servers. Doing so makes the risk profile much more  
manageable and assures proper resource utilisation (servers doing  
password verification means that there's less resource for doing what  
they're supposed to), and is commonly seen on large Web sites by use  
of cookies as tokens + a dedicated 'login' hostname.

Again, digest has the potential for cross-domain authentication, but  
it isn't widely implemented, probably because people feel there are  
security issues in allowing it in an unmitigated fashion.

Cookies are also attractive because when they're used, the server  
doesn't need to trust that the implementation performs correctly;  
besides scoping issues (which have been mostly worked out), cookies  
generally either work or they don't. Additionally, it's not clear  
whether more than one credential is allowed at a time (IIRC they  
aren't), which limits its usefulness compared to Cookies.

Cheers,

--
Mark Nottingham       [email-masked]

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Thu, 14 Jun 2007 09:40:19 +0200

On Wed, Jun 13, 2007 at 10:16:27AM +1200,
 Adrien de Croy <[email-masked]> wrote 
 a message of 38 lines which said:

> my experience also is that it is extremely rare to encounter public
> web servers that use any HTTP auth mechanism.

We do not live in the same world, then. I worked today on a program (a
del.icio.us link checker) which use it. Many Web APIs use this
mechanism. 

Unless if, by "Web servers" , you mean only "Web servers for
interactive human usage".

---

From: Keith Moore <[email-masked]>
Date: Thu, 14 Jun 2007 07:13:18 -0400

> TLS can do that but it does not gel with the way in which (many) organisations
> are structured.  Those responsible for security, for security credentials and
> their maintenance, do not want to be ferreting around in the depths of a network
> stack, they prefer working at application and database level, a point that has
> already been alluded to in this thread.

how exactly does sending TLS credentials involve ferreting around in the
depths of a network stack?

---

From: "tom.petch" <[email-masked]>
Date: Thu, 14 Jun 2007 11:01:54 +0000

<inline>
Tom Petch

----- Original Message -----
From: "Adrien de Croy" <[email-masked]>
To: "Mark Nottingham" <[email-masked]>
Cc: "Apps Discuss" <[email-masked]>; <[email-masked]>
Sent: Wednesday, June 13, 2007 12:16 AM
Subject: Re: RFC2616 vs RFC2617, was: Straw-man charter for http-bis

The solution I like to this is channel bindings, as promoted by Nico Williams,
where a weakly authenticated tunnel is set up and then stronger
authentication - which can rely on the strong encryption which is now in place -
is performed via it, eg at application level in the stack.

---

From: "tom.petch" <[email-masked]>
Date: Thu, 14 Jun 2007 18:09:29 +0000

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "tom.petch" <[email-masked]>
Cc: "Adrien de Croy" <[email-masked]>; "Apps Discuss" <[email-masked]>;
<[email-masked]>
Sent: Thursday, June 14, 2007 1:13 PM
Subject: Re: RFC2616 vs RFC2617, was: Straw-man charter for http-bis
> depths of a network stack?

It doesn't:-)  Those responsible for the creation and maintenance of security
credentials - which I see as the major ongoing work of security - prefer to do
at an application level, using appropriate databases, which are
somewhat removed from the lower layers in which TLS sits.  So TLS has a
different set of credentials or none, which is the problem that channel binding
overcomes.

Tom Petch

---

From: Keith Moore <[email-masked]>
Date: Thu, 14 Jun 2007 16:19:20 -0400

> different set of credentials or none, which is the problem that channel binding
maybe what I think of as "application level" is different than how you
think of this term, but I've never heard of a client application that
uses TLS where TLS wasn't being called by the application, and where the
application wasn't in a position to supply credentials via TLS to the
server.

I'm not trying to be picky here.  Rather I think there's probably an
important principle here that needs to be teased out.

Keith

---

From: Adrien de Croy <[email-masked]>
Date: Fri, 15 Jun 2007 09:20:17 +1200

The reason I raised it is that security and authentication / 
identification are 2 concepts that go hand in hand more often than not.

we have a plethora of auth schemes in use on the net - one per 
application protocol.  They sometimes are similar (i.e. SASL ones), but 
often are not.

TLS is a protocol that provides security of communications, and allows 
for mutual identification through certificates.

So it would make sense to solve both problems at the same time, instead 
of one with TLS, and the other with a horde of higher level 
authentication protocols.  Then it gets designed and debugged once, 
instead of once per protocol, with obvious improvements in reliability 
of implementation.

Adrien

---

From: Chris Newman <[email-masked]>
Date: Fri, 15 Jun 2007 12:23:48 -0700

Keith Moore wrote on 6/14/07 16:19 -0400:
>> credentials - which I see as the major ongoing work of security - prefer to
>> do at an application level, using appropriate databases, which are
>> different set of credentials or none, which is the problem that channel
>> binding overcomes.

TLS software stacks are starting to ossify into both operating systems and 
hardware devices.  The TLS stacks are already hideously complex and I'm seeing 
increasing resistance to change in both the providers and consumers of TLS 
software.  If you've ever looked closely at GSSAPI or the CMU SASL API, adding 
a generic authentication API to a TLS software stack would _greatly_ increase 
the API complexity.  So much so that I'd personally choose to avoid using a TLS 
software stack that included such complexity.

While I have no doubt that tightly coupling authentication and the security 
layer produces a more secure system in theory, I'm also skeptical that such a 
service can achieve all the requirements of both while having enough simplicity 
to be sustainable/maintainable/secure in practice.  SSL/TLS has been around for 
years, is probably our most mature (and heavily used) security service beyond 
plaintext passwords and we're still finding deployed security holes in the 
software stacks.

The TLS channel bindings concept decouples the security layer software from the 
authentication service software so they can evolve separately.  I consider this 
such an important architecture/design improvement that it more than outweighs 
the slight impact on security from separating the two stacks (indeed as the 
separated systems are each simpler, I expect it will improve security in 
practice over a tightly-coupled design).

The major problems involved with the security layer are related to hardening of 
cryptographic algorithms and I/O stack management.  As it's a continuous 
service it has to be rock solid.  Issues with bad APIs, configuration 
complexity and whatnot can largely be hidden by higher layers as long as the 
cryptography and I/O handling are solid.  The major problems with 
authentication are centralized management and identity migration.  Again and 
again I've seen customers ignore more advanced authentication algorithms if 
they fail to achieve the management/migration requirements.  Although the 
authentication algorithms may be cool to geeks like us, they are of secondary 
importance in the real world.  The primary skill sets needed to work on a 
security layer vs. an authentication service are thus almost completely 
disjoint.  As a result I consider the architectural separation and channel 
bindings concept absolutely critical to producing a workable replacement for 
today's web-forms+passwords model.  The management/migration/branding/usability 
requirements are the non-negotiable ones, the actual authentication algorithm 
used is irrelevant absent a solution to the primary requirements.

                - Chris

---

From: Chris Newman <[email-masked]>
Date: Fri, 15 Jun 2007 14:58:38 -0700

> ah, but what's the reason for all of those implementation-imposed

I would speculate it's because the following mandatory pieces of a complete 
DIGEST-based solution were never written down:

* Client UI requirements
* How to store DIGEST H(A1) in a central authentication repository such as
  LDAP or RADIUS in an interoperable fashion
* How web CGIs, PHP modules, etc. interface to the HTTP server's DIGEST support
* How to tie the DIGEST identity to whatever real identity system happens to
  be deployed at the web site.  From an IETF standards perspective, that
  means getting the LDAP directory entry and/or RADIUS/DIAMETER attributes for
  the user to the subsystem that needs that information.  In practice this can
  also involve a SQL database with unspecified keys and attributes.
* How a web page can securely associate branding with a DIGEST authentication
  UI in the browser
* How to migrate an existing password repository using an arbitrary
  one-way-function to store password verifiers to one that's DIGEST compatible,
  and do so in a way that won't generate service calls from customers.

Since implementations already have all this infrastructure for plaintext 
passwords (existing standards are sufficient due to plaintext simplicity), why 
should they waste time doing a one-off version of all this work if there aren't 
enough standards written for it to interoperate?  Besides, DIGEST without TLS 
is now so weak in a 10-cents-per-windows-zombie-CPU-week world that I question 
the value of doing any of this work just for DIGEST.

                - Chris

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 17 Jun 2007 13:37:32 +0200

> DIGEST-based solution were never written down:

Sure, but how much of this is the role of IETF?

Since when do IETF specify client implementation besides what impacts
the wire protocols?

Also don't forget that HTTP has widespread use in many applications
where there is no UI at all.

>   LDAP or RADIUS in an interoperable fashion

Many server implementations uses LDAP as H(A1) store already, with
reasonable options for secure channels server<->backend.

And RADIUS has Digest support, now at PROPOSED STANDARD level (July
2006) and circulated as a draft for quite some years before that. In
RADIUS it's the RADIUS server being the Digest endpoint and the HTTP
server just relays the Digest exchanges and getting told the user
identity (and other RADIUS attributes) on successful authentication.

SASL has had Digest support for quite some time, but is not entirely
compatible with HTTP Digest.

> * How web CGIs, PHP modules, etc. interface to the HTTP server's DIGEST support

As first point above, but server implementation.

>   also involve a SQL database with unspecified keys and attributes.

HTTP is no different here than any other protocol in this regard. IPSec,
TLS, PPP, FTP, SMTP, SNMP, BGP, IMAP, etc etc.. all have similar needs
in identification of the involved parties, and all fight the same
integration problems more or less. But in several of those areas the
problems discussed regarding HTTP authentication is more accepted as
"how things work" and not considered such big problem..

But it's also true that the use of plaintext is very common due to much
the same reasons in integration and migration.

> * How a web page can securely associate branding with a DIGEST authentication

Yes.

>   and do so in a way that won't generate service calls from customers.

The simple fact that there is some migration is usually sufficient to
stop such project even at the planning stage. But it's unavoidable in
order to get away from plain-text exchanges.

It's not a very hard migration, especially not if the password backend
already periodically expires passwords. But it requires the backend to
support Digest (or whatever scheme is used), and for password storage
security reasons to be configured with the realm details..

> enough standards written for it to interoperate?

Even with standards chances are high the needed support would not be
enabled by default, or even implemented.

> Besides, DIGEST without TLS 

I wouldn't say Digest is that weak. Sure, it's possible to perform a
offline brute-force password guessing attack based on the exchanges, and
due to all the legacy crap it's possible for a man in the middle to
downgrade Digest slightly if the client and server is willing to.

But yes, there is a need for a stronger replacement. Especially one
without all the compatibility fallbacks crap.

But more importantly there is a need for a robust framework where new
schemes can be plugged in more easily, and most importantly a way to
make HTTP authentication more visually and functionality attractive in
the browser world.

Even if IETF comes up with the technically most secure authentication
scheme it's unlikely to gain widespread foothold as "the" HTTP
authentication scheme and there will always be major vendors doing their
own thing I am afraid.

Regards
Henrik

---

From: Keith Moore <[email-masked]>
Date: Sun, 17 Jun 2007 11:16:17 -0400

> Since when do IETF specify client implementation besides what impacts
Traditionally we try to not constrain client design more than necessary,
but that doesn't mean we strictly limit ourselves to defining what
happens on the wire.  And to the extent that we have limited ourselves,
if we're learning that this is not sufficient, that's a good reason for
relaxing that limitation.

> where there is no UI at all.
IMHO it would be reasonable to define a profile of HTTP for use with
interactive web browsers.
> As first point above, but server implementation.
I don't see why we shouldn't describe these things if doing so helps
acceptance and interoperability.  Perhaps as informational rather than
normative text, but we should make sure that there's a complete
picture.  These things are important to the success of an authentication
system.  A lot of our failings in the area of authentication have been
due to failure to consider more than just the on-the-wire protocol - we
haven't tried very hard to make our technology accessible to potential
users.

Keith

---

From: "tom.petch" <[email-masked]>
Date: Mon, 18 Jun 2007 09:52:23 +0000

----- Original Message -----
From: "Keith Moore" <[email-masked]>
To: "tom.petch" <[email-masked]>
Cc: "Adrien de Croy" <[email-masked]>; "Apps Discuss" <[email-masked]>;
<[email-masked]>
Sent: Thursday, June 14, 2007 10:19 PM
Subject: Re: RFC2616 vs RFC2617, was: Straw-man charter for http-bis

> > credentials - which I see as the major ongoing work of security - prefer to
do
> > different set of credentials or none, which is the problem that channel
binding
> important principle here that needs to be teased out.

It is good to be picky; it ensures I am thinking clearly (or not as the case may
be).

I think the problem arises at the receiving end, where TLS needs to validate
credentials and where, as in the more secure organisations that I see, the
credentials may be stored in a proprietary database (Active Directory, Domain
Controller and other manufacturers' equivalents).  I do not see the interface
from TLS to this store so that whatever the incoming credentials need to be
compared with can be obtained by TLS.  Hence TLS gets its own set.

Also, while I see no issue with the passing around of a certificate between
application and TLS, I do see an exposure doing this with other forms of
authentication eg passwords. (Certificates, like IPv6, are a good solution to a
widespread problem that the world at large irritatingly refuses to adopt:-(.

With channel bindings, TLS passes on a secure identifier of the channel which
the application can then use to check that there is no Man In The Middle, so
there are no such concerns there.

Tom Petch