Subject: [Fwd: I-D ACTION:draft-decroy-http-progress-00.txt]

From: Julian Reschke <[email-masked]>
Date: Fri, 02 Feb 2007 09:54:51 +0100

-------- Original-Nachricht --------
To: [email-masked]
From: [email-masked]
Date: Thu, 01 Feb 2007 15:50:02 -0500
Subject: I-D ACTION:draft-decroy-http-progress-00.txt
Reply-To: [email-masked]
<https://www1.ietf.org/mailman/listinfo/i-d-announce>, 
<mailto:[email-masked]?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/i-d-announce>
List-Post: <mailto:[email-masked]>
List-Help: <mailto:[email-masked]?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/i-d-announce>, 
<mailto:[email-masked]?subject=subscribe>
Errors-To: [email-masked]
X-GMX-Antivirus: -1 (not scanned, may not use virus scanner)
X-GMX-Antispam: 0 (Mail was not recognized as spam)
X-GMX-UID: nmL3dEhwbUkpFqMKL2gnodtkZ2hlN0rf

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: Progress notifications for HTTP
	Author(s)	: A. de Croy
	Filename	: draft-decroy-http-progress-00.txt
	Pages		: 19
	Date		: 2007-2-1
	
    This document specifies extenions to HTTP to allow progress messages
    for user-agents during lengthy transactions, and to allow better
    flow-control of large message body submission in cases where HTTP
    authentication is required by servers and/or intermediaries.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-decroy-http-progress-00.txt

To remove yourself from the I-D Announcement list, send a message to
[email-masked] with the word unsubscribe in the body of
the message.
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the
username "anonymous" and a password of your e-mail address. After
logging in, type "cd internet-drafts" and then
"get draft-decroy-http-progress-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-decroy-http-progress-00.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce

---

From: Julian Reschke <[email-masked]>
Date: Sat, 03 Feb 2007 11:46:00 +0100

I haven't read the document in detail yet, but one thing that should be 
considered is the choice of the new status code, 102. It collides with 
the definition in RFC2518, a standards track RFC (see 
<http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.10.1>).

Best regards, Julian

Julian Reschke schrieb:
> Content-Type: Multipart/Mixed; Boundary="NextPart"
> Mime-Version: 1.0
> Message-Id: <[email-masked]>
> X-Spam-Score: -2.5 (--)
> X-Scan-Signature: 3002fc2e661cd7f114cb6bae92fe88f1
> X-BeenThere: [email-masked]
> X-Mailman-Version: 2.1.5
> Precedence: list
> List-Id: i-d-announce.ietf.org
> List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/i-d-announce>, 
> ------------------------------------------------------------------------
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Sat, 3 Feb 2007 13:42:08 -0800

On Feb 3, 2007, at 2:46 AM, Julian Reschke wrote:
> I haven't read the document in detail yet, but one thing that  
> should be considered is the choice of the new status code, 102. It  
> collides with the definition in RFC2518, a standards track RFC (see  

Well, considering the rationale is based on using NTLM
authentication (a non-standard authentication mechanism that does
not work outside an intranet and isn't valid for HTTP anyway) for
requests to proxies performing whole-content filtering (an architecture
that definitely won't work with NTLM), I don't think you should
worry about it.  Band-aids are not sufficient for amputated limbs.

....Roy

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 04 Feb 2007 01:08:47 +0100

> worry about it.  Band-aids are not sufficient for amputated limbs.

Pretty much the same problems is seen for the standard schemes as well,
just not as frequent. In basic it's seen on the first request to the
realm, and in digest it's seen whenever the nonce has expired, isn't
suitable for the request or if the client does not support nonce count.

100 Continue helps, but only partially. It would be good if the
connection could be held persistent without having to transmit the
request body, at least in the digest cases which is automated responses
not requiring any user input for completion.

Regarding the second half of the draft, the "Progress" header, is it
really suitable to overload 100 Continue for this purpose as well?
Specifically 3.1.2 Use of 100 continue for progress messages.

Regards
Henrik

---

From: "S. Mike Dierken" <[email-masked]>
Date: Sat, 3 Feb 2007 23:10:10 -0800

Couple questions about the draft:

"Frequently problems occur where users of client agents give up waiting for
visible progress."
When I read this, I first thought the issue was not displaying progress of
the upload itself - but it seems that the issue is with waiting for an
indication of a response, or an indication that a response is being worked
on. Is that correct?

"There is a clear need for upstream agents to be able to provide timely
progress notifications to downstream users [...]"
Later in the document, an example of multiple back-and-forth messages is
given - could the user agent indicate that this back-and-forth messaging is
actually a sign of progress?

Much of the document describes a single use-case dealing with authentication
and NTLM specifically. Are there other specific situations other than
authentication that can be described? Are there other styles of
authentication other than NTLM that would benefit from a solution?

What other approaches have been considered? 
Regarding a connection oriented authentication approach, if the client
submitted a request with no body (like a GET) would that be sufficient to
establish the credentials that could be used with a larger request later on?
Regarding large message bodies, would using chunked transfer-encoding with a
small initial chunk be useful to quickly 'probe' for these chained
authentication routes?

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Julian Reschke
> Sent: Friday, February 02, 2007 12:55 AM
> To: HTTP Working Group
> Subject: [Fwd: I-D ACTION:draft-decroy-http-progress-00.txt]
> 
> 
> 
> List-Unsubscribe: 
> A New Internet-Draft is available from the on-line 
> Internet-Drafts directories.
> 
> 
> To remove yourself from the I-D Announcement list, send a 
> message to [email-masked] with the word 
> unsubscribe in the body of the message.
> Internet-Drafts are also available by anonymous FTP. Login 
> with the username "anonymous" and a password of your e-mail 
> address. After logging in, type "cd internet-drafts" and then 
> http://www.ietf.org/shadow.html or 
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
> 
> Below is the data which will enable a MIME compliant mail 
> reader implementation to automatically retrieve the ASCII 
> version of the Internet-Draft.
> 
>

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 04 Feb 2007 23:30:13 +0100

> authentication other than NTLM that would benefit from a solution?

The draft is about two quite independent problems.

1. POST/PUT and authentication challenges. Mostly a problem for the
non-compliant connection oriented NTLM/Negotiate/Kerberos authentication
schemes, but may also be noticeable for Digest to some extent even if
just marginally so. To solve this the draft proposes a new 1xx response
type to indicate "please DO NOT send the request body, I am not
interested in what it contains", asking the client to diverge even more
from the standard message format. This is mostly a band-aid to make the
non-compliant connection oriented authentication schemes survive a large
PUT/POST as initial request. 

2. Servers taking a long time to finish processing a request before the
final response is known. To solve this the draft proposes a new
"Progress" header (both request and response) and reusing 100 Continue
to relay this to the user-agent. Very similar to the 102 Processing
response defined by rfc2518 but with a new header to provide additional
information to the end user explaining what is going on.

> What other approaches have been considered? 

For the second problem (servers taking a long time to process a request)
RFC 2518 already defines 102 Processing. But the new "Progress" header
proposed in this draft adds valuable information which the user agent
may display to inform the user why the request is taking so long to
complete.

The first problem has a couple of alternative solutions, none of which
is any better..
  * If an auth challenge is seen, close the connection and then retry
with a 0-length body until the credentials can be provided.
  * As above, but instead use another temporary request to initiate the
authentication handshake, assuming one knows a such request in the same
protection space..
  * Probably other hacks is possible as well to get around the problem.

> establish the credentials that could be used with a larger request later on?

It is. Problems arise if there is no open authenticated connection where
to send the request, and the request has a large request body.

> small initial chunk be useful to quickly 'probe' for these chained

Not sure I follow what you think on here.

As for seeing that authentication is required before transmitting the
request body 100 Continue is quite sufficient.

Ah, now I follow. Yes, sending the request using chunked
transfer-encoding also allows getting around the PUT/POST authentication
problem very nicely. Just terminate the body immediately when seeing the
authentication challenge instead of 100 Continue.. this is how it should
be done. Nice catch.

So the draft should be reduced to just the new Progress header, suitable
to be used in the already defined "102 Processing" response, and
additionally implementation advice how to handle authentication
challenges to avoid having to repeatedly transmit the request body by
making use of chunked transfer encoding once it's known the resource is
a HTTP/1.1 resource, easily detected by the 401 challenge.

Regards
Henrik

---

From: Adrien de Croy <[email-masked]>
Date: Sun, 04 Feb 2007 23:27:45 +0000

Hi all

thanks for forwarding this - comments below.

Should I subscribe to ietf-http-wg?

Henrik Nordstrom wrote:

Yes, a definite band-aid.  Problem has been in definition of how long a
client should wait for a 100 Continue.

RFC2616 talks about a method based on RTT for connection to the
server.  However, when the server is a proxy

this RTT is usually extremely small, and the upstream RTT is shielded
from the downstream agent.

The real issue with 100 is that it is a green light to send the
content, but there are no hard and fast rules about how

long a client should wait before sending POST or PUT data in absence of
receiving a 100 Continue.  This is the crux

of the problem, since the client cannot ever rely on ever getting a 100
Continue.

The other problem is the definition of using the "Expects" header.  The
spec states that

a) a server must not send a 100 Continue if the client did not send an
Expects header containing "100-Continue".

Many web servers violate this and send it regardless, and in fact I
haven't seen the latest versions of IE or firefox

ever sending an Expects header - possibly out of fear of what will
happen if the server actually obeys the requirement

for unhandled expectations.

b) that any server receiving an Expects header it can't fulfil must
bounce the request with a 417.

This makes the usability of the Expects header much more restricted.
Any client wishing to indication a desired

functionality must use a different header than "Expects", since
"Expects" designates required functionality,

where the request must not be completed without it.  So this doesn't
allow for indication of optional desired functionality

The proposed 102 (which will have to be a different code because of
WebDav) is a red light.

A red light is fundamentally different to a green light.

Anyway, I did consider what alternatives there should be, and how the
spec was intended to work in this respect.

But the spec really only considers auth to an origin server, not
multiple auths along the chain to an origin server.

If you look at how IE and Firefox actually behave in this respect,
you'll see the dilemma that their devs found with the

spec.

IE for instance sends a POST with Content-length: 0 if it believes it
is about to be challenged for authentication.

It often gets this wrong however, and sends requests like this over
connections that already have established credentials.

We had to add explicit code into WinGate to catch this case, where we
have to re-start the auth process (only alternative

is to hard-close the connection).

however, IE (6 and 7) only does this the first time.  So that will
allow it to authenticate only one link in the chain.  If there are 2
links

which is common, you still end up sending the content 3 times to the
upstream server for NTLM, or 2 times for

Basic.  The more links the worse it gets.

Also IE doesn't do this if it's talking to a proxy, it sends the full
amount from the start.

Firefox 1.x and 2 always sends the full amount each time.

The spec states that if a client receives some sort of error response
whilst it is uploading content, then it should

stop uploading, and break the connection (since there's no way to
resynchronise the connection).  This is fine if the auth

is basic, but no good for session-negotiated auth such as NTLM or
anything else that uses a challenge-response.

Using chunking for uploads in this respect whilst it might allow a
resynchronisation. I think you'll

find that this causes implementation problems.  I don't think many
proxies even support chunked data from a client

no current clients that I've seen ever send chunked data, since they
always know the content-length.

It would be a bigger re-write of the client to get it to send chunked
data than it would be to have it honour a new

red-light code.  So you'll find more resistance to implement it, and
more problems with faulty implementations etc.

Also a red light code is an explicit solution, rather than an implicit
one.  The client is left under no doubt as to what

will happen and under what conditions it may send the body.   The
100-Continue does not provide this certainty.

So I still think a red-light code will be a simpler implementation for
people, and the fallback is that we are in the same

situation we are now.

There are also benefits to be had from having the Content-length field
in the requests from the beginning, since then

a server or proxy can use that information in its decision (i.e.
disallowing uploads above a certain size).  Chunking would

not allow for that sort of functionality.

Actually the proposal (albeit maybe not too clearly) states that such a
header could be returned in any 1xx message.

I was unaware of the WebDAV proposal in this respect, so the 102
defined there would be an obvious choice.

this doesn't cope with more than one auth challenge in a session.

Many transparent or intercepting proxies are configured to require
auth, so in such cases the proxy

sends back a WWW-authenticate field instead of a Proxy-authenticate one
(since the client believes

it is talking to a server).  So a client can actually receive multiple
401 WWW-authenticate responses

from chained agents even though it thinks it is only talking to one
server.

The browsers out there actually cope with this and prompt for next-hop
authentication credentials.

like for a dummy resource?  tricky to know what sort of request would
be usable to set up auth

also, server may still close the connection once it thinks it has
fulfilled the request, whether the proxy

maintains the connection to the client or not.  So I wouldn't pursue
this path.

I couldn't think of any, and I thought about it for a long time....
that's why I wrote the spec in the end :)

It needs a bit of work, but I think the concepts are worth-while.  My
support-desk certainly do.

Only for the immediate connection for the reasons above (being the
client doesn't know how long to wait).

If the current spec was changed so that the client MUST wait for a 100
continue, we would be fine, but

it makes it optional on the part of the server and client.

This creates also serious problems for HTTP/1.1 proxies talking to
upstream HTTP/1.0 servers.

In such cases the proxy would need to spool the entire content locally
before it could upload it

the next hop.  This then creates issues where the client sent the
request really quickly to the proxy

but the proxy then takes ages to get it the next hop before sending a
response. So the user is left

hanging wondering what is going on. The progress header would help
here, but we find in general

it is best to manage flow-control end to end rather than hop by hop,
then the progress indications

of the browser are applicable, the user expects to see the content take
a while to upload, and then

shortly afterwards see some sort of response.  If we have to buffer
everything in the middle, then

the user won't see a response until some indeterminate period of time
after the upload has completed

according to them.

So I think whilst the chunked uploading may allow the existing spec to
kinda cope with the problem

it's not an elegant solution which provides for the best user
experience.

And users have a habit of complaining if their expectations aren't
met.  educating them is very

expensive.

Regards

Adrien de Croy

Regards
Henrik

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Tue, 06 Feb 2007 08:40:45 +0100

>> <http://greenbytes.de/tech/webdav/rfc2518.html#rfc.section.10.1>).

I haven't read either of these before, but isn't the HTTP 1.1-defined "202  
Accepted" exactly what 102 tries to be? From HTTP 1.1:

   The request has been accepted for processing, but the processing has not
   been completed.

The word "processing" is even mentioned twice in the first sentence.

-- 
Asbjørn Ulsberg     -=|=-    http://virtuelvis.com/quark/
«He's a loathsome offensive brute, yet I can't look away»

---

From: Jeffrey Mogul <[email-masked]>
Date: Tue, 06 Feb 2007 11:31:33 -0800

Remember: there is an IANA "HTTP Status Code Registry", at
<http://www.iana.org/assignments/http-status-codes>, so people
should check this registry before choosing new status codes.

Not that draft-decroy-http-progress-00.txt necessarily justifies
one, of course.

-Jeff

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 06 Feb 2007 22:21:30 +0100

> The word "processing" is even mentioned twice in the first sentence.

The two differ quite significantly.

202 means "I have accepted the request, and will probably act on it some
time in future, no need to send the request again. I can't tell you what
the result will be, but if you know how you may find the results
somewhere when I am done if or when I choose to act on your request".

102 means "I am currently processing the request. Please wait, the
response is coming, just be patient with me while I process your
request".

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Sat, 17 Feb 2007 19:23:29 +0100

> Not that draft-decroy-http-progress-00.txt necessarily justifies

I wasn't aware of the registry, nor were many other people I asked. It's 
very well hidden in RFC2817 ("Upgrading to TLS Within HTTP/1.1").

So how about moving it into a separate spec for easier maintenance, and 
better visibility?

Proposal at 
<http://greenbytes.de/tech/webdav/draft-reschke-http-status-registry-latest.html>.

Best regards, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 18 Feb 2007 01:07:30 +0100

> So how about moving it into a separate spec for easier maintenance, and 

Or better yet add the IANA http status code registry reference to
RFC2616bis declaring that IANA is responsible for maintaining the status
code registry?

Having "registry" type RFCs never works out well as these needs to be
updated before the next draft needing a new status code is published.
This job is much better done by IANA.

6.1.1 Status Code and Reason Phrase

just after the status code class definitions add something like the
following.

   The Internet Assigned Numbers Authority (IANA) acts as a registry
   for HTTP status codes and suggested reason phrases. The values
   defined here is only the initial set defined for HTTP/1.1.

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Sun, 18 Feb 2007 10:24:58 +0100

> RFC2616bis declaring that IANA is responsible for maintaining the status

That's of course ok as well, but at this point we can't be sure that 
there will be a RFC2616bis anytime soon.

> This job is much better done by IANA.

Yep. This one is not intended to be one of those. It only has a list in 
order to update what's there.

 > ...

Best regards, Julian

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 18 Feb 2007 11:14:02 +0100

> there will be a RFC2616bis anytime soon.

Well, to get there we can only do our best in updating the document and
correct the issues seen and trying to get it accepted.

I very much prefer getting RFC2616bis out as a draft with the minimum
needed updates compared to having complementary drafts published which
doesn't even intend to become an RFC..

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Sun, 18 Feb 2007 11:37:57 +0100

> doesn't even intend to become an RFC..

So it really should be on the issues list 
(<http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/issues/>) then. Mark, 
could you please add it?

Best regards, Julian