Subject: Re: [google-gears-eng] Re: Deploying new expectation-extensions

From: Mark Nottingham <[email-masked]>
Date: Fri, 12 Sep 2008 14:25:26 +1000

Catching up a bit late...

On 29/08/2008, at 11:13 PM, Julian Reschke wrote:
>>>
>>> The problem in offering widely differing ways to do the same thing  
>>> is that
>>> at least in the beginning, clients and servers would need to  
>>> implement all
>>> of them in order to achieve some kind of interoperability.  It  
>>> would be
>>> great if this could be avoided.
>> Good point. We'll definitely consider that.
>
> I think the biggest simplification would be to select either the  
> ETag based approach or the URI-based approach (where I think the  
> latter works better).

Big +1. Simpifying is a good thing. Don't try to keep *everybody* happy.

Although I started this e-mail with no real preference, I'm leaning  
towards the ETag-based approach, for the reasons below (although there  
may be a way to address them).

>>> It would be interesting to hear why you think these additional  
>>> parts are
>>> needed.
>> Content-Range request header: used to specify which byte range is
>> being sent. Roy's proposal simply suggested appending bytes, but that
>> leads to requests which are non-idempotent.
>
> It depends. If the "append" operation always uses POST/If-Match,  
> then each of these requests is idempotent (it will either succeed,  
> or the server will reject it because the ETag isn't current anymore).

...but that's if you're using an ETag. My reading was that ETags vs.  
different URLs would be an either/or thing in this proposal... If one  
of them is picked and the other discarded, it would be easier to make  
a judgement about this.

>> the precedent of byte-range GETs. If a byte-range POST is being
>> performed, then by all means let's specify which range of bytes are
>> being sent!
>
> I think sending the Content-Range header is harmless, but not  
> required for the whole thing to work.

If it's being done across multiple URIs, you're really bending if not  
breaking the semantics of Content-Range.

> As far as I can tell, if you use the ETag based approach, and  
> multiple clients try to post to the same collection (POST URI), then  
> you'll have to disambiguate the requests. That problem would go away  
> if each of them would use a different URL.

I read the doc as saying that the server would provide unique ETags  
somehow...

One other issue; caches use a set of rules against requests to  
determine when to invalidate their entries (section 13.10); generally,  
it's desirable to do so when the response is received (i.e., when the  
request body is stored/processed on the origin server). The ETag-based  
approach will work nicely with this, but the URL-based one will not;  
the request URI will be invalidated when the request has started --  
but potentially *not* finished -- on the origin server.

Also, I haven't thought through it completely, but it seems like it  
would be compelling to implement this feature in an intermediary  
(i.e., an accelerator in front of your servers). Using the ETag  
approach would avoid having to worry about coordinating URI space  
between the intermediary and origin server, although there are  
probably other issues to consider (e.g., doing the initial handshake).

Cheers,

--
Mark Nottingham       [email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Fri, 12 Sep 2008 09:25:48 +0200

Mark Nottingham wrote:
> ...
>> I think the biggest simplification would be to select either the ETag 
>> based approach or the URI-based approach (where I think the latter 
>> works better).
> 

Looking at your feedback I realize that what I really was thinking of 
was an URI-based approach *with* (strong) ETags.

> ...
>> It depends. If the "append" operation always uses POST/If-Match, then 
>> each of these requests is idempotent (it will either succeed, or the 
>> server will reject it because the ETag isn't current anymore).
> 
> different URLs would be an either/or thing in this proposal... If one of 
> them is picked and the other discarded, it would be easier to make a 
> judgement about this.

Right. See above -- ETags should be used anyway (no matter whether 
special URIs are used or not).

>> I think sending the Content-Range header is harmless, but not required 
>> for the whole thing to work.
> 

Maybe I wasn't clear what I was proposing: each resumable transfer would 
get a specific URI, and the various stages of that transfer would be 
identified by a specific ETag. Thinking of it, in this case the ETag 
*could* be built based on the number of bytes and the digest of bytes 
received.

>> As far as I can tell, if you use the ETag based approach, and multiple 
>> clients try to post to the same collection (POST URI), then you'll 
>> have to disambiguate the requests. That problem would go away if each 
>> of them would use a different URL.
> 

Disambiguating by ETag probably would work, but that doesn't feel right 
to me. If multiple resumable transfers can be in progress at the same 
point of time, then this really sounds like multiple resources (thus 
multiple URIs), not multiple variants of the same resource to me.

> One other issue; caches use a set of rules against requests to determine 
> when to invalidate their entries (section 13.10); generally, it's 
> desirable to do so when the response is received (i.e., when the request 
> body is stored/processed on the origin server). The ETag-based approach 
> will work nicely with this, but the URL-based one will not; the request 
> URI will be invalidated when the request has started -- but potentially 
> *not* finished -- on the origin server.
> 
> would be compelling to implement this feature in an intermediary (i.e., 
> an accelerator in front of your servers). Using the ETag approach would 
> avoid having to worry about coordinating URI space between the 
> intermediary and origin server, although there are probably other issues 
> to consider (e.g., doing the initial handshake).

That's an interesting idea, but again, using the same URI to handle 
multiple transfers at once doesn't feel right to me...

BR, Julian

---

From: "Brian McBarron" <[email-masked]>
Date: Fri, 12 Sep 2008 10:53:43 -0400

On Tue, Jul 22, 2008 at 2:47 PM, Julian Reschke <[email-masked]>wrote:
>
> ETags are good. No problem with that.
>
> That being said, I liked Roy's proposal in where the resumable request gets
> a separate URI assigned, exposed in the Location header. Maybe I read
> something the wrong way, or maybe I'm confusing what the two documents
> propose, but wouldn't it be best to always use that design?
>
> The problem in offering widely differing ways to do the same thing is that
> at least in the beginning, clients and servers would need to implement all
> of them in order to achieve some kind of interoperability.  It would be

I want to call out one point here.  In our proposal, the server is in full
control of the upload URI and the presence/absence of an ETag.  This has
several advantages:

1) The server can decide whether it is most convenient (from an
implementation point of view) to identify operations via a unique URI and/or
an ETag.
2) Regardless of the two potential mechanisms which can be used, a server
only needs to use one.  The server never has to "implement all of them".
3) The client or intermediary will always have a URI to deal with,
regardless of whether it is unique or not.  Indeed, the client doesn't have
enough information to know if a URI is unique.
4) Thus, the only added complexity to the system of supporting either/both
methods, is that a client or intermediary must correctly forward an ETag
when it _optionally_ appears.

To me, the overhead of (4) is very minor considering the power and
flexibility of (1).

On another note, I think it's very much worth designing a protocol that is
compatible with our original intent, which I summarize below.  While this
isn't feasible while intermediaries are not 1.1 compliant, we can hope that
will change eventually, enabling this extension to really shine, IMO.

Assume that a client is conducting requests as normal, with no regard for
whether the server supports this extension.  At some point, it makes any
non-idempotent request to the server, and based on the characteristics of
the request, the server realizes the operation has a probability of failure
due to network disconnect.  This could be due to a particularly long running
upload, a lossy network connection, or a back-end delay in processing the
request (such as a credit card transaction).  Dynamically, the server
decides to enable resumability for the request, by pushing down a 103
intermediate response.  If the client is compliant, it will either ignore
the 103 if it doesn't support resumability, or it will properly enable
support for continuing the request in case of network failure.  In the case
of an upload, it means we can continue where we left off.  In the case of a
long-running credit-card transaction, it means we can safely poll for the
final response code, without the risk of double billing the user.  The
benefit is that no additional planning had to be done between the client and
server ahead of time, so there is no overhead in regular traffic.

---

From: Julian Reschke <[email-masked]>
Date: Sat, 13 Sep 2008 10:09:13 +0200

Brian McBarron wrote:
> I want to call out one point here.  In our proposal, the server is in 
> full control of the upload URI and the presence/absence of an ETag. 
>  This has several advantages:
> 
> implementation point of view) to identify operations via a unique URI 
> and/or an ETag.
> 2) Regardless of the two potential mechanisms which can be used, a 
> server only needs to use one.  The server never has to "implement all of 
> them".
> regardless of whether it is unique or not.  Indeed, the client doesn't 
> have enough information to know if a URI is unique.
> 4) Thus, the only added complexity to the system of supporting 
> either/both methods, is that a client or intermediary must correctly 
> forward an ETag when it _optionally_ appears.

OK, so a server can just choose one approach, and for the client it 
doesn't matter.

So the only overhead is in the specification, which, I think, is still 
sub-optimal. I'd really prefer a single way to do that if that doesn't 
conflict with another goal.

> To me, the overhead of (4) is very minor considering the power and 

Correctly using ETags is something we really should require from parties 
using this protocol (so I wouldn't even consider that an "overhead").

> On another note, I think it's very much worth designing a protocol that 
> is compatible with our original intent, which I summarize below.  While 
> this isn't feasible while intermediaries are not 1.1 compliant, we can 
> hope that will change eventually, enabling this extension to really 
> shine, IMO.
> 
> Assume that a client is conducting requests as normal, with no regard 
> for whether the server supports this extension.  At some point, it makes 
> any non-idempotent request to the server, and based on the 
> characteristics of the request, the server realizes the operation has a 
> probability of failure due to network disconnect.  This could be due to 
> a particularly long running upload, a lossy network connection, or a 
> back-end delay in processing the request (such as a credit card 
> transaction).  Dynamically, the server decides to enable resumability 
> for the request, by pushing down a 103 intermediate response.  If the 
> client is compliant, it will either ignore the 103 if it doesn't support 
> resumability, or it will properly enable support for continuing the 
> request in case of network failure.  In the case of an upload, it means 
> we can continue where we left off.  In the case of a long-running 
> credit-card transaction, it means we can safely poll for the final 
> response code, without the risk of double billing the user.  The benefit 
> is that no additional planning had to be done between the client and 

Good summary.

It seems to me that if we can rely on HTTP/1.1 throughout, this is 
relatively easy to achieve. My recommendation is to define the simplest 
possible extension based on HTTP/1.1 (optimal number of optional stuff: 
zero :-), and only then consider extensions/workarounds to get it work 
with HTTP/1.0. -- Speaking of which, for HTTP/1.0 is there anything 
besides the missing 1xx status codes?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 15 Sep 2008 12:05:59 +1000

On 12/09/2008, at 5:25 PM, Julian Reschke wrote:

>>> multiple clients try to post to the same collection (POST URI),  
>>> then you'll have to disambiguate the requests. That problem would  
>>> go away if each of them would use a different URL.
> Disambiguating by ETag probably would work, but that doesn't feel  
> right to me. If multiple resumable transfers can be in progress at  
> the same point of time, then this really sounds like multiple  
> resources (thus multiple URIs), not multiple variants of the same  
> resource to me.

Huh. That's very revealing, I think (if unintentional :) POST can  
already create a new resource with a new, server-selected URI, and the  
pattern for doing so is already described with POST, 201 and Location.

Question: If I want to make this sort of request resumeable, do I do  
this?

REQ: POST /a
REQ: Content-Range: bytes */100
RES: 308 Resume Incomplete
RES: Location: /b

REQ: POST /b
REQ: Content-Range: 0-100/100
REQ: [bytes]
RES: 200 OK

or this?

REQ: POST /a
REQ: Content-Range: bytes */100
RES: 308 Resume Incomplete
RES: Location: /z

REQ: POST /z
REQ: Content-Range: 0-100/100
REQ: [bytes]
RES: 201 Created
RES: Location: /b

?

The important part here is: is this protocol defining a "temporary"  
resource (with a very specific interface) for the Location in a 308  
refers to, or is the Location in a 308 referring to a "regular"  
resource that's used for more than that?

It's interesting to note that the second approach (with the temp  
resource) preserves the 201 status code in the interchange, while in  
the former approach, it's not there (308 usurps it).

Now look at it with PUT (to a not-yet-existent resource);

REQ: PUT /a
REQ: Content-Range: bytes */100
RES: 308 Resume Incomplete
RES: Location: /b

REQ: PUT /b
REQ: Content-Range: 0-100/100
REQ: [bytes]
RES: 201 Created
RES: Location: /a

Here, if we use URIs, /b *has* to be a "temporary" resource with a  
very specifically defined behaviour; it accepts PUTs and has a side  
effect of having its bytes copied to /a (presumably when the final 201  
is sent).

My point here is that there are actually some pretty deep differences  
between the URI approach and the ETag approach; the URI approach is  
much more intrusive and needs to be specified in a different way  
(e.g., talking about what methods to use, the nature of the resource  
created, etc.).

Back to your comment;

> Disambiguating by ETag probably would work, but that doesn't feel  
> right to me. If multiple resumable transfers can be in progress at  
> the same point of time, then this really sounds like multiple  
> resources (thus multiple URIs), not multiple variants of the same  
> resource to me.

I don't know that I agree; with PUT, it's very natural to use ETags  
(you avoid creating the temporary resource, and have the option of  
409'ing any concurrent PUTs after the first), whereas with POST,  
you're just pushing the assignment of a final identifier for the  
created resource until the entire request entity is received (which is  
the case with the URI-based approach anyway, unless you're arguing  
that POST is a special case and *doesn't* create a temporary resource,  
unlike PUT), and you still have the option of not assigning it any  
identity (just as many POST processors do today).

So, I'm firmly leaning in the direction of the ETags-only approach  
now; I think the selection of a URI for created resources is  
separable, and should be separate.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 15 Sep 2008 12:10:10 +1000

On 13/09/2008, at 12:53 AM, Brian McBarron wrote:

> I want to call out one point here.  In our proposal, the server is  
> in full control of the upload URI and the presence/absence of an  
> ETag.  This has several advantages:
>
> implementation point of view) to identify operations via a unique  
> URI and/or an ETag.
> server only needs to use one.  The server never has to "implement  
> all of them".
> regardless of whether it is unique or not.  Indeed, the client  
> doesn't have enough information to know if a URI is unique.

Sorry, I *really* don't buy that. You're pushing more complexity and  
ambiguity (see previous post) onto the clients, which history shows  
are more numerous, harder to implement and easier to stuff up. On the  
server side the difference between using URIs and using ETags is more  
a matter of style than implementation complexity.

I appreciate that you're trying to foster adoption by making your  
proposal more flexible, but my experience is that this kind of  
flexibility is what puts protocols and extensions at greater risk for  
failure.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Mon, 15 Sep 2008 15:13:50 +0200

Mark Nottingham wrote:
>> right to me. If multiple resumable transfers can be in progress at the 
>> same point of time, then this really sounds like multiple resources 
>> (thus multiple URIs), not multiple variants of the same resource to me.
> 
> 

Of course POST can do that. Did anybody argue something else?

> Question: If I want to make this sort of request resumeable, do I do this?
> 
> REQ: POST /a
> refers to, or is the Location in a 308 referring to a "regular" resource 
> that's used for more than that?

If I understand correctly, in the first example the server immediately 
assigns the URI for the resource-to-be-created, let's the client know 
it, and lets it transfer the remaining bytes to that resource. In this 
case, the server would need to make sure that this resource is only 
available to the client until the transfer is completed.

In the second case, the server assigns a temporary URI which is just 
used to complete the transfer. Once that's done, the "final" resource is 
being created. This looks similar to what Roy proposed in 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2008AprJun/0082.html>.

I think the second approach is more versatile, because it also covers 
cases where the server wouldn't create a new URI upon POST.

> resource) preserves the 201 status code in the interchange, while in the 
> former approach, it's not there (308 usurps it).
> 
> Here, if we use URIs, /b *has* to be a "temporary" resource with a very 
> specifically defined behaviour; it accepts PUTs and has a side effect of 
> having its bytes copied to /a (presumably when the final 201 is sent).

Yes.

BTW: it wouldn't necessarily be PUT; it could be anything that allows 
"appending", such as POST or PATCH.

> between the URI approach and the ETag approach; the URI approach is much 
> more intrusive and needs to be specified in a different way (e.g., 
> talking about what methods to use, the nature of the resource created, 
> etc.).

Well, not entirely.

Let's say you've got three concurrent resumable transfers started to /a, 
and the server has assigned the etags "C1-1", "C2-1" and "C3-1" to the 
three clients.

Client 1 starts its upload:

REQ: POST /a
REQ: If-Match: "C1-1"
REQ: Content-Range: 0-50/100
REQ: [bytes]
RES: 200 OK
RES: ETag: "C1-2"

...what I'm concerned if is that we're essentially introduce ETag-based 
variant-selection here.

So what do these ETags represent in requests *other* than resumable 
uploads, such as in:

REQ: GET/a
REQ: If-Match: "C1-1"

?

Will it have an effect?

So just avoiding new URIs may look simpler first, but it also requires 
additional specification work.

>> right to me. If multiple resumable transfers can be in progress at the 
>> same point of time, then this really sounds like multiple resources 
>> (thus multiple URIs), not multiple variants of the same resource to me.
> 
> I don't know that I agree; with PUT, it's very natural to use ETags (you 
> avoid creating the temporary resource, and have the option of 409'ing 
> any concurrent PUTs after the first), whereas with POST, you're just 

That assumes that PUT with Content-Range can be used today, which really 
isn't the case, unless the client can be confident that the server 
actually understands PUT with ranges.

> pushing the assignment of a final identifier for the created resource 
> until the entire request entity is received (which is the case with the 
> URI-based approach anyway, unless you're arguing that POST is a special 
> case and *doesn't* create a temporary resource, unlike PUT), and you 
> still have the option of not assigning it any identity (just as many 
> POST processors do today).
> 
> So, I'm firmly leaning in the direction of the ETags-only approach now; 
> I think the selection of a URI for created resources is separable, and 
> should be separate.

I agree with that part; the URI assigned for the upload really should be 
temporary.

With that, the approaches are almost identical; in both cases unique 
identifiers are minted (ETags or URIs), the server needs to deal with 
house keeping, and the impact of other methods must be understood and 
specified.

BR, Julian

---

From: "Brian McBarron" <[email-masked]>
Date: Mon, 15 Sep 2008 09:41:02 -0400

On Sun, Sep 14, 2008 at 10:10 PM, Mark Nottingham <[email-masked]>wrote:

> ambiguity (see previous post) onto the clients, which history shows are more
> numerous, harder to implement and easier to stuff up. On the server side the
> difference between using URIs and using ETags is more a matter of style than
> implementation complexity.

Rather than waving hands about history, let's keep the conversation a bit
more pragmatic.  If we can find agreement that one of the models is
sufficient, I'd be happy to narrow the specification.

I can imagine cases where redirection is key to efficient implementation of
resumability.  Here are some use cases:
1) A server farm where the majority of traffic is handled by essentially
diskless front-ends.  In order to support resumability, a server would need
to redirect the client to a separate pool of servers who are provisioned to
do the necessary caching.
2) A server farm where requests are load balanced across a pool of servers.
 Even if the load-balancing mechanism implements a reasonable affinity
mechanism, it's likely that that mechanism is just an approximation, and
that any particular request may end up on a variety of machines.  I can
imagine that an efficient implementation of resumability would be to ensure
the user connects to the same machine, where the request may be cached on
local disk.  Therefore, the resumable URI could be the name of that machine,
rather than the load balanced hostname.

I much prefer the ETag mechanism, in general.  I feel it is easier to
understand, and in some cases I think it would be easier to implement.
 Charles Fry created a reference implementation of this protocol in Apache,
and I think his changes were smaller with the ETag approach, due to
similarity to existing concepts in the server.  With that said, I can't
think of any cases where ETags would be necessary for ease of
implementation, so if I had to pick just one approach, I think URI's is the
way to go.

---

From: Stefan Eissing <[email-masked]>
Date: Mon, 15 Sep 2008 16:16:04 +0200

Am 15.09.2008 um 15:41 schrieb Brian McBarron:

> >On Sun, Sep 14, 2008 at 10:10 PM, Mark Nottingham <mnot@yahoo- 
> inc.com> wrote:
> >[...]
> >Sorry, I *really* don't buy that. You're pushing more complexity  
> and ambiguity (see previous post) onto the clients, which >history  
> shows are more numerous, harder to implement and easier to stuff  
> up. On the server side the difference between using >URIs and using  
> ETags is more a matter of style than implementation complexity.
>
> Rather than waving hands about history, let's keep the conversation  
> a bit more pragmatic.  If we can find agreement that one of the  
> models is sufficient, I'd be happy to narrow the specification.

I agree with Mark that optional features/alternate behaviours make  
clients more complex and results usually in poor interoperability.  
But, boy, do I dislike ETags. They are like visits to the dentist,  
necessary but no pleasure.

How would a client cancel an upload scenario which just uses ETags?  
(With temp URIs the obvious solution seems to be a DELETE on that  
uri. I sure would feel uncomfortable with a DELETE/if-match: Etag on  
the original POST target...)

//Stefan

--
<green/>bytes GmbH, Hafenweg 16, D-48155 Münster, Germany
Amtsgericht Münster: HRB5782

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Sep 2008 09:25:45 +1000

On 15/09/2008, at 11:13 PM, Julian Reschke wrote:

>>>>> then you'll have to disambiguate the requests. That problem  
>>>>> would go away if each of them would use a different URL.
>>>> I read the doc as saying that the server would provide unique  
>>>> ETags somehow...
>>>
>> already create a new resource with a new, server-selected URI, and  
>> the pattern for doing so is already described with POST, 201 and  
>> Location.
>

No, just laying the groundwork...

> If I understand correctly, in the first example the server  
> immediately assigns the URI for the resource-to-be-created, let's  
> the client know it, and lets it transfer the remaining bytes to that  
> resource. In this case, the server would need to make sure that this  
> resource is only available to the client until the transfer is  
> completed.

... or use ETags on the requests to differentiate them.

> used to complete the transfer. Once that's done, the "final"  
> resource is being created. This looks similar to what Roy proposed  
> in <http://lists.w3.org/Archives/Public/ietf-http-wg/2008AprJun/0082.html 
> >.
>
> I think the second approach is more versatile, because it also  
> covers cases where the server wouldn't create a new URI upon POST.

... as would using ETags to differentiate requests. :)   Understand  
I'm not pushing that solution strongly (yet), It's just that they're  
functionally equivalent so far.

>> resource) preserves the 201 status code in the interchange, while  
>> in the former approach, it's not there (308 usurps it).
>> effect of having its bytes copied to /a (presumably when the final  
>> 201 is sent).
>
> Yes.
>
> BTW: it wouldn't necessarily be PUT; it could be anything that  
> allows "appending", such as POST or PATCH.

Good point.

>> My point here is that there are actually some pretty deep  
>> differences between the URI approach and the ETag approach; the URI  
>> approach is much more intrusive and needs to be specified in a  
>> different way (e.g., talking about what methods to use, the nature  
>> of the resource created, etc.).
>
> Well, not entirely.
>
> Let's say you've got three concurrent resumable transfers started  
> to /a, and the server has assigned the etags "C1-1", "C2-1" and  
> "C3-1" to the three clients.
>
> ...what I'm concerned if is that we're essentially introduce ETag- 
> based variant-selection here.
>
> So just avoiding new URIs may look simpler first, but it also  
> requires additional specification work.

It will require additional spec work if this use of ETags is  
incompatible with existing ones. If it is, they shouldn't be used for  
this purpose at all, but I don't *think* they are (happy to be proven  
wrong, as always).

I.e., yes, if-match will work as specified; what else needs to be said?

>> you're just
> That assumes that PUT with Content-Range can be used today, which  
> really isn't the case, unless the client can be confident that the  
> server actually understands PUT with ranges.

That seems to be a problem with all the approaches on the table,  
according to the flows in the current document. By the letter of the  
law, if the server doesn't understand a Content-* header on a PUT  
request, it should refuse it, but we already have an open issue or two  
(#79, #102) on that...

>> pushing the assignment of a final identifier for the created  
>> resource until the entire request entity is received (which is the  
>> case with the URI-based approach anyway, unless you're arguing that  
>> POST is a special case and *doesn't* create a temporary resource,  
> I agree with that part; the URI assigned for the upload really  
> should be temporary.
>
> identifiers are minted (ETags or URIs), the server needs to deal  
> with house keeping, and the impact of other methods must be  
> understood and specified.
>
> BR, Julian
>

BTW, we should really be talking about an Internet-Draft at this  
point, rather than a wiki page. Google guys, when will that happen?

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 16 Sep 2008 10:45:03 +1000

> the original POST target...)

Probably rely on server-side garbage collection of incomplete requests  
(just as with a dropped "normal" connection).

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Tue, 16 Sep 2008 10:11:54 +0200

Mark Nottingham wrote:
> ...
>> Well, not entirely.
>>
>> Let's say you've got three concurrent resumable transfers started to 
>> /a, and the server has assigned the etags "C1-1", "C2-1" and "C3-1" to 
>> the three clients.
>>
>> ...what I'm concerned if is that we're essentially introduce 
>> ETag-based variant-selection here.
>>

OK, let's rephrase: what I'm concerned with is this:

- most of the time, resources that support POST for upload have a 
GETtable representation

- these representations (again, most of the time), are not those of 
previous upload, but a form, a collection listing, or both

- so the resource posted to has a very different nature from those it 
creates

- if, for GET/HEAD, the resource assigns and supports ETags, they have a 
very different purpose than those assigned for resumable uploads, and 
those will need to be distinguished upon GET -- so some If-Match headers 
will cause conneg, some won't

So my concern is that we use two different types of ETags on the same 
resource.

> request, it should refuse it, but we already have an open issue or two 

We *couöd' state that minting a new URI for resumable upload implies 
proper server support for PUT/Range; but of course that doesn't help 
with broken intermediaries.

BR, Julian

---

From: "Charles Fry" <[email-masked]>
Date: Tue, 16 Sep 2008 07:22:48 -0400

>>>> to me. If multiple resumable transfers can be in progress at the same point
>>>> of time, then this really sounds like multiple resources (thus multiple
>>>> URIs), not multiple variants of the same resource to me.
>>>
>>> avoid creating the temporary resource, and have the option of 409'ing any
>>> concurrent PUTs after the first), whereas with POST, you're just
>>
>> isn't the case, unless the client can be confident that the server actually
>> understands PUT with ranges.
>
> That seems to be a problem with all the approaches on the table, according
> to the flows in the current document. By the letter of the law, if the
> server doesn't understand a Content-* header on a PUT request, it should
> refuse it, but we already have an open issue or two (#79, #102) on that...

So clearly a client should not start sending resumable PUT/POST
requests to a server that doesn't support them. The HTTP/1.0 version
of the protocol is _not_ self-discoverable. The only way we propose
self-discoverability is in the form of 103 responses sent by the
server to HTTP/1.1 compliant clients (through HTTP/1.1 compliant
proxies).

> BTW, we should really be talking about an Internet-Draft at this point,
> rather than a wiki page. Google guys, when will that happen?

At a high-level we'd been hoping to at least get some high-level
community feedback confirming that we are ready to move onto that
step. If there is agreement that it is time to proceed there then
we'll work on putting one together.

And thanks to all for all of the feedback and discussion.

Charles

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 16 Sep 2008 14:19:55 +0100

> So my concern is that we use two different types of ETags on the same 

I agree, this is a good point.

Why use the Etag header for resumable POSTs?  If Etags are used (and
why not - it's handy not to have to carve out special URL space), I
still don't see any advantage to using the ETag _header_ for it in
resumed POSTs, as opposed to inventing a Resume-POST header.

-- Jamie

---

From: "Brian McBarron" <[email-masked]>
Date: Tue, 16 Sep 2008 09:26:58 -0400

> resumed POSTs, as opposed to inventing a Resume-POST header.

Here are my use cases where ETags may not be sufficient, restated from
earlier in this thread.  I'd love for someone to explain why these aren't
issues, since I also like the ETag approach.

---

From: "Charles Fry" <[email-masked]>
Date: Tue, 16 Sep 2008 09:31:33 -0400

> we'll work on putting one together.

But to be explicit here, we are trying to balance our internal
deployment and experience with this protocol with the external
discussion, which is just to say that some additional time will
clearly go by before we actually complete the lofty goal of
transforming this into an Interntet Draft. That said, we clearly plan
to mine this thread for feedback as we undertake that process, so this
is not a lost discussion. :-)

Charles

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 16 Sep 2008 14:46:32 +0100

Brian McBarron wrote:
> Here are my use cases where ETags may not be sufficient, restated
> from earlier in this thread.  I'd love for someone to explain why
> these aren't issues, since I also like the ETag approach.
>
> 1) A server farm where the majority of traffic is handled by
> essentially diskless front-ends.  In order to support resumability,
> a server would need to redirect the client to a separate pool of
> servers who are provisioned to do the necessary caching.
>
> 2) A server farm where requests are load balanced across a pool of
> servers.  Even if the load-balancing mechanism implements a
> reasonable affinity mechanism, it's likely that that mechanism is
> just an approximation, and that any particular request may end up on
> a variety of machines.  I can imagine that an efficient
> implementation of resumability would be to ensure the user connects
> to the same machine, where the request may be cached on local disk.
> Therefore, the resumable URI could be the name of that machine,

These aren't a problem, as long as the front-end redirector is able
to select targets according to the Resumable-POST Etag as well as the URL.

If the front-end redirection is still approximate, there would need to
be back-end redirection too, or shared storage for partial uploads.
Lots of other variations are possible.  It's not hard to think of solutions.

It's probably useful to structure the Resumable-POST Etag to assist
front-end rediration,
e.g. "upload-foo-server1.example.com-54069843-50395-309534-059345".
How it's structured is entirely a server-side implementation detail
and doesn't need to be specified in an RFC.  Also it's not strictly
necessary, it just might make some implementations easier.  It could
also be concealed and/or signed cryptographically, if useful.  Or it
could just be a hash, with a few bits used to select upload server.

-- Jamie

---

From: "Brian McBarron" <[email-masked]>
Date: Tue, 16 Sep 2008 09:58:28 -0400

> could just be a hash, with a few bits used to select upload server.

Requiring a header-inspecting load balancer and/or implementing a
multi-layer serving architecture seems like a steep cost to get into this
protocol.  Especially when explicit URI redirection would solve the issue
for no added cost.

---

From: Stefan Eissing <[email-masked]>
Date: Tue, 16 Sep 2008 16:03:11 +0200

> resumed POSTs, as opposed to inventing a Resume-POST header.

Not that it is important, but *I* do not understand why there is a  
discussion about a new protocol feature when URI space management is  
able to solve the problem most elegantly. URI usage seems even easier  
for clients, so the burden is on the server implementation where  
there are not so many.

The use case I heard about is adding resumable upload to a cloud at  
the very edge by some sort of server proxy. Such a proxy is not  
master of the served namespace and cannot simply invent new URIs. So  
such proxy would need additionally a namespace of their own, but that  
does not sound like a show stopper. It would allow for load  
balancing, as was already mentioned here.

If someone would be so kind as to help me overcome my ignorance in  
this matter? Thanks.

//Stefan

---

From: Jamie Lokier <[email-masked]>
Date: Tue, 16 Sep 2008 17:07:40 +0100

Brian McBarron wrote:
> multi-layer serving architecture seems like a steep cost to get into
> this protocol.  Especially when explicit URI redirection would solve
> the issue for no added cost.

There is a cost, because you may have to change your infrastructure to
support URLs directly addressing individual servers within a cluster,
and you have to add a different kind of load-balancing to handle when
individual servers are taken down in the cluster so another server can
take over those URLs for the clients still using them.

But I agree that you might be able to use existing URL-based front-end
load-balancing kit, and that might reduce the cost of implementation
substantially.

-- Jamie

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 17 Sep 2008 07:58:47 +1000

There's a more practical aspect to this; some people are going to be  
wary of giving you feedback until it's surrounded by the template that  
says you agree to disclose any IP you become aware of immediately.

It can still change when it becomes an I-D; the important part is that  
there are people willing to engage and who think this is a useful  
thing to be doing.

Cheers,

On 16/09/2008, at 9:22 PM, Charles Fry wrote:

>> point,

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 17 Sep 2008 08:09:04 +1000

> load-balancing kit, and that might reduce the cost of implementation

Is there a hidden assumption there that this is done in hardware?

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 17 Sep 2008 10:11:34 +1000

On 16/09/2008, at 9:22 PM, Charles Fry wrote:

>>> really
>> according

Well, HTTP/1.0 doesn't define PUT; it first shows up in RFC2068, which  
says:

> the recipient of the entity MUST NOT ignore any Content-* (e.g.  
> Content-Range) headers that it does not understand or implement and  
> MUST return a 501 (Not Implemented) response in such cases.

(as 2616 does).

So, in theory you can PUT with Content-Range and know that if the  
server doesn't support resumable requests, you'll get a 501. In  
practice, of course, may be a completely different kettle of fish.

Cheers,

--
Mark Nottingham     http://www.mnot.net/

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Tue, 16 Sep 2008 19:21:56 -0500

Mark Nottingham wrote:
> So, in theory you can PUT with Content-Range and know that if the server 
> doesn't support resumable requests, you'll get a 501. In practice, of 
> course, may be a completely different kettle of fish.

The reason this isn't a solution is that the server must swallow the body
of the request or lose keepalive and pipeline optimizations; 501 isn't an
intermediate response.

I wonder if a new 100-class code, the inverse of CONTINUE, wouldn't be of
value in HTTP/next to designate that a the request body can be omitted since
a final determination is available.  The client would still have to send a
body (since there is no ack) but using a chunked send, a chunk of 0 bytes
to finalize an empty request body would be sufficient.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 17 Sep 2008 10:43:33 +1000

On 17/09/2008, at 10:21 AM, William A. Rowe, Jr. wrote:

>>> Content-Range) headers that it does not understand or implement  
>>> and MUST return a 501 (Not Implemented) response in such cases.
>> (as 2616 does).

Sounds nice, but I don't know that the introduced complexity is worth  
keeping connections open in a fail case; implementers already seem to  
have enough trouble with continue...

Cheers,

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Tue, 16 Sep 2008 20:27:07 -0500

> have enough trouble with continue...

:)

Speaking of which, if 99.9% of the client implementors finally get their
expect: 100-continue behavior correct by HTTP/next, it really wouldn't
be impossible to suggest that this status for an HTTP/next request line
must honor a 105 ABORT BODY response, even for a C-L: nnn based request.

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 17 Sep 2008 12:25:35 +0100

> Is there a hidden assumption there that this is done in hardware?

No.  By "kit" I mean readily available hardware and/or software, which
does URL-based affinity HTTP request forwarding for load balancing.

-- Jamie

---

From: "Charles Fry" <[email-masked]>
Date: Wed, 17 Sep 2008 07:36:05 -0400

> to finalize an empty request body would be sufficient.

But if there will be any new 1xx response code, then it should be the
103 (Checkpoint) from our proposal. Because if you can use that then
you can just include the entire body in the first post, using the 103
to obtain whatever information you need to resume upon failure (e.g.
etag or new location).

Charles

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 18 Sep 2008 08:36:48 +1000

OK. I agree that doing it in hardware is more expensive (as you have  
no choice but to involve a vendor), but doing it in software is  
relatively easy; high-performance servers that can act as  
intermediaries are getting pretty common, and pretty performant.

Cheers,

---

From: "Charles Fry" <[email-masked]>
Date: Fri, 19 Sep 2008 10:28:13 -0400

A few more thoughts on URLs and ETags:

- While it isn't clear in the examples on the wiki page, the initial
handshake is optional. In other words, it represents one extreme end
of the protocol, but the primitive requirement is that unique
identifier (whether URL or ETag) be obtained. This could very well be
present in an HTML page, for example, allowing a resumable upload
without the initial handshake.

 - Regarding ETags vs. URLs, one point of reference for me is CGI
scripts. A single CGI script running behind a fixed URL should (in my
mind) be able to perform a resumable upload, but this would require a
unique identification mechanism other than the URL.

Charles

On Wed, Sep 17, 2008 at 18:36, Mark Nottingham <[email-masked]> wrote:
> OK. I agree that doing it in hardware is more expensive (as you have no
> choice but to involve a vendor), but doing it in software is relatively
> easy; high-performance servers that can act as intermediaries are getting
> pretty common, and pretty performant.
>
> Cheers,
>
>

---

From: Julian Reschke <[email-masked]>
Date: Fri, 19 Sep 2008 16:34:28 +0200

> unique identification mechanism other than the URL.

Really? How about a query parameter?

BR, Julian

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 19 Sep 2008 09:01:48 -0700

> unique identification mechanism other than the URL.

That isn't relevant -- any CGI script (and just about any part of a
server's URI space) can be overlapped as many times as you want.
In any case, ETag is not a unique identifier.

I already explained how to do this functionality in
<mid:[email-masked]> on Apr 4.

The ETag examples you provide don't make any sense.  Etags only
have meaning within the scope of a resource's current representations.
If you are posting to a service that has no corresponding resource
mapping, the etag in response must correspond to the etag that you
would have received if the request were a GET on that service.
In essence, this allows a service to be versioned, though such
interfaces are not typically used on the Web because we prefer to
use backwards compatible services rather than strict version coupling.

So, the short answer is that only the URI solution fits within
the current web architecture.

....Roy

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 19 Sep 2008 18:27:00 +0100

> Really? How about a query parameter?

Or PATH_INFO, since we're talking CGI.  CGI scripts receive a whole
subspace of URLs to play with as they like.

-- Jamie