Subject: HTTP PUT method variations

From: Michel Drescher <[email-masked]>
Date: Thu, 17 Feb 2005 14:26:21 +0000

Dear working group,

I am exploring the capabilities of the HTTP protocol for implementing 
parallel data distribution using established protocols.

Studying the HTTP 1.1 specification (RFC 2616), figuring out how this 
could work using HTTP GET method is rather easy:

- For as many parallel data streams you want, open a connection to the 
desired server for each of the streams
- On each of these connections, emit a HTTP GET request carrying each 
time the same URI (and the usual common headers), but each carrying
a "Range:" header, specifying a different range of the requested resource.

These requests using 3 parallel data streams could look like as follows 
(incomplete for brevity) for a resource of 10,000 bytes:

Request 1:
     GET /some/resource HTTP/1.1
     Range: bytes=0-3332
     ...

Request 2:
     GET /some/resource HTTP/1.1
     Range: bytes=3333-6665
     ...

Request 3:
     GET /some/resource HTTP/1.1
     Range: bytes=6666-9999
     ...

The answers I can expect to these requests would look like this 
(assuming the resource exists and can be served etc.):

Response 1:
     HTTP/1.1 206 Partial Content
     Content-Length: 3333
     Content-Range: bytes 0-3332/10000
     ...

     [the bytes of the resource as requested]

Response 2:
     HTTP/1.1 206 Partial Content
     Content-Length: 3333
     Content-Range: bytes 3333-6665/10000
     ...

     [the bytes of the resource as requested]

Response 3:
     HTTP/1.1 206 Partial Content
     Content-Length: 3334
     Content-Range: bytes 6666-9999/10000
     ...

     [the bytes of the resource as requested]

This is easy if I want to request some data. Variations of partial GETs, 
using multiple ranges (and receiving then a response having the header 
"Content-Type: multipart/byteranges") etc. are also fairly easy to 
figure out, thanks to the clearly written specification.

Now, for my target environment, it definitely will come handy if we also 
would be able to use HTTP PUT to upload data (resources, in general HTTP 
speak) to a server. Of course, one can model this using HTTP GET in a 
callback environment: Server A, instead of directly uploading the data 
to Server B, instructs Server B to download the data from Server A. On 
the other hand, using the direct way HTTP PUT greatly reduces complexity 
in this scenario.

I know there are serious security considerations with HTTP PUT, but be 
assured that the target environment uses appropriate mechanisms to 
secure the whole transaction.

So I digged into the details of HTTP PUT.

Obviously, it is fairly simple to upload a whole resource at once using 
HTTP PUT. My confusion starts when I wanted to upload fractions of a 
resources to establish an upload way symmetric to the parallel download 
way as described above, especially because the two most important 
headers in this case, "Range:" and "Content-Range:", are wholly 
described from the viewpoint of *downloading* data with HTTP GET.

Basically, I can see two scenarios here, depending on whether certain 
HTTP headers are allowed in an HTTP PUT request or not:

The header "Range:" is clearly a request header and "Content-Range:" an 
entity header. In a GET request, the "Content-Range:" header doe not 
make any sense, since there is no entity to describe in the GET message 
body. In a PUT request, the descriptions are ambiguous (I would've loved 
to read a section "14.35.3 Range Upload Requests"), since both "Range:" 
and "Content-Length:" are possible, maybe even allowed.

The scenarios I can see here are thus as follows, including a sample 
request/response sequence for each, using the same sample resource as above:

Scenario A: Uploading one fraction of a resource in one request
---------------------------------------------------------------
I am pretty sure that this is covered by the specification, only the 
correct header usage is somewhat unclear. Both headers, "Range:" and 
"Content-Length:" are appropriate here in the request header section, so 
two, possibly three, alternative request methods are as follows, with 
the same response (i.e. either a "200 OK" or a "201 Created" response, 
possibly a "203 Accepted" or "204 No Content"):

Alternative 1:
     PUT /some/resource HTTP/1.1
     Range: bytes=0-3332
     Content-Length: 3333
     ....

     [bytes 0 to 3332 of the resource]

Alternative 2:
     PUT /some/resource HTTP/1.1
     Content-Range: bytes 0-3332/10000
     Content-Length: 3333
     ....

     [bytes 0 to 3332 of the resource]

Alternative 3:
     PUT /some/resource HTTP/1.1
     Range: bytes=0-3332
     Content-Range: bytes 0-3332/10000
     Content-Length: 3333
     ....

     [bytes 0 to 3332 of the resource]

Note: The requests alternatives for the second and third part of the 
resources are left as an exercise to the reader.
Note 2: Again, the samples are kept short for brevity.

Scenario B: Uploading two or more fractions of a resource in one request
------------------------------------------------------------------------
Although this scenario is of no direct consern to me at the moment, I 
still would like to see this clarified.
In this case, the "Content-Range:" header is clearly not appropriate in 
the request header, since "Content-Range:" can only specify one range. 
Instead, the ranges might be specified using the "Range:" header, and 
the entity described as "Content-Type: multipart/byteranges". The entity 
in the message body itself then is split into several parts, 
corresponding to the amount of byte ranges specified.

So the only alternative I can see here is as follows:

     PUT /some resource HTTP/1.1
     ...
     Range: bytes=0-3332,3333-6665,6666-9999
     Content-type: multipart/byteranges; boundary=THIS_STRING_SEPARATES

     --THIS_STRING_SEPARATES
     ...
     Content-range: bytes 0-3332/10000

     [the bytes of the first part]
     --THIS_STRING_SEPARATES
     ...
     Content-range: bytes 3333-6665/10000

     [the bytes of the second part]
     --THIS_STRING_SEPARATES--
     ...
     Content-range: bytes 6666-9999/10000

     [the bytes of the third and last part]
     --THIS_STRING_SEPARATES--

Note: Again, the sample is kept short for brevity.

--

Now, my overall questions are as follows:

Q1: Is Scenario 1 covered by RFC 2616?
Q2: If the answer to Q1 is yes, which alternatives are correct?
Q3: Is Scenario 2 covered by RFC 2616?
Q4: Is there anything wrong with my examples?
Q5: Would you please not try to kill me for asking about HTTP PUT? ;-)

Cheers,
Michel Drescher

---

From: Scott Lawrence <[email-masked]>
Date: Thu, 17 Feb 2005 13:02:17 -0500

> parallel data distribution using established protocols.

As you correctly illustrated, HTTP can (and is) used to 'pull' data in
parallel requests.

Trying to overload PUT in the way you describe is more or less sure to
fail as soon as you encounter a server that doesn't understand it.  That
being the case, you are better off defining a body type for what you
want to do and using POST.  

-- 
Scott Lawrence <[email-masked]>
http://skrb.org/scott/

---

From: Michel Drescher <[email-masked]>
Date: Thu, 17 Feb 2005 18:15:22 +0000

> want to do and using POST.  

It is a controlled environment - we provide both clients and server, so 
we can make the server understand these requests.

All I (more or less) wanted to know is that if my interpretation of the 
RFC is valid (so we would be fully compliant) or not.

Thanks for the answer.

Cheers,
Michel

---

From: Jamie Lokier <[email-masked]>
Date: Thu, 17 Feb 2005 18:41:50 +0000

> RFC is valid (so we would be fully compliant) or not.

Do any of the widely-used servers offer "resume upload" functionality?
If so, they might have defined a syntax for range PUT that you can
check you are compatible with.

-- Jamie

---

From: Scott Lawrence <[email-masked]>
Date: Thu, 17 Feb 2005 15:05:02 -0500

> we can make the server understand these requests.

Then why use HTTP at all?

> RFC is valid (so we would be fully compliant) or not.

No - you are assuming constraints in server implementations that the RFC
does not require them to observe.