Subject: Re: HTTP Partial Download Query

From: "Yogesh Bang" <[email-masked]>
Date: Mon, 9 Dec 2002 10:58:30 +0530

Dear Scott and Patrick,

Firstly Many thanks for your quick and precise inputs on this query.
But still I have following doubts.

1)With your replies it seems that its the responsibility of the BROWSER
to make the MULTIPLE GET request to get a LARGE OBJECT(lets say MOBILE RING TONES) 
by using the RANGE header of HTTP request and the server merely send those 
requested BYTES.
In such case how one can track if the download is complete or not?
How to track that the request is coming from same authenticated user?
I know that I can extract HTTP header information in FILTERS(in IIS terminology) or 
MODULES(in Apache terminolgy) but any other mechanism will be highly appreciated.

2)If you refer to section 19.2 of RFC 2616(http://rfc-2616.rfclist.com/rfc-2616-165.htm),
it depicts a scenario of HTTP Partial download. In this scenario the requested 
object is send as MULTIPART/BYTERANGES type seperated by a seperator.
It seems it is a SINGLE response send as MULTIPART with each part cut into small
chunks (which can be accepted by browser.). 
If this is the way HTTP PARTIAL DOWNLOAD works then to implement this 
functionality what care a Web-Programmer needs to take ?
Does he need to write a seperate DOWNLOAD program which will cut the requested object
into chunks ,put the seperators and send it to browser as single response?

Please clear me if I am wrong in my understanding.

Thanks once again for your valuable time.

With Regards,
Yog

--------------------------------------------------------------------------------
Previous Communication related to HTTP PARTIAL DOWNLOAD
--------------------------------------------------------------------------------
> I was browsing through RFC 2616 for HTTP1.1
> One of the smart feature it mensioned was of PARTIAL DOWNLOAD.
> Means browser can specify a range of bytes (e.g 100-200 bytes of img123.gif)
> in the HTTP header and webserver will send only those bytes.
> 
> I have following query about PARTIAL DOWNLOAD
> 
> If the browser is capable of accepting say 100KB at a time (This 
> is most likely scenario in case of browsers on Mobile phones)
> then is it browser's responsibility to make request for every 100KB 
> of that object(which is to be downloaded) and the webserver will send requested 
> bytes? 
> OR
> Is it the server who will send the requested Object in chunk of 100KB
> with seperator and as Multipart response?

your first scenario is much closer.. the server will send only a
subset of the document.. if the client wants other subsets it must
make multiple requests. Also note that the server is allowed to just
send you the whole document even if you only ask for part of it.

imho, your issue appears to be one of flow control, which isn't really
what the range mechanism was meant for. (but then I'm not one to talk,
I've done an experiment to try and interleave different documents from
the same web page across the same persistent tcp stream using range).You should
really use your transport protocol to accomplish what you're
after. (e.g. in TCP advertise a 0 rwin and you won't get any more data
until you've consumed the last chunk.)

-Patrick
--------------------------------------------------------------------------
The client (browser) may request a specified range of the object; if
it wants another range, it must make another request.  The request is
made using the Range request header (see section 14.35 of RFC 2616).

Note however, that since a server MAY ignore a Range request header
and return the entire resource, a client that has a limited capacity
cannot always rely on getting a subrange response.  The Content-Range
header is included in the response if it is a subrange.

There is a mechanism (the 'Accept-Ranges' response header) designed to
allow the server to declare whether or not it will support range
requests for a resource - I don't know how widely it is implemented.
If it is implemented by the server, then an OPTIONS or HEAD request to
the URL of interest would return an Accept-Ranges header, without
having to deal with the entire object.

  Scott Lawrence
---------------------------------------------------------------------------

---

From: Alex Rousskov <[email-masked]>
Date: Mon, 9 Dec 2002 10:15:04 -0700 (MST)

On Mon, 9 Dec 2002, Yogesh Bang wrote:

> 1) With your replies it seems that its the responsibility of the
> BROWSER to make the MULTIPLE GET request to get a LARGE OBJECT(lets
> say MOBILE RING TONES)  by using the RANGE header of HTTP request
> and the server merely send those requested BYTES.

Allow me to outline client-side and server-side options to clarify the
picture:

	Client option 1: send a simple GET request for the entire
	object and rely on transport mechanism (e.g., TCP buffer
	sizes) to limit sizes of incoming response chunks. This
	is probably "The Right Way" to do it as previous replies on
	this thread indicated.

	Client option 2: send a sequence of Range GET requests,
	each for a single subrange of the response small enough to
	satisfy your buffering requirements. This option relies on the
	server (and all HTTP intermediaries on the path!) supporting
	Range requests and not sending a complete response regardless
	of the Range header. Thus, using this option does not really
	make any practical sense unless you control the server and
	all HTTP intermediaries on the path.

	Client option 3: send a single Range GET request with several
	subranges, each small enough to satisfy your buffering
	requirements. Rely on transport mechanism to limit sizes of
	incoming response chunks. Note that the client may or may not
	receive multipart/byteranges response, depending on whether
	the server can handle Range requests and whether the server
	will merge overlapping subranges and realize that they are
	equivalent to the whole object. Thus, using this option does
	not really make any practical sense unless you control the
	server and all HTTP intermediaries on the path.

	Server option 1: ignore Range header and send complete
	response.

	Server option 2: honor Range request and send just the
	requested range(s). Use multipart/byteranges if there
	is more than one range requested.

	Server option 3: honor Range request and merge requested
	ranges if possible. If the result is an entire object,
	send a normal response as if the Range header was ignored.

If you look at all the options above, you will realize that you cannot
rely on Range requests to limit transfer chunks _unless_ you control
the server and all intermediaries.

> In such case how one can track if the download is complete or not?

The client would know if it received the whole object because a
partial response contains the content length of the complete response.
The server cannot reliably aggregate individual request sizes for
reporting unless server authentication is used and request threads for
the same URL do not overlap. If server authentication is used, one can
aggregate sizes of individual ranged responses on a per-user per-URL
basis.

> How to track that the request is coming from same authenticated

Each request should have authentication information attached, of
course.

> 2)If you refer to section 19.2 of RFC 2616, it depicts a scenario of
> HTTP Partial download. In this scenario the requested object is send
> as MULTIPART/BYTERANGES type seperated by a seperator. It seems it
> is a SINGLE response send as MULTIPART with each part cut into small

Yes, in general, you get a single response for a single request
(regardless of Range header presence or value). If you want multiple
responses, you have to send multiple requests (probably with different
Range headers).

> If this is the way HTTP PARTIAL DOWNLOAD works then to implement
> this functionality what care a Web-Programmer needs to take ? Does
> he need to write a seperate DOWNLOAD program which will cut the
> requested object into chunks ,put the seperators and send it to
> browser as single response?

That would be one option. It is difficult to give a specific design
advice without knowing a lot more about your environment. Please note
that most decent HTTP servers (e.g., Apache) support Range requests
for static objects. Please also note that an HTTP intermediary can
change a Range request into a simple GET.

Overall, just based on the information you gave us so far, it seems
that using Ranges is the wrong mechanism to limit response chunks in a
general Web environment. I would expect the transport mechanism (e.g.
TCP) in the browser to handle browser buffering limitations.

Thanks,

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance