Subject: Re: Server-side roles in the HTTP

From: [email-masked]
Date: Wed, 1 Sep 1999 13:36:00 -0600

> Servers MUST support a persistent connection if the content
>   generator supplies a Content-Length header. If it is not available,
>  the server SHOULD attempt to buffer the response in order to
>   generate one, although this MAY be circumvented if:
>   * the server does not have resources (i.e., memory) to do so, or
>   * the object is very large, and overall latency becomes
>   unacceptable, or
>   * the time required to generate the object adds unacceptable latency
...
>  Servers MUST serve chunked encoding responses for all objects, if:
>   * Content-Length is unavailable and impractical to generate
>   and
>   * the client advertises itself as HTTP/1.1 capable, or
>   * the client includes 'chunked' in a TE header

Why wouldn't streaming the response with the "chunked" encoding be preferable to
buffering and generating a Content-Length?  I don't think the server should
attempt to buffer the response unless it's talking to an HTTP/1.0 client.

> 6.4 Transfer-Encoding

>   Servers SHOULD reply to requests that allow transfer encoding of
>  objects (i.e., TE header present) with appropriate encoding, in a
>  fashion transparent to the content generator.

This paragraphs seems to suggest that transfer encoding is only allowed if the
TE request-header is present.  However, if the TE field-value is empty or if no
TE field is present, the transfer-coding "chunked" is allowed, and is always
acceptable.

>   There MUST be a method by which content generators can specify that
>   content is not to be buffered; this MAY be performed by a pseudo-
>   HTTP header that is consumed by the server.

I think you're only addressing generated (response) content in this statement.
I'd also like to have a method by which content generators can specify that the
received (request) content is not to be buffered.   Sometimes the "content
generator" can be doing something useful with the content of a large POST
request while it's still being received (or even while it's still being
generated by the client).  In some applications (e.g., IPP), POST content may be
so large that you don't even want the server to attempt to buffer it.

     -Carl Kugler

---

From: "Mark Nottingham" <[email-masked]>
Date: Thu, 2 Sep 1999 08:55:27 +1000

Carl,

Thanks for that. I should clarify in the draft that it's aimed at 1.1 AND
1.0 connections (hence, buffering and so forth).

> attempt to buffer the response unless it's talking to an HTTP/1.0 client.

That's what is intended; will clarify.

> TE field is present, the transfer-coding "chunked" is allowed, and is
always

Intention here was more on the compress, deflate, gzip side than chunked.
perhaps
(e.g., compression-related TE request headers)
instead of
(i.e., TE header present)
?

> so large that you don't even want the server to attempt to buffer it.

Ah, interesting. yes, the request side would be good to put in there - will
reread the IPP> chunked post messages.

I hestiated to put in actual suggestions for buffering thresholds.
Suggestions?

Thanks,

---

From: [email-masked] (Julien Pierre)
Date: Wed, 01 Sep 1999 17:04:13 -0700

Hi,

[email-masked] wrote:

> buffering and generating a Content-Length?

It depends on the size of the object and how long it takes to generate the content.
If the content is small and generated quickly, it makes sense for the server to
buffer the response and transparently add a content-length header, as opposed to
using chunking. Chunking for small objects will waste bandwidth unnecessarily and
provide no benefits if the full content is immediately available.
If the application generates too much content to buffer, or ends up taking too long
to generate little content, then the server can choose to chunk.
But this may be a matter for webmasters do decide depending on their applications,
how much bandwidth they have, and how much latency they can accept, so we are making
those time and buffer size thresholds configurable. We may also add settings for the
minimum size of the chunks themselves to conserve bandwidth.

> TE field is present, the transfer-coding "chunked" is allowed, and is always

Good point - looks like Mark should add a provision for this case.

> so large that you don't even want the server to attempt to buffer it.

This won't work for some content generators.
For example, you can't start a CGI before you know the length of the POST data
because you have to set it in the CGI process environment variables.

--
for a good time, try kill -9 -1