Subject: HTTP request header truncated

From: Patrick Thun <[email-masked]>
Date: Thu, 14 Aug 2003 12:17:47 -0400 (EDT)

Hi, my name is patrick. I'm a student doing a project on embedded system. I've wrote a httpd for the embedded system. I've added Digest authentication to the httpd for access control of the web resources. 

When I test the httpd with Mozilla and Navigator 7.1 in redhat 8, everything works fine.When I test it with IE5, IE6 and navigator 7.1 on win98, my Digest authentication fails. I used a HTTP sniffer to trace
the HTTP transaction and discovered that the Authorization header field in the request were truncated. The Authorization header was the last header in the request.

My httpd only supports a subset of the HTTP1.1 headers. Is there anyway or method to make the browser send only the authorization header and on others like Accept, Accept-language etc?

Yahoo! Games
- Who Wants to Be A Millionaire? Play now!

---

From: Alex Rousskov <[email-masked]>
Date: Fri, 15 Aug 2003 16:00:17 -0600 (MDT)

On Thu, 14 Aug 2003, [iso-8859-1] Patrick Thun wrote:

> When I test the httpd with Mozilla and Navigator 7.1 in redhat 8,
> everything works fine.When I test it with IE5, IE6 and navigator 7.1
> on win98, my Digest authentication fails. I used a HTTP sniffer to
> trace the HTTP transaction and discovered that the Authorization
> header field in the request were truncated. The Authorization header
> was the last header in the request.

I assume you mean that the HTTP header did not fit into a single TCP
packet (was split across several packets), and that your program does
not handle partial reads from the network socket.

> My httpd only supports a subset of the HTTP1.1 headers. Is there
> anyway or method to make the browser send only the authorization
> header and on others like Accept, Accept-language etc?

Not in general. You can install a header-stripping proxy in front of
your server, but that is probably not what you are after. If your
httpd cannot handle partial reads, you should fix it.

Also, please note that the browser is not the only agent that can add
HTTP headers to requests. HTTP intermediaries such as caching proxies
do that as well, further increasing header length.

HTH,

Alex.

-- 
                            | HTTP performance - Web Polygraph benchmark
www.measurement-factory.com | HTTP compliance+ - Co-Advisor test suite
                            | all of the above - PolyBox appliance

---

From: "Wilfred Nilsen" <[email-masked]>
Date: Thu, 21 Aug 2003 14:30:44 -0700

I have a page that is generated dynamically and I therefore set the
"Cache-Control: No-Cache" header. The page contains a meta refresh tag
and is automatically polled every so often.

The page contains links to images that never change so in order to
prevent the browser from attempting to fetch new versions of the
images, I set the header value "Cache-Control: max-age=86400". The
problem is that the browsers I have tested this with, IE 6.0 and
Mozilla 1.13, seems to ignore the max-age header value and are always
checking for newer versions. The browsers are set to get a new page
when the current page is out of date.

I also tried to set the "Expires" header but that did not help either.

Thanks,
Wilfred

---

From: Alex Rousskov <[email-masked]>
Date: Sat, 23 Aug 2003 10:42:18 -0600 (MDT)

Do the browsers send If-Modified-Since (IMS) requests or regular GETs
when refreshing the images? If they send IMS requests, you may be out
of luck. If they send regular gets, you may want to experiment with
image headers some more: try adding Cache-Control: public as the first
Cache-Control header or removing all Cache-Control headers and adding
Expires header instead.

BTW, please keep in mind that a Cache-Control: no-cache does not mean
that that cache cannot store (cache) the object. It means the cache
has to revalidate it every time (with an IMS request, for example).

HTH,

Alex.

---

From: Robert Olofsson <[email-masked]>
Date: Sat, 23 Aug 2003 20:06:04 +0200

>when the current page is out of date.
For IE have you enabled the HTTP/1.1 extensions?
(I guess that you have, but check it please).

For Mozilla I think that you should try the 1.4 release since that is
stable version for now.
Have you checked bugzilla.mozilla.org for bugs in mozilla regarding
this?

How big are the images?
How big cache do you have?
Depending on the two above questions the image may be deleted from the
cache directly if the image is too big. I have not studied the
algorithms used for mozilla or IE when it comes to cache cleaning so
I can't say if your images are likely to be killed directly.

Are your images the result of a POST (I guess that they are not).
Are your images the result of a GET with a query string?
GET /image?num=b HTTP/1.1
query string can complicate matter:
"caches MUST NOT treat responses to such URIs as fresh unless
 the server provides an explicit expiration time" (RFC2616).
It does seem like you do provide expiration time so that should not
be a problem.

Do the browsers send nice requests that get a 304 Not Modified as response
or do they send plain GETs?

Also note that you can not force a user to have his cache enabled so that
the browser check/get the image every time is not a violation of the HTTP
specification.

/robo

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 23 Aug 2003 13:00:44 -0700

Keep in mind that caches aren't obliged to cache when headers allow it; 
they may have strategies that they believe work better.

In my experience, browser caches are pretty simplistic, and very 
conservative. They may have the images inheriting freshness information 
from the page they're referenced from, or they may interpret the 
refresh tag as a forced refresh for everything on the page.

Cheers,

---

From: "Wilfred Nilsen" <[email-masked]>
Date: Sun, 24 Aug 2003 19:20:35 -0700

> refresh tag as a forced refresh for everything on the page.

I believe you are right.  I did a few tests and it seems that a
browser sends a new GET request if you press the reload button and
that includes all images in the page, which seems to inherit the
refresh from the page.  Looks like the browser always bypasses the
cache when the reload button is pressed, no matter the content of any
cache headers.  I presume that a page with a "meta refresh" tag
behaves as if the reload button was pressed.

Thank you Mark, Robert and Alex for your comments.

/Wilfred