Subject: remove/deprecate User-Agent header?

From: Paul Wise <[email-masked]>
Date: Thu, 19 Aug 2010 11:31:38 +0000

Hi all,

I found out about httpbis and the first thing I thought of was my hatred
of the User-Agent header. Would it be possible to make it illegal in
HTTP/1.2 so that there are less stupid websites? To see just how much
FAIL User-Agent has created, check out some of these strings:

http://www.zytrax.com/tech/web/browser_ids.htm#chrome
http://www.zytrax.com/tech/web/browser_ids.htm#epiphany
http://www.zytrax.com/tech/web/browser_ids.htm#gnuzilla
http://www.zytrax.com/tech/web/browser_ids.htm#galeon
http://www.zytrax.com/tech/web/browser_ids.htm#mozilla

Examples of stupid websites:

https://bugzilla.gnome.org/show_bug.cgi?id=592665
http://blog.kov.eti.br/?p=119
http://glandium.org/blog/?p=145
http://home.kairo.at/blog/2007-06/the_fight_for_the_suckiest_ua_string
https://bugzilla.mozilla.org/show_bug.cgi?id=385999
https://bugzilla.mozilla.org/show_bug.cgi?id=384721
http://svn.gnome.org/viewvc/epiphany/branches/gnome-2-18/data/weasel-ua-pref.js.in?view=markup&pathrev=7101

-- 
bye,
pabs

http://bonedaddy.net/pabs3/

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 Aug 2010 14:38:59 +0200

> HTTP/1.2 so that there are less stupid websites? To see just how much

Clarifying: we aren't defining HTTP/1.2.

> http://svn.gnome.org/viewvc/epiphany/branches/gnome-2-18/data/weasel-ua-pref.js.in?view=markup&pathrev=7101

I agree that many uses of U-A are bad. That doesn't mean there aren't 
good ones.

Independently of that, what *effect* do you think making it "illegal" has?

Best regards, Julian

---

From: Paul Wise <[email-masked]>
Date: Thu, 19 Aug 2010 12:57:53 +0000

> Clarifying: we aren't defining HTTP/1.2.

Hmmm, Ok.

> That doesn't mean there aren't good ones.

Do you have any examples? I'm unable to think of any.

> Independently of that, what *effect* do you think making it "illegal" has?

Hopefully to stop user-agents from sending the header at all, servers
from providing access to it in their logs or environment variables and
web applications and sites from using it in hacks to discriminate
against users of browsers from the wrong vendor.

If you mean at the protocol level, I would suggest servers return 400
Bad Request when a client sends User-Agent.

-- 
bye,
pabs

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 Aug 2010 15:12:39 +0200

On 19.08.2010 14:57, Paul Wise wrote:
> ...
> ...

Sometimes it's necessary to do the Right Thing, although some UAs break 
when you attempt to.

For instance, using RFC 2231 / RFC 5987 encoding in Content-Disposition 
(fails with IE/Safari/Chrome), or when sending a proper Vary response 
header (which causes all kinds of trouble in old IE versions).

> Bad Request when a client sends User-Agent.

U-A is widely used for working around UA bugs. If we made it illegal we 
probably would need to replace it with something that provides that 
functionality. I'm not sure how that would be different from what we 
have today, nor how to deploy it in practice.

Best regards, Julian

---

From: J Ross Nicoll <[email-masked]>
Date: Thu, 19 Aug 2010 14:20:12 +0100

> Do you have any examples? I'm unable to think of any.

The button tag in IE 6 is handled differently to IE 8 (unsure about IE 7). Under IE 6, the button tag cannot be used to submit a form (well, not without using some JS and onclick), while under IE 8 it actually follows the specification. We have an application that generates HTML for really nice big buttons with images and text in them under IE 8, Firefox, Chrome, etc. and a standard old <input type="submit"... button for IE 6 (and 7, because we weren't sure). It decides which to generate based on the UA string.

Less critically, we also decide whether to put HTML 5 date fields into pages based on the UA. I know they're meant to fall back onto text entry inputs elegantly, however I don't think our users would agree that "1979-10-14T12:00:00.001-04:00" is exactly a date format they're used to, so we only put HTML 5 date formatting in for browsers that will handle the fields correctly and display a usable UI.

That's just cases where we use the user agent string to work around differences, I'm sure there are plenty of others.

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 19 Aug 2010 22:47:07 +0200

> Do you have any examples? I'm unable to think of any.

There are people in the finance domain who use it to track automated
fraud attempts that are often performed by malware running inside the
browser and acting without the users' consent. Without giving details,
basically keep in mind that it's a browser signature that has no reason
to change within an application session.

Also, what causes the most UA-related breakage you often see on the web
in my opinion are the checks that are performed within the browser itself
using JS and that are never updated to consider fixed browsers. And you
won't fix that by removing the UA header.

> against users of browsers from the wrong vendor.

That will not change anything. If you suggest to remove something which
has some uses, people will continue to use it as before. Just see how
cookies are used despite the two RFCs that tried to restrict them.

> Bad Request when a client sends User-Agent.

What will happen then is that noone will enable this option because
admins want to get visits and not to kick valid visitors off of their
site.

Regards,
Willy