Subject: NEW ISSUE: Use of "Client" in 14.4

From: "Travis Snoozy (Volt)" <[email-masked]>
Date: Thu, 4 Jan 2007 12:26:25 -0800

Section 14.4, page 105, reads as follows:

   As intelligibility is highly dependent on the individual user, it is
   recommended that client applications make the choice of linguistic
   preference available to the user. If the choice is not made available,
   then the Accept-Language header field MUST NOT be given in the request.

Note that *user agents* are really the only clients in the position to make
this choice readily available to users. A strict interpretation of this
would mean that other non-user-agent clients (e.g., proxies) "MUST NOT" give
an Accept-Language header in their requests, because they can not (easily)
give users a "choice of linguistic preference". That could lead to all sorts
of silliness (e.g., proxies stripping Accept-Language before forwarding)
that very likely isn't intended.

I propose the following fix:

   As intelligibility is highly dependent on the individual user, it is
   recommended that <del>client applications</del><ins>user agents</ins>
   make the choice of linguistic preference available to the user. If the
   choice is not made available, then the Accept-Language header field MUST
   NOT be given in the request.

Thoughts?

-- Travis

---

From: Paul Leach <[email-masked]>
Date: Thu, 4 Jan 2007 12:53:42 -0800

I don't think its worth making that change. 

We need to be considerate of the people reading the updated spec and
comparing it with the old one to see if they need to do anything. The
diffs should be minimized, and since this one won't actually cause
anyone to do anything to their code, it might just as well be omitted.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Travis Snoozy (Volt)
Sent: Thursday, January 04, 2007 12:26 PM
To: [email-masked]
Subject: NEW ISSUE: Use of "Client" in 14.4

   As intelligibility is highly dependent on the individual user, it is
   recommended that client applications make the choice of linguistic
   preference available to the user. If the choice is not made
available,
   then the Accept-Language header field MUST NOT be given in the
request.

Note that *user agents* are really the only clients in the position to
make
this choice readily available to users. A strict interpretation of this
would mean that other non-user-agent clients (e.g., proxies) "MUST NOT"
give
an Accept-Language header in their requests, because they can not
(easily)
give users a "choice of linguistic preference". That could lead to all
sorts
of silliness (e.g., proxies stripping Accept-Language before forwarding)
that very likely isn't intended.

   As intelligibility is highly dependent on the individual user, it is
   recommended that <del>client applications</del><ins>user agents</ins>
   make the choice of linguistic preference available to the user. If
the
   choice is not made available, then the Accept-Language header field
MUST
   NOT be given in the request.

Thoughts?

-- Travis

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Thu, 04 Jan 2007 15:11:08 -0600

> I don't think its worth making that change. 

Agreed.

> anyone to do anything to their code, it might just as well be omitted.

Moreso...

> A strict interpretation of this would mean that other non-user-agent 
> clients (e.g., proxies) "MUST NOT" give an Accept-Language header in
> their requests, because they can not (easily) give users a "choice of
> linguistic preference". That could lead to all sorts of silliness (e.g.,
> proxies stripping Accept-Language before forwarding) that very likely
> isn't intended.

Well, first off, you presume the spec replaces common sense of the authors
of said proxy, but worse...

>   If the choice is not made available, then the Accept-Language header field
>   MUST NOT be given in the request.

not made available by ??...??  I presume you mean the end user of the proxy.

But if the proxy can discern a locale by, say, the regional IP assignment
of the end client, then that would be it's choice to 'fill in the gap' here.
Or a special purpose proxy could very well prefer a specific language family
based on it's anticipated user base.

I'd suggest we please leave the existing text alone and leave it to the
implementor to determine if and how Accept-Language should be presented.

---

From: "Travis Snoozy (Volt)" <[email-masked]>
Date: Thu, 4 Jan 2007 13:13:05 -0800

Paul Leach said:
> diffs should be minimized, and since this one won't actually cause anyone
> to do anything to their code, it might just as well be omitted.

... but that is then inconsiderate of anyone who is reading the spec from
scratch, is it not? And since this *does* center around a MUST-level
requirement, I for one think it's very important to fix. Even though the
likely intent is clear to anyone who thinks about it long enough, what the
spec *says* will cause clients that do the sane thing to be non-conformant.

On the other hand, I don't think that anyone really cares if their
client/server is "HTTP/1.1 conformant" at this point. I can't recall anyone
ever claiming conformance, and literal conformance is probably impossible
anyway (due to internal requirement conflicts). From this perspective, we've
got only an ad-hoc standard at best, and the MUSTs aren't really worth
anything -- all that matters is "does my [client/server] work with
[Apache/IIS/IE/Firefox/libwww/.NET]?"

Thanks,

-- Travis

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Thu, 04 Jan 2007 15:24:13 -0600

> spec *says* will cause clients that do the sane thing to be non-conformant.

I read a MUST-NOT, not a MUST.  If there is no way to infer the user-agent's
preference, do not present an Accept-Language header.  This statement applies,
in equal part, to an endpoint client agent as well as any middle layer agent.

> client/server is "HTTP/1.1 conformant" at this point.

I would choose words more wisely when emailing from a work account.
Developers at Apache, Mozilla, certainly even at Microsoft do pay close
attention to the specification.  What they choose to do from that point
forward is their own foolishness.

The risk is that conformant++ behavior often becomes --conformant in actual
practice, witness the attempt to be more forgiving of spaces between header
fields and the separating colon of header strings, and the resulting HTTP
request spoofing vulnerabilities between varying vendor agent implementations.

---

From: "Travis Snoozy (Volt)" <[email-masked]>
Date: Thu, 4 Jan 2007 13:28:43 -0800

> of said proxy, but worse...

I think it's more egregious to presume that -any- author has common sense --
to me, that's just common sense ;).

> >   If the choice is not made available, then the Accept-Language header
> >   field MUST NOT be given in the request.
>

The prior sentence was omitted. Please elide omissions; taking that sentence
out of context does screw up the meaning, but that is an artifact of how the
original text was phrased. My proposed changes are not even in the clause
you quoted; they were in the omitted portion. The full proposal, again, was:

> of the end client, then that would be it's choice to 'fill in the gap'
> here. Or a special purpose proxy could very well prefer a specific
> language family based on it's anticipated user base.

None of these make the "linguistic preference available to the user". They make a choice, but the user is cut out of the loop (what if I use TOR, and get a German IP? Should the proxy rewrite my Accept-Language to be German, even though I want English?). Thus, none of the options you suggest conform to the spec.

> implementor to determine if and how Accept-Language should be presented.

... It's totally up to the implementer how to present Accept-Language options right now, and in my modification; I agree with you that this should be a choice on the implementer's part. But implementers of clients (proxies, user agents, or otherwise) do have to be present a linguistic preference choice -somehow- (according to the spec, right now) in order to transmit an Accept-Language header. This is the case even for things (like proxies) that don't have a reasonable way to present a choice to the user.

(I think I might be being unclear somehow; please, ask questions and I'll try to explain myself better.)

Thanks,

-- Travis

---

From: "Travis Snoozy (Volt)" <[email-masked]>
Date: Thu, 4 Jan 2007 15:34:11 -0800

William A. Rowe, Jr. said:
> > ... but that is then inconsiderate of anyone who is reading the spec
> > from scratch, is it not? And since this *does* center around a MUST-
> > level requirement, I for one think it's very important to fix. Even
> > though the likely intent is clear to anyone who thinks about it long
> > enough, what the spec *says* will cause clients that do the sane thing
> > to be non-conformant.
>
> I read a MUST-NOT, not a MUST.

You read correctly (well, aside from the hyphen, but that's splitting hairs
:). I'm expressing the MUST NOT requirement in the positive sense (as a
MUST) -- if I've made a semantic change in this transformation, please point
it out.

> If there is no way to infer the user-agent's preference, do not present an
> Accept-Language header.

The spec doesn't say "infer" or "user-agent"; it says that an -option has to
be made available- to the -user- (by [all] client[s]).

An example to demonstrate the problem:

Fred runs a browser (user-agent). Fred has configured his browser for the
Yiddish language. Thus configured, the browser sends a request with an
Accept-Language header to Fred's border proxy. Everyone is happy so far.

Now, what happens? Fred's border proxy is a client; as such, it MUST make a
linguistic option available to the user before giving an Accept-Language
header in the request (which it's forwarding). However, in order to do that,
Fred's proxy has to communicate back with Fred somehow. Thus, Fred's poor
proxy is left in the dilemma of either having to discard the Accept-Language
header (which means Fred isn't likely to get Yiddish), or send a page back
to Fred ("Do you really want Yiddish?"), or *violate the spec* and pass the
Accept-Language header to the origin server *without making a language
option available to Fred*.

Why should the proxy have to make a language option available to Fred? The
user-agent already did that. The proxy has no business doing it again, and
it just gets in the way. The sane thing to do is to just pass the Accept-
Language header along -- and that's why I think the wording should be
changed, so this behavior is allowed.

Now, all that said, the proposed wording could certainly be phrased better -
- most notably, because it should prohibit the proxy from sticking in
arbitrary Accept-Language headers on behalf of an unwitting user. But the
pass-through case should most certainly be permitted.

> This statement applies, in equal part, to an endpoint client agent as well
> as any middle layer agent.

Yep. That's the problem -- middle-layer agents don't have a way to make the
option available to the user, nor do they have a way to verify that the
option was -actually- made available to the user from whatever upstream HTTP
application gave them the message. So, middle-layer agents either have to
strip the Accept-Language header, or do something silly to try and get the
user to explicitly pick a language. That's why the requirement needs to be
narrowed down to "user agents" instead of "clients."

> I would choose words more wisely when emailing from a work account.

"The opinions expressed in this E-mail are that of the author, and do not
constitute any official position held by the author's employers or
affiliates..."

I'd like to think that choosing wise words is a skill that is independent of which keyboard I'm at. That said, I'd rather be wrong and corrected than right and silent. :)

> attention to the specification.

I never intended to imply that they didn't. What I said was that I do not
think anyone really cares if their client/server is "HTTP/1.1 conformant".
In straightforward terms: is it preferable to have a browser that works as
expected, or a browser that's HTTP/1.1 compliant (assuming mutual
exclusion)? A cache? A server? From the discussion so far, folks seem very
much more interested in having things that *work*, even though they aren't
*technically to spec*.

I, myself, am interested in pointing out and fixing warts in the spec such
that literal adherence leads to the creation of something that works as
expected. Strict conformance and (inter)operability should not be mutually
exclusive; likewise, implementers should be rewarded for following
directions to the T (i.e., not interpreting *anything*), not punished by
being broken-as-specified because they *didn't* interpret things. The spec-
writer is the one who should know best, IMO -- if the spec says jump off a
bridge, do it (and figure that the spec picked a low one with deep water
under it).

An excellent example provides itself, here:

> What they choose to do from that point forward is their own foolishness.
>
> The risk is that conformant++ behavior often becomes --conformant in
> actual practice, witness the attempt to be more forgiving of spaces
> between header fields and the separating colon of header strings, and the
> resulting HTTP request spoofing vulnerabilities between varying vendor
> agent implementations.

I feel like there's a contradiction. Earlier, it seemed to me like the
implication was to interpret the spec more loosely ("If there is no way to
infer the user-agent's preference, do not present an Accept-Language
header"), and do the right thing (violating the spec). Now, "conformant++"
(violating the spec) is a Bad Thing. This appears to be a double-standard;
am I missing something?

Thanks,

-- Travis

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Thu, 04 Jan 2007 18:02:07 -0600

>    NOT be given in the request.

Thank you for the clarification.  I have a better rational for your change;
"user agents" appears repeatedly in the specification; "client applications"
occurs only in this specific paragraph.  The other rational alternative is
to drop the word client.

Remember that an application/user agent may not even be interactive, and may
be an application with a preferences.conf file which can be configured
appropriately.

> None of these make the "linguistic preference available to the user". They
> make a choice, but the user is cut out of the loop (what if I use TOR, and
> get a German IP? Should the proxy rewrite my Accept-Language to be German,
> even though I want English?). Thus, none of the options you suggest conform
> to the spec.
> 
> ... It's totally up to the implementer how to present Accept-Language options 
> right now, and in my modification; I agree with you that this should be a choice
> on the implementer's part. But implementers of clients (proxies, user agents,
> or otherwise) do have to be present a linguistic preference choice -somehow-
> (according to the spec, right now) in order to transmit an Accept-Language
> header. This is the case even for things (like proxies) that don't have a
> reasonable way to present a choice to the user.

You presume.  Let's say that in your example above that as TOR's outbound proxy,
an assortment of proxy services are offered, e.g.

http://fr.tor.example.com
http://dk.tor.example.com
http://en.tor.example.com

I can make the rational argument that these three are user agents acting on
behalf of the endpoint user agent, even with the language change you propose.

> I'll try to explain myself better.)

Of course.

---

From: "Travis Snoozy (Volt)" <[email-masked]>
Date: Thu, 4 Jan 2007 16:55:35 -0800

William A. Rowe, Jr. said:
> >    the choice is not made available, then the Accept-Language header
> "user agents" appears repeatedly in the specification;
> "client applications" occurs only in this specific paragraph.

Well, "user agents" is defined; I *assume* "client applications" is
semantically equivalent to "client(s)" (which is defined), but that's just
my interpretation.

> The other rational alternative is to drop the word client.

Yes, but then everything has to make a choice available to the user.

Now, I think I should probably stop and back up a bit: when I say "user", I
mean "the person who told the user-agent to go make an HTTP request" -- not
"the person who is running the server/proxy/gateway." The word "user" does
not appear to be defined (at least, not in section 1.3), so I don't know
what the spec considers users to be, per se. With that said...

> Remember that an application/user agent may not even be interactive, and
> may be an application with a preferences.conf file which can be configured
> appropriately.

Keep in mind that "user agents" are special (the word is defined), whereas
"applications" are not (I read "application" to mean "anything that
interprets HTTP/1.1 messages").

> > ... It's totally up to the implementer how to present Accept-Language
> > options right now, and in my modification; I agree with you that this
> > should be a choice on the implementer's part. But implementers of
> > clients (proxies, user agents, or otherwise) do have to present a
> > linguistic preference choice -somehow- (according to the spec, right
> > now) in order to transmit an Accept-Language header. This is the case
> > even for things (like proxies) that don't have a reasonable way to
> > present a choice to the user.
>
> You presume.  Let's say that in your example above that as TOR's outbound
> proxy, an assortment of proxy services are offered, e.g.
>

Yes, but then why would I bother configuring my browser to Yiddish? What
happens if, e.g., there is no yi.tor.example.com? What happens if the TOR
endpoint (which I can't control) goes through a proxy before or after the
xx.tor.example.com?

As far as I can tell, the only things that have business with Accept-
Language are the end-user/user agent and the origin server (and, possibly,
an intermediate cache that can service the request in the appropriate
language).

> I can make the rational argument that these three are user agents acting
> on behalf of the endpoint user agent, even with the language change you
> propose.

Well, section 1.3 isn't rock-solid in its definition, but I think that it's
fairly clear that proxies don't "initiate" requests, they "forward"
requests. Proxies are not _usually_ user agents (though nothing stops you
from implementing a browser that also just happens to run a proxy on :8080).
Also, I think that the intent is that all "user agents" are endpoints (with
origin servers being the other endpoint) -- again, I don't see this anywhere
explicitly, other than the word "initiate" in the definition of "user
agent".

Thanks,

-- Travis

---

From: Paul Leach <[email-masked]>
Date: Thu, 4 Jan 2007 17:13:43 -0800

I'm not saying that changing it to "user-agent" isn't better. Or that
replacing MUST with SHOULD isn't a good idea. However, what's there
isn't hard to understand. Its intent is clear: if a client can't be
configured to express language preference, then it should not express
any. (That does _not_ mean that proxies should strip Accept-Language.)
We should focus on more important issues.

-----Original Message-----
From: Travis Snoozy (Volt) 
Sent: Thursday, January 04, 2007 1:13 PM
To: Paul Leach; [email-masked]
Subject: RE: NEW ISSUE: Use of "Client" in 14.4

> diffs should be minimized, and since this one won't actually cause
anyone

from 

likely intent is clear to anyone who thinks about it long enough, what
the
spec *says* will cause clients that do the sane thing to be
non-conformant.

On the other hand, I don't think that anyone really cares if their 
client/server is "HTTP/1.1 conformant" at this point. I can't recall
anyone 
ever claiming conformance, and literal conformance is probably
impossible 
anyway (due to internal requirement conflicts). From this perspective,
we've 
got only an ad-hoc standard at best, and the MUSTs aren't really worth 
anything -- all that matters is "does my [client/server] work with 
[Apache/IIS/IE/Firefox/libwww/.NET]?"

Thanks,

-- Travis

---

From: Paul Leach <[email-masked]>
Date: Thu, 4 Jan 2007 17:20:34 -0800

-----Original Message-----
From: William A. Rowe, Jr. [mailto:[email-masked]] 
Sent: Thursday, January 04, 2007 1:11 PM
To: Paul Leach
Cc: Travis Snoozy (Volt); [email-masked]
Subject: Re: NEW ISSUE: Use of "Client" in 14.4

> proxies stripping Accept-Language before forwarding) that very likely
[Paul Leach] A proxy is "configured" for the Accept-Language headers of
the requests it makes by the requests it receives.

[Paul Leach] Back to what William said:
I'd suggest we please leave the existing text alone and leave it to the
implementor to determine if and how Accept-Language should be presented.
[Paul Leach] I suggest they follow the intention of the spec, which is
clear enough.

---

From: Paul Leach <[email-masked]>
Date: Thu, 4 Jan 2007 17:33:14 -0800

Maybe I wasn't clear -- the reason no one would do anything to their
code if we made the change is not that I believe they don't care about
HTTP/1.1 conformance, its that they already are compliant with the
suggested new wording (because the old wording made the intent clear
enough).

It's all about priorities -- until the HTTP/1.1 WG focused on just
getting three things done well enough, we were making progress at a very
slow rate, as everyone strove for perfection.

(Those three things: caching, keep-alive and the Host: header. They were
the minimum deemed essential to "keep the internet from melting down"
under the pressure of Web traffic using earlier HTTP versions.)

It was just another instance of "the best being the enemy of the good".

-----Original Message-----
From: Travis Snoozy (Volt) 
Sent: Thursday, January 04, 2007 1:13 PM
To: Paul Leach; [email-masked]
Subject: RE: NEW ISSUE: Use of "Client" in 14.4

anyone

Thanks,

-- Travis

---

From: "Travis Snoozy (Volt)" <[email-masked]>
Date: Fri, 5 Jan 2007 13:49:47 -0800

Paul Leach said:
> Maybe I wasn't clear -- the reason no one would do anything to their code
> if we made the change is not that I believe they don't care about HTTP/1.1
> conformance, its that they already are compliant with the suggested new
> wording (because the old wording made the intent clear enough).

I understood what you meant. However, it seems that I was unclear: despite
the fact that these implementations are doing what is *right*, they aren't
doing what is *spec'd*. In this case, the spec needs to change so that these
people are in conformance instead of violation (hence my comment that
"nobody cares about conformance" -- they'd rather do what's right than be
broken-as-spec'd).

<snip>

Thanks,

-- Travis