Subject: Issue 113, was: Proposed resolution for Issue 13 (language tags)

From: Julian Reschke <[email-masked]>
Date: Fri, 01 Aug 2008 14:15:24 +0200

Felix Sasaki wrote:
> ...
> RFC 4647 defines a basic language range in sec. 2.1 which is the same as 
> the language range of sec. 14.4 of RFC 2616. Note also the information 
> in sec. 2.1 of RFC 4647:
> "Note that
> the ABNF [RFC4234] in [RFC2616] is incorrect, since it disallows the
> use of digits anywhere in the 'language-range' (see [RFC2616errata])."
> So you could just refer to basic language ranges of RFC 4647 and be fine.
> ...

Indeed, thanks for pointing this out. I missed the fact that RFC 4647 
defines basic ranges. Will make a concrete proposal later on.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 01 Aug 2008 21:05:04 +0200

Julian Reschke wrote:
> 
> Felix Sasaki wrote:
>> ...
>> RFC 4647 defines a basic language range in sec. 2.1 which is the same 
>> as the language range of sec. 14.4 of RFC 2616. Note also the 
>> information in sec. 2.1 of RFC 4647:
>> "Note that

Proposed text:

-- snip --
6.4.  Accept-Language

    The Accept-Language request-header field is similar to Accept, but
    restricts the set of natural languages that are preferred as a
    response to the request.  Language tags are defined in Section 3.5.

      Accept-Language = "Accept-Language" ":"
                        1#( language-range [ ";" "q" "=" qvalue ] )
      language-range  =
                <language-range, defined in [RFC4647], Section 2.1>

    Each language-range MAY be given an associated quality value which
    represents an estimate of the user's preference for the languages
    specified by that range.  The quality value defaults to "q=1".  For
    example,

        Accept-Language: da, en-gb;q=0.8, en;q=0.7

    would mean: "I prefer Danish, but will accept British English and
    other types of English."

    For matching, the "Basic Filtering" matching scheme, defined in
    Section 3.3.1 of [RFC4647], is used:

       A language range matches a particular language tag if, in a case-
       insensitive comparison, it exactly equals the tag, or if it
       exactly equals a prefix of the tag such that the first character
       following the prefix is "-".

    The special range "*", if present in the Accept-Language field,
    matches every tag not matched by any other range present in the
    Accept-Language field.
-- snip --

This

1) delegates the definition of language-range to RFC4647, Section 2.1, and

2) delegates the definition of the matching function to RFC4647, Section 
3.3.1 ("Basic Filtering").

Note that the proposed text *repeats* the definition (it's just one 
sentence, so I'm trying to do the reader a favor by not actually having 
to loop up the text).

I also note that "Basic Filtering" is case-insensitive, which it wasn't 
in RFC2616. This is intentional (right?); so should it noted in the 
"Changes from RFC2616" section?

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Fri, 1 Aug 2008 22:43:04 +0200

Julian Reschk wrote:

[...] 
>> Felix Sasaki wrote:
>>> ...
>>> RFC 4647 defines a basic language range in sec. 2.1
[...]

> Proposed text:
[...]

Fine.

> represents an estimate of the user's preference for the languages

Optionally s/MAY/can/.  The syntax has it clear that "q=" <qvalue>
can be omitted.

Do you have a special reason to write "q" "=" instead of "q=" ?
Is this a place where you actually want some kind of *WSP "=" *WSP ?

> would mean: "I prefer Danish, but will accept British English and

Maybe note that this list is not supposed to be sorted by <qvalue>s.

> matches every tag not matched by any other range present in the

Better add an example here, where the <qvalue> of "*" is not the
smalles in the list.  That is a pathological case, you'd get get
an "anything is better than something" effect:

   Accept-Language: da, en-gb;q=0.8, en;q=0.7, *;q=0.75

What does this mean if only "en" and "es" content is available ?

> I also note that "Basic Filtering" is case-insensitive, which it
> wasn't in RFC2616.

 [RFC 2616 page 29]
| all tags are case-insensitive

Nothing new here, IMO.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Sat, 02 Aug 2008 08:37:29 +0200

> Optionally s/MAY/can/.  The syntax has it clear that "q=" <qvalue>

+1

> Is this a place where you actually want some kind of *WSP "=" *WSP ?

This is text inherited from RFC 2616. Let's discuss LWS issues 
separately from this issue.

> Maybe note that this list is not supposed to be sorted by <qvalue>s.

I'd prefer not to change things that aren't broken. Has this come up 
before? Would it be sufficient just to shuffle the language-ranges in 
the example to make the point?

> What does this mean if only "en" and "es" content is available ?

Again, did this come up before? Is there a problem we need to solve?

I'm not against examples, but we should use them carefully.

> | all tags are case-insensitive

OK.

BR, Julian

---

From: Felix Sasaki <[email-masked]>
Date: Sat, 02 Aug 2008 16:03:58 +0900

Julian Reschke さんは書きました:
>
> Julian Reschke wrote:
>>
>> Felix Sasaki wrote:
>>> ...
>>> RFC 4647 defines a basic language range in sec. 2.1 which is the 
>>> same as the language range of sec. 14.4 of RFC 2616. Note also the 

looks fine to me as well. Re case-insensitive, see mail from Frank.

Felix

> 2) delegates the definition of the matching function to RFC4647, 
> Section 3.3.1 ("Basic Filtering").
>
> sentence, so I'm trying to do the reader a favor by not actually 
> having to loop up the text).
>
> wasn't in RFC2616. This is intentional (right?)

---

From: "Frank Ellermann" <[email-masked]>
Date: Sat, 2 Aug 2008 21:41:17 +0200

Julian Reschke wrote:

> Would it be sufficient just to shuffle the language-ranges in 

Yes, maybe swap en;q=0.7 and en-GB;q=0.8

 
> Again, did this come up before? Is there a problem we need to solve?

The problem to solve is to tell me how I should implement
this... ;-)  The example can be simpler and more obscure:

     Accept-Language: da, en-gb;q=0.8, en;q=0.7, *

What does that mean ?  Is it just broken, and implementations
can do whatever they like, e.g. ignore "*" ?  Or always use
a qvalue 0.001 for "*" no matter what the "wanted" value is ?

Or does this mean "I want anything, with en as worst case" ?

 Frank

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 03 Aug 2008 01:31:29 +0200

> Or does this mean "I want anything, with en as worst case" ?

Correct.. and if en is the only family of available languages then en-br
is preferred over any other english.. (and da is redundant)

A more useful example using * is

  Accept-Language: da, en-gb;q=0.8, en;q=0.7, *;q=0.001

Regards
Henrik

---

From: "Frank Ellermann" <[email-masked]>
Date: Sun, 3 Aug 2008 09:20:36 +0200

>  Accept-Language: da, en-gb;q=0.8, en;q=0.7, *;q=0.001

Then let's add it to the spec., with an explanation:
"Typically '*' is used with the lowest <qvalue>" 

Folks can still shoot into their foot, but they got
a fair warning.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Sun, 03 Aug 2008 15:06:49 +0200

> "Typically '*' is used with the lowest <qvalue>" 

Is that better than not specifying value for "*" at all?

I just checked FF3, and the default Accept-Language it sends for me does 
not include "*", but sure, servers typically send content outside the 
language ranges it sends.

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Sun, 3 Aug 2008 15:19:08 +0200

> Is that better than not specifying value for "*" at all?

Dunno.  Clearly this question needs the *same* answer for
all "*" cum <qvalue> contructs in various Accept-* header
fields.  That might be a new issue (not limited to #113).

 Frank

---

From: Henrik Nordstrom <[email-masked]>
Date: Sun, 03 Aug 2008 22:06:06 +0200

> language ranges it sends.

Uaing * makes the "I accept anything" fallback explicit. Otherwise
servers MAY return 406 if the response is not acceptable, even if most
prefer to return their default language instead.

Regards
Henrik

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 08:17:30 +0200

> prefer to return their default language instead.

Understood.

But apparently servers choose a default anyway. So the defaulting may be 
different for different Accept-* headers.

So again, what problem do we want to solve here?

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 4 Aug 2008 09:07:26 +0200

Julian Reschke wrote:

> again, what problem do we want to solve here?

The problem of "*" or "*/*" or other Accept-* stars,
where the implicit or explicit <qvalue> is not the
smallest <qvalue> in the Accept-* list.

Including the special cases *;q=0, *;q=1, and *. 

This is an issue in its own right, not limited to
#113 and Accept-Language.  It is a wildcard-qvalue
issue.  2616bis should tell implementors what this
means.  If this is hopelessly non-interoperable the
concept of Accept-* wildcards could be deprecated.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 09:20:15 +0200

> smallest <qvalue> in the Accept-* list.

It's only a (potential) problem if clients send it. Do you have any 
evidence this happens in practice?

> concept of Accept-* wildcards could be deprecated.

The case where "*" has qvalue bigger than an other value in the list is 
an edge case. I don't see why a client ever would send it.

The handling of "*" is defined separately for each header. So we'd 
really have to restate whatever we want to say for each of them, or to 
restructure that stuff.

Not convinced.

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 4 Aug 2008 11:02:01 +0200

> It's only a (potential) problem if clients send it.
> Do you have any evidence this happens in practice?

I've no evidence that it does NOT happen in practice.

I know for sure that a "quick locale switcher" add-on
of my UA managed to add some weird chrome://whatever
string to Accept-Language until I fixed it manually.

<about:config> claims that my current Accept-Charset
might be iso-8859-1,*,utf-8 (sic!).

And <http://delorie.com:81/> tells me that it is in
fact Accept-Charset: windows-1252,utf-8;q=0.7,*;q=0.7

So yes, this obviously happens, PEBKAC or otherwise.

> The case where "*" has qvalue bigger than an other
> value in the list is an edge case. I don't see why
> a client ever would send it.

I also don't see why my UA should ever wish to send
garbage, nevertheless it does.  And RFC 2616 doesn't
even say that this is garbage, so arguably it is not.

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 11:54:35 +0200

> might be iso-8859-1,*,utf-8 (sic!).

Which UA?

My FF3 says:

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

...so I don't see any problem here.

> fact Accept-Charset: windows-1252,utf-8;q=0.7,*;q=0.7

For me it says:

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

which matches what LiveHeaders reported.

> So yes, this obviously happens, PEBKAC or otherwise.

PEBKAC?

> even say that this is garbage, so arguably it is not.

I'm not convinced yet that your UA *does* send garbage. <about:config> 
apparently leaves the qvalues out, so you really need to look at the 
HTTP traffic.

BR, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 12:22:40 +0200

> looks fine to me as well. Re case-insensitive, see mail from Frank.

OK, applied with <http://tools.ietf.org/wg/httpbis/trac/changeset/303>.

BR, Julian

---

From: Yves Lafon <[email-masked]>
Date: Mon, 4 Aug 2008 08:53:39 -0400 (EDT)

> Including the special cases *;q=0, *;q=1, and *.

Well, the text is quite explicit, the default qvalue is 1, so * = *;q=1
and *;q=0 should be equivalent to not having *.

The fact that most servers returns something when * is not present and a 
406 when *;q=0 is just an implementation choice, driven by the text of 
part 3 section 5.

> concept of Accept-* wildcards could be deprecated.

Can you point to a specific non-interoperable issue? The example you gave 
didn't seem hopelessly broken.

-- 
Baroula que barouleras, au tiéu toujou t'entourneras.

         ~~Yves

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 4 Aug 2008 20:07:51 +0200

Yves Lafon wrote:

> Can you point to a specific non-interoperable issue?
> The example you gave didn't seem hopelessly broken.

| And <http://delorie.com:81/> tells me that it is in
| fact Accept-Charset: windows-1252,utf-8;q=0.7,*;q=0.7

* and utf-8 have the same <qvalue>, and one of the two
can't be serious.  Either * should be actuallly 0.001
(here: anything less than 1 and 0.7, but not 0), or
utf-8 should be anything between 0.7 and 1, say 0.701

Julian found a similar oddity with his UA:

| Which UA?
|
| My FF3 says:
|
| Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
|
| ...so I don't see any problem here.

I see a problem, the <qvalue> of * is not smaller than 
all other non-zero <qvalue>s.  BTW, my IUT is an FF 2.

I didn't test any other UA, but it is fairly simple to
end up with a dubious <qvalue> for *, */*, or similar.

|> So yes, this obviously happens, PEBKAC or otherwise.
|
| PEBKAC?

<http://www.google.com/search?q=define:PEBKAC>
"Problem Exists Between Keyboard And Chair":  I wasn't
sure if it was me who created this odd Accept-Charset.

The windows-1252 detail was me, but Julian has the same
q=0.7 issue behind the original ISO-8859-1.  I now say
"otherwise", no PEBKAC ;-)

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 20:22:19 +0200

> all other non-zero <qvalue>s.  BTW, my IUT is an FF 2.

So FF sends a header that contains unnecessary information (include 
utf-8, where leaving it out would mean the same).

I'm still not sure how this is a problem, unless you can show that this 
causes interop problems somewhere.

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 4 Aug 2008 21:09:04 +0200

Julian Reschke wrote:

> I'm still not sure how this is a problem, unless you can
> show that this causes interop problems somewhere.

The <qvalue>s are supposed to mean something.  In the case
of Accept-Language: en,de,frr;q=0.1,frs;q=0.1  I'd guess
(and implement) that this means "user wants 'en' or 'de', 
if both are unavailable frr or frs".  If 'en' *and* 'de' 
are available toss a coin, or take always the first / last, 
the user doesn't care.  Ditto if frr *and* frs exist, but
no 'en' and no 'de'.

So far it is simple, and no reason to talk about it in the
spec.  Now back to our Accept-Charset cases cases, assume
that Koi8-R, BOCU-1, and UTF-8 are available:

All match *;q=0.7.  Toss a coin ?  Take always the last,
i.e. for utf-8;q=0.7,*;q=0.7 take the * and toss a coin ?

This is _relatively_ harmless (unless you get BOCU-1 and
your UA can't handle it, obviously).  

But it is strange when the <qvalue> of * isn't one of the
smallest non-zero <qvalue>s, and I'm surprised that folks
here refuse to see it.

But why ?  If everybody does something else the wildcard
can be simply deprecated.  Or if desired fixed in some
way, e.g., state that a wildcard <qvalue> greater or
equal than the smallest non-zero <qvalue> elsewhere MUST
be interpreted as *;q=0.001.  Or SHOULD.  Or something
better than "dunno, who cares, toss a coin, get BOCU-1".

 Frank

---

From: Julian Reschke <[email-masked]>
Date: Mon, 04 Aug 2008 21:21:52 +0200

> smallest non-zero <qvalue>s, and I'm surprised that folks

It is a problem if it occurs in practice. Does it?

> better than "dunno, who cares, toss a coin, get BOCU-1".

Of course we can spend time on edge cases like these.

But if we do, I'd prefer to talk about repeating Content-Length or 
Content-Type headers first, because those may have security implications.

As a matter of fact, we *did* discuss those a few months ago, and 
decided not to include error handling instructions into the spec.

So why start here, without any evidence that we're working on a problem 
that indeed affects people?

BR, Julian

---

From: "Robert Brewer" <[email-masked]>
Date: Mon, 4 Aug 2008 12:24:31 -0700

> better than "dunno, who cares, toss a coin, get BOCU-1".

I don't think "toss a coin" is the only answer. For example, CherryPy
allows web app authors to provide a default encoding per resource [1],
which will be used if '*' is the first acceptable match in an
'Accept-Encoding' request header. Another option is to allow the app
author to provide an ordered list of desired values; CP uses this
technique for the 'Accept' header [2]. This is also specifiable per
resource.

Hmm...maybe we should unify those. ;)

Robert Brewer
[email-masked]

[1] http://www.cherrypy.org/browser/trunk/cherrypy/lib/encoding.py#L151
[2] http://www.cherrypy.org/browser/trunk/cherrypy/lib/cptools.py#L366

---

From: "William A. Rowe, Jr." <[email-masked]>
Date: Mon, 04 Aug 2008 14:24:27 -0500

Frank Ellermann wrote:
> the user doesn't care.

Actually, not quite flip-a-coin; the server can (and in the case of httpd,
it will) order preferences according to it's own qvalue list.  So in this
case of en == de, where the document was composed in German and translated
into english, the website author can reasonably attempt to serve an original
document rather than a translation.

---

From: "ryah dahl" <[email-masked]>
Date: Mon, 4 Aug 2008 22:52:04 +0200

> Content-Type headers first, because those may have security implications.

Isn't the only use for q-value to give equal preference to several
values? As far as I see it, the whole q-value topic is an edge case. I
think it should be depreciated in favor of ordered lists.

In the case where this might be meaningful, Accept-Language, UA do not
give the ability of the user to specify something more detailed than
an ordered list of perfered languages. Accept, Accept-Charset - who
cares? Is an ordered list ever not enough?

ry

---

From: Adrien de Croy <[email-masked]>
Date: Tue, 05 Aug 2008 09:14:23 +1200

> think it should be depreciated in favor of ordered lists.
ordered lists don't allow you to specify holes in a range of acceptable 
values, e.g.

Accept-Language: en, en-us;q=0

which means any english except US.

I don't see any problem with the current spec.  If UAs really don't want 
to specify a real preference (i.e. assign same q value, either 
explicitly or implicitly to multiple options), then as long as that is 
really a representation of the human user's wishes, what's the problem?

If we start to consider having a qvalue on "*" to be a problem, we might 
be tempted to take steps which will arbitrarily limit us (and cause 
actual problems) later.  But logically, being able to specify a qvalue 
for anything is just part of being able to define logical statements.

Regards

Adrien

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 04 Aug 2008 23:08:01 +0200

> different for different Accept-* headers.

Yes, defaulting is different for different Accept-* headers. Implemeters
are free to select how to handle the case when there is no acceptable
variant. Some may do very elaborate guessing, some just use a default,
some return 406...

For most it's very natural and common sense to return the servers defaul
language if none of the available languages is indicated as acceptable
by Accept-Language. The result will still render fine, even if the user
probably won't understand much.  But it's not the same for most other
Accept-* headers as unacceptable type or encoding most likely will
result in a popup for download in the browser...

But there is no difference in the spec. Implementers are free to use
common sense on when to use a default and when not, and clients should
indicate their preference and not assume all servers have the same
default behavior.

> So again, what problem do we want to solve here?

No idea.. only jumped in to explain what the specs says which aparently
wasn't understood and have not followed the discussion before. (language
is not really in my interest, but cache and negotiation model is)

Regards
Henrik

---

From: "Robert Brewer" <[email-masked]>
Date: Mon, 4 Aug 2008 14:29:27 -0700

Frank Ellermann wrote:
> William's observation that they can have their own list of
> priorities helps, but doesn't guarantee that you get UTF-8
> in this scenario - the server might prefer BOCU-1 because
> it needs less bandwidth.

If the client wants a guarantee, it needs to assign a higher qvalue to
utf-8 than to *.

Robert Brewer
[email-masked]

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 4 Aug 2008 23:24:31 +0200

> It is a problem if it occurs in practice. Does it?

We both had this *;q=0.7 oddity with different UAs, that's
a practical case.  All HTTP servers we visit are forced to
interpret it _somehow_  Because I'm always curious I like
to know what they are supposed to do.

Roberts proposal goes in a similar direction:  There are 
ways to deal with "*" in some appropriate way if it's the
winner (or one of the winners) determined by <qvalue>.

Ry's proposal would be nice, get rid of the <qvalue> and
use ordered lists.  But we're not supposed to change the
spec. so radically.

> Of course we can spend time on edge cases like these.

I don't insist on spending much time on it, I just want
it to be clear.  How about this:

| Note that "*" or similar wildcards depending on theare interpreted as

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 4 Aug 2008 23:47:27 +0200

Sorry, sent before I was ready...  Here's the proposal:

| Note that "*" or similar wildcards depending on the
| Accept header fields are interpreted as "*;q=1".
|  
| If a wildcard is associated with a non-zero <qvalue>,
| then equal or smaller non-zero <qvalue>s for other
| choices are in essence pointless.  Clients MAY assist
| users to avoid this scenario.

All variations of MAY are fine, e.g., SHOULD, or some
vaguer statement with "desirable" and "might".
 
 [Julian asked:]
> So why start here

Maybe adopt the proposal above, and be done with it.

 Frank

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 04 Aug 2008 23:48:22 +0200

> smallest <qvalue> in the Accept-* list.

And what is the problem with that?

For Accept-Language the priority is clear.

The most specific language range matching the content tag, with * being
the least specific possible..

* is a very non-specific language tag and sets the qvalue for any other
language. If a more specific language tag has a smaller qvalue than *
then it's a negative saying "Any other language is better than X". 

"en;q=0.1, sv;q=1, *;q=0.5" means

1. sv
2. any other language except for en
3. en

> Including the special cases *;q=0, *;q=1, and *. 

*;q=0 is a no-op as q=0 is the default anyway (except in Accept-Charset
where ISO-8859-1;q=1 is a special implicit default if * is not used...
damn legacy crap)

* and *;q=1 is the same thing as q=1 is the default if the qvalue is not
specified.

But yes there is a slight issue in the description of Accept-Encoding
which uses a slightly different selection priority than the other
Accept* headers where it could be read as *;q=1 is given a higher
priority than gzip;q=0 even if common sense says that "*;q=1, gzip;q=0"
should mean any encoding except for gzip just as it does for the other
Accept* headers..

> concept of Accept-* wildcards could be deprecated.

And where does it not tell what the wildcard means or which priority it
gets in relation to other specifications?

Accept-Language which is the header discussed says:

Which gives * the lowest possible selection priority.

And further the following is also of relevant to your question:

   A language-range matches a language-tag if
   it exactly equals the tag, or if it exactly equals a prefix of the
   tag such that the first tag character following the prefix is "-".

tag in this context is the language tag of the candidate resource
variant on the server when determining the qvalue of that variant.

Content-Negotiation works in such way that the sever selects the variant
with the highest qvalue as per the Accept* headers evaluated. Variants
with a qvalue of 0 in any dimension is not acceptable. In effect you
start out with the set of all available variants, filter out the
variants with qvalue in any dimension and then sort them to find the
best match..

Specifications do not define the seletion priority between different
Accept* dimensions if they conflict (i.e. Accept, Accept-Language,
Accept-Charset). 

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 04 Aug 2008 23:59:23 +0200

> fact Accept-Charset: windows-1252,utf-8;q=0.7,*;q=0.7

And what's the problem with that? Yes,your user-agent apparently have a
bit of a  problem making up it's mind.. (the utf-8;q=0.7 part is
redundant... same qvalue as *) but It's very clear how the header should
be read. But maybe it's a workaround to broken implementations not
understanding *...

redundant details is just redundant, not garbage, and not the source of
failuers.

The definition of * (and no *) for Accept-Charset is:

   The special value "*", if present in the Accept-Charset field,
   matches every character set (including ISO-8859-1) which is not
   mentioned elsewhere in the Accept-Charset field. If no "*" is present
   in an Accept-Charset field, then all character sets not explicitly
   mentioned get a quality value of 0, except for ISO-8859-1, which gets
   a quality value of 1 if not explicitly mentioned.

And default qvalue is defined just above this as:

   Each charset MAY
   be given an associated quality value which represents the user's
   preference for that charset. The default value is q=1.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 05 Aug 2008 00:04:19 +0200

> utf-8 should be anything between 0.7 and 1, say 0.701

Yes, but where it the specification problem with this?

> all other non-zero <qvalue>s.  BTW, my IUT is an FF 2.

What problem?

For variants with the same qvalue the server is free to select one.

a;q=0.5, *;q=0.5

is really not any different in principle from

a;q=0.5, b;q=0.5

if available variants are 'a' or 'b'.

> end up with a dubious <qvalue> for *, */*, or similar.

And where is the problem we need to address?

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 05 Aug 2008 00:23:13 +0200

On mån, 2008-08-04 at 21:09 +0200, Frank Ellermann wrote:
> assume

Toss a coin.

All three have a qvalue of 0.7. It's not relevant that only one got the
qvalue from a specific declaration and the other from a wildcard. The
result should be the same if the header was listing all three explicitly
as in "utf-8;q=0.7, koi8-r;q=0.7; bocu-1;q=0.7".

> smallest non-zero <qvalue>s, and I'm surprised that folks

I don't see it stange. It's a negative prose. "I really prefer anything
else than...". Useful as an optimization when you have problems with
something specific (a specific media type, a specific charset, language
or whatever..)

"bocu-1;q=0.5;Koi8-r,*;q=0.7"

* prefer Koi8-r.
* use bocu-1 only if no other is availabe

Or using Accept to illustrate the need for negative proses better:

"*/*; q=0.5, movie/mpeg;q=0, image/jpeg, text/*"

for an application that understands most content, have some preferences
on preferred content, but knows it can not decode mpeg content (patent
issues).

> better than "dunno, who cares, toss a coin, get BOCU-1".

I'd say the specs is fine as they are on this. The rules is pretty clear
with no real room for confusion. You just have to keep in mind that the
specifications is written from a server perspective here starting out
with all available variants of the resource and finding the qvalue for
each of them and then filtering and sorting the result..

Implementers are however free to implement server driven negotiation
differently if they wish, with the specifications being no more than a
recommendation (no MUSTs in there.. only SHOULD or MAY). But the rules
specified is easy to implement, and not hard to understand for technical
people (even if Accept-Language concepts is far from intuitive to most
end-users, as already explained)

Regards
Henrik

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 5 Aug 2008 00:50:17 +0200

> Toss a coin.

Okay, that answers that:  The utf-8;q=0.7 is redundant, but
actually harmless.  Maybe Julian's or my UAs shouldn't do
it, as it could cause harm for discussions on this list ;-)

 [* qvalue not the smallest non-zero qvalue]
> I don't see it stange. It's a negative prose. "I really
> prefer anything else than...". Useful as an optimization
> when you have problems with something specific
[...]

> * prefer Koi8-r.
> * use bocu-1 only if no other is available

Oh, you found a plausible meaning for non-zero and smaller
than *.  That is good, so far I assumed that this _must_ be
erroneous.

> for an application that understands most content, have
> some preferences on preferred content, but knows it can
> not decode mpeg content (patent issues).

Yes, I got this idea for zero, but not for non-zero.  Maybe
a better example is application/pdf:

..., text/*, */*;q=0.2, application/pdf;q=0.1

"If all else fails I'm willing to try my luck with PDF 
 before giving up".

> no real room for confusion. You just have to keep in mind
> that the specifications is written from a server 
> perspective here starting out with all available variants
> of the resource and finding the qvalue for each of them
> and then filtering and sorting the result..

Well, it did confuse me.  Now that you've shown me a simple
example it's perfectly clear.  But maybe it was not only me,
others apparently also didn't exactly know what the purpose
of a wildcard with a "medium" <qvalue> is.

So that IMO needs a plausible example, explaining what you
wrote above (not the mpeg;q=0 case, that's IMO too simple).

 Frank

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 05 Aug 2008 01:37:48 +0200

> cares? Is an ordered list ever not enough?

No, as the differen variants on the server do not need to be of equal
quality.

This is best explained in the description of Accept. Read that part of
the spec again, then continue thinking on the topic of preference..

   The example

       Accept: audio/*; q=0.2, audio/basic

   SHOULD be interpreted as "I prefer audio/basic, but send me any audio
   type if it is the best available after an 80% mark-down in quality."

Which also means that even if audio/basic IS avilable it does not need
to be the prefferred if the quality of that representation on the server
is far worse than something else (more than 80% mark-down in quality)..

charset does not have the same quality level. But language may, even if
most user-agents currently does not give the user any easy way to
indicate his level of understanding of the language. But many
multi-lingual users read several languages fluently and really want's
the best representation in any of those languages with no strict
preference.. 

If you know the language reasonably well then the original language
something was written in is often the preferred variant even if
translations of varying quality is available.  For example in technical
literature I have a strong prefernce for english over my native swedish
language as english is often the original language and of better quality
than the swedish translations..

So while we are not quite there yet in actual deployments the framework
is there to be used and does it's thin without complicating life for
those who don't care. Don't toss it out.

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 05 Aug 2008 01:43:19 +0200

> | users to avoid this scenario.

Not true as there may be negative preference for something. I.e. a
dialect you really don't like, a media type your U-A have problem
decoding (technical or legal), etc..

Regards
Henrik

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 05 Aug 2008 02:01:51 +0200

> than *.  That is good, so far I assumed that this _must_ be

But you did get the meaning of q=0 and smaller than *?

Regards
Henrik

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 5 Aug 2008 04:53:05 +0200

> But you did get the meaning of q=0 and smaller than *?

Almost too late to ask that question, I fear it was in the
direction of:

"0 means I do not want it.  Anything else means I want it
 to some degree, and if I want 'anything' at some level it
 is pointless to want something specific at a lower or the
 same level."

I completely missed the concept to *not* want something at
non-zero levels.  

 Frank