Subject: Re: [Ltru] Language tag education and negotiation

From: Nicolas Krebs <[email-masked]>
Date: Sun, 4 May 2008 19:12:27 +0200

Martin Dürst wrote about rfc-822-like-message-header 
"Accept-Language" 

>Date: Wed, 30 Apr 2008 14:25:28 +0900
>To: Asmus Freytag <[email-masked]>,
>        Andrew Cunningham <[email-masked]>
>From: Martin Duerst <[email-masked]>
>CC: [email-masked]
> Archived-At: <http://www.w3.org/mid/6.0.0.20.2.20080430142025.0b2154e0@localhost>

>Actually, it IS handled by the current scheme, with what's called
>q-values. Q-values are used to indicate relative preferences from
>the client side, and relative quality on the server side. So a
>server could give original documents a q value of 1.0, and translated
>documents a q-value of e.g. 0.5 (q-values are always between 1.0 and
>0.0).
>
>q-values are sent by some browsers, which translate a preference
>list of e.g. en, ja, de, fr into something like
>   en, ja;q=0.9, de;q=0.8, fr;q=0.7
>or so. Apache also understands them. But they are difficult to set
>up, so they are not really used much.

And the specification of "Accept-Language" is in rfc 3282 section 3
urn:ietf:rfc:3282 
http://tools.ietf.org/html/rfc3282#section-3 

There is a specification for http protocol only in 
rfc 2616 section 14.4
urn:ietf:rfc:2616
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.4
http://tools.ietf.org/html/rfc2616#section-14.4

The RFC 2616 is currently updated by a working group of IETF and W3C 
http://www.ietf.org/html.charters/httpbis-charter.html
http://tools.ietf.org/wg/httpbis

If someone has bug-report or improvement about 
http://tools.ietf.org/html/draft-ietf-httpbis-p3-payload-02#section-6.4
or http://tools.ietf.org/html/draft-ietf-httpbis-p3-payload-02#section-3.5
( urn:ietf:draft-ietf-httpbis-p3-payload-02 section 6.4 and 3.5), 
please send feedback to the forum (wich is a mailing list) 
http://lists.w3.org/Archives/Public/ietf-http-wg/ . 

See also 
http://www3.tools.ietf.org/wg/httpbis/trac/ticket/13
http://www3.tools.ietf.org/wg/httpbis/trac/ticket/113

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Mark Davis" <[email-masked]>
Date: Sun, 4 May 2008 13:29:06 -0700

Unfortunately, the specs are ill-defined regarding the q values. Take the
following example:

a, b;q=0.7, c, d;q=0.5, e, f;q=0.9, g

The specs do not distinguish between at least two different possible
reasonable interpretations of what the q values of c, e, and g are:

   1. c;q=1.0, e;q=1.0, g;q=1 // always 1
   2. c;q=0.7, e;q=0.5, g;q=0.9 // always same as previous

Our guess is that the user meant #2, but it is only a guess.

And even once that ambiguity is cleared up, nobody knows what the meaning of
the numbers really is. My native tongue is English, I can understand Swiss
German and German, my French is rusty, and Italian basic. Which of these
should I use?

   1. en, gsw, de, fr, it
   2. en, gsw;q=0.9, de;q=0.9, fr;q=0.8, it;q=0.7
   3. en, gsw;q=0.5, de;q=0.4, fr;q=0.3, it;q=0.2
   4. en, gsw;q=0.99, de;q=0.95, fr;q=0.03, it;q=0.02

All are descending order, but depending on what algorithm the consumer of
the tags uses, they could have very different results. Without being able to
have any consistent expectations for what the q numbers mean, producers of
tags don't know what settings to provide or what difference it will make,
and consumers of tags don't know what the producers meant.

Mark

On Sun, May 4, 2008 at 10:12 AM, Nicolas Krebs <
[email-masked]> wrote:

> > Archived-At: <
> http://www.w3.org/mid/6.0.0.20.2.20080430142025.0b2154e0@localhost>
>
>

-- 
Mark

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Sun, 4 May 2008 22:59:43 +0200

On Sun, May 04, 2008 at 01:29:06PM -0700,
 Mark Davis <[email-masked]> wrote 
 a message of 185 lines which said:

> Unfortunately, the specs are ill-defined regarding the q values. 

My main problem with language negotiation in HTTP, and the reason why
I did not configure my browser to list my favorite language (french)
first, is that HTTP make no distinction between original and
translated texts. If I say to the HTTP server "I prefer french, then
english", I get, most of the time, poorly translated and/or
out-of-date pages in french. (Just compare
http://www.microsoft.com/en/us/ and http://www.microsoft.com/fr/fr/.)

What I really would like to express is "I prefer original text in
french or english, then translated text in one of these two
languages." But that's work for RFC 4646 ter :-)
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Ira McDonald" <[email-masked]>
Date: Sun, 4 May 2008 18:56:58 -0400

Hi Stephane,

Actually, I think it's *not* work for RFC 4646 ter.

The metadata for original versus translation doesn't belong in
a language tag.  Continuing to add optional fields to language
tags is just going to make like for more difficult for all of us.

Cheers,
- Ira

-- 
Ira McDonald (Musician / Software Architect)
Chair - Linux Foundation Open Printing WG
Blue Roof Music/High North Inc
email: [email-masked]
winter:
 579 Park Place Saline, MI 48176
 734-944-0094
summer:
 PO Box 221 Grand Marais, MI 49839
 906-494-2434

---

From: Felix Sasaki <[email-masked]>
Date: Mon, 05 May 2008 08:09:40 +0900

> tags is just going to make like for more difficult for all of us.

+1.

Felix

---

From: "Andrew Cunningham" <[email-masked]>
Date: Mon, 5 May 2008 11:40:49 +1000

What you're discussing is metadata about an object (not language
tagging), and belongs in an appropriate metadata standard

assuming web developers use it

in html its possible to have link elements point to alternative
translations of a document, but i've rarely seem it implemented.

Andrew

-- 
Andrew Cunningham
Vicnet Research and Development Coordinator
State Library of Victoria
Australia

[email-masked]
[email-masked]

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 05 May 2008 00:40:00 -0700

> languages." But that's work for RFC 4646 ter :-)
I have precisely the same problem! You express it very clearly.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Mon, 05 May 2008 08:31:10 -0700

On 5/5/2008 7:39 AM, Frank Ellermann wrote:
> Asmus Freytag wrote:
>
>   
>>> What I really would like to express is "I prefer original text
>>> in french or english, then translated text in one of these two
> +1  It infuriates me when xyzzy.com asks me if I want to go to
> xyzzy.de, or worse just redirects to xyzzy.de, when I prefer
> xyzzy.com for some reason of my own.  But that is unrelated to
> language tags, it is some "Geo IP" stuff, unfortunately often
> "working as designed" (from their POV).
>
>   
We don't think that the issue as originally described is unrelated to 
language negotiation at all.

Language negotiation, as implemented seems to make the implicit 
assumption that where users provide more than one language, the second 
(and following) are to be treated as fallback, so *any* page in the 
first language, is preferable to the user, whether its a complete 
translation or just a partial service intended for those monolingual in 
the those languages.

What you are describing is the further assumption by some sites, that if 
your first choice of language matches some of their subsidiaries, that 
you would rather work in that 'locale'. Treating the language preference 
as what you call 'geo ip' appears to be a clear misuse of the language 
negotiation feature, whereas the other issue is one of a shortcut in 
implementation of a scheme, which, if correctly implemented and adopted 
(including the lack of assigning q values to the contents) might 
actually have led to the correct results.

But +1 to being infuriated about that misuse as well ;-)
A./
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Peter Constable <[email-masked]>
Date: Mon, 5 May 2008 18:05:37 -0700

+1

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Ira McDonald
> Sent: Sunday, May 04, 2008 3:57 PM
> To: Stephane Bortzmeyer; Ira McDonald
> Cc: [email-masked]; [email-masked]
> Subject: Re: [Ltru] Language tag education and negotiation
>
> Hi Stephane,
>

---

From: Shawn Steele <[email-masked]>
Date: Mon, 5 May 2008 20:33:23 -0700

I agree as well, but I'm curious where this distinction might be made?  (should it be part of HTTP?)

Personally I'm in the other group, marginal English might be better for me than German, which would do in a pinch.

- Shawn

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Peter Constable [[email-masked]]
Sent: Monday, May 05, 2008 6:05 PM
To: Ira McDonald; Stephane Bortzmeyer
Cc: [email-masked]; [email-masked]
Subject: Re: [Ltru] Language tag education and negotiation

+1

---

From: Martin Duerst <[email-masked]>
Date: Wed, 07 May 2008 16:31:55 +0900

>Unfortunately, the specs are ill-defined regarding the q values.

Yes indeed. I have reported this to the HTTP WG.

>Take the following example:
>
>The specs do not distinguish between at least two different possible reasonable interpretations of what the q values of c, e, and g are: 
>    * c;q=1.0, e;q=1.0, g;q=1 // always 1 
>    * c;q=0.7, e;q=0.5, g;q=0.9 // always same as previous 

My guess would be to use 1.0 as a default, because this is
consistent with other headers, and allows concatenation and
splitting of headers without going through hoops. Please
remember that the above is protocol syntax, not what the
user should see or would edit.

>And even once that ambiguity is cleared up, nobody knows what the meaning of the numbers really is. My native tongue is English, I can understand Swiss German and German, my French is rusty, and Italian basic. Which of these should I use? 
>    * en, gsw, de, fr, it 
>    * en, gsw;q=0.9, de;q=0.9, fr;q=0.8, it;q=0.7 
>    * en, gsw;q=0.5, de;q=0.4, fr;q=0.3, it;q=0.2 
>    * en, gsw;q=0.99, de;q=0.95, fr;q=0.03, it;q=0.02 
>All are descending order, but depending on what algorithm the consumer of the tags uses, they could have very different results. Without being able to have any consistent expectations for what the q numbers mean, producers of tags don't know what settings to provide or what difference it will make, and consumers of tags don't know what the producers meant.

There is a certain aspect of tuning, which indeed currently doesn't work
because there isn't enough actual use on the server side. But basically,
the idea is simple: the values get multiplied with the corresponding
quality values for the documents on the server side.

So the way you should think is that you set gsw to 0.5 if you would
be satisfied to get the English document if the server thinks that the
English document it has is more than half as good than the corresponding
gsw document, but you would prefer to get the gsw document if the server
thinks that the corresponding English document is less than half as
good in quality than the corresponding gsw document.

Of course, "half as good in quality" is then the difficult piece of
information, but ideas for measuring this might be the time it takes
to read for a native reader, the amount of "updateness" (i.e. 70%
of the document are up-to-date -> document quality 0.7), or similar
measures. You can go into very fine details (and they way I know you,
you probably will), but while my considerations above won't help
you figure out the *exact* values for your q-values, they should
be enough to help you decide among the above listed four variants.

My personal guess is that
   en, gsw;q=0.5, de;q=0.4, fr;q=0.3, it;q=0.2
maybe most reasonable, but it's up to you to decide.

Regards,   Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: "Mark Davis" <[email-masked]>
Date: Wed, 7 May 2008 08:23:23 -0700

I sent in a mail, and the responses I got back was that 1 is always the
default, and that as far as the spec is concerned, the only reliable
information is the relative magnitude of the values. Neither the client nor
server can depend on anything else in the absence of negotiation. That is,
for all the server can depend on,

"en, fr;q=0.99999, de;q=0.99998"
    is the same as
"en, fr;q=0.00002, de;q=0.00001".

Mark

On Wed, May 7, 2008 at 12:31 AM, Martin Duerst <[email-masked]>
wrote:

> >And even once that ambiguity is cleared up, nobody knows what the meaning
> of the numbers really is. My native tongue is English, I can understand
> Swiss German and German, my French is rusty, and Italian basic. Which of
> these should I use?
> >    * en, gsw, de, fr, it