Subject: Charsets and language

From: Graham Klyne <[email-masked]>
Date: Mon, 26 Apr 1999 08:31:50 +0100

This post derives from an offline discussion...

An intention to register a feature for "charset" capability/preference has
been suggested to me.  Looking at the negotiation framework for HTTP
(specifically, the "accept-charset" and "accept-language" headers) I am
wondering if it would make sense to also register a "language" preference
feature at the same time.

This in turn raises the idea that "charset" is a media feature, but
"language" is a user preference.  It is my understanding that constraining
the conneg work to "media features" was not intended to avoid
discussion/registration of user preferences;  rather, it was intended to
avoid us getting drawn into issues of transmission options.  Thus, it seems
to me that the focus of our effort has been "content features" as distinct
from, say, "transmission options".

The foregoing leads me to two questions:
(a) are registrations of features that relate to user preferences in scope
for our registration scheme?
(b) more specifically, is a registration of a feature for language
preference (in the same sense as that used for HTTP "accept-language")
appropriate?

#g

------------
Graham Klyne
([email-masked])

---

From: "Martin J. Duerst" <[email-masked]>
Date: Mon, 26 Apr 1999 17:11:38 +0900

> preference (in the same sense as that used for HTTP "accept-language")

If you do, please make sure that the behaviour with respect to
"sub-languages" is exactly the same as in HTTP (and other protocols).
There are some subtleties involved, but repeated discussion has shown
that this is the best way to do things.

I.e. if asked for "en", the server may send back "en-us" or "en-gb",
but if asked for "en-us", the server must not send back "en-gb" nor
"en", only "en-us" or "en-us-*".

Regards,   Martin.

#-#-#  Martin J. Du"rst, World Wide Web Consortium
#-#-#  mailto:[email-masked]   http://www.w3.org

---

From: Al Gilman <[email-masked]>
Date: Mon, 26 Apr 1999 10:01:05 -0400

Standard disclaimer: pardon my ignorance, I have not reviewed the
registration process RFC well enough to know some of the things I am
commenting on here.

From a product perspective, Yes, the vocabulary of resource features should
include the notions of language and charset.

From a process perspective, No, the media features registry is not the
right process for establishing appropriate values of these features; there
are senior authorities already controlling vocabulary for designating
language and charset and these should be followed.

This gets back to the issue of how much semantics there is in the registry.
 Ideally there would be in the registry mechanics a formal
define-by-reference capability so that a knowledge agent could parse the
registration record and comprehend that language and charset values are
used in this application as they are defined in designated authorities
external to the registry.

So, failing my review of the registry document, I see two ways to cover the
notions of language and charset.  One is a standards-track annex to the
appropriate RFC  which extends the predefined vocabulary of features.
This, being a standards-track action can use natural language to impart its
semantics.  The other is a define-by-reference mechanism in the registry
and a registration under this clause which binds usage for these notions to
the usage already controlled by existing authorities.  The latter should
probably be formal in the sense that it defines a syntax which connotes
normative reference.

Al

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Mon, 26 Apr 1999 08:16:50 -0700

Actually, after talking to Graham, I decided to register both "charset" and 
"language". I had forgotten that language *is* a media feature for programs 
that "display" text by reading it aloud, such as text-to-speech converters 
that are used by blind people.

I submitted the draft yesterday, and it should appear in the next day or 
two. I'll copy the pointer here when it comes out.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Al Gilman <[email-masked]>
Date: Mon, 26 Apr 1999 12:57:28 -0400

Summary:

Yes, it would be good to capitalize on best practice from elsewhere, and
synchronize the normal behavior of this information as much as is reasonable.

No, it is not best for any exact behavior to be uniquely what is associated
with any pattern of information.  There should be multiplicity in the
behavior/information association.

>that this is the best way to do things.

I would be interested in the experience behind the discussion.  Can you
furnish references to backups for this conclusion?

If the overarching general rule is that there should be zero, one or many
ways to do something, then the default assumption should be 'many,' because
for universal access there should generally be at least two.

>"en", only "en-us" or "en-us-*".

This presumes that the client has the skill to recognize the distinction
between "en" and "en-us" and send a search list articulating the priorities
for each option at the maximum fineness involved in any statement.  The
argument for why this is a global optimum behavior would be interesting to
read.  It is contrary to the existing mode of operation in natural language.

I had thought the consensus best practice was that only root languages are
restrictive, and that all sublanguage designations are advisory.  In other
words, "en" always matches a request for "en-us" if there is no "en-us"
designated alternative available.

Where is the rationale for the strict version articulated in public?

Given the limited experience base of HTTP negotiation, it is not clear one
should bless the HTTP decisions as "what should be exactly replicated
everywhere."

Al