Subject: Re: RFC2616 erratum "languagetag"

From: Julian Reschke <[email-masked]>
Date: Sun, 25 Nov 2007 18:23:44 +0100

(see also <http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/issues/#i13>)

Julian Reschke wrote:
> ...
>> Regarding replacing the syntax by just a link to RFC4646 will break a 
>> nice feature of rfc2616, the fact that the syntax is contained in the 
>> spec, with no need to get something else. It means also that we should 
>> change/upgrade the definition of language-range (RFC2616#14.4)
> 
> The problem is that RFC4646 has made that much harder, because now two 
> lines wouldn't be sufficient anymore (unless we deviate from RFC4646's 
> ABNF).

...and, as a matter of fact, the RFC2616 grammar already relies on rules 
imported from the URI spec.

> ...

I have looked at this again, and made experimental changes in 
<http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/draft-lafon-rfc2616bis-latest.html#rfc.issue.languagetag>.

The old text was:

3.10.  Language Tags

    A language tag identifies a natural language spoken, written, or
    otherwise conveyed by human beings for communication of information
    to other human beings.  Computer languages are explicitly excluded.
    HTTP uses language tags within the Accept-Language and Content-
    Language fields.

    The syntax and registry of HTTP language tags is the same as that
    defined by [RFC1766].  In summary, a language tag is composed of 1 or
    more parts: A primary language tag and a possibly empty series of
    subtags:

      language-tag  = primary-tag *( "-" subtag )
      primary-tag   = 1*8ALPHA
      subtag        = 1*8ALPHA

    White space is not allowed within the tag and all tags are case-
    insensitive.  The name space of language tags is administered by the
    IANA.  Example tags include:

        en, en-US, en-cockney, i-cherokee, x-pig-latin

    where any two-letter primary-tag is an ISO-639 language abbreviation
    and any two-letter initial subtag is an ISO-3166 country code.  (The
    last three tags above are not registered tags; all but the last are
    examples of tags which could be registered in future.)

The current edits say:

3.10.  Language Tags

    A language tag identifies a natural language spoken, written, or
    otherwise conveyed by human beings for communication of information
    to other human beings.  Computer languages are explicitly excluded.
    HTTP uses language tags within the Accept-Language and Content-
    Language fields.

    The syntax and registry of HTTP language tags is defined by
    [RFC4646]:

      Language-Tag  = <defined in [RFC4646], Section 2.1>

    White space is not allowed within the tag and all tags are case-
    insensitive.  The name space of language tags is administered by the
    IANA.  Example tags include:

        en, en-US, en-cockney, i-cherokee, x-pig-latin

    where any two-letter primary-tag is an ISO-639 language abbreviation
    and any two-letter initial subtag is an ISO-3166 country code.  (The
    last three tags above are not registered tags; all but the last are
    examples of tags which could be registered in future.)

So essentially I have removed the 2nd paragraph, replacing it with a 
pointer to RFC4646, and changed the BNF to just import the 
"Language-Tag" rule from there).

I'm not so sure about the remainder of the subsection? Should we keep 
the explanations and examples? If we do, are they correct as per RFC4646?

BR, Julian

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 26 Nov 2007 09:11:42 +0100

Julian Reschke wrote:

> essentially I have removed the 2nd paragraph, replacing it with
> a pointer to RFC4646, and changed the BNF to just import the 

Sounds good.  We can check it with the LTRU folks (if necessary).

> I'm not so sure about the remainder of the subsection? Should we
> keep the explanations and examples? If we do, are they correct
> as per RFC4646?

s/any two-letter/any two- or three-letter/ (alpha3 primary tags
are also ISO 639 languag codes).  Get rid of "i-cherokee", it's
not a registered (valid + grandfathered) i-anything, the code 
for Cherokee is chr (language) or Cher (script) respectively.
 
s/en-cockney/en-scouse/ (cockney is no registered en-variant at
the moment).  Maybe pick en-scotland or hy-arevmda instead.

Language ranges (in other words the RFC 3066 "*") are covered
by RFC 4647, please look into it, maybe 2616bis should get a
pointer to 4647.  IIRC 4647 got a reference to the RFC 2616
errata.  LTRU was too timid to risk an "updates 2626"... ;-)

 Frank

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Mon, 26 Nov 2007 11:53:12 +0100

On Sun, Nov 25, 2007 at 06:23:44PM +0100,
 Julian Reschke <[email-masked]> wrote 
 a message of 87 lines which said:

> > > a nice feature of rfc2616, the fact that the syntax is contained
> > > in the spec, with no need to get something else.

Is there a general rule somewhere against cross-RFC references?
Because a lot of RFC do it (even RFC 2616, as you mention).

...
>        Example tags include:

As mentioned by Frank Ellermann, it is bad practice to include invalid
(even if they are well-formed) tags like en-cockney.

i-cherokee is worse since it is not even well-formed (I agree it is
not clear in RFC 4646 but much better said in 4646bis).

x-pig-latin is not valid but it is a private tag, so I find it OK.

>    where any two-letter primary-tag is an ISO-639 language
>    abbreviation and any two-letter initial subtag is an ISO-3166
>    country code.

No, this is no longer true since RFC 4646 and the introduction of the
IANA registry. For instance, "yu" as a country is no longer in ISO
3166 but still in the registry.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 27 Nov 2007 21:22:28 -0800

I've updated i13 <http://www.w3.org/Protocols/HTTP/1.1/rfc2616bis/ 
issues/#i13> to include this.

On 14/10/2006, at 5:13 AM, Julian Reschke wrote:

> In the meantime, RFC3066 has been obsoleted by RFC4646, and the  
> original grammar production defining subtags seems to be gone:
>
>    Language-Tag  = langtag
>                  / privateuse             ; private use tag
>                  / grandfathered          ; grandfathered  
> registrations
>
>    langtag       = (language
>                     ["-" script]
>                     ["-" region]
>                     *("-" variant)
>                     *("-" extension)
>                     ["-" privateuse])
>
>    language      = (2*3ALPHA [ extlang ]) ; shortest ISO 639 code
>                  / 4ALPHA                 ; reserved for future use
>                  / 5*8ALPHA               ; registered language subtag
>
>    extlang       = *3("-" 3ALPHA)         ; reserved for future use
>
>    script        = 4ALPHA                 ; ISO 15924 code
>
>    region        = 2ALPHA                 ; ISO 3166 code
>                  / 3DIGIT                 ; UN M.49 code
>
>    variant       = 5*8alphanum            ; registered variants
>                  / (DIGIT 3alphanum)
>
>    extension     = singleton 1*("-" (2*8alphanum))
>
>    singleton     = %x41-57 / %x59-5A / %x61-77 / %x79-7A / DIGIT
>                  ; "a"-"w" / "y"-"z" / "A"-"W" / "Y"-"Z" / "0"-"9"
>                  ; Single letters: x/X is reserved for private use
>
>    privateuse    = ("x"/"X") 1*("-" (1*8alphanum))
>
>    grandfathered = 1*3ALPHA 1*2("-" (2*8alphanum))
>                    ; grandfathered registration
>                    ; Note: i is the only singleton
>                    ; that starts a grandfathered tag
>
>    alphanum      = (ALPHA / DIGIT)       ; letters and numbers
>
> (<http://tools.ietf.org/html/rfc4646#page-5>).
>
> So shouldn't RFC2616 (Section 3.10) stop defining these things, and  
> just normatively refer to RFC4626 for the definition of "Language- 
> Tag"?

--
Mark Nottingham     http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Thu, 29 Nov 2007 17:18:05 +0100

Frank & Stephane,

thanks for the feedback; I'll probably not be able to update the doc 
before next week though.

Do we have lazy consensus that HTTP should use the language matching 
defined in RFC4647? Just checking...

BR, Julian

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 29 Nov 2007 16:21:50 +0000

> defined in RFC4647? Just checking...

+1 from me.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 30 Nov 2007 10:10:04 +0100

On Thu, Nov 29, 2007 at 05:18:05PM +0100,
 Julian Reschke <[email-masked]> wrote 
 a message of 11 lines which said:

> defined in RFC4647?

Yes for me, in order to avoid making the HTTP RFC even bigger.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 30 Nov 2007 10:18:12 +0100

> Yes for me, in order to avoid making the HTTP RFC even bigger.

My goal is to make it smaller :-). For instance, there's also lots of 
historic information about URIs which I hope is better covered in RFC3986...

BR, Julian