Subject: RE: [Ltru] Reference lists; AND ABNF GRAMMARS

From: "Addison Phillips" <[email-masked]>
Date: Tue, 19 Jul 2005 10:24:35 -0700

Hi Kent,

The grammar in matching is different on purpose.

1. The matching draft allows users to specify *either* a primary language subtag *or* an extended language sequence. That is, extlangs, for matching purposes, are considered unitary with the primary language. You either specify the extlangs or you don't. You cannot have a sequence like:

*-cmn-*-Latn

You may have sequences like:

zh-*-Latn, zh-cmn-*-Latn

In other words, assuming an extended language range, the wild card can only appear at the END of a specific field. This is also illegal:

en-Latn-US-*-boont // specifies multiple variants before a specific variant.

2. Extensions are specifically excluded from extended language range matching: their matching semantics are allowed to be different and they are not necessarily considered to be "part" of the language identifier proper. Of course they can be matched on a pure subtag basis. Contributions about extensions are invited.

> extlang         = 2*3ALPHA	%%%%%% I'm not sure about the non-trivial
> differences from -registry here
>                   *2("-" 3ALPHA)
>                   ("-"(3ALPHA / "*"))

This says "(concrete) primary language followed by one or more extlangs or star"

Best Regards,

Addison

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Kent Karlsson
> Sent: 2005?7?19? 10:07
> To: Kent Karlsson
> Cc: LTRU Working Group
> Subject: Re: [Ltru] Reference lists; AND ABNF GRAMMARS
> 
> I wrote:
> 
> >>    [RFC2234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for
> Syntax
> >>               Specifications: ABNF", RFC 2234, November 1997.
> >
> > I think that should be an informative reference (it is normative for
> > -registry, but not for -matching, I would say). B.t.w., I don't find
> > an actual reference in the text.
> 
> Oops. It should stay normative.
> 
> 
> The grammars in -registry and -matching should "look more similar";
> formatting, order, comments, and structure. Here is a preliminary cleanup:
> 
> ------------------------------------------------------------
> 
> %% For -registry
> 
> Language-Tag    = langtag
>                 / privateuse         ; private use tag
>                 / grandfathered      ; grandfathered registrations
> 
> langtag         = (lang
>                    *3("-" extlang)
>                    ["-" script]
>                    ["-" region]
>                    *("-" variant)
>                    *("-" extension)
>                    ["-" privateuse])
> 
> lang            = 2*4ALPHA           ; shortest ISO 639 code
>                 / registered-lang
> 
> registered-lang = 4*8ALPHA          ; registered language subtag
> 
> extlang         = 3ALPHA             ; reserved for future use
> 
> script          = 4ALPHA             ; ISO 15924 code
> 
> region          = 2ALPHA             ; ISO 3166 code
>                 / 3DIGIT             ; UN country number
> 
> variant         = 5*8alphanum       ; registered variants
>                 / ( DIGIT 3alphanum )
> 
> extension       = singleton 1*("-" (2*8alphanum))
> 
> singleton       = %x41-57 / %x59-5A / %x61-77 / %x79-7A / DIGIT
>                 ; "a"-"w" / "y"-"z" / "A"-"W" / "Y"-"Z" / "0"-"9"
>                 ; Single letters: x/X is reserved for private use
> 
> privateuse      = ("x"/"X") 1*("-" (1*8alphanum))
> 
> grandfathered   = 1*3ALPHA 1*2("-" (2*8alphanum))
>                                     ; grandfathered registration
>                                     ; Note: i is the only singleton
>                                     ; that starts a grandfathered tag
> 
> alphanum        = (ALPHA / DIGIT)   ; letters and numbers
> 
> 
> 
> --------------------------------------------------------------------------
> -
> 
> %% for -matching
> 
> Language-Range
>                 = langrange
>                 / privateuse         ; private use tag
>                 / grandfathered      ; grandfathered registrations
> 
> langrange       = (lang
>                    %%%%% I don't see why *3("-" extlang) is not included
> here
>                    ["-"script]
>                    ["-" region]
>                    *("-" variant)
>                    %%%%% I don't see why *("-" extension) is not included
> here
>                    ["-" privateuse])
> 
> lang            = 2*8ALPHA	%%%%%% I'm not sure about the non-trivial
> differences from -registry here
>                 / extlang
>                 / "*"
> 
> extlang         = 2*3ALPHA	%%%%%% I'm not sure about the non-trivial
> differences from -registry here
>                   *2("-" 3ALPHA)
>                   ("-"(3ALPHA / "*"))
> 
> script          = 4ALPHA             ; ISO 15924 code
>                 / "*"
> 
> region          = 2ALPHA             ; ISO 3166 code
>                 / 3DIGIT             ; UN country number
>                 / "*"
> 
> variant         = 5*8alphanum       ; registered variants
>                 / ( DIGIT 3alphanum )
>                 / "*"
> 
> privateuse      = ("x"/"X") 1*("-" (1*8alphanum))
> 
> grandfathered   = 1*3ALPHA 1*2("-" (2*8alphanum))
>                                     ; grandfathered registration
>                                     ; Note: i is the only singleton
>                                     ; that starts a grandfathered tag
> 
> alphanum        = (ALPHA / DIGIT)   ; letters and numbers
> 
> --------------------------------------------------------------------------
> --
> 
> I'm not sure about some non-trivial differences; like why is
> not "singleton" (and its reference) included in the -matching version?
> 
>     /kent k
> 
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Kent Karlsson" <[email-masked]>
Date: Wed, 20 Jul 2005 00:50:41 +0200 (CEST)

Ok, thanks for the explanation Addison.

After a bit more grammar mangling I've got the following
changes for -registry ABNF and -matching ABNF to my
draft cleanup. I'm not repeating unproblematic parts
of the grammars that I sent earlier. This is (supposed
to be) pure grammar beautification and parallelisation
between the two grammars.

---------------------------
%% For -registry

langtag         = (langext
                   ["-" script]
                   ["-" region]
                   *("-" variant)
                   *("-" extension)
                   ["-" privateuse])

langext         = lang
                / extlang

lang            = 2*4ALPHA           ; shortest ISO 639 code
                / 4*8ALPHA           ; registered language subtag

extlang         = (lang
                   1*3("-" 3ALPHA)   ; reserved for future use

---------------------------
%% for -matching

langrange       = (langext
                   ["-"script]
                   ["-" region]
                   *("-" variant)
                   ["-" privateuse])

langext         = lang
                / extlang
                / "*"

lang            = 2*4ALPHA           ; shortest ISO 639 code
                / 4*8ALPHA           ; registered language subtag

extlang         = (lang
                   *2("-" 3ALPHA)
                   ("-"(3ALPHA / "*")))
                                     ; reserved for future use

---------------------------

            /kent k