Subject: Re: [Ltru] draft-15 posted...

From: "Doug Ewell" <[email-masked]>
Date: Wed, 11 Jun 2008 07:28:06 -0600

Kent Karlsson <kent dot karlsson14 at comhem dot se> wrote:

> While this might not affect registry entries, this grammar is 
> confusing and very far from my liking. Please keep these problem cases 
> ("no-bok" / "no-nyn" / "zh-min" / "zh-min-nan") as "irregular" even if 
> they are also (incorrectly) covered by other parts of the grammar, 
> noting that the "regular" part of the grammar MUST NOT be used for 
> these problem cases.

Tags like "no-bok" are syntactically regular, but semantically 
irregular: they don't mean what you would assume by looking at the 
individual subtags.  I don't know if ABNF is supposed to capture that. 
I'm sure there are ABNF experts who can address this.

> B.t.w. I'd like to see the non-extlang versions of the now introduced 
> extended double-coding to be the preferred coding. Indeed, I would 
> still prefer to go back to the -14 grammar, and no extlangs at all.

We could go back and open the extlang discussion all over again, and 
spend another month or two debating it, I suppose.

--
Doug Ewell  *  Arvada, Colorado, USA  *  RFC 4645  *  UTN #14
http://www.ewellic.org
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages  ˆ

_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 11 Jun 2008 15:51:02 +0200

On Wed, Jun 11, 2008 at 07:28:06AM -0600,
 Doug Ewell <[email-masked]> wrote 
 a message of 7 lines which said:

> irregular:

The politically correct terms would be "Tags like 'no-bok' are
well-formed but invalid".

> I don't know if ABNF is supposed to capture that. 

No, it cannot, in the same way that it cannot capture that ar-Cyrl-CP
is absurd.

---

From: John Cowan <[email-masked]>
Date: Wed, 11 Jun 2008 10:47:02 -0400

> well-formed but invalid".

Sort of.  The irregular tags would not be well-formed if they were not
specially provided for in the BNF and the registry.  The grandfathered
but not irregular tags would be well-formed and not valid if they were
not specially provided for in the registry.  Since they are provided for,
they are all well-formed and valid.

-- 
John Cowan    http://ccil.org/~cowan    [email-masked]
Mr. Henry James writes fiction as if it were a painful duty.  --Oscar Wilde
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Phillips, Addison" <[email-masked]>
Date: Wed, 11 Jun 2008 08:01:24 -0700

> well-formed but invalid".

Ur... not exactly. "no-bok" is well-formed and valid. What it isn't is composed of subtags with independent meaning. That is, it is semantically irregular.

In fact, the subtag 'bok' will mean the language "Bonjo" as a primary language subtag. The tag "no-bok" does not mean Norwegian Bonjo.

Addison

---

From: Peter Constable <[email-masked]>
Date: Thu, 12 Jun 2008 07:54:41 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Doug Ewell

> individual subtags.

Do we have an easier means to capture the semantic irregularity than to treat these tags as syntactically irregular?

Peter

---

From: Peter Constable <[email-masked]>
Date: Thu, 12 Jun 2008 07:58:09 -0700

Sorry, not thinking clearly: we want to keep these grandfathered to capture the semantic irregularity, right?

> -----Original Message-----
> Peter Constable
> Sent: Thursday, June 12, 2008 7:55 AM
> To: LTRU Working Group
> Subject: Re: [Ltru] draft-15 posted...
>
> Do we have an easier means to capture the semantic irregularity than to
> treat these tags as syntactically irregular?
>
>
>
> Peter

---

From: "Mark Davis" <[email-masked]>
Date: Thu, 12 Jun 2008 08:23:57 -0700

I agree. The ABNF defines syntactic validity; the registry provides validity
and semantic interpretation. As part of that, grandfathering is used for
these semantically irregular tags.

Mark

---

From: John Cowan <[email-masked]>
Date: Thu, 12 Jun 2008 11:33:26 -0400

Peter Constable scripsit:

> Sorry, not thinking clearly: we want to keep these grandfathered to
> capture the semantic irregularity, right?

Exactly.

-- 
A mosquito cried out in his pain,               John Cowan
"A chemist has poisoned my brain!"              http://www.ccil.org/~cowan
        The cause of his sorrow                 [email-masked]
        Was para-dichloro-
Diphenyltrichloroethane.                                (aka DDT)
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Phillips, Addison" <[email-masked]>
Date: Thu, 12 Jun 2008 11:00:58 -0700

While I agree, I also see what Kent is getting at. There is a way we can satisfy both: we can provide a separate ABNF production in 2.2.9 (conformance) that lists the 3066-registrations. Such an ABNF could have separate productions for redundant, potentially redundant, semantically distinct but syntactically redundant, and irregular sets of tags. These turn out to be distinct lists that comprise, if I’m not mistaken, the complete set of registrations. Some implementers might then choose to match this production before doing a well-formed check using the “regular” ABNF.

Addison

Addison Phillips
Globalization Architect -- Lab126

Internationalization is not a feature.
It is an architecture.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Mark Davis
Sent: Thursday, June 12, 2008 8:24 AM
To: Peter Constable
Cc: LTRU Working Group
Subject: Re: [Ltru] draft-15 posted...

I agree. The ABNF defines syntactic validity; the registry provides validity and semantic interpretation. As part of that, grandfathering is used for these semantically irregular tags.

Mark
On Thu, Jun 12, 2008 at 7:58 AM, Peter Constable <[email-masked]<mailto:[email-masked]>> wrote:
Sorry, not thinking clearly: we want to keep these grandfathered to capture the semantic irregularity, right?

> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of
> Peter Constable
> > From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf
> Of
> > Doug Ewell
>
>
> [email-masked]<mailto:[email-masked]>

_______________________________________________
Ltru mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Thu, 12 Jun 2008 14:05:25 -0400

Phillips, Addison scripsit:

> While I agree, I also see what Kent is getting at. There is a way
> we can satisfy both: we can provide a separate ABNF production in
> 2.2.9 (conformance) that lists the 3066-registrations. Such an ABNF
> could have separate productions for redundant, potentially redundant,
> semantically distinct but syntactically redundant, and irregular sets
> of tags. These turn out to be distinct lists that comprise, if I’m
> not mistaken, the complete set of registrations. Some implementers
> might then choose to match this production before doing a well-formed
> check using the “regular” ABNF.

Please see my 2.2.8 proposal.

-- 
Mark Twain on Cecil Rhodes:                    John Cowan
I admire him, I freely admit it,               http://www.ccil.org/~cowan
and when his time comes I shall                [email-masked]
buy a piece of the rope for a keepsake.
_______________________________________________
Ltru mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ltru

---

From: "Kent Karlsson" <[email-masked]>
Date: Fri, 13 Jun 2008 01:17:15 +0200

Phillips, Addison wrote: 

I'm not at all keen on having two stages of grammars. And while the generative view of grammars is just to produce a set of strings without any analysis, grammars as used in computing are most often used to generate some kind of parse with a structure, and that structure then guiding further processing (in this case registry lookup). And while I can certainly special treat the problem cases, I do want to make sure that that is clearly communicated to other implementers. Other implementers who, by the way, are used to (somewhat) ambiguous grammars like those for most programming languages (though perhaps not used to process very ambiguous grammars as those constructed for modelling natural languages).

Note that the 4646 ABNF allowed for the correct parse of all language tags, the draft -15 ABNF forces an incorrect parse of some language tags, which I still find is something to howl about...

Whether these tags are obsolete, deprecated, just nine of them, or the solution is gerrymandering, is quite irrelevant for this issue. Despite the howling complaints against my suggestion, I still stand by my suggestion, and in all details it is as follows.

The additions to the ABNF are in bold black:

Language-Tag  = langtag

              / privateuse             ; private use tag

              / irregular              ; tags grandfathered by rule

              / falselyregular         ; tags grandfathered by rule,

                                       ;   but look like regular tags

langtag       = (language

                 ["-" script]

                 ["-" region]

                 *("-" variant)

                 *("-" extension)

                 ["-" privateuse])

language      = 2*3ALPHA               ; shortest ISO 639 code

                [extlang]              ; sometimes followed by 

                                       ;   extended language subtags

              / 4ALPHA                 ; reserved for future use

              / 5*8ALPHA               ; registered language subtag

extlang       = "-" 3ALPHA             ; selected ISO 639 codes,

                                       ;   '-bok', '-nyn', and '-min'

                                       ;    must not match

                *2[ "-" 3ALPHA]        ; permanently reserved

script        = 4ALPHA                 ; ISO 15924 code

region        = 2ALPHA                 ; ISO 3166-1 code

              / 3DIGIT                 ; UN M.49 code

variant       = 5*8alphanum            ; registered variants

                                       ;   '-lojban' must not match

                                       ;   if the prefix is 'art'

                                       ;   '-gaulish' must not match

                                       ;   if the prefix is 'cel'

                                       ;   '-guoyo', '-hakka', and

                                       ;   '-xiang' must not match

                                       ;   if the prefix is 'zh'

              / (DIGIT 3alphanum)

extension     = singleton 1*("-" (2*8alphanum))

singleton     = %x41-57 / %x59-5A / %x61-77 / %x79-7A / DIGIT

              ; "a"-"w" / "y"-"z" / "A"-"W" / "Y"-"Z" / "0"-"9"

              ; Single alphanumerics

              ; "x" is reserved for private use

privateuse    = "x" 1*("-" (1*8alphanum))

irregular     = "en-GB-oed" / "i-ami" / "i-bnn" / "i-default"

              / "i-enochian" / "i-hak" / "i-klingon" / "i-lux"

              / "i-mingo" / "i-navajo" / "i-pwn" / "i-tao"

              / "i-tay" / "i-tsu" / "sgn-BE-FR" / "sgn-BE-NL"

              / "sgn-CH-DE"

falselyregular= "art-lojban" / "cel-gaulish" / "no-bok" / "no-nyn"

              / "zh-guoyu" / "zh-hakka" / "zh-min" / "zh-min-nan"

              / "zh-xiang"            ; these tags do not contain

                                      ;   extlangs nor variants

alphanum      = (ALPHA / DIGIT)       ; letters and numbers

I do not propose any changes to the proto-registry (4645bis) because of this. There was some spin-off from the discussion, but that is irrelevant for this ABNF proposal as such (but is relevant to the second suggested sentence below, but that is not a vital sentence).

The text in 4646bis should say something like the following (since the comments aren't normative, but should still of course be correct):

“The ‘falselyregular’ productions take priority over other productions. However, all of the ‘falselyregular’ language tags are deprecated.”

The last sentence is assuming that you accept John’s spin-off suggestion to deprecate cel-gaulish.

        /kent k

---

From: "Kent Karlsson" <[email-masked]>
Date: Fri, 13 Jun 2008 07:07:52 +0200

Editorial nit:
 
I note that for some reason the leading "-" is included in the extlang production, wheras for other items with a leading "-",
the leading "-" is external to the item production. I'd suggest the same for the extlang production. So for 'language' (two first lines ony):

                ["-" extlang]          ; sometimes followed by 

...
and for 'extlang':

extlang       = 3ALPHA                 ; selected ISO 639 codes,

                                       ;   'bok', 'nyn', and 'min'

                                       ;    must not match

                *2["-" 3ALPHA]         ; permanently reserved

I'm not sure why the latter line should not say *2("-" 3ALPHA) instead.

Also remove the spurious "-":s I had in the comment for 'variant'.

    /kent k

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of Kent Karlsson
Sent: Friday, June 13, 2008 1:17 AM
To: 'LTRU Working Group'
Subject: Re: [Ltru] draft-15 posted...

Phillips, Addison wrote: 

Language-Tag  = langtag

langtag       = (language

                 ["-" script]

                 ["-" region]

                 *("-" variant)

                 *("-" extension)

                 ["-" privateuse])

                                       ;    must not match

              / 3DIGIT                 ; UN M.49 code

                                       ;   if the prefix is 'art'

                                       ;   if the prefix is 'cel'

                                       ;   if the prefix is 'zh'

              / (DIGIT 3alphanum)

              ; Single alphanumerics

              / "sgn-CH-DE"

                                      ;   extlangs nor variants

 

 

 

 

        /kent k