Subject: RE: [Ltru] Re: Registry in record-jar format

From: "Peter Constable" <[email-masked]>
Date: Thu, 24 Mar 2005 07:41:42 -0800

> From: [email-masked] [mailto:[email-masked]]
On
> Behalf Of JFC (Jefsey) Morfin

> I am just interested to know if what you consider spurious is
"non-ASCII"
> or "lots". And if you think user's descriptive fields MUST or SHOULD
be in
> ASCII (obviously only a MUST would go your way). Also, when you speak
of
> normative data, do you speak of normative codes and numbers or of all
the
> data in the norm, i.e. including French texts, (and happily sometimes,
for
> the language norms Chinese, Russian, Arabic, etc. will be supported -
> again
> only a question of budget and international agreements we do not
control)?

Please do NOT consider adding names of languages, scripts, regions, etc.
in multiple languages (the Chinese name for "Devanagari" etc.) to the
registry. This should not be a localization-data repository. The last
thing we need is to have the registry data file bloat to some huge size.

Peter Constable

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 24 Mar 2005 08:36:31 -0800

Normative data in 3066bis would be the subtags, subtag type identifiers, ISO 8601 dates, canonical mappings, and recommended prefixes. Non-normative fields would be descriptions and notes.

These normative fields are restricted to ASCII because language tags are or because ISO 8601 says so.

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: JFC (Jefsey) Morfin [mailto:[email-masked]]
> Sent: jeudi 24 mars 2005 06:25
> To: Addison Phillips; [email-masked]
> Subject: RE: [Ltru] Re: Registry in record-jar format
> 
> At 14:32 24/03/2005, Addison Phillips wrote:
> >The question about lots of non-ASCII (JFC's comment) is a spurious one:
> by
> >rule it should be limited to the name of the described language or
> >language part in that language or to descriptive fields (both
> >informative). The normative data would still be restricted to ASCII.
> 
> I am always happy to read constructive comments. Obviously ASCII fits the
> ASCII Internet. The question is will it fit the Multilingual Internet we
> all work on?
> 
> I am just interested to know if what you consider spurious is "non-ASCII"
> or "lots". And if you think user's descriptive fields MUST or SHOULD be in
> ASCII (obviously only a MUST would go your way). Also, when you speak of
> normative data, do you speak of normative codes and numbers or of all the
> data in the norm, i.e. including French texts, (and happily sometimes, for
> only a question of budget and international agreements we do not control)?
> ASCII will not even fully support French which is a normative language.
> 
> jfc
> 
>

---

From: "Peter Constable" <[email-masked]>
Date: Thu, 24 Mar 2005 10:46:56 -0800

to which Addison replied:

> For the record, the suggestion was to replace or augment the current:
> 
>    8. Native name of the language or variation (transcribed into
ASCII):
> 
> ... with the native name of the language or variation.

I was responding to our not-always-helpful contributor JFC, who, because
he believes we should be working toward "multilingualization" rather
than "internationalization" (in quotes because I cannot guarantee that
he uses that term to mean what anybody else means), appears very
definitely to be wanting to be able to document the names of languages
as expressed in every other language. He replied off-list to my comment
above saying so.

Peter Constable

---

From: Misha Wolf <[email-masked]>
Date: Thu, 24 Mar 2005 18:50:27 +0000

I suggest we 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of 
Sent: 24 March 2005 18:47
To: [email-masked]
Subject: RE: [Ltru] Re: Registry in record-jar format

Peter Constable

-----------------------------------------------------------------
        Visit our Internet site at http://www.reuters.com

To find out more about Reuters Products and Services visit http://www.reuters.com/productinfo 

Any views expressed in this message are those of  the  individual
sender,  except  where  the sender specifically states them to be
the views of Reuters Ltd.

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 24 Mar 2005 20:41:16 -0800

Well, one doesn't want to claim too much for any specific format. Validation of the registry's structure would be limited by the limits of what a DTD can reasonably do.

I like XML because there are lots of tools for working with it and doing useful things, but frankly you're correct that the Ur-format really doesn't matter. All of these formats are essentially just permutations of the same thing (absent that XML can do some structural backflips with multiple entries that are harder in record-jar format).

Size is also not a real limiting factor, especially once one factors in the ability to transform from one format to another.

I must admit that I think it a good idea, though, to choose one format for the normative registry and make the others an informative mirror of that. That answers the "where do I look" problem. I noticed yesterday, for example, that IANA's new format web page doesn't link properly to the language tag registration forms (the URI is mangled in the href), making certain information difficult to access. If there is more than one normative format and the registry copies ever get out of sync for some (human error?) reason, it is useful to know which registry version one is supposed to look at.

Otherwise, despite earlier arguments, the format isn't that important to me. Finishing the work is far more so.

Addison

Addison P. Phillips

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Frank Ellermann
> Sent: jeudi 24 mars 2005 20:04
> To: [email-masked]
> Subject: [Ltru] Re: Registry in record-jar format
> 
> JFC (Jefsey) Morfin wrote:
> 
> > Please read RFC 3688 limitations including Part 6 on IANA
> > considerations.
> 
> Thanks, the last time I looked into this was when MARID tried
> to get rid of some obscure XML-over-DNS ideas.  In the case
> here I don't see a problem, the worst I'll ever try is to use
> the W3C validator with the registry, because Addison said that
> one advantage of XML is validation.
> 
> For a DTD with a SYSTEM identifier that works, I've tested it
> with a SYSTEM "http://xml.resource.org/authoring/rfc2629.dtd"
> 
> For a FPI I'd know where I could ask the W3C validator team to
> add it to their <http://validator.w3.org/sgml-lib> collection.
> 
> "Registry validation" is a very special case, nothing we need
> to worry about.  A DTD is not good enough to handle the finer
> points of the syntax like number of characters per subtag, and
> the W3C validator is not tuned to handle XML, but these are all
> Addison's problems.  I'm happy with ABNF and record-jar / 2822.
> 
> > As one of those who will pay for that (ccTLDs) and will pay
> > for mirrors (AFRAC), I thank those who have figures in mind
> > to address my pending questions.
> 
> How you mirror IANA is your problem, but if you can handle the
> about 1 MB in 50,000 lines of a future 3066ter registry, then
> you can also handle about 2 MB in an equivalent XML format.
> 
> You're IMHO not forced to squeeze this in a mobile device, and
> you're not limited by display resolutions for Unicode fonts.
> Or maybe you are, but then the size of a language tag registry
> is not your most pressing problem.
> 
>                           Bye, Frank
> 
> 
>

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 24 Mar 2005 21:52:44 -0800

Addison Phillips <addison dot phillips at quest dot com> wrote:

> ... with the native name of the language or variation. In some cases
> there may be several of these (witness zh-hakka, although one suspects
> these are all represented by the same or at least a smaller set of
> ideographs). There are also some regions that have naming problems in
> ASCII (ISO 3166 shows the Åland Islands surrounded by a sea of ASCII).

Mark Davis <mark dot davis at jtcsv dot com> responded:

> If we did that, it should be an optional, additional field.

I agree.  UN/LOCODE solves this problem by having two "name" fields, one
in Latin-1 and another smashed to ASCII (i.e. each non-ASCII letter
converted 1-to-1 to an ASCII letter, with no attempt to be
linguistically correct).  It seems to solve a problem they had.

I would be concerned about an attempt to expand "native name" beyond
Latin-1, though.  Although Unicode-capable systems are commonplace now
and we have the technology to display "Armenian" in Armenian and
"Chinese" in Han, real people still have to be able to read this stuff.
Not everyone using the registry will be capable of reading multiple
scripts, and we shouldn't look like we're doing it just to show off.

I'd just as soon give the Åland Islands back their ring, give Côte
d'Ivoire its circumflex and Réunion its acute, and pretty much leave it
at that.

-Doug Ewell
 Fullerton, California
 http://users.adelphia.net/~dewell/

---

From: "McDonald, Ira" <[email-masked]>
Date: Fri, 25 Mar 2005 09:14:15 -0800

Hi,

Please don't mis-use XML attributes to be the equivalent of
'attributes' in the object-oriented sense.  They aren't and
the protocol specs I've seen that used them caused nightmares
when trying to morph them into proper hierarchical elements.

And bear in mind what Scott Hollenbeck just reminded us about,
to whit, _nothing_ is up to IANA to figure out.  This WG has
to specify complete processes and formats for use by IANA in 
our 'IANA Considerations' section.

Cheers,
- Ira

Ira McDonald (Musician / Software Architect)
Blue Roof Music / High North Inc
PO Box 221  Grand Marais, MI  49839
phone: +1-906-494-2434
email: [email-masked]

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On
Behalf Of Doug Ewell
Sent: Friday, March 25, 2005 2:02 AM
To: LTRU Working Group
Subject: [Ltru] Re: Registry in record-jar format

Addison Phillips <addison dot phillips at quest dot com> wrote:

> If we're going to consider different formats for the registry, might I
> suggest that an even better format would be XML?

You might.

Oh, all right, I'll do an XML version too and post it, so you guys can
see that one too.

XML has all the advantages and disadvantages that people have mentioned
so far, and probably many more of both.  XML is sort of like California:
anything anyone can say about it is true.

Actually I had wondered before about converting the registry to XML, but
wasn't sure how it should be structured.  (The eternal question:
"Elements or attributes?  Which will serve me best?")  For this version,
I'll use Addison's example blindly, but without any extra information
(such as URLs) that isn't already in the existing registry.  Don't hold
your breath; I'll announce when it's ready.

-Doug Ewell
 Fullerton, California

---

From: "Addison Phillips" <[email-masked]>
Date: Fri, 25 Mar 2005 09:52:01 -0800

A real XML DTD would have to be carefully designed. My example in the email was very much a back-of-the-envelope example.

I agree that everything has to be in the RFC governing the registry. If there are to be multiple formats, then they all have to be documented, along with their production methods, for IANA and others.

The key is getting consensus on what to do.

Right now I think the options are:

1. Change to record-jar format.
2. Change to an XML format. A proper DTD needs to be designed in this case.
3. Neither.
4. Both (record-jar is normative)
5. Both (XML is normative)

Right this second I support (2), but would be (almost) as happy with (1).

Addison

Addison P. Phillips

> Behalf Of McDonald, Ira
> Sent: vendredi 25 mars 2005 09:14
> To: 'Doug Ewell'; LTRU Working Group

---

From: Misha Wolf <[email-masked]>
Date: Sat, 26 Mar 2005 16:37:42 +0000

Doug Ewell wrote:
 
<quote>
My point was that 600,000 bytes isn't all that much in comparison to the
things we store and pass around nowadays.  That's less than half a
floppy disk, and everyone knows we don't use floppies any more because
they're considered too small to hold anything useful.
</quote>

Do you have in mind an 8 inch or a 5.25 inch floppy disk?

Misha

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 26 Mar 2005 17:21:20 -0800

Misha Wolf <Misha dot Wolf at reuters dot com> wrote:

> <quote>
> My point was that 600,000 bytes isn't all that much in comparison to
> the things we store and pass around nowadays.  That's less than half a

I have in mind either a 5.25-inch or a 3.5-inch high-density diskette,
either of which can hold at least 1.2 MB.  I haven't seen a computer
built in the last 25 years that used an 8-inch diskette.  We had some in
the college bookstore warehouse where I worked in the '80s; we
considered them museum pieces even then.

The 5.25-inch HD diskette has been available since 1984, and the
3.5-inch HD since 1987.  Very few PCs sold since the mid-'90s support
only the low-density (720K) diskettes, which would still be large enough
to hold a 600K registry.  5.25-inch drives are getting quite hard to
find by now.

I shouldn't have brought this up.  The registry won't be anywhere near
this big unless and until the draft makes a concrete provision to
support ISO 639-3 codes.  Right now it defines a placeholder for them,
but does not specify anything else about them (e.g. when are they used
as primary language subtags and when as extlang subtags).

-Doug Ewell
 Fullerton, California

---

From: "Addison Phillips" <[email-masked]>
Date: Tue, 29 Mar 2005 17:19:25 -0800

bs-Latn was registered, or should have been. I suppose one could dig out the approval note and bug Michael about it.

> BTW, should we deprecate en-GB-oed and zh-min-nan ? 

Maybe, although it isn't necessary. After the draft is completed a new variant can be registered and the canonical mapping field updated to deal with these few cases. This could be done in the prototype registry for completeness sake, I suppose.

> And / or
> list the few grandfathered exceptions from the new general
> pattern directly in the ABNF ?

I think listing the grandfathered tags in the ABNF just makes very hard to read ABNF. Besides, the list can change up to the point where the draft takes effect. The text of the draft does a good enough job of enumerating these beasts, IMHO.

Addison

Addison P. Phillips

> Sent: mardi 29 mars 2005 17:10
> To: [email-masked]
> > let alone the newer ones like "bs-Latn" and "sl-rozaj".
> 
> The IANA/assignments/lang-tags/ Web page doesn't list bs-Latn.
> The most recent IANA tag is still en-GB-oed (2003-07-09).
> 
> BTW, should we deprecate en-GB-oed and zh-min-nan ?  And / or
> list the few grandfathered exceptions from the new general
> pattern directly in the ABNF ?
> 
>                         Bye, Frank
> 
> 
>

---

From: "Addison Phillips" <[email-masked]>
Date: Wed, 30 Mar 2005 09:12:31 -0800

> If you're thinking about registering new subtags to replace these,
> remember:  Grandfathered tags are forever.  They don't drop off the
> registry just because they've been deprecated.  All that does is create
> a duplicate tag with the same meaning, which is not a desirable thing in
> and of itself.

I would add that there isn't a requirement that every grandfathered tag start life as a deprecated tag. Registering a new variant subtag to replace (for example) en-GB-oed would provide a way of referencing that particular orthographic distinction (OED preferred spellings of English) for use with the generative mechanism or in combination with other variations or extensions.

But there isn't a requirement that this be done and the tag "en-GB-oed" will still be valid for representing such content.

Addison

Addison P. Phillips

> Sent: mardi 29 mars 2005 22:00
> To: LTRU Working Group
> Frank Ellermann <nobody at xyzzy dot claranet dot de> wrote:
> 
> That's the problem.  Nine tags *have* been registered since then, but
> they're still not listed on the IANA page.
> 
> Why would we want to do that?
> 
> If you're thinking about registering new subtags to replace these,
> remember:  Grandfathered tags are forever.  They don't drop off the
> registry just because they've been deprecated.  All that does is create
> a duplicate tag with the same meaning, which is not a desirable thing in
> and of itself.
> 
> > And / or list the few grandfathered exceptions from the new general
> No, for exactly the reasons Addison mentioned.
> 
> -Doug Ewell
>  Fullerton, California