Subject: RE: [Ltru] Registry in record-jar format

From: "Addison Phillips" <[email-masked]>
Date: Wed, 23 Mar 2005 12:14:19 -0800

If we're going to consider different formats for the registry, might I suggest that an even better format would be XML?

XML can be structured in a way that would allow multiple separate elements to be defined when multiple values are called for. Adding new recommended prefixes, for example, could be done by adding <prefix> elements to the record.

This would also allow for some other useful things. For example, the registry could use the xml:lang attribute (ooh, self-referential objects are so cool) to provide alternate language descriptions or native language descriptions (since an XML file could use UTF-8 as the encoding or at least NCRs). The registry could also use a URI (or rather, an IRI) to reference the registration form itself (tying registration to the registry).

The DTD could also enforce a certain level of validity on the registry and could itself be versioned to accommodate 3066ter and 639-3 as necessary. 

Finally, unlike a flat text file, an XML format would be extensible. Future changes to the RFC could add fields to the registry without breaking existing registry processors. (RJ format can do this too, of course.)

Note that XML files are text files, so the presence of an XML processor itself is not required, especially if the registry uses NCRs for non-ASCII values.

Consider:

Type: variant
Subtag: hakka
Description: Hakka
Date: 1999-12-18
Recommended_Prefix: zh

Versus:

<subtag type="variant" value="hakka" date="1999-12-18">
  <description>Hakka</description>
  <description xml:lang="zh-hakka">.....</description>
  <reference>http://www.iana.org/assignments/lang-tags/zh-hakka</reference>
  <prefix>zh</prefix>
  <prefix>zh-Hant</prefix>  <!-- example only -->
  <note>You can put all sorts of useful information 
    into an XML document.
  </note>
  <!-- <canonical>...not for this one...</canonical> -->
</subtag>

Of course, an XML (or any format) version of the registry can be made from (say) the RJ format. We could pick one canonical format and consider whether to render informative versions in other formats. XML offers lots of flexibility for automating views of the data for that matter.

Just an idea........

Addison

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Randy Presuhn
> Sent: mercredi 23 mars 2005 10:54
> To: LTRU Working Group
> Subject: Re: [Ltru] Registry in record-jar format
> 
> Hi -
> 
> > From: "Doug Ewell" <[email-masked]>
> > To: "LTRU Working Group" <[email-masked]>
> > Sent: Tuesday, March 22, 2005 10:44 PM
> > Subject: [Ltru] Registry in record-jar format
> >
> 
> > At least one person expressed some interest in seeing the draft /
> > proposed / sample registry in record-jar format, so here it is:
> >
> > http://users.adelphia.net/~dewell/lstreg-rj.txt
> >
> > This has no official sanction or consensus; it's just there so people
> > can visualize it (which I guess is true for the other one as well).
> ...
> 
> Thanks for making this available.  It's always helpful to be able to see
> just how a proposal might be realized.
> 
> (co-chair hat off)
> I like this format better than the current one, for the following reasons:
>    1) it's much easier to read, and consequently less error-prone for
>       whoever ends up doing the actual edits to the registry files
>    2) it's still machine-parseable
>    3) it would permit the initial registry content to be documented in
>        an internet draft
> 
> Randy
> 
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 24 Mar 2005 07:16:27 +0100

Addison Phillips wrote:

> If we're going to consider different formats for the
> registry, might I suggest that an even better format would
> be XML?

<http://xml.resource.org/authoring/rfc2629.dtd> would be nice
to create I-Ds and later RfCs, see <http://xml.resource.org>

> This would also allow for some other useful things.

Apparently you want a completely new DTD for this purpose.
Does _any_ IANA registry use XML ?

> provide alternate language descriptions or native language
> descriptions (since an XML file could use UTF-8 as the
> encoding or at least NCRs).

The resulting plain text ASCII RfC and registry could be messy,
from the POV of a reader.  One or two literal &#1234; or u+04D2
in a paragraph are okay, but numerous of these beasts are ugly.

> The registry could also use a URI (or rather, an IRI) to
> reference the registration form itself (tying registration
> to the registry).

That's a good idea, add an URL as (optional) 7th column to the
registry, pointing to the approval of the tag reviewer, e.g.
http://www.iana.org/assignments/lang-tags/en-scouse

In the record-jar part of a future I-D or RfC something like:

%%
Type: variant
Subtag: scouse
Description: Scouse
Date: 2000-05-25
Recommended_Prefix: en
URL: http://www.iana.org/assignments/lang-tags/en-scouse
%%

> The DTD could also enforce a certain level of validity on the
> registry and could itself be versioned to accommodate 3066ter
> and 639-3 as necessary.

Is that a good idea ?  Matching language tags shouldn't depend
on XML parsers.  It's also recursive, 3066bis is a definition
of what we do in a xml:lang, and if that definition has its own
XML DTD not mentioned in the document with the xml:lang then
it's confusing.  If we'd want DC we'd know where to find it.

> XML offers lots of flexibility for automating views of the
> data for that matter.

True, but gawk is also a nice tool, it has no problem with the
record-jar (or similar) format.  But it's not exactly the best
choice to parse XML.
                        Bye, Frank

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Thu, 24 Mar 2005 10:41:35 +0100

Just a minute, gentlemen!
Could you please investigate first what will be the resulting use?

The IANA is a reference source, for a "few" developers or politics to 
consult. What was documented by the RFC 3066 up to now was a more flexible 
manner to register, store and make accessible for free what ISO do with 
paid papers.

The load resulting from RFC 3066 bis is more substantial since there are 
far more languages, possible subtags and resulting langtags to register and 
manage. If we consider the workload represented by registrations 
(proposition received from Michael Everson, registration, verification, 
possible errors and support) we are here in the range of a very large ccTLD 
with top quality services (we are only a few checking registered DNs). 
Also, before that I am not sure ietf-languages can cope with that workload 
(with a 15 days debate, etc.). Let remember that we have 7500 languages and 
240 countries (rough figures), I suppose that the total amount of 
registrations may be in the range of  30-100.000. Plus private registration 
- over the years this may amount to millions if SNHNs start playing the way 
every indication show they will. The first question about this is: who will 
pay?

Now, if you start talking of DTD, with several versions, and of XML 
presentations with additional data and linked URLs and IRIs, questions are:
- who is going to maintain the entries?
- there will be millions/billions of pages referencing the DTD, the urls, 
etc. which will lead to a IANA access 24/366. Some evaluations are needed 
about the resulting traffic and the CPU load before IANA can just discuss 
it. I am not sure, but we are replicating bigger than the DNS root here. 
One has to be aware that DoS and bug are very heavy on this kind of WHOIS 
is like systems (Iegitimate calls to the DNS root amount to 2.5% - 97.5% 
being more or less junk). I am not sure the IANA agreement with SRI or who 
ever else can stand the load.

I note also that for such a standardized and hopefully stable formatted 
system, the format can certainly be prototyped in using XML but a more
compact solution should to be considered. Most probably UDP supported.

I do not know the INRIA, etc. project we will see documented. But I note 
that this is a paid access service.

Resources have a cost. IANA is a "free" service because it is donated to 
the Internet community. I understand that Unicode/W3C want to use IANA 
"free" resources, but how much are these close paying membership structures 
ready to contribute to permit everyone to get this free service?

Right now ccTLDs are discussing how much to pay ICANN for its IANA service. 
And want to keep it very very low. Since the more we would ask from IANA 
the more we would depend on ICANN, so the more our economies and people 
would depend on a foreign uncontrolled source. (this is a common concern by 
every ccTLD, that USA share: no one knows who will manage the IANA two 
years from now).

We will certainly want to look in detail at this RFC 3066 ter service since 
our communities will most probably be large users of it. And we will 
certainly question its use and design to make sure we are requested to pay 
for the most appropriate solution. Several countries (and this is my own 
direct concern with AFRAC) consider how best to replicate and extend the 
IANA system for surety, security, stability, scalability, support, 
simplicity, sovereignty, innovation reasons. Economics, data consistency 
and operations considerations are also of the essence, since the 
granularity of these referencing systems will certainly extend down to the 
SNHNs.

I do not know why but I feel we are considering a gas refinery where a 
straw would be enough to sip the wiskey in the -jar.
jfc

---

From: "Jon Hanna" <[email-masked]>
Date: Thu, 24 Mar 2005 11:54:48 -0000

> in a paragraph are okay, but numerous of these beasts are ugly.

It would (and I'd hope eventually will) be much nicer if the RFC rules
allowed for UTF-8 wouldn't it?

Still, I'd imagine that the said XML section would be an appendix that
anyone studying in earnest would view as XML, rather than as ASCII source
formatted to fit the RFC line-width rules, so I don't foresee this being a
major issue.

Regards,
Jon Hanna
Work: <http://www.selkieweb.com/>
Play: <http://www.hackcraft.net/>
Chat: <irc://irc.freenode.net/selkie>

---

From: "Scott Hollenbeck" <[email-masked]>
Date: Thu, 24 Mar 2005 07:09:01 -0500

> Does _any_ IANA registry use XML ?

Yes -- take a look at these:

http://www.iana.org/assignments/xml-registry-index.html

-Scott-

---

From: John Cowan <[email-masked]>
Date: Thu, 24 Mar 2005 08:54:29 -0500

JFC (Jefsey) Morfin scripsit:

> I suppose that the total 
> amount of registrations may be in the range of  30-100.000.

The number of potential tags far exceeds that already.  The number of
useful tags is rather less.  But in any case, the whole point of
RFC 3066bis is to register only tag parts (and in 99% of cases directly
tracking the various ISO standards).

> Plus private 
> registration - over the years this may amount to millions if SNHNs start 
> playing the way every indication show they will.

What are these "indications"?

> compact solution should to be considered. Most probably UDP supported.

Nobody supposes that the tag registry must or should be retrieved in
real time as the DNS is.  Caching, lots of private caching, is the
order of the day, and it is already going on.

> straw would be enough to sip the wiskey in the -jar.

The straw in question is the strawman argument you are making.

-- 
Do NOT stray from the path!             John Cowan <[email-masked]>
        --Gandalf                       http://www.ccil.org/~cowan

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Thu, 24 Mar 2005 15:50:13 +0100

At 14:54 24/03/2005, John Cowan wrote:
>useful tags is rather less.

I rise a question, I do not dispute anything.
Can you quantify/qualify what is useful?

>But in any case, the whole point of

So, what is the need for any registration?

>What are these "indications"

You are welcome to work on this yourself. You will certainly better 
document them than I could.

>order of the day, and it is already going on.

I am extremely interested if you could document these IANA caching 
mechanisms. Distributed IANA is precisely what we (AFRAC) work on as a 
protoype Context Reference Center.

>The straw in question is the strawman argument you are making.

Oh! no problem: I am then a strawman for a few 30 years :-)

jfc

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 25 Mar 2005 03:11:48 +0100

> http://www.iana.org/assignments/xml-registry-index.html

In that case some of us would probably love XML, and others
(like me) still prefer record-jar or another 2822-style.

For the latter it's also simple to split it into A..Z IANA
pages with 7000+ languages.

The DTD or schema could be an appendix in the registry draft.

For Unicode-tags (language in their native script) it's better
to translate XML into record-jar format, the other direction
would be a pain.
                 Bye, Frank

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Fri, 25 Mar 2005 03:29:52 +0100

>(like me) still prefer record-jar or another 2822-style.

Please read RFC 3688 limitations including Part 6 on IANA considerations.

As one of those who will pay for that (ccTLDs) and will pay for mirrors 
(AFRAC), I thank those who have figures in mind to address my pending 
questions. I note that if the average load is one access per year per 
Internet user, the load of is 25 access per second. With root system ratio 
(which has been probably grown over time, but gives good examples of users 
behaviors) this represents 1000 accesses per seconds, hence picks (off DoS) 
of 10.000 and average piks of 3.000 (my guesstimate from comparable type of 
site - every correction/experience welcome).

jfc

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 25 Mar 2005 05:03:50 +0100

> Please read RFC 3688 limitations including Part 6 on IANA

Thanks, the last time I looked into this was when MARID tried
to get rid of some obscure XML-over-DNS ideas.  In the case
here I don't see a problem, the worst I'll ever try is to use
the W3C validator with the registry, because Addison said that
one advantage of XML is validation.

For a DTD with a SYSTEM identifier that works, I've tested it
with a SYSTEM "http://xml.resource.org/authoring/rfc2629.dtd"

For a FPI I'd know where I could ask the W3C validator team to
add it to their <http://validator.w3.org/sgml-lib> collection.

"Registry validation" is a very special case, nothing we need
to worry about.  A DTD is not good enough to handle the finer
points of the syntax like number of characters per subtag, and
the W3C validator is not tuned to handle XML, but these are all
Addison's problems.  I'm happy with ABNF and record-jar / 2822.

> As one of those who will pay for that (ccTLDs) and will pay
> for mirrors (AFRAC), I thank those who have figures in mind
> to address my pending questions.

How you mirror IANA is your problem, but if you can handle the
about 1 MB in 50,000 lines of a future 3066ter registry, then
you can also handle about 2 MB in an equivalent XML format.

You're IMHO not forced to squeeze this in a mobile device, and
you're not limited by display resolutions for Unicode fonts.
Or maybe you are, but then the size of a language tag registry
is not your most pressing problem.

                          Bye, Frank