Subject: [Ltru] IANA XML registries

From: David Conrad <[email-masked]>
Date: Tue, 19 Sep 2006 17:41:49 -0700

Hi,

Stephane Bortzmeyer <bortzmeyer at nic dot fr> wrote:
>> Any news about the supposed plan to move all IANA registries to XML?
> > No idea. Do you mind if I ping the IANA about that?

Yes, IANA is undertaking to convert all our registries over to XML.   
We'll be doing this in an incremental fashion of course, starting  
with a few of the simpler registries, getting feedback from people,  
revising and doing more registries, getting feedback, etc. until  
everything is converted over.  Part of this project will also be to  
generate "legacy" registries from the XML in formats that are as  
close to what exists now as we can easily manage (in an undoubtedly  
vain attempt to try to limit the screams of outrage from the poor  
souls who have hacked software to deal with the existing registries).

We anticipate having the first set of registries (and various backend  
systems we use internally) done by sometime around the beginning of  
2007.  A more formal announcement of this effort will probably be  
made at the San Diego IETF.  I'm happy to answer any questions folk  
might have...

Regards,
-drc
General Manager, IANA

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Sep 2006 08:48:00 +0200

David Conrad wrote:

> Part of this project will also be to generate "legacy"
> registries from the XML in formats that are as close to what
> exists now as we can easily manage (in an undoubtedly
> vain attempt to try to limit the screams of outrage from the
> poor souls who have hacked software to deal with the existing
> registries).

That's rather important, e.g. it's not yet mentioned in a new
<draft-narten-iana-considerations-rfc2434bis> submitted five 
days ago.  I saw no discussion about it on the XML-dir list.

It can affect any charset registry cleanup attempts (and Ira's
tool donated to IANA), and of course it's also interesting for
any modifications of the language subtag registry format.

With Stephane's tip "Apricot" and Google I found a note from
you in March about "eIANA", that sounded perfectly harmless,
new Web site, new forms, faster, better, enjoy.  Google didn't
find what you said in January, but apparently half of the IAB
should have known that that's coming, and not a minor issue.

Maybe they ignored it at this time because it was before the
renewal or SLA or whatever it was (IANAL), but forgetting it
was "sub-optimal" (I hope that word has the same connotations
in English as in German).

Thanks for info

Frank

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 20 Sep 2006 09:15:45 +0200

On Wed, Sep 20, 2006 at 08:48:00AM +0200,
 Frank Ellermann <[email-masked]> wrote 
 a message of 32 lines which said:

> I found a note from you in March about "eIANA", that sounded
> perfectly harmless,

eIANA has nothing to do with XML. It is a Web interface for TLD
managers to ask IANA and the DoC about the changes they request in the
root zone file of the DNS. No relationship with the "ordinary"
registry business of IANA (the one we depend on) and no relationship
with LTRU.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 20 Sep 2006 09:17:17 +0200

On Wed, Sep 20, 2006 at 08:48:00AM +0200,
 Frank Ellermann <[email-masked]> wrote 
 a message of 32 lines which said:

> but apparently half of the IAB should have known that that's coming,
> and not a minor issue.

I want to repeat that RFC 4646 is an exception, not the rule. Most of
the RFCs which instruct IANA to create a registry leave the details
completely open and, unlike RFC 4646, never specify a format for the
registry.

IMHO, it means IANA is quite free to change the master format.

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Sep 2006 10:14:27 +0200

Stephane Bortzmeyer wrote:

> eIANA has nothing to do with XML. It is a Web interface for
> TLD managers to ask IANA and the DoC about the changes they
> request in the root zone file of the DNS.

While they're at it whois.iana.org should be integrated into
WDPRS, there are many bogus whois servers and broken URLs in
their data.  Or at least that was the case some months ago.

> No relationship with the "ordinary" registry business of IANA

Registry is registry, when I read "better forms" I thought it
could be an improved version of:

http://www.iana.org/protocols/forms.htm

> (the one we depend on) and no relationship with LTRU.

 From my POV it's related, I'm here because I didn't like what
the pre-LTRU draft said about country codes, and that's because
of my whois client with TLDs including ccTLDs.  "Ask IANA" was
about the first thing it got right (after "ask abuse.net").

Frank

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 20 Sep 2006 10:40:19 +0200

Stephane Bortzmeyer wrote:

> Most of the RFCs which instruct IANA to create a registry
> leave the details completely open and, unlike RFC 4646, 
> never specify a format for the registry.

The _intended_ format of RFC 3864 is apparently like what you
find in RFC 4021, the published registry in...

http://www.iana.org/assignments/message-headers/perm-headers.html

...is rather minimalistic.

> IMHO, it means IANA is quite free to change the master format

For stuff that's in essence a collection of <name> ":" <value>
fields in registration templates (= a "record" of record-jar)
XML is not the ideal choice.  Admittedly for the few registries
I care about XML would be better than the status quo.  Minus
LTRU, that's fine.

Frank

---

From: David Conrad <[email-masked]>
Date: Wed, 20 Sep 2006 08:18:23 -0700

Frank,

> days ago.  I saw no discussion about it on the XML-dir list.

To clarify, the XML-ization of IANA registries is an _internal_  
effort, primarily aimed at allowing us to simplify/automate registry  
creation and modification.  We will take what efforts we can to  
insure there is no significant change to the "legacy" registries.  We  
will _also_ publish the initial set of XML-ized registries (at  
different URLs of course) and request comments.  Assuming the initial  
forays into XML-ization are deemed acceptable, we'll publish more.   
If not (hey, it could happen), we'll not publish more (but we'll  
still use the XML registries internally).

> new Web site, new forms, faster, better, enjoy.

As Stephane indicated, eIANA is completely unrelated to the registry  
XML-ization effort.

> Google didn't

Actually, I believe it is a minor issue (at least from an external  
perspective).

Rgds,
-drc

---

From: David Conrad <[email-masked]>
Date: Wed, 20 Sep 2006 08:28:51 -0700

Frank,

> their data.  Or at least that was the case some months ago.

IANA has another project to replace the current whois server system  
we use.  This project will, of course, be related to the deployment  
of eIANA.  However, in the meantime, I will look into linking into  
WDPRS.

> http://www.iana.org/protocols/forms.htm

No, that's yet another IANA project (as Stephane said in a previous  
message, we're pretty busy :-)) but it is somewhat related in the  
sense that one of the reasons for XMLization of the registries is to  
normalize how the data is represented internally so we can more  
easily automate forms for registry modification requests.

Rgds,
-drc

---

From: David Conrad <[email-masked]>
Date: Wed, 20 Sep 2006 09:01:41 -0700

Jefsey,

Lots of questions, most of which I suspect are not really related to  
ltru.  I'll answer these questions here, but I suspect there are  
better places for continued discussion on these topics.

On Sep 20, 2006, at 5:18 AM, r&d afrac wrote:
> Do you have a completion date in mind?

As I've indicated, we hope to have the first set of XML-ized  
registries available for comment around the beginning of 2007.   
Further publication will depend on input from the affected communities.

> Do you take advantage of this to make them ISO 11179 conformant?  
> Will it be organised as an ontology? Will you adopt a Dublin Core  
> approach?

If these are appropriate.  I am not an XML expert and will look to  
others to provide advice on the best way of representing IANA  
registry data externally.

> Will you include a secure update annoucement list?

I'm unsure of what you mean.  We will, in the relatively near future,  
be PGP signing IANA announcements.

> Will you include a consistency check solution for comparing two  
> versions of a registry?

Yes.

> Will you include a daily distributed status of these checks to  
> avoid millions of unnecessary down loads.

Hadn't considered it, but if this is desirable, IANA can publish  
diffs of the registry updates.

> Will you include a single IANA procedure for registry management  
> and review (mailing list, reviewer, appeals, cross registry  
> consistency, CVS).

Having a consistent approach to registry management is a goal.  How  
we manage registries internally is something that is evolving (as you  
might guess).

> Will you support a multilingual approach?

Not IANA's call.  What is contained within the vast majority of  
registries is defined by the IETF.

> Have you planned a distributed dissemination mechanism?

There have been discussions along these lines, but nothing formalized  
as yet.

> Have you planned a procedure/hooks for personal extensions?

No.

Rgds,
-drc

---

From: John Cowan <[email-masked]>
Date: Wed, 20 Sep 2006 12:46:22 -0400

> diffs of the registry updates.

This is indeed a valid LTRU concern, because at the end of the RFC 4646bis
process our registry will grow to about a megabyte.  We definitely don't
want to see widely distributed software hitting the iana.org server to
download it daily or even more often, given that changes will be much
more infrequent (based on history).

This list has discussed several methods of avoiding such an outcome,
from encouraging the use of the HTTP HEAD method or the If-Modified-Since
header, to a separate file containing the most recent registry file date,
to full provision of diffs.  IANA clearly has a stake in the outcome of
that conversation.

(My opinion FWIW is that a separate short file containing just the
file-date record, without full diffs, is the best and simplest proposal.)

-- 
John Cowan   [email-masked]    http://ccil.org/~cowan
[R]eversing the apostolic precept to be all things to all men, I usually [before
Darwin] defended the tenability of the received doctrines, when I had to do
with the [evolution]ists; and stood up for the possibility of [evolution] among
the orthodox -- thereby, no doubt, increasing an already current, but quite
undeserved, reputation for needless combativeness.  --T. H. Huxley

---

From: David Conrad <[email-masked]>
Date: Wed, 20 Sep 2006 10:52:06 -0700

John,

> file-date record, without full diffs, is the best and simplest  

Internally, we use subversion for version control.  It would be  
relatively straight forward for us to provide an opt-in mailing list  
that broadcasts the contextual diffs of any registry update.  We can,  
of course, also look at other alternatives people might suggest.

Rgds,
-drc

---

From: John Cowan <[email-masked]>
Date: Wed, 20 Sep 2006 14:18:23 -0400

> of course, also look at other alternatives people might suggest.

That would be useful for someone like Doug (except that he's the Official
Doug of ietf-languages, and already knows what the changes will be),
whose langtag-picking program has a version of the registry compiled in.

But if I write a picker or validator that does *not* have a compiled-in
registry, I would want something more scalable than downloading the whole
registry every time the program starts up.  If the current File-Date: is
provided as a separate IANA resource, I can instead cache the registry
locally and then compare the cached File-Date: with the current one,
and then download a new instance of the registry only if it has changed.
That's equivalent to using the HTTP HEAD method, but easier to do
using higher-level facilities that just fetch URIs rather than providing
general HTTP capability.

Such a scheme is much easier than applying diffs sent out by you, not
to mention that I probably won't have access to the user's mail agent
to subscribe to the list or collect the diffs from it.

-- 
John Cowan   [email-masked]
    "Mr. Lane, if you ever wish anything that I can do, all you will have
        to do will be to send me a telegram asking and it will be done."
    "Mr. Hearst, if you ever get a telegram from me asking you to do
        anything, you can put the telegram down as a forgery."

---

From: David Conrad <[email-masked]>
Date: Wed, 20 Sep 2006 11:38:37 -0700

Hi,

On Sep 20, 2006, at 11:18 AM, John Cowan wrote:
> registry every time the program starts up.

Makes sense.

> If the current File-Date: is

The implication of this is that the entire registry would need to be  
fetched, right?  Given I'm told the registry under discussion is a  
bit on the large-ish size, this might be somewhat problematic.  Can  
you give me an idea of the anticipated scale of the number of  
applications that will be wanting to fetch this registry?

I'm curious: is there a way the registry could be broken down in a  
hierarchical fashion such that only the sub-trees or leaves that  
change would need to be downloaded?  Obviously I'm a newcomer to this  
list and ignorant of previous discussions so apologies if this has  
been discussed to death previously...

Rgds,
-drc

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 20 Sep 2006 22:04:54 +0200

On Wed, Sep 20, 2006 at 11:38:37AM -0700,
 David Conrad <[email-masked]> wrote 
 a message of 39 lines which said:

> Can you give me an idea of the anticipated scale of the number of

Do note that RFC 4646 clearly mentions that access to the registry is
*not* necessary for successful parsing of language tags. Indeed, the
RFC authors take great pain to ensure that tags could be checked
offline, without a copy of the registry.
 
[Side note: what JFC Morfin wrote about my parser is completely
false.]

> change would need to be downloaded?

The issue of performance is a complex one because there are many
possible solutions (Web caches like Squid being an obvious
one). Distributions by redistributors, and local storage, like it is
currently done with XML schemas, to avoid killing the W3C Web servers,
is another one.

I do not think there is *one* approach to this problem.

---

From: "Debbie Garside" <[email-masked]>
Date: Wed, 20 Sep 2006 21:08:45 +0100

David Conrad wrote:

> I'm curious: is there a way the registry could be broken down 
> in a hierarchical fashion such that only the sub-trees or 
> leaves that change would need to be downloaded?  Obviously 
> I'm a newcomer to this list and ignorant of previous 
> discussions so apologies if this has been discussed to death 
> previously...

I think this could be a very good idea and should be investigated. However,
one way to do this is to update the registry but to create "Newsletter" type
pages with any additions (bit like they do for ISO 3166-1).  The
WatchThisPage function could then be utilised.  Those code elements included
in the Newsletters could then be appended to the local registry file, thus
negating the need to download the whole registry again.  This would work in
database terms but only for NEW registry items, not changes to existing code
elements. I cannot say how this would work with bespoke applications.

I am sure Doug can give us the statistics for New Items and Change Items in
the past 12 months!  Doug?

I think John's idea of publishing the "Registry Last Updated" date is a good
one too.   

Both suggestions would encourage less downloads of the whole Registry file.

Best regards

Debbie Garside 

> -----Original Message-----
> From: David Conrad [mailto:[email-masked]] 
> Sent: 20 September 2006 19:39
> To: John Cowan
> Cc: [email-masked]
> Subject: Re: [Ltru] IANA XML registries
> 
> Hi,
> 
> > But if I write a picker or validator that does *not* have a 
> > compiled-in registry, I would want something more scalable than 
> > downloading the whole registry every time the program starts up.
> 
> Makes sense.
> 
> The implication of this is that the entire registry would 
> need to be fetched, right?  Given I'm told the registry under 
> discussion is a bit on the large-ish size, this might be 
> somewhat problematic.  Can you give me an idea of the 
> anticipated scale of the number of applications that will be 
> wanting to fetch this registry?
> 
> I'm curious: is there a way the registry could be broken down 
> in a hierarchical fashion such that only the sub-trees or 
> leaves that change would need to be downloaded?  Obviously 
> I'm a newcomer to this list and ignorant of previous 
> discussions so apologies if this has been discussed to death 
> previously...
> 
> Rgds,
> -drc
> 
> 
> 
>

---

From: John Cowan <[email-masked]>
Date: Wed, 20 Sep 2006 16:09:23 -0400

David Conrad scripsit:

> bit on the large-ish size, this might be somewhat problematic.    

The idea is to fetch the registry only when it changes, and
looking over the history of the registry and its predecessor,
it has only changed three times a year on average.

> applications that will be wanting to fetch this registry?

It's not the number of applications, it's the number of *instances*
of those applications that is at stake, and that's unforeseeable
right now.

> been discussed to death previously...

Not really.  There are six kinds of entries in the registry (language,
region, script, variant, grandfathered, redundant), but a change is
equally likely in any of them.  In any case, most people will only need
a few of the registry's entries, but it will be unpredictable which few.

-- 
Principles.  You can't say A is         John Cowan <[email-masked]>
made of B or vice versa.  All mass      http://www.ccil.org/~cowan
is interaction.  --Richard Feynman

---

From: John Cowan <[email-masked]>
Date: Wed, 20 Sep 2006 16:27:02 -0400

> offline, without a copy of the registry.

Parsing is indeed possible without the registry: with a very small
amount of embedded permanent knowledge, an RFC 4646 language tag
can be decomposed into language, script, region, variant, extension,
and private-use subtags, or else identified as a grandfathered tag
that cannot be decomposed.

However, selecting language tags or determining their validity
(as opposed to well-formedness) requires a copy of the registry.

> currently done with XML schemas, to avoid killing the W3C Web servers,

We've already heard, however, that a large portion of the load on
those same W3C servers is from validating parsers that are unnecessarily
fetching DTDs for common XML document types like XHTML.

> I do not think there is *one* approach to this problem.

+1

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
Female celebrity stalker, on a hot morning in Cairo:
"Imagine, Colonel Lawrence, ninety-two already!"
El Auruns's reply:  "Many happy returns of the day!"

---

From: Ted Hardie <[email-masked]>
Date: Wed, 20 Sep 2006 13:28:45 -0700

>offline, without a copy of the registry.

I've just been having a side discussion on this topic, so let me
be sure I'm clear about this.  There are two types of check:  for
well-formedness and for validity.  For well-formedness, you do need access to
the list of grandfathered tags, since they are part of the check
in 2.2.9 of 4646:

>  An implementation that claims to check for well-formed language tags
>   MUST:
>
>   o  Check that the tag and all of its subtags, including extension and
>      private use subtags, conform to the ABNF or that the tag is on the
>      list of grandfathered tags.
<snip>

Validity is associated with registry date for which the implementation
performs validity checks, and access to a copy of the registry for
that date is needed:

   An implementation that claims to be validating MUST:

    * Check that the tag is well-formed.

   o  Specify the particular registry date for which the implementation
      performs validation of subtags.

   o  Check that either the tag is a grandfathered tag, or that all
      language, script, region, and variant subtags consist of valid
      codes for use in language tags according to the IANA registry as
      of the particular date specified by the implementation.

  o  Specify which, if any, extension RFCs as defined in Section 3.7
      are supported, including version, revision, and date.

   o  For any such extensions supported, check that all subtags used in
      that extension are valid.

   o  For variant and extended language subtags, if the registry
      contains one or more 'Prefix' fields for that subtag, check that
      the tag matches at least one prefix.  The tag matches if all the
      subtags in the 'Prefix' also appear in the tag.  For example, the
      prefix "es-CO" matches the tag "es-Latn-CO-x-private" because both
      the 'es' language subtag and 'CO' region subtag appear in the tag.

<snip>

The documents statement on accessibility of the registry is:

>Although the specification of valid subtags for an extension (see
>   Section 3.7) MUST be available over the Internet, implementations
>   SHOULD NOT mechanically depend on it being always accessible, to
>   prevent denial-of-service attacks.

All present and correct?

			Ted

---

From: John Cowan <[email-masked]>
Date: Wed, 20 Sep 2006 16:33:04 -0400

> All present and correct?

Yes.  In RFC 4646bis, we expect all these things to be maintained;
however, the registry will become much larger (~ 1MB), so any
performance-related issues will bite all the harder.

-- 
Verbogeny is one of the pleasurettes    John Cowan <[email-masked]>
of a creatific thinkerizer.             http://www.ccil.org/~cowan
   -- Peter da Silva

---

From: Addison Phillips <[email-masked]>
Date: Wed, 20 Sep 2006 14:49:42 -0700

> (as opposed to well-formedness) requires a copy of the registry.

One small nit: selecting language tags does not necessarily require a 
copy of the registry. Most implementations I'm aware of just have a list 
of common (valid) tags for the user to choose from or perform 
rudimentary well-formedness checks on user input. Validating these 
requires a registry copy, of course.

+1

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: Frank Ellermann <[email-masked]>
Date: Thu, 21 Sep 2006 01:20:43 +0200

David Conrad wrote:

> leaves that change would need to be downloaded?

It has "records" (roughly reflecting a registration template)
of different "types" (language, script, region, etc.), and
while RFC 4646 allows you to organize it in any way you like,
it does reqire an organization "by type".

So you can sort say languages in any way that makes sense for
you, e.g. chronologically (by "added"), or alphabetically (by
"subtag"), or in the input order (by subtag length and then
alphabetically within the same length) as provided by Doug.

Breaking it up "by type" is logical, but doesn't help much,
most records are languages.  We could remove the requirement
to organize it by type in 4646bis.  Then you'd be free to
organize it alphabetically [0..9A..Za..z] (case-sensitive).

Unfortunately that (also) won't help much for "what's new"
queries, it would only allow faster access on given subtags
for users without clue about the various types.

The idea (so far) is that you organize it as required, and
that third parties offer the various interesting "views" on
the complete data.

Frank

  Obviously I'm a newcomer to this

---

From: "Doug Ewell" <[email-masked]>
Date: Wed, 20 Sep 2006 22:30:14 -0700

Debbie Garside <debbie at ictmarketing dot co dot uk> wrote:

> I am sure Doug can give us the statistics for New Items and Change 
> Items in the past 12 months!  Doug?

Since the initial rollout of the Language Subtag Registry dated 
2005-10-16, there have been 8 updates (including one just today, on 
2006-09-20), for an average of one update every 8 weeks.

--
Doug Ewell
Fullerton, California, USA
http://users.adelphia.net/~dewell/
RFC 4645  *  UTN #14

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:05:27 +0900

>file-date record, without full diffs, is the best and simplest proposal.)

My personal opinion is that the HTTP mechanisms are best,
because already widely implemented. In addition to the two
mechanisms mentioned (HEAD, If-Modified-Since), there are also
Etags (hash values that can be used to check for changes) and
I seem to remember that there is also some diff mechanism
(I guess from WebDAV, Lisa?).

Regards,    Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:05:27 +0900

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:05:27 +0900

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:54:35 +0900

Hello David,

>still use the XML registries internally).

Many thanks for this clarification, and also for helping with the
discussions concerning the size of our new registry and possible
consequences of this size.

The main reason that you got contacted was that to some extent,
we were considering to move from what could be called a 'home-brewn'
format to something XML-based. One argument for it was that there
was a rumor that IANA would be moving to XML, which you now have
confirmed.

Given that above, you describe the purpose of XML as mainly internal,
I'm wondering whether, in your/IANAs view, an XML format defined by
a WG would be:

a) desirable (e.g. less work for IANA; helpful external input,...)
b) irrelevant (the WG can do what they want, we'll do what we need,...)
c) harmful (we'll have to re-do it anyway; it would confuse people,...)

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:54:35 +0900

Hello David,

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:54:35 +0900

Hello David,

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 21 Sep 2006 18:28:23 +0900

Hello David,

>of course, also look at other alternatives people might suggest.

If you use subversion internally already, the best way to allow
differential updates would be to make subversion available publicly.
I understand that it would have to be considered carefully,
but it's really the protocol designed for the job.

Also, as an alternative to mail broadcasts, I'd suggest using
Atom (RFC 4287).

Regards,     Martin.

t) with SMTP id
	k8L9tF35023420
	for <[email-masked]>; Thu, 21 Sep 2006 18:55:15 +0900 (JST)
Date: Thu, 21 Sep 2006 18:28:23 +0900
To: David Conrad <[email-masked]>, John Cowan <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: [Ltru] IANA XML registries
Cc: [email-masked]

Hello David,

>of course, also look at other alternatives people might suggest.

If you use subversion internally already, the best way to allow
differential updates would be to make subversion available publicly.
I understand that it would have to be considered carefully,
but it's really the protocol designed for the job.

Also, as an alternative to mail broadcasts, I'd suggest using
Atom (RFC 4287).

Regards,     Martin.

t) with SMTP id
	k8L9tF35023420
	for <[email-masked]>; Thu, 21 Sep 2006 18:55:15 +0900 (JST)
Date: Thu, 21 Sep 2006 18:28:23 +0900
To: David Conrad <[email-masked]>, John Cowan <[email-masked]>
From: Martin Duerst <[email-masked]>
Subject: Re: [Ltru] IANA XML registries
Cc: [email-masked]

Hello David,

>of course, also look at other alternatives people might suggest.

If you use subversion internally already, the best way to allow
differential updates would be to make subversion available publicly.
I understand that it would have to be considered carefully,
but it's really the protocol designed for the job.

Also, as an alternative to mail broadcasts, I'd suggest using
Atom (RFC 4287).

Regards,     Martin.

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 21 Sep 2006 06:23:11 -0700

John Cowan <cowan at ccil dot org> wrote:

> However, there are only 31 distinct "Added:" dates in the whole 
> registry going back to 1995, which is really what counts from the 
> frequency-of-updates point of view.  Even if we add thousands of items 
> at once, that's *one* update.

I was answering Debbie's specific question about the past 12 months.  I 
agree that this period has been unusually busy, with the increased 
attention on language tags spurred by the development and release of RFC 
4646, and that this rate of change probably won't continue indefinitely 
(though my crystal ball works poorly at 6:20 am).

---

From: John Cowan <[email-masked]>
Date: Thu, 21 Sep 2006 09:09:22 -0400

> (I guess from WebDAV, Lisa?).

Well, except for the last we already have those things; there's
nothing to do (ergo nothing to discuss).  The question for
discussion is, what else if anything should we do?

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
And now here I was, in a country where a right to say how the country should
be governed was restricted to six persons in each thousand of its population.
For the nine hundred and ninety-four to express dissatisfaction with the
regnant system and propose to change it, would have made the whole six
shudder as one man, it would have been so disloyal, so dishonorable, such
putrid black treason.  --Mark Twain's Connecticut Yankee

---

From: John Cowan <[email-masked]>
Date: Thu, 21 Sep 2006 09:07:26 -0400

---

Date: Thu, 21 Sep 2006 09:07:26 -0400

> 2006-09-20), for an average of one update every 8 weeks.

However, there are only 31 distinct "Added:" dates in the whole registry
going back to 1995, which is really what counts from the frequency-of-
updates point of view.  Even if we add thousands of items at once,
that's *one* update.

-- 
John Cowan    [email-masked]    http://ccil.org/~cowan
Heckler: "Go on, Al, tell 'em all you know.  It won't take long."
Al Smith: "I'll tell 'em all we *both* know.  It won't take any longer."

i
Resent-From: [email-masked]
Resent-Date: Thu, 21 Sep 2006 09:17:03 -0400
Resent-To: [email-masked]
Resent-Message-Id: <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> 2006-09-20), for an average of one update every 8 weeks.

However, there are only 31 distinct "Added:" dates in the whole registry
going back to 1995, which is really what counts from the frequency-of-
updates point of view.  Even if we add thousands of items at once,
that's *one* update.

-- 
John Cowan    [email-masked]    http://ccil.org/~cowan
Heckler: "Go on, Al, tell 'em all you know.  It won't take long."
Al Smith: "I'll tell 'em all we *both* know.  It won't take any longer."

---

From: David Conrad <[email-masked]>
Date: Thu, 21 Sep 2006 16:21:40 -0700

Martin,

> c) harmful (we'll have to re-do it anyway; it would confuse  

I'd say "a".  I'm all in favor of less work for IANA... :-)

More seriously, as far as I'm aware, there is no real specification  
on how registries must be created, so if a WG decides to go to the  
trouble to define a registry using XML (and it doesn't break anything  
at IANA), we'd probably just use that definition.

However, I should note that the registry XMLization project is  
something done by IANA primarily for IANA and has neither been  
requested (to my knowledge) nor reviewed by the IESG or IAB.  We will  
be providing access to the XML registries, but if the IETF community  
decides we should publish registries in some other fashion, we'd have  
to translate the work you all (and we) did.  Not that I think that  
would happen...

Rgds,
-drc

---

From: Martin Duerst <[email-masked]>
Date: Fri, 22 Sep 2006 12:17:10 +0900

Dear LTRU WG,

We have earlier discussed whether we should move to XML or not
for the registry. One unknown in these discussions was what
was (or wasn't) going on at IANA.

Thanks to David's help, we now have a much better idea of
what IANA is up to. Given that information, I'd like us to
make a firm decision on whether to use XML or not for RFC464xbis
soon. Please (re)state your preference.

Regards,    Martin.

---

From: Addison Phillips <[email-masked]>
Date: Thu, 21 Sep 2006 20:32:12 -0700

My preference:

-1 to XML

While I think XML would have been a better choice 'ab initio', changing 
the file format strikes me as a "no benefit" change, with plenty of 
pointless risk. Other IANA registries apparently are not as 
well-specified as ours. While I was a (if not *the*) proponent for XML 
originally, changing now will require substantial, pointless change to 
our documents.

I would support providing an additional, non-normative, copy of the 
registry using an XML format.

Addison

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 22 Sep 2006 09:12:53 +0200

On Fri, Sep 22, 2006 at 12:17:10PM +0900,
 Martin Duerst <[email-masked]> wrote 
 a message of 55 lines which said:

> Given that information, I'd like us to make a firm decision on
> whether to use XML or not for RFC464xbis soon. Please (re)state your
> preference.

My humble vote goes to XML for the following reasons (in order of
*decreasing* strength):

* in the long term (registries last a long time), XML is a safe bet,

* XML makes easy to produce tools to process and *check* the registry,
  thanks to schema languages like RelaxNG or Schematron,

* there are a lot of tools, libraries, editors, etc for XML, for every
  programming language and every licence,

* there are a lot of experience, knowledge and documentation for XML,

* it is better to stay in synch with IANA.

PS: the way the survey is casted is strange. People should be
requested to state a preference to a specific technology (XML, JSON,
YAML, record-jar, CSV) not just "XML or not" because "not XML" would
leave a lot of things to discuss.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 22 Sep 2006 09:16:23 +0200

On Thu, Sep 21, 2006 at 08:32:12PM -0700,
 Addison Phillips <[email-masked]> wrote 
 a message of 83 lines which said:

> Other IANA registries apparently are not as well-specified as ours.

Since it seems a criticism, I want to emphasize what Frank Ellermann
already said: most other registries do not have to be
well-specified. They are extremely simple, with mostly just pairs
"attribute: value". They do not have structured values with integrity
constraints.

That's one of the reasons why nobody (to my limited knowledge) before
LTRU had the issue.

---

From: Addison Phillips <[email-masked]>
Date: Fri, 22 Sep 2006 00:25:52 -0700

> Since it seems a criticism, I want to emphasize what Frank Ellermann

That isn't really my point: I don't mean it as a criticism. I just mean 
to point out that IANA is free to change the formats of other 
registries, as it is NOT free to do with LTRU.

Addison

---

From: Martin Duerst <[email-masked]>
Date: Fri, 22 Sep 2006 17:10:24 +0900

>leave a lot of things to discuss.

To clarify:
- The survey is "Switch to XML or stay with record-jar"
  Details, including use of schema language, character encoding,
  and so on are left for later discussion. Other formats haven't
  been seriously proposed in this round, so I do not see the point
  of including them in this survey. Of course, if anybody wants
  to seriously propose a switch to something like JSON or YAML,
  they can do that separately (better sooner than later).
- The survey is about the one official and normative format.
  This does neither include nor exclude additional non-normative
  formats.

Hope this helps,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Fri, 22 Sep 2006 17:13:52 +0900

>discussion is, what else if anything should we do?

Setting up a file that contains only the date of the last
change may need as much of an instruction to IANA as setting
up a compressed version that can be content negotiated or
a version that can be downloaded incrementally. HEAD and
If-Modified-Since should indeed come for free with a decent
Web server.

Regards,    Martin.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 22 Sep 2006 13:46:35 +0200

Martin Duerst wrote:

> Please (re)state your preference.

NCRs => record-jar, UTF-8 => less clear record-jar, I didn't
look at the proposed charset feature for record-jar so far.

Frank

---

From: Ned Freed <[email-masked]>
Date: Fri, 22 Sep 2006 06:31:55 -0700 (PDT)

> soon. Please (re)state your preference.

+1 in favor of shifting to XML. I don't think consumption of the current format
is sufficiently widespread that changing now is going to be a problem. And as
others have pointed out, an immense array of tools are available for processing
XML, which may speed up adoption and possibly get us past a few of the
boneheaded implementation blunders that invariably seem to creep in to
implementations of newly standardized stuff.

				Ned

---

From: Lisa Dusseault <[email-masked]>
Date: Fri, 22 Sep 2006 10:34:51 -0700

Yes, there's many ways to skin this cat, getting a registry (or any  
resource) from an HTTP server only if it has been updated.

1.  HEAD or GET (or any other method) can be used with "If-Modified- 
Since" or "If-None-Match", depending on whether the client has a  
previous Last-Modified date or an ETag.  The client might as well do  
the conditional GET as it's one request/response in either changed or  
unchanged case, and the server is instructed to do the right thing in  
returning or not returning the resource.  Using an ETag is preferred  
to using a timestamp in theory.

2.  HEAD can be used without a conditional header to find out the  
value of "Last-Modified" or the ETag.  This is not as reliable as  
using a conditional, because the client might see that a resource was  
last modified since the most recent download according to its  
understanding of the times involved.  But because the client knows  
the time of the most recent download and the server knows the last- 
modified time, there's  a possibility for clock skew, timezone errors  
etc.

3.  *IF* the server supports RFC3229, "Delta Encoding in HTTP", the  
client can make a single request for a compressed diff from the  
server if something has changed:

GET /registry.xml HTTP/1.1
Host: example.org
If-None-Match: "[value-of-last-etag]"
A-IM: gdiff, gzip

This is about the most efficient standardized thing to do for keeping  
a single large resource from an HTTP server up-to-date.  I'm not sure  
how widespread support for RFC3229 is.  Still, if the server doesn't  
support RFC3229, the request will behave just like a proper request  
designed for case 1 above.

  I don't think WebDAV adds anything for this case; WebDAV might be  
useful to keep a number of resources stored in the same collection on  
a single server, up-to-date on the client (by doing PROPFIND, asking  
for ETag values, and noting which ones had changed and required new  
downloads)

Hope this wasn't more info than y'all wanted :)

Lisa

---

From: John Cowan <[email-masked]>
Date: Fri, 22 Sep 2006 13:54:42 -0400

> to using a timestamp in theory.

In this case, timestamp is probably easier, because the registry
contains a last-modified timestamp in its content.  So if you have
a copy of the registry by whatever means, you can use that to
construct a proper If-Modified-Since header.

> modified time, there's  a possibility for clock skew, timezone errors  

Changes to the registry are separated by weeks to months, so clock
differences should not be a problem.

> [3.] This is about the most efficient standardized thing to do for keeping  

The trick here is to arrange for the client to support it.

-- 
John Cowan   [email-masked]    http://ccil.org/~cowan
Original line from The Warrior's Apprentice by Lois McMaster Bujold:
"Only on Barrayar would pulling a loaded needler start a stampede toward one."
English-to-Russian-to-English mangling thereof: "Only on Barrayar you risk to
lose support instead of finding it when you threat with the charged weapon."