Subject: Last Call: language root file system

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sat, 27 Aug 2005 13:25:16 +0200

http://www.ietf.org/internet-drafts/draft-ietf-ltru-initial-04.txt

The proposed RFC 3066 bis Draft under current review actually creates 
a "LTS" (language tag system) having similarities with the old Host.txt system.

It is made of:
- a standard track description calling for tag resolution libraries 
like http://www-306.ibm.com/software/globalization/icu/index.jsp .
- a registry, managed by [email-masked]
- a root file: its initial version is given by the Draft above (under 
LC) to be hosted by the IANA servers.

The terse Draft version of the "langroot" is of 80,430 chars (for 
information the current size of the DNS root is 63,858 today). Zipped 
it is 11.413 chars while the DNS root is 17.320. The DNS root is 
updated around 60 times a year. It is likely that the langroot is 
currently similarly updated with new langtags. These files are 
therefore today of equivalent magnitude.

However, there will be a sharp increase next year when all the ISO 
639-3 language codes are added, if ISO accepts it, as its authors 
thinks, by the end of the year. We will have an increase from 500 
languages to 7450. When ISO 639-6 is accepted further on, there will 
be 20.000 languages subtags more. Then we may have additional 
regional tags (for example ISO 3166-2, E.164, X.121 or UN-zones, 
etc.) which may multiply the size by two or three.

The need for the users to obtain an updated langroot information is 
equivalent to the DNS root (see below) with the following differences:
- there is no intermediary service like ISP nameservers: the users 
will directly call the langroot servers (hence my comparison with host.txt)
- there is to date no cache solution, nor TTL considered yet.
- the number of calls to the registry data (not necessarily to the 
langroot server) by the users is superior to the DNS. The DNS 
supports less than 50% of the Internet calls. The langtag resolution 
will be needed for every HTML, XML, email page being read.

There is no possibility to know how developers will address the 
langtag resolution need. But the foreseen evolution of the IANA 
registry and the needs of the market plead in favor of an XML or an 
ASN.1 solution. The clearing-house related architecture is not clear 
at this stage, as the proposed Draft does not discuss this point. 
This is left to the IANA (hence my interest to know about the exact 
timing). The draught-back is that if a versatile and fast solution is 
available programers will tend to use it real time. This means a 
system comparable to the DNS root, with a root file which may be 40 
to 50 times larger, called upon 100 to 500 times more often (I know 
that 97.5% of the DNS root calls are illegitimate, that the langroot 
will not have the same historic and will take time to grow, what will 
certainly drastically initially reduce that last figure; but on the 
long range this figure stands).

The problems are the analysis of this system (its real need, its 
alternative, its architecture), of its cost. Today the IANA servers 
are not used in real time dynamic operations. Even if the user cache 
their 12.000 to 600.000 k zip file when they boot, or accept an 
update every week or month, we are in the logic of an anti-virus 
update. The IANA system can certainly support it in using an AKAMAI 
like solution, but we are entering a commercial approach. Is that 
what we want?

The proposition I made is to use IETF Drafts. This means that the 
current Draft would be made a permanent solution, with a Draft 
updated every month (this would probably provide a shorter delay than 
the IANA and stay under the full control of a dedicated IETF WG). The 
Internet community is used to rely on the RFC Editor, no one accesses 
it real time, and the new ISOC/IASA organisation provides a reliable 
and trusted structure. It would mean that updates would be provided 
once a month, possibly in a diff way or rsync to ISP and from ISP to users.

This also means that the "Draft as a default not as an exclusive" 
solution I propose, will permit a simple integration of specialised 
schemes: in addition or instead of using the Monthly WG Draft data, 
the libraries will use the data from the scheme they want to use. 
This approach is no cost, off the shelves, proven, immediate, and no 
conflict as totally open and distributed.

NB: a comment could be that the IANA could provide a simple ASCII jar 
file, some other service could distribute (like in my proposition). 
This is what we can actually expect. If my "Draft as a default not as 
an exclusive" proposition was not accepted, it would mean that this 
other service would be de facto market exclusive of the exclusive 
file. It would then probably be financed in selling related services 
and informations: this would build a commercial core for the 
Multilingual Internet.

jfc

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: David Hopwood <[email-masked]>
Date: Sat, 27 Aug 2005 17:11:46 +0100

JFC (Jefsey) Morfin wrote:
> [...] The DNS root is updated around 60 times a year. It is likely that the
> langroot is currently similarly updated with new langtags.

No, that isn't likely at all.

[...]
> The langtag resolution will be needed for every HTML, XML, email page being read.

Patent nonsense. In practice the list will be hardcoded into software that
needs it, and will be updated when the software is updated. This is perfectly
sufficient. After all, font or character encoding support for new scripts and
languages (e.g. support for Unicode version updates) has to be handled in the
same way. Languages, scripts, countries, etc. are not domains.

-- 
David Hopwood <[email-masked]>

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sat, 27 Aug 2005 20:38:48 +0200

>No, that isn't likely at all.

Dear David,
your opposition is perfectly receivable. But it should be documented.

The order of magnitude is the same. I did not note the number of 
entries in the IANA file during the last months. This is something 
that I will certainly maintain if the registry stabilises. But in the 
last months there have been more than six and far less than six 
hundred. May be will you work out a list of the registrations to 
prove me wrong?

>needs it, and will be updated when the software is updated.

Then? the langtag resolution is the translation of the langtag into a 
machine understandable information. It will happen every time a 
langtag is read, the same as domain name resolution is needed 
everytime an URL is called.

>This is perfectly sufficient. After all, font or character encoding 
>support for new scripts and
>same way.

I am afraid you confuse the process and the update of the necessary 
information. And you propose in part the solution I propose :-) .

The problem however is not to propose but to survive the users 
inadequate usage. If you have been around for some times in 
networking you know what I mean.

>  Languages, scripts, countries, etc. are not domains.

The DNS root tend to be much more stable. What count is not the 
number of changes, but their frequency.
- there is no difference between ccTLDs and country codes. We 
probably can say that there is one change a year. At least.
- scripts. ISO 15924 currently used supports 100 scripts, real life 
is more than 200 stabilised from what I gather.
- languages are 500 in ISO 639-2, 7500 in ISO 639-3 and 20.000 in ISO 639-6

The purpose of the IANA registry is to permit to adapt to reality 
(missing languages) and to transition towards ISO new tables. When 
you consider that the langtags are supposedly multimode and currently 
only support scripts, this means a steady need to adapt to 
multimedia. You should probably contact the BSI people to get more 
information on language analysis (I suggest you consider the 
preparatory work of ISO 639-4, the lack of definition of a language, 
a text, a script in a network context, lead to think, ISO will 
eventually come with much more flexibility).
Considering 50 to 200 updates a years seems very conservative - 
independently from the big structural changes of new ISO codes or the 
support of new tables - and completely live aside the 
variant/extension issue - whatever the way it will be addressed in 
the further planned Drafts.

Now, if there are updates, this means there are needs to use them, 
now - not in some years time. Same considerations as other processes. 
In a global system new releases are not localised. So either you have 
to update one billion softwares or one billion users.

BTW, your position tend to say that one do not need the root server 
system. What is an interesting idea.

jfc

PS. The problem is: one way or another one billion users, with 
various systems and appliances must get a reasonably maintained 
related information which today weight 15 K and is going to grow to 
600 K at some future date, with a change from every week to every day 
(IMHO much more as people start mastering and adapting a tool 
currently not much adapted to cross lingual exchanges). From a single 
source (in exclusive case) or from hundreds of specialised sources in 
an open approach. This should not be multiplied by all the languages 
that will progressively want to support langtags, but will multiply 
the need by two or three.For example an Ukrainian will want langtags 
in Ukrainian, in Latin and Cyrillic scripts and in most of the case 
in ASCII English, probably in Russian, etc. With the variants. etc.

---

From: David Hopwood <[email-masked]>
Date: Sat, 27 Aug 2005 23:23:36 +0100

JFC (Jefsey) Morfin wrote:
>>> [...] The DNS root is updated around 60 times a year. It is likely 
>>> that the langroot is currently similarly updated with new langtags.
>>

For the long-term, sustained rate of updates to the registry to be 60 a year,
there would have to be real-world changes in the status of countries or in
the classification of languages and scripts that occurred at the rate of 60
a year (i.e. every 6 days). And even in times of significant political
upheaval, that is simply implausible.

> The order of magnitude is the same. I did not note the number of entries 
> in the IANA file during the last months. This is something that I will 
> certainly maintain if the registry stabilises.

Exactly; the registry has not stabilised. It will do, but until it does,
there is little point in arguing statistics on how frequently it is updated.

>> Patent nonsense. In practice the list will be hardcoded into software 
>> that needs it, and will be updated when the software is updated.
> 
> machine understandable information. It will happen every time a langtag 
> is read, the same as domain name resolution is needed everytime an URL 
> is called.

The langtags would already be encoded in a form that can be interpreted
directly by each application. You were trying to imply that repeatedly
downloading this information would impose significant logistical costs:

# Even if the user cache their 12.000 to 600.000 k zip file when they boot,
# or accept an update every week or month, we are in the logic of an
# anti-virus update.

In fact there is unlikely to be any additional cost apart from that of
upgrading software using existing mechanisms.

>> support for new scripts and languages (e.g. support for Unicode version
>> updates) has to be handled in the same way.
> 

If it is sufficient to upgrade software using existing mechanisms, then
there is no problem that is not already solved.

> The DNS root tend to be much more stable. What count is not the number 
> of changes, but their frequency.
> - there is no difference between ccTLDs and country codes. We probably 
> can say that there is one change a year. At least.

What happens if the change isn't immediately picked up by all software?
Not much. Only use of that particular country code is affected.

[...]
> Now, if there are updates, this means there are needs to use them, now - 
> not in some years time.

And if they do, they will upgrade their software -- which is what they
have to do anyway to actually make use of any new localisations, scripts,
etc.

> PS. The problem is: one way or another one billion users, with various 
> systems and appliances must get a reasonably maintained related 
> information which today weight 15 K and is going to grow to 600 K at 
> some future date,

The subset of the information needed by any particular application will
typically be much less than 600K. If there is a real issue of database size,
operating systems will start providing shared libraries to look up this
information, so that only an OS update is needed (and similarly for the
Unicode data files, which are already significantly more than 600K).

> with a change from every week to every day (IMHO much 
> more as people start mastering and adapting a tool currently not much 
> adapted to cross lingual exchanges). From a single source (in exclusive 
> case) or from hundreds of specialised sources in an open approach. This 
> should not be multiplied by all the languages that will progressively 
> want to support langtags, but will multiply the need by two or three.For 
> example an Ukrainian will want langtags in Ukrainian, in Latin and 
> Cyrillic scripts  [...]

You pick one of the very few languages that are written in more than
one script, and use that example to imply that the total number of
language-script combinations used in practice is 2 to 3 times the number
of languages. Please stop exaggerating.

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sun, 28 Aug 2005 05:16:37 +0200

Dear David,
I am afraid this debate leads to nowhere because you suppose 
implemented solutions instead of considering how to implement them. 
Saying "operating systems will start providing ... so that only an OS 
update ....".

Everyone would be happy if the DNS was supported by the OS (MS said 
they would support IDNA in Windows should the market shows it the 
proper thing to do ....). Louis Pouzin defined the mail concept in 
64, Tom Van Wleck developed it in 65. The first spam was in 67 (ask 
Tom). Since then there are people saying they have the anti-spam solution...

However, you have a solution: Unicode. Why not? But you need to document it.

At 00:23 28/08/2005, David Hopwood wrote:
>>>>[...] The DNS root is updated around 60 times a year. It is 
>>>>likely that the langroot is currently similarly updated with new langtags.
>>>

Please stop removing the responses I already gave. I documented that 
this rate, for a database meant to add all what is missing, is a very 
very low rate. This is not in denying reality that you build yourself 
a credibility for your proposition. You should document the number of 
yearly changes in the Unicode files: consult 
http://www.unicode.org/versions/ - there are 20 versions on-lines.

>>that I will certainly maintain if the registry stabilises.
>

??? Nobody is arguing, are you? There is a problem which is to be 
assessed, documented and addressed. However a BCP the Draft does not 
document this (this was discussed, not addressed and decided as out 
of scope). The responsibility has been left with the IANA. The IANA 
is all of us. This is for the IESG to decide, and to bear the responsibility.

I documented that stabilisation means tens of thousands of entries. 
When (if) such a stabilisation occurs, the problem of size will be 
still more important.

>>>Patent nonsense. In practice the list will be hardcoded into 
>>>software that needs it, and will be updated when the software is updated.
>>Then? the langtag resolution is the translation of the langtag into 
>>a machine understandable information. It will happen every time a 
>directly by each application.

I do not understand what this may mean? The Draft is about that. What 
is discussed here is the update of each application.

>You were trying to imply that repeatedly downloading this 
>information would impose significant logistical costs:
>

I try to imply nothing. I document that applications (many possible 
applications) have the need to access data from a big database. We 
need the simplest, most secure, least costly, most stable, most open 
to innovation, fastest and most efficient way to give that access. 
Because I am among those who will pay for it and who will be blocked 
if it does not work, I feel concerned as I see no credible solution.

I am not interested in your "no"s, but in your/IESG "how"s I could be 
happy with.

>upgrading software using existing mechanisms.

Like updating mail servers to anti-spam solutions, ISPs to IPv6, IE to IDNA ?

>>>This is perfectly sufficient. After all, font or character 
>>>encoding support for new scripts and languages (e.g. support for 
>>>Unicode version
>If it is sufficient to upgrade software using existing mechanisms, 
>then there is no problem that is not already solved.

OK. you imply the Unicode solution is your solution?

>Not much. Only use of that particular country code is affected.

Thank you for the "not much" for the affected country code. Let say 
that some time we will have to switch from "uk" or "gb" to "en", with 
all the changes it would mean. Not much a problem if only England is affected ?

The same, no big deal if the DNS root is not kept updated? OS could 
update it in computers every now and then (ICANN has sometimes four 
months delay)? So, let suppress the root servers: I do not object to 
this, but I wish to know if this is your proposition?

I proposed to use the DNS to support that information. This was 
opposed. Do you think your Unicode-like solution is better?

>>now - not in some years time.
>

The problem is not with people upgrading. The problem is with the 
servers providing this upgrade. If all the current users upgraded 
their current langroot file once a year, over the year, in trying to 
avoid any peak, no error, no DoS, etc. This would represent today 400 
K a second.

In reality this would represent at least be 4 Meg peaks. Up to IESG 
and to IANA to say they can take the load and the risk. Size 
increase, frequency increase, DoS risks, probably call for 400 Megs.

Again DNS would most probably dramatically distribute the load. CRCs 
(Common Reference Centers) I work on are to support this without 
problem, and add a lot added value. But it seems the opposition we 
have is here. The authors favor, like you, a Unicode oriented solution.

>>600 K at some future date,
>

This is like saying that the subset of DNS information a use will 
typically need is much less than the hundred of millions of FQDNs.

Do you mean that if there is a problem, there will be langserver 
system developed in a rush outside of any standard, so the 
standardizers are not to bother? And that, in any case, we can copy Unicode?

Then I would be interested in the Unicode solution? In the traffic 
data and user support of Unicode? Of the Unicode distribution system? 
Of the way people maintain their Unicode applications (I suppose 
there is a reason why they maintain 20 releases online)?

But I am afraid you confuse the usage of Unicode data and of language 
identification data. This is not a table, this is a network cross 
negotiation, calling for far more information. The currently 
discussed Draft on filtering/matching shows the versatility users 
should obtain...

>>with a change from every week to every day (IMHO much more as 
>>people start mastering and adapting a tool currently not much 
>>adapted to cross lingual exchanges). From a single source (in 
>>exclusive case) or from hundreds of specialised sources in an open 
>>approach. This should not be multiplied by all the languages that 
>>will progressively want to support langtags, but will multiply the 
>>need by two or three.For example an Ukrainian will want langtags in 
>>Ukrainian, in Latin and Cyrillic scripts  [...]
>

??? This is an odd remark! When you develop a solution, you do not 
target supporting needs a minima. You support a maxima. And this 
example is not really a big one. Just think what can be the demand 
for an international multilingual directory....

The question is to know if the Unicode solution can be adapted to the 
IANA or to replace it ,if needed? if it scales or cost too much. 
Without making the Internet dependant of commercial solutions? Will 
you warranty this?

jfc

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sun, 28 Aug 2005 15:55:46 +0200

Last Call: http://www.ietf.org/internet-drafts/draft-ietf-ltru-initial-04.txt

I started documenting some of the problems resulting from the 
expected size of the language tag registry and the capacity of the 
langtag solution to fulfill the WG-ltru Charter. Here are two inputs 
from the author of the Draft above on the WG-ltru list, Doug Ewell:

- "I've already built a hypothetical RFC 3066ter registry.  The 
changes alone add up to 35,700 lines, or more than 740 pages in RFC format.  It
might reopen the question of whether an I-D is the best vehicle for 
delivering this amount of information to IANA."

- "I still have significant concerns about the assumption that ISO 
639-6 will be, or should be, automatically integrated into a language tagging
scheme. [snip] Meanwhile, the claim that there are "over 20,000 
languages" to be tagged is being used as an argument against the 
current RFC 3066bis effort and the plan to support 7,600 languages in 
RFC 3066ter."

I fully share the concerns of Doug Ewell. There is only a difference 
of timing. I rose questions they took as oppositions and he now 
discovers as problems. Would the WG-ltru have first analysed its 
charter we would have identified them a long ago. I list some in annex.

The Charter says: "The RFC 3066 standard for language tags has been 
widely adopted in various protocols and text formats, including HTML, 
XML, and CLDR, [... the first document] is also expected to provide 
mechanisms to support the evolution of the underlying ISO standards, 
in particular ISO 639-3, mechanisms to support variant registration 
and formal extensions, as well as allowing generative private use 
when necessary."

1. BTW the Charter speaks of "adopted _standard_" not of "generalised 
_practice_".

2. the documented upgrade enlarge the size from 80 K (11.5 K zipped) 
to 650 K (100 K zipped). This information, updates and additions MUST 
be available to each of the on-line application of the devices of 
billions of users. The Draft does not explain how.

2. One of the author has _legitimate_ concerns about the capacity of 
the proposition and the reasonability of the Charter expectation to 
support the ISO 639-6 evolution of the underlying ISO standards.

But he is wrong is in assuming that I use this as an argument against 
the current RFC 3066bis effort. To the contrary, I use it for a an 
argument to support the proposed Draft as default solution and 
support extensions and practical information distribution through 
other adapted solutions introduced by a singleton. The comment given 
by Debbie Garside, the author of ISO 639-6 shows there is no concern 
for our plans and developments using ISO 639-6 [except two months 
delay on our projected calendar], in line with ISO 11179 (what the 
WG-ltru and the author if ISO 639-3, Peter Constable, chose to disregard).

No problem in specifying this extension capacity through a 
specialised Draft. If this Draft is a complement, not a competition. 
But I would prefer to work on a sample ISO 639-6 code and get inputs 
from the WGIG multilingual forum under discussion. The three 
approaches (not even alluded by the Charter) address three different layers:
- practical, script oriented needs (ISO 639-1, 2, 3, ISO 15924)
- comprehensive multimedia, multimodal, multitechnologies and 
extension to information systems (ISO 639-6)
- lingual community networks (privateuse systems)

Years of work ahead, I would prefer being made within the IETF rather 
than against it.
jfc

Annex: a quick list of questions that need to be addressed.

- the nature and the size of the involved information
- the size and the frequency of its additions which is not 
documented. However we know that the I-D bases are just the support 
for additions.
- the availability of this information,addition and updates to the users
- the redundancies of this information between three different 
visions of the language tagging supported by
   - ISO 639-3: a bare list of 7450 languages calling for variant information
   - ISO 639-6 : a list of 20.000 pre-formed language tag meant to 
deliver language information to ISO 11179 systems
   - the real user systems planned to be supported by the Draft 
"privateuse" solution.
   and their correlations (bridges, equivalence, conflict resolution, etc.)
- the additional related information such as "locale" files (quoted 
in the Charter as CLDR, the Unicode project to publish all the 
locales of all the OS directed by one of the authors of the Draft). 
This related information is actually without limit as it may relates 
with every other information system (ISO 11179). However efforts like 
ISO 11179 have not yet addressed two major points:
   - networking what represents a huge area of research and 
applications (interoperation of an ISO complete system per... user avatar)
   - multilingualism, this means that everything supported in ASCII 
English must be able to be supported in every language
- the change in nature of languages, texts, scripts by multilingual, 
multitechnology, multilateral computer supported relations
- the multimodal (script, oral, sign, etc.) nature of the internet exchanges
- the architectural evolution to support the "brain to brain" 
interintelligibility (languages) and the community interculturation 
(vernacularisation) layers.
- the privacy violation and the intelligence leaks security issues this rises.
- the purpose and architecture of the applications and services using 
language identifications
- the relations to cultures and content, UNESCO, GAC
- the identification, information and adhesion of lingual and culture 
experts and publics
- the correlation with MPEG, MINC, ITU and other SDOs
- the governance and the distribution of the language information system

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 28 Aug 2005 17:05:53 -0700

JFC (Jefsey) Morfin <jefsey at jefsey dot com> wrote:

> changes alone add up to 35,700 lines, or more than 740 pages in RFC
> format.  It might reopen the question of whether an I-D is the best
> vehicle for delivering this amount of information to IANA."

Some of you who have not had the joy of witnessing this sort of gross
misrepresentation on LTRU over the past eight months might be a bit
confused.

At no time did I ever say, or imply, or MEAN, that the eventual size of
the registry would be a problem in and of itself, nor that the solution
developed by the LTRU WG would not fulfill the charter.

What I said, as anyone can see from reading the quote above, is that a
740-page I-D might be unwieldy enough that the IETF might consider a
different procedural mechanism for delivering the information to IANA.

> 639-6 will be, or should be, automatically integrated into a language
> tagging scheme. [snip] Meanwhile, the claim that there are "over
> 20,000 languages" to be tagged is being used as an argument against
> the current RFC 3066bis effort and the plan to support 7,600 languages
> in RFC 3066ter."

Since the charter neither refers nor alludes to ISO 639-6, there is no
conflict with the charter if WG members disagree in regard to the
*eventual* expansion of the scope of language tags to involve ISO 639-6.

The argument against the RFC 3066bis effort on the basis of the asserted
existence of "20,000 languages" is attributable to M. Morfin alone.  He
is not being truthful in saying that he does not oppose the draft; he
has spent the entire lifetime of the LTRU WG, and before, shouting his
opposition from the rooftops.

> I fully share the concerns of Doug Ewell.

M. Morfin does not share any concerns with me, except to the extent he
can twist my words to mean something I do not intend.

I hereby disassociate myself with any statements made by M. Morfin
concerning the drafts produced by the LTRU WG.  I hope that is clear
enough for IETF members.

> billions of users. The Draft does not explain how.

The WG decided this was an IANA implementation issue, and out of scope.
Clearly, if some consider it wrong to specify both a registry format and
a registration procedure within the same draft, it would be that much
worse to dictate to IANA how it should manage its resources.

> support the ISO 639-6 evolution of the underlying ISO standards.

Any talk within the LTRU WG regarding ISO 639-6 is just that: talk.
There is no charter requirement to support ISO 639-6.  (There *is* a
charter requirement to begin paving the way for support of ISO 639-3,
and we have addressed that requirement within the limitation that ISO
639-3 is still not an approved standard.)

> other adapted solutions introduced by a singleton.

I invite any interested IETF member to peruse the WG archives, and judge
for himself whether M. Morfin supports the draft or not.

Since there are no detectable RFC 3683 or 3934 constraints on M.
Morfin's right to post anything he likes, I expect the usual scathing
personal attack in response.  (Don't bother sending it to me personally;
I do have a Blocked Senders list.)

--
Doug Ewell
Fullerton, California
http://users.adelphia.net/~dewell/

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 29 Aug 2005 03:53:12 +0200

I am sorry to impose that kind of response - and a large number of 
mails. But this shows the problem to protect an open R&D capacity 
against a chapel, which as some good points. And a non-profit R&D 
against people sharing in a commercial competing consortium.

>different procedural mechanism for delivering the information to IANA.

Correct.
I do not see the need of all the innuendo above to repeat what I quoted.

>*eventual* expansion of the scope of language tags to involve ISO 639-6.

This repeated allusion to the Charter neither refering nor alluding 
to ISO 639-6 is to be compared with the text of the Charter 
(http://ietf.org/html.charters/ltru-charter.html) which says "It is 
also expected to provide mechanisms to support the evolution of the 
underlying ISO standards, in particular ISO 639-3".

This kind of problem may be related to the refusal of the WG to start 
from/analyse the Charter requirments?

>can twist my words to mean something I do not intend.

I do not know if I share concerns with Doug. I do share the concerns of Doug.

>concerning the drafts produced by the LTRU WG.  I hope that is clear

Since I said I supported his Draft ....

>The WG decided this was an IANA implementation issue, and out of scope.

Correct. This is exactly what I say.

>worse to dictate to IANA how it should manage its resources.

This is an interesting idea: because I disagree that a general 
registration procedure is documented in the same document as a 
specific format (but I propose to address the point in considering it 
as the default format) ... I would support the idea to pass to others 
what one does not know to address.

I note that this has been done with much success in the IDNA case.

>Any talk within the LTRU WG regarding ISO 639-6 is just that: talk.

Doug said in another mail "I just think the two figures (7,600 and 
20,000) could be seen as representing a fundamental disagreement." 
This is a legitimate concern at a time a LC is to engage the whole 
IETF in a direction where you think this is unclear. I would not 
qualify that of ust "talk". This is something loyalty calls you to 
say, even if it is to explain why there is no fundamental disagreement.

>639-3 is still not an approved standard.)

See remark above on the Charter.

NB. As a user, the langtag solution does not provide as much 
possibilities, simplicity, flexibility than solutions using ISO 639-6 
(with the possible use of ISO 639-1/2/3 when necessary. This is IETF not ISO.

>for himself whether M. Morfin supports the draft or not.

I have documented enough (too much :-)) during these last days that I 
support (one can read my mails of end of December 2004) the Draft 
with the provision it is not exclusive and serves the Internet 
community rather than exclude its R&D and innovation capacity.

I make no mistery that I proposed or opposed some points, knowing 
that some persons, in particular the authors of the Drafts, would 
oppose them. This permitted to have a very clear/protected ABNF we 
can now use as a default, without risk of pollution from other 
specific sub-formats.

>I do have a Blocked Senders list.)

Happily I support my Unicode co-Member Doug's own Draft and I thanked 
him for the work he consciensiously achieved over the months 
maintaining the base he now proposes as a Draft! (even if I doubt a 
740 pages I-D can be of real use in the future).

jfc

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 28 Aug 2005 21:11:21 -0700

> against people sharing in a commercial competing consortium.

Since most of M. Morfin's messages make at least passing mention of this
supposed conflict between "open" and "commercial" interests -- the
forces of good vs. the forces of evil, as it were -- I suppose some
explanations about my own motives are in order.

I do not have any commercial or financial stake in the success of the
language-tag drafts.  I am a software developer, working for a company
that produces an internationalized software product, for (frankly)
rather small values of "internationalized."  Since we are working with
the .NET platform, most of us have at least some familiarity with what
the documentation calls "culture names," which are based on RFC 3066 but
make use of some non-standard extensions.  In addition to regular
development, I am responsible for maintaining what we call "code lists,"
which are basically resource files shared between systems that otherwise
know little about each other.

I got involved in the language-tag project at its early stages, partly
to avoid repeating some of the misconceptions I had had years earlier
about RFC 3066.  I had been keeping track of changes to ISO 639 and 3166
for many years anyway, so when the concept of a comprehensive language
subtag registry came up, I mocked up my own "example" registry that
matched the description in the draft.  That "example" eventually became
the proposed initial registry.

I have developed, on my own time and with no company assistance, a small
utility program that generates and validates language tags according to
the current draft.  It contains a database that is compiled
programmatically from the text registry (into a DLL).  It is flexible in
the sense that the database can be updated independently from the main
program.  It understands the concept of extended-language subtags,
although none are defined in the current draft.  The IETF may wish to
consider this "running code," or a "working implementation," for
purposes of evaluating the drafts, and if IETF members are interested in
examining it to help with serious evaluation of the drafts -- not just
to find reasons to tear it down -- I can make it freely available.  (I
have not released it yet, of course, because the RFCs on which it is
based are only I-Ds.)

Very importantly, this program is **NOT** a commercial endeavor.  It
could conceivably be made into one, or it culd be offered as freeware or
shareware (although I have no specific plans to do either), but it could
just as easily be converted to an "open source" project of the type
often mentioned by M. Morfin.

The point is that there is NO dichotomy between "free" (or "open" or
"non-profit") and "commercial" (or "corporate") when it comes to the
current drafts.  The draft contains NO restrictions against open-source
or non-profit implementation (I would think they would be strongly
encouraged).  There is nothing in the ABNF that prevents open-source
development based on the proposed BCP.  There are no IP constraints on
any of the technology used in its development.  Any existing protocols
that make use of language tags, and stand to benefit from the proposed
update, will benefit equally regardless of whether they are commercial
or not.

M. Morfin has argued that there are prevalent "running code"
implementations that use his expanded language-tag syntax, which is not
compatible with the syntax in the present draft.  However, to date I
have not seen any pointers to such an implementation, nor any evidence
of the "prevalence" of this alternative syntax.  Obviously any developer
can write code that *does not conform* to a given standard, or proposed
standard, but that says nothing about the standard itself, or about
whether the non-conformant code or the syntax it does follow is somehow
better.

The arguments that RFC 3066 is widely ignored are specious as well;
there are numerous examples of protocols that adhere to the RFC 3066
model.  The fact that non-conformant implementations (like the culture
identifiers) exist is not a justification for throwing away the model.
To cite an example from another thread, we acknowledge that some mail
software implements "Reply-To" poorly, but that does not mean the whole
notion of "Reply-To" should be thrown away as irrelevant, or "not
supported by open R&D solutions."

I apologize for the length of this particular point, but I feel it is
important.  Not everyone who supports the draft does so out of
commercial greed, and even those who happen to work for Big Commercial
Behemoths might have higher motives.  Some people just like the feel of
an "open software vs. closed software" struggle, á la Richard Stallman
or Eric Raymond -- note M. Morfin's reference to a "chapel," evoking
images of Raymond's "The Cathedral and the Bazaar" -- but the truth is
that this image simply doesn't apply to the LTRU draft.

>> What I said, as anyone can see from reading the quote above, is that
>> a 740-page I-D might be unwieldy enough that the IETF might consider
>> a different procedural mechanism for delivering the information to
>> IANA.
>
> Correct.

It was quoted as though it proved "some of the problems resulting from
the expected size of the language tag registry and the capacity of the
langtag solution to fulfill the WG-ltru Charter."  It does nothing of
the sort.  Whether IETF or IANA wants to deal with a 740-page I-D is
entirely up to them.

>> Since the charter neither refers nor alludes to ISO 639-6, there is
>> no conflict with the charter if WG members disagree in regard to the

Peter Constable has already addressed the meaning of "the underlying ISO
standards."  ISO 639-6 is not an underlying ISO standard for purposes of
this draft.  (It's not even a standard, in fact, not even a CD yet.)
For that matter, ISO 6709 is also not an underlying ISO standard for
purposes of this draft, so the draft does not permit tags of the form
"fr+484816+0020708" either.

> from/analyse the Charter requirments?

I understand the charter requirements just fine.  Almost all of the WG
does.

> I do not know if I share concerns with Doug. I do share the concerns

The concern quoted above was that a 740-page I-D might be unwieldy.  I
acknowledge that M. Morfin might share that concern.  Happily, it is a
concern for the next draft, not this one.

> Since I said I supported his Draft ....

"Disassociate" does not mean "disagree."  It means I do not speak so as
to bolster his point.

>>> to 650 K (100 K zipped). This information, updates and additions
>>> MUST be available to each of the on-line application of the devices
>>> of billions of users. The Draft does not explain how.
>>

I suppose if we want to pursue this issue of huge daily updates and
bandwidth consumption, we should start by asking how many
implementations currently perform daily updates on the existing IANA
Language Tag Registry, a resource which (of course) is necessary to
implement RFC 3066 fully.

> say, even if it is to explain why there is no fundamental

1.  The charter does not refer to ISO 639-6, which will eventually
encode 20,000 "languages and dialects."

2.  The charter does refer to ISO 639-3, which will encode 7,600
languages, but only for future planning, for "evolution."

3.  The proposed initial registry contains subtags based only on the
existing standards: ISO 639-1 and 639-2, ISO 3166-1, ISO 15924, and
United Nations M.49, as well as "variant" and "grandfathered" subtags
based on the RFC 3066 registry.

4.  Normative references to a not-yet-completed standard are bad, for
the same reason that normative references to an I-D are bad.

> (with the possible use of ISO 639-1/2/3 when necessary. This is IETF

THERE IS NO ISO 639-6.  There will be, some day, but such a standard
does NOT exist now.  Good heavens, the data is not even publicly
available yet!  Of course the language tag solution is not going to be
based on a vapor standard.

> community rather than exclude its R&D and innovation capacity.

The purpose of language tags is interoperability.  This purpose is not
served if every organization invents its own extensions.  The sections
of the draft that deal with private-use tags and subtags are heavily
laden with caveats to explain this.

> 740 pages I-D can be of real use in the future).

I acknowledge M. Morfin's support for draft-initial, and thank him for
his restraint in this message.

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 28 Aug 2005 22:15:10 -0700

> based on the RFC 3066 registry.

Should have been:
"variant" subtags and "grandfathered" tags

There is no such thing in the draft as a grandfathered subtag.

---

From: r&d afrac <[email-masked]>
Date: Mon, 29 Aug 2005 09:13:56 +0200

>explanations about my own motives are in order.

Dear Doug,
I read very carefully your mail. I apologise if you felt personally 
concerned. None more than myself can understand that. You may realise 
that I am even accused to have defended my own consulting practice 
(you are an employee, I am an independent, sharing in non-profit I 
often also subsidise) because I started losing clients because a 
member used mails with his corporate name.

My own involvement is in this area dates of 25 years and I accepted 
much to defend the users needs. This lead to many experimentations. 
The potential is huge for the users (to get back for everyone to what 
I did in 1985 for countries). All this is directly endangered by the Draft.

>often mentioned by M. Morfin.

I refer to the piratical capacity to freely develop what they want. 
Open to all, but also open to innovation.

>current drafts.

I do not speak of "free". I actually want to talk of "open standard" 
but I am not sure there is a consensus on the meaning?
I do not really consider commercial money, but commercial dominance. 
Of Open Culture. This is in particular described in 
http://nicso.org/equilang.tm equal lingual opportunity.

>The draft contains NO restrictions against open-source
>encouraged).

The problem is that the nature of the market makes that Draft 
favorable to market dominant. And the threat on open source is 
important. This is not the place to discuss this analysis. It was 
made by Gov officials and international analysts I trust.

>There is nothing in the ABNF that prevents open-source
>development based on the proposed BCP.

The problem is not the ABNF, it is the pretension to exclusive. 
Because the ABNF does not address _all_ the situations.
Status quo by nature favors commercial dominants. Status quo is 
opposed to innovation.

>  There are no IP constraints on
>any of the technology used in its development.

Can you warranty that? Unfortunately the experience does not say that....

>Any existing protocols

I have documented enough the dot-root experiment which is at the 
basis of our effort. The daily information we publish on the network. 
We are currently blocked by the lack of data (we wait for stability 
of ISO 639-3 and ISO 639-6 availability). The work engaged is of 
magnitude and is based upon underlying effort. We have a plan and try 
to stick to it at our expense.  The cost the WG-ltru and the threat 
it represents on us is significant (hence my delegation as an 
interface). We try to be free to stay independent in advising Govs 
and international entities.

>The arguments that RFC 3066 is widely ignored

Never said that !!! I say it is loosely enforced. And that the target 
of the Draft is to use it much more.

>are specious as well;

???? I am _defending_it_ if you really like it !!! I just want it 
_not_to_want_to_be_exclusive_ and to warn people about its dangers.
You are throwing away the model: in wanting to oppose others, you 
will lead a generalised approach to develop which will also deprecate yours.

>an "open software vs. closed software" struggle, Ã¡ la Richard Stallman

I am sure this is the way you see me. But read what I say, do not imagine it!

I documented enough that I defend industry non-dominants! I say that 
I defend Govs and cultures! I am for _free_ commercial and trade 
against a de facto monopoly. We do not want a _unique_ limited 
langtag removing the capacity to refer non-dominant industries, and 
imposing a vision foreign to people vernacular culture/way of life 
(context). Because  it does not sell. And does not permit open 
source. Let take an example: TCP/IP is open to all. Can you give me 
the source of a Windows socket?

Do you know what is the turn-over of the printing industry? Imposing 
a single langtag helps the market, technology unification. A few may 
want to become book/press industries Majors. You will understand many 
do not want.

A chapel/church is usually (and this is the way it is often used in 
here) as a religious way to support an idea received as a faith, 
rather as the result of an experimentation or a serious analysis. A 
serious analysis would have started with a clean sheet reading of the Charter.

You may recall that I started this WG in proposing to be a co-writer 
of the Draft. The response was something like "no, they are Addison 
and Mark. They will publish the WG Draft tomorrow"

>the sort.  Whether IETF or IANA wants to deal with a 740-page I-D is

It was quoted period. So people may just decide that!

>standards."

I am not interested in "underlying" but in "evolution"....

>ISO 639-6 is not an underlying ISO standard for purposes of

Do you really think this detail carry any weight when you consider 
engaging the final lingual orientation of the Internet, hence of the world?

>I understand the charter requirements just fine.  Almost all of the WG

How do you know? Randy documented 57 participants at some stage. May 
be twenty participated. Really less than a dozen ....

>concern for the next draft, not this one.

See above. Do you buy a car without considering the cost of 
insurance; repairs, gas, etc. ?

>implement RFC 3066 fully.

Very good idea. Do you have figures?
My experience of networks is different from yours. Let add experience 
and experimentation.
I have no predefined solution. But I have questions no one wants to address....

>encode 20,000 "languages and dialects."

ISO 639-6 is an officially planned "evolution" of ISO standard (refer 
to Charter). In any case we support Linguasphere.

>languages, but only for future planning, for "evolution."

This is totally irrelevant remark: what does it change? We talk of 
the Internet of the world.

>based on the RFC 3066 registry.

Yes. Then?

>the same reason that normative references to an I-D are bad.

I am not interested in paper work, I am interested in stability, 
security, surety, innovation capacity, multilingualism, multimode, 
multitechnology etc. The common interest is that we can agree on 
solutions. If we disagree (like SID and SPF) the user will decide, 
not the vendor. This takes more time, delays everyone, cost to 
everyone: so it is better than users and vendors agree.

>oes NOT exist now.  Good heavens, the data is not even publicly

..... in which world do you live?

>The purpose of language tags is interoperability.

This is the problem: the need of the langtag is interintelligibility.

>This purpose is not served if every organization invents its own extensions.

The purpose is served if every community can invent what it needs. 
More complex for the vendor. Risk of local solutions being sold....

>The sections

There is no private-use tags.

>his restraint in this message.

I am afraid that I have been often hurt - for example on my Franglish 
which is a very interesting demonstration of the difficulty of the 
participants to deal with multilingualism :-) - despised, joked at, 
etc. I do not think I ever retaliated. If I did I was wrong. I prefer 
to explain. Even if it takes and not read. No one knows.

jfc