Subject: [Ltru] Re: Last Call: language root file size

From: "Doug Ewell" <[email-masked]>
Date: Sun, 28 Aug 2005 17:05:53 -0700

JFC (Jefsey) Morfin <jefsey at jefsey dot com> wrote:

> I started documenting some of the problems resulting from the
> expected size of the language tag registry and the capacity of the
> langtag solution to fulfill the WG-ltru Charter. Here are two inputs
> from the author of the Draft above on the WG-ltru list, Doug Ewell:
>
> - "I've already built a hypothetical RFC 3066ter registry.  The
> changes alone add up to 35,700 lines, or more than 740 pages in RFC
> format.  It might reopen the question of whether an I-D is the best
> vehicle for delivering this amount of information to IANA."

Some of you who have not had the joy of witnessing this sort of gross
misrepresentation on LTRU over the past eight months might be a bit
confused.

At no time did I ever say, or imply, or MEAN, that the eventual size of
the registry would be a problem in and of itself, nor that the solution
developed by the LTRU WG would not fulfill the charter.

What I said, as anyone can see from reading the quote above, is that a
740-page I-D might be unwieldy enough that the IETF might consider a
different procedural mechanism for delivering the information to IANA.

> - "I still have significant concerns about the assumption that ISO
> 639-6 will be, or should be, automatically integrated into a language
> tagging scheme. [snip] Meanwhile, the claim that there are "over
> 20,000 languages" to be tagged is being used as an argument against
> the current RFC 3066bis effort and the plan to support 7,600 languages
> in RFC 3066ter."

Since the charter neither refers nor alludes to ISO 639-6, there is no
conflict with the charter if WG members disagree in regard to the
*eventual* expansion of the scope of language tags to involve ISO 639-6.

The argument against the RFC 3066bis effort on the basis of the asserted
existence of "20,000 languages" is attributable to M. Morfin alone.  He
is not being truthful in saying that he does not oppose the draft; he
has spent the entire lifetime of the LTRU WG, and before, shouting his
opposition from the rooftops.

> I fully share the concerns of Doug Ewell.

M. Morfin does not share any concerns with me, except to the extent he
can twist my words to mean something I do not intend.

I hereby disassociate myself with any statements made by M. Morfin
concerning the drafts produced by the LTRU WG.  I hope that is clear
enough for IETF members.

> 2. the documented upgrade enlarge the size from 80 K (11.5 K zipped)
> to 650 K (100 K zipped). This information, updates and additions MUST
> be available to each of the on-line application of the devices of
> billions of users. The Draft does not explain how.

The WG decided this was an IANA implementation issue, and out of scope.
Clearly, if some consider it wrong to specify both a registry format and
a registration procedure within the same draft, it would be that much
worse to dictate to IANA how it should manage its resources.

> 2. One of the author has _legitimate_ concerns about the capacity of
> the proposition and the reasonability of the Charter expectation to
> support the ISO 639-6 evolution of the underlying ISO standards.

Any talk within the LTRU WG regarding ISO 639-6 is just that: talk.
There is no charter requirement to support ISO 639-6.  (There *is* a
charter requirement to begin paving the way for support of ISO 639-3,
and we have addressed that requirement within the limitation that ISO
639-3 is still not an approved standard.)

> But he is wrong is in assuming that I use this as an argument against
> the current RFC 3066bis effort. To the contrary, I use it for a an
> argument to support the proposed Draft as default solution and
> support extensions and practical information distribution through
> other adapted solutions introduced by a singleton.

I invite any interested IETF member to peruse the WG archives, and judge
for himself whether M. Morfin supports the draft or not.

Since there are no detectable RFC 3683 or 3934 constraints on M.
Morfin's right to post anything he likes, I expect the usual scathing
personal attack in response.  (Don't bother sending it to me personally;
I do have a Blocked Senders list.)

--
Doug Ewell
Fullerton, California
http://users.adelphia.net/~dewell/

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 28 Aug 2005 21:11:21 -0700

JFC (Jefsey) Morfin <jefsey at jefsey dot com> wrote:

> I am sorry to impose that kind of response - and a large number of
> mails. But this shows the problem to protect an open R&D capacity
> against a chapel, which as some good points. And a non-profit R&D
> against people sharing in a commercial competing consortium.

Since most of M. Morfin's messages make at least passing mention of this
supposed conflict between "open" and "commercial" interests -- the
forces of good vs. the forces of evil, as it were -- I suppose some
explanations about my own motives are in order.

I do not have any commercial or financial stake in the success of the
language-tag drafts.  I am a software developer, working for a company
that produces an internationalized software product, for (frankly)
rather small values of "internationalized."  Since we are working with
the .NET platform, most of us have at least some familiarity with what
the documentation calls "culture names," which are based on RFC 3066 but
make use of some non-standard extensions.  In addition to regular
development, I am responsible for maintaining what we call "code lists,"
which are basically resource files shared between systems that otherwise
know little about each other.

I got involved in the language-tag project at its early stages, partly
to avoid repeating some of the misconceptions I had had years earlier
about RFC 3066.  I had been keeping track of changes to ISO 639 and 3166
for many years anyway, so when the concept of a comprehensive language
subtag registry came up, I mocked up my own "example" registry that
matched the description in the draft.  That "example" eventually became
the proposed initial registry.

I have developed, on my own time and with no company assistance, a small
utility program that generates and validates language tags according to
the current draft.  It contains a database that is compiled
programmatically from the text registry (into a DLL).  It is flexible in
the sense that the database can be updated independently from the main
program.  It understands the concept of extended-language subtags,
although none are defined in the current draft.  The IETF may wish to
consider this "running code," or a "working implementation," for
purposes of evaluating the drafts, and if IETF members are interested in
examining it to help with serious evaluation of the drafts -- not just
to find reasons to tear it down -- I can make it freely available.  (I
have not released it yet, of course, because the RFCs on which it is
based are only I-Ds.)

Very importantly, this program is **NOT** a commercial endeavor.  It
could conceivably be made into one, or it culd be offered as freeware or
shareware (although I have no specific plans to do either), but it could
just as easily be converted to an "open source" project of the type
often mentioned by M. Morfin.

The point is that there is NO dichotomy between "free" (or "open" or
"non-profit") and "commercial" (or "corporate") when it comes to the
current drafts.  The draft contains NO restrictions against open-source
or non-profit implementation (I would think they would be strongly
encouraged).  There is nothing in the ABNF that prevents open-source
development based on the proposed BCP.  There are no IP constraints on
any of the technology used in its development.  Any existing protocols
that make use of language tags, and stand to benefit from the proposed
update, will benefit equally regardless of whether they are commercial
or not.

M. Morfin has argued that there are prevalent "running code"
implementations that use his expanded language-tag syntax, which is not
compatible with the syntax in the present draft.  However, to date I
have not seen any pointers to such an implementation, nor any evidence
of the "prevalence" of this alternative syntax.  Obviously any developer
can write code that *does not conform* to a given standard, or proposed
standard, but that says nothing about the standard itself, or about
whether the non-conformant code or the syntax it does follow is somehow
better.

The arguments that RFC 3066 is widely ignored are specious as well;
there are numerous examples of protocols that adhere to the RFC 3066
model.  The fact that non-conformant implementations (like the culture
identifiers) exist is not a justification for throwing away the model.
To cite an example from another thread, we acknowledge that some mail
software implements "Reply-To" poorly, but that does not mean the whole
notion of "Reply-To" should be thrown away as irrelevant, or "not
supported by open R&D solutions."

I apologize for the length of this particular point, but I feel it is
important.  Not everyone who supports the draft does so out of
commercial greed, and even those who happen to work for Big Commercial
Behemoths might have higher motives.  Some people just like the feel of
an "open software vs. closed software" struggle, á la Richard Stallman
or Eric Raymond -- note M. Morfin's reference to a "chapel," evoking
images of Raymond's "The Cathedral and the Bazaar" -- but the truth is
that this image simply doesn't apply to the LTRU draft.

>> What I said, as anyone can see from reading the quote above, is that
>> a 740-page I-D might be unwieldy enough that the IETF might consider
>> a different procedural mechanism for delivering the information to
>> IANA.
>
> Correct.
> I do not see the need of all the innuendo above to repeat what I
> quoted.

It was quoted as though it proved "some of the problems resulting from
the expected size of the language tag registry and the capacity of the
langtag solution to fulfill the WG-ltru Charter."  It does nothing of
the sort.  Whether IETF or IANA wants to deal with a 740-page I-D is
entirely up to them.

>> Since the charter neither refers nor alludes to ISO 639-6, there is
>> no conflict with the charter if WG members disagree in regard to the
> This repeated allusion to the Charter neither refering nor alluding
> to ISO 639-6 is to be compared with the text of the Charter
> (http://ietf.org/html.charters/ltru-charter.html) which says "It is
> also expected to provide mechanisms to support the evolution of the
> underlying ISO standards, in particular ISO 639-3".

Peter Constable has already addressed the meaning of "the underlying ISO
standards."  ISO 639-6 is not an underlying ISO standard for purposes of
this draft.  (It's not even a standard, in fact, not even a CD yet.)
For that matter, ISO 6709 is also not an underlying ISO standard for
purposes of this draft, so the draft does not permit tags of the form
"fr+484816+0020708" either.

> This kind of problem may be related to the refusal of the WG to start
> from/analyse the Charter requirments?

I understand the charter requirements just fine.  Almost all of the WG
does.

> I do not know if I share concerns with Doug. I do share the concerns
> of Doug.

The concern quoted above was that a 740-page I-D might be unwieldy.  I
acknowledge that M. Morfin might share that concern.  Happily, it is a
concern for the next draft, not this one.

> Since I said I supported his Draft ....

"Disassociate" does not mean "disagree."  It means I do not speak so as
to bolster his point.

>>> to 650 K (100 K zipped). This information, updates and additions
>>> MUST be available to each of the on-line application of the devices
>>> of billions of users. The Draft does not explain how.
>>
> Correct. This is exactly what I say.

I suppose if we want to pursue this issue of huge daily updates and
bandwidth consumption, we should start by asking how many
implementations currently perform daily updates on the existing IANA
Language Tag Registry, a resource which (of course) is necessary to
implement RFC 3066 fully.

> Doug said in another mail "I just think the two figures (7,600 and
> 20,000) could be seen as representing a fundamental disagreement."
> This is a legitimate concern at a time a LC is to engage the whole
> IETF in a direction where you think this is unclear. I would not
> qualify that of ust "talk". This is something loyalty calls you to
> say, even if it is to explain why there is no fundamental
> disagreement.

1.  The charter does not refer to ISO 639-6, which will eventually
encode 20,000 "languages and dialects."

2.  The charter does refer to ISO 639-3, which will encode 7,600
languages, but only for future planning, for "evolution."

3.  The proposed initial registry contains subtags based only on the
existing standards: ISO 639-1 and 639-2, ISO 3166-1, ISO 15924, and
United Nations M.49, as well as "variant" and "grandfathered" subtags
based on the RFC 3066 registry.

4.  Normative references to a not-yet-completed standard are bad, for
the same reason that normative references to an I-D are bad.

> NB. As a user, the langtag solution does not provide as much
> possibilities, simplicity, flexibility than solutions using ISO 639-6
> (with the possible use of ISO 639-1/2/3 when necessary. This is IETF
> not ISO.

THERE IS NO ISO 639-6.  There will be, some day, but such a standard
does NOT exist now.  Good heavens, the data is not even publicly
available yet!  Of course the language tag solution is not going to be
based on a vapor standard.

> I have documented enough (too much :-)) during these last days that I
> support (one can read my mails of end of December 2004) the Draft
> with the provision it is not exclusive and serves the Internet
> community rather than exclude its R&D and innovation capacity.

The purpose of language tags is interoperability.  This purpose is not
served if every organization invents its own extensions.  The sections
of the draft that deal with private-use tags and subtags are heavily
laden with caveats to explain this.

> Happily I support my Unicode co-Member Doug's own Draft and I thanked
> him for the work he consciensiously achieved over the months
> maintaining the base he now proposes as a Draft! (even if I doubt a
> 740 pages I-D can be of real use in the future).

I acknowledge M. Morfin's support for draft-initial, and thank him for
his restraint in this message.

---

From: "Doug Ewell" <[email-masked]>
Date: Sun, 28 Aug 2005 22:15:10 -0700

> based on the RFC 3066 registry.

Should have been:
"variant" subtags and "grandfathered" tags

There is no such thing in the draft as a grandfathered subtag.

---

From: r&d afrac <[email-masked]>
Date: Mon, 29 Aug 2005 09:13:56 +0200

>explanations about my own motives are in order.

Dear Doug,
I read very carefully your mail. I apologise if you felt personally 
concerned. None more than myself can understand that. You may realise 
that I am even accused to have defended my own consulting practice 
(you are an employee, I am an independent, sharing in non-profit I 
often also subsidise) because I started losing clients because a 
member used mails with his corporate name.

My own involvement is in this area dates of 25 years and I accepted 
much to defend the users needs. This lead to many experimentations. 
The potential is huge for the users (to get back for everyone to what 
I did in 1985 for countries). All this is directly endangered by the Draft.

>often mentioned by M. Morfin.

I refer to the piratical capacity to freely develop what they want. 
Open to all, but also open to innovation.

>current drafts.

I do not speak of "free". I actually want to talk of "open standard" 
but I am not sure there is a consensus on the meaning?
I do not really consider commercial money, but commercial dominance. 
Of Open Culture. This is in particular described in 
http://nicso.org/equilang.tm equal lingual opportunity.

>The draft contains NO restrictions against open-source
>encouraged).

The problem is that the nature of the market makes that Draft 
favorable to market dominant. And the threat on open source is 
important. This is not the place to discuss this analysis. It was 
made by Gov officials and international analysts I trust.

>There is nothing in the ABNF that prevents open-source
>development based on the proposed BCP.

The problem is not the ABNF, it is the pretension to exclusive. 
Because the ABNF does not address _all_ the situations.
Status quo by nature favors commercial dominants. Status quo is 
opposed to innovation.

>  There are no IP constraints on
>any of the technology used in its development.

Can you warranty that? Unfortunately the experience does not say that....

>Any existing protocols

I have documented enough the dot-root experiment which is at the 
basis of our effort. The daily information we publish on the network. 
We are currently blocked by the lack of data (we wait for stability 
of ISO 639-3 and ISO 639-6 availability). The work engaged is of 
magnitude and is based upon underlying effort. We have a plan and try 
to stick to it at our expense.  The cost the WG-ltru and the threat 
it represents on us is significant (hence my delegation as an 
interface). We try to be free to stay independent in advising Govs 
and international entities.

>The arguments that RFC 3066 is widely ignored

Never said that !!! I say it is loosely enforced. And that the target 
of the Draft is to use it much more.

>are specious as well;

???? I am _defending_it_ if you really like it !!! I just want it 
_not_to_want_to_be_exclusive_ and to warn people about its dangers.
You are throwing away the model: in wanting to oppose others, you 
will lead a generalised approach to develop which will also deprecate yours.

>an "open software vs. closed software" struggle, Ã¡ la Richard Stallman

I am sure this is the way you see me. But read what I say, do not imagine it!

I documented enough that I defend industry non-dominants! I say that 
I defend Govs and cultures! I am for _free_ commercial and trade 
against a de facto monopoly. We do not want a _unique_ limited 
langtag removing the capacity to refer non-dominant industries, and 
imposing a vision foreign to people vernacular culture/way of life 
(context). Because  it does not sell. And does not permit open 
source. Let take an example: TCP/IP is open to all. Can you give me 
the source of a Windows socket?

Do you know what is the turn-over of the printing industry? Imposing 
a single langtag helps the market, technology unification. A few may 
want to become book/press industries Majors. You will understand many 
do not want.

A chapel/church is usually (and this is the way it is often used in 
here) as a religious way to support an idea received as a faith, 
rather as the result of an experimentation or a serious analysis. A 
serious analysis would have started with a clean sheet reading of the Charter.

You may recall that I started this WG in proposing to be a co-writer 
of the Draft. The response was something like "no, they are Addison 
and Mark. They will publish the WG Draft tomorrow"

>the sort.  Whether IETF or IANA wants to deal with a 740-page I-D is

It was quoted period. So people may just decide that!

>standards."

I am not interested in "underlying" but in "evolution"....

>ISO 639-6 is not an underlying ISO standard for purposes of

Do you really think this detail carry any weight when you consider 
engaging the final lingual orientation of the Internet, hence of the world?

>I understand the charter requirements just fine.  Almost all of the WG

How do you know? Randy documented 57 participants at some stage. May 
be twenty participated. Really less than a dozen ....

>concern for the next draft, not this one.

See above. Do you buy a car without considering the cost of 
insurance; repairs, gas, etc. ?

>implement RFC 3066 fully.

Very good idea. Do you have figures?
My experience of networks is different from yours. Let add experience 
and experimentation.
I have no predefined solution. But I have questions no one wants to address....

>encode 20,000 "languages and dialects."

ISO 639-6 is an officially planned "evolution" of ISO standard (refer 
to Charter). In any case we support Linguasphere.

>languages, but only for future planning, for "evolution."

This is totally irrelevant remark: what does it change? We talk of 
the Internet of the world.

>based on the RFC 3066 registry.

Yes. Then?

>the same reason that normative references to an I-D are bad.

I am not interested in paper work, I am interested in stability, 
security, surety, innovation capacity, multilingualism, multimode, 
multitechnology etc. The common interest is that we can agree on 
solutions. If we disagree (like SID and SPF) the user will decide, 
not the vendor. This takes more time, delays everyone, cost to 
everyone: so it is better than users and vendors agree.

>oes NOT exist now.  Good heavens, the data is not even publicly

..... in which world do you live?

>The purpose of language tags is interoperability.

This is the problem: the need of the langtag is interintelligibility.

>This purpose is not served if every organization invents its own extensions.

The purpose is served if every community can invent what it needs. 
More complex for the vendor. Risk of local solutions being sold....

>The sections

There is no private-use tags.

>his restraint in this message.

I am afraid that I have been often hurt - for example on my Franglish 
which is a very interesting demonstration of the difficulty of the 
participants to deal with multilingualism :-) - despised, joked at, 
etc. I do not think I ever retaliated. If I did I was wrong. I prefer 
to explain. Even if it takes and not read. No one knows.

jfc

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Sun, 28 Aug 2005 15:55:46 +0200

Last Call: http://www.ietf.org/internet-drafts/draft-ietf-ltru-initial-04.txt

- "I've already built a hypothetical RFC 3066ter registry.  The 
changes alone add up to 35,700 lines, or more than 740 pages in RFC format.  It
might reopen the question of whether an I-D is the best vehicle for 
delivering this amount of information to IANA."

- "I still have significant concerns about the assumption that ISO 
639-6 will be, or should be, automatically integrated into a language tagging
scheme. [snip] Meanwhile, the claim that there are "over 20,000 
languages" to be tagged is being used as an argument against the 
current RFC 3066bis effort and the plan to support 7,600 languages in 
RFC 3066ter."

I fully share the concerns of Doug Ewell. There is only a difference 
of timing. I rose questions they took as oppositions and he now 
discovers as problems. Would the WG-ltru have first analysed its 
charter we would have identified them a long ago. I list some in annex.

The Charter says: "The RFC 3066 standard for language tags has been 
widely adopted in various protocols and text formats, including HTML, 
XML, and CLDR, [... the first document] is also expected to provide 
mechanisms to support the evolution of the underlying ISO standards, 
in particular ISO 639-3, mechanisms to support variant registration 
and formal extensions, as well as allowing generative private use 
when necessary."

1. BTW the Charter speaks of "adopted _standard_" not of "generalised 
_practice_".

But he is wrong is in assuming that I use this as an argument against 
the current RFC 3066bis effort. To the contrary, I use it for a an 
argument to support the proposed Draft as default solution and 
support extensions and practical information distribution through 
other adapted solutions introduced by a singleton. The comment given 
by Debbie Garside, the author of ISO 639-6 shows there is no concern 
for our plans and developments using ISO 639-6 [except two months 
delay on our projected calendar], in line with ISO 11179 (what the 
WG-ltru and the author if ISO 639-3, Peter Constable, chose to disregard).

No problem in specifying this extension capacity through a 
specialised Draft. If this Draft is a complement, not a competition. 
But I would prefer to work on a sample ISO 639-6 code and get inputs 
from the WGIG multilingual forum under discussion. The three 
approaches (not even alluded by the Charter) address three different layers:
- practical, script oriented needs (ISO 639-1, 2, 3, ISO 15924)
- comprehensive multimedia, multimodal, multitechnologies and 
extension to information systems (ISO 639-6)
- lingual community networks (privateuse systems)

Years of work ahead, I would prefer being made within the IETF rather 
than against it.
jfc

Annex: a quick list of questions that need to be addressed.

- the nature and the size of the involved information
- the size and the frequency of its additions which is not 
documented. However we know that the I-D bases are just the support 
for additions.
- the availability of this information,addition and updates to the users
- the redundancies of this information between three different 
visions of the language tagging supported by
   - ISO 639-3: a bare list of 7450 languages calling for variant information
   - ISO 639-6 : a list of 20.000 pre-formed language tag meant to 
deliver language information to ISO 11179 systems
   - the real user systems planned to be supported by the Draft 
"privateuse" solution.
   and their correlations (bridges, equivalence, conflict resolution, etc.)
- the additional related information such as "locale" files (quoted 
in the Charter as CLDR, the Unicode project to publish all the 
locales of all the OS directed by one of the authors of the Draft). 
This related information is actually without limit as it may relates 
with every other information system (ISO 11179). However efforts like 
ISO 11179 have not yet addressed two major points:
   - networking what represents a huge area of research and 
applications (interoperation of an ISO complete system per... user avatar)
   - multilingualism, this means that everything supported in ASCII 
English must be able to be supported in every language
- the change in nature of languages, texts, scripts by multilingual, 
multitechnology, multilateral computer supported relations
- the multimodal (script, oral, sign, etc.) nature of the internet exchanges
- the architectural evolution to support the "brain to brain" 
interintelligibility (languages) and the community interculturation 
(vernacularisation) layers.
- the privacy violation and the intelligence leaks security issues this rises.
- the purpose and architecture of the applications and services using 
language identifications
- the relations to cultures and content, UNESCO, GAC
- the identification, information and adhesion of lingual and culture 
experts and publics
- the correlation with MPEG, MINC, ITU and other SDOs
- the governance and the distribution of the language information system

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 29 Aug 2005 03:53:12 +0200

>different procedural mechanism for delivering the information to IANA.

Correct.
I do not see the need of all the innuendo above to repeat what I quoted.

>can twist my words to mean something I do not intend.

I do not know if I share concerns with Doug. I do share the concerns of Doug.

>worse to dictate to IANA how it should manage its resources.

This is an interesting idea: because I disagree that a general 
registration procedure is documented in the same document as a 
specific format (but I propose to address the point in considering it 
as the default format) ... I would support the idea to pass to others 
what one does not know to address.

I note that this has been done with much success in the IDNA case.

say, even if it is to explain why there is no fundamental disagreement.

>639-3 is still not an approved standard.)

See remark above on the Charter.

NB. As a user, the langtag solution does not provide as much 
possibilities, simplicity, flexibility than solutions using ISO 639-6 
(with the possible use of ISO 639-1/2/3 when necessary. This is IETF not ISO.

I make no mistery that I proposed or opposed some points, knowing 
that some persons, in particular the authors of the Drafts, would 
oppose them. This permitted to have a very clear/protected ABNF we 
can now use as a default, without risk of pollution from other 
specific sub-formats.

jfc