Subject: [Ltru] Re: Description field ordering

From: "Doug Ewell" <[email-masked]>
Date: Thu, 14 Dec 2006 23:57:17 -0800

Mark Davis <mark dot davis at icu dash project dot org> wrote:

> Where the source standard has a Reference Name, that name SHOULD be 
> the first description listed. In the special case of language subtags 
> taken from ISO 639-1 or ISO 639-2, the Reference Name is considered to 
> be the corresponding value from ISO 639-3. When there is no 
> corresponding Reference Name, the first Description SHOULD be the 
> clearest and most common description for the entity.

Mark, do you remember when I proposed some sort of significance to be 
applied to the order of Description fields, and the size of the 
cannonballs with which that proposal was shot down?  I wonder what it 
will mean if the same proposal coming from you is accepted.

Addison Phillips <addison at yahoo dash inc dot com> wrote:

> This text should go in RFC 4645bis (registry-update), not 4646bis.

I think if it the concept going to be adopted, it belongs in both.

> I don't see a really good reason to make our non-normative 
> descriptions  have all sorts of arcane requirements about ordering, 
> etc.

The premise is that the Description field wouldn't be entirely 
non-normative any more.  That is the cardinal question that remains 
unanswered, and as long as we are not using issue trackers or being 
forcibly guided toward consensus, it probably never will be.

> Since we permit editing and addition of descriptions, making 
> requirements to track particular names would consume a certain amount 
> of time/effort on the part of the Official Doug.

As most know, I had planned to make the ISO 639-3 Reference Name the 
first Description field for each subtag, but not publicize that fact. 
That way we would not be bound to it, since it was clear some did not 
want to be.

Why don't we make a formal WG decision on this particular article of 
religion rather than getting it in through the side door via a minor 
wording change?

--
Doug Ewell  *  Fullerton, California, USA  *  RFC 4645  *  UTN #14
http://users.adelphia.net/~dewell/
http://www1.ietf.org/html.charters/ltru-charter.html
http://www.alvestrand.no/mailman/listinfo/ietf-languages

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 15 Dec 2006 16:46:12 +0100

Doug Ewell wrote:
 
> do you remember when I proposed some sort of significance to be
> cannonballs with which that proposal was shot down?  I wonder what
> it will mean if the same proposal coming from you is accepted.

<http://article.gmane.org/gmane.ietf.ltru/6526/match=description+order>

I think I'm to some degree consistent.  But somebody claimed that
being "consistent" actually means "closed mind" and "stubborn" :-)

Frank

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 15 Dec 2006 13:36:50 -0800

Hi -

As a technical contributor, I still think making the order
of description fields significant would be a mistake.
(Think back to your introduction to relational databases
if the reason isn't totally obvious.  :-)

If we need to convery an additional semantic, the
correct solution is to add a new field.

Randy

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 15 Dec 2006 13:43:41 -0800

I don't see the problem. The registry itself is heavily dependent on order
of lines, and most protocols that people would translate it into also allow
ordering, such as XML. And even if you read the registry into an internal
relational database, it is pretty trivial to put the first Description into
a differently named field if you wanted. So could you explain a bit more why
you think this will be a problem in practice?

Mark

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 15 Dec 2006 14:07:17 -0800

Hi -

As a technical contributor -

> From: "Mark Davis" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>
> Cc: "LTRU Working Group" <[email-masked]>
> Sent: Friday, December 15, 2006 1:43 PM
> Subject: Re: [Ltru] Re: Description field ordering
>
...

This type of argument is easily extended to claim that we
should collapse the Type and Subtag / Tag and Added fields into
the Description field - make the first Description Type, the
second one Subtag / Tag, the third one Added, and the fourth one
the "privileged" description, and the fifth and subsequent ones
"ordinary" descriptions.  Obviously, this would be silly.

If the first description is to be made different from other
descriptions, then it really should be a distinct field.
To do otherwise is to regress back to the earlier, order-dependent
form of registry entries.

It's not a question of whether it can be made to work.
Lots of ugly things can be made to work.  But why make
something ugly when there is nothing to be gained from
the ugliness?

Randy

---

From: Addison Phillips <[email-masked]>
Date: Fri, 15 Dec 2006 14:16:25 -0800

Uh...

 > The registry itself is heavily dependent on order
 > of lines,

Actually, it isn't. Nowhere in RFC 4646 is there any reference to field 
order. My implementations don't rely on it either---in fact, since I use 
Hashtables here and there, I suspect that I lose the order along the way 
somewhere.

I think I understand your "ulterior motives". Let's see if I can ascribe 
them accurately: you're worried about CLDR localizers needing to choose 
which form of the language name to translate. Is it "Old Church 
Slavonic" or "Church Slavic" or what? At present, you'd have to identify 
which one you want translated manually (or risk either getting them all 
or getting different ones for each target language).

Did I get that right?

Addison

Mark Davis wrote:
> ------------------------------------------------------------------------
> 

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 15 Dec 2006 15:10:42 -0800

(1) I said, literally: "order of lines". And clearly we are
line-order-dependent:

File-Date: 2006-12-12
%%
Type: language
Subtag: aa
Description: Afar
Added: 2005-10-16
%%
Type: language
Subtag: ab
Description: Abkhazian
Added: 2005-10-16
Suppress-Script: Cyrl
%%

is interpreted as different from the reordered-line version:

%%
%%
%%
Added: 2005-10-16
Added: 2005-10-16
Description: Abkhazian
Description: Afar
File-Date: 2006-12-12
Subtag: aa
Subtag: ab
Suppress-Script: Cyrl
Type: language
Type: language

(2) If you don't care about the first Description, then you can go ahead
with your current implementation. And if you do care, as noted earlier, you
can pick out the first name as your own separate field and put the rest into
a HashSet.

(3) I am only concerned with having the Reference Name be distinguished in
some way, and using "first Description" looked like the smallest change. As
far as the choice between implementing this as either of the following, I'm
completely agnostic. The only reason to prefer #2 is that it is a smaller
change.

   1. Different tag for "Reference Name"
   2. Use first "Description Field"

Mark

---

From: John Cowan <[email-masked]>
Date: Fri, 15 Dec 2006 19:09:43 -0500

> (1) I said, literally: "order of lines". And clearly we are

Okay, okay.  *Within each record* we are line-order independent, and the
order of records doesn't matter either (though for some reason we
partly prescribe it).

>   1. Different tag for "Reference Name"

What would this mean in the case of a script or region subtag?

-- 
John Cowan                              <[email-masked]>
            http://www.ccil.org/~cowan
                .e'osai ko sarji la lojban.
                Please support Lojban!          http://www.lojban.org

---

From: Addison Phillips <[email-masked]>
Date: Fri, 15 Dec 2006 16:50:41 -0800

> (1) I said, literally: "order of lines". And clearly we are

(laughing) Okay, okay. But intra-record order doesn't matter, at present.

> can pick out the first name as your own separate field and put the rest 

There are ordered collections and what not. I'm just trying to point out 
that implementations may exist that are allergic to ordering 
restrictions. That said, an agreement to put the reference name first is 
fine, if it has no special status.

What I'm trying to get at is whether it needs a special status and why. 
If there is a reason it needs a special status, we should give it one 
(i.e. it's own field type). You didn't really answer the 
"whys-and-wherefores" question, so I can't tell whether the use case is 
compelling (to me).

Question: if we did your #1, would we do this:

  Reference-Name: Chuvash
  Description: Chuvash

Or just:

  Reference-Name: Chuvash

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Fri, 15 Dec 2006 17:13:16 -0800

> (laughing) Okay, okay. But intra-record order doesn't matter, at present.

(smiles) True. (Although I wouldn't be surprised if someone's tool breaks if
we gratuitously change the order from the current.)

I think if we did #1, then just the second option would be sufficient. That
is, let's suppose that Chuvash is the Reference Name, and there is one other
name Yabba Dabba Doo, which is less commonly used. Then here are the
options.

#0 "Status Quo" (user doesn't know for order)

Description: Yabba Dabba Doo
Description: Chuvash

#1 "Reference Name"
Reference-Name: Chuvash
Description: Yabba Dabba Doo

#1a "Reference Name Clone"
Reference-Name: Chuvash
Description: Yabba Dabba Doo
Description: Chuvash

#2 "First Description"
Description: Chuvash
Description: Yabba Dabba Doo

The reasons why I have a slight preference for #2 is that people who don't
care about order don't need to change anything, and it can extend to
non-language subtags.

Mark

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 16 Dec 2006 11:36:32 +0100

Randy Presuhn wrote:

> Lots of ugly things can be made to work.

A variant of ltru2xml.awk destroying the source order of
descriptions / prefixes / comments is in theory possible,
but it would need some rather convoluted or "ugly" code.

What you get with a separate name is in essence that the
code would be not only "ugly" but officially "broken".

> But why make something ugly when there is nothing to be
> gained from the ugliness?

Because "backwards incompatible" is the mother of all
ugliness.  You offer no concept what a "reference name"
might be for all other subtags, it's only well-defined
for language subtags derived from 639-3.

A 4646bis going to a Last Call as "we added the 639-3
languages as promised" will pass with zero complications.

For a 4646bis going to Last Call as "while we were at it
we screwed with the 4646 syntax, update all your tools
implemented for 4646" I'd cry foul.

And IFF we screwed with the syntax invalidating all 4646
tools folks might ask why we didn't invent an UTF-8 XML
format and what else for it.  It's a can of worms for a
small theoretical advantage.  For practical purposes it
is more interesting to create tools transforming the
registry into forms digestable by Excel and/or Access.

And if that means Tab-Separated-Values with ASCII FS as
"subfield" delimiter we'll try that.  But maybe the ugly
<alt /> kludge already does the trick.

Frank

---

From: Addison Phillips <[email-masked]>
Date: Sat, 16 Dec 2006 11:00:59 -0800

Okay, I prefer #2, but only as generalized instructions to the 
reviewer/Official Doug. Perhaps:

--
For fields of type 'language' or 'extlang', where multiple descriptions 
are available, either in the original source standard or via subsequent 
registration, the first 'Description' field appearing in the Registry 
matches the Reference Name assigned by ISO 693-3. This helps facilitate 
cross-referencing between ISO 639 and the registry.
--

Notice that this requires additional work on the part of the 
LSR/Official Doug. But it doesn't give the first Description any 
particularly special status.

This suggests, please note, that we don't do uninversion or other edits 
on that description (and, further, suggests that we don't do any edits 
on descriptions whatsoever).

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Sat, 16 Dec 2006 11:09:27 -0800

The direction is good. However, I think the issue of "touching up" is really
somewhat orthogonal, and should be decided on (and clearly described,
whatever conclusion we reach) in other text. And as far as wording, in the
case where there are no multiple descriptions, still the first field will
correspond, so the two clauses that make the sentence more complicated are
not necessary. So for this paragraph I suggest the following simplification:

For fields of type 'language' or 'extlang', the first 'Description' field
appearing in the Registry
corresponds to the Reference Name assigned by ISO 693-3. This helps
facilitate
cross-referencing between ISO 639 and the registry.

Look ok to you?

Mark

On 12/16/06, Addison Phillips <[email-masked]> wrote:
> >> > with your current implementation. And if you do care, as noted
> earlier,
> >> you
> >> > can pick out the first name as your own separate field and put the
> rest
> >> > into
> >> > a HashSet.
> >>

---

From: Martin Duerst <[email-masked]>
Date: Sat, 16 Dec 2006 11:20:21 +0900

At 08:10 06/12/16, Mark Davis wrote:
>(1) I said, literally: "order of lines". And clearly we are line-order-dependent:
>
>File-Date: 2006-12-12
>%%
>Type: language
>Subtag: aa
...

>is interpreted as different from the reordered-line version:

Strictly speaking, you're right, but that's usually called structure,
not order. Our registry is structured into entries. The order of the
entries doesn't matter. The order of items in an entry doesn't matter.
The structure matters.

>(2) If you don't care about the first Description, then you can go ahead with your current implementation. And if you do care, as noted earlier, you can pick out the first name as your own separate field and put the rest into a HashSet. 

That's not the issue. For me, the main question is whether
we can and want to enforce anything on future registries,
which means on the community that deals with them.

Regards,     Martin.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: Addison Phillips <[email-masked]>
Date: Sun, 17 Dec 2006 13:36:16 -0800

It can't be entirely orthogonal, Mark. If we say "the first description 
is the reference name from ISO 639-3", then really it ought to be the 
reference name and not "something similar to the reference name". That 
is, if I go search www.sil.org/iso639-3, I expect to find "Frisian, 
Western" under "F", whereas "Western Frisian" would appear under "W".

If the reference name features inversions and other things, we would 
need to preserve them (double spaces and other cruft should probably be 
fixed, per Doug's message), at least for the reference name description. 
Hence my thinking that touch-ups and reference names are at least 
somewhat intertwined.

That said, your text works perfectly, regardless of which option we 
choose (since "correspond" is not "equal to"). I inserted your version 
into the editor's copy.

Addison

Mark Davis wrote:
> The direction is good. However, I think the issue of "touching up" is 
> really somewhat orthogonal, and should be decided on (and clearly 
> described, whatever conclusion we reach) in other text. And as far as 
> wording, in the case where there are no multiple descriptions, still the 
> first field will correspond, so the two clauses that make the sentence 
> more complicated are not necessary. So for this paragraph I suggest the 
> following simplification:
> 
> For fields of type 'language' or 'extlang', the first 'Description' 
> field appearing in the Registry