Subject: [Ltru] IPA, NSAI and ANSI

From: Marion Gunn <[email-masked]>
Date: Mon, 16 Oct 2006 12:55:43 +0000

To say Ireland (NSAI) is responsible for the refusal to register IPA  
as an ISO 15924 script is as unfair as it is untrue.

I am writing to confirm that Irish users were as put out by that  
negative decision as users elsewhere, and that that negative decision  
was not made by NSAI.

ISO 15924 is nominally the responsibility of ISO/TC 46, although not  
under normal TC 46 control, either, and that negative decision was  
made by the ISO 15924 advisory committee, currently made up as follows:
One representative of the Registration Authority:
Michael Everson, Evertype
One representative of the ISO 639-2/RA:
Randall K. Barry, Library of Congress
One representative of ISO TC37:
Håvard Hjulstad, Språk- og terminologitjenester
One representative of ISO TC46:
François Demay, Encyclopaedia Universalis
Two representatives of ISO/IEC JTC1/SC2:
Rick McGowan, Unicode Consortium
Ken Whistler, Sybase

That AC failed to offer a cogent argument against tagging IPA in ISO  
15924 and, during a public exchange on the LTRU list, its members all  
failed to address most of the points raised, for example, when Mark  
Davis of International Components for Unicode pointed out that [ISO  
15924] already "incorporates variations which are essentially  
orthography (Hant vs Hans), and other differences that are simply  
glyphic variation (Fraktur Latin)" their answer was silence.

The grounds for registering IPA as a script have been long since  
established: it looks like a script, behaves like a script and only  
requires standard ISO tagging to facilitate proper electronic script  
processing. An IPA tag is urgently needed by NSAI's Irish-speaking  
communities, (so I must continue pressing for it), as well as being  
needed by university researchers and practical speech therapists in  
so many countries that a positive decision to tag it for use needs to  
made without delay, to open the door for Net searches for texts  
written in IPA.

Because that particular ISO AC was set up with a built-in US majority  
(4 out of 6), it is the logical responsibility of ANSI (_not_ NSAI)  
to ensure that such overloading in a big country's favour not be any  
excuse for refusing/delaying reasonable service requests from smaller  
countries. No country, however big it is, has any right to have such  
control over any international standards — however obtained, that  
amount of control carries with it an equal amount of answerability  
for concomitant results.

Accordingly, I respectfully request that ANSI use its influence to to  
good effect, to persuade such members of the ISO 15924 AC as it can  
influence to reverse their negative decision in re registering IPA in  
ISO 15924.

FWIW, I wish to affirm that I still believe in ISO 15924's potential  
quite as much as I did many years ago, when I was NSAI's sole  
representative in ISO/TC 37, where I first brought its first NWI  
draft to ISO/TC 37's attention (and after it was blocked in TC 37,  
supported its adoption in ISO/TC 46), that I regard its inception by  
Michael Everson as something very much to his credit, and that I  
would also regard it as to Mr Everson's credit were he now to reverse  
his vote (withdraw his opposition to registing IPA as a script).

I hold it as a reasonable expectation that the rest would probably  
follow.
mg
Convener, NSAI ISO/TC 46 IG

- -
Marion Gunn * EGTeo (Estab.1991)
27 Páirc an Fhéithlinn, Baile an
Bhóthair, Co. Átha Cliath, Éire.
* [email-masked] * [email-masked] *

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 16 Oct 2006 21:52:34 -0700

Hi -

As co-chair:

It's odd that Marion's message was addressed to the ltru working group,
and not to whatever the relevant ANSI-accredited committee would
be, since the message appears to be a request for action to be
taken by ANSI.

As a technical contributor:

The fact that someone happens to come from the US does not
imply that ANSI has any influence whatsoever over their actions.
ANSI is a private organization, not a government body, and it is
just one of many standards-generating orgnizations in the US.
It is an especially significant one, but by no means the only one.

Randy

----- Original Message ----- 
From: "Marion Gunn" <[email-masked]>
To: "LTRU Working Group" <[email-masked]>
Cc: "Sue Ellen Wright" <[email-masked]>; "TC46-L Information and Documentation Technology List" <[email-masked]>
Sent: Monday, October 16, 2006 5:55 AM
Subject: [Ltru] IPA, NSAI and ANSI

- -

---

From: Erkki Kolehmainen <[email-masked]>
Date: Thu, 26 Oct 2006 18:10:57 +0300

I'm a little late in adding my comment, but I have quite some difficulty 
in understanding this. To me, IPA is but one of the surprisingly many 
phonetic alphabets of this world, albeit probably the most commonly 
used. Others include e.g. UPA (the Uralic Phonetic Alphabet) and LMA 
(the Swedish Landsmålsalfabetet), both of which I'm somewhat familiar 
with, since I've been involved in the encoding of them in the 
UCS/Unicode. The usage of the phonetic alphabets also has a tendency to 
be based on extensive cross-boundary character loans even from other 
scripts than the Latin script, under which they are now listed.

   However, it would make a lot of sense to be able to identify a text 
as having been written using _some_ phonetic alphabet (for which a 
separate script code could be used if a more suitable solution cannot be 
found). This could then be further identified by the code for the 
specific, most dominant alphabet, such as IPA, used to write the text. 
My point is that the IPA identification problem should not be solved by 
such a special treatment that it could not be applied to the other 
phonetic alphabets.

Sincerely, Erkki I. Kolehmainen

Marion Gunn wrote:

> To say Ireland (NSAI) is responsible for the refusal to register IPA  as 
> an ISO 15924 script is as unfair as it is untrue.
> 
> under normal TC 46 control, either, and that negative decision was  made 
> by the ISO 15924 advisory committee, currently made up as follows:
> needed by university researchers and practical speech therapists in  so 
> many countries that a positive decision to tag it for use needs to  made 
> without delay, to open the door for Net searches for texts  written in IPA.
> 
> (4 out of 6), it is the logical responsibility of ANSI (_not_ NSAI)  to 
> ensure that such overloading in a big country's favour not be any  
> amount of control carries with it an equal amount of answerability  for 
> concomitant results.
> 
> representative in ISO/TC 37, where I first brought its first NWI  draft 
> to ISO/TC 37's attention (and after it was blocked in TC 37,  supported 
> its adoption in ISO/TC 46), that I regard its inception by  Michael 
> Everson as something very much to his credit, and that I  would also 
> regard it as to Mr Everson's credit were he now to reverse  his vote 
> (withdraw his opposition to registing IPA as a script).
> 
> I hold it as a reasonable expectation that the rest would probably  follow.
> mg

---

From: "Jukka K. Korpela" <[email-masked]>
Date: Thu, 26 Oct 2006 20:14:51 +0300 (EEST)

On Thu, 26 Oct 2006, Erkki Kolehmainen wrote:

>  However, it would make a lot of sense to be able to identify a text as 
> having been written using _some_ phonetic alphabet

I'm afraid I fail to see reasons to that. "Phonetic" is just a general 
attribute here, referring to the principle that a writing system tries to 
represent the sounds of speech exactly, as opposite to systems that more 
or less reflect historical developments or apply the principle of using 
the same spelling for a phoneme, irrespectively of allophonic variation.

So it's not a matter of having a "phonetic script" as opposite to all 
other scripts, but a class or property of scripts. In practice, the 
attribute "phonetic" tends to exclude scripts that are actually used for 
writing human languages, as opposite to writing systems used in 
linguistic research or teaching.

> This 
> could then be further identified by the code for the specific, most dominant 
> alphabet, such as IPA, used to write the text.

A phonetic writing system need not have _anything_ to do with IPA, except 
for the very abstract principle of representing sounds by characters (for 
some meaning of "sound").

> My point is that the IPA 
> identification problem should not be solved by such a special treatment that 
> it could not be applied to the other phonetic alphabets.

I can't follow that logic, but I think I can see the implicit argument 
that registering the IPA would be a precedent to endless requests for 
registering different phonetic writing systems - or, so to say, would open 
a can of worms.

What puzzles me most is that the _reasons_ for registering scripts and 
using script metainformation have not been discussed. This isn't just a 
game of labeling things, is it? There must be something practical to be 
achieved, even if it is practical only for the progress of theoretical 
research (in linguistics, for example). But what _are_ the realistic 
applications?

For example, if I didn't misunderstand some of the previous discussion, 
labeling text as being written in IPA is expected to help in searching for 
information on the web. While this is surely _imaginable_, I cannot regard 
it as realistic. We already have experience about _language_ 
identification, which is a much easier thing and more understandable to 
the layman. Yet, the vast majority of web pages contain either no language 
metainformation or, much worse, _wrong_ metainformation, like pages 
claiming to be in Sanskrit when they are in Northern Sámi or pages 
claiming to be in English just because that's the default of an authoring 
program.

Thus, it is quite understandable that _no_ significant web search engine
uses language metainformation at all but instead applies heuristics to 
guess the language. I see little reason to expect that script information 
would be treated differently.

If, on the other hand, script information would be used by database 
applications, language processing software, or dedicated site-specific 
search systems, for example, then the question is what these applications 
would need. I would expect them to have rather specific needs, and any 
general method of specifying script information would be just one 
interesting possibility _if it is useful enough_. Embedding script 
information into a language (or locale) code might be feasible, but that 
would be overkill if you just need, for example, to keep track of which 
strings are in normal writing and which strings are in a phonetic 
notation.

-- 
Jukka "Yucca" Korpela, http://www.cs.tut.fi/~jkorpela/

---

From: Peter Constable <[email-masked]>
Date: Thu, 26 Oct 2006 10:18:42 -0700

Hence my suggestion a while back: a script-variant ID in 15924 for Latin-based phonetic transcription systems, "Lphn".

Peter

> -----Original Message-----
> From: Erkki Kolehmainen [mailto:[email-masked]]
> Sent: Thursday, October 26, 2006 8:11 AM
> To: Marion Gunn
> Cc: TC46-L Information and Documentation Technology List; Sue Ellen Wright; LTRU
> Working Group
> Subject: Re: [Ltru] IPA as a script [was: IPA, NSAI and ANSI]
>

---

From: Peter Constable <[email-masked]>
Date: Thu, 26 Oct 2006 10:28:20 -0700

You asked for application scenarios:

- The various repositories that are part of the Open Language Archive Community have a real need to tag content that is in some kind of phonetic transcription. OLAC allows for content in the distributed repository to be search according to the language/written-form attributes, and the distinction between practical orthography and phonetic transcription is part of that.

- There are several entities that create applications for linguistic or anthroplogical analysis in which phonetic transcription as well as practical orthographies need to be supported. Those applications need a tag for infrastructure use to associate certain data with appropriate data categories within their database structure as well as resources that need to be used for that data -- locale IDs, as it were, with separate IDs for (e.g.) Lahu text in some practical orthography vs. a phonetic transcription.

Peter

> From: Jukka K. Korpela [mailto:[email-masked]]
> Sent: Thursday, October 26, 2006 10:15 AM
> To: LTRU Working Group
> Cc: TC46-L Information and Documentation Technology List

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 27 Oct 2006 10:52:44 +0200

Jukka K. Korpela wrote:

> I think I can see the implicit argument that registering the IPA would be
> a precedent to endless requests for registering different phonetic writing
> systems - or, so to say, would open a can of worms.

Maybe not "endless", but enough for a decent extension registry managed by
those who are supposed to know what they do / need / want.   The arguments
that IPA is only one of many similar "non-scripts" were convincing - that
it's the most important "non-script" is beside the point.

> research (in linguistics, for example). But what _are_ the realistic

IIRC we've discussed this here when LTRU started.  From my POV it's mainly
a kludge.  Without Unicode if I get Latin-1 or similar charsets I know, or
rather my applications know, that I can read it / they can handle it.

Without Unicode if I get some Cyrillic charset I know that I can't read it.
Maybe I can display it, but one year in school some decades ago isn't
good enough for me to decode say the charset registrations for "obscure"
(my POV) kz-charsets.

For other traditional charsets it's perfectly clear that I can't read it,
and my applications can't display it.  This crude but working zoo^Wsystem
of traditional charsets is about to be destroyed by Unicode.  Folks get
documents and have no clue at all how much of it they can display or read.

Maybe it's orcish written in Shavian.  Therefore they need scripts as a
hint, e.g. "Latn" means "there's a good chance that I can read and display
some of these Unicode points" (modulo conversion to windows-1252 depending
on the application).  "Shav" means "forget it, don't waste time trying to
download this 2.5MB PDF with a modem unless you have fonts for that script".

Frank

---

From: John Cowan <[email-masked]>
Date: Fri, 27 Oct 2006 10:05:39 -0400

> research (in linguistics, for example). But what _are_ the realistic

Let us return to first principles.  Why tag documents with language
at all?  Because if they are in the wrong language (with respect to the
end user) they are utterly unreadable.  Give me a document in Finnish and
I can't do anything with it (except recycle the paper, if it's printed).
For you it's quite otherwise.

Similarly, why tag documents with script?  Because even if the language is
English (the only language I fully command), if the script is Braille,
it might as well be in Finnish for all I can do with it.  And while
I like Frank can decipher short stretches of English in Cyrillic,
I certainly wouldn't read a whole book unless my life depended on it.

The utility of script tagging is completely separate from questions of
encoding and fonts.  It applies just as much to paper documents as to
electronic ones, and its utility would remain even if all systems were
equally Unicode-capable and had very complete sets of fonts.

Fortunately, most languages are written in only one script (a) except
in very particular circumstances, such as IPA and Braille transcription.
Some languages, however, are (b) very frequently written in alien scripts
(Chinese and Yiddish, e.g.), some languages have (c) no dominant script,
and some languages have (d) changed scripts, sometimes repeatedly.
Factors a through d, taken jointly, make it worthwhile to allow script
metainformation to be incorporated into language tags.

Frank Ellerman scripsit:

> Maybe it's orcish written in Shavian.  Therefore they need scripts as
> a hint, e.g. "Latn" means "there's a good chance that I can read and
> display some of these Unicode points" (modulo conversion to windows-1252
> depending on the application).  "Shav" means "forget it, don't waste
> time trying to download this 2.5MB PDF with a modem unless you have
> fonts for that script".

Don't bother at all, in fact, unless you can read Shavian script -- the
fonts make that possible, but not easy.  (Not to mention the fact that
Orkish cannot be written in Shavian, which lacks some necessary letters.)

-- 
Overhead, without any fuss, the stars were going out.
        --Arthur C. Clarke, "The Nine Billion Names of God"
                John Cowan <[email-masked]>

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 27 Oct 2006 12:35:54 -0700

Hi -

As a technical contributor...

> From: "John Cowan" <[email-masked]>
> To: <[email-masked]>
> Sent: Friday, October 27, 2006 7:05 AM
> Subject: Re: [Ltru] Re: IPA as a script
...
> equally Unicode-capable and had very complete sets of fonts.

I think this is a very important point.  The inclusion of script (or any
other information conveyed by a tag) is not for the convenience of
the software.  (Even admitting that there are a few cases where
knowledge of the language *might* be used by an excessively refined
rendering system.)

> metainformation to be incorporated into language tags.

And the key question here is whether this WG is willing to produce a
coherent framework for handling phonetic / phonemic transcriptions in
general, or would prefer to give a partial solution that addresses the
most widely-used one, namely IPA.  If the WG goes the general route,
then the question of how much information (phonetic vs phonemic,
transcription "system", etc.) needs to be encoded in the subtag(s)
should be decided.  I think, however, that we need to keep in mind
the reason for tagging in the first place: to be able to label content in
a way so that users can select information in representations that they
think they might be able to use.

So, going back to the days when I did linguistics work, what would *I*
have cared about when searching for documents?  (1) language
(of course) and possibly region  (2) script - I'm comfortable with IPA,
and not the others, including the "Latin only" ones I've used (3) phonetic /
phonemic / broad / narrow distinctions.  (1) and (2) fit in nicely with
what we already have.  What about (3)?

Randy

---

From: Addison Phillips <[email-masked]>
Date: Fri, 27 Oct 2006 15:49:30 -0700

> knowledge of the language *might* be used by an excessively refined

You mean "... is not *just* for the convenience of the software", right?

Software can use language tags to extract meaning from content. There 
are many applications which need to know the language in order to 
provide the best processing (which sometimes includes rendering). 
Currently many of these must use heuristics to determine language. Web 
search, for example, depends on the language of the content and search 
terms.

More prevalent and consistent use of language tags would help 
generalized search applications greatly. Their non-use in applications 
of this nature is more a function of the relative lack of consistent use 
than a lack of necessity.

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: "Randy Presuhn" <[email-masked]>
Date: Fri, 27 Oct 2006 20:15:09 -0700

Hi -

> From: "Addison Phillips" <[email-masked]>
> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Friday, October 27, 2006 3:49 PM

Actualy, I did not.  The primary motivations for language tagging have
to do with helping humans get stuff they can understand.  Everything else
is just "gravy", in my personal opinion.  This very old-fashioned attitude
is consistent with BCP 18, which, even while giving many other reasons why
language tagging is desirable, as I read it gives the most emphasis to the
needs of humans to obtain material in languages they can understand.

> search, for example, depends on the language of the content and search 

I'd still categorize this as finding material in a language the user wants.
Yes, I realize there are additional implications for non-isolating / inflected
languages, but adding transcriptions to the mix doesn't change this problem one bit.

The question of whether to add support for IPA as a one-off or as part of
a general framework for transcriptions is what this thread is all about.
There are all sorts of language-related attributes one might want to include,
but if we are to finish this work in a reasonable time, we should stick to
the minimum set that can handle what we see as the important use cases.

...

No argument.

But the only reason for doing this at all is to reduce the amount of
unintelligible stuff returned in response to a user's request.  And so
the question remains: what is a reasonable level of granularity for handling
transcriptions?  For example, do we need something horribly elaborate like
-{transcription}-{phonetic/phonemic/morphological/whatever}-{system} to
further qualify these?  Or would one simply infer the granularity of the
transcription from the system, even though doing so would prevent doing
a wildcard search for, say, any phonetic transcriptions of frs, regardless
of the system used?

Randy