Subject: [Ltru] Re: script tag for IPA

From: "Doug Ewell" <[email-masked]>
Date: Thu, 14 Sep 2006 22:45:20 -0700

Martin Hosken <martin underscore hosken at sil dot org> wrote:

> My understanding of RFC 4646 is that extensions like intlphon need to 
> be registered *per language*. There seems to be no concept of script 
> extensions. So either we have to hide it behind a chr-Lphn-t-ipa-phn 
> (ipa phonemic) or do something else? But I may have misread the RFC.

Extensions do not have to be language-specific AFAICT.  This would be 
completely dependent on how the extension RFC is written.  Unlike 
variants, this would work right out of the box, without any changes to 
the RFC 4646 wording.

--
Doug Ewell
Fullerton, California, USA
http://users.adelphia.net/~dewell/
RFC 4645  *  UTN #14

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Fri, 15 Sep 2006 01:54:01 -0400

Doug Ewell scripsit:

> Extensions do not have to be language-specific AFAICT.  

I think Martin means variants rather than extensions.

-- 
John Cowan    http://ccil.org/~cowan    [email-masked]
Mr. Henry James writes fiction as if it were a painful duty.  --Oscar Wilde

---

From: Martin Hosken <[email-masked]>
Date: Fri, 15 Sep 2006 13:14:14 +0700

Dear John,

> I think Martin means variants rather than extensions.

Sorry. Indeed I should have written variants rather than extensions.

As I see it we have a problem on our hands regarding scripts. I am not
just talking here about transliteration schemes, I'm talking about
scripts and script variants, including the Polytonic greek question, the
myriad of Arabic and Burmese and who knows what else script variants
that are out there. I think it is unreasonable to try to pressure ISO
15924 to become a standard aimed at supporting something it was designed
to support. But further than that we do need to acknowledge that many of
these things are script variants and not just new scripts (despite my
feelings about IPA :))

Mark has shown the difficulty of using the existing variant scheme for
scripts in that script variants, since they constitute writing systems,
have a greater effect on a text than does regional variation and that
because current variants are stored after the region identifier are
considered of lower importance.

I would like to offer a possible solution for people to consider/laugh
at/whatever.

I suggest that we introduce the concept of a script variant that
modifies a script subtag, that is valid for all languages and is only
scoped by the script it occurs following. The subtag shall be 4
characters long and start with an initial capital letter just like an
existing script subtag. Given the space within ISO 15924 I'm sure we can
come to some agreement over carving out a codespace for such variants.
Failing that we could make the variant start and finish with a capital
letter or use any capitalisation trick we feel like using to mark this
thing. Since it is exactly 4 chars long it looks like a script tag to
existing mechanisms (although if they are hyper strict they will get fed
up) and would occur before the region subtag and after the script tag.
They would be clearly identifiable since they are neither the right size
for a region subtag or a variant subtag.

The effect would be to add a mechanism of script variation that would
take priority over regions, be clearly identifiable and would resolve
many of the frustrations over ISO 15924.

I hasten to add that I am *not* suggesting this as a solution to the
transcription extensions issue, which I agree should be handled via an
extension. I *am* suggesting it as a solution to the scripts question
where a particular script variant may occur across many languages and is
identifiably different in some key respect (here unspecified) to the
script it is varying from.

Thank you for your consideration.

Yours,
Martin Hosken

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 15 Sep 2006 11:14:18 +0200

Martin Hosken wrote:

> a possible solution for people to consider/laugh at/whatever.

Picking whatever:  "Case insensitive" is a fundamental concept
in 1766 / 3066 / 4646, and the default for anything you do in
the IETF - the complete ABNF is per default "case insensitive"
(nothing elaborated, only US-ASCII).

Changing that is straight out.  Better try a similar idea with
an extension regitry, e.g. s-whatever for "script variants", or
s-Latn-whatever for "Latn-variants" (etc.)

The "whatever" part does not need its own registry, if anything
folks like to use goes you can say so in the required "s-" RFC.

Frank

---

From: Martin Hosken <[email-masked]>
Date: Fri, 15 Sep 2006 16:51:52 +0700

Dear Frank,
> (nothing elaborated, only US-ASCII).

Thanks for that.
> s-Latn-whatever for "Latn-variants" (etc.)
No because the script variation needs to come before region in priority
terms.

In addition we need to indicate which extension marker marks a script
extension and which a language extension.

Yours,
Martin

---

From: Martin Duerst <[email-masked]>
Date: Fri, 15 Sep 2006 15:51:04 +0900

The solution proposed by Martin Hosken seems to make quite a bit
of sense to me. I only see two problems:
- This would need a change to the RFC 4646 ABNF. This is serious.
- Everything related to lanuguage tags is case-insensitive. The
  proposal below contains some case rules; these should (and can
  easily) be dropped.

Regards,     Martin.

>Thank you for your consideration.

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: John Cowan <[email-masked]>
Date: Fri, 15 Sep 2006 08:11:52 -0400

> - This would need a change to the RFC 4646 ABNF. This is serious.

Not a big one; basically we would need to allow multiple script subtags,
just as we allow multiple variant subtags today.  Order is enough to
tell which is primary and which are secondary, so we need play no tricks
with case; and there is no ambiguity, because currently a well-formed
tag can have only one non-initial 4-letter subtag.

I'm not necessarily saying I approve of this, just saying it's not a
big deal technically.  Changing the ABNF at all is indeed a big deal in
terms of our commitments, however, so the sooner we decide to do this
or not do this the better.

-- 
John Cowan  [email-masked]   ccil.org/~cowan
Dievas dave dantis; Dievas duos duonos          --Lithuanian proverb
Deus dedit dentes; deus dabit panem             --Latin version thereof
Deity donated dentition;
  deity'll donate doughnuts                     --English version by Muke Tever
God gave gums; God'll give granary              --Version by Mat McVeagh

---

From: "Doug Ewell" <[email-masked]>
Date: Fri, 15 Sep 2006 06:31:16 -0700

>  proposal below contains some case rules; these should (and can

- There are only 50 private-use code elements in ISO 15924, despite its 
huge capacity.  We cannot "carve out" an area beyond this.  Code 
elements starting with X and Z, for example, are already allocated.

- This adds yet another mechanism that is expected to be of limited 
use -- if it's not for transliterations or transcriptions, it's probably 
just for IPA and IPA-like systems.  Section 2.2.3 says we should be 
using variants for something like this.

- The "priority" argument applies only if such a subtag is used with 
regions and if we assume nobody will use extended matching.

-1.

---

From: Addison Phillips <[email-masked]>
Date: Fri, 15 Sep 2006 08:40:41 -0700

>   proposal below contains some case rules; these should (and can

-1

Changing the ABNF is a horrible idea.

The proper "repair" to this issue is to fix ISO 15924. Multiple script 
subtags would be very difficult for users to understand and use 
consistently. And we'd have to deal with canonical ordering rules, 
prefix checking, and all sorts of other nastiness---all to figure out 
which Latin transcription was used? Bah.

Not to mention: if script variations aren't registered in 15924, where 
will they come from? What rules will be applied to their registration? 
Why does anyone think ietf-languages will be a good arbiter of said 
variations?

Addison

-- 
Addison Phillips
Globalization Architect -- Yahoo! Inc.

Internationalization is an architecture.
It is not a feature.

---

From: Addison Phillips <[email-masked]>
Date: Fri, 15 Sep 2006 08:48:18 -0700

> No because the script variation needs to come before region in priority

Extensions can semantically modify whatever one wants them to in the 
language tag. An implementation is free to ignore an extension, 
whereupon it will fall off the tag first.

> extension and which a language extension.

No. You need to define what the semantics are for the specific 
extension. This could be that the extension describes script variation.

Addison

---

From: John Cowan <[email-masked]>
Date: Fri, 15 Sep 2006 19:51:52 -0400

> elements starting with X and Z, for example, are already allocated.

The capacity is not so large as it looks.  Every 4-letter code has a
corresponding 3-digit code, so the absolute limit is 1000 scripts, and
the first digit of the numeric code encodes the type of script
according to the following table:

0 - hieroglyphic/cuneiform
1 - RTL alphabets/abjads
2 - LTR alphabets/abjads
3 - abugidas
4 - syllabaries
5 - morphosyllabaries
6 - undeciphered
9 - private use, alias, special

-- 
Yes, chili in the eye is bad, but so is your    John Cowan
ear.  However, I would suggest you wash your    [email-masked]
hands thoroughly before going to the toilet.    http://www.ccil.org/~cowan
        --gadicath

---

From: Martin Hosken <[email-masked]>
Date: Sat, 16 Sep 2006 09:26:28 +0700

Dear Addison,
> which Latin transcription was used? Bah.

The registrar of ISO 15924 has indicated that he has no intention of
ever giving IPA a script code and that it is a variant of Latn. Perhaps
you can get him to change his mind, but I doubt it. So where does that
leave me? How do I tag text in the IPA script that can be in any
language? You are asking me to live between a rock and a hard place.

As Mark has stated, we need something to indicate that a script variant
is more significant than a region. For example, please prioritise the
aspects of "UK Glaswegian English written in IPA" in terms of the
components that have the most significance on the text and you will find
that UK comes last and Glaswegian second to last. But if IPA is marked
by an extension, it will come last.

I estimate that there are probably in the order of 10 scripts needing on
average 2-3 variants each. But that depends on what we see as being a
script variant.

In discussions with the ISO 15924 registrar on this, he seemed open to
the idea of extending the private use script code space. In addition, I
agree that since a script variant (in my 4 character scheme) would
always occur after a real script, there is no need to worry about
codespace overlap.
> Why does anyone think ietf-languages will be a good arbiter of said

ISO 15924 hasn't scored too highly for us so far. Addressing what a
script variant really is will need some discussion, of course.

Remember that ISO 15924 isn't our standard to control. It's coming from
TC46.

In the meantime, please send me the form to request 7000 language
variants or extensions (since both are registered by language).

I would encourage folks to think about how the language tag can be made
productive. If every possible language tag in effect has to be
registered, it will push many tags underground and the x- extension
space will become far more popular than we might want it to be. I, for
example, have to deal with emerging writing systems and storing data in
them for archival purposes long before such writing systems are well
established (in some cases), if every such instance needs to be
registered RFC4646 will be seen as a bottle kneck to be worked around
rather than in collaboration with.

Yours,
Martin

---

From: "Doug Ewell" <[email-masked]>
Date: Fri, 15 Sep 2006 21:13:51 -0700

John Cowan <cowan at ccil dot org> wrote:

>> - There are only 50 private-use code elements in ISO 15924, despite 
>> its huge capacity.  We cannot "carve out" an area beyond this.  Code 
> corresponding 3-digit code, so the absolute limit is 1000 scripts

Seen in that light, the percentage of total ISO 15924 code space 
allocated to private use (5.0 percent) compares well to ISO 639 (2.9 
percent), ISO 3166 (6.2 percent), and ISO 10646 (12.3 percent).  This is 
much better than 50 out of 456,976 possible 4-letter code elements, or 
0.00011 percent.

We still can't unilaterally carve out an area of the ISO 15924 code 
space for our own purposes.

I'd be tempted to write an RFC for the t- extension myself, as convinced 
as I am that that's the path we should be taking, but I'd have to set up 
and moderate a discussion mailing list and I have no time for that.

---

From: "Doug Ewell" <[email-masked]>
Date: Fri, 15 Sep 2006 21:22:38 -0700

Martin Hosken <martin underscore hosken at sil dot org> wrote:

> ever giving IPA a script code and that it is a variant of Latn. 
> Perhaps you can get him to change his mind, but I doubt it. So where 
> does that leave me? How do I tag text in the IPA script that can be in 
> any language? You are asking me to live between a rock and a hard 
> place.

EXTENSIONS.  Somebody read Section 3.7, please.  This mechanism was 
added for a reason.

> As Mark has stated, we need something to indicate that a script 
> variant is more significant than a region. For example, please 
> prioritise the aspects of "UK Glaswegian English written in IPA" in 
> terms of the components that have the most significance on the text 
> and you will find that UK comes last and Glaswegian second to last. 
> But if IPA is marked by an extension, it will come last.

Then it would be really silly to include a region subtag, wouldn't it?

> variants or extensions (since both are registered by language).

You're kidding, of course.  You really have examples of 7,000 languages 
written in IPA?  Awyi and Brokskat and Caluyanun and Dusner and all the 
rest?

Is everyone aware that extension subtags are NOT required to be specific 
to a single language?

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 16 Sep 2006 06:49:54 +0200

Martin Hosken wrote:
 
> The registrar of ISO 15924 has indicated that he has no
> intention of ever giving IPA a script code and that it is a
> variant of Latn. Perhaps you can get him to change his mind,
> but I doubt it.

For the Unicode Consortium (the ISO 15924/RA) I've no idea how
appeals work.  You probably need a rejected "Latp" registration
request, and arguments why that decision violates ISO 15924 (?)

> As Mark has stated

BTW, Mark certainly knows everything about any "Unicode appeal
process", if it exists.  Almost all contributors here are in
some way Unicode members, Randy (maybe) and I are exceptions.

If they don't know how to overrule their appointed registrar -
maybe he's simply right and IPA is no script or variant as
defined in the standard ?  I've just read the page about the
ISO 15924 history - the author of the rules would know what
they mean.

For RFC 4646bis I hope that we'll add a requirement that the
subtag reviewer MUST indicate that (s)he has read the RFC, and
is willing to implement it, and is appointed for a defined 
period of time, ideally by the IETF NomCom.  Or something in
that direction, making sure that the reviewer has still read
and write access on the review list - at the moment that isn't
relevant for the language list, but for other review lists it
can be an issue if they're silent for months (or even years).

> In discussions with the ISO 15924 registrar on this, he
> seemed open to the idea of extending the private use script
> code space.

Apparently there's also the possibility to reserve codes if
they are rejected by the JAC.  At the moment that won't help
you, because reserved codes can't be used in language tags -
same idea as EU wrt country codes.  "Officially", of course
you're always free to do what you want as long as you don't
claim to follow rules after you've clearly broken them... :-)

> In addition, I agree that since a script variant (in my 4
> character scheme) would always occur after a real script,
> there is no need to worry about codespace overlap.

It would break all 4646 parsers.  It would also break a main
feature of 4646, minus grandfathered cruft you can decompose
a tag into subtags, and you always know what each subtag is
if you keep the leading hyphen:  -1234 => must be a variant, 
-abcd => must be a script, -123 => must be an UN M.49 number,
-ab => must be a 3166 CC, -abc => extlang, abc => language,
ab => language, abcd => language (not yet possible), etc.

You can also recreate the original tag from its pieces to a
certain degree.  For extlang and variant you need the prefix
as noted in the registry, for generic variants you lose, but
maybe 4646bis kills generic variants by one s/SHOULD/MUST/.

Frank

---

From: Martin Hosken <[email-masked]>
Date: Sat, 16 Sep 2006 22:32:10 +0700

Dear Doug,
>
> You're kidding, of course.

Yes I'm kidding about filling out the forms. But I am serious that we
need to mark IPA for everything.

> You really have examples of 7,000 languages written in IPA?

Not yet, no. But hang on. The process of identifying a language for
inclusion in the Ethnologue involves a process called survey. They
primary means of doing survey is to take wordlists from locations and
transcribe them into IPA, so that they can be cross compared. So the
probability is that there exists some text somewhere in IPA for nearly
all these languages.

So, in summary, yes we need IPA for all languages.
>   Awyi and Brokskat and Caluyanun and Dusner and all the rest?

Having looked at their entries in the Ethnologue, I would be surprised
if no wordlist had been taken and therefore surprised if there were not
some text in IPA in those languages.

Please excuse my playful pedantry on that one :)

Yours,
Martin

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 16 Sep 2006 10:30:31 -0700

Martin Hosken brings up a good case for the existence of text written 
in, if not 7,000 languages, at least a large percentage of them, using 
IPA.

Let me ask a general, related question.  We have talked about Wade-Giles 
and pinyin, and to a much lesser extent McCune-Reischauer and ALA-LC and 
ISO 9 and other standards and/or conventions for transcribing and/or 
transliterating languages into a different writing system (not, not 
"script") for some purpose.

In what way -- other than the actual repertoire of characters used -- do 
IPA and other phonetic and phonemic notation systems, differ from those?

---

From: Addison Phillips <[email-masked]>
Date: Sat, 16 Sep 2006 10:55:48 -0700

> language? You are asking me to live between a rock and a hard place.

Very few ISO standards are entirely under the thumb of one person's 
opinion. Just because Michael indicated his opposition on an 
unaffiliated email list doesn't mean that a carefully prepared request 
to ISO 15924 would fail.

I'm not asking you to live between a Scylla and Charybdis. I'm asking 
you to carefully consider what you're asking for. The idea you presented 
would break a number of principles and commitments in the formation of 
language tags. Furthermore, I note that ietf-languages history as a 
registry is somewhat uneven---something that was reined in under RFC 
4646 on purpose. I also note that adding more complex and arcane 
structures to language tags strikes me as questionable. The "finger of 
doubt" points at 15924 as the source of scripts. I would exhaust that 
angle first.

> is more significant than a region. 

For starters, I don't believe in "script variants". I think there is a 
disconnect between the granularity of "script" as currently embodied in 
15924 and "script" as needed in language tags. Mark and I were of the 
(possibly mistaken) impression that 15924 would be a bit more expansive 
in their interpretation of "script".

 > For example, please prioritise the

So? Language tags cannot do everything. At some point one must look at 
the fact that two bodies of text have different tags and concluded that 
they are different, possibly mutually unintelligible, entities. RFC 4646 
even says this in Section 4.2. While the relative importance of various 
subtags is a key feature, there may be times that the subtags cannot 
entirely reflect the real relationships of the variations.

And I think the correct resolution to the problem would be to get 15924 
to register additional codes or to get a source for codes that mirrors 
15924 while adding additional codes to suit the needs of (say) language 
and locale identification. "en-Lipa-GB-glasgow" would be the best 
solution to your conundrum.

> always occur after a real script, there is no need to worry about

Script "variants" are just scripts in another container. In a language 
tag, I see very little benefit to having a tag like "el-grek-mono" 
instead of an "el-mono" or an "en-Latn-Lipa" over an "en-Lipa".

> script variant really is will need some discussion, of course.

Yes, and from hard experience I don't believe that ietf-languages is a 
better solution. A few email exchanges with ISO 15924 folks does not 
indicate utter failure and other ideas might be useful before creating 
new language subtag fauna.

> Remember that ISO 15924 isn't our standard to control. It's coming from

"Controlling a standard" implies some measure of responsibility. In this 
case, RFC 4646 strives to *avoid* making up its own rules and "working 
around" the underlying ISO standards. In fact, registration of language 
tags has a long and dubious history in this regard: many existing 
registrations have later been repented when (for example) ISO 639 took 
action at a later date.

The most likely possibility is that ISO 15924, by its definition, does 
not define all of the orthographic variations that are needed in 
language tags. While you (or I) might not like the 15924 JAC's 
decisions, I must admit that they have a certain logic.

However, I also note that modifying language tags is probably not the 
best method of overcoming this deficiency. There is precedent for 
creating a project such as an ISO 15924 Part 2 "Codes for the 
representation of scripts and script variations". Such a project, if it 
shared a codespace with 15924-1, would be a more methodical and 
consistent way to register the values you seek.

> variants or extensions (since both are registered by language).

Extensions are NOT AT ALL associated with particular languages (unless 
the RFC that defines them says they are).

Variants are NOT required to be associated with specific languages, 
although there is stern resistance by some to "generic" variants. If you 
were to propose (say) 'phonipa' as a variant with no Prefix field, I 
would probably support it.

And the form is in RFC 4646.

> space will become far more popular than we might want it to be. 

Not really. Hardly anyone uses the registered stuff. And the x- private 
use space was made productive with subtags to facilitate those with 
specialized needs. I *hope* that more people can derive use from tags 
such as en-x-ipa-glasgow (oh, hey! there's your tag :-) ).

> I, for

Your use case is explicitly what private-use subtags are for: 
idiosyncratic or specialized requirements.

Yes, every subtag you want to use has to be registered if you want it to 
be non-private-use. But previously you needed to registered entire tags 
(including every variation you wanted), so 4646 greatly improves your 
situation.

But I would start with really trying harder with 15924, followed by 
looking at creating a registry for "scripts and script variations", 
preferably within TC 46. Nothing that does not confront us today will 
prevent us from adding "script variants" in the future if they are truly 
necessary. But I don't see that as strictly necessary yet.

Addison

---

From: "Mark Davis" <[email-masked]>
Date: Sat, 16 Sep 2006 12:45:56 -0600

I generally agree with Addison's comments. A couple of further items.

1. Procedurally, the possibility of getting an IPA script variant is not
zero. It was proposed, there was very little discussion, and dismissed
without a lot of discussion (I'm just an observer). So if a more reasoned
document is presented there could be different results. Unlike other cases,
it is not up to one person capriciousness.

2. I think the private use codes (for script, etc) are misunderstood. If an
authority designates a code as private use code, that means that it is
recognized as a valid code, but that authority doesn't attach a meaning to
it, and never will.

It does not mean that nobody else can attach a meaning to it -- if that were
the case it would be completely pointless to have private use codes. For
example, the Unicode Consortium defines a meaning for Qaai (
http://www.unicode.org/reports/tr24/). Any application conformant to the
Unicode Standard must interpret Qaai as applied with that meaning. It also
attaches meaning to certain private use tags in CLDR (LDML), notably for
some region codes:
QO Outlying Oceania  QU European Union  ZZ Unknown or Invalid TerritoryBut
it only takes part of the range: "The private use codes from XA..XZ will
never be used by CLDR, and are thus safe for use for other purposes by
applications using CLDR data".

It would be possible for RFC4646bis to define a range of private use codes
that it will use. This would need to be very carefully done, and we might
decide that it is not appropriate, but it is certainly a possibility.

Mark

---

From: John Cowan <[email-masked]>
Date: Sat, 16 Sep 2006 14:46:58 -0400

> IPA and other phonetic and phonemic notation systems, differ from those?

I would say in that IPA and its rivals are not orthographies; that is,
the meaning of [f] in IPA is standardized, but the choice of when
to use it in the representation of some language depends on the
acuity of the transcriber.  Per contra, it is very clear when "f"
is to be written in the various romanizations of Japanese, whether the
sound represented is IPA [f] or not.

And I do agree that IPA (and rival) sound transcription systems are
a very different matter from mere variants in transcription like
pinyin vs. Wade-Giles or Hepburn vs. kunrei shiki.

-- 
Even a refrigerator can conform to the XML      John Cowan
Infoset, as long as it has a door sticker       [email-masked]
saying "No information items inside".           http://www.ccil.org/~cowan
        --Eve Maler

---

From: Peter Constable <[email-masked]>
Date: Sat, 16 Sep 2006 16:04:15 -0700

So, if we're entertaining ABNF changes, what about this?

langtag = language ["-" script ["-" scriptvar] ...

scriptvar = 4alpha 

where scriptvar subtags are defined per a registration and *not* interpreted in terms of ISO 15924.

That's a limited change in syntax and gives 456,976 possible script-variant subtags (should be more than plenty for transcriptions and transliterations).

Peter

> -----Original Message-----
> From: John Cowan [mailto:[email-masked]]
> Sent: Friday, September 15, 2006 5:12 AM
> To: Martin Duerst
> Cc: [email-masked]
> Subject: Re: [Ltru] Re: script tag for IPA
> 
> Martin Duerst scripsit:
>

---

From: Peter Constable <[email-masked]>
Date: Sat, 16 Sep 2006 16:10:02 -0700

> From: Martin Hosken [mailto:[email-masked]]

> How do I tag text in the IPA script that can be in any
> language? 

Just to clarify: IPA can be used to transcribe any language, but any instance of IPA transcription is in *some* particular language.

Peter Constable

---

From: "Doug Ewell" <[email-masked]>
Date: Sat, 16 Sep 2006 16:28:13 -0700

John Cowan <cowan at ccil dot org> wrote:

>> In what way -- other than the actual repertoire of characters used --  
>> do IPA and other phonetic and phonemic notation systems, differ from 
>> those?
>
> the meaning of [f] in IPA is standardized, but the choice of when to 
> use it in the representation of some language depends on the acuity of 
> the transcriber.  Per contra, it is very clear when "f" is to be 
> written in the various romanizations of Japanese, whether the sound 
> represented is IPA [f] or not.

I agree that there are differences in the details of their use, but what 
I was getting at was that both are alternative "ways of writing" 
language (note that incredibly vague phrase).  Pinyin (et al.) is not a 
"normal" way of writing Chinese (et al.); it is a special application 
for learners and other non-native readers of the language.  (This is 
different from, say, Serbian in Cyrillic/Latin, where native readers 
might use either script.)

Likewise, IPA (et al.) is not a "normal" way of writing English (et 
al.); it is a special application for linguists and others who need to 
represent the exact (or nearly exact) pronunciation of the language.

> And I do agree that IPA (and rival) sound transcription systems are a 
> very different matter from mere variants in transcription like pinyin 
> vs. Wade-Giles or Hepburn vs. kunrei shiki.

OK, well, that wasn't the way I saw it, but since almost everyone on 
this list probably has more linguistic training than I do, perhaps I 
should back off on this.

Maybe the thing to consider in 4646bis is whether we should remove the 
extension mechanism altogether.  If transcription and transliteration 
systems aren't a good application for extensions, I really don't know 
what is, and nobody else has come forward with a "favorite" potential 
application for them.  The examples in 4646 are all contrived. 
Extensions have never been valid in real-world language tags (since none 
are defined), so they could be removed without breaking anything, and 
doing so would simplify parsers.  "Deprecating" them, of course, would 
be completely pointless.

---

From: "Jukka K. Korpela" <[email-masked]>
Date: Sun, 17 Sep 2006 08:41:31 +0300 (EEST)

On Sat, 16 Sep 2006, Peter Constable wrote:

> Just to clarify: IPA can be used to transcribe any language, but any 
> instance of IPA transcription is in *some* particular language.

Normally so, but the particular language may be unknown; this however is 
no different from a case where you have, say, text in Latin letters and 
you do not know its language (yet). Moreover, it is possible to use IPA to 
describe words that appear the same (in pronunciation) in different 
languages, in which case we would probably use "mul" if we had to tag the 
text. So there are situations where there the language tagging is not 
obvious, but this may happen for any script, not just IPA.

-- 
Jukka "Yucca" Korpela, http://www.cs.tut.fi/~jkorpela/

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 17 Sep 2006 13:56:43 +0200

Peter Constable wrote:
 
> That's a limited change in syntax 

Breaking two out of two LTRU "tools" I have...   I could live
with it, but it would break _all_ RFC 4646 tools worldwide :-(

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Sun, 17 Sep 2006 21:09:40 +0200

On Sat, Sep 16, 2006 at 06:49:54AM +0200,
 Frank Ellermann <[email-masked]> wrote 
 a message of 68 lines which said:

> It would also break a main feature of 4646, minus grandfathered
> cruft you can decompose a tag into subtags, and you always know what
> each subtag is if you keep the leading hyphen: -1234 => must be a
> variant, -abcd => must be a script, -123 => must be an UN M.49
> number, -ab => must be a 3166 CC, -abc => extlang, abc => language,

I do not believe it is true. In "fr-x-abcd", "-abcd" is not a script.

---

From: John Cowan <[email-masked]>
Date: Sun, 17 Sep 2006 16:06:53 -0400

> I do not believe it is true. In "fr-x-abcd", "-abcd" is not a script.

Well, it's true if you deal with private-use and extension subtags first.

-- 
LEAR: Dost thou call me fool, boy?      John Cowan
FOOL: All thy other titles              http://www.ccil.org/~cowan
             thou hast given away:      [email-masked]
      That thou wast born with.

---

From: Martin Duerst <[email-masked]>
Date: Sun, 17 Sep 2006 16:49:56 +0900

At 02:30 06/09/17, Doug Ewell wrote:
>Martin Hosken brings up a good case for the existence of text written in, if not 7,000 languages, at least a large percentage of them, using IPA.
>
>Let me ask a general, related question.  We have talked about Wade-Giles and pinyin, and to a much lesser extent McCune-Reischauer and ALA-LC and ISO 9 and other standards and/or conventions for transcribing and/or transliterating languages into a different writing system (not, not "script") for some purpose.
>
>In what way -- other than the actual repertoire of characters used -- do IPA and other phonetic and phonemic notation systems, differ from those?

Well, it's pretty obvious that Wade-Giles is for transcribing Chinese, and only
Chinese. Same for pinyin. McCune-Reischauer is for Korean. Single language, as
opposed to IPA, which is used widely across languages.

ALA-LC is a collection of transcription rules for different languages (in some cases)
and scripts (in other cases). There may be some commonalities (my guess would
be that one fairly wide commonality is use of consonants close to (some) English
pronounciation), but also some differences because these systems are usually
based on pre-existing customs (e.g. Japanese following Hepburn, Chinese following
pinyin, etc.). So even if syntactically, we could define a variant "alalc",
semantically, its meaning would change depending on the language (and script)
applied to.

ISO 9 is a standard for romanizing Cyrillic, independent of language.

All of the examples you mention are very far away from being a script,
while IPA is at least close.

Regards,    Martin.

---

From: Peter Constable <[email-masked]>
Date: Sun, 17 Sep 2006 23:19:42 -0700

Mark:

You mentioned you were an observer to the ISO 15924 process. Can you fill us in on that process?

-          Can the public observe the process, or are you just one of the five non-voting observers in an otherwise closed process?

-          Who are the reps from each of TC46, TC37, JTC1/SC2 and the RA?

Just curious…

Peter

________________________________

From: Mark Davis [mailto:[email-masked]] 
Sent: Saturday, September 16, 2006 11:46 AM
To: Martin Hosken
Cc: Doug Ewell; LTRU Working Group
Subject: Re: [Ltru] Re: script tag for IPA

1. Procedurally, the possibility of getting an IPA script variant is not zero. It was proposed, there was very little discussion, and dismissed without a lot of discussion (I'm just an observer). So if a more reasoned document is presented there could be different results. Unlike other cases, it is not up to one person capriciousness. 

2. I think the private use codes (for script, etc) are misunderstood. If an authority designates a code as private use code, that means that it is recognized as a valid code, but that authority doesn't attach a meaning to it, and never will.

It does not mean that nobody else can attach a meaning to it -- if that were the case it would be completely pointless to have private use codes. For example, the Unicode Consortium defines a meaning for Qaai ( http://www.unicode.org/reports/tr24/). Any application conformant to the Unicode Standard must interpret Qaai as applied with that meaning. It also attaches meaning to certain private use tags in CLDR (LDML), notably for some region codes: 

QO

Outlying Oceania

QU

European Union

ZZ

Unknown or Invalid Territory

But it only takes part of the range: "The private use codes from XA..XZ will never be used by CLDR, and are thus safe for use for other purposes by applications using CLDR data".

It would be possible for RFC4646bis to define a range of private use codes that it will use. This would need to be very carefully done, and we might decide that it is not appropriate, but it is certainly a possibility. 

Mark

---

From: Peter Constable <[email-masked]>
Date: Sun, 17 Sep 2006 23:29:32 -0700

> From: Jukka K. Korpela [mailto:[email-masked]]

> Normally so, but the particular language may be unknown;

Even if it is unknown, it is still in some particular language.

> this however is
> you do not know its language (yet). 

Yes. ISO 639-2/-3 have "und" for that situation.

> Moreover, it is possible to use IPA to
> text.

I suppose one could use IPA notation to cite an abstracted utterance. Abstracted usage -- i.e. independent of any language -- is often done when discussing individual phonemes, but I don't know that I've ever seen a single instance of a transcription of an abstracted utterance.

But I'd suggest that, if a document contained such a thing, it would make most sense either to leave it untagged, or to tag it "zxx" as it is not content in any language, let alone multiple languages; it is just a abstract phonetic sound.

> So there are situations where there the language tagging is not

Quite so; and again no tag or "zxx" makes more sense to me than "mul".

Peter Constable

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 18 Sep 2006 08:46:44 +0200

Peter Constable wrote:
 
> Just curious

Same here, that's how I found the page
<http://www.unicode.org/iso15924/iso15924jac.html>

Frank

---

From: "Jukka K. Korpela" <[email-masked]>
Date: Mon, 18 Sep 2006 09:58:56 +0300 (EEST)

On Sun, 17 Sep 2006, Peter Constable wrote:

> I suppose one could use IPA notation to cite an abstracted utterance. 
> Abstracted usage -- i.e. independent of any language -- is often done 
> when discussing individual phonemes, but I don't know that I've ever 
> seen a single instance of a transcription of an abstracted utterance.

Just to clarify: I did not mean an abstract utterance but something like
"The word [kala] 'fish' has probably remained the same from the Uralian 
protolanguage to Finnish and Estonian but often changed considerably
in other Uralian languages."
(I'm cheating a bit here, in order to be able to use ASCII only, since [a]
is not the most adequate vowel symbol here.)
So it would not be an abstract utterance but a word that is allegedly the 
same in three languages, one of which is an uncertain reconstruction 
without a specific code.

The same situation may, of course, also appear when words are written 
using some normal script instead of IPA.

> But I'd suggest that, if a document contained such a thing, it would 
> make most sense either to leave it untagged, or to tag it "zxx" as it is 
> not content in any language, let alone multiple languages; it is just a 
> abstract phonetic sound.

Perhaps. It depends on the context and requirements. I think the actual 
_utilization_ of language tagging in software is far behind the 
specification of codes for it, and it is therefore difficult to estimate 
the practical impact of tagging decisions in borderline cases. Most texts 
in the world have no language tagging, still less tagging at the level of 
individual words (or characters) even in cases where the tagging decision 
would be straightforward (e.g., in tagging personal names that clearly 
belong to a language other than the surrounding text).

---

From: Peter Constable <[email-masked]>
Date: Mon, 18 Sep 2006 00:09:56 -0700

> in other Uralian languages."

In that case, the transcription is of a reconstructed word form from proto Uralic -- it's a word in a specific (reconstructed) language.

Now, mind you, ISO 639 does *not* provide IDs for historically-reconstructed proto-languages.

Peter Constable

PA
Date: Mon, 18 Sep 2006 00:09:56 -0700
From: Peter Constable <[email-masked]>
To: <[email-masked]>

> in other Uralian languages."

In that case, the transcription is of a reconstructed word form from proto Uralic -- it's a word in a specific (reconstructed) language.

Now, mind you, ISO 639 does *not* provide IDs for historically-reconstructed proto-languages.

Peter Constable

---

From: "Mark Davis" <[email-masked]>
Date: Mon, 18 Sep 2006 00:49:58 -0700

The people are http://www.unicode.org/iso15924/iso15924jac.html
The process is at http://www.unicode.org/iso15924/standard/index.html#annex

I haven't looked it over at all, myself.

Mark

On 9/17/06, Peter Constable <[email-masked]> wrote:
> -          Can the public observe the process, or are you just one of the
> five non-voting observers in an otherwise closed process?
>
>   ------------------------------
>
> *From:* Mark Davis [mailto:[email-masked]]
> *Sent:* Saturday, September 16, 2006 11:46 AM
> *To:* Martin Hosken
> *Cc:* Doug Ewell; LTRU Working Group
> *Subject:* Re: [Ltru] Re: script tag for IPA
>
>
>
> 2. I think the private use codes (for script, etc) are misunderstood. If
> an authority designates a code as private use code, that means that it is
> recognized as a valid code, but *that *authority doesn't attach a meaning
> to it, *and never will.
>
> *It does not mean that nobody else can attach a meaning to it -- if that
> were the case it would be completely pointless to have private use codes.
> For example, the Unicode Consortium defines a meaning for Qaai (
> But it only takes part of the range: "The private use codes from XA..XZ
> will never be used by CLDR, and are thus safe for use for other purposes by

---

From: Martin Duerst <[email-masked]>
Date: Tue, 19 Sep 2006 22:21:33 +0900

>Just to clarify: I did not mean an abstract utterance

Talking about abstract utterances, the sounds produced by
our six month old doughter could be an interesting source,
but I doubt IPA could cover all of them, and I'm not eough
of an IPA user to actually try to write some of them down :-).

Regards,    Martin.