Subject: RE: [Ltru] Re: Language transformation

From: "Addison Phillips" <[email-masked]>
Date: Thu, 31 Mar 2005 09:06:22 -0800

Why not an extension? Using variants for this strikes me as stretching the use of language tags beyond their reasonable usefulness.

Transliteration seems like it might be handled using script codes in a gross sense. For the finer sense this draft wants to convey, an extension seems like a better fit. 

Addison

Addison P. Phillips
Globalization Architect, Quest Software
Chair, W3C Internationalization Core Working Group

Internationalization is not a feature.
It is an architecture. 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Frank Ellermann
> Sent: mercredi 30 mars 2005 21:12
> To: [email-masked]
> Subject: [Ltru] Re: Language transformation
> 
> M.T. Carrasco Benitez wrote:
> 
> > I am sending this today to the editor.
> 
> Interesting, if I get it right you want LTRU to use a scheme
> which allows the later addition of transliterations.  You
> identify five relevant parts:
> 
> 1: source language, 2: source script, 3: target laguage(s),
> 3: target script, 5: (registered) transliteration scheme.
> 
> Two of these parts could be irrelevant, the source script,
> and target language(s) if they are implied by the scheme.
> 
> This would fit into a lang + script + variant pattern, if
> you consider a "registered transliteration scheme" as some
> kind of "registered variant".
> 
> Is that good enough for your purposes, and do you want it
> as an example for "variants" in the registry draft ?
> 
>                    Bye, Frank
> 
> 
> 
> _______________________________________________
> Ltru mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ltru

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: "Peter Constable" <[email-masked]>
Date: Thu, 31 Mar 2005 09:04:48 -0800

> From: [email-masked] [mailto:[email-masked]]
On Behalf Of
> Doug Ewell

> > zh-wadegiles
> 
> I think this might make a good candidate for an extension RFC.  It's
not
> strictly an attribute of the language, but might be important for
> content storage and retrieval.  For example, it might be valuable to
> specify whether "kr-Latn" text is romanized using McCune-Reischauer,
> Yale, or Revised Romanization, or whether "ru-Latn" is romanized using
> ISO 9, GOST, or some other language-dependent scheme (which is common
> for Russian).

I don't see why an extension RFC should be necessary at all. Think of
these different Romanizations for kr as being comparable to different
spelling conventions. A variant subtag should be sufficient:

kr-Latn-mccnrsch
kr-Latn-yalermn
kr-Latn-revdrmn

ru-Latn-iso9rmn
ru-Latn-gostrmn

zh-Latn-wadegile

(Or perhaps "Latn" is redundant here and can be omitted.)

> Transcription schemes that don't have a name, but are dependent on the
> target language (again, common for Russian), might be indicated with a
> 2-character subtag representing the target language.  Thus
> "ru(-Latn)-t-de" would be a transcription of Russian intended for
German
> readers, while "ru(-Latn)-t-fr" would be the same for French speakers.
> In that case, subtags that don't represent a target language would
have
> to be 3 or more characters long.

I'd rather (i) avoid the use of extensions, and (ii) see particular
conventions documented somewhere and given an ID that can be used as a
variant subtag. Except for cases that are general enough, such as
perhaps IPA, that it might make sense to get an alpha-4 ID added to ISO
15924.

Peter Constable

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 18:58:00 +0100 (BST)

> I agree that "transliterations" are not as cut and dried as TC46 makes them out to be. My only
> point was that there are many things that fall in the category of "transcription" (clearly that
> and not transliteration), such as IPA, that really aren't conversions or transformations.

I am ready to accepts almost any term; as long as it is decide once and for all.
A secret; do not tell anybody -:)  I preferred term is "transliteration", but I got hammered by
the parties that want to make a difference with "transcription".

Regards
Tomas

Send instant messages to your online friends http://uk.messenger.yahoo.com

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 19:03:36 +0100 (BST)

> (Or perhaps "Latn" is redundant here and can be omitted.)

The script token should not be present and it shoudl be implied y the scheme. 
Hence,

 zh-wadegile

Tomas

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 19:07:47 +0100 (BST)

> Transliteration seems like it might be handled using script codes in a gross sense. For the
> finer sense this draft wants to convey, an extension seems like a better fit. 

Script are not sufficient. Examples:

 <span lang="zh-widegiles">Peking</span>
 <span lang="zh-pinyin">Beijing</span>

Regards
Tomas

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 19:47:17 +0100 (BST)

> However, using variants complicates matters. The transformation applied to the text is only one
> dimension of the content. It may still have a language ("zh"), a dialect (in 3066ter a 639-3
> extland code, such as wuu???), a script (Grek, Latn, Hant, etc.), a region (CN, HK, SG, TW...),
> and a variant not related to the transformation itself. These may be as important to tag the
> content with as the transformation applied.
> 
> Extensions allow this kind of semi-orthogonal, language-related distinction to be applied.
> Consider:
> 
>   zh-wuu-CN-t-wadegile

Corrent. This the requirement. I do not know if this would be the appropiate syntax.

Regards
Tomas

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 31 Mar 2005 13:29:38 -0800

Extensions are defined in the draft. They are separated from the rest of the tag and identified by a singleton---a one letter subtag such as 't'. 

My examples in this thread show an extension whose identifier is 't'.

Addison

Addison P. Phillips

> Behalf Of Debbie Garside
> Sent: jeudi 31 mars 2005 13:25
> To: 'Peter Constable'; [email-masked]
> Subject: RE: [Ltru] Re: Language transformation
> 
> Peter Constable wrote:
> 
> >I don't see why this is superior in any way to zh-wuu-CN-wadegile, and
> can
> >see reasons why it's inferior: the extension is ignorable by anything
> that
> > knows only RFC 3066bis and not some other extensions RFC.
> 
> Good point... unless of course the -t- is added to RFC3066bis
> 
> Debbie
> 
> Behalf Of Peter Constable
> Sent: 31 March 2005 21:46
> To: [email-masked]
> Subject: RE: [Ltru] Re: Language transformation
> 
> > Addison Phillips
> > However, using variants complicates matters. The transformation
> applied to the text
> > is only one dimension of the content...
> 
> The complication you site is that variants can distinguish dialectal
> variants, or variants in written form (just as region subtags can
> distinguish regional variations in dialect or regional variations in
> written form), and the two are not distinguish (any more than they are
> for region subtags).
> 
> 
> > Extensions allow this kind of semi-orthogonal, language-related
> distinction to be
> > applied. Consider:
> >
> >   zh-wuu-CN-t-wadegile
> 
> I don't see why this is superior in any way to zh-wuu-CN-wadegile, and
> can see reasons why it's inferior: the extension is ignorable by
> anything that knows only RFC 3066bis and not some other extensions RFC.
> 
> 
> 
> Peter Constable
> 
> 
> 
>

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 22:59:01 +0100 (BST)

--- Mark Davis <[email-masked]> wrote:
> > uri:http://somewhere.xxx/speficationofwidegiles
> 
> 
> That very policy has been a disaster with IANA charsets. The references
> don't supply a complete description, or are not stable. That's why I think
> this is premature.

It is not realistic for LTRU to try to specify every imaginable language transformation.
The scope should be limited to provide the mechanism. The Registry for Schemes for
Language Transformation should try to maintain a minimal discipline. But at the end of
the date, it is up to the community interested in any particular transformation to take
responsability.

Regards
Tomas

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 23:12:15 +0100 (BST)

--- "JFC (Jefsey) Morfin" <[email-masked]> wrote:
> What should be considered is the result to the user.

Yes. This is the point.

> A recording or a 
> transcoding resulting in the same voice-text, look the same to me? I do not 
> see script difference between a voice, in Russian and a voice in English 
> speaking the same way? Questions can be about the sex, accent, timber, age 
> of the voice and about the speaking style? Same a short-hand or phonetic or 
> icons or music or menu figures (as long as there is a common understanding 
> of what means what - what is a script?)
> 
> Describing the process would be like documenting the version of Apache 
> being used to show an HTML page?

There is real need for tagging transliterations and the requirements are as
discussed before.

Some of your comments above might well fall in a similar model to
transliteration; so any volunteer ? Other comments looks more like metadata,
though the language is also metadata.

Regards
Tomas

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 23:26:05 +0100 (BST)

--- John Cowan <[email-masked]> wrote:
> I propose the term "trans*tions" for these unnatural acts of orthography,
> thus covering transliteration, transcription, and transformation.

"trans*tions" would probably create more confusion than "transformation";
this last term covers both transliteration and transcription. TC46/SC2 uses
the term "convertion" with the intention of covering also boths aspects. 

The term that most people associate with these unnatural acts is
"transliteration". To use this we just have to:

 - Define it to cover also transcription and similar nuances.
 - Resist the attack of the purists.
 
> I further propose that the Right Thing with these is to use the
> RFC 3066bis language, optional script, and variant fields, using
> variants specifically chosen for the purpose:  thus Wade-Giles could
> be zh-Latn-wadegiles, or just zh-wadegiles.  The English-language
> trans*tion of Russian might be ru-transen or ru-Latn-transen; I think
> it is unnecessary to introduce an extension feature for this purpose.
> 
> But I think it would be good to introduce an example (perhaps the
> wadegiles one) into the draft, to make it clear that variants may be
> orthographic as well as linguistic.

On purpose, I was staying away from the syntax. Now I am going to have
a harder look to the syntax.

Regards
Tomas

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 31 Mar 2005 14:44:50 -0800

Yes, or at least from nearly the outset: the RFC would be adopted first. Then section 3.4 could be used to define -t/T- as a singleton for a transmutation extension.

Addison

Addison P. Phillips

> From: Debbie Garside [mailto:[email-masked]]
> Sent: jeudi 31 mars 2005 14:37
> To: Addison Phillips; 'M.T. Carrasco Benitez'; [email-masked]
> Addison Phillips wrote:
> >Extensions are defined in the draft. They are separated from the rest of
> >the tag and identified by a singleton---a one letter subtag such as 't'.
> 
> So... under section 3.4 a -t/T- singleton could be registered as defining
> transliteration subtags from the outset?
> 
> Debbie
> 
> 
> Behalf Of Addison Phillips
> Sent: 31 March 2005 23:20
> To: M.T. Carrasco Benitez; [email-masked]
> So let me understand...
> Are you proposing to create a *separate* standard for tagging
> transliterations of content with its own registry? Or are you proposing
> the
> inclusion of these items into language tags proper?
> 
> Addison
> 
> Addison P. Phillips
> On
> > Behalf Of M.T. Carrasco Benitez
> > Sent: jeudi 31 mars 2005 14:12
> > To: [email-masked]
> > Subject: Re: [Ltru] Re: Language transformation
> >
> > > transcoding resulting in the same voice-text, look the same to me? I
> do
> > not
> > > of the voice and about the speaking style? Same a short-hand or
> phonetic
> > or

---

From: "M.T. Carrasco Benitez" <[email-masked]>
Date: Thu, 31 Mar 2005 23:47:18 +0100 (BST)

--- Addison Phillips <[email-masked]> wrote:
> So let me understand...
> 
> Are you proposing to create a *separate* standard for tagging transliterations of content with
> its own registry? Or are you proposing the inclusion of these items into language tags proper?

Transliteration should in the language tag proper. But LTRU should only provide the mechanism
for including transliteration in the language tag. Example (pseudo-code):

 zh-wadegiles

As proposed in section 4 of "Codes for language transformation"
http://www1.ietf.org/mail-archive/web/ltru/current/msg00387.html

"Two registries are need: 
    
 - IANA  
 - Schemes for Language Transformation"

Lets call "wadegiles" the "transliteration identifier". As commented in a previous message,
the IANA register should contain the list of unique transliteration identifier. Example:

# Transformations
transformation | widegile | | yyyy-mm-dd | |
transformation | pinyin| | yyyy-mm-dd | |

The Registry of Schemes for Language Transformation (out of scope of LTRU)
should address the description of the transformation. Example:

%%
id:widegile
sourcelang:zh
targetlang:en
uri:http://somewhere.xxx/speficationofwidegiles
foo:foo
%%
id:pinyin
sourcelang:zh
targetlang:en
uri:http://somewhereelse.xxx/speficationofpinyin
foo:foo
%%

This registry should not address the methodology of the transformation. It should
just point with the URI to the entity maintaining the methodology.

Regards
Tomas

---

From: "Addison Phillips" <[email-masked]>
Date: Thu, 31 Mar 2005 15:00:42 -0800

Okay, I understand. It wasn't clear to me previously which way you wanted to go.

Some pointers and comments, then.

First, for backward compatibility reasons subtags in language tags are limited to eight characters (so 'wadegile', not 'wadegiles').

Then, I would point out that 3066bis as it sits defines a complete syntax for language tags. The subtags are broken out according to type and are identifiable by length and position in the tag.

As John Cowan pointed out, we could have an orthographic variation such as transliteration as an example of the variant type in the 3066bis document. At present the only subtag type that can accommodate transformation subtags would be registered variants. And variants subtags must be defined in the language subtag registry to be valid in a language tag.

Syntax of the language tags and the interrelation between different subtags is actually quite critical to progress of the 3066bis draft. I would urge you to consider the current design carefully and think about how your proposal could either fit into the existing design or how to adapt the designs to one-another.

I think that the options for what you want to do would be:

a) add a new subtag type, distinguished from variants in some form, with additional verbiage in the 3066bis draft describing any additional information and usage information for that type

b) don't distinguish transformations per-se, just register variants for the different ones you're interested in on an ad hoc basis under 3066bis

c) write an extension per the rules in 3066bis, which would allow you to have a separate registry and syntax for the subtags and use them in language tags

I would very strongly oppose (a), personally, as I don't see transformations as being significantly different from other language variations to warrant greatly complicating the design of language tags further. You may (or may not) find (b) to be sufficient to your needs.

Thanks,

Addison

Addison P. Phillips

> Sent: jeudi 31 mars 2005 14:47
> To: [email-masked]
> transliterations of content with

---

From: "Peter Constable" <[email-masked]>
Date: Thu, 31 Mar 2005 15:22:20 -0800

> From: [email-masked] [mailto:[email-masked]]
On Behalf Of M.T.
> Carrasco Benitez

> Language Transformation should try to maintain a minimal discipline.
But at the end
> of

Let's be clear about the difference between documenting and providing
some identifier for "transformations" on the one hand, and providing
tags to declare the linguistic variety and form of content that may
result from such "transformations" on the other. 

Only the latter is relevant for RFC 3066bis; I will assume RFC 3066bis
as proposed will be sufficient for such purposes unless scenarios
suggesting otherwise can be provided.

I'd have to take another look at the charter to see if the former is
reasonably within scope for LTRU, but it doesn't seem unreasonable that
it could be.

Peter Constable

---

From: "Doug Ewell" <[email-masked]>
Date: Thu, 31 Mar 2005 20:29:00 -0800

M.T. Carrasco Benitez <mtcarrascob at yahoo dot com> wrote:

> By the way, Armenians are celebrating this year the 1600th anniversary
> of their script. It would be nice to go to the "Targmanchats Ton"
> (translators day). This is a case of very successful standardisation:
> would LTRU be here in 1600 years ?  -:)

Well, certainly not the working group.  We're only chartered to do 2
documents, and the second is scheduled to go into Last Call this August.

If you mean, "Will the revised language tagging standard exist and be
relevant in 1600 years?" the answer is probably no.  But then, most
technology standards don't last as long as most writing systems, so that
seems an odd comparison.

-Doug Ewell
 Fullerton, California
 http://users.adelphia.net/~dewell/