Subject: [Ltru] Preparations for smart matching

From: Frank Ellermann <[email-masked]>
Date: Sat, 19 Mar 2005 14:56:50 +0100

Hi, so far en-Latn-US-boont not matching en-boont is one of my
standard examples why I have a problem with 3066bis as it is.

This could be solved later by a smarter matching strategy.  Or
by subtags used in the order of their importance, something
like en-boont-Latn-US would match en-boont with the normal RTL
strategy.

But it would require complex restrictions allowing to identify
the class of some subtags independent of their position.  Most
probably that was discussed years ago before the first of the
old drafts, and it simply doesn't work.

So the last chance is smart matching.  To be smart the matching
algorithm needs some input.  For variants this input could be
the corresponding default language.   If boont has a pointer to
en, or if 1996 has a pointer to de, a smart matching algorithm
could identify fr-1996 or sl-boont as "garbage" and stop any
attempt to be smart.

For laguages the registry could list a default script, with a
"SHOULD NOT be used" somewhere in the draft.  For de / en / fr
that would be Latn, and the SHOULD NOT then reduces a dubious
en-Latn-boont to en-boont.  Or a de-Latn-1996 to de-1996.

The smart matching algorithm knows this and reduces all kinds
of en-Latn-* to en-* in a second matching attempt.  It also
knows that the US in any en-US-* is "only" a region code, so
it could start a third pass with en-*.  The longest match after
up to three attempts (default script removed, region removed)
wins.  Then en-Latn-US-boont finally matches en-boont.

And it would not match en-other-boont or en-other-US-boont with
en-boont.  For de-Latn-*-boont-* it would give up after the
reduction to de-boont-* shows that it's not a known en-boont-*.

So de-Latn-US-boot won't match de-boont, because boont is only
registered for use with en.  Does that make any sense ?  If yes
we need optional default scripts for languages, and maybe also
default languages for all variants in the registry.  Bye, Frank

_______________________________________________
Ltru mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ltru

---

From: John Cowan <[email-masked]>
Date: Sat, 19 Mar 2005 13:19:27 -0500

> standard examples why I have a problem with 3066bis as it is.

The real trouble is with en-boont, which is not even remotely mutually
intelligible with other kinds of en; it really isn't a "variety" in the
usual sense.  In an RFC 3066bis regime, I would probably press for it
to be registered as simply boont.

(This particular argument probably makes more sense for en, which is
not really a very variable language as languages go.)

> old drafts, and it simply doesn't work.

In fact it does work, provided that the algorithm keeps around a short list
of exceptions (much less than the whole registry).  Other than the
exceptions, one can identify a tag's class knowing only if it is initial
or non-initial, thus:

	2-letter initial:  ISO 639-1 language
	3-letter initial:  ISO 639-2 language, may also be an
		ISO 639-3 language in RFC 3066ter
	5-or-more-letters initial:  currently not allowed, will be a language
		in RFC 3066bis
	2-letter non-initial: ISO 3166-1 country
	3-letter non-initial: currently not allowed, will be an ISO 639-3
		language in RFC 3066ter
	4-letter non-initial: ISO 15924 script
	3-digit non-initial: UN M.49 country
	anything else non-initial: variety

Note that ISO 639-2 and ISO 639-3 language codes are compatible, so one does
not need to determine (in an RFC 3066ter regime) which is which.

> So the last chance is smart matching.  

I think it will be straightforward to design an algorithm that does
matching on the above facts, without having to worry about
what-goes-with-what rules.  (Indeed, if another language turned out
to have reformed its spelling in 1996, it would make sense to reuse
the variety tag "1996" for it.)

-- 
John Cowan    http://www.ccil.org/~cowan   <[email-masked]>
    "Any legal document draws most of its meaning from context.  A telegram
    that says 'SELL HUNDRED THOUSAND SHARES IBM SHORT' (only 190 bits in
    5-bit Baudot code plus appropriate headers) is as good a legal document
    as any, even sans digital signature." --me

---

From: John Cowan <[email-masked]>
Date: Sat, 19 Mar 2005 13:34:31 -0500

> exceptions, one can identify a tag's class knowing only if it is initial

I meant to add here that we do not need to match language-ranges lacking
an initial subtag; a range like *-Latn wouldn't represent any Real
World requirement.

-- 
But that, he realized, was a foolish            John Cowan
thought; as no one knew better than he          [email-masked]
that the Wall had no other side.                http://www.ccil.org/~cowan
        --Arthur C. Clarke, "The Wall of Darkness"

---

From: "Jon Hanna" <[email-masked]>
Date: Sat, 19 Mar 2005 18:42:30 -0000

> an initial subtag; a range like *-Latn wouldn't represent any Real

There are no real world systems capable of dealing with Latin text in any
language but not with all scripts?

Regards,
Jon Hanna
Work: <http://www.selkieweb.com/>
Play: <http://www.hackcraft.net/>
Chat: <irc://irc.freenode.net/selkie>

---

From: John Cowan <[email-masked]>
Date: Sat, 19 Mar 2005 14:36:47 -0500

> language but not with all scripts?

Fonts, maybe, but fonts don't care about language one way or another.
The Latn subset used by en-Latn isn't the one used by ie-Latn, e.g.

-- 
What asininity could I have uttered     John Cowan <[email-masked]>
that they applaud me thus?              http://www.reutershealth.com
        --Phocion, Greek orator         http://www.ccil.org/~cowan