Subject: Re: Last Call: 'Tags for Identifying Languages' to BCP

From: Bruce Lilly <[email-masked]>
Date: Sun, 28 Aug 2005 10:15:00 -0400

>  Date: 2005-08-25 20:55
>  From: "JFC (Jefsey) Morfin" <[email-masked]>

> the privacy problem is the "what you read, who you are" intelligence 
> leak.

That is to some extent true of any negotiation mechanism and negotiated
value.

> Today langtags are not yet much used (say the W3C people in the  
> WG-ltru) when compared with  what they should in XML, HTML, etc.

XML, HTML, etc. are not IETF protocols and should not be the main
consideration in IETF work on IETF documents, especially as language tags
are heavily used by IETF protocols, notably MIME (RFCs 2045, 2047, 2231,
3282) and widely-deployed core IETF application protocols which use MIME
(e.g. the Internet Message Format and its applications (email, news, voice
messaging, EDI, etc.) and HTTP and applications using HTTP as a substrate.

> This  
> is all what this proposition is about. This proposition is to give 
> _one_shot_ in a _standardised_ way the language, the script and the 
> country.

This was discussed during Last Call of the previous non-IETF (individual
submission) attempt.  IIRC David Singer brought up several examples of
other pieces of information (e.g. legal/copyright variations) that could
also be negotiated and which might affect the presentation of content (or
choice among alternative content).  Lumping all of these separate items into
one tag is a poor design as it impedes negotiation and tends toward lengthy
tags which are incompatible with fixed-length mechanisms such as MIME
encoded-words.  While there is some mention of this issue in the document
under discussion, its treatment and resolving the underlying issue in a
manner that would minimize the problems are lacking.

> It uses for that ISO codes. ISO never wanted to propose such  
> a code where:
> 
> ar-arab-us are texts destined the people interested in US Arabic 
> community issues.
> iw-hebr-ru are texts destined to people interested in Jewish Russian 
> community,
> etc.
> 
> When you browser accept that langtags and you pursue the relation, 
> this structured information can be filtered by ISP (for police, 
> censoring, intelligence gathering, etc.) to know about their users. 
> It can be used for searches on a large scale in search engines to 
> know the mail you responded, etc. I suppose that in most of the world 
> countries this is subject to privacy laws. I think that in France it 
> is subject to the anti-racist law (the one used against Yahoo a few years ago).

Let's separate three issues:
1. privacy
2. tagging
3. negotiation

The privacy issue exists whenever any information is conveyed; the user
needs to balance privacy concerns with facilitation of communication.
Mechanisms such as TLS can be used to limit the visibility of the information
to the end points of communication; ultimately it boils down to a matter of
trust in the end-point partner in the communication exchange.  I believe
that the issue is dealt with adequately in the security considerations
section of the document under discussion (some mention of transport-level
protection of privacy would be welcome).

Tagging identifies characteristics of a particular piece of content.  For
that purpose alone, it makes little difference (other than regarding the
aforementioned compatibility issues with existing IETF mechanisms) whether
the characteristics are lumped or separate.  There are existing IETF
mechanisms which permit handling of either lumped or individual characteristics
(e.g. the extensible header field mechanism of RFC 2045 and the "feature/filter"
mechanism of RFC 2533/2738/2912).  Tagging per se identifies characteristics
of content.  While that may be used to infer something about the content
provider, such inferences may be unreliable, particularly for providers that
support a wide variety of characteristics for the content in question.

Negotiation of characteristics is where several issues arise.  One such
issue, as discussed here in December 2004/January 2005 relates to an
algorithm for matching content characteristics (e.g. between a particular
piece of content and a specified range of acceptance (as in an RFC 3282
Accept-Language field).  RFC 3066 skirted that issue as it stopped short of
specification of an algorithm, and as it specified a mere two particular
characteristics (language per se, and country) which could be combined in
a tag.  That was not true of the individual submission, which combined at
least 5 characteristics and specified an algorithm.  As a result of issues
with that approach, the LTRU WG was established with a charter to produce a
BCP (for registration procedures) and a separate Standards Track document
for topics such as algorithms which are unsuitable for BCP.  A related issue
is the interaction of the established negotiation mechanism (viz. the RFC
3282 Accept-Language field) and potential use of the other (feature/filter)
mechanism for negotiation.  The Accept-Language field provides for
specification of language ranges and for associating a preference value
with specific languages (as defined in RFC 3066) or ranges.  The proposed
mechanism in the individual submission of late last year (essentially
unchanged in the LTRU product (see discussion below)) does not address the
language range issue, and that issue is greatly complicated by conflating
separate characteristics into a single tag.  Addressing the language range
issue is not a WG work item and, unfortunately, the algorithm issue is
scheduled to be a later work item than the registry issue.  Added to that
is the fact that the specification of the tag format is mixed with
registration procedures.  Negotiation of separate characteristics is much
simpler than that of a combined conflation of characteristics; each
characteristic can be assigned separate preference values, and irrelevant
characteristics (e.g. script w.r.t. spoken language) can be easily ignored.

As negotiation and related issues represent a critical technical issue for
the design of language tags (viz. keeping separate characteristics out of
*language* tags), it is essential that such negotiation issues be considered
carefully before specifying the format of tags.  Unfortunately, that has not
been done, and considering the published WG milestones it appears that that
issue has not been taken into consideration.  It should be pointed out that
such issues have been raised, both in the discussion during Last Call of the
individual submission and as a result of subsequent work.  However, it
appears that the WG has not considered the issues, with the effect that the
WG product lacks the "particular care" expected of BCP documents (RFC 2026).
Note that it is not the registration procedural issues that are typical of
BCP documents that are problematic; rather it is the conflation of separate
characteristics into a single tag syntax, specified in the same document,
which raises problems related to content negotiation.

Part of the problem is the scheduling of WG work items as noted above
(viz. negotiation issues are critical to design of tag syntax, and should not
have been deferred until after syntax specification).  Another large part of
the problem is WG management; in addition to the issues raised by John
Klensin the last time that LTRU participation was discussed on the IETF
discussion list -- and with which I wholeheartedly agree -- it appears that
management of WG participant conduct has been rather lax; proponents of the
individual submission effort who are participating in the WG tend to resort
to ad-hominem attacks when a problem is identified or when an alternative
approach is raised, with no visible intervention by the WG co-chairs.  That
has also (i.e. in addition to the factors which John identified) had the
effect of limiting WG participation by individuals.

Specification of "language" tag syntax which conflates other content
characteristics prior to open and professional discussion of negotiation
issues and alternative approaches would be a premature lock-in of a design
choice.  As the document under discussion specifies a conflation of such
characteristics without open discussion -- indeed hampered by unchecked
unprofessional conduct -- it should not be approved as BCP in its current
form.  Separation of syntax specification to a separate document, to be
specified after due consideration of negotiation issues, leaving purely
procedural issues of registration, would be one approach to enable making
a decision on BCP registration procedures independently of an in advance of
a concrete specification of negotiation issues and tag syntax.  However,
as it stands, the document cannot be evaluated for soundness of the tag
syntax design in the absence of a specification that addresses negotiation
issues (in a backwards-compatible manner with the existing negotiation
mechanisms (viz. MIME Content- and Accept- fields and feature/filter
negotiation).

Therefore, at minimum, I recommend that the IESG defer a decision on the
subject document until such time as the full impact of the early design
choice to conflate multiple characteristics into a single tag can be fully
evaluated w.r.t. proposed matching algorithms and impact on existing
IETF-approved negotiation mechanisms.   Revision to move the syntax
specification to a separate document, as mentioned above, would permit
evaluation of the registration procedures per se independently of such
concerns, and would be one way to move forward on those registration
procedures quickly (i.e. independently of analysis of the syntax design)
if that is deemed desirable.

Aside form that, the IESG (via the cognizant ADs) should address the issues
of WG charter work items and milestones as they relate to consideration of
negotiation issues prior to locking down a tag syntax specification, should
emphasize the importance of backwards compatibility with established,
approved, and widely deployed IETF protocols and mechanisms, and should
discuss WG participant conduct (viz. ad-hominem attacks) and mailing list
issues (as identified by JCK) with the WG co-chairs.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Bruce Lilly <[email-masked]>
Date: Sun, 28 Aug 2005 10:42:53 -0400

> On 00:40 26/08/2005, David Hopwood said:

> >This objection seems to be correct: URI tags include characters not 
> >allowed by RFC 3066.
> 
> Then? The purpose of this work is to address the limitations of RFC 
> 3066. URI tags did not exist when RFC 3066 was written.

RFC 1738 certainly existed, not only at the time of RFC 3066, but its
predecessor RFC 1766 as well.

> please document how do you do, while respecting the hybrid format of 
> the proposed ABNF where information is not indentified by fixed 
> position, but also relative position and size, with "-" as sole 
> separator. And they want to keep labels between "-" 8 characters 
> long. Tell me how you support IDNs.
> 
> Let suppose that I have "lang-tags.org:" as a scheme.
> or "xn--abcdef.com:". Tell me how you support them

It's unclear what you're trying to get at here.  A URI scheme is a
protocol element (an "assigned number") registered by IANA, not a
piece of text (see RFCs 1958 and 2277).  As such, it has no need of
an indication of language, for it has no language; it is a language-
independent protocol element.  Confusing protocol element issues with
language will only muddy the water; try to stay focused on real
problems.

For that matter, DNS labels are public names (i.e. protocol elements,
again see RFC 1958 (sect. 4.3, noting that "text" there has a different
meaning than in RFC 2277)) and as such there should not have been any
reason to overload the semantics and baggage of internationalized
text (in the RFC 2277 sense).  Now, having made the decision to
nevertheless do so, you might well point out that per RFC 2277, there
ought to be a means of indicating language in IDNs.  However, that is
primarily an issue with the IDN specification(s), not with the document
under discussion (except to the extent that the document under
discussion extends the likely length of tags in a way that is likely to
conflict with the DNS label length and domain name length limits, *if*
there were in fact provision in IDN for the use of language tags. You
might also point out that as IDNs use utf-8 exclusively as a charset, and
as script is easily inferred from the Unicode code points corresponding
to utf-8, that the length-increasing provision for conflating script with
language would be unnecessary and redundant *if* IDNs had provision for
language tags.  But IDNs have no such provision at this time.

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 28 Aug 2005 22:25:31 +0200

Bruce Lilly wrote:

> While there is some mention of this issue in the document
> under discussion, its treatment and resolving the underlying
> issue in a manner that would minimize the problems are
> lacking.

That's a last call, if you have better ideas than those in the
draft speak up.  Your Content-Script idea is good, but won't
help e.g. in encoded words (2047+2231).  We definitely tried
to minimize especially this problem.  

This is a ready-for-Bruce's-review draft as far as I can judge
this, but for obvious reasons only you can really judge it. ;-)

> Addressing the language range issue is not a WG work item
> and, unfortunately, the algorithm issue is scheduled to be a
> later work item than the registry issue.

Only my personal view of course, but the matching draft offers
a syntactical form for ranges, and the Suppress-Script in the
registry provides for backwards compatibility where possible.

> it is essential that such negotiation issues be considered
> carefully before specifying the format of tags.
> Unfortunately, that has not been done

IBTD, we considered the script issues.  Anything else is as
good or bad as it is with 3066 - some minor problems fixed of
course, if ISO 3166-1 pulls another CS 3066bis will handle it
better than 3066 (no potential worldwide retagging confusion).

> the WG product lacks the "particular care" expected of BCP
> documents (RFC 2026).

IBTD as far as scripts are concerned.

> it appears that management of WG participant conduct has been
> rather lax

IBTD, the WG Chairs and the responsible AD did a very good job.

> as it stands, the document cannot be evaluated for soundness
> of the tag syntax design in the absence of a specification
> that addresses negotiation issues (in a backwards-compatible
> manner with the existing negotiation mechanisms (viz. MIME
> Content- and Accept- fields and feature/filter negotiation).

IBTD, see above.  Your idea to split tag and registry syntax
from all procedural aspects of tag registration is possible,
but you get the same effect by "ignore the procedural stuff
in chapter 3" (= about 14 of the 60 pages in the draft).

> Revision to move the syntax specification to a separate
> document, as mentioned above, would permit evaluation of the
> registration procedures per se

You can also read chapter 3 per se, the mentioned 14 pages
plus 3.1 as introduction (5 pages, format of the registry).

I'm not violently against splitting the document, but it's
IMHO unnecessary.
                  Bye, Frank (also posted on the LTRU list)

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 29 Aug 2005 01:14:09 +0200

At 16:42 28/08/2005, Bruce Lilly wrote:
>piece of text (see RFCs 1958 and 2277).

A proposition I received from the WG-ltru was to slice URI tags into 
8 aphanum labels, etc. Among many problems URI tags accept domain 
names, mail names and IP addresses as registered identifiers. The 
main problem with the ABNF is therefore the use of "-" as a separator 
since "-" is a legitimate character in domain name. The support of 
IRI tags is impossible.

Our work is on CRC (common reference centers). offering such 
references. URI tags are the correct solution (with some restrictions 
we will probably document once the RFC is published)

>As such, it has no need of
>there were in fact provision in IDN for the use of language tags.

Fully correct. The response is with the approved pending URI Tags RFC.
IRT IDN, as a "multilingualiser" I disapprove IDNA. But whatever the 
final solution the MLDN charsets may use a langtag like solution. 
Hence the interest. Another interest is that currently the IANA uses 
RFC 3066 language tags to identify the IDN tables. What is IMHO an error.

>You

Correct. The MLDN problem is IMHO a different issue. However I say:

1. a langtag may be associated to a locale (this is in the WG-ltru 
Charter [Unicode CLDR project and our own ISO 11179 related solution])
2. we think there should be DNS locale for some important sites and services
3. DNS locale could also be the proper place to distribute MLDN 
virtual zones charsets (several concepts to discuss, specify and deploy).

jfc

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Mon, 29 Aug 2005 02:33:30 +0200

Dear Bruce,
I will try to quickly comment/respond/suggest on some of your well made points.

>That is to some extent true of any negotiation mechanism and negotiated

True. The problem are:
- the unecessary accumulation of orthogonal information
- the easily identified characteristic format: an enormous difference 
between "xx-xx-xxx-xx" (Draft) and "xxxx" (ISO 639-6)
- the lack of alternative (are we sure there are no other 
architectural way to address the same need without information leak)
- the lack of encryption
- the "spam" aspect: I am imposed to receive the langtag.

>consideration in IETF work on IETF documents,

They are specifically quoted by the Charter. Also is CLDR a private 
proposition to unify "locale" file which has interest but also competition.

>especially as language tags

RFC 2231 is among the reference quoted. I more interested in R&D. My 
concern is that OPES have been disregarded.

>manner that would minimize the problems are lacking.

The work we carried on language in a common reference center (where 
are stored the common parameter of a relation) shown us that must be 
included in negociation two classes of additional information. The 
parameters in the community (we call referent: i.e. dictionary, etc) 
and the context of the exchange (style, personal meanings, 
circumstances, etc.). These elements are necessary for OPES call-out 
servers supervising a relation. These elements are by default used by 
... Word (language, script, country, dictionary, style).

The Draft proposes a system which permits to evaluate the locale the 
computer should support for end to end interoperability purposes. It 
does not necessarily permit to establish, maintain and serve a brain 
to brain interintellibility.

>protection of privacy would be welcome).

Not really: see above. The concept is an help to privacy violation:
- more secure alternatives should be investigated and proposed
- the danger is not worth the result, necessary information is missing.

>support a wide variety of characteristics for the content in question.

This confusion will be an increasing problem. More and more the 
"architext" we use (the data from which we infer the text we read) 
become intelligent and multilingual. I currently use a site 
multilingual generator. This means that it uses multilingual texts to 
generate unilingual version of a web site. It uses a default langtag 
scheme (:xxx) to indicate the language of the lingual parts.

>scheduled to be a later work item than the registry issue.

The language negociation issue is independent from any language 
identfier format. But obviously langtag formats may or not better 
serve language negociation.

>Added to that is the fact that the specification of the tag format 
>is mixed with

At this stage many negociation elements are missing. The elements 
related to the referent and to the context are missing. For example a 
traveler will accept more easily a foreign language when it comes to 
the location he tours (context). And a professional when it comes to 
a technical discussion (referent). All the more than terminology OPES 
services or on the fly traduction assistance can be provided

>WG product lacks the "particular care" expected of BCP documents (RFC 2026).

It is to note that ISO 639-4 work is about discussing guidelines in 
that area. This work is under way and was not considered.

>effect of limiting WG participation by individuals.

I will not object that remark. The advantage was that proposing an 
alternative approach resulted in an improvement of the ABNF to 
impeach it. The result is a relatively clean default ABNF which now 
permits to avoid confusion with specific solutions introduced by 
reserved singleton. This permits to support:
- my Draft as a default proposition
- to specify easily other formats and conceptions (such as based upon 
ISO 639-6, or ISO 11179 conformant, etc.) without risking conflicts.

>form.  Separation of syntax specification to a separate document,

Yes!!!

>to be specified after due consideration of negotiation issues, leaving purely
>procedural issues of registration,

Yes!!! supporting multimodal competences (not only script, but also 
signs, voice, icons, moods, style, etc.)

>  would be one approach to enable making
>IETF-approved negotiation mechanisms.

At that time we should have running services. ISO 639-6 authors just 
announced that sample table will be available in Novembre. And ISO 
639-3 author expects it to be published by the end of the year. The 
we can start experimentation. Locking the multilingual internet core 
system into a final ABNF seems premature.

>  Revision to move the syntax
>approved, and widely deployed IETF protocols and mechanisms,

and documented efforts such as OPES, document the way langtags will 
be used and their applications documented.
jfc