Subject: [Slim] draft-tomkinson comments

From: "Phillips, Addison" <[email-masked]>
Date: Wed, 21 Oct 2015 02:10:06 +0000

Hi,

As promised, a number of comments on the current draft-tomkinson follow my .sig. Comments are labeled by section in the document.

Addison

Addison Phillips
Principal SDE, I18N Architect (Amazon)
Chair (W3C I18N WG)

Internationalization is not a feature.
It is an architecture.

=====

Abstract: The term "language code" should be replaced with the term "language tag". It's helpful to be consistent to avoid confusing with ISO standard elements (which are generally referred to as 'codes') and to adopt the same terminology as BCP 47.

3.1 Health warning: the explanatory section may need to use different scripts to handle all of the languages. This suggests that the explanatory block should use the UTF-8 encoding. A health warning about character encodings and the recommendation for using UTF-8 would be useful if included here?

3.1 "language (or locale)": the term locale is unnecessary here, unless you want to explain that the language might be derived from the end-user's locale. Should there be a terminology section? W3C is working on a document that defines some of these, but it is not mature enough to serve as a reference (http://www.w3.org/TR/ltli/). Unicode TR 35 talks about the same issue: that languages and locales are fundamentally related and that you can, in many cases (such as this one) ignore the locale concept.

3.2 "language message parts are translations of the same message" This presumes that they are translations. They might not be exact translations and might not be exactly the same. While the *intention* of the document is that they are translations, there is no guarantee of such. Perhaps "...are usually translations..."

3.2 The recommendation to apply a 'Name'  parameter kind of presumes that the name is meaningful in ASCII, since this is an header field and QP or MIME encoding won't be anything but a bunch of gibberish. Rendering into ASCII or into English is not very international friendly and having the example be 'english.txt' is ignoring the problem?

3.3 The unmatched message part seems problematic to me. It requires the sender to provide a version of the message (increasing message size--some systems have size limits!) that will be shown when language negotiation fails, rather than allowing one of the already extant sections to be selected as the default. There is already a preamble section for cases that multipart/multilingual is not supported. And the unmatched message (probably falsely) affects a pose of having "no language". Why not allow one of the language bearing sections be tagged as the "default" message? If a sender wishes to make the "default" (unmatched) message be language neutral somehow, they can tag it using a tag provided for that purpose, such as zxx or und.

Alternatively, why not use the Preface section? If the end user wants to read the message, presumably they will need to choose one of the languages to do it in.

4. This section is introduced:

--
   The logic for selecting the message part to render and present to the
   recipient is quite straightforward and is summarised in the next few
   paragraphs.
--

It seems hyperbolic to say "quite straightforward". Eliminate that phrase?

4. The section says:

--
...select the best match for
   the user's preferred language from the language message parts
   available
--

But describes that as:

--
The selection of language part may be implemented in a
   variety of ways and is a matter for the email client and its user
   preferences.
--

Isn't this the nub of the problem? I know that, for example, Mark Davis suggests that some clients may wish to use more complex or helpful matching schemes than those defined in BCP 47 (which are pretty simplistic). We don't have to forbid something more elegant. But fundamental matching is improved if there are some base recommendations.

4. There is a note about the subject line:

--
Similarly, the subject to display (for example in a
   message listing) should be chosen from the selected language message
   part if it is available.
--

And slightly later:

--
The top-level Subject header field value should be used
   whenever a suitable translation cannot be identified.
--

These requirements should be pulled together and made consistent. In particular, the message needs to have a Subject even if the "selected translations" does not. 

5. There is this recommendation:

--
While RFC 5646 provides a mechanism accommodating
   increasingly fine-grained distinctions, in the interest of maximum
   interoperability, each Content-Language value SHOULD be restricted to
   the largest granularity of language tags; in other words, it is
   RECOMMENDED to specify only a Primary-subtag and NOT to include
   subtags (e.g., for region or dialect) unless the languages might be
   mutually incomprehensible without them.
--

I think this text is potentially harmful, as it might interfere with the needs of senders and clients to distinguish text based on a variety of factors and ignores the fact that some language variation is not mutually comprehensible at the primary language level. 

An implementation restriction to the primary language subtag would be a disadvantage to languages that have important script, region, or other variations such as extlang or variant. BCP 47 has a whole section (http://tools.ietf.org/html/bcp47#section-4.1) about the choice of language tag that basically recommends similar things and I understand why it's important for senders not to add unnecessary subtags. But I think you'd be better off not having quite so much normative language in this specification and a more focused recommendation that does not impinge on the needs of senders and clients to provide adequate language information. 

7. ISO 8859-1? How about using the UTF-8 encoding instead?

Should the document point out here that each example Subject field is in a separate multipart/multilingual block?

---

From: Nik Tomkinson <[email-masked]>
Date: Tue, 27 Oct 2015 09:51:58 +0000

Thanks very much for the extensive comments. I am working through them now
and should have responses for you in the next day or so.

Nik.

On 21 October 2015 at 03:10, Phillips, Addison <[email-masked]> wrote:

> Hi,
>
> As promised, a number of comments on the current draft-tomkinson follow my
> .sig. Comments are labeled by section in the document.
>
> Addison
>
> Addison Phillips
> Abstract: The term "language code" should be replaced with the term
> "language tag". It's helpful to be consistent to avoid confusing with ISO
> standard elements (which are generally referred to as 'codes') and to adopt
> the same terminology as BCP 47.
>
> 3.1 Health warning: the explanatory section may need to use different
> scripts to handle all of the languages. This suggests that the explanatory
> block should use the UTF-8 encoding. A health warning about character
> encodings and the recommendation for using UTF-8 would be useful if
> included here?
>
> 3.1 "language (or locale)": the term locale is unnecessary here, unless
> you want to explain that the language might be derived from the end-user's
> locale. Should there be a terminology section? W3C is working on a document
> that defines some of these, but it is not mature enough to serve as a
> reference (http://www.w3.org/TR/ltli/). Unicode TR 35 talks about the
> same issue: that languages and locales are fundamentally related and that
> you can, in many cases (such as this one) ignore the locale concept.
>
> 3.2 "language message parts are translations of the same message" This
> presumes that they are translations. They might not be exact translations
> and might not be exactly the same. While the *intention* of the document is
> that they are translations, there is no guarantee of such. Perhaps "...are
> usually translations..."
>
> 3.2 The recommendation to apply a 'Name'  parameter kind of presumes that
> the name is meaningful in ASCII, since this is an header field and QP or
> MIME encoding won't be anything but a bunch of gibberish. Rendering into
> ASCII or into English is not very international friendly and having the
> example be 'english.txt' is ignoring the problem?
>
> 3.3 The unmatched message part seems problematic to me. It requires the
> sender to provide a version of the message (increasing message size--some
> systems have size limits!) that will be shown when language negotiation
> fails, rather than allowing one of the already extant sections to be
> selected as the default. There is already a preamble section for cases that
> multipart/multilingual is not supported. And the unmatched message
> (probably falsely) affects a pose of having "no language". Why not allow
> one of the language bearing sections be tagged as the "default" message? If
> a sender wishes to make the "default" (unmatched) message be language
> neutral somehow, they can tag it using a tag provided for that purpose,
> such as zxx or und.
>
> Alternatively, why not use the Preface section? If the end user wants to
> read the message, presumably they will need to choose one of the languages
> to do it in.
>
> Isn't this the nub of the problem? I know that, for example, Mark Davis
> suggests that some clients may wish to use more complex or helpful matching
> schemes than those defined in BCP 47 (which are pretty simplistic). We
> don't have to forbid something more elegant. But fundamental matching is
> improved if there are some base recommendations.
>
>
> These requirements should be pulled together and made consistent. In
> particular, the message needs to have a Subject even if the "selected
> translations" does not.
>
> I think this text is potentially harmful, as it might interfere with the
> needs of senders and clients to distinguish text based on a variety of
> factors and ignores the fact that some language variation is not mutually
> comprehensible at the primary language level.
>
> An implementation restriction to the primary language subtag would be a
> disadvantage to languages that have important script, region, or other
> variations such as extlang or variant. BCP 47 has a whole section (
> http://tools.ietf.org/html/bcp47#section-4.1) about the choice of
> language tag that basically recommends similar things and I understand why
> it's important for senders not to add unnecessary subtags. But I think
> you'd be better off not having quite so much normative language in this
> specification and a more focused recommendation that does not impinge on
> the needs of senders and clients to provide adequate language information.
>
> _______________________________________________
> SLIM mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/slim
>

-- 

-----------------------------------------------------------------
Multiple Language Content Type Internet Draft:

<http://datatracker.ietf.org/doc/draft-tomkinson-multilangcontent/>
http://datatracker.ietf.org/doc/draft-tomkinson-slim-multilangcontent/

-----------------------------------------------------------------

---

From: Nik Tomkinson <[email-masked]>
Date: Wed, 28 Oct 2015 20:30:09 +0000

My responses are inline below in the [ ]:

Nik.

=====

Abstract: The term "language code" should be replaced with the term
"language tag". It's helpful to be consistent to avoid confusing with
ISO standard elements (which are generally referred to as 'codes') and
to adopt the same terminology as BCP 47.

[Yes, I agree - Updated]

3.1 Health warning: the explanatory section may need to use different
scripts to handle all of the languages. This suggests that the
explanatory block should use the UTF-8 encoding. A health warning
about character encodings and the recommendation for using UTF-8 would
be useful if included here?

[Do you have any suggested wording regarding the 'health warning'?
I'll try to get something together for the requirement and
recommendation of UTF-8.]

3.1 "language (or locale)": the term locale is unnecessary here,
unless you want to explain that the language might be derived from the
end-user's locale. Should there be a terminology section? W3C is
working on a document that defines some of these, but it is not mature
enough to serve as a reference (http://www.w3.org/TR/ltli/). Unicode
TR 35 talks about the same issue: that languages and locales are
fundamentally related and that you can, in many cases (such as this
one) ignore the locale concept.

[Agreed, Locale was a legacy term in this document and has now been removed]

3.2 "language message parts are translations of the same message" This
presumes that they are translations. They might not be exact
translations and might not be exactly the same. While the *intention*
of the document is that they are translations, there is no guarantee
of such. Perhaps "...are usually translations..."

[Good point, updated to "...are typically translations..."]

3.2 The recommendation to apply a 'Name'  parameter kind of presumes
that the name is meaningful in ASCII, since this is an header field
and QP or MIME encoding won't be anything but a bunch of gibberish.
Rendering into ASCII or into English is not very international
friendly and having the example be 'english.txt' is ignoring the
problem?

[There was a small doubt in my mind about exactly this, but I thought
that an acceptable workaround is for the sending email client to use
the closest (or most recognised) ASCII version of the language name in
the absence of a better solution. The issue is that unnamed
attachments are given generated filenames (eg. ATT00001.c ) which make
it hard to tell which is which and therefore difficult for the user to
find the right language. This is only an issue when the message is
rendered in non-conforming legacy email clients. An alternative could
be the IANA language code eg. es.txt, cz.txt, it.txt etc but I don't
think this is better. I am expecting the testing stage to help here.]

3.3 The unmatched message part seems problematic to me. It requires
the sender to provide a version of the message (increasing message
size--some systems have size limits!) that will be shown when language
negotiation fails, rather than allowing one of the already extant
sections to be selected as the default. There is already a preamble
section for cases that multipart/multilingual is not supported. And
the unmatched message (probably falsely) affects a pose of having "no
language". Why not allow one of the language bearing sections be
tagged as the "default" message? If a sender wishes to make the
"default" (unmatched) message be language neutral somehow, they can
tag it using a tag provided for that purpose, such as zxx or und.

[I think there are a couple of points to address here: Firstly the
multilingual preface is not related to language negotiation at all,
this is for legacy MUAs that don't understand multipart/multilingual.
Secondly the 'unmatched message part' is OPTIONAL and should only be
used when the sender wants to provide a language-independent version
(eg. an image) suitable for unmatched users. The first language
message part (the first part with a Language Tag) is considered
'default' by the fact that it is the first. This means that if the
default part is not considered (by the sender) to be generic enough to
be understood by an unmatched language recipient then the language
independent content should be provided. I think that the use of the
different parts is correct but the explanation in the I-D was
insufficient and misleading. I will think of some better wording for
this section and include the zxx language tag in the
'unmatched/language-independent' part.]

Alternatively, why not use the Preface section? If the end user wants
to read the message, presumably they will need to choose one of the
languages to do it in.

[Yes, it has become less straightforward over time but this text
remained intact, I have removed it.]

4. The section says:

But describes that as:

Isn't this the nub of the problem? I know that, for example, Mark
Davis suggests that some clients may wish to use more complex or
helpful matching schemes than those defined in BCP 47 (which are
pretty simplistic). We don't have to forbid something more elegant.
But fundamental matching is improved if there are some base
recommendations.

[Good observation, I will add some text with reference to
https://tools.ietf.org/html/rfc4647]

And slightly later:

[OK, I will add a new section to deal with Subject Selection]

I think this text is potentially harmful, as it might interfere with
the needs of senders and clients to distinguish text based on a
variety of factors and ignores the fact that some language variation
is not mutually comprehensible at the primary language level.

An implementation restriction to the primary language subtag would be
a disadvantage to languages that have important script, region, or
other variations such as extlang or variant. BCP 47 has a whole
section (http://tools.ietf.org/html/bcp47#section-4.1) about the
choice of language tag that basically recommends similar things and I
understand why it's important for senders not to add unnecessary
subtags. But I think you'd be better off not having quite so much
normative language in this specification and a more focused
recommendation that does not impinge on the needs of senders and
clients to provide adequate language information.

[Would you recommend removing this text completely or replacing it
with something less restrictive but more suitable? If so do you (or
does anyone else reading this) have any suggested text?]

[Updated]

Should the document point out here that each example Subject field is
in a separate multipart/multilingual block?

[This should be clear from the new selection of subject section]