Subject: Re: More on mail message header fields
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: Keith Moore <[email-masked]>, [email-masked], Jacob Palme <[email-masked]>
Date: Fri, 30 Apr 2004 12:19:48 -0400
List-Id: <ietf-822.imc.org>

IMHO The Encrypted field never was adequately specified.  If 2026 
criteria had been in effect when rfc 822 was published, 822 would have 
had to been revised to remove Encrypted in order to move it to Draft.

so yes it's obsolete, but not because it was omitted from 2822.  do you 
have a status "fatally flawed"?

On Apr 30, 2004, at 10:08 AM, Graham Klyne wrote:

>
> In response to an off-list comment, I'm making a small revision of the 
> mail message header registry draft [1] to mark header fields defined 
> in RFC822 (as well as RFC2822) as standard rather than just 
> "standards-track".  This raised two questions:
>
> 1.  Are there any other mail header fields that have achieved full 
> standard status?
>
> 2.  RFC2822 dropped the "Encrypted" header field which was defined in 
> RFC822.  I take that to mean it is regarded as obsolete;  would this 
> be correct?  My current proposed entry for this is:
> [[
> <http://id.ninebynine.org/wip/2002/IETF/HeaderField#Encrypted>
>     a hdr:HeaderField ;
>     hdr:fieldName "Encrypted" ;
>     hdr:label "Message encryption information" ;
>     hdr:protocol hdr:Protocol_mail ;
>     hdr:status "obsolete" ;
>     hdr:author hdr:Author_IETF ;
>     hdr:controller hdr:Author_IETF ;
>     hdr:specification hdr:RFC822 ;
>     hdr:comment
>         """Defined by RFC 822, but removed in RFC 2822.
>            Current practice is to use separate encryption, such
>            as S/MIME or OpenPGP, possibly in conjunction with
>            RFC 1847 MIME security multiparts.
>         """ .
> ]]
>
> #g
> --
>
> [1] http://www.ietf.org/internet-drafts/draft-klyne-hdrreg-mail-04.txt
>
>
> ------------
> Graham Klyne
> For email:
> http://www.ninebynine.org/#Contact
>

Date: Fri, 30 Apr 2004 15:08:34 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: More on mail message header fields
Cc: Jacob Palme <[email-masked]>
Sender: [email-masked]

In response to an off-list comment, I'm making a small revision of the mail 
message header registry draft [1] to mark header fields defined in RFC822 
(as well as RFC2822) as standard rather than just "standards-track".  This 
raised two questions:

1.  Are there any other mail header fields that have achieved full standard 
status?

2.  RFC2822 dropped the "Encrypted" header field which was defined in 
RFC822.  I take that to mean it is regarded as obsolete;  would this be 
correct?  My current proposed entry for this is:
[[
<http://id.ninebynine.org/wip/2002/IETF/HeaderField#Encrypted>
     a hdr:HeaderField ;
     hdr:fieldName "Encrypted" ;
     hdr:label "Message encryption information" ;
     hdr:protocol hdr:Protocol_mail ;
     hdr:status "obsolete" ;
     hdr:author hdr:Author_IETF ;
     hdr:controller hdr:Author_IETF ;
     hdr:specification hdr:RFC822 ;
     hdr:comment
         """Defined by RFC 822, but removed in RFC 2822.
            Current practice is to use separate encryption, such
            as S/MIME or OpenPGP, possibly in conjunction with
            RFC 1847 MIME security multiparts.
         """ .
]]

#g
--

[1] http://www.ietf.org/internet-drafts/draft-klyne-hdrreg-mail-04.txt

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Mon, 26 Apr 2004 09:14:37 +0100
To: Frank Ellermann <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re:  draft-klyne-hdrreg-mail-04
Cc: [email-masked]
Sender: [email-masked]

Frank,

Thank you for your comments.   It's not clear from your message if you are 
aware that the registration procedure has been approved for BCP 
publication, and is currently in the RFC editor's queue.  The approved 
Internet Draft is:
   http://www.ietf.org/internet-drafts/draft-klyne-msghdr-registry-07.txt
approved for publication per:
   http://www.ietf.org/mail-archive/ietf-announce-old/Current/msg28827.html

As it happens, I do not propose to make any changes, but I do feel your 
comments at least deserve an explanation:

At 18:14 23/04/04 +0200, Frank Ellermann wrote:
>Hi, that's a very interesting document.  Maybe you could add
>"structured" resp. "unstructured" to the listed mail headers,
>because the MIME-encoding of 8-bit characters in unstructured
>headers is rather simple.

The format of the registry is defined by the registratyion procedure, 
which, as noted above, has been approved as BCP.

The purpose of the registry is to provide a means for designers to find 
header field descriptions, rather than to actually fully describe 
them.  Any additional information aims to support administrative rather 
than technical needs, which are (or should be) addressed by the technical 
specification.

Also, the registry may, in principle, contain field names from other 
protocols for which there is no concept of structured and unstructured.

>The "Status: standards track" for RfC 2822 might be confusing,
>because RfC 822 is a standard.  Date: and From: are not "only"
>on standards track, but required.  At least one of To;, Cc:,
>and Bcc: is also required.  And Message-ID: is now a SHOULD,

My response here is really variation of the above:  the term "standards 
track" indicates that the field name is part of an approved standards-track 
specification, and has no bearing on whether it MUST or SHOULD appear in 
any message, that being part of its technical specification.

>PICS-Label is no internet standard, your list says that this
>is some technical report of the W3C.

Indeed.  The permanent header registry has provision for including header 
fields defined by other bodies:

[[
   o  A Permanent Message Header Field Registry, intended for headers
       defined in IETF standards-track documents, those that have
       achieved a comparable level of community review, or are generally
       recognized to be in widespread use. The assignment policy for such
       registration is "Specification Required", as defined by RFC 2434
       [3], where the specification must be published in an RFC
       (standards-track, experimental, informational or historic), or as
       an "Open Standard" in the sense of RFC 2026, section 7 [1].]]
-- http://www.ietf.org/internet-drafts/draft-klyne-msghdr-registry-07.txt
    Section 2.1

I have been advised that a W3C recommendation counts as an "Open Standard" 
in the sense indicated:
[[
Date: Wed, 20 Feb 2002 11:38:36 -0500 (EST)
From: Scott  Bradner <[email-masked]>
Message-Id: <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: RFC 2026, Open Standard publishers
Sender: [email-masked]
Precedence: bulk
Status:

 > Do W3C count as a publisher of Open Standards for the purposes of section 7
 > of RFC 2026?

yes

Scott
]]
-- a message to the Poisson working group mailing list,
    an archived copy of which I can't easily find.

I trust you find this is responsive to your comments.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Mon, 26 Apr 2004 02:34:48 -0400 (EDT)
From: Sean Donelan <[email-masked]>
To: [email-masked]
Subject: Best practice headers
Sender: [email-masked]

Much of the discussion is about parsing grammer for various headers
seen in the wild.  I'm wondering are there any collections of "perfect"
headers a developer could use as templates for creating new headers?
It would still be liberal accepting other headers, but what should
a conservative sender's headers look like?

Date: Fri, 23 Apr 2004 08:50:34 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: draft-klyne-hdrreg-mail-04
Cc: Jacob Palme <[email-masked]>
Sender: [email-masked]

Further to my previous message [1], to which no major complaints were 
received, and a few minor editorial changes, I believe that:

   http://www.ietf.org/internet-drafts/draft-klyne-hdrreg-mail-04.txt

is now ready for last-call as informational RFC, and initial registration 
of standard Mail and MIME header fields.  I propose to send a formal 
request to IESG in the next couple of days.

An HTML version is available [2].

...

Also, for information, I have been working to help Charles Lindsey prepare 
a similar document for Netnews (RFC1036) header fields [3], the first 
version of which has been published as I-D.

#g
--

[1] http://www.imc.org/ietf-822/mail-archive/msg04094.html

[2] http://www.ninebynine.org/IETF/Messaging/draft-klyne-hdrreg-mail-04.html

[3] http://www.ietf.org/internet-drafts/draft-lindsey-hdrreg-netnews-00.txt
     http://www.ninebynine.org/IETF/Messaging/HdrRegistry/draft-lindsey-hdrreg-news-00.html

and some generated web pages:
     http://www.ninebynine.org/IETF/Messaging/HdrRegistry/lindsey-hdrreg-news.html

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Sat, 17 Apr 2004 09:58:50 -0700 (PDT)
From: [email-masked]
Subject: Re: Mail/MIME permanent header registrations
To: Graham Klyne <[email-masked]>
Cc: [email-masked], Jacob Palme <[email-masked]>
Sender: [email-masked]

> Following approval of the message header registry [1][2], Jacob and I have
> been working on a document to contain initial registrations of
> standards-track email and MIME headers [3].

> Modulo some small editorial fixes, I believe this is ready to request a
> last-call (for publication as informational RFC, and subsequent
> registration of the entries contained therein).

I agree.

> Possibly contentious items are:
>    Accept-language[4]

What's there seems reasonable enough to me.

>    Encoding (experimental) [5]

The encoding field is an interesting case. On the one hand, it is specified in
an experimental RFC so it does belong in the registry. On the other hand,
actual use of the field varies widely from what's in the RFC. In particular,

(1) Some agents use byte counts in the field, not line counts. (Needless to
    say, this doesn't interoperate well.)

(2) Some implementations, in an effort to provide better name and type
    information, embed filenames in the field.

(3) Support for nesting as well as support for many of the encodings
    specified in RFC 1505 is rare.

And then there's the pesky problem that this whole scheme depends on line or
byte counts, which tend to get disrupted as messages pass through the transport
infrastructure. (Used to be that line folding, and tab expansion caused the
disruption. These days its more likely to be insertion of disclaimers or
charset conversions that cause trouble.)

The current registration text makes it clear that this field has interop
problems, and I personally think that's sufficient.

				Ned

Date: Fri, 16 Apr 2004 11:01:03 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Mail/MIME permanent header registrations
Cc: Jacob Palme <[email-masked]>
Sender: [email-masked]

Following approval of the message header registry [1][2], Jacob and I have 
been working on a document to contain initial registrations of 
standards-track email and MIME headers [3].

Modulo some small editorial fixes, I believe this is ready to request a 
last-call (for publication as informational RFC, and subsequent 
registration of the entries contained therein).

Possibly contentious items are:
   Accept-language[4]
   Encoding (experimental) [5]

(I just noticed a typo, content-disposition is listed as 
experimental.  Fixed in my working database of MIME header fields.)

#g
--

[1] http://www1.ietf.org/mail-archive/ietf-announce-old/Current/msg28827.html

[2] http://www.ietf.org/internet-drafts/draft-klyne-msghdr-registry-07.txt

[3] http://www.ietf.org/internet-drafts/draft-klyne-hdrreg-mail-03.txt
HTML version:
http://www.ninebynine.org/IETF/Messaging/draft-klyne-hdrreg-mail-03.html

[4] 
http://www.ninebynine.org/IETF/Messaging/draft-klyne-hdrreg-mail-03.html#hdr.Accept-Language

[5] 
http://www.ninebynine.org/IETF/Messaging/draft-klyne-hdrreg-mail-03.html#hdr.Encoding

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Tue, 13 Apr 2004 17:58:48 +0100
To: "Charles Lindsey" <[email-masked]>, [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Sender: [email-masked]

At 10:51 08/04/04 +0000, Charles Lindsey wrote:

>In <[email-masked]> Graham Klyne 
><[email-masked]> writes:
>
> >This header field [[[Accept-language]]] is commonly used in email, but some
> >problems have been noted, including but not limited to:  determination of
> >the email address to which it refers;  use of different field names names
> >by some mail agents for the same purpose;  lack of consistent recognition
> >and use by receiving agents;  cost and lack of effective
> >internationalization of email responses;  problems with interpretation of
> >language subtags;  problems determining what character set encoding should
> >be used (UTF-8 is not universally supported).
> >]]
>
>I would remove the mention of different field names.

Done.

>The main purpose of
>the Registry AISI is to discourage unnecessary proliferation of lots of
>headers all doing the same thing. So whichever the registry endorses, that
>effectively causes the others to be deprecated.

>As to the problems in determining the address to which it refers (I
>presume that particular issue does not arise in its HTTP usage), either it
>is too severe a problem, in which case the proper procedure is for someone
>to write a standards-track proposal, possibly introducing some additional
>syntax; or else it is OK as it stands, in which case it should be
>registered. If someone does produce such a draft, then of course it could be
>registered in the provisional registry.

HTTP usage is per-session, so the problem doesn't arise.  But the suggested 
text here is specifically with regard to email use.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

To: [email-masked]
Xref: clerew local.mime:3133
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Date: Thu, 8 Apr 2004 10:51:32 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Graham Klyne <[email-masked]> writes:

>This header field [[[Accept-language]]] is commonly used in email, but some 
>problems have been noted, including but not limited to:  determination of 
>the email address to which it refers;  use of different field names names 
>by some mail agents for the same purpose;  lack of consistent recognition 
>and use by receiving agents;  cost and lack of effective 
>internationalization of email responses;  problems with interpretation of 
>language subtags;  problems determining what character set encoding should 
>be used (UTF-8 is not universally supported).
>]]

I would remove the mention of different field names. The main purpose of
the Registry AISI is to discourage unnecessary proliferation of lots of
headers all doing the same thing. So whichever the registry endorses, that
effectively causes the others to be deprecated.

As to the problems in determining the address to which it refers (I
presume that particular issue does not arise in its HTTP usage), either it
is too severe a problem, in which case the proper procedure is for someone
to write a standards-track proposal, possibly introducing some additional
syntax; or else it is OK as it stands, in which case it should be
registered. If someone does produce such a draft, then of course it could be
registered in the provisional registry.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Thu, 8 Apr 2004 11:13:09 +0200
To: IETF mailing list on MIME and e-mail  <[email-masked]>
From: Jacob Palme <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Sender: [email-masked]

At 07.43 -0400 04-04-07, Keith Moore wrote:
>"Accept-language wasn't designed for email, but has been found to be useful
>as input into the generation of automatic replies.  It should apply to the
>SMTP MAIL FROM or Return-Path address rather than some other address."

That sounds reasonable. Because Accept-Language will mostly
be used to control automatically generated responses, and
those should not be sent to all the members of a mailing
list. For one-to-one messages, the MAIL FROM and the From
mailbox is usually identical.
-- 
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/jpalme/

Date: Wed, 07 Apr 2004 13:49:08 +0100
To: "Charles Lindsey" <[email-masked]>, [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Sender: [email-masked]

At 09:40 07/04/04 +0000, Charles Lindsey wrote:
>This raises a more general problem regarding headers that migrate from one
>medium to another.

I see no great difficulty here.  Header fields can be listed under multiple 
protocols so that these subtleties can be captured faithfully.  The 
registration document also notes that an entry may refer to multiple 
specifications in cases like this:

[[
    In some cases, the same field name may be specified differently (by
    different documents) for use with different application protocols;
    e.g.  The Date: header field used with HTTP has a different syntax
    than the Date: used with Internet mail. In other cases, a field name
    may have a common specification across multiple protocols (ignoring
    protocol-specific lexical and character set conventions);  e.g. this
    is generally the case for MIME header fields with names of the form
    'Content-*'.

    Thus, we need to accommodate application-specific fields, while
    wishing to recognize and promote (where appropriate) commonality of
    other fields across multiple applications. Common repositories are
    used for all applications, and each registered header field specifies
    the application protocol for which the corresponding definition
    applies.  A given field name may have multiple registry entries for
    different protocols; in the Permanent Message Header Field registry,
    a given header field name may be registered only once for any given
    protocol. (In some cases, the registration may reference several
    defining documents.)
]]
-- http://www.ietf.org/internet-drafts/draft-klyne-msghdr-registry-07.txt
    section 2.2.1

In the case of Accept-Language, it is defined generically in RFC 3282, but 
it wasn't clear to me that it is defined for use specifically with email, 
hence my question here.  It is defined for use with HTTP, and I had 
originally anticipated that it would be included in the registry for use 
with HTTP.

In summary, the registry gives us an opportunity to record the use of 
common headers with several protocols with which they are applicable.

As for the designated expert mechanism:  you are correct.  But that should 
not be seen as displacing discussion on the mailing lists appropriate for 
the protocol(s) concerned, which is why I raised the matter of 
Accept-Language *for email* on this list.

For netnews-defined header fields that are also used in email, I think it 
is appropriate for the registration *as an email header field* to be 
discussed by the email community.  Personally, in this case, I think it 
appropriate that the mail use registration be contained in a separate 
document for which review by the email community is sought, said document 
referencing Usefor for the technical specification.

#g
--

PS:  while talking about the registry in general, I have recently revived 
some report generation software I adapted for generating 
registration-template XML2RFC source code and browseable HTML pages from 
header field descriptions in RDF/N3.  The most recent version of the 
software is not yet documented, but the Python source code is on the web 
[1].  An earlier version of the software is described [2], et link.  I have 
recently implemented a program to compile a more friendly form of "report 
definition" (the current definition for header field registry generation is 
[3]) into RDF/N3 (e.g. see [4]) for interpretation by the report 
generator;  the Haskell [8] source code of this compiler is at 
[5][6][7].  More information about RDF/N3 (aka Notation3) can be found at 
[9].  All this software is work-in-progress.

[1] http://www.ninebynine.org/Software/PythonN3/
     (The main module is N3GenReport.py)

[2] http://www.ninebynine.org/Software/Intro.html#RDFReportGenerator
     http://www.ninebynine.org/RDFNotes/RDFForLittleLanguages.htm

[3] Source of header registry report definition:
     http://www.ninebynine.org/Software/HdrRegistry/GenHeaderRegistry.rep
     Directory with related data:
     http://www.ninebynine.org/Software/HdrRegistry/

[4] Compiled to Notation3 (not for human consumption [dogfood only?]):
     http://www.ninebynine.org/Software/HdrRegistry/GenHeaderRegistry.n3

[5] http://www.ninebynine.org/Software/CompileRDF/
     (Main program is RepToRDF.hs)
[6] http://www.ninebynine.org/Software/HaskellRDF/
[7] http://www.ninebynine.org/Software/HaskellUtils/

[8] Info about Haskell:
     http://www.haskell.org/

[9] More info about RDF/Notation3:
     http://www.w3.org/DesignIssues/Notation3.html
     http://www.w3.org/2000/10/swap/Primer.html
     http://infomesh.net/2002/notation3/
     See also:
     http://www.w3.org/2000/10/swap/doc/cwm.html

At 09:40 07/04/04 +0000, Charles Lindsey wrote:

>In <[email-masked]> Graham Klyne 
><[email-masked]> writes:
>
> >Notwithstanding the operational problems you mention, this suggests to me
> >that Accept-language should be included in the initial (permanent) registry
> >of email message headers, possibly carrying a warning about the operational
> >issues you note?
>
>This raises a more general problem regarding headers that migrate from one
>medium to another.
>
>AIUI, Accept-Language is already an official HTTP header. Therefore it
>will appear in the registry already, which itself should act as a strong
>hint that it is the preferred header-name for a similar feature in other
>media.
>
>But if you want to go further and list it under those other media, then
>you are required to refer to a defining document, and in such a case the
>defining document will say nothing about those other media. So is that
>allowed? I think the answer has to be that you go through the mechanism
>defined in Graham's RFC-to-be which is to ask the IESG (via their
>"designated expert"), following discussion in the "desginated email
>discussion list". Well, none of that machinery is set up yet, so I suppose
>discussing it on this list is the next best thing (BTW, could this list be
>designated for that purpose? I doubt the suggested [email-masked] will
>receive many subscriptions.)
>
>A similar problem arises with the User-Agent header. Currently, this is
>defined for use in HTTP, but is quite widely used in email and news. In
>this case, the USEFOR WG has taken it on board and is defining it as an
>official News header, with a throwaway remark that "It is also intended
>that this header be suitable for use in Email".
>
>Now I have just been writing the registration templates for the USEFOR
>IANA Considerations section, so naturally the User-Agent header appears in
>it. But also, on the stregth of that throwaway remark, I have tentatively
>included is for both netnews and email. Opinions welcomed on that.
>
>--
>Charles H. Lindsey ---------At Home, doing my own 
>thing------------------------
>Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: 
>http://www.cs.man.ac.uk/~chl
>Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, 
>U.K.
>PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 
>AB A5

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Wed, 7 Apr 2004 07:50:49 -0400
From: Keith Moore <[email-masked]>
To: "Charles Lindsey" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Is Accept-language an email header field?
Sender: [email-masked]

> AIUI, Accept-Language is already an official HTTP header. Therefore it
> will appear in the registry already, which itself should act as a strong
> hint that it is the preferred header-name for a similar feature in other
> media.

Why can't people understand that HTTP and email are different protocols
that serve very different purposes and operate very differently?

It is NOT desirable to encourage people to assume that it's okay to 
adopt fields from one protocol to another.  Even if the fields keep the
same syntax, they end up having different uses and different meanings,
ambiguities can be introduced, etc. 

-- 
Regime Change 2004 - better late than never. 

Date: Wed, 7 Apr 2004 07:43:18 -0400
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Is Accept-language an email header field?
Sender: [email-masked]

Mumble.  I've always been opposed to the idea of a registry that would 
endorse the decisions that the market makes, because while the market 
might serve as a rough indication of what features are needed, the market
sucks at protocol design.  And yet I do to some degree accept the argument
that once the market has chosen a particular feature, it's very difficult to
get the market to  adopt something that is significantly different.
Another way of saying this is that if we fail to produce a good design 
sufficiently early, the market will produce a bad design in our place.

Still, I'd rather have a published statement that says 

"Accept-language wasn't designed for email, but has been found to be useful 
as input into the generation of automatic replies.  It should apply to the 
SMTP MAIL FROM or Return-Path address rather than some other address." 

than a published statement that, by failing to specify usage, encourages 
even more variant use than we're currently seeing.  Because even if we 
claim that the registry is just documenting current practice, it's going
to be cited as an authority for desirable practice.

Keith

> Personally, I'm neutral about its inclusion.  So far I have a very small 
> sample of opinions that are 2:1 in favour of inclusion, but clearly there 
> are some concerns.  To the extent that the registry is intended to document 
> actual behaviour, not define new behaviour, I suggest inclusion as 
> "informational" with a comment along the following lines:
> 
> [[
> Indicates a language that the message sender requests be used for responses.
> 
> This header field [[[Accept-language]]] is commonly used in email, but some 
> problems have been noted, including but not limited to:  determination of 
> the email address to which it refers;  use of different field names names 
> by some mail agents for the same purpose;  lack of consistent recognition 
> and use by receiving agents;  cost and lack of effective 
> internationalization of email responses;  problems with interpretation of 
> language subtags;  problems determining what character set encoding should 
> be used (UTF-8 is not universally supported).
> ]]
> 
> But if it proves difficult to quickly find an acceptable form of words to 
> accompany its inclusion, I propose to withdraw it from an *initial* set of 
> header field registrations, so as to not hold up the inclusion of standard 
> header fields for which there should be no dissent.

Date: Wed, 07 Apr 2004 11:53:09 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

Re:  Accept-language in permanent email header field registry?

Personally, I'm neutral about its inclusion.  So far I have a very small 
sample of opinions that are 2:1 in favour of inclusion, but clearly there 
are some concerns.  To the extent that the registry is intended to document 
actual behaviour, not define new behaviour, I suggest inclusion as 
"informational" with a comment along the following lines:

[[
Indicates a language that the message sender requests be used for responses.

This header field [[[Accept-language]]] is commonly used in email, but some 
problems have been noted, including but not limited to:  determination of 
the email address to which it refers;  use of different field names names 
by some mail agents for the same purpose;  lack of consistent recognition 
and use by receiving agents;  cost and lack of effective 
internationalization of email responses;  problems with interpretation of 
language subtags;  problems determining what character set encoding should 
be used (UTF-8 is not universally supported).
]]

But if it proves difficult to quickly find an acceptable form of words to 
accompany its inclusion, I propose to withdraw it from an *initial* set of 
header field registrations, so as to not hold up the inclusion of standard 
header fields for which there should be no dissent.

#g
--

PS:
Current published I-D for initial registrations is:
http://www.ietf.org/internet-drafts/draft-klyne-hdrreg-mail-02.txt

Current unpublished work-in-progress is at:
http://www.ninebynine.org/IETF/Messaging/draft-klyne-hdrreg-mail-03.html
http://www.ninebynine.org/IETF/Messaging/draft-klyne-hdrreg-mail-03.txt
I hope to publish a version ready for last call review in the next few 
days, pending a decision on this question.

At 14:46 06/04/04 -0700, [email-masked] wrote:
>> > Let me start by pointing out that the accept-language is currently in 
>> fairly
>> > widespread use in email. A number of very popular clients generate these
>> > headers and a fair number of automatic response agents honor them. Strong
>> > customer demand led us to support them in the autoresponses our product
>> > generates a few years ago, which means I have a fair amount of
>> > experience  with
>> > them and the problems they do and do not have.
>
>>Notwithstanding the operational problems you mention, this suggests to me
>>that Accept-language should be included in the initial (permanent) registry
>>of email message headers, possibly carrying a warning about the operational
>>issues you note?
>
>To the extent that having a single, standardized header for this would 
>create a
>trend towards using a single field name and would prevent additional field
>names from being used for this purpose, I support having this in the registry.
>
>OTOH, I have more important things to do than engage in a protracted wrangle
>about how this field binds to addresses or anything similar. The market 
>appears
>to have decided on a "good enough" solution to this problem; the IETF can
>either back this solution with a particular field name or not. It isn't going
>to be able to effect more change than that.
>
>                                 Ned

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

To: [email-masked]
Xref: clerew local.mime:3128
Newsgroups: local.mime
Path: clerew!chl
From: "Charles Lindsey" <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Date: Wed, 7 Apr 2004 09:40:41 GMT
Lines: 47
Sender: [email-masked]

In <[email-masked]> Graham Klyne <[email-masked]> writes:

>Notwithstanding the operational problems you mention, this suggests to me 
>that Accept-language should be included in the initial (permanent) registry 
>of email message headers, possibly carrying a warning about the operational 
>issues you note?

This raises a more general problem regarding headers that migrate from one
medium to another.

AIUI, Accept-Language is already an official HTTP header. Therefore it
will appear in the registry already, which itself should act as a strong
hint that it is the preferred header-name for a similar feature in other
media.

But if you want to go further and list it under those other media, then
you are required to refer to a defining document, and in such a case the
defining document will say nothing about those other media. So is that
allowed? I think the answer has to be that you go through the mechanism
defined in Graham's RFC-to-be which is to ask the IESG (via their
"designated expert"), following discussion in the "desginated email
discussion list". Well, none of that machinery is set up yet, so I suppose
discussing it on this list is the next best thing (BTW, could this list be
designated for that purpose? I doubt the suggested [email-masked] will
receive many subscriptions.)

A similar problem arises with the User-Agent header. Currently, this is
defined for use in HTTP, but is quite widely used in email and news. In
this case, the USEFOR WG has taken it on board and is defining it as an
official News header, with a throwaway remark that "It is also intended
that this header be suitable for use in Email".

Now I have just been writing the registration templates for the USEFOR
IANA Considerations section, so naturally the User-Agent header appears in
it. But also, on the stregth of that throwaway remark, I have tentatively
included is for both netnews and email. Opinions welcomed on that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

Date: Tue, 06 Apr 2004 14:46:23 -0700 (PDT)
From: [email-masked]
Subject: Re: Is Accept-language an email header field?
To: Graham Klyne <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > Let me start by pointing out that the accept-language is currently in fairly
> > widespread use in email. A number of very popular clients generate these
> > headers and a fair number of automatic response agents honor them. Strong
> > customer demand led us to support them in the autoresponses our product
> > generates a few years ago, which means I have a fair amount of
> > experience  with
> > them and the problems they do and do not have.

> Notwithstanding the operational problems you mention, this suggests to me
> that Accept-language should be included in the initial (permanent) registry
> of email message headers, possibly carrying a warning about the operational
> issues you note?

To the extent that having a single, standardized header for this would create a
trend towards using a single field name and would prevent additional field
names from being used for this purpose, I support having this in the registry.

OTOH, I have more important things to do than engage in a protracted wrangle
about how this field binds to addresses or anything similar. The market appears
to have decided on a "good enough" solution to this problem; the IETF can
either back this solution with a particular field name or not. It isn't going
to be able to effect more change than that.

				Ned

Date: Tue, 06 Apr 2004 21:03:53 +0100
To: [email-masked], Keith Moore <[email-masked]>
From: Graham Klyne <[email-masked]>
Subject: Re: Is Accept-language an email header field?
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

At 09:49 06/04/04 -0700, [email-masked] wrote:
>Let me start by pointing out that the accept-language is currently in fairly
>widespread use in email. A number of very popular clients generate these
>headers and a fair number of automatic response agents honor them. Strong
>customer demand led us to support them in the autoresponses our product
>generates a few years ago, which means I have a fair amount of 
>experience  with
>them and the problems they do and do not have.

Notwithstanding the operational problems you mention, this suggests to me 
that Accept-language should be included in the initial (permanent) registry 
of email message headers, possibly carrying a warning about the operational 
issues you note?

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

Date: Tue, 06 Apr 2004 09:49:48 -0700 (PDT)
From: [email-masked]
Subject: Re: Is Accept-language an email header field?
To: Keith Moore <[email-masked]>
Cc: Graham Klyne <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> I don't think it makes sense to use the accept-language field, as it's
> currently defined, with email.   partially this is because there's no clear
> indication as to whose preferences are being described, and partially because
> a reply to a message (the most likely use of accept-language) might go to the
> author, the reply-to field, some subset of to and cc recipients, etc..

> more generally, I don't think it makes sense to try to add descriptive
> information about an address in a header field by using other fields
> that don't explicitly reference that address.  (note that these addresses
> are sometimes changed in transit while leaving the other fields intact)

Let me start by pointing out that the accept-language is currently in fairly
widespread use in email. A number of very popular clients generate these
headers and a fair number of automatic response agents honor them. Strong
customer demand led us to support them in the autoresponses our product
generates a few years ago, which means I have a fair amount of experience  with
them and the problems they do and do not have.

As you might expect, I have observed a number of operational problems with
this header:

(1) Far and away the most common problem has been the absence of a single,
    recommended field being defined for this purpose. As is so often the
    case, this has led to a number of different fields being used, some
    supported by some agents and others not. The fields I've observed
    operationally are: X-Accept-language, Accept-Language and
    Preferred-Language. I've observed X-Accept-Language to be the
    most popular, but that's from a small and probably unrepresentative
    sample.

(2) Lack of use of the field in all cases. Although several popular clients
    generate the field, other popular clients do not. The result is that
    sometimes you get nicely internationalized responses and other times
    you don't. This violates the least astonishment principle, and leads
    to frenzied attempts to guess the appropriate language to use from other
    data, e.g. the domain of some address ends in .jp, use Japanese. These
    other mechanisms don't work nearly as well, of course, and have been
    known to cause more problems than they solve.

(3) There are a lot of languages out there, and internationalization
    is expensive. Implementations have to decide what languages they want
    to support - there's no way to support them all. Additionally, some
    products are still caught up in the old localization mindset, where
    you have to take extra steps to install "language support". This all
    leads to sporadic support for properly internationalized responses,
    which again violates the least astonishment principle.

(4) Support for language subtags is hard to get right. Cases exist
    where it is better to fall back to English than to use the wrong
    dialect. 

(5) Charsets. Figuring out the right charset to use can be a problem since
    support for utf-8 is nowhere near universal and there can be disparate
    user communities that use the same language but written with different
    charsets.

Notable by its absence from my list is Keith's concern that the lack of a
binding of this information to a specific address or addresses in the header
will lead the wrong language being used in some cases. Simply put, I have never
encountered a case where this has been a problem. The reality seems to be that
language choice is a fairly coarse thing, and that if the originator of the
message expresses a preference in the header of the message, it seems to work
at least as well as using some sort of implementation default (usually
English). Alternately, since I'm dealing with automatically generated responses
here, I suppose you could say I'm mostly using a binding to the MAIL FROM
address and finding that it works pretty well.

Another potential issue I haven't found to be a problem in practice is the
syntax of the field itself. The HTTP syntax for the field is rather complex and
allows for each value to have an attached weight. Weights make sense when there
are other factors to consider when deciding what document to return, but
they're just unnecessary complexity for email and I doubt that most agents that
look at these fields handle them properly. But this has been a nonissue in
practice - the only fields with weights in them I've ever seen have been ones
I've generated myself.

In summary, I think this is a case where we've let the best be the enemy of the
good in a fairly major way. Is there a potential problem with multiplicity of
addresses in the header and with there not being a way to attach language
preference information to each address? Sure, but in practice the biggest
problem with having a single header for this information has been the lack of
single, standardized field, which we could have fixed easily had we been able
to get past the binding issue.

				Ned

Date: Tue, 6 Apr 2004 12:27:52 -0400
From: Keith Moore <[email-masked]>
To: Graham Klyne <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Is Accept-language an email header field?
Sender: [email-masked]

I don't think it makes sense to use the accept-language field, as it's 
currently defined, with email.   partially this is because there's no clear 
indication as to whose preferences are being described, and partially because 
a reply to a message (the most likely use of accept-language) might go to the 
author, the reply-to field, some subset of to and cc recipients, etc..

more generally, I don't think it makes sense to try to add descriptive
information about an address in a header field by using other fields 
that don't explicitly reference that address.  (note that these addresses
are sometimes changed in transit while leaving the other fields intact)

Keith

> Following its definition in RFC 3282, should the Accept-language header 
> field be considered to be an email header field, in the sense of being a 
> recognized extension of RFC 2822.  (If so, what does it mean in the context 
> of an RFC2822  mail message?)

--
Power corrupts; Powerpoint corrupts absolutely. 	- Vint Cerf

Date: Tue, 06 Apr 2004 16:42:42 +0100
To: [email-masked]
From: Graham Klyne <[email-masked]>
Subject: Is Accept-language an email header field?
Cc: [email-masked]
Sender: [email-masked]

Following its definition in RFC 3282, should the Accept-language header 
field be considered to be an email header field, in the sense of being a 
recognized extension of RFC 2822.  (If so, what does it mean in the context 
of an RFC2822  mail message?)

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact