Subject: [EAI] Proposed resolution of List-* and downgrade- Issues

From: Randall Gellens <[email-masked]>
Date: Mon, 28 Jul 2008 10:34:57 -0700

----------------------------------------------
Proposed resolution of "List-*" headers:

EAI to define new versions of List-* headers which can have unencoded 
UTF8 values. This is just a trivial revision of the current forms. 
Like the current forms, each header can have multiple IRI, URL, URL, 
etc.  Note that this is orthogonal to new "mailto" -- when mailto-bis 
is ready, it just drops in as any other scheme such as HTTP.

This is accomplished by creating a new WG document for 2369-bis to 
extend List-* headers to "permit UTF8 whenever UTF8 is otherwise 
permitted in header fields".  There isn't a need to do more than 
adding this one sentence to the existing 2369, I think.  So hopefully 
it will be very simple to produce.  We need to be very clear that, as 
2369-bis is being done by EAI, its scope is strictly limited to 
permitting UTF8 and not to any other changes that anyone might desire 
in 2369.

A preference is for List-ID to be considered special and remain ASCII 
(same as Message-ID). There is an open question on internationalized 
domain names: since these are used in Sieve scripts, users need to be 
able to type them in.

Note that allowing UTF8 in List-* headers is equivalent to allowing 
UTF8 in address headers.  Presumably, only EAI-aware clients will see 
either.

----------------------------------------------
New text in -downgrade:

List-* headers which contain native UTF8 should be copied into
Downgrade-List-* and the UTF8 IRI should be dropped (alternate:
encoded per IRI).

Add informative reference to new 2369-bis draft along the lines of 
"Since [2369-bis] permits UTF8 in List-* headers, which are vital to 
user's experience of mailing lists, this document needs to specify 
how such headers are to be downgraded and remain valid."

----------------------------------------------
New text in -mailinglist:

Where available, mailing lists SHOULD include an ASCII form of List-* 
headers as well as a UTF8 form. Because these headers are vital to 
good operations and use of mailing lists, special rules are defined 
in [downgrade] so that such headers are not stripped during 
downgrade, but are preserved with working values whenever possible.

----------------------------------------------
New text in -framework-bis:

New versions of List-* headers are also defined, which permit native UTF8.

----------------------------------------------
-utf8headers:

My suggestion is to create an I-D for utf8headers-bis which adds
List-* headers, and keep it around as an I-D for when we go to
standards track.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Nobody can be exactly like me. Even I have trouble doing it.
                                        --Tallulah Bankhead
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 28 Jul 2008 20:43:09 +0200

Randall Gellens wrote:
 
> We need to be very clear that, as 2369-bis is being done by EAI,
> its scope is strictly limited to permitting UTF8 and not to any
> other changes that anyone might desire in 2369.

Why not promote 2369 to DS, upgrade its 1738 reference to STD 66,
and allow IRIs and UTF-8 in the context of a message/global in a
very short i18n consideratins section ?

Apparently your proposal covers 1.5 lines of my 2.5 lines.  There
are no errata, apparently.  It *might* be really simple, because
2369bis is not about mailto-bis (etc.) details.  

> There is an open question on internationalized domain names:
> since these are used in Sieve scripts, users need to be able to
> type them in.

Sorry, I don't get that.  A-labels are ASCII, and U-labels can be
UTF-8, where's the problem wrt IDN ?

> -downgrade:

[I'll look at -downgrade in an IETF Last Call, here makes no sense]
 
> Where available, mailing lists SHOULD include an ASCII form of
> List-* headers as well as a UTF8 form.

I'd prefer MUST.  If a user loses EAI connectivity in some odd way
"help" and "unsubscribe" MUST still work.

 Frank

---

From: Randall Gellens <[email-masked]>
Date: Tue, 29 Jul 2008 01:34:10 -0700

>  "help" and "unsubscribe" MUST still work.

I think we decided a long time ago not to require mailing lists to 
have ASCII addresses for list functions, hence the "where available" 
in the suggested text.  We could say "MUST", but a what that means 
is, "if you have an ASCII address for this list function, you MUST 
include it," which seems strange given that they don't need to have 
an ASCII address at all.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
There will be no lasting peace on our earth as long as politicians and
dictators continue to lead masses of people who are endemically
neurotic and sexually sick.                            --Wilhelm Reich
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 29 Jul 2008 10:41:50 +0100

On Mon, 28 Jul 2008 18:34:57 +0100, Randall Gellens <[email-masked]>  
wrote:

> UTF8 values. This is just a trivial revision of the current forms. Like  
> the current forms, each header can have multiple IRI, URL, URL, etc.   
> Note that this is orthogonal to new "mailto" -- when mailto-bis is  
> ready, it just drops in as any other scheme such as HTTP.

But is that not equivalent to just saying that IRIs are to be permitted  
(within the UTF8SMTP universe) in places where URIs are currently allowed?

And if it is not equivalent, then why cannot it be made so?

> (same as Message-ID).

Agreed.

> ... There is an open question on internationalized domain names: since  
> these are used in Sieve scripts, users need to be able to type them in.

Ah! So if we have
    List-Post: <mailto:[email-masked]>
(which is presumably an IRI), then is the corresponding URI just a  
%-encoding of "mañana", or is it a punycode thereof? Actually, I think we  
have to wait for mailto-bis to answer this question, because mailto-bis  
might well allow something of the form
    List-Post: <mailto:[email-masked]<[email-masked]>>
(i.e. and Alt address).
>
> Note that allowing UTF8 in List-* headers is equivalent to allowing UTF8  
> in address headers.  Presumably, only EAI-aware clients will see either.
>

No! Don't drop it. Downgrade it to a URI, and let mailto-bis sort out the  
result.
> user's experience of mailing lists, this document needs to specify how  
> such headers are to be downgraded and remain valid."
>
> headers as well as a UTF8 form. Because these headers are vital to good  
> operations and use of mailing lists, special rules are defined in  
> [downgrade] so that such headers are not stripped during downgrade, but  
> are preserved with working values whenever possible.

Having invented alt-addresses for use elsewhere within EAI, let us stick  
with them here. The last thing we want is two mechanisms for achieving the  
same result.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131                       
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: Randall Gellens <[email-masked]>
Date: Tue, 29 Jul 2008 05:59:36 -0700

At 10:41 AM +0100 7/29/08, Charles Lindsey wrote:

>  On Mon, 28 Jul 2008 18:34:57 +0100, Randall 
> Gellens <[email-masked]> wrote:
>
>>  EAI to define new versions of List-* headers 
>> which can have unencoded UTF8 values. This is 
>> just a trivial revision of the current forms. 
>> Like the current forms, each header can have 
>> multiple IRI, URL, URL, etc.  
>>  Note that this is orthogonal to new "mailto" 
>> -- when mailto-bis is ready, it just drops in 
>> as any other scheme such as HTTP.
>
>  But is that not equivalent to just saying that 
> IRIs are to be permitted (within the UTF8SMTP 
> universe) in places where URIs are currently 
> allowed?
>

I'm no expert in IRIs, but my understanding is 
that an IRI can be represented either in UTF8 or 
in percent-encoded form.  I specifically said 
"unencoded UTF8 values" because I think this is 
the key point.

>
>>  A preference is for List-ID to be considered 
>> special and remain ASCII (same as Message-ID).
>
>  Agreed.

Yes, but, there is the sticky point in the next paragraph....

>
>>  ... There is an open question on 
>> internationalized domain names: since these 
>> are used in Sieve scripts, users need to be 
>> able to type them in.
>
>  Ah! So if we have
>  (which is presumably an IRI), then is the 
> corresponding URI just a %-encoding of 
> "mañana", or is it a punycode thereof? 
> Actually, I think we have to wait for 
> mailto-bis to answer this question, because 
> mailto-bis might well allow something of the 
> form

The point about internationalized domain names 
and Sieve scripts is specifically about List-ID. 
Since this is the unique identifier for a list 
and hence used in Sieve scripts, it should be 
permitted to be something meaningful that users 
can type in.  So, the example might be

	List-ID: list.mañana.com

Which would be meaningful, and presumably easy 
for users to remember and to type in.  Forcing 
this List-ID to be in punycode turns it from 
something meaningful and easy to type into 
gobbledegook.  However, permitting UTF8 means 
that during downconversion it will be encoded, 
and then there is the question as to if anything 
will recognize an encoded List-ID as being 
identical to the unencoded version.

>>
>>  Note that allowing UTF8 in List-* headers is 
>> equivalent to allowing UTF8 in address 
>> headers.  Presumably, only EAI-aware clients 
>> will see either.
>>
>  No! Don't drop it. Downgrade it to a URI, and 
> let mailto-bis sort out the result.

mailto is just one possible scheme that can be 
used in a List-* header.  I'm OK with encoding 
it, if the encoded result is always as valid and 
usable as the unencoded form.

>>
>>  Add informative reference to new 2369-bis 
>> draft along the lines of "Since [2369-bis] 
>> permits UTF8 in List-* headers, which are 
>> vital to user's experience of mailing lists, 
>> this document needs to specify how such 
>> headers are to be downgraded and remain valid."
>>
>>  Where available, mailing lists SHOULD include 
>> an ASCII form of List-* headers as well as a 
>> UTF8 form. Because these headers are vital to 
>> good operations and use of mailing lists, 
>> special rules are defined in [downgrade] so 
>> that such headers are not stripped during 
>  Having invented alt-addresses for use elsewhere 
> within EAI, let us stick with them here. The 
> last thing we want is two mechanisms for 
> achieving the same result.

I don't follow.  What are the two versions?  The 
point is that an EAI-aware list can be created 
that operates behind a UTF8 email address which 
has no ASCII version.  That's perfectly OK, if 
people want to do this.  Presumably, the list is 
only for people who have EAI-aware clients.

-------------- Randomly-selected tag: ---------------
Never doubt that a small group of thoughtful, committed citizens can
change the world.  Indeed, it's the only thing that ever has.
                                                     --Margaret Meed
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: Martin Duerst <[email-masked]>
Date: Wed, 30 Jul 2008 15:18:15 +0900

For the Archived-At header, defined in http://www.ietf.org/rfc/rfc5064.txt,
which is related to list headers but somewhat different because it
indicates per-message archival location, I took a forward-looking
approach and included this:

2.4.  Syntax Extension for Internationalized Message Headers

   There are some efforts to allow non-ASCII text directly in message
   header field bodies.  In such contexts, the URI non-terminal in the
   syntax defined in Section 2.1 is to be replaced by an
   Internationalized Resource Identifier (IRI) as defined in [RFC3987].
   The specifics of the actual octet encoding of the IRI will follow the
   rules for general direct encoding of non-ASCII text.  For conversion
   between IRIs and URIs, the procedures defined in [RFC3987] are to be
   applied.

My understanding is that this is enough for people to figure out
that with EAI, they can use IRIs (encoded in UTF-8) in the
Archived-At header(s) of EAI messages. If somebody thinks that's
not enough, and a revision of RFC 5064 is needed, please tell me.

Regards,    Martin.

At 02:34 08/07/29, Randall Gellens wrote:
>EAI to define new versions of List-* headers which can have unencoded UTF8 values. This is just a trivial revision of the current forms. Like the current forms, each header can have multiple IRI, URL, URL, etc.  Note that this is orthogonal to new "mailto" -- when mailto-bis is ready, it just drops in as any other scheme such as HTTP.
>
>This is accomplished by creating a new WG document for 2369-bis to extend List-* headers to "permit UTF8 whenever UTF8 is otherwise permitted in header fields".  There isn't a need to do more than adding this one sentence to the existing 2369, I think.  So hopefully it will be very simple to produce.  We need to be very clear that, as 2369-bis is being done by EAI, its scope is strictly limited to permitting UTF8 and not to any other changes that anyone might desire in 2369.
>
>A preference is for List-ID to be considered special and remain ASCII (same as Message-ID). There is an open question on internationalized domain names: since these are used in Sieve scripts, users need to be able to type them in.
>
>Note that allowing UTF8 in List-* headers is equivalent to allowing UTF8 in address headers.  Presumably, only EAI-aware clients will see either.
>
>Add informative reference to new 2369-bis draft along the lines of "Since [2369-bis] permits UTF8 in List-* headers, which are vital to user's experience of mailing lists, this document needs to specify how such headers are to be downgraded and remain valid."
>
>Where available, mailing lists SHOULD include an ASCII form of List-* headers as well as a UTF8 form. Because these headers are vital to good operations and use of mailing lists, special rules are defined in [downgrade] so that such headers are not stripped during downgrade, but are preserved with working values whenever possible.
>

#-#-#  Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-#  http://www.sw.it.aoyama.ac.jp       mailto:[email-masked]

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 30 Jul 2008 11:10:47 +0200

Martin Duerst wrote:

> My understanding is that this is enough for people to figure
> out that with EAI, they can use IRIs (encoded in UTF-8) in
> the Archived-At header(s) of EAI messages.

> If somebody thinks that's not enough, and a revision of RFC
> 5064 is needed, please tell me.

Not at this time, let's wait until the dust of IDNAbis and EAI
settles.  BCP 9 mumbles something about "six months", but IMO
an RFC 5064 promotion can wait until EAI, 2822upd, and IDNAbis
catch up.

 Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 30 Jul 2008 11:31:15 +0100

On Tue, 29 Jul 2008 13:59:36 +0100, Randall Gellens <[email-masked]>  
wrote:

>>  On Mon, 28 Jul 2008 18:34:57 +0100, Randall Gellens  
>> <[email-masked]> wrote:

>>> UTF8 values....
>>  But is that not equivalent to just saying that IRIs are to be  
>> permitted (within the UTF8SMTP universe) in places where URIs are  
>> currently allowed?
>>
> I'm no expert in IRIs, but my understanding is that an IRI can be  
> represented either in UTF8 or in percent-encoded form.  I specifically  
> said "unencoded UTF8 values" because I think this is the key point.

I think we need Martin to come in here and explain it all to us.
> Yes, but, there is the sticky point in the next paragraph....

But the my paragraph was not addressed to List-ID, but to those List-*  
headers that currently use URIs.
>
>>
>>>  ... There is an open question on internationalized domain names:  
>>> since these are used in Sieve scripts, users need to be able to type  
>>> them in.

>
> The point about internationalized domain names and Sieve scripts is  
> specifically about List-ID. Since this is the unique identifier for a  
> list and hence used in Sieve scripts, it should be permitted to be  
> something meaningful that users can type in.  So, the example might be
>
> 	List-ID: list.mañana.com

Well either we allow UTF-8 in there, or we don't. If we do, then we have  
to address the consequences.

>>  No! Don't drop it. Downgrade it to a URI, and let mailto-bis sort out  
>> the result.
>
> mailto is just one possible scheme that can be used in a List-* header.   
> I'm OK with encoding it, if the encoded result is always as valid and  

If it's a valid IRI (of whatever scheme), then there is already an agreed  
mechanism to turn it into a valid URI. Whether it is "valid" delepnds on  
the scheme. The Mailto case is off interest because it is likely to be  
modified to handle EAI addresses, and we are in a position to influence  
the nature of that modification. Again, we need Martin to come in here.

>>  Having invented alt-addresses for use elsewhere within EAI, let us  
>> stick with them here. The last thing we want is two mechanisms for  
> I don't follow.  What are the two versions?  The point is that an  
> EAI-aware list can be created that operates behind a UTF8 email address  
> which has no ASCII version.  That's perfectly OK, if people want to do  
> this.  Presumably, the list is only for people who have EAI-aware  
> clients.

There may be mailing lists which are only intended for EAI-aware clients.

There may be mailing lists where EAI messages are accepted, but are  
downgraded before forwarding.

There may be mailing lists where EAI messages are accepted and forwarded,  
but will get downgraded when sent to non-EAI subscribers.

There may be mailing lists which are all-ASCII, but which accept  
subscriptions from EAI addresses.

All of these cases need to work in a sensible manner.

---

From: Randall Gellens <[email-masked]>
Date: Wed, 30 Jul 2008 09:02:49 -0700

>  not enough, and a revision of RFC 5064 is needed, please tell me.

I think it would be more clear if, instead of saying "The specifics 
of the actual octet encoding of the IRI will follow the rules for 
general direct encoding of non-ASCII text," it actually said UTF8. 
(For example, "The IRI is encoded in UTF8.")  However, I'm not sure 
this is worth a revision.  If you're going to revise it anyway, then 
add it.

-------------- Randomly-selected tag: ---------------
It has always seemed to me that the most difficult part of building
a bridge would be the start.                      --Robert Benchley
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: Randall Gellens <[email-masked]>
Date: Wed, 30 Jul 2008 09:09:22 -0700

At 11:31 AM +0100 7/30/08, Charles Lindsey wrote:

>> and Sieve scripts is specifically about 
>> List-ID. Since this is the unique identifier 
>> for a list and hence used in Sieve scripts, it 
>> should be permitted to be something meaningful 
>> that users can type in.  So, the example might 
>> be
>>
>>  	List-ID: list.mañana.com
>
>  Well either we allow UTF-8 in there, or we 
> don't. If we do, then we have to address the 
> consequences.

Yes.  Hence the question (to allow UTF8 in List-ID or not).

-------------- Randomly-selected tag: ---------------
The Macintosh uses an experimental pointing device called a "mouse."
There is no evidence that people want to use these things.
    --John C. Dvorak (well-known computer pundit), Feb 1984
_______________________________________________
IMA mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 31 Jul 2008 11:03:54 +0100

> an RFC 5064 promotion can wait until EAI, 2822upd, and IDNAbis

Indeed so.

What we would want to write is
    Archived-At: <http://archive.mañana.com>
which is an IRI (using UTF-8, as always with IRIs) which, upon conversion  
to a URI, would simply %-encode that 'ñ'.

But would that then constitute a valid URI? Who knows until  
IDNAbis/whoever clarifies the matter. They might well decree that valid  
URIs for that purpose have to be in punycode, which would in turn require  
the IRI to be in punycode, which would mean a special downgrading at any  
UTF8SMTP-ASCII interface.

The same applies to mailto: [IU]RIs, except that mailto-bis is more likely  
to be whatever Martin proposes, and we are in a better position to lean on  
Martin :-) .

---

From: "Frank Ellermann" <[email-masked]>
Date: Thu, 31 Jul 2008 17:44:35 +0200

Charles Lindsey wrote:

> which is an IRI (using UTF-8, as always with IRIs)

JFTR, s/as always with IRIs/as always with EAI/ 

EAI always uses UTF-8, while IRIs can be in any charset
determined by the context.  And for EAI that context is
of course UTF-8.  But in a KOI8-R document it's KOI8-R:

http://hmdmhdfmhdjmzdtjmzdtzktdkztdjz.googlepages.com/IDN-IRI-koi8-r.html

> which, upon conversion to a URI, would simply %-encode
> that 'ñ'.

For an <ipath>, <iquery>, or an <ifragment>.  But your
example has the ñ in the <ihost>, and there you would
use IDNA(bis) rules to convert this into an URL:

Archived-At: <http://archive.xn--maana-pta.com/>

Extended by some Message-ID, as Archived-At gives you
the location per message, not only some archive URL.

Nobody (maybe except some XML and LEIRI geeks) has the
faintest idea what a <http://archive.ma%C3%b1ana.com/>
might be.  It is a "syntactically valid" URL in some
non-DNS parallel universe I don't care about, but it
is not produced by following RFC 3987 rules.

Excluding XML LEIRI abominations unrelated to a "real"
world as defined by DNS, Googlebot, and W3C validator.

> IDNAbis/whoever clarifies the matter. They might well
> decree that valid URIs for that purpose have to be in
> punycode

IDNAbis won't touch URIs with a stick.  The IRI to URI
rules are in RFC 3987, and that uses IDNA for <ihost>:
No percent characters in labels of host names, percent
is not LDH.

> mailto-bis is more likely to be whatever Martin 
> proposes, and we are in a better position to lean on  
> Martin :-) 

Mailto-bis IRIs on a KOI8-R page would be interesting:

If they are transformed to UTF-8 (the first step in a
IRI to URI conversion) they might work with EAI UTF-8.

But if they are transformed to a mailto-bis URL they
might not more work, neither EAI nor 2822upd.

Mailto-bis is very near to hopeless, I'd be delighted
if it at least manages all STD 66 and 2822upd hurdles,
ignoring additional mailto-i18n obstacles.

 Frank
-- 
<http://omniplex.blogspot.com/2008/07/html5-asinine-selfish.html>