Subject: [EAI] #1507 NO-WS-CTL: Proposed resolution (direction)

From: Harald Alvestrand <[email-masked]>
Date: Sun, 07 Oct 2007 14:36:25 +0200

Dealing with something relatively non-controversial:

I believe the WG thinks it's reasonable for UTF8HDR and DSN to specify
that "utf8-text" builds on the definitions in RFC 2822, in such a way
that if 2822 is updated to remove the NO-WS-CTL characters form the
specification, no textual change will need to be made in the EAI
specifications.

If nobody objects to this resolution, I hope that Alexey, Frank and the
WG can work out appropriate text.

                  Harald

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 8 Oct 2007 14:32:18 +0200

> specification, no textual change will need to be made in the EAI

I'm not sure about that approach, 2822upd almost certainly will be
a DS, that seriously limits its "innovations" wrt NO-WS-CTL.  The
last draft still allows NO-WS-CTL in <dtext> (RHS characters of
addresses and even Message-IDs).

Actually #1507 consists of two parts:

1 - who needs <utf8-quoted-pair> ?  I think it's bogus, the original
    2822upd <quoted-pair> is bad enough.  Without <utf8-quoted-pair>
    the UTF8HDR I-D doesn't need any <utf8-text> in ABNF.  

2 - Without <utf8-text> in the UTF8HDR I-D the DSN I-D is free to use
    whatever it needs, e.g. utf8-text = WSP / VCHAR / UTF8-NON-ASCII

Plan B could be "whatever 2822upd or UTF8HDR say, the DSN I-D can
limit itself to WSP / VCHAR / UTF8-NON-ASCII".  Actually it could do
better when I-D.klensin-net-utf8 made it, but that's a race condition
at the moment.

Unrelated:  In I-D.klensin-unicode-escapes the perl notation ended up
as "NOT RECOMMENDED", how about switching to its 5.1 notation ?

 Frank

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 11 Oct 2007 19:49:55 +0100

>     the UTF8HDR I-D doesn't need any <utf8-text> in ABNF.

So you want to say "quoting, as in <quoted-pair>s, is only to be used on
ASCII characters, since it is never of benefit in <utf8-xtra>s". The
reason it is there is only to avoid treading on 2822's toes too hard, but
one could argue that removing it makes
yat-another-arbitrary-an-ugly-exception. I share your dislike of
<quoted-pair>, but in some (few) contexts it is needed. But is is also
true that all escape mechanisms introduced into character strings
ultimately lead to more complicated code, and the greater likelihood of
obscure bugs. 'Twas ever thus :-( .

This is just comment. I am not arguing for or against any change.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 
   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 15 Oct 2007 12:57:35 +0200

Charles Lindsey wrote:
 
> So you want to say "quoting, as in <quoted-pair>s, is only to be
> used on ASCII characters, since it is never of benefit in 
> <utf8-xtra>s".

Yes.  I vaguely recall that John hoped to get rid of quoted-pairs
two or three years ago wrt EAI when we talked about what's now 
listed as RFC 3696 erratum.

> The reason it is there is only to avoid treading on 2822's toes
> too hard

Now's a good time to tread on its toes, before it's a DS... ;-)
Seriously, 2822bis isn't affected by what EAI does wrt unnecessary
quoted-UTF8-pairs.  It would be a pain for users, implementors,
and likely 3696bis.  

> yat-another-arbitrary-an-ugly-exception.

Not arbitrary, well considered.  It took us months to deal with
this "feature" in USEFOR, looking in the approved I-D we apparently
settled on "don't talk about it" for mail-complaints-to parameters.
Not exactly honest... :-|

Unicode is bad^Winteresting enough, it doesn't need this additional
obs-cenity to make it worse.  (I had an overdose of TUS 5.0 ch. 5.8
"newline guidelines" yesterday.  We should make sure that the blame
for it falls on TUS and not anything including EAI in the IETF.)

 Frank

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 18 Oct 2007 23:23:45 +0200

> WG can work out appropriate text.
I don't think we can take it upon ourselves to declare messages that are
legal per 2822 illegal
in the EAI context without specific reason (even though I sometimes wish
we could).
So dropping "utf8-quoted-pair" is not reasonable for us. But I still
think we should go out of our way to not give the impression that we've
explicitly decided to allow control characters. So the argument above
still seems valid to me.

Now with a more specific proposal:

write the production of utf8-text as:

  utf8-text = text / UTF8-non-ascii

(replacing all occurences of UTF8-xtra-char with UTF8-non-ascii, as per
Frank's suggestion).

As per the intro to section 4, "text" is already imported from RFC 2822.

Looks better?
Does this resolve #1507?

                     Harald

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 18 Oct 2007 23:36:57 +0200

> (replacing all occurences of UTF8-xtra-char with UTF8-non-ascii, as per
.... and of course I forgot to say that this applies equally to -dsn.
RFC 3464 uncritically imports "text" from 2822, and we're striving for
the least amount of incompatibility that we can manage, which says
that we should stick to that if at all possible.

                Harald

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 22 Oct 2007 10:45:38 +0200

>>> WG can work out appropriate text.

For that solution the I-Ds should actually import <text> from RFC 2822,
example for the DSN I-D:

-| utf8-text      = %d1-9 /      ; Any Unicode character except for NUL,
-|                  %d11 /       ; CR and LF, encoded in UTF-8
-|                  %d12 /
-|                  %d14-127 /
-|                  UTFMB
-|
-| UTFMB          = UTF2 / UTF3 / UTF4

+| utf8-text      = text /         ; text as specified in RFC 2821
+|                  utf8-non-ascii ; or other UTF-8 Unicode points
+|
+| utf8-non-ascii = UTF2 / UTF3 / UTF4

>> I don't think we can take it upon ourselves to declare messages that
>> are legal per 2822 illegal> in the EAI context without specific reason
>> (even though I sometimes wish we could).

Maybe, see above.  If you like to remove NO-WS-CTL from <text> in 2822upd
please say so on the rfc822 list.  So far Charles and others including me
focussed their attacks on <dtext>, <qtext>, and <quoted-pair>.  Maybe
it's better to remove NO-WS-CTL from <text> instead of <quoted-pair> (?)

>> So dropping "utf8-quoted-pair" is not reasonable for us.

Here I disagree.  We can leave the RFC 2822 <quoted-pair> as is, and we
should not try to make it worse.  Maybe existing parsers will interpret
a <quoted-pair> as "take the next byte as a literal".  That's likely
not what we want for 2..4 UTF-8 bytes.  I think we don't need a "new" 
<utf8-quoted-pair>, what should it do for EAI, obscure C1 controls ?

>>   utf8-text = text / UTF8-non-ascii

ACK, something like this mentioning the source (RFC 2822) for <text>.

> RFC 3464 uncritically imports "text" from 2822

It imports it from RFC 822 (STD 11).  RFC 3464 is a DS, it couldn't 
import anything from a PS like RFC 2822 without procedural tricks.

The RFC 822 <text> includes NUL, bare CR, bare LF, and 2822 NO-WS-CTL,
I'm sure that we don't want it in any EAI draft... ;-)

 Frank