Subject: Re: [EAI] help to define Ucharacter and sub-domain by ABNF

From: Chris Newman <[email-masked]>
Date: Mon, 01 May 2006 10:59:43 -0700

It occurs to me that "Uatext" might be a better name for the ABNF rule.  It's 
not a general purpose character.

                - Chris

YAO Jiankang, [email-masked] wrote on 4/28/06 19:31 -0700:

> This ABNF should work:
>
>   Ucharacter = atext / UTF8-2 / UTF8-3 / UTF8-4
>
> Where "atext" is defined in RFC 2822 and UTF8-2, UTF8-3, UTF8-4 are defined
> in RFC 3629.
>
>                 - Chris
>
> YAO Jiankang wrote on 4/28/06 11:14 +0800:
>
>>
>> Dear all,
>>
>>     Could some kind experts help to refine the below definition of Ucharacter
>> and sub-domain by ABNF in the IMA SMTP extension document?
>>    It seems  not easy to find a very good appropriate definition.
>>
>>   thanks a lot.
>>
>>
>> --------------------------------------------------------------------------
>>
>>  According to the description above, define the syntax of an IMA
>>    mailbox with ABNF [RFC4234] as
>>
>>
>>          Mailbox = Local-part "@" Domain
>>
>>          Local-part = Dot-string / Quoted-string
>>                ; MAY be case-sensitive
>>
>>          Dot-string = Atom *("." Atom)
>>
>>          Atom = 1*Ucharacter
>>          Ucharacter = <any UNICODE character,
>>              except ASCII characters that are not permitted in "atext" and
>> characters that can not pass Stringprep>
>>
>>          Quoted-string = DQUOTE *qcontent DQUOTE
>>
>>          Domain = (sub-domain 1*("." sub-domain)) / address-literal
>>          sub-domain = Let-dig [Ldh-str] /
>>              <any internationalized domain label specified by IDNA>
>>
>>
>> YAO Jiankang
>> Best Regards
>> Tel:86-10-58813007
>> Fax:86-10-62559892
>> --------------------------------------------------
>> China Internet Network Information Center (CNNIC)
>> 4, South 4th Street, Zhongguancun,
>> Haidian district,
>> Beijing 100080, China
>> POB: Beijing 349, Branch 6
>> http://www.cnnic.cn
>> Email: [email-masked]
>> [email-masked]
>>
>
>
>
>
>
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ima
>

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 03 May 2006 14:47:11 +0100

On Fri, 28 Apr 2006 04:14:40 +0100, YAO Jiankang <[email-masked]> wrote:

> Dear all,
>    Could some kind experts help to refine the below definition of  
> Ucharacter and sub-domain by ABNF in the IMA SMTP extension document?

Presumably you are asking this with regard to the smtpext document.  
However, it might be better to establish the syntax for the utf8headers  
document first.

Sometime ago, I promised to dig out the syntactic changes we made in the  
USEFOR WG at a time when we were considering I18N of headers for Netnews,  
and particularly for the Newsgroups header. We never went ahead with this  
(though we might go back to it once this present IMA work is completed).

So the following syntax is taken from
http://www.imc.org/ietf-usefor/drafts/draft-ietf-usefor-article-09.unpaged,
with changes made to accomodate it to the IMA project. It also shows up  
some syntactic issues that we need to think about. It is not, of course,  
the only way to go about defining this syntax, but I hope it is of use.

    UTF8-xtra-char       = UTF8-2 / UTF8-3 / UTF8-4
[Then you can say that <UTF8-xtra-char>s are further restricted by  
normalizing
them to level NFC, and in other ways as set out in
draft-klensin-net-utf8-00.txt]

    UTF8-2               = %xC2-DF UTF8-tail
    UTF8-3               = %xE0 %xA0-BF UTF8-tail /
                           %xE1-EC 2(UTF8-tail) /
                           %xED %x80-9F UTF8-tail /
                           %xEE-EF 2(UTF8-tail)
    UTF8-4               = %xF0 %x90-BF 2(UTF8-tail) /
                           %xF1-F7 3(UTF8-tail)
[Those three are taken from RFC 3629, so maybe do not need to be repeated]

[The next rules are intended to supersede the corresponding rules in RFC  
2822
(though we might decide that they should not be given free rein in the
obs-syntax)]

    ctext                = NO-WS-CTL /      ; all of <text> except
                           %d33-39 /        ; SP, HTAB, "(", ")"
                           %d42-91 /        ; and "\"
                           %d93-126 /
                           UTF8-xtra-char
[That gives you UTF-8 in comments]

    qtext                = NO-WS-CTL /      ; all of <text> except
                           %d33 /           ; SP, HTAB, "\" and DQUOTE
                           %d35-91 /
                           %d93-126 /
                           UTF8-xtra-char
[That gives you UTF-8 in quoted-strings, and hence in phrases.
It also gives you UTF-8 on the RHS of parameters as defined in RFC 2045,  
but
this needs to be pointed out as an extension to RFC 2045 (with RFC 2231  
used
as the downgrade).]

    text                 = %d1-9 /          ; all UTF-8 characters except
                           %d11-12 /        ; US-ASCII NUL, CR and LF
                           %d14-127 /
                           UTF8-xtra-char
[That gives you UTF-8 in quoted-pairs]

    utext                = NO-WS-CTL /      ; Non white space controls
                           %d33-126 /       ; The rest of US-ASCII
                           UTF8-xtra-char
    unstructured         = 1*( [FWS] utext ) [FWS]
[So now we have covered all the cases where RFC 2047 is currently used (and
will still be used for downgrades)]

    atext                = ALPHA / DIGIT /
                           "!" / "#" /      ; Any character except
                           "$" / "%" /      ; controls, SP, and specials.
                           "&" / "'" /      ; Used for atoms
                           "*" / "+" /
                           "-" / "/" /
                           "=" / "?" /
                           "^" / "_" /
                           "`" / "{" /
                           "|" / "}" /
                           "~" /
                           UTF8-xtra-char
[That gives you UTF-8 in local-parts (quoted or not) and in domains, but it
needs careful examination to see whether we have gone too far]

    addr-spec            = local-part "@" domain
    local-part           = ( dot-atom / quoted-string ) [ atomic ]
    atomic               = ???????????????
[It is quite unclear to me what the syntax of <atomic> is intended to be,  
but
since it applies to local-parts it seems to belong here. Presumably it  
MUST be
absent unless either the dot-atom or the quoted-string contains some
UTF8-xtra-char.]

    angle-addr           = [CFWS] "<" addr-spec [ alt-address ] ">" [CFWS]
    alt-address          = alt-separator strict-addr
    alt-separator        = <"," or "|" or whatewver we decide, possibly with
                            some WSP or FWS around it>
[<strict-addr> will be more or less the <addr-spec> as currently defined in
RFC 2822. The syntax follows - it is extremely tedious, because it has to  
undo
all the changes made to intorduce UTF-8, but here it is, anyway.]

    strict-addr          = strict-local-part "@" strict-domain
    strict-local-part    = strict-dot-atom / strict-quoted-string
    strict-dot-atom      = [CFWS] strict-dot-atom-text [CFWS]
    strict-dot-atom-text = 1*strict-atext *( "." 1*strict-atext 0
    strict-atext         = ALPHA / DIGIT /
                           "!" / "#" /      ; Any character except
                           "$" / "%" /      ; controls, SP, and specials.
                           "&" / "'" /      ; Used for atoms
                           "*" / "+" /
                           "-" / "/" /
                           "=" / "?" /
                           "^" / "_" /
                           "`" / "{" /
                           "|" / "}" /
                           "~"
    strict-quoted-string = [CFWS] DQUOTE
                              *( [FWS] strict-qcontent ) [FWS]
                              DQUOTE [CFWS]
    strict-qcontent      = strict-qtext / strict-quoted-pair
    strict-qtext         = NO-WS-CTL /      ; qtext restricted to
                           %d33 /           ; US-ASCII
                           %d35-91 /
                           %d93-126
    strict-quoted-pair   = "\" strict-text
    strict-text          = %d1-9 /          ; text restricted to
                           %d11-12 /        ; US-ASCII
                           %d14-127

[Here the possibilities divide. If you are going to use "," as the
<alt-separator>, then you simply say:]

    strict-domain        = strict-dot-atom / domain-literal
[The <domain-literal> defined by RFC 2822 already allows many things,
including most of ASCII, that do not really belong in IP addresses. I hope
nobody is suggesting allowing any UTF-8 in there.]

[But if you want to use "|" as the <alt-separator>, then it would be  
better to
restrict the domains in an alt-address to what RFC 2821 will accept.]
    strict-domain        = (sub-domain 1*("." sub-domain)) / address-literal
    sub-domain           = Let-dig [Ldh-str]
    address-literal      = "[" IPv4-address-literal /
                               IPv6-address-literal /
                               General-address-literal "]"
[Though I do not altogether like the <IPv6-address-literal> from RFC 2821,  
and
I gather that implementors may not actually be following it.]

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5