Subject: [EAI] rfc5336bis-09: Downgrade in Submission server

From: [email-masked]
Date: Wed, 13 Apr 2011 19:56:58 +0900 (JST)

I have some commens for rfc5336bis-09.

Rewriting envelope, headers, message materials described in section
3.2 shoud be separated to another document because the part does not
describe how to rewrite non-ASCII envelope, headers, messages.

RFC 4409 does not allow unrestricted modifications.

| 3.2.  The UTF8SMTPbis Extension
| 
| 2.  If and only if the EAI-aware SMTP client (sender) is a Message
|        Submission Agent ("MSA") [RFC4409] [RFC5598], it MAY rewrite the
|        envelope, headers, or message material to make them entirely
|        ASCII [ASCII] and consistent with the provisions of RFC 5321
|        [RFC5321] and RFC 5322 [RFC5322].

The comment may be too late to say. I missed making the comment when
RFC 5336 was standardized.

-- 
Kazunori Fujiwara, JPRS <[email-masked]>

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 13 Apr 2011 12:57:35 +0100

Hi,

>RFC 5336 was standardized.
I think the purpose of the existing text is to emphasize that such 
modifications are allowed. I wouldn't make this text any more detailed. 
So any further details should go into the downgrade-on-submission document.

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 13 Apr 2011 07:58:31 -0700

On 4/13/2011 4:57 AM, Alexey Melnikov wrote:
>> | | 2. If and only if the EAI-aware SMTP client (sender) is a Message
> I think the purpose of the existing text is to emphasize that such modifications
> are allowed. I wouldn't make this text any more detailed. So any further details
> should go into the downgrade-on-submission document.

I'll second the concern raised by Fujiwara-san.

The Section 3.2 #2 text is offered almost as if it were a minor comment, when it 
actually is quite major.  Frankly, the best choice is to remove #2 entirely.

The authorization for an MSA to effect a downgrade is not stated in any other 
document.  So this one brief entry is the entire specification for MSA 
downgrading.  More text is needed, but it is needed in another document.

The current text cannot merely be a 'reminder' that modifications are allowed, 
because there is no specification yet authorizing such changes.

The current model for 5336bis is of a pure UTF-8 environment.  Keep it that way.

If there is a strong compulsion to refer to downgrading, make the comment 
extremely generic, along the lines of:

      Along the messaging handling path, it is possible that a node will 
translate a message from UTF-8 to be pure ASCII.  Specification of this process 
and its results is outside the scope of this document.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 13 Apr 2011 16:20:10 +0100

Dave CROCKER wrote:
 [...]

> If there is a strong compulsion to refer to downgrading, make the 
> comment extremely generic, along the lines of:
>
>      Along the messaging handling path, it is possible that a node 
> will translate a message from UTF-8 to be pure ASCII.  Specification 
> of this process and its results is outside the scope of this document.

Actually your proposed text is not correct: there was a very explicit 
discussion of this point and downgrading anywhere but MSA or MDA is 
prohibited.

---

From: Dave CROCKER <[email-masked]>
Date: Wed, 13 Apr 2011 08:26:26 -0700

On 4/13/2011 8:20 AM, Alexey Melnikov wrote:
>> Along the messaging handling path, it is possible that a node will translate a
>> message from UTF-8 to be pure ASCII. Specification of this process and its
>> results is outside the scope of this document.
>
> Actually your proposed text is not correct: there was a very explicit discussion
> of this point and downgrading anywhere but MSA or MDA is prohibited.

1. Please cite the relevant text.  I don't find the word "downgrade" in 
rfc5336bis and the word 'rewrite' occurs only in #2.

2. It is a mistake to have rfc5336bis define a 'pure' utf-8 model and then make 
statements about the rules for "leaving" that model, especially when it is not 
specifying the details.

Again, the text of #2 is not a mere reminder and it is out of place as a 
specification.

It has been a long journey from the very mixed set of goals the work started 
with, getting to what is a rather simple model that is easy to understand.  The 
document should stay with that improvement and remove text that taints it.

d/

-- 

   Dave Crocker

---

From: Joseph Yee <[email-masked]>
Date: Wed, 13 Apr 2011 17:58:48 -0400

On 2011-04-13, at 10:58 AM, Dave CROCKER wrote:

> The Section 3.2 #2 text is offered almost as if it were a minor comment, when it actually is quite major.  Frankly, the best choice is to remove #2 entirely.

RFC4409 permits MSA to rewrite of the message to conform to site's 'local policy'.  RFC5336bis Section 3.2 bulletin 2 is documenting such capability if 'next hop' is not EAI-aware.  Note MSA does not have the highest priority for this scenario.

> 
> The authorization for an MSA to effect a downgrade is not stated in any other document.  So this one brief entry is the entire specification for MSA downgrading.  More text is needed, but it is needed in another document.
> 
> The current text cannot merely be a 'reminder' that modifications are allowed, because there is no specification yet authorizing such changes.

Any addition would likely be repeating statement which we had consensus not to do.  The current text made normative reference to RFC4409 already.   

> If there is a strong compulsion to refer to downgrading, make the comment extremely generic, along the lines of:
> 
>    

[snip]

> Specification of this process and its results is outside the scope of this document.
> 

Personally, I like this sentence.  I think it's good to add it to the end of bulletin 2.  Thoughts?

Regards,
Joseph

> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima

---

From: [email-masked]
Date: Thu, 14 Apr 2011 14:57:52 +0900 (JST)

Hi,

> From: Alexey Melnikov <[email-masked]>
> modifications are allowed. I wouldn't make this text any more
> detailed. So any further details should go into the
> downgrade-on-submission document.

If the purpose of the existing text is to emphasize only,
the text is not necessary.

If the text updates RFC 4409, the text is necessary.

I believe that simple is the best. Unnecessary text should be removed.

> From: Joseph Yee <[email-masked]>

If RFC 4409 already permits the text, the text is useless.
Otherwise, the text updates RFC 4409.

> Personally, I like this sentence.  I think it's good to add it to the end of bulletin 2.  Thoughts?

The sentence seems OK.

But if the text is outside of the scope of the draft, then the text is useless.

New document will be able to update rfc5336bis.

---

From: "Jiankang YAO" <[email-masked]>
Date: Thu, 14 Apr 2011 17:36:57 +0800

----- Original Message ----- 
From: "Joseph Yee" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, April 14, 2011 5:58 AM
Subject: Re: [EAI] rfc5336bis-09: Downgrade in Submission server

> Personally, I like this sentence.  I think it's good to add it to the end of bulletin 2.  Thoughts?

if there were such a sentence, it seems that we are trying to Normalize the possible submission server downgrade document.

the simple one is  better.

so I suggest that we just keep the current words of the rfc5336bis-09.

Jiankang Yao

---

From: Joseph Yee <[email-masked]>
Date: Fri, 15 Apr 2011 14:30:02 -0400

> I believe that simple is the best. Unnecessary text should be removed.

Note of the text, just before all bulletin points, the last sentence at the third paragraph in section 3.2 that leads to the scenario discussion in the draft:

   Instead, if an EAI-aware SMTP client (EAI-
   aware SMTP sender) attempts to transfer an internationalized message
   and encounters an SMTP server that does not support the extension, it
   MUST make one of the following three choices and the priority order
   is 1, 2 and 3.

This text is to describe what could implementors do, with guided preferences, and meant to cover all permitted choices.  If #2 excluded, with rest unchanged, there's an implication of new restriction (change) to MSA under EAI-mode.  

> If RFC 4409 already permits the text, the text is useless.
Not useless, see my reason above.

> Otherwise, the text updates RFC 4409.
Do you mean that if we keep #2, the content must be about updating RFC4409?  Please elaborate.

> But if the text is outside of the scope of the draft, then the text is useless.

The intention of adding this 'out of scope' sentence is to say this draft would not discuss *how* MSA may rewrite an internationalized message.  

Regards,
Joseph

---

From: John C Klensin <[email-masked]>
Date: Tue, 19 Apr 2011 10:37:07 -0400

Speaking for myself only, let me propose what I think may be a
compromise position...

--On Wednesday, April 13, 2011 17:58 -0400 Joseph Yee
<[email-masked]> wrote:

>...
>> The Section 3.2 #2 text is offered almost as if it were a
>> minor comment, when it actually is quite major.  Frankly, the
>> best choice is to remove #2 entirely.
> 
> RFC4409 permits MSA to rewrite of the message to conform to
> site's 'local policy'.  RFC5336bis Section 3.2 bulletin 2 is
> documenting such capability if 'next hop' is not EAI-aware.
> Note MSA does not have the highest priority for this scenario.

The intent of 4409 is fairly clearly to permit the MSA
significant latitude in fixing things that the MUA didn't get
right.  It focuses on fixes to conform with the standards, but
the "local policy" provision that Joseph mentions opens things
up to broader changes.    So, whether this text is correct
and/or desirable or not, it really is a reminder in the sense
that it doesn't update 4409 in any substantive way.

>...
>> The current text cannot merely be a 'reminder' that
>> modifications are allowed, because there is no specification
>> yet authorizing such changes.
> 
> Any addition would likely be repeating statement which we had
> consensus not to do.  The current text made normative
> reference to RFC4409 already.   

yes.

>> If there is a strong compulsion to refer to downgrading, make
>> the comment extremely generic, along the lines of:
>> 
>>    
> 
> [snip]
> 
> 

Concur.

So, at the risk of causing more work for myself, let me suggest
the following:

(i) We remove the offending "reminder" sentence from #2 and
substitute the above sentence (or some editor-chosen variation).

(ii) An EAI-related version/revision of 4409 has always been an
EAI task if the WG concludes it is necessary.  Let's do that,
not with the intent of adding detailed EAI-specific provisions,
but adding a paragraph that explicitly indicates that
conversions of character sets or encodings are within the scope
of things that MSAs can do.  If that is handled as a
clarification --which I think it would be-- than it could be
done as a revision in grade or even as part of a full standard
4409bis (whether that document were processed by EAI or YAM
would be a matter of some indifference to me).

Does that make sense?  

    john

---

From: [email-masked]
Date: Wed, 20 Apr 2011 13:52:47 +0900 (JST)

> From: John C Klensin <[email-masked]>

I agree.

---

From: "Jiankang YAO" <[email-masked]>
Date: Thu, 21 Apr 2011 11:41:12 +0800

Dear all,

I just checked through the jabber log of Prague eai meeting.

I found the follow message:
"
[07:38:17] <resnick> One of my concerns with 5336bis is whether we have a story for why we did the *extension* syntax in 5335bis but are using an *alternate* syntax on 5336bis.
[07:38:52] <resnick> (My current read of 5336bis is that it's still using the alternate syntax.)
"

 ABNF is also an importan comment from Dave.  If I understand Dave's comments to ABNF clearly, 
the core suggestion is to have the revision focus only on the existing, basic 5322  ABNF constructs and
         simply augment them to support UTF-8 instead of using the prefix "u".

The current version of rfc5336bis is still  to use the prefix "u".

My concern is that if the system support both rfc5336bis and 5321, the system needs to recognize wihich address
is mailbox and which one is hte uMailbox. if we just extend it,  it is not easily to distingush the mailbox in implementation because both are named with "mailbox". .

on the other hand, the Email system may choose to recognize both the rfc5336bis address and rfc5321 address as pointed out by Fujiwara in the previous message, thus
mailbox and uMailbox need to be easily separated.

So, my personal favorite is still to keep the current alternate syntax style for rfc5336bis to avoid the possible confusion with rfc5321.

But if this concern above is not a problem, I will follow Dave's model to extend it instead of using prefix "u".

Jiankang Yao

---

From: John C Klensin <[email-masked]>
Date: Wed, 20 Apr 2011 23:59:48 -0400

--On Thursday, April 21, 2011 11:41 +0800 Jiankang YAO
<[email-masked]> wrote:

> [07:38:17] <resnick> One of my concerns with 5336bis is
> whether we have a story for why we did the *extension* syntax
> in 5335bis but are using an *alternate* syntax on 5336bis.
> [07:38:52] <resnick> (My current read of 5336bis is that it's
> still using the alternate syntax.) "

And, of course, we don't have the "extension" syntax in 5335bis
yet either, but I hope we will see it soon.

>  ABNF is also an importan comment from Dave.  If I understand
> Dave's comments to ABNF clearly,  the core suggestion is to
> have the revision focus only on the existing, basic 5322  ABNF
> constructs and          simply augment them to support UTF-8
> instead of using the prefix "u".

I think that is, more or less, the "extension" syntax model, but
Dave can speak for himself.

> The current version of rfc5336bis is still  to use the prefix

As I have said to you several times offlist, whatever the WG
decides to do about a basic ABNF model, it should do with all
documents, especially the three core ones.  I think having their
basic models --e.g., what Pete refers to as "alternate" and
"extension"-- be different is far less acceptable than making
either choice and using it for all three.

> My concern is that if the system support both rfc5336bis and
> 5321, the system needs to recognize wihich address is mailbox
> and which one is hte uMailbox. if we just extend it,  it is
> not easily to distingush the mailbox in implementation because
> both are named with "mailbox". .

> on the other hand, the Email system may choose to recognize
> both the rfc5336bis address and rfc5321 address as pointed out
> by Fujiwara in the previous message, thus mailbox and uMailbox
> need to be easily separated.

That is, I think, the argument for the WG's original conclusion
that we should just an "alternate" syntax, not an "extension"
one that reuses production names.
 
> So, my personal favorite is still to keep the current
> alternate syntax style for rfc5336bis to avoid the possible
> confusion with rfc5321.
> 
> But if this concern above is not a problem, I will follow
> Dave's model to extend it instead of using prefix "u".

Note that, whether we go with the "alternate" or "extension"
syntax, there is a remaining problem of what "mailbox" means --
that issue really needs to be sorted out with/in 5321/5322 and
should not be an EAI problem.

    john

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 21 Apr 2011 10:00:55 -0700

On 4/20/2011 8:41 PM, Jiankang YAO wrote:
> I found the follow message: " [07:38:17]<resnick>  One of my concerns with
> 5336bis is whether we have a story for why we did the *extension* syntax in
> 5335bis but are using an *alternate* syntax on 5336bis. [07:38:52]<resnick>
> (My current read of 5336bis is that it's still using the alternate syntax.)
> "
>
> ABNF is also an importan comment from Dave.  If I understand Dave's comments
> to ABNF clearly, the core suggestion is to have the revision focus only on
> the existing, basic 5322  ABNF constructs and simply augment them to support
> UTF-8 instead of using the prefix "u".
>
> My concern is that if the system support both rfc5336bis and 5321, the system
> needs to recognize wihich address is mailbox and which one is hte uMailbox.
> if we just extend it,  it is not easily to distingush the mailbox in
> implementation because both are named with "mailbox". .

There is a very large change that was made in the approach for doing the EAI 
work.  It changed from defining a complete, hybrid system that supports old and 
new addresses, and instead now focuses only on the UTF-8 case.  That is, the 
current, core specifications do not try to juggle both legacy and new 
characters.  They only seek to handle new (Unicode) characters.

Transition and hybrid issues are therefore deferred to other documents, outside 
of this core.

In my view, this simplification is quite a good choice, especially due to the 
fact that ASCII is a subset of Unicode.  Hence, talking only about UTF-8 means 
that ASCII is still included.

However, the documents still reflect the earlier attempt to juggle both old and 
new.  The ABNF I proposed is designed to deal with the simplified environment 
that is strictly UTF-8.  Hence, the core constructs, like mailbox, do not need 
an old and a new form.  The ABNF merely needs to enhance the definition of 
mailbox to a single, UTF-8 form.

In protocol architecture terms, this simplified approach defines use of UTF-8 as 
a "parallel" protocol stack to the older ASCII-only one.  This is why Pete's 
term "alternate" makes sense.

The term "extension" reflects an attempt to add on to existing ABNF without 
changing its existing core constructs.  Hence it is suited for the older hybrid 
case. However I believe its complexity is neither necessary nor appropriate for 
the current work.

> on the other hand, the Email system may choose to recognize both the
> rfc5336bis address and rfc5321 address as pointed out by Fujiwara in the
> previous message, thus mailbox and uMailbox need to be easily separated.
>
> So, my personal favorite is still to keep the current alternate syntax style
> for rfc5336bis to avoid the possible confusion with rfc5321.

As for rfc5335bis vs. rfc5336bis, I do believe it would be much better to have 
them reflect the same approach to ABNF enhancement.  Obviously my own 
recommendation is to have both of them use the simpler "alternate" definitions, 
rather than the more complicated "extension" approach.

There is a much larger and longer-term issue concerning some redundant 
specifications between RFC5321/RFC5322. It came from what I consider a basic 
mistake that Jon Postel and I made in doing RFC821 and RFC822.  I now class 
redundant specification as evil and I think that the DRUMS effort missed a real 
opportunity in failing to fix this.  But, as noted, this issue is far beyond the 
scope of the EAI working group.

I hope this fully responds to the questions in this thread.

d/
--

   Dave Crocker

---

From: Pete Resnick <[email-masked]>
Date: Sat, 23 Apr 2011 17:17:20 -0500

On 4/21/11 12:00 PM, Dave CROCKER wrote:
> Obviously my own recommendation is to have both of them use the 
> simpler "alternate" definitions, rather than the more complicated 
> "extension" approach. 

Darn. I didn't think that would happen.

When I said "alternate" syntax, I meant using an *alternate* name (like 
"uMailbox").

When I said "extension" syntax, I meant *extending* the 532x syntax 
(like "mailbox =/ blah").

I think (unless I am very confused), Dave swapped them on me.

Maybe we can just call one the "u" syntax. I don't know what to call the 
other. (*mumble*)

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Dave Crocker <[email-masked]>
Date: Sat, 23 Apr 2011 18:25:19 -0700

oops.

d/

-- 
Dave Crocker
bbiw.net

(via mobile)

Pete Resnick <[email-masked]> wrote:

On 4/21/11 12:00 PM, Dave CROCKER wrote: > Obviously my own recommendation is to have both of them use the > simpler "alternate" definitions, rather than the more complicated > "extension" approach. Darn. I didn't think that would happen. When I said "alternate" syntax, I meant using an *alternate* name (like "uMailbox"). When I said "extension" syntax, I meant *extending* the 532x syntax (like "mailbox =/ blah"). I think (unless I am very confused), Dave swapped them on me. Maybe we can just call one the "u" syntax. I don't know what to call the other. (*mumble*) pr -- Pete Resnick<http://www.qualcomm.com/~presnick/>; Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: "Jiankang Yao" <[email-masked]>
Date: Sun, 24 Apr 2011 10:08:03 +0800

----- Original Message ----- 
From: "Pete Resnick" <[email-masked]>
To: <[email-masked]>
Cc: "Dave CROCKER" <[email-masked]>; "Jiankang YAO" <[email-masked]>; <[email-masked]>
Sent: Sunday, April 24, 2011 6:17 AM
Subject: Re: [EAI] uMailbox or mailbox (ABNF for rfc5336bis)

> Maybe we can just call one the "u" syntax. I don't know what to call the 

now we are clear.
it is  "u" syntax  VS  "mailbox =/ blah" model problem.

If we favor the "mailbox =/ blah", another problem is how to extend them.

some names such as mailbox  are difficult to use the   "mailbox =/ blah"  model.

original defintion of mailbox of rfc5321 is ASCII@ASCII.
the new defintion of rfc5336bis should include  ASCII@ASCII., ASCII@utf-8, utf-8@ASCII , utf-8@utf-8 

if we use the  "mailbox =/ blah"  model, the syntax will be similar to mailbox= / (ASCII@utf-8, utf-8@ASCII , utf-8@utf-8)

expressing the (ASCII@utf-8, utf-8@ASCII , utf-8@utf-8) using ABNF is not easy, bring more ABNF complex.

buf if we  use the "mailbox = blah" model for mailbox, it is easy.
for example,

the current definition for mailbox is 

''
 uMailbox = uLocal-part "@" ( uDomain / address-literal )
    ; Replace Mailbox in RFC 5321, Section 4.1.2

   address-literal = <Defined in Section 4.1.2 of RFC 5321>

   uLocal-part = uDot-string / uQuoted-string
    ; MAY be case-sensitive
    ; Replace Local-part in RFC 5321, Section 4.1.2

''

now, if we just change uMailbox to Mailbox, the example above will become to:

''

 Mailbox = uLocal-part "@" ( uDomain / address-literal )
    ; Replace Mailbox in RFC 5321, Section 4.1.2

   address-literal = <Defined in Section 4.1.2 of RFC 5321>

   uLocal-part = uDot-string / uQuoted-string
    ; MAY be case-sensitive
    ; Replace Local-part in RFC 5321, Section 4.1.2

'' 

I am not sure whether the  "mailbox = blah" model is ok or not.

Jiankang Yao