Subject: [EAI] determining the question set for consensus call

From: Joseph Yee <[email-masked]>
Date: Thu, 6 Jan 2011 02:43:04 -0500

Hi all,

Recent reviews raised some concerns and the WG had discussions on various issues including negotiation mechanism, protocol logic, terminologies, document style, etc.  

As John mentioned in separate email; John, Alexey and I had discussion on how to organize consensus calls.  And before heading to consensus call, we will confirm with the WG on the set of questions.  Once the set confirmed, we will proceed in discussion on each topic and will evaluate the results.

(1)
Should the WG add parameter on MAIL FROM to start EAI transaction and avoid the need for deep inspection?

(2)
Should the parameter be permitted only if VRFY or EXPN appears subsequent to an EHLO response that includes UTF8SMTPbis?

(3)
Should the WG remove repetition of normative text from other documents to the extent possible.  Incorporate by reference and, where necessary, note explicitly that tutorial / context-providing references are not normative.

If the WG remove the repetition of normative text, will it make the draft hard to read?  If not, are there any text in current draft may result in inaccurate or confusing information because by update of other RFCs?

(4)
Should the WG replace references to "ASCII", "non-ASCII", and "UTF-8" with new or different term s that precisely and accurately identify what is intended?

(5)
Should the WG add additional text for gateways?

(6)
Should the WG add additional text for ticket systems that interface with email address, internationalized string, log, and traces?

(7)
Should the WG keep the decision about nested encodings and the use of message/global as described in RFC5336 and RFC5336bis? Or is a different model needed?

(8)
Once errors are corrected, is the current metalanguage model (including the 'u' prefixes) acceptable?  If not, should only those rules be included that are modified from RFC 5321 or 5322 respectively, forcing the user to reference the original documents for parts of substantially every rule?

-------
Note: Issues (5) and (6) were extracted from review composed by Claudio Allocchio and there are lack of discussion on them.  For more details, please refer to Claudio's review (http://www.ietf.org/mail-archive/web/ima/current/msg03621.html)

I am also working on the issue tracker, and will send out the track number as soon as possible for questions agreed by the WG.

Regards,
Joseph, co chair of EAI

---

From: Dave CROCKER <[email-masked]>
Date: Sat, 08 Jan 2011 13:51:52 -0800

On 1/5/2011 11:43 PM, Joseph Yee wrote:
>    And before heading to consensus call, we will
> confirm with the WG on the set of questions.  Once the set confirmed, we will
> proceed in discussion on each topic and will evaluate the results.

My own preferences:

> (1) Should the WG add parameter on MAIL FROM to start EAI transaction and
> avoid the need for deep inspection?

Yes.

> (2) Should the parameter be permitted only if VRFY or EXPN appears subsequent
> to an EHLO response that includes UTF8SMTPbis?

It should be permitted whenever VRFY or EXPN is permitted.  In other words, this 
enhancement should not change the basic SMTP state machine.

> (3) Should the WG remove repetition of normative text from other documents to
> the extent possible.  Incorporate by reference and, where necessary, note
> explicitly that tutorial / context-providing references are not normative.

Yes.

> If the WG remove the repetition of normative text, will it make the draft
> hard to read?  If not, are there any text in current draft may result in
> inaccurate or confusing information because by update of other RFCs?

Removal will simplify the current text.  Simpler specifications usually are much 
easier to read and understand.

The changes made by this extension should be relatively specific.  Hence they 
should not require going back and forth between the based SMTP specification and 
this extension specification.  In addition, anyone who implements these changes 
already needs to be familiar with the SMTP specification.  They do not need to 
be reminded of some details but not others.

Concern for inaccuracy or confusion is most serious for the future, not the 
present.  However even in the present there is a problem that copying part of 
the text from another specification means that the copied text loses the context 
in which it was written.  This can produce significant misunderstandings.

> (4) Should the WG replace references to "ASCII", "non-ASCII", and "UTF-8"
> with new or different term s that precisely and accurately identify what is
> intended?

Yes.

> (5) Should the WG add additional text for gateways?

Probably not, unless the working group wants to specify interworking between 
ASCII and UTF-8 environments.  (My understanding is that the WG decided NOT to 
support this interworking.)  The design needs to be reasonably friendly to 
gateway work, but that is different from actually doing specification.

> (6) Should the WG add additional text for ticket systems that interface with
> email address, internationalized string, log, and traces?

I don't have an opinion about this, yet.

> (7) Should the WG keep the decision about nested encodings and the use of
> message/global as described in RFC5336 and RFC5336bis? Or is a different
> model needed?

No.  The working group should not change MIME encoding rules.

> (8) Once errors are corrected, is the current metalanguage model (including
> the 'u' prefixes) acceptable?  If not, should only those rules be included
> that are modified from RFC 5321 or 5322 respectively, forcing the user to
> reference the original documents for parts of substantially every rule?

I think the ABNF changes are better, if 'replacement' rules being with a string 
that can be reliably and accurately parsed, such as 'utf-' rather than 'u'.  The 
concern is to make the string more likely to be unique.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: John C Klensin <[email-masked]>
Date: Sun, 09 Jan 2011 00:02:31 -0500

--On Saturday, January 08, 2011 13:51 -0800 Dave CROCKER
<[email-masked]> wrote:

>...
>> (2) Should the parameter be permitted only if VRFY or EXPN
>> appears subsequent to an EHLO response that includes
>> UTF8SMTPbis?
> 
> It should be permitted whenever VRFY or EXPN is permitted.  In
> other words, this enhancement should not change the basic SMTP
> state machine.

That is interesting.  Although one can read the text of 5321
(and its predecessors back to 1425) to allow this because there
is no obvious and explicit prohibition, we have never before
allowed an extension parameter to be specified prior to its
being authorized by a matching <ehlo-keyword>.   You implicitly
referred to the assumed requirement in your earlier note when
you wrote:

> (There is also the requirement that an EAI client must not
> send UTF-8 data to a non-EAI server, but the proposed SMTP
> extension keyword solves that problem is the simple, usual
> way.)

So it seems to me that there is a choice:

	(1) Permit VRFY or EXPN with the keyword (and possibly
	with a command argument outside the ASCII range) only
	after EHLO and an EHLO response that authorizes EAI
	extensions.
	
	(2) Permit this keyword, and possibly other keywords, on
	VRFY and EXPN wherever they might occur, without the
	protection of the extension mechanism when they are used
	outside a mail session (i.e., before EHLO).

I don't see either of these as changing the state machine.  VRFY
and EXPN can still be issued without a mail session and still
don't change the state of the mail session if they are issued
inside one.   The difference is where the parameter is permitted
and whether we are redesigning the commands themselves.

As someone (Ned?) noted earlier, unlike MAIL and RCPT, there is
no template syntax for parameters in 5321 (or 1425) for VRFY or
EXPN.  Some procedural complications aside (note that, if there
is a problem with how and where parameters are added to VRFY and
EXPN, the problem was committed in RFC 5336 -- it is not new
here), we can certainly add those parameters and invoke the
general extension model, as long as the parameterized versions
of the commands occur only after a mail session is established
with the appropriate ESMTP keyword response.  

The other possibility is to redefine VRFY and EXPN entirely, and
do so outside the bounds of the extension mechanism.  The
redefinition would permitting the Unicode-enabling keyword at
any time the command is permitted, without any expectation of
getting server permission first.  In principle, that is a pretty
big step with some risk of confusion by previously too-robust
servers.  In practice, maybe not.

     john

---

From: Dave CROCKER <[email-masked]>
Date: Sun, 09 Jan 2011 13:17:36 -0800

...
> 	outside a mail session (i.e., before EHLO).

Apologies.  I thought I had read the relevant, previous messages adequately, but 
had not.  I think I've remedied that oversight.  I also thought that what I 
meant was straightforward, but that wasn't true either.

The discussion thread about a new command versus restricted use of the existing 
command -- where the restriction is to require EHLO first -- looks reasonable to 
me, as does its conclusion.

Hence, I think this wg should settle on a parameter on the existing query 
commands, with a restriction that the parameters must not be used unless an EHLO 
has signaled server support for them.

Considering new commands might be interesting, but as noted goes beyond the 
scope of this working group.

d/
-- 

   Dave Crocker

---

From: John Levine <[email-masked]>
Date: 9 Jan 2011 22:08:42 -0000

>Hence, I think this wg should settle on a parameter on the existing
>query commands, with a restriction that the parameters must not be
>used unless an EHLO has signaled server support for them.

Having gone back and looked at the relevant bits of the drafts and of
RFC 5321, I agree.

When I suggested the STARTEAI command I was under the mistaken impression
that VRFY and EXPN allowed arbitrary text after the command, which would
have made whatever the parameter is retroactively a reserved word.  But
on rereading, I see that they take an argument which is one possibly
quoted string, so the parameter doesn't collide with anything.

We definitely need a two-way handshake so both ends of a connection
know that they're speaking EAI, and this seems the least complicated
way to do it.

As far as whether to allow the parameter on EXPN and VRFY before EHLO,
I realize that it changes the state machine to require EHLO first, but
I wouldn't want to make assumptions about what a non-EAI server might
do with a parameter it wasn't coded to handle.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Sun, 09 Jan 2011 18:54:36 -0500

--On Sunday, January 09, 2011 22:08 +0000 John Levine
<[email-masked]> wrote:

>> Hence, I think this wg should settle on a parameter on the
>> existing query commands, with a restriction that the
>> parameters must not be used unless an EHLO has signaled
>> server support for them.
> 
> Having gone back and looked at the relevant bits of the drafts
> and of RFC 5321, I agree.
> 
> When I suggested the STARTEAI command I was under the mistaken
> impression that VRFY and EXPN allowed arbitrary text after the
> command, which would have made whatever the parameter is
> retroactively a reserved word.  But on rereading, I see that
> they take an argument which is one possibly quoted string, so
> the parameter doesn't collide with anything.

Yes.  The only issues are two bits of flexibility that occur in
practice (but violate 5321):

	(1) In 821, the string argument to VRFY or EXPN could
	contain blanks and other odd characters as long as they
	were preceded by backslash (no surrounding double quotes
	required).   It is possible that code to send such
	strings is around somewhere; whether or not we should
	allow for them is a judgment call.
	
	(2) In part because of the confusion created by that odd
	quoting convention in 821, there are, or used to be,
	servers in the wild that would behave in exactly the way
	you were concerned about, i.e., treat the entire command
	line following VRFY or EXPN as the argument, effectively
	quoting the entire argument as a string if it wasn't
	quoted already.  

My personal view right now is that we can probably safely ignore
both of those cases.  But we should be explicit that is what we
are doing, e.g., by making an explicit requirement that any
server that advertises UTF8SMTPbis takes responsibility for a
5321-conforming interpretation of VRFY and EXPN arguments.

> We definitely need a two-way handshake so both ends of a
> connection know that they're speaking EAI, and this seems the
> least complicated way to do it.

Ack.

> As far as whether to allow the parameter on EXPN and VRFY
> before EHLO, I realize that it changes the state machine to
> require EHLO first, but I wouldn't want to make assumptions
> about what a non-EAI server might do with a parameter it
> wasn't coded to handle.

Exactly.

     john

---

From: Barry Leiba <[email-masked]>
Date: Sun, 9 Jan 2011 19:34:27 -0500

> avoid the need for deep inspection?

Absolutely, yes.

> to an EHLO response that includes UTF8SMTPbis?

Yes.  In order to use EAI-enabled VRFY of EXPN, the client MUST do
EHLO first and check for the server support.  If the client wants to
use VRFY or EXPN without EHLO, it's limited to the old, pre-EAI
version (and it will not get Unicode/UTF-8-encoded responses).

> the extent possible.  Incorporate by reference and, where necessary, note explicitly
> that tutorial / context-providing references are not normative.

Yes.

> If the WG remove the repetition of normative text, will it make the draft hard to read?

No.

> If not, are there any text in current draft may result in inaccurate or confusing
> information because by update of other RFCs?

RFCs aren't updated in place; if the references are to the RFCs by
number, the references will be stable.  If an RFC is updated or
obsoleted, that will be noted, but the referenced text in the original
RFC will still be there and will not have changed.

Of course, it might be a good idea to update our documents if/when an
RFC they reference has an updating or obsoleting RFC... but that's a
different issue.

> (4) Should the WG replace references to "ASCII", "non-ASCII", and "UTF-8" with new
> or different term s that precisely and accurately identify what is intended?

I've answered this in my other note: I think we need to define the
terms we use, and then use them rigorously.  And we need to make it
clear when we're referring to "characters" in the abstract sense, and
when we're referring to encoded characters.

> (5) Should the WG add additional text for gateways?

I don't think that's in scope, so I'd say no.  Follow-on informative
advice to gateways might be appropriate, as a separate effort.

> (6) Should the WG add additional text for ticket systems that interface with email
> address, internationalized string, log, and traces?

I think my answer to that is the same as for (5).  I wouldn't mind
brief text -- a paragraph, maybe -- that points out things to watch
for, but any significant treatment should be saved for follow-on
informative advice.

> message/global as described in RFC5336 and RFC5336bis? Or is a different

I'm undecided about this one.

> the 'u' prefixes) acceptable?  If not, should only those rules be included that
> are modified from RFC 5321 or 5322 respectively, forcing the user to reference
> the original documents for parts of substantially every rule?

The documents should introduce new ABNF elements for the new strings,
and update ABNF rules from the other documents.  Yes, we should use
references to the other documents for the original rules.

Barry

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 10 Jan 2011 10:25:55 +0900

My understanding from John and Joseph, and from the Subject of this 
thread, was that Joseph's list of questions was put on the mailing list 
just to check whether they are the right questions to later (when?) ask 
one-by-one. Now I see that some people are just replying to the 
questions here. Can we please do either one or the other?

Regards,   Martin.

On 2011/01/10 9:34, Barry Leiba wrote:
> _______________________________________________
> IMA mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ima
>
>

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: "Jiankang YAO" <[email-masked]>
Date: Mon, 10 Jan 2011 09:46:45 +0800

----- Original Message ----- 
From: ""Martin J. Dürst"" <[email-masked]>
To: "Barry Leiba" <[email-masked]>
Cc: "EAI WG" <[email-masked]>
Sent: Monday, January 10, 2011 9:25 AM
Subject: Re: [EAI] determining the question set for consensus call

>questions here. Can we please do either one or the other?

my understanding is that we can do both.
answer the questions, and add the new question if we  have.

Jiankang Yao

---

From: Joseph Yee <[email-masked]>
Date: Sun, 9 Jan 2011 21:51:13 -0500

Hi,

I am looking for the WG to confirm that we have the right questions to ask, and of course I am looking for suggestions (including new question) or questions to the question-set.  So please hold your answer for little longer, the question will post to the mailing list one by one.  If you answered any question, I take it that the question is a good question from your perspective.  If you like all questions and see no change, please say so in replying this thread too.

I would rather have the questions reviewed and confirmed quick by the WG rather than based on time.  But I don't see that we need lots of time to confirm the questions (I will let the discussion flow to prove my estimation wrong, and will adjust accordingly).  I will make summary on discussions coming before Wednesday Jan 12, 8pm Pacific Time.  I will update the question list in between to help speed up the discussion.

Thanks
Jospeh

---

From: "Martin J. Dürst" <[email-masked]>
Date: Mon, 10 Jan 2011 12:08:43 +0900

I think the questions you listed are the right ones, so please go ahead 
with putting them before the WG.

There is always the possibility that we find an additional question or 
two, but that shouldn't hold us up starting with the questions we 
already have.

Regards,   Martin.

---

From: Joseph Yee <[email-masked]>
Date: Wed, 12 Jan 2011 15:07:47 -0500

Repeating the question listed, with the new one on uAtom, and rephrase issue (2).

-------------------
(1)

(2) 
Should the new parameter to VRFY/EXPN only be permitted after the SMTP client issued EHLO command and saw a response that included UTF8SMTPbis?

(3)

(4)

(5)

(6)

(7)

(8)

(9) 
Do we need to tweak the definition of uAtom, and if yes, how?

------

Some observation from early consensus:
- Issues (5) and (6) were perceived towards editing matters or not in scope.  If no opposition observed, it's likely I will remove them from consensus discussion.

- I received suggestions to move the second part of issue (3) into later discussion, which I personally think is reasonable.

Suggestions welcomed.

Just friendly reminder that this is to confirm the question before getting into it.  I (even lack of reply) take note to opinions expressed so far and will discuss them afterwards.

Regards,
Joseph

On 2011-01-09, at 10:08 PM, Martin J. Dürst wrote:

> I think the questions you listed are the right ones, so please go ahead with putting them before the WG.
> 
> There is always the possibility that we find an additional question or two, but that shouldn't hold us up starting with the questions we already have.
> 
> Regards,   Martin.
>

---

From: Claudio Allocchio <[email-masked]>
Date: Thu, 20 Jan 2011 18:13:28 +0100 (CET)

> advice to gateways might be appropriate, as a separate effort.

wherevet it is, there should be at least a short reference inside the 
document.
> for, but any significant treatment should be saved for follow-on

same as above, and the Security consideration is the correct place for it, 
already in the document.

------------------------------------------------------------------------------
Claudio Allocchio             G   A   R   R          [email-masked]
                         Senior Technical Officer
tel: +39 040 3758523      Italian Academic and       G=Claudio; S=Allocchio;
fax: +39 040 3758565        Research Network         P=garr; A=garr; C=it;

            PGP Key: http://www.cert.garr.it/PGP/keys.php3#ca