Subject: Re: SMTP WG Last-Call

From: "Nick Shelness/SSW/Lotus" <[email-masked]>
Date: Fri, 21 Jul 2000 05:39:05 -0400

Dan,

Now that you have now produced a list of concerns minus personal attacks, 
I will respond.

1.
>   In reality, for interoperability with existing clients, servers
>   recognize \012 as a line terminator ...

This is true, but is independent of whether it should be required under 
the standard. I believe that we reached rough consensus that under the 
standard, <CRLF> be the only valid command terminator.

2.
>   In reality, for interoperability with existing clients, servers
>   accept (for example) EHLO requests containing underscores. ...

Again, this is true, but is independent of whether it should be required 
under the standard. I believe that we reached consensus that under the 
standard EHLO requests MUST NOT contain underscores.

3.
>   In reality, if an SMTP client is in the middle of sending a message,
>   and is unable to read the next portion of the message from disk, it
>   ruthlessly closes the connection. ...

Again, this is true, and the standard should and does state what a server 
and client should do when a connection is closed prior to the transmission 
of a QUIT and receipt of a QUIT response.

To that end, I think that the text in the final paragraph of section 3.9 
(the only one that addresses client behavior) could be expanded slightly 
to make it clear that there are environmental circumstances (in addition 
to those listed) beyond a client's control that may require it to 
terminate a connection. For example,

   SMTP clients that experience events beyond their control, such as an 
   unrecoverable disk read failure, a notification of impeding system 
   shutdown,> a connection close, a connection reset, or other type
   communications failure (in violation of the intent of this 
   specification but sometimes unavoidable) SHOULD, to maintain the 
   robustness of the mail system, treat the mail transaction as if a 451 
   response had been received and act accordingly.

But fundamentally, this was never at issue. What was at issue is whether 
it was valid under the standard for a client to tear down a connection 
after receiving a positive response to a <CRLF> . <CRLF> and with out 
sending a QUIT and receiving a response. Again, I believe that rough 
consensus was reached that it was not.

4.
>   In reality, as of 1999, approximately 90% of the servers that support
>   both VRFY and EHLO do not list VRFY in response to EHLO. ...

So. There was no rough consensus that VRFY and/or EXPN be dropped from the 
standard. I would quite happily see them both dropped, but there wasn't 
enough support to do so. What there was rough consensus on was, that under 
RFC 821bis servers that support EHLO and VRFY MUST announce VRFY support 
in an EHLO response.

5.
>   In reality, servers that do not support EHLO are still able to
>   receive mail. ...

Absolutely, but again there was rough consensus that RFC 821bis should 
encourage the infrastructure to move to employing EHLO in place of HELO.

6.
 >  In reality, clients that have no need for extensions can and should
 >  start with HELO; EHLO adds quite a bit of unnecessary complexity. ...

I have no objection to your use of the word "can", but again there was 
rough consensus that RFC 821bis should encourage the infrastructure to 
move to employing EHLO in place of HELO.

7.
>   In reality, EXPN is obsolete. Thousands of the most heavily used mail
>   servers on the Internet always respond 502 to EXPN. ...

See my response to 4. above

8.
>  In reality, VRFY is obsolete. Thousands of the most heavily used mail
>  servers on the Internet always respond 252 to VRFY. ...

See my response to 4. above

9.
>   In reality, thousands of SMTP clients close the connection
>   immediately after sending QUIT. ...

See my response to 3. above.

10.
>   In reality, connections are abruptly dropped for a wide variety of
>   reasons, including server shutdowns. ...

Absolutely. See my response to 3 above. The question is whether, in the 
absence of abnormal events beyond the control of an SMTP client or server 
task, it is valid under the standard to drop a connection prior to a 
client sending a QUIT and receiving a response. Again, there was not rough 
consensus to make such a change from RFC 821. 

11.
>   In reality, firewalls and POP toasters do not, and should not,
>   support mailing lists. ...

In the case of firewalls, as long as they are completely transparent at 
the connection and command level to an SMTP client and server, RFC 821bis 
is silent as it should be. Where an SMTP relay is deployed as a firewall, 
then it SHOULD (MUST?) comply with the standard. POP toasters are beyond 
the scope of drums.

12.
>   In reality, thousands of SMTP servers ignore such parameters. ...

Yes, but?

13.
>  In reality, firewalls and POP toasters often run pure SMTP servers.
>  They do not send mail through SMTP, and they are not expected to
>  support any SMTP client features. Installing unused code is a
>  violation of some corporate security policies. ...

RFC 821bis describes on the wire behavior. Thus, an SMTP server need only 
support on the wire RFC 821bis server features, and an SMTP client need 
only support on the wire RFC 821bis client features. Where's the problem?

14.
>   In reality, many relays support other protocols for transferring
>   Internet mail messages, such as QMTP. ...

Certainly true, though within the nomenclature of RFC 821bis these would 
be gateways rather than relays. They may be completely transparent 
gateways, but they are gateways none the less. Gateway behavior beyond 
their SMTP client or server behavior is not within the scope of RFC 821bis

15.
>   In reality, there are many common non-MIME body structures. ...

This is also true, but again rough consensus was reached within the 
working group that RCF 821bis and 822bis should strongly encourage the use 
of MIME, to the exclusion of all other body structures.

16.
>   In reality, there will be a year 10000. 

Yup

17.
> You think these are ``opinions''?

In general you state a fact (a reality in your terminology). This is then 
almost always followed by an opinion. I think that Keith was referring to 
your opinions as opinions. Your facts are certainly facts.

Nick

Nick Shelness, IBM Fellow
Chief Technology Officer
Lotus Development Corp
Tel: (617) 693-7484

Assistant: Mary McGurran
Tel: (617) 693-4321

---

From: Dave Sill <[email-masked]>
Date: Fri, 21 Jul 2000 09:02:21 -0400 (EDT)

"Nick Shelness/SSW/Lotus" <[email-masked]> wrote:

>DJB wrote:

The question is not whether it should be required or not, the question
is whether or not servers should be allowed to recognize \012 as a
line terminator. Nobody is arguing that clients should be allowed to
send \012 as a line terminator.

>standard EHLO requests MUST NOT contain underscores.

Same thing: `should servers be allowed to *accept* EHLO requests
containing underscores', not `should clients be allowed to *send*
them'.

>consensus was reached that it was not.

That may be so, but isn't a little silly to go through all the trouble 
to mandate sending the QUIT when possible, if it serves no useful
purpose? Especially when skipping the QUIT exchange saves several
round trips?

If servers have to Do The Right Thing when the connection dies at a
certain point, what is gained by outlawing the intentional breaking of 
the connection?

>only support on the wire RFC 821bis client features. Where's the problem?

The problem is that the draft goes beyond specifying wire behavior: it 
says that not supporting VRFY is a potential interoperability problem,
which it isn't.

>of MIME, to the exclusion of all other body structures.

So the rough consensus of the working group was to lie? It's one thing 
to encourage MIME, and another to pretend that it's the only way to
structure bodies.

-Dave

---

From: "D. J. Bernstein" <[email-masked]>
Date: 23 Jul 2000 04:53:05 -0000

Nick Shelness/SSW/Lotus writes:
> I believe that we reached consensus that under the 

That is not the issue. The issue is Klensin's text saying that servers
``MUST reject'' underscores. RFC 821 doesn't say that, and we certainly
don't have consensus to make the change.

> encourage the infrastructure to move to employing EHLO in place of HELO.

There isn't consensus that clients happy with HELO should be forced to
use EHLO instead.

> What there was rough consensus on was, that under RFC 821bis servers
> that support EHLO and VRFY MUST announce VRFY support in an EHLO response.

I don't believe you. It appears that the requirement was invented by
Klensin in 1995, with no discussion by DRUMS.

In August 1998, I asked what benefit Klensin's requirement had, pointed
out that it was incompatible with widespread practice, and asked when
DRUMS had discussed it. Nobody responded.

In January 1999, I asked again, and again nobody had an answer. The only
response was from Paul Hoffman, agreeing that the requirement was wrong.

A few people have now spoken up in favor of the requirement, and a few
more people have spoken up against it. Obviously most implementors are
against it. Compare this to Klensin's policy statement in December 1996:

   The editor's position is that telling an implementation that clearly
   conforms today that it is no longer conforming requires very strong
   and clear consensus.

Unfortunately, Klensin takes this ``position'' only for changes that he
doesn't find attractive.

---Dan

---

From: "D. J. Bernstein" <[email-masked]>
Date: 23 Jul 2000 19:17:50 -0000

Nick Shelness/SSW/Lotus writes:
  [ clients not needing extensions can start with HELO ]
> move to employing EHLO in place of HELO.

I've already commented that there certainly isn't consensus to force
clients to use EHLO when they're happy with HELO.

In fact, according to Leiba's notes of the August 1998 meeting,
consensus was established _against_ Klensin's change:

   Consensus is that "SHOULD use EHLO" be dropped, and a client that
   doesn't need extensions may use EHLO or HELO at the client's
   discretion.

That's consistent with discussions of the issue on the mailing list.

So why does smtpupd-12 still say ``SHOULD preferentially utilize EHLO''?
How many times do I have to point this out before Klensin undoes his
unauthorized change? And how come certain people---Crocker and Shelness,
for example---are claiming that there's consensus for this ``SHOULD''?

---Dan

P.S. At the same meeting, according to Leiba's notes, it was decided to
``remain with RFC1123 conventions'' for words like ``SHOULD.'' Klensin's
non-1123 definition of ``SHOULD'' appeared in the next draft.

---

From: DRUMS WG Chair <[email-masked]>
Date: Thu, 27 Jul 2000 12:16:16 -0700

--On Sunday, July 23, 2000 19:17 +0000 "D. J. Bernstein" <[email-masked]> 
wrote:
> ...
> ...
> ...

Section 3.2 of smtpupd-09 (and later) complies with WG rough concensus from 
that meeting, including the goal of encouraging use of EHLO.

The editor's comments on this issue are in section X.1.9(iv).

I searched my archives for messages with "HELO" in the subject line. 
According to such messages, one person has objected to this text on at 
least two occasions.  In all cases there was no support expressed for the 
objection.  The chair concludes there is rough concensus that the current 
text is adequate for RFC publication and the issue is permanently closed.

		- DRUMS WG Chair

---

From: "D. J. Bernstein" <[email-masked]>
Date: 27 Jul 2000 20:14:15 -0000

Section 2.2.1 says ``clients SHOULD preferentially utilize EHLO rather
than HELO,'' which is in blatant violation of the decision I quoted from
Leiba's meeting notes.

---Dan

---

From: "D. J. Bernstein" <[email-masked]>
Date: 27 Jul 2000 20:26:48 -0000

> that meeting, including the goal of encouraging use of EHLO.

Reader beware! Newman is trying to mislead you. He's perfectly aware
that section 2.2.1 is in blatant violation of the meeting consensus, but
he doesn't want to admit it, so he ignores section 2.2.1 and talks about
section 3.2.

---Dan

---

From: "D. J. Bernstein" <[email-masked]>
Date: 27 Jul 2000 21:19:55 -0000

Some juicy historical details in this message---including an admission
from the document editor that the continued presence of this text is an
``error'' on his part!

We have consensus against this text. But Crocker says that there's
consensus for the text. Newman agrees and says that the issue is
permanently closed. _They_ are the people interfering with DRUMS.

> I searched my archives for messages with "HELO" in the subject line. 

Those are not the only relevant messages. If you don't like the time it
takes to do a full-text search, blame yourself (and Klensin, of course)
for not keeping track of DRUMS discussions.

> objection.

There have been public objections from Philip Hazel (28 July 1998),
Robert Elz (5 August 1998), John Myers (6 August 1998), and of course
me---not to mention the decision at the in-person meeting, and the fact
that some widespread clients deliberately start with HELO.

Furthermore, on 30 January 1999, after I pointed out that the problem
still hadn't been fixed, Klensin asked me for a list of

   places where the EHLO preferences should be toned down as indicated
   by the minutes .... If I can get a precise list ... they will be
   fixed; any omissions are errors, not intentional. ... I will try to
   fix things according to principles if that is all I get, but I do
   make mistakes.

I took Klensin at his word, and sent a message to DRUMS explaining in
detail exactly what changes had to be made.

How many people here have ever expressed _support_ for the notion that
clients should be forced to use EHLO? Why was it ever added to the
document in the first place?

> The chair concludes there is rough concensus that the current 

Try again, Chris.

---Dan