Subject: Re: FWIW Connection Sharing Stats
From: Hector Santos <[email-masked]>
To: [email-masked]
Cc: "[email-masked]" <[email-masked]>
Date: Mon, 29 Aug 2011 18:55:59 -0400
List-Id: <ietf-smtp.imc.org>

Murray S. Kucherawy wrote:

> I don't think one can tell whether a particular client is 
> connection caching by observing protocol activity in terms of 
> time between commands/disconnect.  

Who said that was the measurement?  My log scanner is measuring the 
delays at each at the state machine.

> I also don't think one can tell based on those same data whether any 
> delays are caused by deliberate software action, CPU scheduling, 
> network latency, or anything else.  

Sure you can, when its persistent to the 100th of the microsecond. 
Maybe your logs are 1 second resolutions.  Ours are not and was change 
precisely for measuring where time is spent across all points in the 
SMTP state machine.  Facebook is persistently around 5.01 second and 
Google around 30.05.  Same with others.

> I might make that conclusion if I saw a client sending periodic 
> RSET or NOOP commands, but I haven't seen any evidence of such.

But no one said they were using a KEEP-ALIVE concept.  What I seen 
with RSET is that its either used or not, because it can start with 
MAIL FROM.

> If someone thinks (for example) that Facebook might be connection 
> caching and wants to find out for sure, one could always go and ask them.

That answers only their behavior, the general issue and concern. Even 
if we did ask them, the answer either way doesn't change the picture. 
  It might suggest though that the they realize CS is not efficient 
and multi-threaded or multi-concurrent connections are better - why? 
For the same reason you think every system is capable of handling any 
load thrown at them.

> I also have my doubts that the bad guys are implementing 
> connection caching.  

Yet, the evidence is showing some are delaying and others are not. 
That generalization is simply not realistic.

Data does not lie. Common sense and basic science doesn't either. If 
GOOGLE is using 30 seconds per QUIT with 99.99% of the time there is 
no multi-transaction within that time frame, its a waste and if a 
server pulled the rug to 5 seconds, there will be a tremendous amount 
of resources save - even for the largest.

> This is starting to sound like a conversation the ASRG might enjoy 
> rather than this forum.

QUICKSAND!

PS: Today's stats shows an average of 1.5 secs QUIT which is a 50% 
reduction from the month average before lowering the QUIT idle time 
from 5 to 1 min.  We have more idle timeouts and yet to see a 
connection throttle (which by the way, our license limit is 25 per 
sec, again, per sec, probably means nothing to you).

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: [email-masked]
Cc: "[email-masked]" <[email-masked]>
Date: Mon, 29 Aug 2011 14:59:36 -0700 (PDT)
Subject: RE: FWIW Connection Sharing Stats
To: "Murray S. Kucherawy" <[email-masked]>
Sender: [email-masked]

+1 to all of Murray's points here, including the final one about the ASRG ;-)

				Ned

> I don't think one can tell whether a particular client is connection caching
> by observing protocol activity in terms of time between commands/disconnect.  I
> also don't think one can tell based on those same data whether any delays are
> caused by deliberate software action, CPU scheduling, network latency, or
> anything else.  I might make that conclusion if I saw a client sending periodic
> RSET or NOOP commands, but I haven't seen any evidence of such.

> If someone thinks (for example) that Facebook might be connection caching and
> wants to find out for sure, one could always go and ask them.

> I also have my doubts that the bad guys are implementing connection caching. 
> Spammers have such enormous compute and network power at their disposal (when
> using botnets, at least) that they are probably not thinking too much about
> efficiency in terms of establishing new connections, which only adds complexity
> > and makes their code bigger.  And, as far as I'm aware, bots typically try to
> do direct connections rather than relaying, and they tend to carry their own
> MTAs and even TCP stacks with them to avoid detection.  That means they're not
> using the open source MTAs that tend to implement connection caching, so those
> connections that do show some idle time are probably not the bad guys.

> So, so far, I haven't seen anything that warrants consideration in terms of
> updating a standard.  I suggest that there are way too many assumptions being
> made about the data to warrant much action from the IETF, even something
> informational.

> This is starting to sound like a conversation the ASRG might enjoy rather
> than this forum.

Date: Mon, 29 Aug 2011 17:08:27 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: The purpose of QUIT
Sender: [email-masked]

Keith Moore wrote:
> On Aug 28, 2011, at 3:59 PM, Hector Santos wrote:

>> I would like to hear what was the (DRUMS) thinking towards 
>> making it a MUST.
> 
> I don't remember much of the DRUMS discussion.  But it may be helpful
> to remember that SMTP is not inherently tied to TCP, and SMTP has been 
> implemented over X.25, DECnet, and I'm sure some other protocols.  An 
> explicit QUIT is useful in the cases where the underlying transport doesn't 
> have a clean "close" operation.

This has always been an infrastructure issue, especially under X.25 
where costly upgrades from PADS to "Smart" PADS was required to allow 
interactive applications to work better. The smart pads were able to 
auto negotiated the channel/line settings and that also included how 
vendors developed their virtual comm I/O drivers, proper cabling was 
always important, etc.

So it always was a file/data transfer protocol issue having an "EOD" 
concept because "drops" were a reality.  For SMTP, that was covered 
with its EOD response.

Anyway, reading DRUMS rev0 to RFC2821, you can see how it evolved 
during 1995-2000 - the struggles and conflicts emerging.  TURN was 
deprecated with reflected the dual roles concepts of client/server 
protocols, and less need for hubs to hold mail (I see a security 
reason, but this was already becoming an old concept).  You can see 
how "Queuing Strategies" evolved to "Retries Strategies" but continued 
to mixed two related but different functionalities and you can see 
there was a debate regarding QUIT.  D.J. Bernstein had this to say:

    http://cr.yp.to/smtp/quit.html

    Most implementors agree that QUIT is useless, and support a
    transition to a world without QUIT. Some servers pester their
    system administrators when they don't see QUIT; I recommend that
    they change this behavior. Someday clients will no longer send
    QUIT. Eventually servers will no longer need to support QUIT.

The prediction was not true with RFC22821 compliant clients always 
issuing QUIT.   But I worry that it can be true once the higher 
consumption of idle time is realized more with the growth of large 
volume emailer like FaceBook and Google and others continue to grow.

Anyway, I read other notes about the pervasiveness of Sendmail so I 
believe that the main push for QUIT was to make sure the the SERVER 
does not drop the line and allow the client to drive the server for 
holding for mail.  I believe if this purpose was explicitly stated, 
then there would of been consensus problem with many, like D.J. that 
would of fought harder for a no QUIT or perhaps, it could of provided 
people an "Ah HA, I see now why it is needed" too.

That is where I think we are at today. We need to spell it out why a 
QUIT requirement is needed - to assist clients to hold for mail.  This 
will help the larger emailers and receivers to better prepare for it. 
Otherwise, I can see more implementators simply taking matters into 
their own hand. I don't like the idea of offering a smaller idle time 
to pull the rug from under the feet of the clients that don't do the 
holding efficiently.

I even think that the "Retries Strategies" section should be split 
into two:

     Queuing Strategies
     Retries Strategies

They are really two different ideas, related but fundamentally different.

and the we need to add more text in "Receiver Strategies."

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Mon, 29 Aug 2011 11:59:48 -0700
Subject: RE: FWIW Connection Sharing Stats
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

--_000_F5833273385BB34F99288B3648C4F06F13512DF955EXCHC2corpclo_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I don't think one can tell whether a particular client is connection cachin=
g by observing protocol activity in terms of time between commands/disconne=
ct.  I also don't think one can tell based on those same data whether any d=
elays are caused by deliberate software action, CPU scheduling, network lat=
ency, or anything else.  I might make that conclusion if I saw a client sen=
ding periodic RSET or NOOP commands, but I haven't seen any evidence of suc=
h.

If someone thinks (for example) that Facebook might be connection caching a=
nd wants to find out for sure, one could always go and ask them.

I also have my doubts that the bad guys are implementing connection caching=
.  Spammers have such enormous compute and network power at their disposal =
(when using botnets, at least) that they are probably not thinking too much=
 about efficiency in terms of establishing new connections, which only adds=
 complexity and makes their code bigger.  And, as far as I'm aware, bots ty=
pically try to do direct connections rather than relaying, and they tend to=
 carry their own MTAs and even TCP stacks with them to avoid detection.  Th=
at means they're not using the open source MTAs that tend to implement conn=
ection caching, so those connections that do show some idle time are probab=
ly not the bad guys.

So, so far, I haven't seen anything that warrants consideration in terms of=
 updating a standard.  I suggest that there are way too many assumptions be=
ing made about the data to warrant much action from the IETF, even somethin=
g informational.

This is starting to sound like a conversation the ASRG might enjoy rather t=
han this forum.

From: [email-masked] [mailto:[email-masked]] On=
 Behalf Of Keith Moore
Sent: Saturday, August 27, 2011 7:31 AM
To: Keith Moore
Cc: Hector Santos; [email-masked]
Subject: Re: FWIW Connection Sharing Stats

On Aug 27, 2011, at 9:53 AM, Keith Moore wrote:

On Aug 26, 2011, at 8:18 PM, Hector Santos wrote:

Among the spammers rejected with delays, many do try a new transaction with=
 the same failed result.  Is this CS client behavior? I seems to appear tha=
t way.

Maybe the spammers are trying to deal with greylisting?

For that matter, I wonder if legitimate senders are trying to deal with gre=
ylisting also.  Maybe there are poor implementations of greylisting that bl=
ock legitimate traffic too often.  Maybe there are some large-volume sender=
s that don't want to deal with having greylisted mail in their queues any l=
onger than necessary.  If retrying after a few seconds works on some greyli=
sts, it's not surprising that some senders would start doing it.

Keith

--_000_F5833273385BB34F99288B3648C4F06F13512DF955EXCHC2corpclo_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40"><head><meta http-equiv=3DContent-Type content=
=3D"text/html; charset=3Dus-ascii"><meta name=3DGenerator content=3D"Micros=
oft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
	{font-family:Helvetica;
	panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
	{font-family:"Cambria Math";
	panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
	{font-family:Calibri;
	panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
	{font-family:Tahoma;
	panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:12.0pt;
	font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
	{mso-style-priority:99;
	color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{mso-style-priority:99;
	color:purple;
	text-decoration:underline;}
span.apple-style-span
	{mso-style-name:apple-style-span;}
span.EmailStyle18
	{mso-style-type:personal-reply;
	font-family:"Calibri","sans-serif";
	color:#1F497D;}
.MsoChpDefault
	{mso-style-type:export-only;
	font-size:10.0pt;}
@page WordSection1
	{size:8.5in 11.0in;
	margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
	{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-US link=3Dblue vli=
nk=3Dpurple style=3D'word-wrap: break-word;-webkit-nbsp-mode: space;-webkit=
-line-break: after-white-space'><div class=3DWordSection1><p class=3DMsoNor=
mal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";colo=
r:#1F497D'>I don&#8217;t think one can tell whether a particular client is =
connection caching by observing protocol activity in terms of time between =
commands/disconnect.&nbsp; I also don&#8217;t think one can tell based on t=
hose same data whether any delays are caused by deliberate software action,=
 CPU scheduling, network latency, or anything else.&nbsp; I might make that=
 conclusion if I saw a client sending periodic RSET or NOOP commands, but I=
 haven&#8217;t seen any evidence of such.<o:p></o:p></span></p><p class=3DM=
soNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"=
;color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><span styl=
e=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If =
someone thinks (for example) that Facebook might be connection caching and =
wants to find out for sure, one could always go and ask them.<o:p></o:p></s=
pan></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"C=
alibri","sans-serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3D=
MsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif=
";color:#1F497D'>I also have my doubts that the bad guys are implementing c=
onnection caching.&nbsp; Spammers have such enormous compute and network po=
wer at their disposal (when using botnets, at least) that they are probably=
 not thinking too much about efficiency in terms of establishing new connec=
tions, which only adds complexity and makes their code bigger.&nbsp; And, a=
s far as I&#8217;m aware, bots typically try to do direct connections rathe=
r than relaying, and they tend to carry their own MTAs and even TCP stacks =
with them to avoid detection.&nbsp; That means they&#8217;re not using the =
open source MTAs that tend to implement connection caching, so those connec=
tions that do show some idle time are probably not the bad guys.<o:p></o:p>=
</span></p><p class=3DMsoNormal><span style=3D'font-size:11.0pt;font-family=
:"Calibri","sans-serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=
=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-se=
rif";color:#1F497D'>So, so far, I haven&#8217;t seen anything that warrants=
 consideration in terms of updating a standard.&nbsp; I suggest that there =
are way too many assumptions being made about the data to warrant much acti=
on from the IETF, even something informational.<o:p></o:p></span></p><p cla=
ss=3DMsoNormal><span style=3D'font-size:11.0pt;font-family:"Calibri","sans-=
serif";color:#1F497D'><o:p>&nbsp;</o:p></span></p><p class=3DMsoNormal><spa=
n style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497=
D'>This is starting to sound like a conversation the ASRG might enjoy rathe=
r than this forum.<o:p></o:p></span></p><p class=3DMsoNormal><span style=3D=
'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p>&n=
bsp;</o:p></span></p><div style=3D'border:none;border-left:solid blue 1.5pt=
;padding:0in 0in 0in 4.0pt'><div><div style=3D'border:none;border-top:solid=
 #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=3DMsoNormal><b><span sty=
le=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><=
span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> owner-ie=
[email-masked] [mailto:[email-masked]] <b>On Behalf Of =
</b>Keith Moore<br><b>Sent:</b> Saturday, August 27, 2011 7:31 AM<br><b>To:=
</b> Keith Moore<br><b>Cc:</b> Hector Santos; [email-masked]<br><b>Subje=
ct:</b> Re: FWIW Connection Sharing Stats<o:p></o:p></span></p></div></div>=
<p class=3DMsoNormal><o:p>&nbsp;</o:p></p><div><div><p class=3DMsoNormal>On=
 Aug 27, 2011, at 9:53 AM, Keith Moore wrote:<o:p></o:p></p></div><p class=
=3DMsoNormal><br><br><o:p></o:p></p><div><div><div><p class=3DMsoNormal>On =
Aug 26, 2011, at 8:18 PM, Hector Santos wrote:<o:p></o:p></p></div><p class=
=3DMsoNormal><br><br><o:p></o:p></p><p class=3DMsoNormal><span class=3Dappl=
e-style-span><span style=3D'font-size:13.5pt;font-family:"Helvetica","sans-=
serif"'>Among the spammers rejected with delays, many do try a new transact=
ion with the same failed result. &nbsp;Is this CS client behavior? I seems =
to appear that way.</span></span><span style=3D'font-size:13.5pt;font-famil=
y:"Helvetica","sans-serif"'><br><br></span><o:p></o:p></p></div><p class=3D=
MsoNormal><o:p>&nbsp;</o:p></p><div><p class=3DMsoNormal>Maybe the spammers=
 are trying to deal with greylisting?<o:p></o:p></p></div></div><p class=3D=
MsoNormal><o:p>&nbsp;</o:p></p></div><div><p class=3DMsoNormal>For that mat=
ter, I wonder if legitimate senders are trying to deal with greylisting als=
o. &nbsp;Maybe there are poor implementations of greylisting that block leg=
itimate traffic too often. &nbsp;Maybe there are some large-volume senders =
that don't want to deal with having greylisted mail in their queues any lon=
ger than necessary. &nbsp;If retrying after a few seconds works on some gre=
ylists, it's not surprising that some senders would start doing it.<o:p></o=
:p></p></div><div><p class=3DMsoNormal><o:p>&nbsp;</o:p></p></div><div><p c=
lass=3DMsoNormal>Keith<o:p></o:p></p></div><div><p class=3DMsoNormal><o:p>&=
nbsp;</o:p></p></div></div></div></body></html>=

--_000_F5833273385BB34F99288B3648C4F06F13512DF955EXCHC2corpclo_--

Subject: Re: The purpose of QUIT
From: Keith Moore <[email-masked]>
Date: Mon, 29 Aug 2011 10:31:35 -0400
Cc: [email-masked]
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On Aug 28, 2011, at 3:59 PM, Hector Santos wrote:

> 
> In the 1982, but still currently the official SMTP standard, STD10 (RFC821) has QUIT as a SHOULD. In the 2000 document RFC2821 and the newer (2009) RFC5321, it was made into a MUST.
> 
> I would like to hear what was the (DRUMS) thinking towards making it a MUST.

I don't remember much of the DRUMS discussion.  But it may be helpful to remember that SMTP is not inherently tied to TCP, and SMTP has been implemented over X.25, DECnet, and I'm sure some other protocols.  An explicit QUIT is useful in the cases where the underlying transport doesn't have a clean "close" operation.

Keith

Date: Sun, 28 Aug 2011 20:07:31 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: The purpose of QUIT
Sender: [email-masked]

Robert A. Rosenberg wrote:
>> I would like to hear what was the (DRUMS) thinking towards making it a 
>> MUST.
> 
> While I do not know DRUMS reason, I can think of one reason to require 
> the QUIT. It signals that the client is done and has received all of its 
> mail and has logged all of them. If I remember correctly the deletes of 
> mail sent from the server to the client un response to the client's DEL 
> commands is performed when the QUIT is received by the Server not as 
> each DEL is received. If the connection drops before that, the deletes 
> will not occur and the messages will be offered again at the next 
> connection.

Yes, as Klensin noted, that is the POP3 design. Another posted brought 
up the idea that this could be part of the "confusion" and reason for 
equating POP3 QUIT behavior with SMTP.

Under POP3, its classified as moving the server into UPDATE mode where 
it can do the final markings of the messages received or delete 
status.  The idea of supporting "Roaming Users" is central to all 
this.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 28 Aug 2011 19:46:02 -0400
From: John C Klensin <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>, Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: The purpose of QUIT
Sender: [email-masked]

--On Sunday, August 28, 2011 19:17 -0400 "Robert A. Rosenberg"
<[email-masked]> wrote:

> While I do not know DRUMS reason, I can think of one reason to
> require the QUIT. It signals that the client is done and has
> received all of its mail and has logged all of them. If I
> remember correctly the deletes of mail sent from the server to
> the client un response to the client's DEL commands is
> performed when the QUIT is received by the Server not as each
> DEL is received. If the connection drops before that, the
> deletes will not occur and the messages will be offered again
> at the next connection.

This sounds like you are talking about POP3 and not SMTP.

   john

Date: Sun, 28 Aug 2011 19:17:00 -0400
To: Hector Santos <[email-masked]>
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: The purpose of QUIT
Cc: [email-masked]
Sender: [email-masked]

At 15:59 -0400 on 08/28/2011, Hector Santos wrote about The purpose of QUIT:

>In the 1982, but still currently the official SMTP standard, STD10 
>(RFC821) has QUIT as a SHOULD. In the 2000 document RFC2821 and the 
>newer (2009) RFC5321, it was made into a MUST.
>
>I would like to hear what was the (DRUMS) thinking towards making it a MUST.
>
>[snip]
>
>--
>Sincerely
>
>Hector Santos
>http://www.santronics.com

While I do not know DRUMS reason, I can think of one reason to 
require the QUIT. It signals that the client is done and has received 
all of its mail and has logged all of them. If I remember correctly 
the deletes of mail sent from the server to the client un response to 
the client's DEL commands is performed when the QUIT is received by 
the Server not as each DEL is received. If the connection drops 
before that, the deletes will not occur and the messages will be 
offered again at the next connection.

Date: Sun, 28 Aug 2011 18:47:41 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: I-D Outline for "Issues and Enhancements for modern SMTP environments"
Sender: [email-masked]

I am outlining an I-D that hopefully covers all of the main issues 
discussed here regarding recent discoveries surrounding the concept of 
"time" in SMTP and how it helped explains many of the long time 
various issues that we had to observed and contend with over the years.

Hopefully, it can serve as a guide for better implementing SMTP, even 
improve mail delivery, understand GOOD senders better and maybe serve 
to help update SMTP.  Although I realize there are  folks who will 
ignore this, but I feel confident (I have too, otherwise I need to 
retire) there are those that can put aside the side-issues and feel 
compel to contribute. So if you wish to comment here or in private 
with your suggestions, that would be very helpful to getting the 
widest input. I have unfortunately no faith it will go anywhere until 
someone else authored it. So maybe it can be a starter for someone 
else to take over the I-D, which would be my preference as long as 
alienation is not the result.

Anyway, here is an initial draft outline (which can be reduced) for an 
I-D, I have initially titled:

    	Issues and Enhancements for modern SMTP environments
          Subtitle: Dealing with a good and bad mail senders.

         (note the plurality of "modern SMTP environments")
	
         (Keep in mind I am trying to do this as general as purpose)

o Background

    o Traditional Mailer Designs
       - Email vs Groupmail
       - Immediate vs Delayed Scheduling, Delivery
    o Timeouts
    o Hold Times for new mail
      - Mail Queuing
      - Connection Sharing
    o Purpose of Quit
    o Extended SMTP Overhead issues
      - TLS, AUTH
      - DATA Filtering
    o Scalability issues
      o Increase resources
      o Loading Limits
      o Service Availability
    o Socket TIME_WAIT state
      o Understanding the TCP state machine
      o Who closes first? Server or Client
      o Port Reusability

o Recommendations

    o Adhering to SMTP standards and recommendations
       - High Efficiency
       - High Throughputs
       - Lower IDLE SMTP Waste
    o Shorter Timeouts
      - when allowing higher timeouts is "good" or "bad"
    o Negotiated Bulk Mail exchanges
      - Using current methods
      - Ideas for new SMTP extensions
    o How to use the QUIT command
    o Recommended updates to RFC5321

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 28 Aug 2011 15:59:59 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: The purpose of QUIT
Sender: [email-masked]

In the 1982, but still currently the official SMTP standard, STD10 
(RFC821) has QUIT as a SHOULD. In the 2000 document RFC2821 and the 
newer (2009) RFC5321, it was made into a MUST.

I would like to hear what was the (DRUMS) thinking towards making it a 
MUST.

Other than the general robustness and gracefulness it provides a 
client/server handshake, it plays no rule in the mail delivery or 
cancellation. Clients and servers can drop with any mail harm. I can 
only see one technical reason for that will benefit the SERVER or 
CLIENT and thats dealing with a socket port TIME_WAIT TCP state.   In 
short, he who closes first, will move the port in TIME_WAIT and unless 
the port is prepared for reusability, it can not be used again into 
TIME_WAIT expires.

Under STD10, it helped for a faster throughput because the client can 
just send, get the 250 and immediately drop.  Since the client 
dropped, the server-side port is not in TIME_WAIT and it better scale 
the server in regards to available ports.  However, the client may be 
in a disadvantage in regards to scaling available ports.

Under RFC5321, with QUIT, we are not really sure who benefits 
depending on the software on both ends waiting for closure by the 
other side.  So if the client issues a QUIT, it can wait for the 
SERVER to close the socket.  Or the Server sees the QUIT and waits for 
the client to close the socket.  Both sides will need to add a sanity 
timeout check to close the socket if the either end doesn't close it 
first - a state machine contention conflict.

Now when we throw in the idea of intentional delays, "Online Hold for 
New mail" or Connection Sharing,  it adds a new purpose for the server 
to wait for a QUIT.

Neither ideas are documented in RFC5321 and maybe in the classic IETF 
wisdom of leaving things open ended, it was left unmentioned and just 
say NO SIDE SHOULD NOT DROP the connection until QUIT is negotiated.

However, it does seem after all these years that there are many 
repercussions, ambiguity, clearly ways to deal with abusers, loosely 
followed parts of the SMTP spec, higher complexity for scale designs 
between an old era and new era all centered around this basic issue of 
timeouts, delays, socket closure and QUIT that it may be time to 
document the reasons why the QUIT requirement is desirable or at least 
better define its purpose.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sat, 27 Aug 2011 16:46:29 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: FWIW Connection Sharing Stats
To: [email-masked]
Sender: [email-masked]

Keith Moore followed upe:

>> Maybe the spammers are trying to deal with greylisting?
> 
> For that matter, I wonder if legitimate senders are trying to deal with 
> greylisting also.  Maybe there are poor implementations of greylisting that 
> block legitimate traffic too often.  Maybe there are some large-volume senders 
> that don't want to deal with having greylisted mail in their queues any longer 
> than necessary.  If retrying after a few seconds works on some greylists, it's 
> not surprising that some senders would start doing it.

I agree.

So anonymous bad guys hurt receivers, receiver invent ways to throttle 
the abuse using  methods that forces SMTP compliancy, anonymous good 
guys must also comply and must also follow the throttle invention. Bad 
guys finds a loop hole around the throttling invention.  Receivers 
find and close the bad guy loop hole unbeknowst to them that they are 
also closing the loophole also used by the good guys.

Serendipitously, the receiver finds there is a intentionally delay 
used by good guys and its built into the software they are using and 
discovers that this delay pattern is also exhibited by bad guys, 
leading to a reasonable conclusion that all prior thinking was 
incorrect and that the good and bad are using the same type of 
software with the intentional delays. The good uses it inefficiently 
for good bulk mail intentions and the bad uses its inefficiently for 
the same bad bulk spam reasons and to get around greylist rejections.

The receivers are now stuck in a conundrum and catch-22 situation in 
dealing with this situation that is only solvable by reducing the idle 
CPU waste time by dropping all clients after the 1st transaction 
attempt is completed - the good and bad.

I will be showing after I get more daily stats, but yesterday I 
enabled the new option to reduce the QUIT time idle time from 5 to 1 
min and I already saw a 60% reduction in lost CPU idle time waste.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sat, 27 Aug 2011 12:57:21 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: FWIW Connection Sharing Stats
To: [email-masked]
Sender: [email-masked]

Keith Moore followed up:

> On Aug 27, 2011, at 9:53 AM, Keith Moore wrote:
>> Maybe the spammers are trying to deal with greylisting?
> 
> For that matter, I wonder if legitimate senders are trying to deal 
> with greylisting also.  Maybe there are poor implementations of greylisting 
> that block legitimate traffic too often.  Maybe there are some large-volume 
> senders that don't want to deal with having greylisted mail in their queues 
> any longer than necessary.  If retrying after a few seconds works on 
> some greylists, it's not surprising that some senders would start 
doing it.

Agree, which makes it part of the dilemma because all (anonymous, 
unauthenticated, unauthorized, non-prearranged) clients are now lumped 
with "The Good, The Bad and the Ugly."  Just saw the classic movie 
again the other day. :-)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sat, 27 Aug 2011 10:41:21 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: FWIW Connection Sharing Stats
To: [email-masked]
Sender: [email-masked]

Keith Moore wrote:
> On Aug 26, 2011, at 8:18 PM, Hector Santos wrote:
> 
>> Among the spammers rejected with delays, many do try a new transaction 
>> with the same failed result.  Is this CS client behavior? It seems to 
>> appear that way.
> 
> Maybe the spammers are trying to deal with greylisting?

Yes and some are Keith.  This was discovered as a greylist loop hole 
as you may recall with the thread back in May - "RSET - possible 
security loophole."  It raised the discussion level regarding the 
recording/tracking/clearing differences of Session Level vs 
Transaction Level status info.

I added logic to my greylister filter to pass back this rejection 
status to SMTP to disable all any more transactions at the section 
level. I didn't have it just drop per SMTP.  I have logs to show it 
works to prevent these greylisted client attempting to hold and to try 
again.  It also was great to see how it stop qreylist rejected clients 
not delaying but trying other different MAIL/RCPT transactions.  I 
have one of these yesterday that no delays

09:19:00.654 C: MAIL FROM:<[email-masked]>
09:19:00.655 S: 250 <[email-masked]> ok
09:19:00.720 C: RCPT TO:<[email-masked]>
09:19:00.894 S: 250 <[email-masked]>... Recipient ok
09:19:00.940 C: DATA
09:19:00.941 S: 354 Start mail input; end with <CRLF>.<CRLF>
09:19:01.055 ** end of data received. (bytes: 4166) (31.9 K/s)
09:19:01.409 ** WCX Process: SmtpFilterHookLoader  ret: 0 (354 msecs)
09:19:01.409 S: 451 Please try again later (TempFail)
09:19:01.455 C: RSET
09:19:01.455 S: 250 Reset State #1
09:19:01.502 C: MAIL FROM:<[email-masked]>
09:19:01.502 S: 451 Transactions disabled due to policy.
09:19:01.548 C: QUIT
09:19:01.548 S: 221 closing connection
09:19:01.563 ** Completed

But now I am wondering if we gave them the ones that did delay too 
much credit and it was just client software behavior.  I see the 
following types with 2nd transactions:

    1) same MAIL and RCPT
    2) same MAIL and different RCPT
    3) different MAIL and/or RCPT

By far, I see #2 and #3, but with no CS mentality, you would think the 
first one was the greylist loophole I observed.  With a CS mentality, 
it may not be a loop hole per see, instead a result of client software 
behavior possibly seeing the same queued file remaining X secs later. 
  A honest explanation or indeed they were clever to see use CS to 
help get around greylisting without scheduling a new connection.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Subject: Re: FWIW Connection Sharing Stats
From: Keith Moore <[email-masked]>
Date: Sat, 27 Aug 2011 10:31:24 -0400
Cc: Hector Santos <[email-masked]>, "[email-masked]" <[email-masked]>
To: Keith Moore <[email-masked]>
Sender: [email-masked]

--Apple-Mail-3--1006647762
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On Aug 27, 2011, at 9:53 AM, Keith Moore wrote:

> On Aug 26, 2011, at 8:18 PM, Hector Santos wrote:
>=20
>> Among the spammers rejected with delays, many do try a new =
transaction with the same failed result.  Is this CS client behavior? I =
seems to appear that way.
>=20
> Maybe the spammers are trying to deal with greylisting?

For that matter, I wonder if legitimate senders are trying to deal with =
greylisting also.  Maybe there are poor implementations of greylisting =
that block legitimate traffic too often.  Maybe there are some =
large-volume senders that don't want to deal with having greylisted mail =
in their queues any longer than necessary.  If retrying after a few =
seconds works on some greylists, it's not surprising that some senders =
would start doing it.

Keith

--Apple-Mail-3--1006647762
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On Aug 27, 2011, at 9:53 AM, Keith Moore wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space; "><div><div>On Aug 26, 2011, at =
8:18 PM, Hector Santos wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><span =
class=3D"Apple-style-span" style=3D"border-collapse: separate; =
font-family: Helvetica; font-style: normal; font-variant: normal; =
font-weight: normal; letter-spacing: normal; line-height: normal; =
orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: =
none; white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
auto; -webkit-text-stroke-width: 0px; font-size: medium; ">Among the =
spammers rejected with delays, many do try a new transaction with the =
same failed result. &nbsp;Is this CS client behavior? I seems to appear =
that way.<br></span></blockquote></div><br><div>Maybe the spammers are =
trying to deal with =
greylisting?</div></div></blockquote><br></div><div>For that matter, I =
wonder if legitimate senders are trying to deal with greylisting also. =
&nbsp;Maybe there are poor implementations of greylisting that block =
legitimate traffic too often. &nbsp;Maybe there are some large-volume =
senders that don't want to deal with having greylisted mail in their =
queues any longer than necessary. &nbsp;If retrying after a few seconds =
works on some greylists, it's not surprising that some senders would =
start doing =
it.</div><div><br></div><div>Keith</div><div><br></div></body></html>=

--Apple-Mail-3--1006647762--

Subject: Re: FWIW Connection Sharing Stats
From: Keith Moore <[email-masked]>
Date: Sat, 27 Aug 2011 09:53:46 -0400
Cc: "[email-masked]" <[email-masked]>
To: Hector Santos <[email-masked]>
Sender: [email-masked]

--Apple-Mail-1--1008905886
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On Aug 26, 2011, at 8:18 PM, Hector Santos wrote:

> Among the spammers rejected with delays, many do try a new transaction =
with the same failed result.  Is this CS client behavior? I seems to =
appear that way.

Maybe the spammers are trying to deal with greylisting?

Keith

--Apple-Mail-1--1008905886
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On Aug 26, 2011, at 8:18 PM, Hector Santos wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><span =
class=3D"Apple-style-span" style=3D"border-collapse: separate; =
font-family: Helvetica; font-style: normal; font-variant: normal; =
font-weight: normal; letter-spacing: normal; line-height: normal; =
orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: =
none; white-space: normal; widows: 2; word-spacing: 0px; =
-webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: =
0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: =
auto; -webkit-text-stroke-width: 0px; font-size: medium; ">Among the =
spammers rejected with delays, many do try a new transaction with the =
same failed result. &nbsp;Is this CS client behavior? I seems to appear =
that way.<br></span></blockquote></div><br><div>Maybe the spammers are =
trying to deal with =
greylisting?</div><div><br></div><div>Keith</div><div><br></div></body></h=
tml>=

--Apple-Mail-1--1008905886--

Date: Fri, 26 Aug 2011 20:42:44 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: FWIW Connection Sharing Stats
Sender: [email-masked]

Hector Santos wrote:
> Murray S. Kucherawy wrote:
> 
>> How are you able to distinguish between an SMTP client that's 
>> implementing connection sharing and one that is not?
> 
> Good and fair question which highlights part of the dilemma.
> ...
>
> On the other hand, how do you explain Facebook when two clients arriving 
> at the same precise microsecond and each sent 1 message and each waited 
> 5 secs to QUIT.  Why didn't it use one session for these two messages?   
> I can only presume it could be a configuration issue or two different 
> clients looked at two different queues in their outbound farm.  Who knows?

Or Murray, there is another reasonable explanation for this FaceBook 
behavior:

       They found it is better not use CS (all or partly) but
       forgot to remove the QUIT delay in cases it does.

In other words, they can send two concurrent messages in 5 secs less 
total time, then it would take to send two messages under 1 session 
which would have at least two 5 secs waits.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Fri, 26 Aug 2011 20:18:58 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: FWIW Connection Sharing Stats
Sender: [email-masked]

Murray S. Kucherawy wrote:

> How are you able to distinguish between an SMTP client 
> that's implementing connection sharing and one that is not?

Good and fair question which highlights part of the dilemma.

Sure, we don't know for sure what the clients are actually doing. All 
I have is the data in front of me, a history of unexplained patterns 
attributed only to "bad guys"  behavior and now a new recognition that 
helps explains it all.

If Connection Sharing means that client connects, sends a transaction 
or get rejected at EOD, holds for X time and tries again or issues 
QUIT, then:

    I have very few of multi-transactions that have QUIT delays, and
    a good significant amount with QUIT delays between 5 and 260 secs
    with 1 transaction or rejected transaction.

Among the spammers rejected with delays, many do try a new transaction 
with the same failed result.  Is this CS client behavior? I seems to 
appear that way.

So the lack of multi-transaction among the domains with delays, lowers 
slightly but not taken the idea they are probably software with an 
online hold for mail concept.

The other clue is the persistent nature of the delay values among the 
sessions from the same domains, like Facebook with 5 seconds, google 
with 30, hoffman with 6, even with spammer domains, etc.

I saw only hoffman send multi-transaction sessions and google with 30, 
I did a manual test by logging on o gmail.com to send two messages ~10 
seconds apart.   On the server side, a single session logs showed the 
1st transaction, then 10 secs later the 2nd transaction was sent. So I 
will google is using the "Connection Sharing" or like idea.

On the other hand, how do you explain Facebook when two clients 
arriving at the same precise microsecond and each sent 1 message and 
each waited 5 secs to QUIT.  Why didn't it use one session for these 
two messages?   I can only presume it could be a configuration issue 
or two different clients looked at two different queues in their 
outbound farm.  Who knows?

But because of Facebook's persistent nature of 5 secs delay, I will 
tend to think it is using this for CS but for some reason its not 
working for them. Otherwise what is their purpose of waiting 5 secs?

All I can conclude is:

   - The bad guys are also using the same good guys software
     with this "Connection Sharing" idea because it explains the
     common behavior among all with these delays,

   - The higher delay is needed to better pick up 2 messages
     making it worth while for the client, but

   - They are not tuned to associate rates vs the hold time
     which is probably very much dependent on the frequency
     of mail going to a host receiver.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Fri, 26 Aug 2011 13:01:23 -0700
Subject: RE: FWIW Connection Sharing Stats
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Friday, August 26, 2011 12:47 PM
> To: [email-masked]
> Subject: FWIW Connection Sharing Stats
> 
> Some of the known domains with high CS delays:

How are you able to distinguish between an SMTP client that's implementing connection sharing and one that is not?

Date: Fri, 26 Aug 2011 15:47:00 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: FWIW Connection Sharing Stats
Sender: [email-masked]

For July:

connects         : 31228
timeouts         : 57
too many connects: 28
auth             : 165
auth time        : 73 msecs
total tls        : 0
tls time         : 0 msecs    < TLS was not enabled
cs time rset     : 263.850  secs
cs time quit     : 18695.359 secs  (5.9 hours of IDLE CPU waste)
data rset time   : 24 msecs
data quit time   : 1701 msecs

For August:

connects         : 25494
timeouts         : 90
too many connects: 259
auth             : 238
auth time        : 78 msecs
total tls        : 67
tls time         : 698 msecs
cs time rset     : 183.326 sec
cs time quit     : 25618.925 sec  (7.1 hours of IDLE CPU waste)
data rset time   : 16 msecs
data quit time   : 2283 msecs

Some of the known domains with high CS delays:

    Facebook.com             5 secs
    moveon.org               5 secs
    Google.com              30 secs
    hoffman.proper.com    6-12 secs
    aol.com                  6 secs to 263 secs
    qualcomm.com             5 secs

The majority have QUIT delays of less than 1 seconds. The majority of 
the CS 1+ delay were 5 secs. Sorting and Eyeballing, it looks like 
more of the high ones from 60 to 260 secs are spammers.

The #1 difficulty is its anonymous nature thus all of these clients 
are lumped with The Good, The Bad and the Ugly and having trouble why 
we are allowing an implementation issue dictate how SMTP is implemented.

Putting aside the negative aspect, it appears to me that the lower the 
Delay, the less efficient it is for the CS side.  For example, 
Facebook with 5 secs delay never had a multi-transaction session. It 
assumes the a single or a collection of users with the same host are 
generated at at a faster rather than 5 seconds.  My wife made an 
interesting point that she didn't thin Facebook was sending comments 
within 5 seconds.  But if that was the case, Faceback is delaying to 
build up comments.  So I check the looks to look at the time 
differences between sessions and they varied but most interesting was 
seeing a few concurrent message (same time to the msec) and they were 
separate single message sections.  Why wouldn't it do a CS here?

On the other hand, there were logs with a few google.com sessions and 
its 30 sec delay. I also manually tested it by creating two gmail.com 
10 secs apart and once the first session and send the transaction, 10 
secs later the 2nd transaction came in.

So disregarding the negatives, it seems higher the better but only 
when there is a higher potential to do more work.  The last work (1 
message) all these 30 secs idle times are pure waste on the receiver.

TLS is highly dependent on the distance or hops between endpoints.  I got

     0.015 secs  across 1000gb LAN - lowest
     0.129 secs  VPN 17 hops
     2.697 secs  rejected spammer - highest seen

The 15 ms on the LAN is pretty darn fast for SSL.  So for CS clients 
working on a private network, TLS time should be insignificant 
overhead.  The VPN speed was pretty good tool, given the fact that its 
a 1/2 mile distance yet it traveled across two states (Up Florida, 
into Atlanta Georgia and back down).  The highest was 2.7 secs but the 
average was 0.6 secs among all the other clients.

I question the need for TLS for CS clients especially when they are 
not sending multi-messages.

No CS client can do AUTH on our server, so the above times is internal 
users and thats pretty fast at 78ms

Anyway, it appears to me that even if one deems CS beneficial, they 
are certainly implementing it in an efficient for themselves or for 
us.  7 hours of idle lost time with no work is pretty wasteful - on 
both ends.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 25 Aug 2011 12:47:43 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: productivity?
Sender: [email-masked]

John C Klensin wrote:
> 
> 
> --On Thursday, August 25, 2011 09:49 -0400 Hector Santos
> <[email-masked]> wrote:
> 
>> Sabahattin Gucukoglu wrote:

>> +1.  Or we can begin with a basic tenet of "do no harm."   It
>> is obvious the issues of a well vetting system perform for a
>> past era where hubs were more dominant and using ideas of
>> connection sharing were isolated to well known connections,
>> conflicts in a modern world where there is less hubbing, more
>> peer to peer and anonymous connectivity.  Just consider how
>> Facebook has become a new high mail sender where it needs to
>> improve its client side loads but its distribution is towards
>> a larger pool of smaller smtp servers.  If it was assumed that
>> every facebook user had a common host account, i.e. gmail.com,
>> then we back to the centralization and hub-hub concepts where
>> this stuff was ideal.
> 
> FWIW, while I have some fundamental concerns about this as a
> strategy (and have a different picture of history than the one
> Hector describes above), 

You see, I see that as unnecessary statement and one that continues 
with the dismissing expertise and the concerns. I'm sure we all do and 
have different perspective.  I can only base mine of actual mailer and 
mail/file distribution, hubbing product development experience with:

    Early local community access of BBS Systems
    The networking of BBS systems
    Gateways of different mail networks
    Fidonet
    UUCP
    FTP HUBBING
    SMTP

where by far, the mail/file distribution topology was much different 
than it is today simply because everyone and their grandma has 24x7 
internet access unlike in the past.

In these mailers, there was one common idea:

  - 1 to 1

    Netmail (Fidonet) and Email (internet) - instant, dynamic 
sending/delivery

  - 1 to many

    Groupware, EchoMail (fido), News (internet) - delayed, queued, 
scheduling

Internet List mail is an email kludge form of groupware and some 
people have it naturally fall under a 1 to 1 model already prepared 
netmail/email instant delivery and others have it prepared for 1 to 
many delayed distribution - a simple delay or dependent on frequency 
of list mail. Scheduling was also tied to who the targets were.

Fidonet mailers had built-in multi-message packet exchanges, so that 
was never an issue.  We designed and sold PPP servers, UUCP and SMTP 
products to many of the early ISPs that began to offer internet 
related services to their Online users and hosted domains.  Many 
operators who were privy to direct connections to the internet offered 
internet and SMTP access and for hosting drop off points with UUCP and 
also FTP.  Many operators with FTP drop/receive mail/files exchange 
points, started the growth of many popular products like InternetRex 
that handled all this for operators where it was yet feasible to 
install a 24x7 access point with ISDN or T1. DSL was still in the 
horizon.

Many hubs also used Planet Connect, the first to offer a satellite 
feed for downloading Mail Bones and File Bones feeds with a small home 
dish.  These hubs offer services to other people - free or for a fee. 
  There were some early IETF people, early member of Fidonet, who used 
their academia internet access to piggyback fidonet over the mail 
bones internationally.

All of this was the norm with industry vendors in the early trade 
shows like OneBBS CON which was sold and renamed to OneISP CON 
(ISPCON) and it also passed away in 2009.  This is an interest foot 
note and thread that amply applies here to discussion:

      http://www.dslreports.com/shownews/ISPCON-No-More-101070

      ... The death of ISPCON and ISP Planet serves as kind of a 
footnote to
      the slow but steady death of smaller, independent ISPs in the 
United States.
      "In the long term, carriers will survive, ISPs will not," the 
CEO of one
      ISP carrier tells me.

Never mind the fact that this prediction assumes every corporation in 
the world will switch to GMAIL.COM, the fact that we are still in 
business negates that prediction as a whole, in fact, I see the 
opposite.

What is true is that many smaller ISPs simply lower their desire to 
compete with the larger ISPs - their services were cut and they began 
to focus on the vertical markets needs rather the horizontal. Its the 
economy of scale - you don't need 100% of the market, sometimes 1% or 
even less is good enough.  So while we don't concentrate on a general 
hosting server for people to setup, many do it for private 
communications and special needs.

The point is, data distribution was spread out with different ways and 
most of the time, unless a service operator didn't care, and certainly 
not one where a client presumes a server didn't care, only with 
special access was one able to do these larger mail exchange concepts 
- regardless of the transport mechanism.   The fact that SMTP has 
prevail, does not change the issue.

So sure, people will see "history different" - bull or noise, mine 
came on decades of direct experiences working cooperatively with my 
peers and offering products that served a wide spread of ISPes and 
data distribution needs.  Today, anyone can have a "cheap" 24x7 
connection and preparing a SMTP server host site is very cheap to do.

> Let's get a draft in front of us that spells out the details,
> analyzes the side effects, and that includes a Security
> Considerations section that enumerates and evaluates the
> possible attacks and how to deal with this.  Then we have have a
> real discussion.  Without it, all we have is more noise on the
> list, people making suggestions that other people agree with (or
> not) depending on their perceptions of what is being proposed
> (perceptions that may not exactly match the intentions of the
> original proposal), and so on.

I agree and I would normally be eager to consider writing it myself, 
but unfortunately thats a non-starter so it would be better of someone 
else step up to the plate and begin write it. I would not mind 
starting it if someone else takes it over.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 25 Aug 2011 10:52:22 -0400
From: John C Klensin <[email-masked]>
To: [email-masked]
Subject: Re: productivity?
Sender: [email-masked]

--On Thursday, August 25, 2011 09:49 -0400 Hector Santos
<[email-masked]> wrote:

> Sabahattin Gucukoglu wrote:
> 
>> The answer has got to be a new SMTP extension, or pair of
>> extensions, to  do maximum throughput sending on one
>> connection and multi-message  transaction advisory mechanisms
>> to support co-operation between bulk software  and the MTA
>> for knowing the delivery load in advance, thus neutralising
>> the  discussion over server administrative policy, port or
>> connection slot exhaustion,  client selfishness, time to wait
>> until shutdown, event-driven programming vs  process-based
>> servers, OS wars and all the other nonsense.  We need an 
>> improved checkpointing facility.  That's what we should do, I
>> think.
> 
> +1.  Or we can begin with a basic tenet of "do no harm."   It
> is obvious the issues of a well vetting system perform for a
> past era where hubs were more dominant and using ideas of
> connection sharing were isolated to well known connections,
> conflicts in a modern world where there is less hubbing, more
> peer to peer and anonymous connectivity.  Just consider how
> Facebook has become a new high mail sender where it needs to
> improve its client side loads but its distribution is towards
> a larger pool of smaller smtp servers.  If it was assumed that
> every facebook user had a common host account, i.e. gmail.com,
> then we back to the centralization and hub-hub concepts where
> this stuff was ideal.

FWIW, while I have some fundamental concerns about this as a
strategy (and have a different picture of history than the one
Hector describes above), the bottom line is "write it up".
Let's get a draft in front of us that spells out the details,
analyzes the side effects, and that includes a Security
Considerations section that enumerates and evaluates the
possible attacks and how to deal with this.  Then we have have a
real discussion.  Without it, all we have is more noise on the
list, people making suggestions that other people agree with (or
not) depending on their perceptions of what is being proposed
(perceptions that may not exactly match the intentions of the
original proposal), and so on.

best,
    john

Date: Thu, 25 Aug 2011 09:49:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: [email-masked]
Subject: Re: productivity?
To: [email-masked]
Sender: [email-masked]

Sabahattin Gucukoglu wrote:

> The answer has got to be a new SMTP extension, or pair of extensions, to 
> do maximum throughput sending on one connection and multi-message 
> transaction advisory mechanisms to support co-operation between bulk software 
> and the MTA for knowing the delivery load in advance, thus neutralising the 
> discussion over server administrative policy, port or connection slot exhaustion, 
> client selfishness, time to wait until shutdown, event-driven programming vs 
> process-based servers, OS wars and all the other nonsense.  We need an 
> improved checkpointing facility.  That's what we should do, I think.

+1.  Or we can begin with a basic tenet of "do no harm."   It is 
obvious the issues of a well vetting system perform for a past era 
where hubs were more dominant and using ideas of connection sharing 
were isolated to well known connections, conflicts in a modern world 
where there is less hubbing, more peer to peer and anonymous 
connectivity.  Just consider how Facebook has become a new high mail 
sender where it needs to improve its client side loads but its 
distribution is towards a larger pool of smaller smtp servers.  If it 
was assumed that every facebook user had a common host account, i.e. 
gmail.com, then we back to the centralization and hub-hub concepts 
where this stuff was ideal.

So its an issue of recognition that using this across the board has a 
direct impact with most of the SMTP servers which there are 
significant amount of the smaller systems combined. And it may be just 
a matter of costing one to scale up or scale out by 1 or 2 CPU units. 
  Small? Sure, but dollars are dollars and its harder to justify 
expense these days especially if it was discovered that the extra 
expense is due to CS clients.

      "Boss, we can reduce the need to buy more computers or improve 
the ones we
       have simply by disconnecting these connection hogging clients."

Don't you think that will get the attention of most cost cautious 
managers?

Who knows?  But I do know that alienation does not help anyone and 
perpetuates lack of cooperation as a group and continue conflicts with 
the SMTP specs.

> PS: The C10K document is inspired, and I'm a Unix person.  My stuff is 
> event-driven.  Just so you know. :-)

Better and in many aspects ideal, its still open to contention tuning 
issues :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 25 Aug 2011 08:54:57 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: SMTP Interest Group <[email-masked]>
Subject: Improving Timeouts
To: [email-masked]
Sender: [email-masked]

Hector Santos wrote:
> 
> Paul Smith wrote:
> 
>> Just because something isn't seen as an immediate problem doesn't mean 
>> it may not become so in the future. Hopefully we've learned from the 
>> previous things that have become abusive that it's better to strike 
>> early rather than leave it until it's too late.
> 
> +1.
> 
> We don't even now if the bad guys took the idea a step further already - 
> you don't need to wait after the 1st transaction, just wait after each 
> state, 5 seconds at each state perhaps.
> 
> Just consider what is the maximum allowed SMTP session time is by SMTP 
> standards using the minimum commands:
> 
>     EHLO   5 mins
>     MAIL   5 mins
>     RCPT   5 mins * N  where N is total recipients (default 1)
>     DATA   5 mins
>     EOD   10 mins
>     QUIT   5 mins
> 
>   SMTP Standard Allowed Time = 30 + 5*N
> 
> With N=1, SMTP by technical standards, a client is allowed to use 35 
> minutes and still be 100% compliant!
> 

There is also a potential for a 10 mins "time hole" when first connecting:

    Server 5 mins to send a 220
    Server 5 mins for Client to send the first command. i.e. EHLO

So if the server is holding connected clients in a load session queue, 
and the client has not disconnect yet, once the server issue 220, the 
client can still wait 5 minutes to send its first command.

This is one area where the SMTP spec can be improved to set the idle 
time to:

     5 mins - minus the time it took to send the 220

So if it held the client for 3 minutes, the server should only need 
set an idle timeout of  2 mins for the client to send its first command.

The client has its own 5 mins timer and if it hasn't disconnected 
waiting for the 220, then most likely it will send the first command 
immediately after seeing it 220.  The server does not need to set a 5 
minutes idle time if its has already delayed the client.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 25 Aug 2011 08:20:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Peter J. Holzer wrote:
> On 2011-08-24 14:41:33 +0200, Peter J. Holzer wrote:
>> There are a few mails in the queue, so sendmail begins by connecting to
>> the primary MX (MX2 in this case). The MX returns (after 20 seconds,
>> because CBV can't connect to delphi and times out) a temporary error.
> 
> And it turns out that this was a DNS timeout, not a timeout in
> connect(2).

I venture you will now adjust the Filters DNS query timeout to some 
lower number? 10 secs?

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 24 Aug 2011 15:31:23 +0200
From: "Peter J. Holzer" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Mail-Followup-To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

--uQr8t48UFsdbeI+V
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-08-24 14:41:33 +0200, Peter J. Holzer wrote:
> There are a few mails in the queue, so sendmail begins by connecting to
> the primary MX (MX2 in this case). The MX returns (after 20 seconds,
> because CBV can't connect to delphi and times out) a temporary error.

And it turns out that this was a DNS timeout, not a timeout in
connect(2).

	hp

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--uQr8t48UFsdbeI+V
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--uQr8t48UFsdbeI+V--

Date: Wed, 24 Aug 2011 09:05:56 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Paul Smith <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

Paul Smith wrote:

> Just because something isn't seen as an immediate problem doesn't mean 
> it may not become so in the future. Hopefully we've learned from the 
> previous things that have become abusive that it's better to strike 
> early rather than leave it until it's too late.

+1.

We don't even now if the bad guys took the idea a step further already 
- you don't need to wait after the 1st transaction, just wait after 
each state, 5 seconds at each state perhaps.

Just consider what is the maximum allowed SMTP session time is by SMTP 
standards using the minimum commands:

     EHLO   5 mins
     MAIL   5 mins
     RCPT   5 mins * N  where N is total recipients (default 1)
     DATA   5 mins
     EOD   10 mins
     QUIT   5 mins

   SMTP Standard Allowed Time = 30 + 5*N

With N=1, SMTP by technical standards, a client is allowed to use 35 
minutes and still be 100% compliant!

The times were set for the era with things were slower, network 
related issues higher.  My first  TCP/IP programming book had an idiom 
(related to built-in error correction)

           There is no guarantee packets will reach its end point,
           but if it does, its guaranteed to be correct.

Another major point, early on, there was more hubbing, hub to hub, 
more path routing, UUCP, etc and not everyone had SMTP servers where 
we can all go direct.

Quite frankly, its easy to see why timeouts are high when it was done 
in an age where Telneting to the console based client/server protocols 
was par for the course. It would be extremely irritating to be 
developing this stuff using short time outs closing your interactive 
session!  Even today, I can't imagine a internet C/S console based 
protocol implementator not using telnet to test their wares.  Can you 
imagine getting knock out even at 1 minute?

So spammers have probably already exploited this but not going 
overboard and just use seconds.  I have noticed the delays between 
rejected RCPT in my logs, with patterns:

     C: RCPT TO: <FOOEY1>
     S: 550 Sorry Charlie! Not here!
     C: RCPT TO: <FOOEY2>            <<--- 2-5 seconds
     S: 550 Sorry Charlie! Not here!
     C: RCPT TO: <FOOEY3>            <<--- 2-5 seconds
     S: 550 Sorry Charlie! Not here!
     C: RCPT TO: <FOOEY4>            <<--- 2-5 seconds
     S: 550 Sorry Charlie! Not here!
     C: QUIT                         <<--- microsecs!

So this Connection Holding For Mail concept need not apply to after 
the 1st transaction.  It can be spread out.

> Caching connections to or from a small company's mail server is going to 
> be a waste of their resources 99.99% of the time, as it is extremely 
> unlikely that there'll be two messages soon after each other between the 
> same two MTAs. It's a different matter if you're talking about Gmail 
> sending to Hotmail where connection caching could be a huge benefit, but 
> most mail servers aren't in that situation. In my view connection 
> caching should always be disabled by default, and allow the admins to 
> set up cached connections to other specific domains (or automate it 
> based on how many times a cached connection would have been useful). 
> Having a default of always caching connections is just wrong.

Facebook is currently the #1 sender of mail on our server.  Each 
session has a near perfect 5 second delay before QUIT and I have yet 
so see one with a 2nd transaction.  It is definitely contributed to 
the increase average and this month we have 4-5 DoS attacks and 
serious amount sparse 421.  Who knows if Facebook with the others 
holding idle for long periods has contributed to 421s. Without a 
detail analysis its hard to tell.  But what I thought was part for the 
course in dealing with this Spam World, now I am wondering how much of 
this is due these Connection Sharing software.

> I do realise that there are people here who think connection caching is 
> good 100% of the time, so I'm probably flogging a dead horse. Maybe 
> because I'm coming at it from the viewpoint of companies with 5-50 
> users, rather than from ISPs or big companies I have a different viewpoint.

Paul, don't water yourself down. There will be no internet (as we know 
it today) if it wasn't the millions of the smaller systems combined 
compare to the relative far fewer larger mail handlers. So we are all 
in this together and we are all important.  Its everyone's problem, 
some  will be more sensitive to it than others.  Companies of all size 
TCO is always important.  It means more computers and resources and 
the direction is to scale up, with less computers as a very big way to 
lower your TCO. A "What If" analysis will show if this is cost factor 
and a cost reduction and once it is highlighted, most CEO/CTO and 
operations managers will look into this.  I don't be surprise it if 
equates to a cost factor measured by CPU+RAM simply by lowering the 
timeouts allowed by SMTP.

> (PS - for the discussion about ephemeral ports, reusing connections is 
> only useful if you can. If you hold a connection open for 5 seconds 
> longer than necessary and don't reuse it, then that's 5 seconds longer 
> before that port number can be reused - so ephemeral port limits can be 
> an argument against connection caching as well)

For socket operations that do not (or can't) use the socket option 
SO_REUSEADDR to reuse the port in the TIME_WAIT state or closes the 
socket first which put it into TIME_WAIT, yes, the delays increase 
port exhaustion potential.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 24 Aug 2011 14:41:33 +0200
From: "Peter J. Holzer" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Mail-Followup-To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-08-21 12:07:06 +0200, Peter J. Holzer wrote:
> After a RSET it's a bit different: Only 76% send a QUIT after 0 or 1
> seconds, 10% after 5 seconds and there seem to be small peaks of
> decreasing size every 20 seconds.

Actually, the 20 seconds peak is probably due to the interaction between
sendmails connection caching, callback sender verification and a
misconfigured sendmail.=20

I found this in the logfiles (domain names and local parts anonymized):

MX2 2011-08-24T12:02:29+0200  220 mx2.example.com ESMTP qpsmtpd 0.84 ready;=
 send us your mail, but not your spam.
MX2 2011-08-24T12:02:29+0200  dispatching EHLO delphi.example.com
MX2 2011-08-24T12:02:29+0200  250 STARTTLS
MX2 2011-08-24T12:02:29+0200  dispatching STARTTLS
MX2 2011-08-24T12:02:29+0200  220 Go ahead with TLS
MX2 2011-08-24T12:02:29+0200  dispatching EHLO delphi.example.com
MX2 2011-08-24T12:02:29+0200  250 SIZE 20971520
MX2 2011-08-24T12:02:29+0200  dispatching MAIL From:<[email-masked]>=
 SIZE=3D60373
MX2 2011-08-24T12:02:29+0200  250 <[email-masked]>, sender OK - how =
exciting to get mail from you!
MX2 2011-08-24T12:02:29+0200  dispatching RCPT To:<[email-masked]>
MX2 2011-08-24T12:02:49+0200  450 <[email-masked]> cannot be verifie=
d (server unreachable?) (#4.1.8)
MX2 2011-08-24T12:02:49+0200  dispatching DATA
MX2 2011-08-24T12:02:49+0200  503 RCPT first
MX2 2011-08-24T12:02:49+0200  dispatching RSET
MX2 2011-08-24T12:02:49+0200  250 OK

MX1 2011-08-24T12:02:50+0200  220 mx1.example.com ESMTP qpsmtpd 0.40 ready;=
 send us your mail, but not your spam. [LOGINFO]
MX1 2011-08-24T12:02:50+0200  dispatching EHLO delphi.example.com [LOGINFO]
MX1 2011-08-24T12:02:50+0200  250 SIZE 20971520 [LOGINFO]
MX1 2011-08-24T12:02:50+0200  dispatching MAIL From:<[email-masked]>=
 SIZE=3D60373 [LOGINFO]
MX1 2011-08-24T12:02:50+0200  250 <[email-masked]>, sender OK - how =
exciting to get mail from you! [LOGINFO]
MX1 2011-08-24T12:02:50+0200  dispatching RCPT To:<[email-masked]> [LOGINFO]
MX1 2011-08-24T12:03:12+0200  450 <[email-masked]> cannot be verifie=
d (server unreachable?) (#4.1.8) [LOGINFO]
MX1 2011-08-24T12:03:12+0200  dispatching DATA [LOGINFO]
MX1 2011-08-24T12:03:12+0200  503 RCPT first [LOGINFO]
MX1 2011-08-24T12:03:12+0200  dispatching RSET [LOGINFO]

MX2 2011-08-24T12:03:12+0200  dispatching RSET
MX2 2011-08-24T12:03:12+0200  250 OK
MX2 2011-08-24T12:03:12+0200  dispatching MAIL From:<[email-masked]>=
 SIZE=3D4589
[...]

MX1 2011-08-24T12:03:12+0200  250 OK [LOGINFO]
MX1 2011-08-24T12:03:32+0200  dispatching RSET [LOGINFO]
MX1 2011-08-24T12:03:32+0200  250 OK [LOGINFO]
MX1 2011-08-24T12:03:32+0200  dispatching MAIL From:<[email-masked]>=
 SIZE=3D4589 [LOGINFO]
[...]

After your explanation about sendmail's queueing behaviour it is clear
what happens:

Sendmail on delphi is misconfigured to attach the full hostname instead
of only domainname to the username to create mail addresses. Since
addresses like this aren't supposed to exist, port 25 of delphi is
unreachable and there is no MX record either. The recipient R has CBV
enabled.

There are a few mails in the queue, so sendmail begins by connecting to
the primary MX (MX2 in this case). The MX returns (after 20 seconds,
because CBV can't connect to delphi and times out) a temporary error. So
sendmail sends a RSET and leaves to connection open.

There is a secondary MX (confusingly called MX1), so it connects to
that. Again, RCPT fails after about 20 seconds. So it sends a RSET and
leaves the connection open. The mail is currently undeliverable and goes
back into the queue.

The next mail should again be delivered to MX2, and sendmail already has
a connection. So it sends a RSET and tries the delivery, which again
fails after a timeout of 20 seconds and the same game starts with MX1
=2E..

So during the entire queue run the two connections stay open and
delivery switches from one MX to the other every 20 seconds - not
because the client has a 20 second limit somewhere but because the
server takes 20 seconds to reject each message.

	hp

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--ZPt4rx8FFjLCG7dd
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--ZPt4rx8FFjLCG7dd--

Date: Wed, 24 Aug 2011 10:20:48 +0100
From: Paul Smith <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

On 23/08/2011 20:05, Murray S. Kucherawy wrote:
>  So we're only talking about "law-abiding citizens" here, and I don't 
> think MTAs implementing connection caching have any reckless or 
> malicious intent behind those designs, but rather are trying to 
> improve the throughput on both sides
Well, there are two parts to an SMTP implementation - the software and 
the configuration.

While I agree that law abiding SMTP software implementations won't have 
malicious intent, I think that you are very likely to find law abiding 
MTA configators have a disregard for other people - either through 
selfishness or ignorance. We see this frequently, especially with the 
big mail providers who think everyone has to bow to their needs, and 
with small ISPs (and some big ones) who don't know what they are doing. 
The software is OK, but the configuration is very bad. Possibly the 
software authors never thought the configuration would be made so bad...

So, while an MTA may cache connections by default for (say) 1-5 seconds 
(which is bad IMHO, but not too bad ATM), if it allows the configurator 
to change that to 5+ minutes, then you can bet that someone will do it 
at some point. (Just as I guess most MTAs have default timeouts over 1 
minute, but we often see timeouts of 10 seconds or less - someone has 
configured that, not realising the implications).

Thus, notes for the software implementer may be useful as it may give 
them ideas about limits they may want to impose on configurable 
settings. Also, it may help them realise that, even though they are 
concentrating on optimising sending, their optimisations may adversely 
impact the remote receiver.

> , and probably did some testing of the idea before rolling it out. So 
> far the industry as a whole hasn't found it to be a problem (which to 
> me means it's possibly even a benefit).
Well, spam wasn't considered a problem 15 years ago, and things like the 
% redirector and open relays were considered a benefit then as well.

Just because something isn't seen as an immediate problem doesn't mean 
it may not become so in the future. Hopefully we've learned from the 
previous things that have become abusive that it's better to strike 
early rather than leave it until it's too late.

I would like to see what testing any mail sender has done with regards 
to connection caching from a large number of senders to a few receivers. 
Sending from their particular sender to lots of receivers it probably 
has a huge benefit, but going the other way around, it potentially 
causes huge problems.

Caching connections to or from a small company's mail server is going to 
be a waste of their resources 99.99% of the time, as it is extremely 
unlikely that there'll be two messages soon after each other between the 
same two MTAs. It's a different matter if you're talking about Gmail 
sending to Hotmail where connection caching could be a huge benefit, but 
most mail servers aren't in that situation. In my view connection 
caching should always be disabled by default, and allow the admins to 
set up cached connections to other specific domains (or automate it 
based on how many times a cached connection would have been useful). 
Having a default of always caching connections is just wrong.

I do realise that there are people here who think connection caching is 
good 100% of the time, so I'm probably flogging a dead horse. Maybe 
because I'm coming at it from the viewpoint of companies with 5-50 
users, rather than from ISPs or big companies I have a different viewpoint.

(PS - for the discussion about ephemeral ports, reusing connections is 
only useful if you can. If you hold a connection open for 5 seconds 
longer than necessary and don't reuse it, then that's 5 seconds longer 
before that port number can be reused - so ephemeral port limits can be 
an argument against connection caching as well)

Subject: Re: productivity? (was: Re: Mail Data termination)
Date: Wed, 24 Aug 2011 07:13:01 +0100
To: [email-masked]
From: Sabahattin Gucukoglu <[email-masked]>
Reply-To: Sabahattin Gucukoglu <[email-masked]>
Sender: [email-masked]

On 22 Aug 2011, at 15:04, Dave CROCKER wrote:
> I see a couple of folk suggesting that this thread has run its course.  I've been wondering about that.
> 
> Given how long the thread has been and how much senior email talent has contributed to it, I'm entirely unclear what it has accomplished or where it can or should go.
> 
> So I thought I'd ask.

All right.  Since you ask, I'll answer. :-)

The truth is, I've found much of it frustratingly noisy and unproductive.  I mean, nothing changes here.  How could it?  Much of what is being brought into question is thoroughly vetted and well-understood, as John very correctly pointed out.  I'd never advocate for a document which decided one way or the other.  Even the respective protagonists in the war for and against connection caching seem to have some sense of the various pros and cons.  If there's any positive outcome, it's that I have questions I didn't have for a while now that I'm going to need answers for in order to progress with my implementation.  I suppose that qualifies as future document material, but that's probably not the kind you meant.

> Perhaps this has merely been an extended consulting session to help a few folk?
> 
> Given the amount of sage commentary has been posted, is there a document that should be produced?

I think the question that needs answering most is not whether or not connection caching is bad, but how we can best reduce the need for a compromise between starting new client sessions and holding them open in the first place.  The answer has got to be a new SMTP extension, or pair of extensions, to do maximum throughput sending on one connection and multi-message transaction advisory mechanisms to support co-operation between bulk software and the MTA for knowing the delivery load in advance, thus neutralising the discussion over server administrative policy, port or connection slot exhaustion, client selfishness, time to wait until shutdown, event-driven programming vs process-based servers, OS wars and all the other nonsense.  We need an improved checkpointing facility.  That's what we should do, I think.

Cheers,
Sabahattin

PS: The C10K document is inspired, and I'm a Unix person.  My stuff is event-driven.  Just so you know. :-)

Date: Tue, 23 Aug 2011 20:08:25 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
To: [email-masked]
Sender: [email-masked]

Murray S. Kucherawy wrote:
>>
>>      The ConnectionCacheTimeout ( K) option specifies the maximum time
>>      that any cached connection will be permitted to idle. When the idle
>>      time exceeds this value the connection is closed. *This number should be
>>      small (under ten minutes) to prevent you from grabbing too many resources
>>      from other hosts. The default is five minutes.*
> 
> But that's the RFC5321 default anyway.  So if you're holding that up as 
> the Right Way, it would seem you agree no change is actually required.

Since your focus this behavior is OK  and my focus is its not OK, to 
the extent I believe it is borderline unethical engineering (just how 
I was trained to think), we won't be disagreeing on much on anything.

When an client application acknowledges it can hurt remote systems, 
and you didn't see that was what I was talking about when thats all I 
being talking about since the thread started, its really rings how 
different our thinking is when it comes security and "do no harm" 
concepts. Its not the first time. It just won't cross my mind to do 
anything at this level that didn't include some negotiation of some 
form. Its bound to hurt something and even if you think it hasn't yet, 
the evidence exist it already has marginally, but it doesn't exclude 
the fact the potential is realistically there to do some damage.  I 
could done a quite a few things that were considered once considered 
engineering taboos in the past with my networking client/server 
hosting products since the 80s, but I just didn't do it for ethical 
reasons - but eventually people did, made tons of bucks, cracked up 
the door and the world has problems putting it back into the bottle.

Do you know if this default behavior or you have to enable it?

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 23 Aug 2011 17:01:21 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

Peter J. Holzer wrote:
>> So really you would only ever see a cached connection stay open that
>> long if, for a connection cache of size N, it had to connect to some
>> small server, but the remaining (rather large) queue went only to a
>> specific N-1 other servers, and so the first one sat idle for that
>> long.  Seems rather like a corner case.
> 
> Right. Especially since N is usually very small (default is 1 according
> to the 8.12 docs, 2 for 8.13 on a Redhat box), it seems rather unlikely
> that a connection would sit idle for an extended time during a queue
> run.

There is really very little that needs to be done. Ideally,  SMTP 
should have just a few notes about this.  Starter text:

       SMTP clients using Connection Caching methods SHOULD NOT wait more
       than a few seconds to prevent grabbing too many resources from SMTP
       servers.  If the SMTP client has no cached messages to send, it 
SHOULD
       immediately end the session by issuing the QUIT command.  It is
       highly recommended for SMTP clients to queue up the messages before
       connecting rather than connect and hold the connection sitting idle
       waiting for new mail to send.

Really, that is all I think that is needed. Just make both the client 
and server aware and leave it open ended.  Where in the docs? 
Somewhere :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 23 Aug 2011 21:54:12 +0200
From: "Peter J. Holzer" <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Mail-Followup-To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

--lEGEL1/lMxI0MVQ2
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-08-23 12:15:20 -0700, Murray S. Kucherawy wrote:
> > BTW, I've never observed a sendmail to keep open an idle connection for
> > that long. Maybe that limit doesn't apply to "normal" connections
> > (according to the document you cited it only applies to "IPC
> > connections"[1]) or maybe that limit is only reached under rare
> > circumstances. I just tested an out-of-the-box sendmail-8.13 (with
> > ConnectionCacheTimeout=3D5m) and it sends the QUIT immediately after the
> > 250 response to DATA.
>=20
> Sendmail's connection cache only applies during queue runs.  The
> entire connection cache is capped at a certain number of connections,
> which is where the LRU comes in; when it needs to make a connection to
> a server that is not currently cached, the most idle one is closed
> down and the new one is established.  When the queue run completes,
> any cached connections remaining are also closed.
>=20
> So really you would only ever see a cached connection stay open that
> long if, for a connection cache of size N, it had to connect to some
> small server, but the remaining (rather large) queue went only to a
> specific N-1 other servers, and so the first one sat idle for that
> long.  Seems rather like a corner case.

Right. Especially since N is usually very small (default is 1 according
to the 8.12 docs, 2 for 8.13 on a Redhat box), it seems rather unlikely
that a connection would sit idle for an extended time during a queue
run.

Thanks for the explanation.

	hp

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--lEGEL1/lMxI0MVQ2
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--lEGEL1/lMxI0MVQ2--

From: "Murray S. Kucherawy" <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Date: Tue, 23 Aug 2011 12:15:20 -0700
Subject: RE: productivity?
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Peter J. Holzer
> Sent: Tuesday, August 23, 2011 2:09 AM
> To: SMTP Interest Group
> Subject: Re: productivity?
> 
> BTW, I've never observed a sendmail to keep open an idle connection for
> that long. Maybe that limit doesn't apply to "normal" connections
> (according to the document you cited it only applies to "IPC
> connections"[1]) or maybe that limit is only reached under rare
> circumstances. I just tested an out-of-the-box sendmail-8.13 (with
> ConnectionCacheTimeout=5m) and it sends the QUIT immediately after the
> 250 response to DATA.

Sendmail's connection cache only applies during queue runs.  The entire connection cache is capped at a certain number of connections, which is where the LRU comes in; when it needs to make a connection to a server that is not currently cached, the most idle one is closed down and the new one is established.  When the queue run completes, any cached connections remaining are also closed.

So really you would only ever see a cached connection stay open that long if, for a connection cache of size N, it had to connect to some small server, but the remaining (rather large) queue went only to a specific N-1 other servers, and so the first one sat idle for that long.  Seems rather like a corner case.

> [1] I couldn't find out what that is - the old sendmail paper from the
> 1980's uses the term as a synonym for "socket connection", but I'm not
> sure if that's current usage.

"IPC connections" refers to anything that talks SMTP, in that context.  The other mailers (in sendmail parlance) are pipes to local programs.

-MSK

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 23 Aug 2011 12:10:48 -0700
Subject: RE: productivity?
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Carl S. Gutekunst
> Sent: Monday, August 22, 2011 4:37 PM
> To: Hector Santos
> Cc: [email-masked]
> Subject: Re: productivity?
> 
> None of which has any relevance to any existing RFCs. But tuning
> connection reuse might make for an interesting Informational doc or
> BCP.

See, that's interesting.

What sort of tuning do you think might be helpful?

-MSK

From: "Murray S. Kucherawy" <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Date: Tue, 23 Aug 2011 12:05:02 -0700
Subject: RE: productivity?
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Monday, August 22, 2011 2:23 PM
> To: SMTP Interest Group
> Subject: Re: productivity?
> 
> For those interested, if basic laws of operations research, physics,
> science and Thermodynamics isn't good enough, then Believe Sendmail!
> 
>      http://www.sendmail.org/~ca/email/doc8.12/op-sh-4.html
> 
>      The ConnectionCacheTimeout ( K) option specifies the maximum time
>      that any cached connection will be permitted to idle. When the idle
>      time exceeds this value the connection is closed. *This number should be
>      small (under ten minutes) to prevent you from grabbing too many resources
>      from other hosts. The default is five minutes.*

But that's the RFC5321 default anyway.  So if you're holding that up as the Right Way, it would seem you agree no change is actually required.

An SMTP extension that tells a server how many sockets/threads/processes/whatever to allocate seems like a very convenient denial-of-service attack to me.  It's the same discussion we had with the SIZE extension way back when.  The real use of it would be "no, I can't support that many new connections from you right now," although you've already established the one you have, so that's just weird.

I agree with Dave that if there's some useful thing we could document out of this discussion, we should.  But at the moment I'm having trouble seeing what that might be.  It's certainly true that a client that wants to be selfish could open any number of connections and hold them open, replacing any the server closes, regardless of what we put in any standards track or BCP document.  So we're only talking about "law-abiding citizens" here, and I don't think MTAs implementing connection caching have any reckless or malicious intent behind those designs, but rather are trying to improve the throughput on both sides, and probably did some testing of the idea before rolling it out.  So far the industry as a whole hasn't found it to be a problem (which to me means it's possibly even a benefit).

If someone's really keen to do so, some comparisons between running a queue with connection caching and running the same queue without connection caching, monitoring both the client and the server for total cost, would be a good place to start.

And now, if I may borrow from old schoolyard antics:  NOT IT!

-MSK

Date: Tue, 23 Aug 2011 20:06:37 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 22.08.2011 15:00, Paul Smith wrote:
> On 22/08/2011 13:46, John C Klensin wrote:
>> gather up everything you can find and send it as part of the same
>> connection and, especially if the connection took several tries to
>> establish, maybe keep it open for short while to see if anything else
>> can be pushed through it

That confirms that the idle time ought to be proportional to the time
necessary for establishing and terminating the connection (that I called
overhead in a previous message).  The proportionality factor can take into
account statistical prediction from recent history.  It should be at least as
long as to allow a local busy mailer to actually submit the next message, and
may be further constrained by ephemeral ports/TIME_WAIT considerations.

However, what we mean by "idle" was not specified clearly.  The acceptance of
the current message by the server doesn't seem to be relevant, thus we can as
well consider "idle" the time taken by the receiver to run any mail filters
before replying.  Consider the following SMTP-fictional snippet:

  C:  ...blah blah the end of the message.
  C:  <CRLF>.<CRLF>
  S:  100 Hold on.
  C:  Yes, please take your time.

      ... this is some idle time...

  S:  250 Ok, message queued.

      ... this is more idle time...

  C:  QUIT

> I still think that 'pre-queuing' rather than 'post-waiting' is more community
> spirited

Maybe.  But at pre-queuing time a sender doesn't really know which MXes will
be reachable.  Vanity domains may resolve to the same MX, an MX may have
multiple IP addresses, a domain may sport multiple primary MXes, ...

>> In that environment, the strategy is not only not antisocial, it is hugely
>> beneficial, not just to the sender but to the receiver and the
>> users/mailboxes it supports. 
> Not if it prevents other senders getting their messages through while the
> first sender was sitting with an idle connection using up the connection quota.

The queueing model implied by SMTP provides for servicing the current client
for as many transactions as it wishes.  Other senders, who are unable to
connect to their target MX, can be considered to form a virtual queue.
Servers shape their queues according to their policies and configurations,
including load limits, max open connections [per IP| class], {white| gray|
black}listing, and any such means.

A server can disconnect a client, but cannot invite it to wait idle for a
while, nor disclose its current idea of optimal waiting factor.

> Keeping connections open 'too long' is fine if you assume that the receiver
> has an unlimited number of connections available. Once you accept that this
> is a limited resource in many cases, then it becomes a much nastier thing to
> do. If there's an unlimited number of connections, then an idle connection
> costs the receiver nothing. If the receiver is limited to, say, 10
> simultaneous connections, then the idle connection costs the receiver 10% of
> its capacity.

What if the sender is also limited to 10 connections?  It is sacrificing its
chance to deliver messages destined to other MXes for the sake of the
currently connected server's optimization.  Such clients altruistically
relieve servers from repeating unneeded operations, don't they?  :-/

Date: Tue, 23 Aug 2011 09:51:55 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
To: [email-masked]
Sender: [email-masked]

Dave CROCKER wrote:
> 
> 
> 
> On 8/22/2011 1:30 PM, Steve Atkins wrote:
>>> Given the amount of sage commentary has been posted, is there a 
>>> document that should be produced?
>>
>> Measuring actual costs (what is the resource cost of session setup and 
>> teardown, and of maintaining an open session, for well-implemented, 
>> mainstream MTAs?) would seem to be something to do before discussing 
>> such a document in much detail.
> 
> 
> That's certainly a concrete and potentially useful operational 
> optimization study that could be performed.
> 
> Hmmm.  I wonder how much of such a study would be applicable to other 
> apps, such as http?

Yes. Anything related to flow rates (IN=OUT) and the derivatives of 
the rates applies. Bank waiting lines with limited tellers to 
Fluidized Catalytic Crackers (barrels of oil processing into various 
components from gases (lightest), oils, gasoline to tar (heaviest)). 
All use the same basic principles to maximize performance and avoid 
overflowing chaotic conditions.

> Are any hands-on researchers watching this list?  If anyone knows such 
> folk, it might be worth whispering in their ear.

Google is your friend.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 23 Aug 2011 09:32:00 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: productivity?
Sender: [email-masked]

Hector Santos wrote:
> 
> Timing is everything, 
> yet it needs to be understood how it can give you unpredictability.  
> Windows is not an RTOS (Real Time Operating System) where such timing 
> predictions are guaranteed.   For example, it is often misunderstand 
> what the actually elapse time are.
> 
>       Sleep(0)  15 ms,  return immediately
>       Sleep(1)  15 ms,  Thread is Queued
>       Sleep(21) 20 ms,  Thread is Queued
>       Sleep(36) 45 ms,  Thread is Queued

Oops, Sleep(21) is 30 ms!  The granularity is 15ms.

FYI,

You can use change it to 1 ms using the Multimedia Media API library 
functions.

    timeBeginPeriod(1);
    timeEndPeriod(1);

http://msdn.microsoft.com/en-us/library/dd757624(v=vs.85).aspx

And this can be another factor in confusing developers that their 
software is perfect but can't explain why other people have problems. 
  Its a global setting and if some other application running (doesn't 
have to be multi media related) changes it for its own needs, you 
might begin to develop code and think its all perfect, but then PUFF - 
during testing or other users that don't have the global 1ms 
resolution turned on, have problems.

That is one of many reasons synchronization based on time can bite you.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 23 Aug 2011 08:36:42 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

Peter J. Holzer wrote:
>> For those interested, if basic laws of operations research, physics,  
>> science and Thermodynamics isn't good enough, then Believe Sendmail!
>>
>>     http://www.sendmail.org/~ca/email/doc8.12/op-sh-4.html
>>
>>     The ConnectionCacheTimeout ( K) option specifies the maximum time
>>     that any cached connection will be permitted to idle. When the idle
>>     time exceeds this value the connection is closed. *This number  
>> should be
>>     small (under ten minutes) to prevent you from grabbing too many  
>> resources
>>     from other hosts. The default is five minutes.*
> 
> I'm not sure what you are arguing here. That five minutes is a
> reasonable idle time for connection caching? That there are MTAs with
> ridiculously long defaults? 

I guess I am disappointed that the focus was lost here.  The initial 
problem of causing delays and resource issues on the receiver to one 
where its "OK" as long as we fine tune it.

Producing a Tuning doc is fine, but it should not the key focus of 
using this to continue abusing receivers in unauthorized and 
undesirable ways.  It doesn't matter what a client thinks what it good 
or bad - its what the receiver thinks. It has absolutely nothing to do 
with "mainstream" systems or not - that mentality continues to hurts 
groups like this.  Gets us no way and never solves problems across the 
board for the entire community.

I can only see an SMTP extension that exposes a permission token, 
CSHARE=#, perhaps that allows a receiver to allocate the channels it 
wants to allow for these special connections without degradation the 
system loading limits and also not hurting other clients from getting in.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 23 Aug 2011 08:29:23 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: productivity?
Sender: [email-masked]

Peter J. Holzer wrote:
> I don't see how this can make a difference. TIME_WAIT is 240 seconds
> (per RFC, some implementations use shorter defaults). If your ephemeral
> ports are limited to the range 1024-4999 (which used to be popular), you
> are limited to (5000-1024)/240 = 16.6 connections per second. Putting in
> a sleep(0) doesn't change this, lowering TIME_WAIT or extending the
> range of ephemeral ports does. (Waiting for the other side to close the
> connection does, too - but unless I'm missing something moving the
> problem to the other side doesn't help if you want to communicate with
> the other side). I'm with Carl here - if you need to send 17+ mails per
> second, it's much better to reuse connections than to permanently close
> and reopen them.

Devil details.

I mentioned the poor man's poke because I seen it too often when a 
port to windows is done and it WINTEL fundamentals are not very well 
understood.  You got hardware interrupts (networking and disk) 
services where bottlenecks occur and other worker threads to cooperate 
with, so poking as a poor man's way to wake up other workers without 
losing must of your own time. Its not the 100% Wintel Way, but it can 
used. I might use it to help wake up hardware related concepts. 
Timing is everything, yet it needs to be understood how it can give 
you unpredictability.  Windows is not an RTOS (Real Time Operating 
System) where such timing predictions are guaranteed.   For example, 
it is often misunderstand what the actually elapse time are.

       Sleep(0)  15 ms,  return immediately
       Sleep(1)  15 ms,  Thread is Queued
       Sleep(21) 20 ms,  Thread is Queued
       Sleep(36) 45 ms,  Thread is Queued

etc, in other words for robust well understood system, you must have a 
understanding the CPU Quantum size and the whole timing paradigm under 
Windows. I have seen huge huge monster debates when unix thinking 
simply doesn't apply in Windows because they thought there was a 1 ms 
resolution with timing, they get hung on on Linux or other *nix timing 
slicing granularity (1ms vs 15ms) and believe that it equally applies 
to Windows - it doesn't and those that continue to think so continue 
to have problems or just live what they have achieved.

16.6 cps is very slow.

Was ASYNC or SYNCH used?  Threads?  Sequential programming?  You can 
blast 16 threads and you can't use reuseability.  You also need to 
take in account that Windows also have caching too.  So opening and 
closing is not as wasteful as you think.

As I noted to Carl off list, I just did a quick test using our 
internal SmtpStress tool that can blasting X threads with Y messages 
per thread, and without changing the license limit of 25 concurrent 
connections on our SMTP server test machine, I was able to blast 25 
message under 1 second. It wasn't even a full second.  Changing our 
logs to microseconds was all part of this profiling done early this 
year.  And I know more connections were possible because using X=50, 
there were a 25 421 server responses.

Note: That is another thing in RFC5321, it doesn't explicitly say that 
421 can be used at the connection point. It mentions 554, but that 
requires you to hold the connection until the client issues a QUIT.

Finally, I do think, what you touched base with, a half-close shutdown 
needs to be check if this is needed.  Why?  Because when the 
Download/Upload speeds differences became dramatic, all of a sudden 
applications such as IE, Apache and so many others, including our own 
Wildcat! Web Server began to experience timing and synchronizations 
problems.  The infamous IE "Page Not Found" problem was directly 
related to servers not using Half-Close socket Shutdown.  It was all 
because the of that final ACK was lost. I specially recall reluctantly 
trying Sleeps() to try to get it to work, but what worked here, didn't 
work there. The solution was to use half-close shutdowns.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 23 Aug 2011 11:09:28 +0200
From: "Peter J. Holzer" <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Mail-Followup-To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

--Yylu36WmvOXNoKYn
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-08-22 17:23:26 -0400, Hector Santos wrote:
> Steve Atkins wrote:
>> Measuring actual costs (what is the resource cost of session setup
>> and teardown, and of maintaining an open session, for
>> well-implemented, mainstream MTAs?) would seem to be something to do
>> before discussing such a document in much detail.

Definitely. The assumption that the number of concurrent connection is
the only (or at least most important) limit doesn't seem realistic to
me.

> For those interested, if basic laws of operations research, physics, =20
> science and Thermodynamics isn't good enough, then Believe Sendmail!
>
>     http://www.sendmail.org/~ca/email/doc8.12/op-sh-4.html
>
>     The ConnectionCacheTimeout ( K) option specifies the maximum time
>     that any cached connection will be permitted to idle. When the idle
>     time exceeds this value the connection is closed. *This number =20
> should be
>     small (under ten minutes) to prevent you from grabbing too many =20
> resources
>     from other hosts. The default is five minutes.*

I'm not sure what you are arguing here. That five minutes is a
reasonable idle time for connection caching? That there are MTAs with
ridiculously long defaults?=20

BTW, I've never observed a sendmail to keep open an idle connection for
that long. Maybe that limit doesn't apply to "normal" connections
(according to the document you cited it only applies to "IPC
connections"[1]) or maybe that limit is only reached under rare
circumstances. I just tested an out-of-the-box sendmail-8.13 (with
ConnectionCacheTimeout=3D5m) and it sends the QUIT immediately after the
250 response to DATA.

	hp

[1] I couldn't find out what that is - the old sendmail paper from the
1980's uses the term as a synonym for "socket connection", but I'm not
sure if that's current usage.

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--Yylu36WmvOXNoKYn
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--Yylu36WmvOXNoKYn--

Date: Tue, 23 Aug 2011 10:33:50 +0200
From: "Peter J. Holzer" <[email-masked]>
To: [email-masked]
Subject: Re: productivity?
Mail-Followup-To: [email-masked]
Sender: [email-masked]

--Dxnq1zWXvFF0Q93v
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-08-22 21:17:07 -0400, Hector Santos wrote:
> Sounds like a bug or poor windows socket implementation, not yours, the=
=20
> other guy. :)  #1 rule of thumb in Synchronization 101 is to not depend=
=20
> on time.

Too bad the designers of TCP didn't know this ...

> Inevitably it will bite you, maybe this year or even never, but=20
> somehow it will rear its head. :) So if lowering TIME_WAIT was one=20
> solution, it generally means there is some kind of sync design problem=20
> and it might be a matter of issue a one time poke (Sleep(0)) to give=20
> other threads time, but immediately coming back.  Sometimes it can make=
=20
> all the difference. :)

I don't see how this can make a difference. TIME_WAIT is 240 seconds
(per RFC, some implementations use shorter defaults). If your ephemeral
ports are limited to the range 1024-4999 (which used to be popular), you
are limited to (5000-1024)/240 =3D 16.6 connections per second. Putting in
a sleep(0) doesn't change this, lowering TIME_WAIT or extending the
range of ephemeral ports does. (Waiting for the other side to close the
connection does, too - but unless I'm missing something moving the
problem to the other side doesn't help if you want to communicate with
the other side). I'm with Carl here - if you need to send 17+ mails per
second, it's much better to reuse connections than to permanently close
and reopen them.

	hp

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--Dxnq1zWXvFF0Q93v
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--Dxnq1zWXvFF0Q93v--

Date: Mon, 22 Aug 2011 21:17:07 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "Carl S. Gutekunst" <[email-masked]>
CC: [email-masked]
Subject: Re: productivity?
Sender: [email-masked]

Sounds like a bug or poor windows socket implementation, not yours, 
the other guy. :)  #1 rule of thumb in Synchronization 101 is to not 
depend on time. Inevitably it will bite you, maybe this year or even 
never, but somehow it will rear its head. :) So if lowering TIME_WAIT 
was one solution, it generally means there is some kind of sync design 
problem and it might be a matter of issue a one time poke (Sleep(0)) 
to give other threads time, but immediately coming back.  Sometimes it 
can make all the difference. :)

Carl S. Gutekunst wrote:
> 
> Hector Santos wrote:
>>> One issue not raised here: ephemeral port exhaustion. It's a rather 
>>> specialized problem, more of an issue for Windows folks than Linux, 
>>> and mostly an issue for routers, proxies, and similar devices that 
>>> tend to funnel a large volume of mail to smaller number of 
>>> destinations. But it's a real problem, one where reusing existing 
>>> client connections can provide a complete and permanent resolution.
>>
>> The ephemeral port exhaustion exploit began with Microsoft DNS server 
>> DoS attacks....
> 
> I'm talking about the normal operational condition where an SMTP sender 
> connects repeatedly to the same receiver. On platforms where the default 
> ephemeral port space is small (e.g., Windows Server 2003 and older BSD 
> kernels), the sender can exhaust the ephemeral ports. For those of us in 
> the filtering business, this is a constant nuisance.
> 
> The problem can be mitigated by increasing the number of ephemeral ports 
> (Vista defaults to the IANA recommend range), reducing TIME_WAIT, or 
> (tada!) reusing connections.
> 
> None of which has any relevance to any existing RFCs. But tuning 
> connection reuse might make for an interesting Informational doc or BCP.
> 
> <csg>
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 22 Aug 2011 16:37:21 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: Hector Santos <[email-masked]>
CC: [email-masked]
Subject: Re: productivity?
Sender: [email-masked]

Hector Santos wrote:
>> One issue not raised here: ephemeral port exhaustion. It's a rather 
>> specialized problem, more of an issue for Windows folks than Linux, 
>> and mostly an issue for routers, proxies, and similar devices that 
>> tend to funnel a large volume of mail to smaller number of 
>> destinations. But it's a real problem, one where reusing existing 
>> client connections can provide a complete and permanent resolution.
>
> The ephemeral port exhaustion exploit began with Microsoft DNS server 
> DoS attacks....

I'm talking about the normal operational condition where an SMTP sender 
connects repeatedly to the same receiver. On platforms where the default 
ephemeral port space is small (e.g., Windows Server 2003 and older BSD 
kernels), the sender can exhaust the ephemeral ports. For those of us in 
the filtering business, this is a constant nuisance.

The problem can be mitigated by increasing the number of ephemeral ports 
(Vista defaults to the IANA recommend range), reducing TIME_WAIT, or 
(tada!) reusing connections.

None of which has any relevance to any existing RFCs. But tuning 
connection reuse might make for an interesting Informational doc or BCP.

<csg>

Date: Mon, 22 Aug 2011 18:29:56 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
To: [email-masked]
Sender: [email-masked]

Carl S. Gutekunst wrote:
> 
> One issue not raised here: ephemeral port exhaustion. 

I believe I raised that issue early on, but it didn't really apply 
here so maybe I didn't actually post it.

> It's a rather
> specialized problem, more of an issue for Windows folks than Linux, and 
> mostly an issue for routers, proxies, and similar devices that tend to 
> funnel a large volume of mail to smaller number of destinations. But 
> it's a real problem, one where reusing existing client connections can 
> provide a complete and permanent resolution.

The ephemeral port exhaustion exploit began with Microsoft DNS server 
DoS attacks and their quick patch solution was to preallocate a large 
range of the 64K ports, but that created other port exhaustion 
problems with other socket and RPC server applications. That is 
probably what you meant above with other application servers reported 
connection problems all due to the DNS ports preallocation solution.

It was particular irritating when using TCP viewing tools and it 
listed, display thousands of the preallocated ports for DNS. It was 
useless.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: [email-masked]
Cc: "[email-masked]" <[email-masked]>
Date: Mon, 22 Aug 2011 15:02:46 -0700 (PDT)
Subject: RE: Mail Data termination
To: "Murray S. Kucherawy" <[email-masked]>
Sender: [email-masked]

> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> > Sent: Sunday, August 21, 2011 7:14 PM
> > To: [email-masked]
> > Subject: Re: Mail Data termination
> >
> > If you don't get caught up with the "Windows-hate" comments (and
> > incorrect assumptions about Windows), try to catch on that the
> > technology employed in Windows is a direction for some of the *nix
> > flavors. But the above is not about windows, but how Unix is trying to
> > catch up! <g>

> I didn't say anything about any specific operating system, nor am I interested in engaging such a dialog.

> What I did say is that if the OS (or indeed, the SMTP server) where you're seeing this can't tolerate a few idle connections hanging around, I'd start wondering if it's capable of operating at Internet scale and might benefit from a design review.

> At any rate, I agree with John Klensin's comment that this isn't a problem with the RFCs, so I think it's time to let this thread die.

+1

				Ned

Date: Mon, 22 Aug 2011 14:47:27 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: Steve Atkins <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

Steve Atkins wrote:
> Measuring actual costs (what is the resource cost of session setup and teardown, and of maintaining an open session, for well-implemented, mainstream MTAs?) would seem to be something to do before discussing such a document in much detail.

I've been measuring this for my own product (an SMTP proxy) mostly in 
the course of designing more rigorous test tools. (An unbreakable toy is 
useful for breaking other toys....)

Not surprisingly, at 1 second between messages, the load on my SMTP 
receiver is dramatically lower (and throughput markedly higher) when 
senders hold the connection open; doubly so for TLS, where the 
certificate exchange dominates. Equally unsurprising, when the sender 
holds the connection for 5 minutes and more, I'm having problems with 
hitting my limit on concurrent connections. I'm still looking for where 
the "sweet spot" is between. At present, I blindly 421 any sender that's 
been idle for 3 minutes, but that's little more than a SWAG. I'd really 
like to implement throttling based on a sender's actual load, not just 
on sockets held open; but I don't have the data to cost justify that. Yet.

One issue not raised here: ephemeral port exhaustion. It's a rather 
specialized problem, more of an issue for Windows folks than Linux, and 
mostly an issue for routers, proxies, and similar devices that tend to 
funnel a large volume of mail to smaller number of destinations. But 
it's a real problem, one where reusing existing client connections can 
provide a complete and permanent resolution.

<csg>

Date: Mon, 22 Aug 2011 17:23:26 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

For those interested, if basic laws of operations research, physics, 
science and Thermodynamics isn't good enough, then Believe Sendmail!

     http://www.sendmail.org/~ca/email/doc8.12/op-sh-4.html

     The ConnectionCacheTimeout ( K) option specifies the maximum time
     that any cached connection will be permitted to idle. When the idle
     time exceeds this value the connection is closed. *This number 
should be
     small (under ten minutes) to prevent you from grabbing too many 
resources
     from other hosts. The default is five minutes.*

Steve Atkins wrote:
> On Aug 22, 2011, at 7:04 AM, Dave CROCKER wrote:
> 
>>
>> Folks,
>>
>> I see a couple of folk suggesting that this thread has run its course.  I've been wondering about that.
>>
>> Given how long the thread has been and how much senior email talent has contributed to it, I'm entirely unclear what it has accomplished or where it can or should go.
>>
>> So I thought I'd ask.
>>
>> Perhaps this has merely been an extended consulting session to help a few folk?
>>
>> Given the amount of sage commentary has been posted, is there a document that should be produced?
> 
> Measuring actual costs (what is the resource cost of session setup and teardown, and of maintaining an open session, for well-implemented, mainstream MTAs?) would seem to be something to do before discussing such a document in much detail.
> 
> Cheers,
>  Steve
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 22 Aug 2011 13:52:43 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Steve Atkins <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: productivity?
Sender: [email-masked]

On 8/22/2011 1:30 PM, Steve Atkins wrote:
>> Given the amount of sage commentary has been posted, is there a document that should be produced?
>
> Measuring actual costs (what is the resource cost of session setup and teardown, and of maintaining an open session, for well-implemented, mainstream MTAs?) would seem to be something to do before discussing such a document in much detail.

That's certainly a concrete and potentially useful operational optimization 
study that could be performed.

Hmmm.  I wonder how much of such a study would be applicable to other apps, such 
as http?

Are any hands-on researchers watching this list?  If anyone knows such folk, it 
might be worth whispering in their ear.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Subject: Re: productivity? (was: Re: Mail Data termination)
From: Steve Atkins <[email-masked]>
Date: Mon, 22 Aug 2011 13:30:36 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On Aug 22, 2011, at 7:04 AM, Dave CROCKER wrote:

> 
> 
> Folks,
> 
> I see a couple of folk suggesting that this thread has run its course.  I've been wondering about that.
> 
> Given how long the thread has been and how much senior email talent has contributed to it, I'm entirely unclear what it has accomplished or where it can or should go.
> 
> So I thought I'd ask.
> 
> Perhaps this has merely been an extended consulting session to help a few folk?
> 
> Given the amount of sage commentary has been posted, is there a document that should be produced?

Measuring actual costs (what is the resource cost of session setup and teardown, and of maintaining an open session, for well-implemented, mainstream MTAs?) would seem to be something to do before discussing such a document in much detail.

Cheers,
 Steve

Date: Mon, 22 Aug 2011 15:19:05 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: [email-masked]
Subject: Re: productivity?
To: [email-masked]
Sender: [email-masked]

John C Klensin wrote:
>> Given the amount of sage commentary has been posted, is there
>> a document that should be produced?
> 
> I think that is exactly what Paul Smith and I were discussing in
> our most recent few exchanges.

I would like to thank Paul for recognizing the problem and concerns I 
highlighted and one that can get worst.

I leave it to the wisdom of the Sages and Senior Email Talent, the 
"usual folks" to do what they think is best for the global good and 
future of SMTP - or not.  SMTP specs breaking Peewee Vendor has 
already taken action to provide peewee SMTP operators the option to 
zap these CS clients and I'm going to market it as a "New and Improved 
50% throughput SMTP enhancement."

Mucho gracias for the amusement.

<CLICK>
-- 
HLS

Date: Mon, 22 Aug 2011 11:17:23 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: productivity? (was: Re: Mail Data termination)
Sender: [email-masked]

--On Monday, August 22, 2011 07:04 -0700 Dave CROCKER
<[email-masked]> wrote:

> 
> 
> Folks,
> 
> I see a couple of folk suggesting that this thread has run its
> course.  I've been wondering about that.
> 
> Given how long the thread has been and how much senior email
> talent has contributed to it, I'm entirely unclear what it has
> accomplished or where it can or should go.
> 
> So I thought I'd ask.
> 
> Perhaps this has merely been an extended consulting session to
> help a few folk?
> 
> Given the amount of sage commentary has been posted, is there
> a document that should be produced?

I think that is exactly what Paul Smith and I were discussing in
our most recent few exchanges.

FWIW, I do believe that at least a subset of the discussions are
substantive and of broad applicability --at least to the
advantages of getting the analysis out there-- and not just
"extended consulting".  Other bits may be fairly close to the
latter, but probably the best way to sort the categories out
would be to start constructing a document.

best,
   john

    john

Date: Mon, 22 Aug 2011 07:04:18 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: productivity? (was: Re: Mail Data termination)
Sender: [email-masked]

Folks,

I see a couple of folk suggesting that this thread has run its course.  I've 
been wondering about that.

Given how long the thread has been and how much senior email talent has 
contributed to it, I'm entirely unclear what it has accomplished or where it can 
or should go.

So I thought I'd ask.

Perhaps this has merely been an extended consulting session to help a few folk?

Given the amount of sage commentary has been posted, is there a document that 
should be produced?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Mon, 22 Aug 2011 14:00:09 +0100
From: Paul Smith <[email-masked]>
To: John C Klensin <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 22/08/2011 13:46, John C Klensin wrote:
> worse. When I resist changing the 5321 limits, I do so both because I 
> am generally concerned about the procedural implications of protocol 
> changes at this state but also because there are still environments 
> around the world with lower-performance clients and/or servers, lower 
> bandwidth than the big server farms, or even intermittent connections. 
> Servers with intermittent or flaky connections make strategies like 
> "once you determine that you can open a connection and get a message 
> through to a host, gather up everything you can find and send it as 
> part of the same connection and, especially if the connection took 
> several tries to establish, maybe keep it open for short while to see 
> if anything else can be pushed through it" really sensible. Our 
> colleagues in the DTN business would probably suggest that "around the 
> world" should be expanded to "around the solar system" (at least) and 
> point out that the same sorts of strategies are even more important 
> for them. 
I wasn't actually suggesting that the timeout suggestions be simply 
lowered, just that I'd like more stuff about why they are that big, why 
they may need to be shorter, what the implications are etc.

FWIW - I'd have thought having an MX server on an intermittent/flaky 
connection would be a "bad thing"... Anyone who wants to get email 
should have a reasonably reliable connection to their MX server, even if 
then the final delivery server is on a flaky connection.

I still think that 'pre-queuing' rather than 'post-waiting' is more 
community spirited - pushing everything you can through once you have a 
connection is eminently sensible - hanging around just in case more 
comes along might stop other people getting messages through to this 
flaky connection which has just had its few seconds of connectivity for 
the day.

> In that environment, the strategy is not only not antisocial, it is 
> hugely beneficial, not just to the sender but to the receiver and the 
> users/mailboxes it supports. 
Not if it prevents other senders getting their messages through while 
the first sender was sitting with an idle connection using up the 
connection quota.

Keeping connections open 'too long' is fine if you assume that the 
receiver has an unlimited number of connections available. Once you 
accept that this is a limited resource in many cases, then it becomes a 
much nastier thing to do. If there's an unlimited number of connections, 
then an idle connection costs the receiver nothing. If the receiver is 
limited to, say, 10 simultaneous connections, then the idle connection 
costs the receiver 10% of its capacity.

> My sense is that there have been some very careful discussions and 
> analyses in this thread (along with the inevitable small amount of 
> complete noise). Probably a good foundation for such a document could 
> be to simply draw that material together, eliminate the noise, and 
> organize it coherently. Because of differences among environments I 
> think it would actually be most useful as a discussion of the issues 
> and analysis of what might make sense than as any sort of normative 
> specification
Agreed.

Date: Mon, 22 Aug 2011 08:46:18 -0400
From: John C Klensin <[email-masked]>
To: Paul Smith <[email-masked]>
cc: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Monday, August 22, 2011 09:42 +0100 Paul Smith
<[email-masked]> wrote:

>...
 
>> This is _not_ a standards problem, at least IMO.
>> 
> Well, I have issues with the timeout suggestions in 5321,
> which I do think need more discussion. I appreciate that the
> 'SHOULD' means you are allowed to ignore that suggestion if
> there are 'valid reasons' to do so, but it is not clear at all
> what implications may be.

Certainly 5321 and both of its predecessors were written for
times in which SMTP clients and servers could be assumed to be
cooperative in getting messages through, in which message
delivery (and NDN delivery) were everyone's primary objectives,
and where network behavior was the main enemy, not attackers of
a variety of types.  Certainly they were not written for
situations in which the attackers included client or servers
designed to get best performance (however measured or evaluated)
for their environments by making performance for everyone else
worse.

When I resist changing the 5321 limits, I do so both because I
am generally concerned about the procedural implications of
protocol changes at this state but also because there are still
environments around the world with lower-performance clients
and/or servers,  lower bandwidth than the big server farms, or
even intermittent connections.  Servers with intermittent or
flaky connections make strategies like "once you determine that
you can open a connection and get a message through to a host,
gather up everything you can find and send it as part of the
same connection and, especially if the connection took several
tries to establish, maybe keep it open for short while to see if
anything else can be pushed through it" really sensible.  Our
colleagues in the DTN business would probably suggest that
"around the world" should be expanded to "around the solar
system" (at least) and point out that the same sorts of
strategies are even more important for them.
 
> To be honest, I'd never thought any major sender would use
> connection caching when sending mail as it is (to my
> viewpoint) a very antisocial thing to do.

See above.  At least some of the thinking that produced that
behavior comes from environments in which the profile, capacity,
and connections of a server made getting a connection really
hard (no matter how many cycles the connection setup and
teardown did or did not take).   In that environment, the
strategy is not only not antisocial, it is hugely beneficial,
not just to the sender but to the receiver and the
users/mailboxes it supports.

Maybe, in a perfect world, a client system would try to figure
out and remember the difference between a server like that and a
high-performance server that was well and continuously
connected.  Like caching MX targets, one should do that only
with great care and sophistication, lest one cause far more
serious problems in an environment that can change between queue
runs.  Maybe it isn't worth the trouble -- as with other things,
I (and many others) can tell you how to make these statistical
projections from history work, but I can't advise on whether
they are sensible or worth the tradeoffs without knowing a huge
amount about a particular situation and code base.

> So, until this
> discussion, it had never entered my mind that we needed to do
> anything about it. We do have to fight against 'DoS' attacks,
> which might, in fact, turn out to be 'legitimate' connection
> caching, but because we've been treating it as attacks, we've
> been handling it differently than if it was a legitimate, but
> unwanted, behaviour.
> 
> Maybe an info or BCP document would be useful

My sense is that there have been some very careful discussions
and analyses in this thread (along with the inevitable small
amount of complete noise).  Probably a good foundation for such
a document could be to simply draw that material together,
eliminate the noise, and organize it coherently.  Because of
differences among environments I think it would actually be most
useful as a discussion of the issues and analysis of what might
make sense than as any sort of normative specification, buy
YMMD.  If someone has both the time and interest, I'd strongly
encourage it.  I'd guess that the usual suspects for producing
such documents are too busy (I know I am), but that should imply
"someone else needs to pick up the pen if it is to be done" and
not "it isn't worthwhile".

    john

Date: Mon, 22 Aug 2011 07:33:11 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Paul Smith wrote:
> 
> To be honest, I'd never thought any major sender would use connection 
> caching when sending mail as it is (to my viewpoint) a very antisocial 
> thing to do. So, until this discussion, it had never entered my mind 
> that we needed to do anything about it. We do have to fight against 
> 'DoS' attacks, which might, in fact, turn out to be 'legitimate' 
> connection caching, but because we've been treating it as attacks, we've 
> been handling it differently than if it was a legitimate, but unwanted, 
> behaviour.

+1.  I felt the same way.

> Maybe an info or BCP document would be useful

I'm on the fence on how it can be approached since opening up this 
door will most likely  require some cleanup of related topics pointing 
to 5321 and still having 1 conflict with 821, starting why QUIT is 
required in 5321 and not in 821.  And if you recall not too long ago, 
the long debate about not slowing down the DATA EOD response.  Well, 
that concern should be moot now with the CS client holding times 
negating all efforts to speed up DATA EOD filtering. :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 22 Aug 2011 12:54:25 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: SMTP Interest Group <[email-masked]>
Subject: Re: Security problems with SRV records
Sender: [email-masked]

On 08/20/2011 01:03 PM, Alessandro Vesely wrote:
> Assume I have a bunch of email addresses, some bots, and some spam to send.
> I'd run something something like this pseudo-code:

Why would you run that instead of copying code from e.g. Thunderbird or 
K9? The K9 source code is available right now from github or whatever, 
it works with many more addreses than the approach you suggest, and you 
may expect it to continue to work well, since choosing those hostnames 
makes client configuration simple.

Arnt

Date: Mon, 22 Aug 2011 06:15:49 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> 
> Hector answers me:
>>> OK, so suppose you're a sender.
>>>
>>> Your best friend is a queue manager.
>>>
>>> You've just finished sending a message. Now you send a message to tell
>>> your best friend you're done. Do you, or don't you, keep the
>>> connection open just in case your friend has more work for the same
>>> connection? How long?
>>
>> Haven't you ever told someone?
> 
> Don't evade, please.

I'm sorry, I thought I did give me take for an answer to your question.

I'll try again.

No, you don't keep the connection open waiting for new mail from your 
friend.  But if you did, you could not wait for more than 5 mins.   So 
you ask, how long that is less than 5 mins?

Well the less the better and hopefully no more than 0.15 seconds or 
CPU Quantum that forces a page swap, context switching.

The point is that you might get away with this for one or two friends, 
but once ALL friends did this,  the receiver and all clients will 
begin to see problems.

The problem overall is some see this is a good thing, it might be for 
some, but I don't think its a good thing in the wide network of 
receivers and clients. The more people do this, the closer you fell 
the negatives and there scientific formulas that will explain why.  We 
already see the negatives with increase sessions times. small but 
definitely there to see.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 22 Aug 2011 11:19:17 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Hector answers me:
>> OK, so suppose you're a sender.
>>
>> Your best friend is a queue manager.
>>
>> You've just finished sending a message. Now you send a message to tell
>> your best friend you're done. Do you, or don't you, keep the
>> connection open just in case your friend has more work for the same
>> connection? How long?
>
> Haven't you ever told someone?

Don't evade, please.

Arnt

Date: Mon, 22 Aug 2011 09:42:06 +0100
From: Paul Smith <[email-masked]>
To: John C Klensin <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 21/08/2011 21:12, John C Klensin wrote:
>
> --On Sunday, August 21, 2011 14:54 +0200 Paul Smith
> <[email-masked]>  wrote:
>
>> ...
>
>> If the receiver does load limiting by restricting the number
>> of open connections, then it is selfish for the sender to
>> presume that it can keep hold of one of these connections for
>> longer than necessary.
>>
>> I agree that a few seconds is probably not significant, but if
>> 5 seconds grows to 15 seconds, then 60 seconds etc, then it
>> could become a big issue for some servers. OK, so the senders
>> are also having larger numbers of open connections, but this
>> is under their control, not the receiver's.
>>
>> The sender also has the privilege of being able to decide when
>> and what to send, whereas the receiver has to respond in a
>> timely manner to the sender's requests - thus the sender is
>> under much less pressure than the receiver. The sender can
>> keep 1000 connections open and know that it will only send to
>> one at once, but if the receiver has 1000 connections open it
>> has to be able to respond to all of them quickly, even if all
>> 1000 decide to send a message at the same time. The receiver
>> has to respond quickly, to stop the sender (which generally
>> use very short timeouts, regardless of what 5321 says) from
>> timing out prematurely and retrying unnecessarily.
>> ...
> Paul,
>
> First, no one has really suggested "grows to 15 seconds, then 60
> seconds etc.".  As far as I can tell from this thread, there is
> general agreement that deliberately waiting a minute or more
> would be abusive.
In this thread - yes, but not in any published document that I know of.

5321 says the server should wait at least 5 minutes for the next 
command. Some client writers may take this as permission to hold an idle 
connection that long.

> This is _not_ a standards problem, at least IMO.
>
Well, I have issues with the timeout suggestions in 5321, which I do 
think need more discussion. I appreciate that the 'SHOULD' means you are 
allowed to ignore that suggestion if there are 'valid reasons' to do so, 
but it is not clear at all what implications may be.

To be honest, I'd never thought any major sender would use connection 
caching when sending mail as it is (to my viewpoint) a very antisocial 
thing to do. So, until this discussion, it had never entered my mind 
that we needed to do anything about it. We do have to fight against 
'DoS' attacks, which might, in fact, turn out to be 'legitimate' 
connection caching, but because we've been treating it as attacks, we've 
been handling it differently than if it was a legitimate, but unwanted, 
behaviour.

Maybe an info or BCP document would be useful

Date: Mon, 22 Aug 2011 09:14:54 +0100
From: Paul Smith <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

On 22/08/2011 06:06, Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
>> Sent: Sunday, August 21, 2011 6:59 PM
>> To: [email-masked]
>> Subject: Re: Mail Data termination
>>
>> Is there a loading limit in your arsenal against DoS attacks?  Is
>> there such a thing as resource limits even for a Modern OS?  And is it
>> reasonable for the CS client to pass the cost burden on the receiver
>> to solve what is only a client problem?
> I think you're ignoring the points others have made that a connection cache benefits both sides, since establishing a connection is a (relatively) expensive action at both ends, especially if SMTP AUTH or STARTTLS are in use.  It doesn't strictly benefit the sender.
It doesn't benefit the receiver anywhere near as much as the sender.

The sender can open 50,000 connections to 50,000 different mail servers, 
and just hold 49,999 of them idle and send one message at once = very 
low CPU/bandwidth load. The receiver can't, as easily, hold open 50,000 
connections from 50,000 different clients and be able to handle up to 
50,000 incoming messages simultaneously in a timely manner.

The sender knows which connections will be used at any time. The 
receiver has to assume they may all be used at any one time, so has to 
have the capacity to handle them all. So, it's a very unequal relationship.

Date: Mon, 22 Aug 2011 02:57:42 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
To: [email-masked]
Sender: [email-masked]

Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
>> Sent: Sunday, August 21, 2011 6:59 PM
>> To: [email-masked]
>> Subject: Re: Mail Data termination
>>
>> Is there a loading limit in your arsenal against DoS attacks?  Is
>> there such a thing as resource limits even for a Modern OS?  And is it
>> reasonable for the CS client to pass the cost burden on the receiver
>> to solve what is only a client problem?
> 
> I think you're ignoring the points others have made that a connection 
> cache benefits both sides, since establishing a connection is a (relatively) 
> expensive action at both ends, especially if SMTP AUTH or STARTTLS are in use.  
> It doesn't strictly benefit the sender.

You didn't answer the questions above. You can't state this without 
answering the above.

I ignored it because they are common factors on both ends and ones 
that a receiver can control. Its part of the SMTP client/server design 
process and the tweaking is already in place which may include 
disabling STARTTLS, AUTH or any other common factor.

We are taking about pure science:

        INPUT = OUTPUT

and a new timing factor that is out of the receiver control that 
alters the entire balance of the system.  Its impossible to maintain 
steady state with a factor the client owns and the receiver has no 
control. There is only one way for the receiver to get back control - 
break the SMTP spec and drop the client after the 250 EOD response.

No impact on the client because he will finally solve its problem by 
waiting 5 seconds to get more mail for the destination host before 
starting its expensive trip.  It could even afford to wait 10, 30 
seconds or even 1 minute to increase the odds of getting more mail and 
then send it as fast as possible and none of this will have any impact 
on the receiver other than more transactions which is an acceptable 
SMTP concept.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sun, 21 Aug 2011 22:12:49 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Sunday, August 21, 2011 7:14 PM
> To: [email-masked]
> Subject: Re: Mail Data termination
> 
> If you don't get caught up with the "Windows-hate" comments (and
> incorrect assumptions about Windows), try to catch on that the
> technology employed in Windows is a direction for some of the *nix
> flavors. But the above is not about windows, but how Unix is trying to
> catch up! <g>

I didn't say anything about any specific operating system, nor am I interested in engaging such a dialog.

What I did say is that if the OS (or indeed, the SMTP server) where you're seeing this can't tolerate a few idle connections hanging around, I'd start wondering if it's capable of operating at Internet scale and might benefit from a design review.

At any rate, I agree with John Klensin's comment that this isn't a problem with the RFCs, so I think it's time to let this thread die.

-MSK

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sun, 21 Aug 2011 22:06:12 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Sunday, August 21, 2011 6:59 PM
> To: [email-masked]
> Subject: Re: Mail Data termination
> 
> Is there a loading limit in your arsenal against DoS attacks?  Is
> there such a thing as resource limits even for a Modern OS?  And is it
> reasonable for the CS client to pass the cost burden on the receiver
> to solve what is only a client problem?

I think you're ignoring the points others have made that a connection cache benefits both sides, since establishing a connection is a (relatively) expensive action at both ends, especially if SMTP AUTH or STARTTLS are in use.  It doesn't strictly benefit the sender.

> Let them connect again if
> they wish to send another message.

I think you wouldn't like the performance impact of clients doing that across the board.

Date: Sun, 21 Aug 2011 22:14:19 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
To: [email-masked]
Sender: [email-masked]

Murray S. Kucherawy wrote:

> I don't see how keeping a connection open creates extra load in a 
> modern operating system.  An open file descriptor is just an entry 
> in a kernel table.

Murray, the following links are two good articles that talks about 
*nix environments issues and evolution dealing with high scale designs:

    http://www.kegel.com/c10k.html
    http://tinyclouds.org/iocp-links.html

If you don't get caught up with the "Windows-hate" comments (and 
incorrect assumptions about Windows), try to catch on that the 
technology employed in Windows is a direction for some of the *nix 
flavors. But the above is not about windows, but how Unix is trying to 
catch up! <g>

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 21 Aug 2011 21:59:18 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy wrote:

> And if you somehow have so many open connections from clients that 
> you can't accept any more and another connection tries to arrive, 
> select the one that's been idle the longest, close it down 
> (releasing resources), and accept the new one.  Or just generally 
> trim your idle timeout down.

Is there a loading limit in your arsenal against DoS attacks?  Is 
there such a thing as resource limits even for a Modern OS?  And is it 
reasonable for the CS client to pass the cost burden on the receiver 
to solve what is only a client problem?

>> If the sender doesn't like closing and opening connections, then surely
>> it's just as beneficial for the sender to wait 10 seconds before
>> starting to send a message, then if another message to the same
>> destination arrives within that 10 seconds, it can batch them up without
>> adding unnecessary load to the receiver.
> 
> That's exactly how the connection cache works in some instances, depending 
> on whether or not the sender is configured to make an immediate delivery 
> attempt, or just wait for a queue run.

Thats better than a Connection Hogging design.  They should wait the 5 
seconds themselves and don't push its problem to receivers.

> I'm really surprised there's so much sudden consternation about 
> this feature given how many MTAs have it, and the fact that 
> it's been around for well over a decade.  Somehow, in that time, 
> the sky hasn't fallen.

It took a long time for people to see the SKY is fallen with pollution 
and constant abuse before action is taken.

The effects has been marginal by keeping small holding time but it has 
already reduced higher potential throughput with average sessions 
times between 4-7 and mostly like explains why some systems see false 
positives in DoS Attacks.

Connection Hogging is not an cooperative engineering design that only 
serves the client only, not the receiver and new SMTP ideas will 
emerge to get back fast session transactions such as a SERVER instant 
drop after the ONE transaction is complete.  Let them connect again if 
they wish to send another message.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 21 Aug 2011 19:06:29 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> 
> On 08/20/2011 09:41 PM, Paul Smith wrote:
>> I'm with Hector on this. I really don't like the idea of a sender
>> keeping a connection open 'just in case'.
> 
> OK, so suppose you're a sender.
> 
> Your best friend is a queue manager.
> 
> You've just finished sending a message. Now you send a message to tell 
> your best friend you're done. Do you, or don't you, keep the connection 
> open just in case your friend has more work for the same connection? How 
> long?

Haven't you ever told someone?

        "Hey, I only have two hands!"

:)

The issue is when you have 5, 10, 20, 30, 100, etc friends doing the 
same and you reached a capacity (not enough hands) not only to handle 
your friends but also unknown people trying to just send you one 
message and has no interest to wait around any longer.

Its cool if its prearranged among a few friends, but I doubt you be 
able to do this with all your friends. :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 22 Aug 2011 00:13:03 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: Paul Smith <[email-masked]>
Cc: [email-masked], "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 08/20/2011 09:41 PM, Paul Smith wrote:
> I'm with Hector on this. I really don't like the idea of a sender
> keeping a connection open 'just in case'.

OK, so suppose you're a sender.

Your best friend is a queue manager.

You've just finished sending a message. Now you send a message to tell 
your best friend you're done. Do you, or don't you, keep the connection 
open just in case your friend has more work for the same connection? How 
long?

Arnt

Date: Sun, 21 Aug 2011 17:29:23 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: Paul Smith <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Sunday, August 21, 2011 16:57 -0400 Hector Santos
<[email-masked]> wrote:

>...
> While I don't think this warrant any immediate concern, the
> day 5321bis does opens up, I think you should be ready for
> discussion regarding timeouts among other related things
> regarding multi-transaction sections.  :)

And, of course, predictions of that type are precisely why there
has been some reluctance to open 5321.

So I will say what I've said about many comments of that general
type.   If you (or others) thing there is an issue that requires
a substantive protocol change in 5321bis, then generate an I-D
to update 5321 itself with the change and see if you you get
consensus on it.  If so, fine.  If not, let's not use it derail
moving forward with a 5321 update.

    john

Date: Sun, 21 Aug 2011 16:57:53 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Paul Smith <[email-masked]>, Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

John C Klensin wrote:

> This is _not_ a standards problem, at least IMO.

I guess the point is the CS holding time is "unnatural" from a 
receiver point of view and if the insight was available long ago, I 
believe it would have changes a quite few things, including a 
consideration in receiver loading limits to take into account that 
capacity limits cause by caused by the increase use of CS clients 
deliberated slowing down the receivers.

If RFC5322 stated:

      A 5 minute idle time is required after a transaction was 
complete in order
      to allow the sender the opportunity to start a new transaction.

Then we have a different story because now both client and server 
designers are in sync with SMTP client/server high scale throughput 
modeling.

But it doesn't not say that.  It does say to allow for new transaction 
but I don't think most  presumed that a client will deliberate wait X 
time at this point because of their (client side) queuing problem is 
easily solved with other client-side only solutions with the same end 
result without impacting SMTP server designs or causing any new SMTP 
implementation considerations, including considerations that further 
goes against its recommendations.

For example, I already added an operator option to lower the Idle Time 
after the first transaction is completed to a default 1 minute.

It changes the design picture when the discovery of client with CS 
behavior is highlighted.

While I don't think this warrant any immediate concern, the day 
5321bis does opens up, I think you should be ready for discussion 
regarding timeouts among other related things regarding 
multi-transaction sections.  :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 21 Aug 2011 16:12:16 -0400
From: John C Klensin <[email-masked]>
To: Paul Smith <[email-masked]>, Alessandro Vesely <[email-masked]>
cc: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Sunday, August 21, 2011 14:54 +0200 Paul Smith
<[email-masked]> wrote:

>...
 
> If the receiver does load limiting by restricting the number
> of open connections, then it is selfish for the sender to
> presume that it can keep hold of one of these connections for
> longer than necessary.
> 
> I agree that a few seconds is probably not significant, but if
> 5 seconds grows to 15 seconds, then 60 seconds etc, then it
> could become a big issue for some servers. OK, so the senders
> are also having larger numbers of open connections, but this
> is under their control, not the receiver's.
> 
> The sender also has the privilege of being able to decide when
> and what to send, whereas the receiver has to respond in a
> timely manner to the sender's requests - thus the sender is
> under much less pressure than the receiver. The sender can
> keep 1000 connections open and know that it will only send to
> one at once, but if the receiver has 1000 connections open it
> has to be able to respond to all of them quickly, even if all
> 1000 decide to send a message at the same time. The receiver
> has to respond quickly, to stop the sender (which generally
> use very short timeouts, regardless of what 5321 says) from
> timing out prematurely and retrying unnecessarily.
>...

Paul,

First, no one has really suggested "grows to 15 seconds, then 60
seconds etc.".  As far as I can tell from this thread, there is
general agreement that deliberately waiting a minute or more
would be abusive.

Second, since you have already moved us into the territory of
"... regardless of what 5321 saves", let me suggest something
for the design of such receivers.  Note that, while I suggest
doing it, I'd be strongly opposed to suggestions to incorporate
it into 5321bis.  As you have probably gathered from my earlier
note, I'm rather a big fan of basing operational decisions on
statistical prediction from recent history.  It would seem
perfectly rational to me for a receiver to measure and keep
track of how long it takes a sender within a given SMTP session
to issue another command after the receiver sends back a reply.
For an SMTP session, that gives the receiver an absolute minimum
of five measurements before EOD arrives (EHLO, MAIL, RCPT, DATA.
beginning of data stream after the 354 is sent).  For a
multiple-recipient message or multiple messages in the same
session, the number of measurements gets bigger.   Now, suppose
the median of the measurements taken is X and the first upper
hinge* is Y.  I'd think that a receiver, after sending a 250
response to EOD, would be perfectly rational to assume that the
sender had either lost touch with the connection or was screwing
with it after waiting an order of magnitude or so more than Y.  

Note that, if the sender took a half-second to respond with a
new command (or data stream) after receiving a response code and
its response times were fairly stable, the five or six seconds
people have been talking about would be pretty much on target
with that model of analysis.   Whether one used that sort of
formula or not, the point is that, if one were a receiver
concerned enough about the interaction of per-sender behavior
with attacks or abuse, it would be pretty easy to differentiate
between a sender that was holding a connection open deliberately
and for an excessive time to see if more traffic came in and one
that was just slow.  What to do with that knowledge would
obviously be a design or operational issue for the receiver.

This is _not_ a standards problem, at least IMO.

    john

Date: Sun, 21 Aug 2011 15:36:44 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Peter J. Holzer wrote:

> Not quite - there is also at least the per-connection data in the MTA -
> this may be only a small data structure or a whole process.
> 
>> I've never seen an idle connection cause distress to a running system.
> 
> We've run into the connection limit a number of times on our MXs, but
> that's because I've set that too conservatively (there was still plenty
> of free RAM when the limit was reached).

I see it all the time, but randomly.  But loading limits design to 
control the attacks and throttle it will allow you and your customers 
to go to sleep like a baby.  I just never considered that CS clients 
can be contributing to these random capacity limits events always to 
be due to 100% related to DoS Attacks.

If you are seeing 6.7 secs and my is averaging about the same, then 
that seems to suggest we already seen the affects of CS clients out 
there lowering the highest potential throughput of receivers.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 21 Aug 2011 14:54:14 +0200
From: Paul Smith <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 21/08/2011 13:21, Alessandro Vesely wrote:
>> Its a selfish idea from the standpoint that it considers its own benefits by
>> ignoring the impact at the receiver. But I do think its a useful method if
>> its prearrange with the receiver in the same way we white list specific
>> senders (i.e, Allow Relay IP tables, smart hosting, etc).
> It is not selfish, since per-connection costs are experienced by receivers too.
>

If the receiver does load limiting by restricting the number of open 
connections, then it is selfish for the sender to presume that it can 
keep hold of one of these connections for longer than necessary.

I agree that a few seconds is probably not significant, but if 5 seconds 
grows to 15 seconds, then 60 seconds etc, then it could become a big 
issue for some servers. OK, so the senders are also having larger 
numbers of open connections, but this is under their control, not the 
receiver's.

The sender also has the privilege of being able to decide when and what 
to send, whereas the receiver has to respond in a timely manner to the 
sender's requests - thus the sender is under much less pressure than the 
receiver. The sender can keep 1000 connections open and know that it 
will only send to one at once, but if the receiver has 1000 connections 
open it has to be able to respond to all of them quickly, even if all 
1000 decide to send a message at the same time. The receiver has to 
respond quickly, to stop the sender (which generally use very short 
timeouts, regardless of what 5321 says) from timing out prematurely and 
retrying unnecessarily.

So, the per-connection cost is NOT the same for the sender as for the 
receiver, it is much cheaper for the sender.

For the receiver to unilaterally 'close the longest idle' connection, as 
has been suggested before, risks penalising slow senders while not 
affecting those which are deliberately being greedy, and would also be 
'frowned upon' by RFC 5321 which says the server should have a timeout 
of at least 5 minutes. (It's allowed for it to be shorter, but why would 
the 5 minute figure be documented, unless it should be stuck to?)

Date: Sun, 21 Aug 2011 13:21:18 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 20.08.2011 15:28, Hector Santos wrote:
> Alessandro Vesely wrote:
>> For example, a mass mailer script that produces personalized messages for
>> each recipient can simply hand them to the MSA in the right order.  The
>> number of seconds that a connection remains dormant needs just be enough for
>> the next message to be generated and submitted.
> 
> Agree, but only from the design perspective of one side to make it work at
> the expense of presuming there is a unlimited number servers and there are no
> other clients doing the same thing and the server capacity is rarely reached
> so that server availability is not impacted and the server cost to maintain
> steady state performance is minimal.

Disregarding MUA-to-MSA traffic, servers acting as SMTP client globally send
about the same number of messages that SMTP servers receive.  Hence, their
availabilities must be just proportional to one another.  Not unlimited.

> Its a selfish idea from the standpoint that it considers its own benefits by
> ignoring the impact at the receiver. But I do think its a useful method if
> its prearrange with the receiver in the same way we white list specific
> senders (i.e, Allow Relay IP tables, smart hosting, etc).

It is not selfish, since per-connection costs are experienced by receivers too.

> This can be easily modeled and simulated using this formula:
> 
>    C = R /(S+H)
> 
> where
> 
>    R = Total number of available receivers
>    S = Average Session Time
>    H = Average Holding Time (CS cache wait time), and
>    C = Arrival rate of new clients

This model only takes into account a disadvantage of Connection Sharing.  It
should have considered the possibility that a connection is actually shared
and multiple messages delivered during a single session.  For example,

     M = R /(S + (H + O)/N)

where the differences from above are

     S = Average _transaction_ time,
     M = Arrival rate of messages,
     N = Average transactions per session, and
     O = Overhead per session (TCP, TLS, EHLO, and QUIT.)

> Example #1 - No CS consideration H=0
> 
>    R = 100 receivers
>    S = 5 secs

Let's say  S + O = 5 secs; e.g.

     S = 4
     O = 1

>    H = 0 secs
> 
> Then for the maximum efficient, high production steady state condition:
> 
>    C = 100/(5+0) or 20 CPS (clients per sec)

Yes, disregarding queues, H = 0 implies N = 1, and thus M = C.

> This means that all 100 receivers are busy 100% at all times. It is steady
> state when C does not increase.
> 
> Example #2: CS considered, H=5

Let's say H = 5 _seconds_, since we said that's more reasonable than 5
minutes.  For this oversimple model, the break even is at H = O*(N-1).
Indeed, if we hypothesize we receive an average of N = 6 messages per
session, we get

     M = 100 /(4 + (5 + 1)/6) = 20 messages per sec, as in example #1.

Date: Sun, 21 Aug 2011 12:07:06 +0200
From: "Peter J. Holzer" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Mail-Followup-To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

--4Ckj6UjgE2iN1+kY
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2011-08-20 23:18:15 -0700, Murray S. Kucherawy wrote:
> > From: Paul Smith [mailto:[email-masked]]
> > Sent: Saturday, August 20, 2011 12:41 PM
> >=20
> > I'm with Hector on this. I really don't like the idea of a sender
> > keeping a connection open 'just in case'.
> >=20
> > If it keeps it open for 10 seconds 'just in case', and no new mail
> > arrives so it has to close the connection, what has that achieved other
> > than extra load on the receiver? Why wouldn't it then say 'well, another
> > message may arrive in the next few seconds, so I'll keep it open just a
> > bit longer', and so on.
>=20
> I don't see how keeping a connection open creates extra load in a
> modern operating system.  An open file descriptor is just an entry in
> a kernel table.

Not quite - there is also at least the per-connection data in the MTA -
this may be only a small data structure or a whole process.

> I've never seen an idle connection cause distress to a running system.

We've run into the connection limit a number of times on our MXs, but
that's because I've set that too conservatively (there was still plenty
of free RAM when the limit was reached).

> Even for an application that creates a complete subprocess to handle
> that idle connection, the process would eventually get paged or
> swapped out in favour of more active processes.

We are talking about a few seconds here: Postfix has a default limit of
2 seconds, 5 seconds also seems to be a rather popular limit (see below).
I think if your MTA processes are being swapped out within 2-5 seconds
of idle time you should probably invest in more RAM (or reduce the
connection limit).

There are also clients (mostly bots, I think) which don't
disconnect at all and just hang around until the server disconnects
them. These might be swapped out, and they could be a problem because
there are potentially a lot of them.

> I'm really surprised there's so much sudden consternation about this
> feature given how many MTAs have it, and the fact that it's been
> around for well over a decade.  Somehow, in that time, the sky hasn't
> fallen.

The consternation is because Hector misread his logfiles (reading 5
minutes instead of 5 seconds) and then didn't want to back down.

I just computed a few statistics over the log files of the last week of
our lowest-numbered MX:

Average session duration is 6.7 seconds, Average delay between the reply
code to the last command and QUIT is 2.2 seconds. So delayed quit does
increase the average session duration by almost 50%.

More interesting is the distribution:

After a DATA command, most clients (86%) send a QUIT after 0 or 1
seconds (our log files only have a time resolution of 1 second, so these
are probably the "send QUIT immediately" cases). If there is a peak at 2
seconds (postfix default) it is two small to be noticed. There is small
peak at 5 seconds (6%), and even smaller at 30 seconds (2.5%) and a
really tiny at 60 seconds (0.3%). The=20

After a RSET it's a bit different: Only 76% send a QUIT after 0 or 1
seconds, 10% after 5 seconds and there seem to be small peaks of
decreasing size every 20 seconds.

If any other command was the last one, the only peak was at 0 seconds.

I haven't checked how long connections stay open which aren't correctly
terminated with QUIT (I probably should, there are a lot of them).

	hp

--=20
   _  | Peter J. Holzer    | Web 2.0 k=F6nnte man also auch =FCbersetzen als
|_|_) | Sysadmin WSR       | "Netz der kleinen Geister".
| |   | [email-masked]         |=20
__/   | http://www.hjp.at/ |  -- Oliver Cromm in desd

--4Ckj6UjgE2iN1+kY
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

--4Ckj6UjgE2iN1+kY--

Date: Sun, 21 Aug 2011 04:35:19 -0400
From: John C Klensin <[email-masked]>
To: Bill McQuillan <[email-masked]>, SMTP Discussion <[email-masked]>
cc: Paul Smith <[email-masked]>, [email-masked], "Murray S. Kucherawy" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Saturday, August 20, 2011 16:19 -0700 Bill McQuillan
<[email-masked]> wrote:

> On Sat, 2011-08-20, Paul Smith wrote:
>... 
>> I'm with Hector on this. I really don't like the idea of a
>> sender keeping a connection open 'just in case'.
> 
>> If it keeps it open for 10 seconds 'just in case', and no new
>> mail arrives so it has to close the connection, what has that
>> achieved other than extra load on the receiver? Why wouldn't
>> it then say 'well, another message may arrive in the next few
>> seconds, so I'll keep it open just a bit longer', and so on.
> 
>> If the sender doesn't like closing and opening connections,
>> then surely it's just as beneficial for the sender to wait 10
>> seconds before  starting to send a message, then if another
>> message to the same  destination arrives within that 10
>> seconds, it can batch them up without adding unnecessary load
>> to the receiver.
>...
 
> It seems to me that the measure is NOT the "number of clients"
> but rather the number of messages handled.
> 
> Each client will have a certain number of messages to transmit
> to the receiver over a time period, regardless of the number of
> sessions used. When a client saves a session teardown and
> reestablishment by delaying QUIT, so does the receiver. So it
> would seem that a delay on the order of that time IS
> warranted. I don't have a good feel for the amount of that
> overhead, but I would guess that a second or two is reasonable
> and ten seconds is approaching too long.
>...

Folks, while this discussion started as a question about the
requirements of SMTP, it seems to be edging past even one about
operational optimizations and into a philosophical debate.  Let
me made one more observation, after which I'm dropping out the
thread.

Optimizations in queuing theory are a complex business,
especially if one tries to be really analytical about them.  In
a perfect world for the optimizer, a sending system would keep
track of traffic (both number and size of messages) to each
destination over a relatively long period of time, progressively
de-weighting older data (rather than just discarding them).  The
patterns of those data would then be used to build Bayesian
predictions of whether a new message to a given destination was
likely to be an isolated event or the beginning of a flow (to
that destination or others) with predictable characteristics.
Those predictions could, in turn, be used to condition decisions
about whether, and for how long, it was useful to keep a
connection open after sending the last queued message (noting
that the discussion has slid from "a few seconds" to "ten
seconds" and that difference, in todays's world, comes pretty
close to setting up and attacking a strawman).

One might use much the same data to predict whether delivery for
a message that is newly-arrived for sending should be tried
immediately (or as close to that as is feasible) or queued.  It
is almost obvious optimization that systems that try immediate
delivery of new message and succeed should drain any messages
queued for that destination in the same connection, but there
are scenarios (albeit usually unlikely ones) that actually make
that a bad strategy.  At least in the absence of deliberate
malicious attacks, one could diagnose or predict those
pathologies from an adequate database and set of predictive
procedures.  In addition, being able to drain the set of
messages queued for a particular destination implies that one's
queuing mechanisms are arranged to permit that: if, for example,
there is only a single queue and it is purely sequential, with
no destination threading or indexing, the amount of time and
effort required to find the messages relevant to a particular
destination and send them while the connection is open may not
be worthwhile.

Now I don't know any mail systems that carry that optimization
effort to an extreme that would amuse the theorists as close to
the possible limits of prediction.  Maybe there is one out
there, but I doubt it because I doubt that the resources that
would go into database-keeping and computationally-intensive
rolling predictions would be nearly justified by the performance
improvements they would yield.  So, in the real world,
implementers make guesses as to what optimizations will buy them
enough to be worth the trouble.  Those guesses are informed by
the observation that, possible specialized submission servers
and authors of spam-sending bots aside, almost no one builds an
SMTP client who is not also building SMTP servers: I believe
that it is fairly well understood in practice that writing
abusive senders that servers see as DoS attackers is, at best, a
really short-term optimization.

So, to the extent it is useful and appropriate, let's share what
we know about things that work and things that don't.  But let's
do so with the understanding that systems differ widely in how
they are organized, what optimizations are possible, and what
resources they can reasonably assume are available when they are
being operated.

     john

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sat, 20 Aug 2011 23:18:15 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: Paul Smith [mailto:[email-masked]]
> Sent: Saturday, August 20, 2011 12:41 PM
> To: [email-masked]
> Cc: Murray S. Kucherawy; [email-masked]
> Subject: Re: Mail Data termination
> 
> I'm with Hector on this. I really don't like the idea of a sender
> keeping a connection open 'just in case'.
> 
> If it keeps it open for 10 seconds 'just in case', and no new mail
> arrives so it has to close the connection, what has that achieved other
> than extra load on the receiver? Why wouldn't it then say 'well, another
> message may arrive in the next few seconds, so I'll keep it open just a
> bit longer', and so on.

I don't see how keeping a connection open creates extra load in a modern operating system.  An open file descriptor is just an entry in a kernel table.  I've never seen an idle connection cause distress to a running system.  Even for an application that creates a complete subprocess to handle that idle connection, the process would eventually get paged or swapped out in favour of more active processes.

And if you somehow have so many open connections from clients that you can't accept any more and another connection tries to arrive, select the one that's been idle the longest, close it down (releasing resources), and accept the new one.  Or just generally trim your idle timeout down.

> If the sender doesn't like closing and opening connections, then surely
> it's just as beneficial for the sender to wait 10 seconds before
> starting to send a message, then if another message to the same
> destination arrives within that 10 seconds, it can batch them up without
> adding unnecessary load to the receiver.

That's exactly how the connection cache works in some instances, depending on whether or not the sender is configured to make an immediate delivery attempt, or just wait for a queue run.

I'm really surprised there's so much sudden consternation about this feature given how many MTAs have it, and the fact that it's been around for well over a decade.  Somehow, in that time, the sky hasn't fallen.

Date: Sat, 20 Aug 2011 16:19:49 -0700
From: Bill McQuillan <[email-masked]>
To: SMTP Discussion <[email-masked]>
CC: Paul Smith <[email-masked]>, [email-masked], "Murray S. Kucherawy" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

On Sat, 2011-08-20, Paul Smith wrote:

> On 18/08/2011 22:56, [email-masked] wrote:
>> But more to the point, you're talking about very different time 
>> regimes here. Caching a connection for anything even close to five 
>> minutes is almost certainly counterproductive - SMTP connection 
>> establishment isn't *that* difficult. But holding on to a connection 
>> for just a few seconds can be quite beneficial for systems handling 
>> high volumes of mail.

> I'm with Hector on this. I really don't like the idea of a sender
> keeping a connection open 'just in case'.

> If it keeps it open for 10 seconds 'just in case', and no new mail
> arrives so it has to close the connection, what has that achieved other
> than extra load on the receiver? Why wouldn't it then say 'well, another
> message may arrive in the next few seconds, so I'll keep it open just a
> bit longer', and so on.

> If the sender doesn't like closing and opening connections, then surely
> it's just as beneficial for the sender to wait 10 seconds before 
> starting to send a message, then if another message to the same 
> destination arrives within that 10 seconds, it can batch them up without
> adding unnecessary load to the receiver.

> It just seems selfish for the sender to say 'I'm so important that I'm
> going to hog one of the receiver's channels just in case someone else
> wants to send a message to them, so the sender doesn't have to wait a
> few seconds longer'.

It seems to me that the measure is NOT the "number of clients"
but rather the number of messages handled.

Each client will have a certain number of messages to transmit to
the receiver over a time period, regardless of the number of
sessions used. When a client saves a session teardown and
reestablishment by delaying QUIT, so does the receiver. So it
would seem that a delay on the order of that time IS warranted. I
don't have a good feel for the amount of that overhead, but I
would guess that a second or two is reasonable and ten seconds is
approaching too long.

-- 
Bill McQuillan <[email-masked]>

Date: Sat, 20 Aug 2011 21:41:00 +0200
From: Paul Smith <[email-masked]>
To: [email-masked]
CC: "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

On 18/08/2011 22:56, [email-masked] wrote:
> But more to the point, you're talking about very different time 
> regimes here. Caching a connection for anything even close to five 
> minutes is almost certainly counterproductive - SMTP connection 
> establishment isn't *that* difficult. But holding on to a connection 
> for just a few seconds can be quite beneficial for systems handling 
> high volumes of mail.

I'm with Hector on this. I really don't like the idea of a sender 
keeping a connection open 'just in case'.

If it keeps it open for 10 seconds 'just in case', and no new mail 
arrives so it has to close the connection, what has that achieved other 
than extra load on the receiver? Why wouldn't it then say 'well, another 
message may arrive in the next few seconds, so I'll keep it open just a 
bit longer', and so on.

If the sender doesn't like closing and opening connections, then surely 
it's just as beneficial for the sender to wait 10 seconds before 
starting to send a message, then if another message to the same 
destination arrives within that 10 seconds, it can batch them up without 
adding unnecessary load to the receiver.

It just seems selfish for the sender to say 'I'm so important that I'm 
going to hog one of the receiver's channels just in case someone else 
wants to send a message to them, so the sender doesn't have to wait a 
few seconds longer'.

Date: Sat, 20 Aug 2011 09:58:27 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: John Levine <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

John C Klensin wrote:
> 
> 
> --On Wednesday, August 17, 2011 17:23 +0000 John Levine
> <[email-masked]> wrote:
> 
>>> How does the 'discard' action of RFC5617 (DKIM ADSP) fit into
>>> this  picture? Although RFC5321 and RFC5617 operate on
>>> different protocol  layers, in real life the two are often
>>> intertwined by way of MTA milter  callouts during SMTP DATA
>>> phase.
>> For people who choose to implement ADSP (and you know how I
>> feel about that), I think it's fair to say that it operates at
>> a layer above SMTP.
> 
> Please also note that 5321 says, very explicitly, that a server
> may make all sorts of exceptions to a close reading of the rules
> to protect itself from attacks.  I'm not going to quote the
> section number again -- the authors of several entries in this
> long thread need to go back and read that spec again.  If one
> views DKIM with ADSP as a necessary attack-prevention mechanism,
> provisions of 5321 that seem contradictory are irrelevant.

I'm not sure which way to read your comment, but I don't see a 
contradiction.

The essential and common, general, NEW design consideration is the 
continued evolution of advanced mail integration by performing payload 
analysis in the DATA state in order to do three things;

    - Accelerate what would be the same result if it was accepted 
always and
      processed after the session was added,

    - Eliminate/reduce the problematic Accept/Bounce Exploits, and

    - the "Throw Away" ideas does not conform via well with the 1986
      US ECPA provisions for "User Expectations" with the long
      tradition of online and also SMTP mandating a rejection 
notification
      adheres too.

 From a product liability standpoint, to avoid censorship claims, the 
direction to provide the rejection notice during the smtp session 
solve two major issues - keep with User Expectation notification 
guidelines and reduce the serious Accept/Bounce Mail attacks.

I have always held the position that this debate and many others, 
centered around two basic modes of operations people use and SMTP 
needs to work under - older original styles of always accepting the 
payload and newer styles of tighter integration with the backend and 
dynamic processing of the payload. With operators mindset with older 
styles, a ADSP would be a NON-SMTP concept.   But the PAYLOAD is part 
of SMTP so i don't agree with the concept.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sat, 20 Aug 2011 09:38:37 -0400
From: John C Klensin <[email-masked]>
To: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Saturday, August 20, 2011 13:36 +0200 Alessandro Vesely
<[email-masked]> wrote:

>...
> One possibility is to sort queued messages by MX, but this
> implies a delay and is error-prone.

Sigh.

Maybe I'm confused about what you are suggesting, but some of
the earliest SMTP senders implemented used a logic of:

 -- when a message arrives (from an MUA or relay
	function), try to deliver it
 -- if it cannot be delivered immediately, queue it for
	retry but sort or thread the queues so that, when a
	redelivery attempt is made and succeeds, all messages
	that are queued for that host are attempted, either in
	the same SMTP session or very soon thereafter.

Variations on that theme are, IIR, discussed in RFC 1123 -- it
is not exactly a new idea. 

In one form or another, that approach has been used by most
non-trivial SMTP clients since.  There are a bunch of
optimizations and one wants to be really careful about any sort
or threading that gets tangled up with MX preferences and
thereby loses critical information, but, more than two decades
later, those methods are rather well-understood. 

The obvious extension to that model is to look around after the
queue for a particular target host has been drained (or a bit
earlier if one has been careful about threading (and lucky in
one's choice of operating systems)) and see if new messages have
arrived for that host while that portion of the queue has been
run.   That optimization is common too, and holding a connection
open for a few additional seconds while the check is being made
to see if something else arrives is too.

So what are we discussing here?

   john

Date: Sat, 20 Aug 2011 09:28:04 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Alessandro Vesely wrote:
> On 20.08.2011 08:38, Hector Santos wrote:
>> Well, It (Connection Sharing) is still a poor one-side only benefit logic and
>> probably this fact was realized long along and simply not employed in any
>> significant way and if it is, probably isolated to specific host and for the
>> rest, if employed, has a very short (perhaps 5 secs) hold time.
> 
> SMTP offers the option to shove several transactions into a single session,
> thereby re-using costly TCP and TLS handshaking for multiple messages.  How
> can a server take advantage of that option?
> 
> One possibility is to sort queued messages by MX, but this implies a delay
> and is error-prone.  Another possibility is to sort recipients by
> domain-part, which can be done by the submitter.
> 
> For example, a mass mailer script that produces personalized messages for
> each recipient can simply hand them to the MSA in the right order.  The
> number of seconds that a connection remains dormant needs just be enough for
> the next message to be generated and submitted.

Agree, but only from the design perspective of one side to make it 
work at the expense of presuming there is a unlimited number servers 
and there are no other clients doing the same thing and the server 
capacity is rarely reached so that server availability is not impacted 
and the server cost to maintain steady state performance is minimal.

Its a selfish idea from the standpoint that it considers its own 
benefits by ignoring the impact at the receiver. But I do think its a 
useful method if its prearrange with the receiver in the same way we 
white list specific senders (i.e, Allow Relay IP tables, smart 
hosting, etc).

This can be easily modeled and simulated using this formula:

    C = R /(S+H)

where

    R = Total number of available receivers
    S = Average Session Time
    H = Average Holding Time (CS cache wait time), and
    C = Arrival rate of new clients

Example #1 - No CS consideration H=0

    R = 100 receivers
    S = 5 secs
    H = 0 secs

Then for the maximum efficient, high production steady state condition:

    C = 100/(5+0) or 20 CPS (clients per sec)

This means that all 100 receivers are busy 100% at all times. It is 
steady state when C does not increase.

Example #2: CS considered, H=5

Now add the artificial CS Holding Time, H=5, assuming the maximum hold 
time allowed by SMTP is 5 mins idle time, and you have:

    C = 100/(5+5*60) or 100/305 or 0.3 CPS or 1 client every 3 seconds!

So without changing total receivers or impacting the service time, the 
client arrival rate decreases.

If we want to maintain a 20 CPS, something has to change, like R or S 
and that is my basic point that CS only benefits the client and can 
potentially have a high impact on the receiver by forcing new loading 
requirements and service availability design issues.

Of course, there are all kinds of other considerations but the above 
shows where your bottlenecks can occur.

I think all agrees that H has to be very small so that the receiver is 
not impacted in any significant way.  The SMTP "Pressure Relief" (i.e. 
221 and Retry strategies) recommendations will help move it or keep it 
stable.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sat, 20 Aug 2011 09:10:47 -0400
From: John C Klensin <[email-masked]>
To: John Levine <[email-masked]>, [email-masked]
cc: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Wednesday, August 17, 2011 17:23 +0000 John Levine
<[email-masked]> wrote:

> 
>> How does the 'discard' action of RFC5617 (DKIM ADSP) fit into
>> this  picture? Although RFC5321 and RFC5617 operate on
>> different protocol  layers, in real life the two are often
>> intertwined by way of MTA milter  callouts during SMTP DATA
>> phase.
> 
> For people who choose to implement ADSP (and you know how I
> feel about that), I think it's fair to say that it operates at
> a layer above SMTP.

Please also note that 5321 says, very explicitly, that a server
may make all sorts of exceptions to a close reading of the rules
to protect itself from attacks.  I'm not going to quote the
section number again -- the authors of several entries in this
long thread need to go back and read that spec again.  If one
views DKIM with ADSP as a necessary attack-prevention mechanism,
provisions of 5321 that seem contradictory are irrelevant.

   john

Date: Sat, 20 Aug 2011 13:36:17 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 20.08.2011 08:38, Hector Santos wrote:
> Well, It (Connection Sharing) is still a poor one-side only benefit logic and
> probably this fact was realized long along and simply not employed in any
> significant way and if it is, probably isolated to specific host and for the
> rest, if employed, has a very short (perhaps 5 secs) hold time.

SMTP offers the option to shove several transactions into a single session,
thereby re-using costly TCP and TLS handshaking for multiple messages.  How
can a server take advantage of that option?

One possibility is to sort queued messages by MX, but this implies a delay
and is error-prone.  Another possibility is to sort recipients by
domain-part, which can be done by the submitter.

For example, a mass mailer script that produces personalized messages for
each recipient can simply hand them to the MSA in the right order.  The
number of seconds that a connection remains dormant needs just be enough for
the next message to be generated and submitted.

jm2c

Date: Sat, 20 Aug 2011 13:03:20 +0200
From: Alessandro Vesely <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: Security problems with SRV records
Sender: [email-masked]

On 19.08.2011 14:56, Arnt Gulbrandsen wrote:
> 
> On 08/19/2011 02:05 PM, Alessandro Vesely wrote:
>> As well as they automate client setup, SRV records also automate cracking.
> 
> How is that?

Assume I have a bunch of email addresses, some bots, and some spam to send.
I'd run something something like this pseudo-code:

foreach ($email-addr in BUNCH)
{
  $srv = lookup_SRV(domain_part($email-addr));
  if ($srv)
  {
    // assume $email-addr is also the login name (or, possibly, BUNCH was
    // harvested by gathering Authentication-Results for the "auth" method,
    // or equivalent auth info stored in MSAs' Received header fields.)
    $passwd = crack_password($email-addr, $srv, BOTNET);

    // otherwise, try the local part
    if (!$passwd)
    {
      $email-addr = local_part($email-addr);
      $passwd = crack_password($email-addr, $srv, BOTNET);
    }

    // send authenticated mail, if cracked
    if ($passwd)
    {
      $bot = choose_a_client(BOTNET);
      run_exploit($email-addr, $passwd, $srv, $bot, SPAM);
    }
  }
}

That only works if SRV records are widespread.  In most cases it is
straightforward to work out an MSA for a given domain, but it has to be done
manually, using some knowledge and insight.  A fully automated tool can be
run by anyone.

Admittedly, the attacks I've seen thus far (MSA and POP3) are not very
clever, but they seem to be getting better.

> Keep in mind that the net is small these days. The bruteforce kiddies sweep
> the entire IPv4 internet, and do it quickly. I set up a new public box the
> other day, it was swept during its first working day

Yes, that's common.  However, until they are forced to try a few role
accounts, e.g. "Administrator", it is quite easy to avoid being cracked.

Hiding login names is a possible alternative.  For example, Section 8.8 of
4409bis says:

   The MSA MAY rewrite local parts and/or domains in the SMTP envelope,
   and optionally in address fields of the header, according to local
   policy.  For example, a site may prefer to rewrite 'JRU' as
   'J.Random.User' in order to hide login names, and/or to rewrite
   'squeaky.sales.example.net' as 'zyx.example.net' to hide machine
   names and make it easier to move users.

Are such tactics deployed often?

And how about IPv6?

> I've heard stories of "rooted in x minutes" for scarily small values of x. 

Yes, by rooting the server it is possible to download a user list that also
includes (possibly encrypted) passwords.  Let's assume that we have some
means to avoid being rooted, at least in theory!

> I venture to suggest that at the moment, sweeping the IPv4 net looking for
> open TCP ports is easier than sweeping the DNS looking for SRV records.

It is ways more difficult to impersonate real users when you know neither
their userids nor their passwords.  Having a server's IP and some login names
is more helpful for crackers[1], and it is the kind of information that we
are willing to make publicly available.  Are there theoretical
countermeasures that avoid cracking but still allow traditional use of
"human" passwords?

-- 

[1] http://tomicki.net/chimera.php
    remote password cracker for a variety of protocols;
    tool's arguments: server, service, login name, kind of passwords.

Date: Sat, 20 Aug 2011 11:29:58 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 08/20/2011 08:38 AM, Hector Santos wrote:
> Well, It (Connection Sharing) is still a poor one-side only benefit
> logic and probably this fact was realized long along and simply not
> employed in any significant way and if it is, probably isolated to
> specific host and for the rest, if employed, has a very short (perhaps 5
> secs) hold time. Holding up channels *anonymous and unsolicited* for an
> extended time would not be very desirable.

One-sided?

TLS setup is equally expensive for client and server. Assessing the SMTP 
client as mail/spam source is more expensive for the server than for the 
client.

Arnt

Date: Sat, 20 Aug 2011 02:38:58 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
To: [email-masked]
Sender: [email-masked]

Murray S. Kucherawy wrote:

>> I have no idea why your servers are seeing longer averages, but I can assure
>> you it isn't our client that's doing it. When there are no more messages we
>> only keep connections open for a few seconds and this is not
>> configurable.
> 
> I also don't know why this would be a change that became visible only in 
> the last year given the fact that at least sendmail has had that 
> feature for something like fifteen years now.

I had a mental relapse in reading the change in our log profiles this 
year from secs to usecs display.  The ave is essentially the same.

Well, It (Connection Sharing) is still a poor one-side only benefit 
logic and probably this fact was realized long along and simply not 
employed in any significant way and if it is, probably isolated to 
specific host and for the rest, if employed, has a very short (perhaps 
5 secs) hold time. Holding  up channels *anonymous and unsolicited* 
for an extended time would not be very desirable.

-- 
HLS

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Fri, 19 Aug 2011 22:52:28 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Thursday, August 18, 2011 6:46 PM
> To: Hector Santos
> Cc: [email-masked]
> Subject: Re: Mail Data termination
> 
> I have no idea why your servers are seeing longer averages, but I can assure
> you it isn't our client that's doing it. When there are no more messages we
> only keep connections open for a few seconds and this is not
> configurable.

I also don't know why this would be a change that became visible only in the last year given the fact that at least sendmail has had that feature for something like fifteen years now.

-MSK

Date: Fri, 19 Aug 2011 22:56:31 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

hmmmmmm, you're right. The change to add .microsecs profiling made me 
dense!  My August stats is ave 7.4 secs, not mins. Man, I humbly 
apologize to all and especially to FaceBook for this nonsense noise. 
Thanks George for picking up on this my mental relapse and error.

George Schlossnagle wrote:
> Am I dense or is that them disconnecting after 5 seconds?
>>
>> 20:24:26.252 C: DATA
>> 20:24:26.460 S: 250 Message received for delivery. (bytes: 8223)
>> 20:24:26.460 ** Changing Post Data Idle time to: 1 mins
>> 20:24:31.545 C: QUIT
>>
>> The change to 1 minute was tested manually with a telnet connection
>> and its kicks out the session in 1 minute after message transaction.
>>
>> Now that represents intentional abuse of receivers, definitely
>> anti-social!
>>
>>
>> -- 
>> Sincerely
>>
>> Hector Santos
>> http://www.santronics.com
>>
>>
>>
> 
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: George Schlossnagle <[email-masked]>
To: Hector Santos <[email-masked]>, "[email-masked]" <[email-masked]>
Date: Fri, 19 Aug 2011 21:40:50 -0400
Subject: Re: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

Am I dense or is that them disconnecting after 5 seconds?

On 8/19/11 9:14 PM, "Hector Santos" <[email-masked]> wrote:

>
>Hector Santos wrote:
>
>> Well, I can't stand for this.  A local policy option to lower the
>>server 
>> idle client timeout to 1 minute (default) after the 1st transaction has
>> been finished will be provided.  I'm calling it Final-Idle-Time.
>
>FWIW:
>
>I added this this exploratory option and tested it against FaceBook
>sending emails to me and they are somehow keeping the line active:
>
>20:24:26.252 C: DATA
>20:24:26.460 S: 250 Message received for delivery. (bytes: 8223)
>20:24:26.460 ** Changing Post Data Idle time to: 1 mins
>20:24:31.545 C: QUIT
>
>The change to 1 minute was tested manually with a telnet connection
>and its kicks out the session in 1 minute after message transaction.
>
>Now that represents intentional abuse of receivers, definitely
>anti-social!
>
>
>-- 
>Sincerely
>
>Hector Santos
>http://www.santronics.com
>
>
>

Date: Fri, 19 Aug 2011 21:14:36 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Hector Santos wrote:

> Well, I can't stand for this.  A local policy option to lower the server 
> idle client timeout to 1 minute (default) after the 1st transaction has 
> been finished will be provided.  I'm calling it Final-Idle-Time.  

FWIW:

I added this this exploratory option and tested it against FaceBook 
sending emails to me and they are somehow keeping the line active:

20:24:26.252 C: DATA
20:24:26.460 S: 250 Message received for delivery. (bytes: 8223)
20:24:26.460 ** Changing Post Data Idle time to: 1 mins
20:24:31.545 C: QUIT

The change to 1 minute was tested manually with a telnet connection 
and its kicks out the session in 1 minute after message transaction.

Now that represents intentional abuse of receivers, definitely 
anti-social!

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Fri, 19 Aug 2011 09:59:13 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Arnt Gulbrandsen <[email-masked]>
CC: Alessandro Vesely <[email-masked]>, SMTP Interest Group <[email-masked]>, Martin Rex <[email-masked]>
Subject: Re: Security problems with SRV records
Sender: [email-masked]

+1. I see this as two different problems, and one of the negatives of 
SRV records is that it opens sites up to discovery issues.  Can't do 
much about that but make sure the site is fortified against attacks.

We use a default 3 attempt login limit across the board and a failure 
will block the IP across all the internet hosting servers. I can 
attest this is extremely well liked feature and it has help people go 
to sleep better.  I don't think this necessary protection has anything 
to do with how the attacker found the site - SRV or otherwise.

Arnt Gulbrandsen wrote:
> 
> On 08/19/2011 02:05 PM, Alessandro Vesely wrote:
>> As well as they automate client setup, SRV records also automate 
>> cracking.
> 
> How is that?
> 
> Keep in mind that the net is small these days. The bruteforce kiddies 
> sweep the entire IPv4 internet, and do it quickly. I set up a new public 
> box the other day, it was swept during its first working day, and I've 
> heard stories of "rooted in x minutes" for scarily small values of x.
> 
> I venture to suggest that at the moment, sweeping the IPv4 net looking 
> for open TCP ports is easier than sweeping the DNS looking for SRV records.
> 
> Arnt
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Fri, 19 Aug 2011 09:32:17 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Sorry Ned, I was referring to the "cheaper" part claim.  Your 
recognition for a shorter idle residence time, cache time, was 
previously noted.

I have yet to compare notes with customers.  Among the top senders is 
FaceBook's mailer with a 5 mins delay before sending QUIT.  It looks 
like Hoffman's imc.org mailer setup has a 4 mins delay and there are 
others from unknown senders.  So its apparent now where these odd 
unexplained delays are now coming from - Connection Sharing clients.

Well, I can't stand for this.  A local policy option to lower the 
server idle client timeout to 1 minute (default) after the 1st 
transaction has been finished will be provided.  I'm calling it 
Final-Idle-Time.  Probably also look at some tuning logic to monitor 
Sender vs Final-Hold-Time to see if that can be used to kick out idle 
clients when new connections come in.   This is part of the problem 
that a CS logic does - increases cost on receivers to avoid the 
potential problems caused by them, and that includes contributing to 
any DoS attacks.

[email-masked] wrote:
> 
> 
>> [email-masked] wrote:
> 
>> >> I don't think the practice of connection caching is particularly 
>> selfish when
>> >> compared to the cost of having the connection torn down and then 
>> re-established
>> >> with some frequency, when it's generally much cheaper for both the 
>> sender and
>> >> the receiver to just leave it open.
>> >
>> > Exactly. Although it is necessarily up to the client to decide, the 
>> server also
>> > benefits as long as the connection isn't cached for very long.
> 
>> I can see this when two systems have prearranged dedicated channesl
>> for high load exchanges, but in general, I disagree that it is
>> "cheaper" and definitely not for the receiver. I don't see how that
>> can be justified. Average SMTP session times last year of 10-15
>> seconds are now up to 4-7 minutes this year  Thats better?
> 
> Please reread my message. Nowhere did I say that clients should cache
> connections up to the 5 minute timeout - in fact I said the exact opposite:
> 
>  Caching a connection for anything even close to five minutes is almost
>  certainly counterproductive - SMTP connection establishment isn't *that*
>  difficult. But holding on to a connection for just a few seconds can be 
> quite
>  beneficial for systems handling high volumes of mail.
> 
> I have no idea why your servers are seeing longer averages, but I can 
> assure
> you it isn't our client that's doing it. When there are no more messages we
> only keep connections open for a few seconds and this is not configurable.
> 
>                 Ned
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Fri, 19 Aug 2011 14:56:43 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: Alessandro Vesely <[email-masked]>
Cc: SMTP Interest Group <[email-masked]>, Martin Rex <[email-masked]>
Subject: Re: Security problems with SRV records
Sender: [email-masked]

On 08/19/2011 02:05 PM, Alessandro Vesely wrote:
> As well as they automate client setup, SRV records also automate cracking.

How is that?

Keep in mind that the net is small these days. The bruteforce kiddies 
sweep the entire IPv4 internet, and do it quickly. I set up a new public 
box the other day, it was swept during its first working day, and I've 
heard stories of "rooted in x minutes" for scarily small values of x.

I venture to suggest that at the moment, sweeping the IPv4 net looking 
for open TCP ports is easier than sweeping the DNS looking for SRV records.

Arnt

Date: Fri, 19 Aug 2011 14:05:57 +0200
From: Alessandro Vesely <[email-masked]>
To: SMTP Interest Group <[email-masked]>
CC: Martin Rex <[email-masked]>
Subject: Security problems with SRV records
Sender: [email-masked]

RFC 6186 is being mentioned in discussions about 4409bis, but I don't think
this belongs to yam@ nor ietf@ lists.  So I try here.

On 16.08.2011 23:01, Martin Rex wrote on [email-masked]:

> Security-wise, the SRV record suggested by rfc6186 seems to create
> additional security problems, so I would also not like to see it being
> "adopted" as is.  :-/

As well as they automate client setup, SRV records also automate cracking.

IMHO, this issue should be fixed in SMTP AUTH.  The Security Considerations
in RFC 4954 say that

   Servers MAY implement a policy whereby the connection is dropped
   after a number of failed authentication attempts.  If they do so,
   they SHOULD NOT drop the connection until at least 3 attempts to
   authenticate have failed.

I don't think that suffices.  What might the replacement text for an errata
suggest instead?

Users of fail2ban probably also drop any further connection attempt from the
relevant IP address(es) for at least some minutes.  This still doesn't
prevent distributed attacks.

Forcing passwords changes annoys users, if it is frequent enough.

The most reasonable thing I can think of is the capability of "blocking" an
account when a number of failures becomes comparable with the entropy of the
current password, wherever they came from.  I mean, if a clever user chooses
a password with 24 bits of entropy, her account should be blocked sometimes
after, say, 16384 failed attempts --2^24/1024, about a thousandth of the
search space, see http://en.wikipedia.org/wiki/Password_strength

There are two difficulties with this approach:

1) "Blocking" implies being able to reach users in some other way.

2) Servers tend to be sparing in their logging failed login attempts.  They
   obviously do so for security reasons...  How do we tackle this catch-22?

TIA for any reply

Subject: Re: Mail Data termination
Date: Fri, 19 Aug 2011 04:21:04 +0100
To: [email-masked]
From: Sabahattin Gucukoglu <[email-masked]>
Reply-To: Sabahattin Gucukoglu <[email-masked]>
Sender: [email-masked]

On 18 Aug 2011, at 18:22, Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: Tony Finch [mailto:[email-masked]] On Behalf Of Tony Finch
>> Sent: Thursday, August 18, 2011 4:14 AM
>> To: Murray S. Kucherawy
>> Cc: [email-masked]
>> Subject: RE: Mail Data termination
>> 
>> Murray S. Kucherawy <[email-masked]> wrote:
>>> One of the more ubiquitous MTAs out there has a feature called
>>> "connection caching" where it will keep some fixed number of outbound
>>> SMTP connections around, cycled using an LRU method, in case a new
>>> message arrives that is routed to a recent destination to avoid the cost
>>> of making a new connection and doing the initial SMTP negotiation.
>> 
>> I believe Postfix and Exchange can do this.
> 
> And I was referring to Sendmail.

Sendmail is certainly a very interesting animal.  Its sorting mechanisms are fairly primitive, which means it has no way to identify when it's safe for any given client (whether running part of the spool or invoked just now for delivery) to close all sessions it has cached.  So it waits until all jobs are done before doing so.  If the administrator wants to get maximum SMTP efficiency by having as many messages as possible be sorted by as few clients as possible at the expense of parallel deliveries, he might choose to run a very small number of delivery clients to run queues, and then the caching could last quite a while during which unrelated messages for different hosts drain.  (If you've ever tried to run a mailing list with Sendmail you've probably seen the script that breaks lists up by hostname so this process can be somewhat optimised.)

> The more SMTP clients there are that do this, the more likely it is that a server demanding QUIT before scheduling a delivery attempt will lose mail.

I don't think it's a good idea.

Just checked, Sendmail is sending RSET to check if the connection is still running.  Postfix does that too.  So I hope RSET is not similar to QUIT in initiating the mail delivery following each transaction.

Here's the details from Postfix:
http://www.postfix.org/CONNECTION_CACHE_README.html

I can certainly understand the need to finish quickly, but I believe the balance is in favour of a small cache time.  If nothing's happening, it's no worse than having to need a connection that's just been torn down and must be renegotiated.

Cheers,
Sabahattin

From: [email-masked]
Cc: "[email-masked]" <[email-masked]>
Date: Thu, 18 Aug 2011 18:46:22 -0700 (PDT)
Subject: Re: Mail Data termination
To: Hector Santos <[email-masked]>
Sender: [email-masked]

> [email-masked] wrote:

> >> I don't think the practice of connection caching is particularly selfish when
> >> compared to the cost of having the connection torn down and then re-established
> >> with some frequency, when it's generally much cheaper for both the sender and
> >> the receiver to just leave it open.
> >
> > Exactly. Although it is necessarily up to the client to decide, the server also
> > benefits as long as the connection isn't cached for very long.

> I can see this when two systems have prearranged dedicated channesl
> for high load exchanges, but in general, I disagree that it is
> "cheaper" and definitely not for the receiver. I don't see how that
> can be justified. Average SMTP session times last year of 10-15
> seconds are now up to 4-7 minutes this year  Thats better?

Please reread my message. Nowhere did I say that clients should cache
connections up to the 5 minute timeout - in fact I said the exact opposite:

  Caching a connection for anything even close to five minutes is almost
  certainly counterproductive - SMTP connection establishment isn't *that*
  difficult. But holding on to a connection for just a few seconds can be quite
  beneficial for systems handling high volumes of mail.

I have no idea why your servers are seeing longer averages, but I can assure
you it isn't our client that's doing it. When there are no more messages we
only keep connections open for a few seconds and this is not configurable.

				Ned

Date: Thu, 18 Aug 2011 21:16:02 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Connection Caching and Little's Law 
Sender: [email-masked]

Murray S. Kucherawy wrote:

> I don't think the practice of connection caching is particularly 
> selfish when compared to the cost of having the connection torn down and 
> then re-established with some frequency, when it's generally much cheaper 
> for both the sender and the receiver to just leave it open.

Only if there is efficient work being done.  Look, maybe it was my 
fault to assume the obviousness of this classic Operational Research 
and Querying Design problem is commonly understood.

One just need to use Little's Law [1] to see how this is a very 
selfish queuing concept that benefits only one part of the system and 
increases demands and cost on other parts.  This is particular true 
you reach a capacity and can no longer remain in steady state. To 
return it back to steady state, adjustments are needed.  The flaw in 
CS is the erroneous presumption that there is unlimited resources on 
the receiver side (service workers) so that the waste in idle service 
time is factored out as arrival rates increased.  As in a bank 
scenario, efficiency is lost if after a teller station finishes 
servicing a customer, he decides to hangs around the station doing 
nothing.  The teller's time is wasted. Less workers available, Queued 
Customers wait longer, Lines get longer. Little's Law will show you this.

[1] http://en.wikipedia.org/wiki/Little's_law

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 18 Aug 2011 19:35:53 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy wrote:

> The five-minute timeout is a SHOULD.  If you're resource constrained, I 
> would argue that issuing a 221 to the most idle open connection and 
> closing it down so the resources can be re-used is just fine.

It has nothing to do with any system being constrained but one that 
can cause them to have new constraints, a higher TCO and need to scale 
up or out in order to satisfy what is guarantee to become new loading 
profile.

>> This should be done using an SMTP extension where a server can expose
>> server timeouts perhaps and willingness to allow a client to sit idle
>> redundantly and repeatedly.
> 
> Or it can simply impose an idle time that's more strict than what 
> 4.5.3.2.7 of RFC5321 says, understanding that one needs to be pretty 
> careful about doing so.

Whats going to happen is that implementors discovery might begin will 
change the timeout after a transaction is completed to 1 minute and 
zap CS clients out! I already have this Penciled in after seeing this 
was causing the increase SMTP session times from what was seconds to 
multiple minutes - TERRIBLE!

We spent a lot time and energy to optimized systems and the times out 
were designed to accommodate client/server handshaking processing and 
networking delay needs.  It was not designed to allow a client to hold 
hostage communication channels chewing up CPU time and system 
availability needs.

No matter what limit is set on total receivers, if the modus operandi 
is the CS design, you will reach limit exhaustion far quicker. This 
can cause a higher potential for initial outbound failures on clients 
sending out mail, delays transactions, etc, etc.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 18 Aug 2011 19:01:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

[email-masked] wrote:

>> I don't think the practice of connection caching is particularly selfish when
>> compared to the cost of having the connection torn down and then re-established
>> with some frequency, when it's generally much cheaper for both the sender and
>> the receiver to just leave it open.
> 
> Exactly. Although it is necessarily up to the client to decide, the server also
> benefits as long as the connection isn't cached for very long.

I can see this when two systems have prearranged dedicated channesl 
for high load exchanges, but in general, I disagree that it is 
"cheaper" and definitely not for the receiver. I don't see how that 
can be justified. Average SMTP session times last year of 10-15 
seconds are now up to 4-7 minutes this year  Thats better?  I don't 
think so. If everyone did this, channel exhaustions will be realized 
much quicker.  Its increase cost if one needs to scale up or out to 
satisfy their service loads and availability needs.

We need to consider changing the timeout after DATA response, waiting 
for QUIT or next transaction to at least 1 minute. The basic mail job 
was done, no need to wait 5 minutes at this point and this is 
especially the case when the majority of sessions are only single 
transactions.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 18 Aug 2011 18:42:20 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
To: [email-masked]
Sender: [email-masked]

I'm sorry if I am passionate about this.  But this explains what I 
have been seeing this year.

This past year, I have been seeing an unexplained increase in the 
average time of SMTP sessions to around 4.5, 5 or more and we spend a 
lot of time adding timing profiles to see where the SMTP processing 
was chewing these greater SMTP times.  I thought it as just stupid 
clients that didn't have robust logic to wait on signals before they 
issued QUIT.  No, it seems to be these CS clients and I don't like it. 
This explains why FACEBOOK waits exactly 5 minutes for each 
transaction it sends. I have not seen it a Facebook transaction that 
has two or more transactions but I guess that depends on the frequency 
of comments.   This is terrible!

     http://www.winserver.com/public/spamstats.wct

August 7.7 mins
July   4.7 mins
June   4.0 mins
May    8.3 mins

etc.  going back 1 year to 2010

August 14 secs
July   12 secs

Geez, enough said - Sendmail CS logic is a problem.  It adds all kinds 
of internet and networking pressures, receiver server availability, 
scalability problems - a problem across the board, and the greater 
this practiced is implementation, the worst the problem is going to 
get, perhaps to the extend the SERVER will take action across these CS 
clients.

There are robust ways to solve the "Same Destination Server" 
redundancy issues - the CS way is a one-sided selfish method that 
creates new problems for receivers.

Client/Server idle times were meant for clients to wait on server 
responses, not to sit there idle doing nothing within its next job to 
come in!

Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
>> Sent: Thursday, August 18, 2011 12:32 PM
>> To: [email-masked]
>> Subject: Re: Mail Data termination
>>
>> First, a server may have Receiver Loading Limits and a client with a
>> tenacity to take up more CPU time than it needs can get it flagged. To
>> use CS for the purpose to reduce its redundancy without considering
>> the load it places on receivers is a very selfish engineering
>> consideration. Chewing up a channel unnecessarily for 5 minutes
>> disallowing other clients connection access increases system available
>> and scalability problems.
>>
>> Second, it uses a dangerous presumption that a server is FIXED on
>> using 5 minutes for idle times. 5 minutes is a SHOULD, not a MUST and
>> for a client to DEPEND on a SERVER using 5 minutes is short-sighted
>> engineering.
> 
> The five-minute timeout is a SHOULD.  If you're resource constrained, I would argue that issuing a 221 to the most idle open connection and closing it down so the resources can be re-used is just fine.
> 
> I don't think the practice of connection caching is particularly selfish when compared to the cost of having the connection torn down and then re-established with some frequency, when it's generally much cheaper for both the sender and the receiver to just leave it open.
> 
>> This should be done using an SMTP extension where a server can expose
>> server timeouts perhaps and willingness to allow a client to sit idle
>> redundantly and repeatedly.
> 
> Or it can simply impose an idle time that's more strict than what 4.5.3.2.7 of RFC5321 says, understanding that one needs to be pretty careful about doing so.
> 
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: [email-masked]
Cc: "[email-masked]" <[email-masked]>
Date: Thu, 18 Aug 2011 13:56:31 -0700 (PDT)
Subject: RE: Mail Data termination
To: "Murray S. Kucherawy" <[email-masked]>
Sender: [email-masked]

> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> > Sent: Thursday, August 18, 2011 12:32 PM
> > To: [email-masked]
> > Subject: Re: Mail Data termination
> >
> > First, a server may have Receiver Loading Limits and a client with a
> > tenacity to take up more CPU time than it needs can get it flagged. To
> > use CS for the purpose to reduce its redundancy without considering
> > the load it places on receivers is a very selfish engineering
> > consideration. Chewing up a channel unnecessarily for 5 minutes
> > disallowing other clients connection access increases system available
> > and scalability problems.
> >
> > Second, it uses a dangerous presumption that a server is FIXED on
> > using 5 minutes for idle times. 5 minutes is a SHOULD, not a MUST and
> > for a client to DEPEND on a SERVER using 5 minutes is short-sighted
> > engineering.

> The five-minute timeout is a SHOULD.  If you're resource constrained, I would
> argue that issuing a 221 to the most idle open connection and closing it down
> so the resources can be re-used is just fine.

Agreed.

But more to the point, you're talking about very different time regimes here.
Caching a connection for anything even close to five minutes is almost
certainly counterproductive - SMTP connection establishment isn't *that*
difficult. But holding on to a connection for just a few seconds can be quite
beneficial for systems handling high volumes of mail.

> I don't think the practice of connection caching is particularly selfish when
> compared to the cost of having the connection torn down and then re-established
> with some frequency, when it's generally much cheaper for both the sender and
> the receiver to just leave it open.

Exactly. Although it is necessarily up to the client to decide, the server also
benefits as long as the connection isn't cached for very long.

				Ned

Date: Thu, 18 Aug 2011 16:26:46 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

To add the problems a client using a CS logic can cause, it won't work 
very well against the growing # of Greylist Ready SMTP receivers.

If an initial transaction is rejected is due to greylist, sitting 
there idle waiting for another same destination message to come in to 
start a new transaction with the receiver isn't going to change the 
initial SENDER rejection status.

Allowing for this, opens the door for GreyList Security Loopholes we 
have documented proof (and discussed here) to occur.

The solution was to remember the transaction status so that bad guys 
don't use 2nd transaction to get around the initial greylist rejection.

Hector Santos wrote:
> Murray S. Kucherawy wrote:
> 
>>>> One of the more ubiquitous MTAs out there has a feature called
>>>> "connection caching" where it will keep some fixed number of outbound
>>>> SMTP connections around, cycled using an LRU method, in case a new
>>>> message arrives that is routed to a recent destination to avoid the 
>>>> cost
>>>> of making a new connection and doing the initial SMTP negotiation.
>>> I believe Postfix and Exchange can do this.
>>
>> And I was referring to Sendmail.
>>
>> The more SMTP clients there are that do this, the more likely it is 
>> that a server demanding QUIT before scheduling a delivery attempt will 
>> lose 
> mail.
> 
> Fortunately, that is *never* the case hence there is no problem until 
> you try to correct a problem.  It helps with robustness by reducing 
> problematic transactions, especially the exploited ones.
> 
> But you did highlight something I need to look at, not because of 5321 
> required QUIT which by the way is only going to be addressed with YASO 
> (Yet Another Server Option),  but rather analyze what the major 
> dangerous presumptions and selfish engineering considerations a client 
> operating under a Connection Sharing (CS) mode imposes on receivers. It 
> might explains some situations seen.
> 
> First, a server may have Receiver Loading Limits and a client with a 
> tenacity to take up more CPU time than it needs can get it flagged. To 
> use CS for the purpose to reduce its redundancy without considering the 
> load it places on receivers is a very selfish engineering consideration. 
> Chewing up a channel unnecessarily for 5 minutes disallowing other 
> clients connection access increases system available and scalability 
> problems.
> 
> Second, it uses a dangerous presumption that a server is FIXED on using 
> 5 minutes for idle times. 5 minutes is a SHOULD, not a MUST and for a 
> client to DEPEND on a SERVER using 5 minutes is short-sighted engineering.
> 
> This should be done using an SMTP extension where a server can expose 
> server timeouts perhaps and willingness to allow a client to sit idle 
> redundantly and repeatedly.
> 
> Whats surreal about this, is the constant conflict of people ramming 
> specs in people's spec yet have this strange tolerance for allowing and 
> accommodating failure.  Very odd.   That doesn't work very well, IMEO, 
> and if you wish to call that a "singular interpretation" thats fine by me.
> 
> As this particular case shown, the ignorance of the failure would of 
> delivered bad messages and also delivered duplicated bad messages.
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Thu, 18 Aug 2011 13:09:19 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Thursday, August 18, 2011 12:32 PM
> To: [email-masked]
> Subject: Re: Mail Data termination
> 
> First, a server may have Receiver Loading Limits and a client with a
> tenacity to take up more CPU time than it needs can get it flagged. To
> use CS for the purpose to reduce its redundancy without considering
> the load it places on receivers is a very selfish engineering
> consideration. Chewing up a channel unnecessarily for 5 minutes
> disallowing other clients connection access increases system available
> and scalability problems.
> 
> Second, it uses a dangerous presumption that a server is FIXED on
> using 5 minutes for idle times. 5 minutes is a SHOULD, not a MUST and
> for a client to DEPEND on a SERVER using 5 minutes is short-sighted
> engineering.

The five-minute timeout is a SHOULD.  If you're resource constrained, I would argue that issuing a 221 to the most idle open connection and closing it down so the resources can be re-used is just fine.

I don't think the practice of connection caching is particularly selfish when compared to the cost of having the connection torn down and then re-established with some frequency, when it's generally much cheaper for both the sender and the receiver to just leave it open.

> This should be done using an SMTP extension where a server can expose
> server timeouts perhaps and willingness to allow a client to sit idle
> redundantly and repeatedly.

Or it can simply impose an idle time that's more strict than what 4.5.3.2.7 of RFC5321 says, understanding that one needs to be pretty careful about doing so.

Date: Thu, 18 Aug 2011 16:03:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
CC: Tony Finch <[email-masked]>, "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

[email-masked] wrote:
> 
>> Murray S. Kucherawy <[email-masked]> wrote:
>>> One of the more ubiquitous MTAs out there has a feature called
>>> "connection caching" where it will keep some fixed number of outbound
>>> SMTP connections around, cycled using an LRU method, in case a new
>>> message arrives that is routed to a recent destination to avoid the cost
>>> of making a new connection and doing the initial SMTP negotiation.
> 
>> I believe Postfix and Exchange can do this.
> 
> Our server also does something along these lines. (It's a little more complex
> because it ties in to how many simultaneous threads you allow to connect to a
> given destination.)

We are doing this as a storage destination queue basis. I think a CS 
logic that depends on sitting idle with a presumed server 5 mins usage 
imposes a serious problem with receiver loading limits, system 
availability and scalability concerns.

For example, if a system allowed for X receivers, the more clients 
follow a CS logic, the closer the connections reach X, the more the 
server availability goes down.

NOT GOOD!

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 18 Aug 2011 15:31:30 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy wrote:

>>> One of the more ubiquitous MTAs out there has a feature called
>>> "connection caching" where it will keep some fixed number of outbound
>>> SMTP connections around, cycled using an LRU method, in case a new
>>> message arrives that is routed to a recent destination to avoid the cost
>>> of making a new connection and doing the initial SMTP negotiation.
>> I believe Postfix and Exchange can do this.
> 
> And I was referring to Sendmail.
> 
> The more SMTP clients there are that do this, the more likely it is that 
> a server demanding QUIT before scheduling a delivery attempt will lose 
mail.

Fortunately, that is *never* the case hence there is no problem until 
you try to correct a problem.  It helps with robustness by reducing 
problematic transactions, especially the exploited ones.

But you did highlight something I need to look at, not because of 5321 
required QUIT which by the way is only going to be addressed with YASO 
(Yet Another Server Option),  but rather analyze what the major 
dangerous presumptions and selfish engineering considerations a client 
operating under a Connection Sharing (CS) mode imposes on receivers. 
It might explains some situations seen.

First, a server may have Receiver Loading Limits and a client with a 
tenacity to take up more CPU time than it needs can get it flagged. To 
use CS for the purpose to reduce its redundancy without considering 
the load it places on receivers is a very selfish engineering 
consideration. Chewing up a channel unnecessarily for 5 minutes 
disallowing other clients connection access increases system available 
and scalability problems.

Second, it uses a dangerous presumption that a server is FIXED on 
using 5 minutes for idle times. 5 minutes is a SHOULD, not a MUST and 
for a client to DEPEND on a SERVER using 5 minutes is short-sighted 
engineering.

This should be done using an SMTP extension where a server can expose 
server timeouts perhaps and willingness to allow a client to sit idle 
redundantly and repeatedly.

Whats surreal about this, is the constant conflict of people ramming 
specs in people's spec yet have this strange tolerance for allowing 
and accommodating failure.  Very odd.   That doesn't work very well, 
IMEO, and if you wish to call that a "singular interpretation" thats 
fine by me.

As this particular case shown, the ignorance of the failure would of 
delivered bad messages and also delivered duplicated bad messages.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: [email-masked]
Cc: "Murray S. Kucherawy" <[email-masked]>, "[email-masked]" <[email-masked]>
Date: Thu, 18 Aug 2011 12:25:33 -0700 (PDT)
Subject: RE: Mail Data termination
To: Tony Finch <[email-masked]>
Sender: [email-masked]

> Murray S. Kucherawy <[email-masked]> wrote:
> >
> > One of the more ubiquitous MTAs out there has a feature called
> > "connection caching" where it will keep some fixed number of outbound
> > SMTP connections around, cycled using an LRU method, in case a new
> > message arrives that is routed to a recent destination to avoid the cost
> > of making a new connection and doing the initial SMTP negotiation.

> I believe Postfix and Exchange can do this.

Our server also does something along these lines. (It's a little more complex
because it ties in to how many simultaneous threads you allow to connect to a
given destination.)

				Ned

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Thu, 18 Aug 2011 10:22:36 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: Tony Finch [mailto:[email-masked]] On Behalf Of Tony Finch
> Sent: Thursday, August 18, 2011 4:14 AM
> To: Murray S. Kucherawy
> Cc: [email-masked]
> Subject: RE: Mail Data termination
> 
> Murray S. Kucherawy <[email-masked]> wrote:
> > One of the more ubiquitous MTAs out there has a feature called
> > "connection caching" where it will keep some fixed number of outbound
> > SMTP connections around, cycled using an LRU method, in case a new
> > message arrives that is routed to a recent destination to avoid the cost
> > of making a new connection and doing the initial SMTP negotiation.
> 
> I believe Postfix and Exchange can do this.

And I was referring to Sendmail.

The more SMTP clients there are that do this, the more likely it is that a server demanding QUIT before scheduling a delivery attempt will lose mail.

Date: Thu, 18 Aug 2011 08:40:55 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Tony Finch wrote:
> Murray S. Kucherawy <[email-masked]> wrote:
>> One of the more ubiquitous MTAs out there has a feature called
>> "connection caching" where it will keep some fixed number of outbound
>> SMTP connections around, cycled using an LRU method, in case a new
>> message arrives that is routed to a recent destination to avoid the cost
>> of making a new connection and doing the initial SMTP negotiation.
> 
> I believe Postfix and Exchange can do this.

This is also basically known as worker pools.  Generally used for 
scalability, as of recent years also used to help address socket/ports 
exhaustion exploits.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 18 Aug 2011 08:26:15 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Tony Finch wrote:
> Murray S. Kucherawy <[email-masked]> wrote:
>> I don't think it clearly indicates that.  I think this is more plausible:
> 
> Or this:
> 
>> - client sends DATA
>> - server sends 354
>> - client starts sending a broken message body that contains an errant <CR><LF>.<CR><LF> plus more after it
>> - server sees the termination, starts message delivery and returns 250
>> - server starts seeing the other message content as SMTP commands, rejecting them all with 5xx
> 
> - server drops the connection
> - client sees the connection drop while still trying to write message data
> - client aborts the transaction and retries later
> 
> Tony.

Digging further on who dropped the connection, well the logged 10054 
socket error is a "remote forced closed" error.  To double check, my 
particular setup did not have a limit set on 500 responses issued. It 
would of logged the disconnection if a limit was set:

     DWORD nCode = 500;
     nCommandErrors++;
     if (MaxCommandErrors && (nCommandErrors > MaxCommandErrors)) {
         Send("%3d '%s': command not understood. (error count: %d) 
Disconnecting!\r\n",
              nCode, cmdline,nCommandErrors);
         Control->Close();
         return;
     }
     Send("%3d '%s': command not understood.\r\n", nCode,cmdline);

But MaxCommandError was set at zero and the server was no dropping the 
line.

Since the client dropped the line (10054 was the real clue here), it 
makes it more speculative as to why it dropped it without knowing its 
comm I/O logic.

I mean, SMTP does not have any design consideration or expectation for 
a client in a DATA state to be reading a socket during its pumping out 
the data.  In other words, short of a drop, there is no official 
method to abort a data exchange.

But I can also see an implementation who uses this non-expectation and 
assume that its reader buffer must remain quiet (empty) and it may 
have "sanity check" just to make sure.  If there is information on its 
reader buffer while it loop pumps out its DATA payload, I can see this 
being used an an unofficial method of an ABORT and hence it drops the 
line.  I guess, one can use the theory that a client should not expect 
any application level noise coming from the receiver during its DATA pump.

One thing we do know, it was all FUBAR and IMV, how liberal we should 
be on accepting non-compliancy such as this makes things more complex, 
more problematic than not. The Mail was no longer valid in terms of 
incomplete capture. It wasn't a good transaction. Passing this junk to 
users is not a good idea, IMV.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 18 Aug 2011 12:13:41 +0100
From: Tony Finch <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
cc: "[email-masked]" <[email-masked]>
Subject: RE: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy <[email-masked]> wrote:
>
> One of the more ubiquitous MTAs out there has a feature called
> "connection caching" where it will keep some fixed number of outbound
> SMTP connections around, cycled using an LRU method, in case a new
> message arrives that is routed to a recent destination to avoid the cost
> of making a new connection and doing the initial SMTP negotiation.

I believe Postfix and Exchange can do this.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Trafalgar: Variable 3 or 4, becoming north 5 at times. Slight or moderate.
Thundery showers. Good, occasionally poor.

Date: Thu, 18 Aug 2011 12:08:40 +0100
From: Tony Finch <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
cc: "[email-masked]" <[email-masked]>
Subject: RE: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy <[email-masked]> wrote:
>
> I don't think it clearly indicates that.  I think this is more plausible:

Or this:

> - client sends DATA
> - server sends 354
> - client starts sending a broken message body that contains an errant <CR><LF>.<CR><LF> plus more after it
> - server sees the termination, starts message delivery and returns 250
> - server starts seeing the other message content as SMTP commands, rejecting them all with 5xx

- server drops the connection
- client sees the connection drop while still trying to write message data
- client aborts the transaction and retries later

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Lundy, Fastnet, Irish Sea: North 4 or 5 backing south or southwest 5 or 6.
Slight or moderate. Showers. Good.

Date: Thu, 18 Aug 2011 12:00:05 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: John C Klensin <[email-masked]>, Keith Moore <[email-masked]>, SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Hector Santos <[email-masked]> wrote:
>
> First, there are plenty of clients who even after seeing a 250 will not
> interpret this as an indicator the message transaction is complete. Until they
> are able to enter the final state and issue QUIT  (or a state where it moves
> into new transaction command), they will continue to retry.

Can you cite specific software that does this?

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Malin, Hebrides, Bailey: Variable 4, becoming southeasterly 6 or 7, perhaps
gale 8 later. Slight or moderate, becoming moderate or rough. Showers then
rain. Moderate or good.

Date: Thu, 18 Aug 2011 11:40:58 +0100
From: Tony Finch <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Hector Santos <[email-masked]> wrote:
>
> I've been seeing (maybe because I'm looking) more transactions from one of
> more particular senders that include a <CRLF>.<CRLF> end of data terminator in
> the body but continue with additional text after the DATA has been accepted.

That reminds me of a bug in Microsoft Exchange 2003 that I encountered
last year: http://fanf.livejournal.com/106903.html

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Biscay: Variable 3 or 4. Slight, occasionally moderate. Showers. Good,
occasionally poor.

Date: Wed, 17 Aug 2011 20:18:18 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Sabahattin Gucukoglu <[email-masked]>
CC: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Sabahattin Gucukoglu wrote:

> Wait a minute.  You didn't advertise PIPELINING.  Do your logs report 
> the lock-step reads and writes of CRLF-terminated lines from the network, or 
> would they print the entire buffer content?  If the former, there is 
> the possibility that the client never finishes sending data to you before it 
> gets a write error, and is thus forced to shut down without completing a mail 
> transaction (it doesn't realise that it has incorrectly terminated the 
> transaction).  This would not indicate that it wants to send QUIT, only that 
> it never got a chance to read the 250 response before you killed the 
connection.

Hi,

Without knowing the devil details regarding the COMM I/O used by the 
client, its all speculation. I lean on the side the client has a read 
buffer that has 250 in there waiting to be read just like the IETF 
mailer have shown.  You would not expect the client in DATA mode 
reading the socket while its sending the payload. But its also 
plausible that all the 500 responses pushed out the 250 at the top of 
the read buffer when it was still sending out text.

No matter what, it was a screwed up transaction to start and I don't 
believe its a good idea to correct it by relaxing compliancy or 
presume that everything is still ok.  When there is irregularities, 
accepting and automatically starting a delivery purely based on 250 is 
more problematic than not. The message was truncated and if signed, no 
longer DKIM valid.

Just consider, if the server option NO-QUIT-CANCEL was disabled (the 
log showed its state when the session started), then our server would 
of started the delivery process immediately (in the background) after 
issuing the 250 and this sender would of still began it additional 
resends with what are now dupes.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: 17 Aug 2011 17:23:49 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Mail Data termination
Organization: 
Sender: [email-masked]

>How does the 'discard' action of RFC5617 (DKIM ADSP) fit into this 
>picture? Although RFC5321 and RFC5617 operate on different protocol 
>layers, in real life the two are often intertwined by way of MTA milter 
>callouts during SMTP DATA phase.

For people who choose to implement ADSP (and you know how I feel about
that), I think it's fair to say that it operates at a layer above SMTP.

If you transported your mail by uucp, DKIM and ADSP would work exactly
the same way as they do if it's transported by SMTP.

R's,
John

Date: Wed, 17 Aug 2011 12:45:17 +0200
From: "Rolf E. Sonneveld" <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 8/17/11 11:40 AM, John C Klensin wrote:
>
>
> --On Tuesday, August 16, 2011 22:58 -0700 "Murray S. Kucherawy"
> <[email-masked]>  wrote:
>
>>> -----Original Message-----
>>> From: [email-masked]
>>> [mailto:[email-masked]] On Behalf Of Murray S.
>>> Kucherawy Sent: Tuesday, August 16, 2011 10:03 PM
>>> To: [email-masked]
>>> Subject: RE: Mail Data termination
>>>
>>> SMTP has always held
>>> that the client is responsible for retrying delivery until
>>> the server gives a 250 or 5xx back after DATA, at which point
>>> the server is now responsible for the message.
>> Sorry, that's not quite right:
>>
>> SMTP has always held that the client is responsible for
>> retrying delivery until the server gives a 250 or 5xx back
>> after DATA, at which point the server has either accepted or
>> finally rejected the message; if the message is accepted, the
>> server is now responsible for its relay or delivery.
> Still not quite right.
>
> 	"...if the message is accepted, the server is now
> 	responsible for its relay, delivery, or rejection via an
> 	NDN ('bounce') message."

(At the risk of opening up a can of worms :-)

How does the 'discard' action of RFC5617 (DKIM ADSP) fit into this 
picture? Although RFC5321 and RFC5617 operate on different protocol 
layers, in real life the two are often intertwined by way of MTA milter 
callouts during SMTP DATA phase.

Maybe we should rephrase the above to:

	"...if the message is accepted, the server is now
	responsible for its disposition."

/rolf

Date: Wed, 17 Aug 2011 05:47:53 -0400
From: John C Klensin <[email-masked]>
To: Randall Gellens <[email-masked]>, Hector Santos <[email-masked]>, Keith Moore <[email-masked]>
cc: SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Tuesday, August 16, 2011 19:29 -0700 Randall Gellens
<[email-masked]> wrote:

> 
> At 8:20 PM -0400 8/16/11, Hector Santos wrote:
> 
>>  NO QUIT means a cancellation because otherwise the only
>>  other way  to end a SMTP session is for the client to close
>> the socket  connection.
> 
> To me, this sounds like a confusion between POP and SMTP.  In
> POP, not sending a QUIT cancels any pending deletions.  In
> SMTP, each MAIL transaction stands alone.

Interesting insight.  Yes.

>  Sending a QUIT at
> the very end is required, but failing to do so has no ill
> effects (and there are a number of clients who don't bother).

Actually, there is a potentially bad effect, but it isn't on
mail transactions.    Depending on the quality of TCP/IP
implementation on the server and some other factors, a client
close may not be noticed for some time.  I.e., the client thinks
the connection had been closed, while the server is still
listening on it.  I'll leave a discussion of all of  the nasty
things that could be done by an attacker getting hold of a
connection in that state as an exercise, but that is at least
one of the reasons why the spec says "handshake then close"
rather than "just close".

   john

Date: Wed, 17 Aug 2011 05:40:48 -0400
From: John C Klensin <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>, [email-masked]
Subject: RE: Mail Data termination
Sender: [email-masked]

--On Tuesday, August 16, 2011 22:58 -0700 "Murray S. Kucherawy"
<[email-masked]> wrote:

> 
>> -----Original Message-----
>> From: [email-masked]
>> [mailto:[email-masked]] On Behalf Of Murray S.
>> Kucherawy Sent: Tuesday, August 16, 2011 10:03 PM
>> To: [email-masked]
>> Subject: RE: Mail Data termination
>> 
>> SMTP has always held
>> that the client is responsible for retrying delivery until
>> the server gives a 250 or 5xx back after DATA, at which point
>> the server is now responsible for the message.
> 
> Sorry, that's not quite right:
> 
> SMTP has always held that the client is responsible for
> retrying delivery until the server gives a 250 or 5xx back
> after DATA, at which point the server has either accepted or
> finally rejected the message; if the message is accepted, the
> server is now responsible for its relay or delivery.

Still not quite right.  

	"...if the message is accepted, the server is now
	responsible for its relay, delivery, or rejection via an
	NDN ('bounce') message."

See the text quoted in my immediately-previous note and the
comments I made about it.

This whole "formal acceptance of responsibility" bit really just
says that a server must not casually discard a message and that
it must be designed to be robust against that happening in
anything but the most drastic situations.  Because the spec was
deliberately intended to permit a server to defer all actual
processing of the message until it got around to it (see
previous note), the "receive EOD, issue Success response"
sequence guarantees only that the responsibility handoff has
occurred.   It is completely conforming for the server to come
back some time later with an NDN that says that none of the
addresses were deliverable.

At one time, with different server speeds and typically
available bandwidth, we believed that "accept, defer all address
and message  processing until after EOD, and bounce if needed"
was a much better strategy than trying to do as much as possible
at SMTP time.  Today, the pendulum has swung in the other
direction, propelled by concerns about blowback, discarded NDNs,
and so on.  But SMTP hasn't changed: decisions about when and
how the message should be examined and processed are strictly up
to the server as long as it doesn't try to avoid responsibility
for messages it has accepted.

   john

Date: Wed, 17 Aug 2011 05:26:26 -0400
From: John C Klensin <[email-masked]>
To: SM <[email-masked]>, Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Tuesday, August 16, 2011 21:51 -0700 SM <[email-masked]>
wrote:

> Hi Hector,
> At 20:23 16-08-2011, Hector Santos wrote:
>> So the only highly subjective argument which I will never buy
>> as  being absolute in either direction is that the receiver
>> MUST ALWAYS  deliver with the 250 even for invalid messages.
> 
> As a clarification, acceptance of a message (SMTP transaction)
> does not equate with delivery, i.e. the SMTP server already
> sent the 250, it cannot send a 550 as it already provided a
> reply code.  John commented on the server sending a NDN if it
> deems fit.  Randall Gellens commented on the timing issue.  I
> haven't seen anyone arguing in this thread that the receiver
> must always deliver.

Two additions:

(1) The somewhat convoluted wording in Section 3.3 of RFC 5321
about the receipt of the end of mail data indicator is there for
a very specific purpose, which is to be sure that, while the
server has accepted the message (and a responsibility handoff),
it is not required to deliver it immediately after receiving the
EOD.  It is also not required to deliver it immediately after
receiving a QUIT or RSET.   As far as the spec is concerned, it
can initiate delivery to a mailbox or mail store any time
between when it receives the EOD and the day the sun burns out.
It can also choose to bounce the message at any time during that
interval.  Of course, waiting days or months after the message
has been accepted before delivering (or bouncing) it would be
terrible operational practice, but it is prohibited only by good
sense, not by 5321.

(2) At least as far as I can remember or been able to find,
nowhere does 5321 say "MUST deliver".  Even if it did, the very
broad language of Section 7.8 (titled "Resistance to Attacks"
and often known as the "operational necessity" provision).  What
it does say instead is very important for this discussion:
Section 2.1 talks about a formal handoff of responsibility when
the EOD is received and a success response delivered.  That
handoff is associated with
	"MUST accept responsibility for either delivering the
	message or properly reporting the failure to do so".
That is a far cry from "MUST deliver".  And, again, it is tied
to receipt of EOD, sending out a success reply, and terminating
the mail transaction state, not to QUIT or any future command.
There is no difference at all between RFC 821 and 2821/5321 in
that regard -- the prohibition on the client closing the
connection without sending QUIT first is about the SMTP session,
not a mail transaction.

   john

Date: Wed, 17 Aug 2011 05:01:15 -0400
From: John C Klensin <[email-masked]>
To: Randall Gellens <[email-masked]>, Hector Santos <[email-masked]>, Keith Moore <[email-masked]>
cc: SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Tuesday, August 16, 2011 19:38 -0700 Randall Gellens
<[email-masked]> wrote:

>>  But, as I hope is clear from the above, no such thing
>>  happens. If the drop occurs after EOD, the mail (message)
>>  transaction is over already, completed by EOD.  If the drop
>>  occurs before EOD, the mail transaction and the SMTP session
>>  are abandoned and, because EOD wasn't received by the
>>  server, nothing is delivered.
> 
> There is of course a timing hole (as there always is): if the
> connection drops after the client sent EOF but before the
> client received a reply, then the client can't know if the
> server is processing the message or not.

Of course, this issue was news ... in 1988.    See RFC 1047.

   john

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 16 Aug 2011 23:57:05 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Tuesday, August 16, 2011 10:22 PM
> To: SM
> Cc: John C Klensin; [email-masked]
> Subject: Re: Mail Data termination
> 
> Again, this is repeated 5 times before it decides to stop. Thats a
> clear indication that the 250 response is not viewed as the
> transaction complete signal that a server has accepted the transaction
> for delivery and the client can "believe" it doesn't need to send the
> QUIT nor retry again and can just drop the line.

I don't think it clearly indicates that.  I think this is more plausible:

- client sends DATA
- server sends 354
- client starts sending a broken message body that contains an errant <CR><LF>.<CR><LF> plus more after it
- server sees the termination, starts message delivery and returns 250
- server starts seeing the other message content as SMTP commands, rejecting them all with 5xx
- client finally reaches the end of its idea of the body
- client then sees the 250 the server sent, which makes it happy
- client then also sees a ton of other stuff it didn't expect, which makes it unhappy
- client assumes the connection has gone south and disconnects without sending QUIT

I suspect the client is not sending QUIT because the two SMTP sessions are clearly out of sync, not because it thinks QUIT is the right way to initiate delivery.

Subject: Re: Mail Data termination
Date: Wed, 17 Aug 2011 07:55:02 +0100
To: [email-masked]
From: Sabahattin Gucukoglu <[email-masked]>
Reply-To: Sabahattin Gucukoglu <[email-masked]>
Sender: [email-masked]

On 17 Aug 2011, at 06:21, Hector Santos wrote:
> SM wrote:
>> At 20:23 16-08-2011, Hector Santos wrote:
>>> So the only highly subjective argument which I will never buy as being absolute in either direction is that the receiver MUST ALWAYS deliver with the 250 even for invalid messages.
>> As a clarification, acceptance of a message (SMTP transaction) does not equate with delivery, i.e. the SMTP server already sent the 250, it cannot send a 550 as it already provided a reply code.  John commented on the server sending a NDN if it deems fit.  Randall Gellens commented on the timing issue.  I haven't seen anyone arguing in this thread that the receiver must always deliver.
> 
> Point taken, and I agree, but I guess I took offense to the incorrect singular interpretation label which is clearly not correct and wouldn't make sense to design products that will operates one way that no one supports.
> 
> Here is one sender log (trust me its more than one sender) that will resend regardless of DATA response
> 
>        250 message accepted for delivery
> 
> simply because it was not able to issue the 2821/5321 SMTP required QUIT.

Wait a minute.  You didn't advertise PIPELINING.  Do your logs report the lock-step reads and writes of CRLF-terminated lines from the network, or would they print the entire buffer content?  If the former, there is the possibility that the client never finishes sending data to you before it gets a write error, and is thus forced to shut down without completing a mail transaction (it doesn't realise that it has incorrectly terminated the transaction).  This would not indicate that it wants to send QUIT, only that it never got a chance to read the 250 response before you killed the connection.

Cheers,
Sabahattin

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 16 Aug 2011 22:58:36 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Murray S. Kucherawy
> Sent: Tuesday, August 16, 2011 10:03 PM
> To: [email-masked]
> Subject: RE: Mail Data termination
> 
> SMTP has always held
> that the client is responsible for retrying delivery until the server
> gives a 250 or 5xx back after DATA, at which point the server is now
> responsible for the message.

Sorry, that's not quite right:

SMTP has always held that the client is responsible for retrying delivery until the server gives a 250 or 5xx back after DATA, at which point the server has either accepted or finally rejected the message; if the message is accepted, the server is now responsible for its relay or delivery.

Date: Wed, 17 Aug 2011 01:21:48 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

SM wrote:
> Hi Hector,
> At 20:23 16-08-2011, Hector Santos wrote:
>> So the only highly subjective argument which I will never buy as being 
>> absolute in either direction is that the receiver MUST ALWAYS deliver 
>> with the 250 even for invalid messages.
> 
> As a clarification, acceptance of a message (SMTP transaction) does not 
> equate with delivery, i.e. the SMTP server already sent the 250, it 
> cannot send a 550 as it already provided a reply code.  John commented 
> on the server sending a NDN if it deems fit.  Randall Gellens commented 
> on the timing issue.  I haven't seen anyone arguing in this thread that 
> the receiver must always deliver.
> 
> Regards,
> -sm

Point taken, and I agree, but I guess I took offense to the incorrect 
singular interpretation label which is clearly not correct and 
wouldn't make sense to design products that will operates one way that 
no one supports.

Here is one sender log (trust me its more than one sender) that will 
resend regardless of DATA response

         250 message accepted for delivery

simply because it was not able to issue the 2821/5321 SMTP required 
QUIT.

The following session log is repeated 5 times so I will only show the 
first one. This is for a XML-DEV mailing list I subscribed and no 
doubt, as I suspect now do some unique message structure. I never had 
a problem with this sender in the near 10 years as a member:

**************************************************************************
Wildcat! ESMTP Server v6.4.454.1
SMTP log started at Sun, 07 Aug 2011  04:20:54
Connection Time: 20110807 04:20:54  cid: 0000A022 tid: 00000F04
SSL-Enabled=NO No-Quit-Cancel=ON Receiver-Bin=ON
Client IP: 66.151.234.57:42705 (unknown) Host IP: 208.247.131.9:25
04:20:54.752 S: 220 winserver.com Wildcat! ESMTP Server v6.4.454.1 ready
04:20:54.804 C: EHLO sf01.oasis-open.org
04:20:54.804 S: 250-winserver.com, Pleased to meet you.
04:20:54.804 S: 250-SIZE 10240000
04:20:54.804 S: 250-8BITMIME
04:20:54.804 S: 250-SUBMITTER
04:20:54.804 S: 250-ETRN
04:20:54.805 S: 250-AUTH CRAM-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
04:20:54.805 S: 250-AUTH=LOGIN
04:20:54.805 S: 250 HELP
04:20:54.945 C: MAIL 
FROM:<xml-dev-return-55772-xml-dev=[email-masked]> 
SIZE=724226
04:20:54.945 S: 250 
<xml-dev-return-55772-xml-dev=[email-masked]>... Sender 
validation pending. Continue.
04:20:54.997 C: RCPT TO:<[email-masked]>
04:20:55.273 ** WCX Process: wcsap ret: -1 (275 msecs)
04:20:55.273 S: 250 <[email-masked]>... Recipient ok
04:20:55.329 C: DATA
04:20:55.329 S: 354 Start mail input; end with <CRLF>.<CRLF>
04:20:55.407 ** end of data received. (bytes: 2622) (24.7 K/s)
04:20:55.591 ** WCX Process: SmtpFilterHookLoader  ret: 1 (183 msecs)
04:20:55.592 S: 250 Message received for delivery. (bytes: 2622)
04:20:55.592 C: Candle's  markup_
04:20:55.592 S: 500 'Candle's  markup_ ': command not understood.
04:20:55.592 C: 
(http://www.candlescript.org/doc/candle-markup-reference.htm) 
language is strongly-typed even without schema, whereas XML
04:20:55.592 S: 500 
'(http://www.candlescript.org/doc/candle-markup-reference.htm) 
language is strongly-typed even without schema, whereas XML  ': 
command not understood.
04:20:55.592 C: is only weakly-typed without schema.
04:20:55.592 S: 500 'is only weakly-typed without schema.  ': command 
not understood.

a bunch of these 500 response finally ending with:

04:25:56.440 ** connection drop - error: 10054 state: 
tMessageQuitPending lastcmd: DATA
04:25:56.441 ** Cancelling message delivery due to incomplete 
transaction (NO QUIT).
04:25:56.441 ** Completed

Again, this is repeated 5 times before it decides to stop. Thats a 
clear indication that the 250 response is not viewed as the 
transaction complete signal that a server has accepted the transaction 
for delivery and the client can "believe" it doesn't need to send the 
QUIT nor retry again and can just drop the line.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 16 Aug 2011 22:03:03 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John C Klensin
> Sent: Tuesday, August 16, 2011 6:14 PM
> To: Hector Santos; Keith Moore
> Cc: SM; [email-masked]
> Subject: Re: Mail Data termination
> 
> Mail transactions are initiated by a MAIL command (see Section
> 3.3) and extend until a QUIT or RSET is issued or a DATA command
> completes, whichever comes first.   I think the spec is very
> clear about that -- if you disagree, point out where and,
> preferably, supply text.

That's always been my understanding as well, and I don't see the ambiguity being claimed.  I've never encountered the reading of the SMTP specs that's being argued here, either in a client or a server.

One of the more ubiquitous MTAs out there has a feature called "connection caching" where it will keep some fixed number of outbound SMTP connections around, cycled using an LRU method, in case a new message arrives that is routed to a recent destination to avoid the cost of making a new connection and doing the initial SMTP negotiation.  That means this MTA would connect, do HELO/EHLO, MAIL, RCPT, DATA, dot, and then wait for more mail to come in (or for the cached connection to expire).  With this interpretation, that means only the last (n-1) messages are ever actually delivered while the connection is held open, and the nth message goes missing if the connection is somehow terminated without QUIT.

A client that implements this could still reasonably expect its mail to be delivered because it is breaking a rule about terminating sessions, not terminating transactions.  And, most importantly, it at least got the post-DATA status code so it knows if the message got accepted or not.  It's certainly broken, but it will still work.

The real concern with that is the case where the server following this logic replies 250 after DATA, so the client thinks the mail went through, but then for network reasons the connection dies before QUIT is received, so the server throws away the mail.  SMTP has always held that the client is responsible for retrying delivery until the server gives a 250 or 5xx back after DATA, at which point the server is now responsible for the message.  Any server violating that tenet seems seriously damaged to me.

-MSK

Date: Tue, 16 Aug 2011 21:51:53 -0700
To: Hector Santos <[email-masked]>
From: SM <[email-masked]>
Subject: Re: Mail Data termination
Cc: John C Klensin <[email-masked]>, [email-masked]
Sender: [email-masked]

Hi Hector,
At 20:23 16-08-2011, Hector Santos wrote:
>So the only highly subjective argument which I will never buy as 
>being absolute in either direction is that the receiver MUST ALWAYS 
>deliver with the 250 even for invalid messages.

As a clarification, acceptance of a message (SMTP transaction) does 
not equate with delivery, i.e. the SMTP server already sent the 250, 
it cannot send a 550 as it already provided a reply code.  John 
commented on the server sending a NDN if it deems fit.  Randall 
Gellens commented on the timing issue.  I haven't seen anyone arguing 
in this thread that the receiver must always deliver.

Regards,
-sm 

Subject: Re: Mail Data termination
From: Dave Crocker <[email-masked]>
Date: Tue, 16 Aug 2011 21:36:43 -0700
To: [email-masked]
Sender: [email-masked]

------DJCJJJBLDC7NBJ1DRYE8815EGL5CH7
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 8bit

+1
-- 

[email-masked] wrote:
> On Aug 16, 2011, at 2:56 PM, Keith Moore wrote:
> > On Aug 16, 2011, at 2:46 PM, Randall Gellens wrote:
> >
> >> This is something that is so broken I think the best course is to reject it.
> >
> > +1

> sorry, I misunderstood what was being suggested. if the server gets DATA
> followed by a message (or a message fragment) followed by CRLF.CRLF, it should
> accept the message (or fragment) and deliver it. no matter what else follows,
> or doesn't follow, during the same SMTP session .

+1

				Ned

/d

--
Dave Crocker
bbiw.net

via mobile
------DJCJJJBLDC7NBJ1DRYE8815EGL5CH7
Content-Type: text/html;
 charset=utf-8
Content-Transfer-Encoding: 8bit

+1<br>
-- <br>
<br>
<br>
[email-masked] wrote:<br>
&gt; On Aug 16, 2011, at 2:56 PM, Keith Moore wrote:<br>
&gt; &gt; On Aug 16, 2011, at 2:46 PM, Randall Gellens wrote:<br>
&gt; &gt;<br>
&gt; &gt;&gt; This is something that is so broken I think the best course is to reject it.<br>
&gt; &gt;<br>
&gt; &gt; +1<br>
<br>
&gt; sorry, I misunderstood what was being suggested. if the server gets DATA<br>
&gt; followed by a message (or a message fragment) followed by CRLF.CRLF, it should<br>
&gt; accept the message (or fragment) and deliver it. no matter what else follows,<br>
&gt; or doesn&#39;t follow, during the same SMTP session .<br>
<br>
+1<br>
<br>
				Ned<br>
<br>
<br>
<br>
/d<br>
<br>
--<br>
   Dave Crocker<br>
   <a href="http://bbiw.net">bbiw.net</a><br>
<br>
   via mobile
------DJCJJJBLDC7NBJ1DRYE8815EGL5CH7--

Date: Wed, 17 Aug 2011 00:12:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Steve Atkins <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Steve Atkins wrote:
> 
> On Aug 16, 2011, at 8:23 PM, Hector Santos wrote:
> 
>> In short, it was technically an invalid RFC5322 message or probably more correct, the SMTP client failed to properly escape dotted lines for whatever reason (I think Ned is correct that it probably did the QP conversion on the fly).
> 
> Can you clarify what you mean by an "invalid RFC 5322 message"? Bare CRs or LFs? Lines longer than 998 characters long? Something else?

As stated above, its probably more correct to read it as a SMTP client 
failing to escape a dotted line (<CRLF>.<CRLF>) which is not the true 
EOD terminator.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 17 Aug 2011 00:10:42 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Keith Moore <[email-masked]>, SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

John C Klensin wrote:
> 
> Mail transactions are initiated by a MAIL command (see Section
> 3.3) and extend until a QUIT or RSET is issued or a DATA command
> completes, whichever comes first.   I think the spec is very
> clear about that -- if you disagree, point out where and,
> preferably, supply text.  

As stated, my server implementation support both 821 and 2821/5321, 
where the former the 250 response is coupled with a immediate signal 
to the router to process the message, then the receiver moves back to 
command mode and waits for more SMTP commands.  A client can drop the 
line with no change in message status. QUIT is not required per 821. 
In the 2821 QUIT required mode, an ungraceful session aborts the 
pending delivery. It has to issue a QUIT or start a new transaction, 
at which the router is signed to finally handle the message.

If you think this an error in interpretation than I am clearly not the 
only one, because there are senders, who irregardless of the 250 
response will wait until they are able to issue a QUIT before making 
the message "Sent" complete.  If they are not able to send the QUIT, 
again even with the 250 issued, they will most definitely try again.

If we are wrong, then the 2821/5321 specs is bad because it has a QUIT 
MUST mandate and you can't blame use because the only logical 
technical different with 821 was that it was not required and the only 
thing left was a 250 signal and thus allow them to drop the connection 
as soon as they saw that.

That is not allowed in 2821/5321.

All I can say that it wasn't done without reason in these day of 
spammers who exploit the ideas of blasting receivers and not 
respecting any server responses.  The only way to catch these is with 
a QUIT requirement. Going back to 821 mode where QUIT is not required 
is guaranteed to cause a system to add more design consideration for 
dupe checking.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 23:44:48 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Randall Gellens <[email-masked]>
CC: Keith Moore <[email-masked]>, SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Randall Gellens wrote:
> 
> At 8:20 PM -0400 8/16/11, Hector Santos wrote:
> 
>>  NO QUIT means a cancellation because otherwise the only other way to 
>> end a SMTP session is for the client to close the socket connection.
> 
> To me, this sounds like a confusion between POP and SMTP.  In POP, not 
> sending a QUIT cancels any pending deletions.  In SMTP, each MAIL 
> transaction stands alone.  Sending a QUIT at the very end is required, 
> but failing to do so has no ill effects (and there are a number of 
> clients who don't bother).

Excellent point Randall, but rest assured it is not issue of 
confusion. In POP3, issuing an QUIT is a requirement in order to move 
the backend into update mode (either to delete or marked mail received 
depending on the backend mail storage policies). There is no question 
about this handshaking requirement.

For SMTP, IMTO, it is an issue of mixed 821 vs 2821 mindsets, old vs 
new era and an old era that worked under a design presumption on no 
errors - mail was RFC822 valid, the state machine as always in sync, 
no reason to presume a redundant sequence of 500 or more command 
unknown responses has a special meaning and could cause a server to 
drop the line versus an era where it all does have a meaning for 
following strong SMTP compliancy.

Just consider if we are saying that a 250 is all a client needs to 
see, the current SMTP MUST for a QUIT is no longer required and all 
clients can just simply drop the line.

I am only seeing this with what appears to be older software. Other 
senders will retry even after a 250 if they can not issue a SMTP 
required QUIT.  If the server dropped the line because of all the 500 
responses, these senders will retry regardless of the 250.

So its not just my interpretation. If a session can not be gracefully 
completely, 5321 senders will retry if they are not mixing it up with 
a SHOULD as stated in 821.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Subject: Re: Mail Data termination
From: Steve Atkins <[email-masked]>
Date: Tue, 16 Aug 2011 20:45:47 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 8:23 PM, Hector Santos wrote:

> In short, it was technically an invalid RFC5322 message or probably more correct, the SMTP client failed to properly escape dotted lines for whatever reason (I think Ned is correct that it probably did the QP conversion on the fly).

Can you clarify what you mean by an "invalid RFC 5322 message"? Bare CRs or LFs? Lines longer than 998 characters long? Something else?

Cheers,
  Steve

Date: Tue, 16 Aug 2011 23:23:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Keith Moore <[email-masked]>, SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

John,

I fully agree it is a state machine which generally means there is 
little room for subjective opinions.  I'm writing here in general and 
not to specifically to you, you know all this.

As a background. RFC821 had a lower case "should not" for closing the 
connection before issuing a QUIT, where RFC2821 (carried on to 
RFC5321) has a upper case MUST NOT mandate of requiring a QUIT.  The 
different between the two are significant.

I am not in a position to take one side only. We have to support both 
modes.  Some (very few) customers want their SMTP server to behave the 
old way and most want to enforce SMTP compliancy.

First, there are plenty of clients who even after seeing a 250 will 
not interpret this as an indicator the message transaction is 
complete. Until they are able to enter the final state and issue QUIT 
  (or a state where it moves into new transaction command), they will 
continue to retry.  The only ones I have seen that stood out that use 
the 250 as the only requirement are older sender software using the 
821 model.  They will not resend.

Second, spammer exploitations of what they assume are continued 
support for relaxed 821 legacy operations has allowed them to blast 
commands and data with no respect to server responses.  So you see 
many transactions where the residue unread text being now read at 
after DATA in the SMTP command state.

Because of these residue text and redundant consecutive 500 response, 
the server limit kicks in and drops the connection.  If the server 
used the 250 message accepted as a trigger to deliver the mail, the 
spammer found a loophole.

Third, it is written, a QUIT is a MUST mandate (stated as a MUST NOT 
drop connection until a QUIT is issued). Not enforcing this mandate, 
returns the server back to 821 mode and the exploitation problems.

Forth, the only reason the issue was highlighted was because it was 
false positive by a IETF mailing list  I am subscribed too and I took 
the time to find out why it happen.

In short, it was technically an invalid RFC5322 message or probably 
more correct, the SMTP client failed to properly escape dotted lines 
for whatever reason (I think Ned is correct that it probably did the 
QP conversion on the fly).

So the only highly subjective argument which I will never buy as being 
absolute in either direction is that the receiver MUST ALWAYS deliver 
with the 250 even for invalid messages.

What is lost here is that the sender was broken and the truncated 
message was indeed no longer invalid.

Also consider a typical philosophical conflict is the idea of SMTP 
level filter vs POST SMTP filtering where perhaps a 250 is always 
issued and delivery is pending the post smtp processing result after 
the session has been ended.   There is no guarantee of delivery.

If we use the old 821 idea that a 250 is all that is needed by a 
client, then QUIT is not a requirement. Client can just drop.

But that is not what I see with many senders.  The transaction is not 
complete until the SMTP mandate QUIT an be issued. I only see the old 
821 behavior with older software and I say that because they don't 
retry when server drops the line due to all the 500 responses.

Which finally, either mode is not a problem until an non-compliant 
operation take place.

John C Klensin wrote:
> 
> 
> --On Tuesday, August 16, 2011 20:20 -0400 Hector Santos
> <[email-masked]> wrote:
> 
>> OK, I don't think it is productive to repeat these semantic
>> difference of opinions of what section 4.1.1.10 says:
>> ...
> 
> Hector, having either written those words or inherited them from
> Jon Postel and RFC 821, maybe I should offer an opinion.  It is
> really important to remember that SMTP is stateful (see 2.3.6
> among other things) and that the DATA command:
> 
> 	-- Is treated as atomic wrt that state model, with the
> 	354 message being a special "go ahead" indicator, not a
> 	command termination acknowledgement
> 	
> 	-- Terminates the mail transaction in progress.
> 
> Mail transactions are initiated by a MAIL command (see Section
> 3.3) and extend until a QUIT or RSET is issued or a DATA command
> completes, whichever comes first.   I think the spec is very
> clear about that -- if you disagree, point out where and,
> preferably, supply text.  
> 
> Note (also in Section 3.3):
> 
> 	The end of mail data indicator also confirms the mail
> 	transaction and tells the SMTP server to now process the
> 	stored recipients and mail data.  If accepted, the SMTP
> 	server returns a "250 OK" reply.
> 
> Note that it does not say "tentatively confirms pending the next
> command" or any such thing.  The server gets EOD (CRLF.CRLF),
> treats it as confirming the transaction from the client's
> standpoint, initiates processing of the message, and treats the
> mail transaction as ended.
>  
>> IMTO, no matter how you wish read this, NO QUIT means a
>> cancellation because otherwise the only other way to end a
>> SMTP session is for the client to close the socket connection.
> 
> QUIT immediately after the DATA command (and hence the mail
> transaction) completes is just a signal to the server that the
> client is initiating an orderly close.  It has no effect on the
> mail transaction because there isn't one open.  When the server
> receives it, the SMTP session is closed.    QUIT (instead of
> DATA) after one or more RCPT commands is another matter, but
> that isn't the case we are discussing.
> 
> Similarly, having the client close the connection after EOD is
> sent is, indeed, "a cancellation", but it is a cancellation of
> the SMTP session... it has nothing to do with any mail
> transactions.  On the other hand, if the client closes the
> connection prior to sending EOD, that inevitably cancels both
> the mail transaction and the SMTP session... having the server
> deliver anything would be a really bad idea and is clearly (I
> believe) outside  the spec.
> 
>> So what this would indirectly suggest the SMTP specs
>> implicitly says:
>>
>>     There is two ways for a client to end a SMTP session:
>>     QUIT or DROP THE LINE
> 
> That is true even though one of those choices is prohibited
> unless the "operational necessity" provisions are invoked.
> 
>> and that does not jive with the written words:
>>
>>     The sender MUST NOT intentionally close the transmission
>>     channel until it sends a QUIT command,
> 
> It aligns perfectly well.  A client that closes the transmission
> channel without first sending a QUIT is in violation of the
> spec.  That doesn't mean it can't do it.  Because it can do it
> and, more important, because all sorts of other things can cause
> a connection to be dropped, a server has to be prepared to deal
> with a dropped connection and to clean up when it occurs.
> 
> I just don't see a problem there.  Please point out what I'm
> missing.
> 
>  Aside: there is a design characteristic here that I wish
> 	were different in SMTP.  I have wished it for any years,
> 	that has done me no good, and I can see no possible way
> 	to change it now, at least at a plausible level of
> 	effort.  SMTP does not anticipate a case in which a
> 	client opens a mail transaction, sends RCPT commands as
> 	needed, sends DATA and then changes its mind and wants
> 	to abort sending without having the message delivered.
> 	With SMTP as now defined (and as it has always been
> 	defined), it can do that _only_ by closing the
> 	connection prior to sending EOD.  While the use cases
> 	are few, that is just unsanitary, especially in a model
> 	in which we expect closing of an SMTP session to be
> 	preceded by a client-server handshake (i.e., sending
> 	QUIT).   In an alternate universe, we might have had
> 	EOD-and-deliver and EOD-and-abort, not just the first.
> 	But we are getting real close to 30 years after the time
> 	to make that choice differently. 
>  
>> I can understood your position, very clearly, but in general,
>> I don't think this is a good way to do robust client/server
>> communications (allowing drops to signal message transaction
>> completion). It opens the door for unsureness.
> 
> But, as I hope is clear from the above, no such thing happens.
> If the drop occurs after EOD, the mail (message) transaction is
> over already, completed by EOD.  If the drop occurs before EOD,
> the mail transaction and the SMTP session are abandoned and,
> because EOD wasn't received by the server, nothing is delivered.
> 
> There is a potential problem with that model, but it is
> connected to EOD in progress, not to the situation you are
> apparently concerned about.  That issue is discussed in great
> detail in RFC 1047.
> 
>     john
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 19:38:24 -0700
To: John C Klensin <[email-masked]>, Hector Santos <[email-masked]>, Keith Moore <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: Mail Data termination
CC: SM <[email-masked]>, <[email-masked]>
Sender: [email-masked]

At 9:14 PM -0400 8/16/11, John C Klensin wrote:

>   Aside: there is a design characteristic here that I wish
>  	were different in SMTP.  I have wished it for any years,
>  	that has done me no good, and I can see no possible way
>  	to change it now, at least at a plausible level of
>  	effort.  SMTP does not anticipate a case in which a
>  	client opens a mail transaction, sends RCPT commands as
>  	needed, sends DATA and then changes its mind and wants
>  	to abort sending without having the message delivered.
>  	With SMTP as now defined (and as it has always been
>  	defined), it can do that _only_ by closing the
>  	connection prior to sending EOD.  While the use cases
>  	are few, that is just unsanitary, especially in a model
>  	in which we expect closing of an SMTP session to be
>  	preceded by a client-server handshake (i.e., sending
>  	QUIT).   In an alternate universe, we might have had
>  	EOD-and-deliver and EOD-and-abort, not just the first.
>  	But we are getting real close to 30 years after the time
>  	to make that choice differently.

If we did have this, it would probably now be used by spammers to 
probe the acceptability of various messages.

>   > I can understood your position, very clearly, but in general,
>   > I don't think this is a good way to do robust client/server
>>  communications (allowing drops to signal message transaction
>>  completion). It opens the door for unsureness.
>
>  But, as I hope is clear from the above, no such thing happens.
>  If the drop occurs after EOD, the mail (message) transaction is
>  over already, completed by EOD.  If the drop occurs before EOD,
>  the mail transaction and the SMTP session are abandoned and,
>  because EOD wasn't received by the server, nothing is delivered.

There is of course a timing hole (as there always is): if the 
connection drops after the client sent EOF but before the client 
received a reply, then the client can't know if the server is 
processing the message or not.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
We adore chaos because we love to produce order.
                                  --M. C. Escher

Date: Tue, 16 Aug 2011 19:29:46 -0700
To: Hector Santos <[email-masked]>, Keith Moore <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: Mail Data termination
CC: SM <[email-masked]>, <[email-masked]>
Sender: [email-masked]

At 8:20 PM -0400 8/16/11, Hector Santos wrote:

>  NO QUIT means a cancellation because otherwise the only other way 
> to end a SMTP session is for the client to close the socket 
> connection.

To me, this sounds like a confusion between POP and SMTP.  In POP, 
not sending a QUIT cancels any pending deletions.  In SMTP, each MAIL 
transaction stands alone.  Sending a QUIT at the very end is 
required, but failing to do so has no ill effects (and there are a 
number of clients who don't bother).

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
Algol was a great improvement on most of its successors.
                                          --C.A.R Hoare

Date: Tue, 16 Aug 2011 17:56:27 -0700
To: Hector Santos <[email-masked]>
From: SM <[email-masked]>
Subject: Re: Mail Data termination
Cc: [email-masked]
Sender: [email-masked]

Hi Hector,
At 16:48 16-08-2011, Hector Santos wrote:
>Hence the dilemma.  It wasn't an valid RFC5322 message and the issue 
>is exasperated when it was DKIM signed and now recorded as a 
>failure. Never mind the fact QUIT (or a new transaction command) is 
>an SMTP requirement before the transaction is considered complete 
>for delivery.  Its in the specs, but really, how much liberalism in 
>failure to comply to

 From Section 3.3:

   "The end of mail data indicator also confirms the mail transaction and
    tells the SMTP server to now process the stored recipients and mail
    data.  If accepted, the SMTP server returns a "250 OK" reply."

 From Section 3.8:

   "An SMTP connection is terminated when the client sends a QUIT
    command."

Keith already commented on "QUIT" and when a transaction is 
considered as complete.

Regards,
-sm 

Date: Tue, 16 Aug 2011 21:14:17 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, Keith Moore <[email-masked]>
cc: SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Tuesday, August 16, 2011 20:20 -0400 Hector Santos
<[email-masked]> wrote:

> 
> OK, I don't think it is productive to repeat these semantic
> difference of opinions of what section 4.1.1.10 says:
>...

Hector, having either written those words or inherited them from
Jon Postel and RFC 821, maybe I should offer an opinion.  It is
really important to remember that SMTP is stateful (see 2.3.6
among other things) and that the DATA command:

	-- Is treated as atomic wrt that state model, with the
	354 message being a special "go ahead" indicator, not a
	command termination acknowledgement
	
	-- Terminates the mail transaction in progress.

Mail transactions are initiated by a MAIL command (see Section
3.3) and extend until a QUIT or RSET is issued or a DATA command
completes, whichever comes first.   I think the spec is very
clear about that -- if you disagree, point out where and,
preferably, supply text.  

Note (also in Section 3.3):

	The end of mail data indicator also confirms the mail
	transaction and tells the SMTP server to now process the
	stored recipients and mail data.  If accepted, the SMTP
	server returns a "250 OK" reply.

Note that it does not say "tentatively confirms pending the next
command" or any such thing.  The server gets EOD (CRLF.CRLF),
treats it as confirming the transaction from the client's
standpoint, initiates processing of the message, and treats the
mail transaction as ended.
 
> IMTO, no matter how you wish read this, NO QUIT means a
> cancellation because otherwise the only other way to end a
> SMTP session is for the client to close the socket connection.

QUIT immediately after the DATA command (and hence the mail
transaction) completes is just a signal to the server that the
client is initiating an orderly close.  It has no effect on the
mail transaction because there isn't one open.  When the server
receives it, the SMTP session is closed.    QUIT (instead of
DATA) after one or more RCPT commands is another matter, but
that isn't the case we are discussing.

Similarly, having the client close the connection after EOD is
sent is, indeed, "a cancellation", but it is a cancellation of
the SMTP session... it has nothing to do with any mail
transactions.  On the other hand, if the client closes the
connection prior to sending EOD, that inevitably cancels both
the mail transaction and the SMTP session... having the server
deliver anything would be a really bad idea and is clearly (I
believe) outside  the spec.

> So what this would indirectly suggest the SMTP specs
> implicitly says:
> 
>     There is two ways for a client to end a SMTP session:
>     QUIT or DROP THE LINE

That is true even though one of those choices is prohibited
unless the "operational necessity" provisions are invoked.

> and that does not jive with the written words:
> 
>     The sender MUST NOT intentionally close the transmission
>     channel until it sends a QUIT command,

It aligns perfectly well.  A client that closes the transmission
channel without first sending a QUIT is in violation of the
spec.  That doesn't mean it can't do it.  Because it can do it
and, more important, because all sorts of other things can cause
a connection to be dropped, a server has to be prepared to deal
with a dropped connection and to clean up when it occurs.

I just don't see a problem there.  Please point out what I'm
missing.

 Aside: there is a design characteristic here that I wish
	were different in SMTP.  I have wished it for any years,
	that has done me no good, and I can see no possible way
	to change it now, at least at a plausible level of
	effort.  SMTP does not anticipate a case in which a
	client opens a mail transaction, sends RCPT commands as
	needed, sends DATA and then changes its mind and wants
	to abort sending without having the message delivered.
	With SMTP as now defined (and as it has always been
	defined), it can do that _only_ by closing the
	connection prior to sending EOD.  While the use cases
	are few, that is just unsanitary, especially in a model
	in which we expect closing of an SMTP session to be
	preceded by a client-server handshake (i.e., sending
	QUIT).   In an alternate universe, we might have had
	EOD-and-deliver and EOD-and-abort, not just the first.
	But we are getting real close to 30 years after the time
	to make that choice differently. 
 
> I can understood your position, very clearly, but in general,
> I don't think this is a good way to do robust client/server
> communications (allowing drops to signal message transaction
> completion). It opens the door for unsureness.

But, as I hope is clear from the above, no such thing happens.
If the drop occurs after EOD, the mail (message) transaction is
over already, completed by EOD.  If the drop occurs before EOD,
the mail transaction and the SMTP session are abandoned and,
because EOD wasn't received by the server, nothing is delivered.

There is a potential problem with that model, but it is
connected to EOD in progress, not to the situation you are
apparently concerned about.  That issue is discussed in great
detail in RFC 1047.

    john

Date: Tue, 16 Aug 2011 18:03:26 -0700
To: John C Klensin <[email-masked]>, <[email-masked]>, Keith Moore <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: Mail Data termination
CC: Randall Gellens <[email-masked]>, Hector Santos <[email-masked]>, <[email-masked]>
Sender: [email-masked]

At 8:42 PM -0400 8/16/11, John C Klensin wrote:

>  Given various attack scenarios, my personal opinion is
>  that a server should not tolerate more than a couple of lines of
>  noise after successful EOD (CRLF.CRLF) before simply shutting
>  things down by closing the connection.

I'd encourage this.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
A server is only as secure as its dumbest administrator
                                         --Steve Dorner

Date: Tue, 16 Aug 2011 20:55:43 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy wrote:

> That's not what QUIT means.  QUIT closes a session, not a transaction.  The end of DATA completes a transaction (Section 4.1.1.4 of RFC5321) and resets state.
> 
> The idea of not delivering a message because the client didn't send QUIT is 
> a singular interpretation of SMTP as far as I can tell.

Sorry. Not so Murray. There are plenty of clients that will resend 
simply because they are waiting to send the QUIT command as mandated 
by SMTP. If that was not the case, they could simple drop the 
connection all the time once the 250 is received.  The fact is, some 
will drop the line and some will drop  but not before pumping a QUIT, 
just in case, even before getting the 250 response.  In all cases, it 
is in violation of SMTP 2821/5321.

The SMTP spec absolutely does not say in any way shape of form that 
one way to complete a transaction is to DROP THE LINE once a 250 is 
received.  It clearly indicate not sending a QUIT is irregular. The 
client MUST NOT drop the line before sending the QUIT, and it is 
extremely not robust to assume a 250 means mail MUST BE delivered by 
simply dropping the line.

Following it will save you dupe problems with invalid RFC5322 messages 
such this example case.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 20:42:03 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], Keith Moore <[email-masked]>
cc: Randall Gellens <[email-masked]>, Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

--On Tuesday, August 16, 2011 15:19 -0700
[email-masked] wrote:

>> sorry, I misunderstood what was being suggested.   if the
>> server gets DATA followed by a message (or a message
>> fragment) followed by CRLF.CRLF, it should accept the message
>> (or fragment) and deliver it.  no matter what else follows,
>> or doesn't follow, during the same SMTP session .
> 
> +1

+1 also

That is certainly what the spec says now and almost certainly
what it should say.  Anything following CRLF.CRLF is either an
SMTP command that is valid in that context or noise (which
should get a syntax error).   Trying to allow for, much less
standardize, the behavior an SMTP receiver should exhibit when
the sending system can't do dot-stuffing properly is a recipe
for many bad things happening as well as an opportunity for
attack.  Given various attack scenarios, my personal opinion is
that a server should not tolerate more than a couple of lines of
noise after successful EOD (CRLF.CRLF) before simply shutting
things down by closing the connection.  The standard explicitly
permits, but does not particularly encourage, much less require,
that behavior.  Again, I think that is how it should be.  

By the way, SM wrote...

> Yes.  The SMTP server mentioned that it is accepting the
> message (up to the end-of-data terminator.  Rejecting the
> message is not an option once the message has been accepted.

But, while doing so has other disadvantages, the spec certainly
permits an SMTP server to accept the message and then send back
an NDN message, potentially with a nastygram.  So, while
rejecting is not an option, the SMTP server still has
standards-conforming options for not delivering it.

   john

Date: Tue, 16 Aug 2011 20:31:57 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: Paul Smith <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

>>    Be 100% liberal in what you receive, and don't worry about being
>>    very conservative in what you send.
> 
> what are you smoking?

Marlboro Lights. :)

> I'm only saying that all we can do is document how things should work.  If we've 
> done that, and done it clearly, adding more documentation probably won't help.

Or perhaps may. In the highly complex world of proper mail 
integration, I can see a little note saying the following being useful 
to someone:

     In your QP algorithm, take into account any truncation at an ending
     period does not result in sending a SMTP an non-escaped 
".<cr></lf>" line.

That might enough to be an "oh, ok" insight and design reminder to add 
an extra if
condition or whatever.  After all, look at what happen - an real bug 
occurred which
could only have occurred because someone lacked the insight or foresee 
it via
experience.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 20:20:48 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: SM <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

OK, I don't think it is productive to repeat these semantic difference 
of opinions of what section 4.1.1.10 says:

IMTO, no matter how you wish read this, NO QUIT means a cancellation 
because otherwise the only other way to end a SMTP session is for the 
client to close the socket connection. So what this would indirectly 
suggest the SMTP specs implicitly says:

    There is two ways for a client to end a SMTP session:
    QUIT or DROP THE LINE

and that does not jive with the written words:

    The sender MUST NOT intentionally close the transmission
    channel until it sends a QUIT command,

I can understood your position, very clearly, but in general, I don't 
think this is a good way to do robust client/server communications 
(allowing drops to signal message transaction completion). It opens 
the door for unsureness.

Keith Moore wrote:
> On Aug 16, 2011, at 7:48 PM, Hector Santos wrote:
> 
>> SM wrote:
>>> At 15:03 16-08-2011, Keith Moore wrote:
>>>> sorry, I misunderstood what was being suggested.   if the server gets DATA followed by a message (or a message fragment) followed by CRLF.CRLF, it should accept the message (or fragment) and deliver it.  no matter what else follows, or doesn't follow, during the same SMTP session .
>>> Yes.  The SMTP server mentioned that it is accepting the message (up to the end-of-data terminator.  Rejecting the message is not an option once the message has been accepted.
>> Hence the dilemma.  It wasn't an valid RFC5322 message and the issue is exasperated when it was DKIM signed and now recorded as a failure. Never mind the fact QUIT (or a new transaction command) is an SMTP requirement before the transaction is considered complete for delivery.
> 
> Where do you get that idea?  RFC 2821 doesn't require this.  It only requires that the server not close the connection until it sees a QUIT.  (It even says that clients SHOULD send a QUIT, implying that things should still work if they don't.)
> 
> Keith
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 16 Aug 2011 17:08:39 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Tuesday, August 16, 2011 4:48 PM
> To: SM
> Cc: [email-masked]
> Subject: Re: Mail Data termination
> 
> Never mind the fact QUIT (or a new transaction command) is an SMTP
> requirement before the transaction is considered complete for
> delivery.  Its in the specs, but really, how much liberalism in
> failure to comply to standards do we continue to tolerant?

That's not what QUIT means.  QUIT closes a session, not a transaction.  The end of DATA completes a transaction (Section 4.1.1.4 of RFC5321) and resets state.

The idea of not delivering a message because the client didn't send QUIT is a singular interpretation of SMTP as far as I can tell.

Subject: Re: Mail Data termination
From: Keith Moore <[email-masked]>
Date: Tue, 16 Aug 2011 19:58:08 -0400
Cc: Paul Smith <[email-masked]>, [email-masked]
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 7:53 PM, Hector Santos wrote:

> Keith Moore wrote:
>> On Aug 16, 2011, at 6:29 PM, Hector Santos wrote:
>>> Agree. I just wanted to show at least the one way found why/how it can happen, otherwise, with my tendency to give everyone the benefit of the doubt first, I would still be pulling my hair on this one. :)
>>> 
>>> There are some other insight to this, that IMV is worthy of implementation notes, but yes, I agree, the SMTP client MUST escape the dot lines if its not the true end of data marker.
>> This is not a new problem; I've occasionally seen clients that botched this ever since the mid-1980s.  (which is to say, ever since I first had anything to do with SMTP).
>> I'm not sure how implementation notes would help here.   If client implementors can't be bothered to read the spec, why would they read the implementation notes?
>> maybe we need to make the protocol more complicated so that they have to do so :)
> 
> Ok, I can see your position, therefore we should use this principle to reduce, eliminate all WG, IETF/IETF reviews, conflicts, strong passions that delay progress when in fact, no one really need to follow anything standard or put in the sweat to at least try and all software really just need to follow one principle:
> 
>    Be 100% liberal in what you receive, and don't worry about being
>    very conservative in what you send.

what are you smoking?

I'm only saying that all we can do is document how things should work.  If we've done that, and done it clearly,
adding more documentation probably won't help.

Keith

Subject: Re: Mail Data termination
From: Keith Moore <[email-masked]>
Date: Tue, 16 Aug 2011 19:57:17 -0400
Cc: SM <[email-masked]>, [email-masked]
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 7:48 PM, Hector Santos wrote:

> SM wrote:
>> At 15:03 16-08-2011, Keith Moore wrote:
>>> sorry, I misunderstood what was being suggested.   if the server gets DATA followed by a message (or a message fragment) followed by CRLF.CRLF, it should accept the message (or fragment) and deliver it.  no matter what else follows, or doesn't follow, during the same SMTP session .
>> Yes.  The SMTP server mentioned that it is accepting the message (up to the end-of-data terminator.  Rejecting the message is not an option once the message has been accepted.
> 
> Hence the dilemma.  It wasn't an valid RFC5322 message and the issue is exasperated when it was DKIM signed and now recorded as a failure. Never mind the fact QUIT (or a new transaction command) is an SMTP requirement before the transaction is considered complete for delivery.

Where do you get that idea?  RFC 2821 doesn't require this.  It only requires that the server not close the connection until it sees a QUIT.  (It even says that clients SHOULD send a QUIT, implying that things should still work if they don't.)

Keith

Date: Tue, 16 Aug 2011 19:53:14 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: Paul Smith <[email-masked]>, [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Keith Moore wrote:
> On Aug 16, 2011, at 6:29 PM, Hector Santos wrote:
> 
>> Agree. I just wanted to show at least the one way found why/how it can happen, otherwise, with my tendency to give everyone the benefit of the doubt first, I would still be pulling my hair on this one. :)
>>
>> There are some other insight to this, that IMV is worthy of implementation notes, but yes, I agree, the SMTP client MUST escape the dot lines if its not the true end of data marker.
> 
> This is not a new problem; I've occasionally seen clients that botched this ever since the mid-1980s.  (which is to say, ever since I first had anything to do with SMTP).
> 
> I'm not sure how implementation notes would help here.   If client implementors can't be bothered to read the spec, why would they read the implementation notes?
> 
> maybe we need to make the protocol more complicated so that they have to do so :)

Ok, I can see your position, therefore we should use this principle to 
reduce, eliminate all WG, IETF/IETF reviews, conflicts, strong 
passions that delay progress when in fact, no one really need to 
follow anything standard or put in the sweat to at least try and all 
software really just need to follow one principle:

     Be 100% liberal in what you receive, and don't worry about being
     very conservative in what you send.

:)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 19:48:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

SM wrote:
> 
> At 15:03 16-08-2011, Keith Moore wrote:
>> sorry, I misunderstood what was being suggested.   if the server gets 
>> DATA followed by a message (or a message fragment) followed by 
>> CRLF.CRLF, it should accept the message (or fragment) and deliver it.  
>> no matter what else follows, or doesn't follow, during the same SMTP 
>> session .
> 
> Yes.  The SMTP server mentioned that it is accepting the message (up to 
> the end-of-data terminator.  Rejecting the message is not an option once 
> the message has been accepted.

Hence the dilemma.  It wasn't an valid RFC5322 message and the issue 
is exasperated when it was DKIM signed and now recorded as a failure. 
Never mind the fact QUIT (or a new transaction command) is an SMTP 
requirement before the transaction is considered complete for 
delivery.  Its in the specs, but really, how much liberalism in 
failure to comply to standards do we continue to tolerant?

If you want to change this, change the spec.  I find it surreal that 
people can have such a deep seeded passion for argument and fighting 
for establishing standards and ideas to the point of delays in WG, 
IETF, IESG reviews, etc, yet there this odd prevailing mindset among 
the same people for tolerating failure.   Honestly, whats the point in 
throwing the RFC2919 book at people of its ok not follow any of this 
and all these design implementation pressures to get things right is 
all for nothing?

Crazy man. :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 19:09:27 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

Murray S. Kucherawy wrote:
>> -----Original Message-----
>>
>> sorry, I misunderstood what was being suggested.   if the server gets
>> DATA followed by a message (or a message fragment) followed by
>> CRLF.CRLF, it should accept the message (or fragment) and deliver it.
>> no matter what else follows, or doesn't follow, during the same SMTP
>> session .
> 
> +1.  Bonus points for sending forensic information upstream, or to the MLM developer, so whoever it is can fix the problem.
> 
> It's possible I also got the same message fragments, but I'd need to know which ones they were to go check.

Although it may not be a big deal, to respect those involved, I will 
send the info to you off-list.  You will be able to see the original 
submission, and check your receiver for the transformed distribution 
to see how it handled it.  For me, I never got the message (was not 
delivered) but when looking at the meta logging of the DKIM 
verification AUTH-RES, I saw this IETF message and then the archive 
message against the truncated and saved canceled message.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 18:49:24 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Keith Moore wrote:

>> The problem is that a 250 was already issued once the <CRLF>.<CRLF> is detected 
>> so the only current possible SMTP "allowed" action is to not deliver (complete 
>> the transaction) due to a NO QUIT command cancellation which is what happened 
>> in our server setup.  I never saw these IETF messages.
> 
> no QUIT is a lousy reason to not deliver a message.  

I agree with both sides of this argument for and against.  I don't 
wish to spend too much time justifying the pros and cons, but the fact 
is, it is a SMTP requirement and for good reasons too, yet, there are 
reasons when its less optimal.

Our smtp server, originally never supported it for multi-threaded and 
RPC related design reasons. Onnce threaded receiver issued a 250 
message acceptance response, it immediately signaled the router 
threads to deal with it.  No waiting for a QUIT or a new transaction 
command (MAIL FROM).

But in the name of conforming with RFC2821,  back in early 2000, we 
added a bunch a stuff related to RFC2821 compliant including the NO 
QUIT SMTP requirement as a default behavior knowing full well there 
could be new surprises here.

For one thing, it slowed down the instant signaling and delivery by 
holding it longer, not something that tickled  me.  But its in the 
specs, an as long a fall back was available for customers affected by 
it, I was fine with it.

Today, I think it has helped reduce problems and I rarely hear of any 
reported problems and if when it did, simple switch flipping resolves 
it.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Subject: Re: Mail Data termination
From: Keith Moore <[email-masked]>
Date: Tue, 16 Aug 2011 18:48:33 -0400
Cc: Paul Smith <[email-masked]>, [email-masked]
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 6:29 PM, Hector Santos wrote:

> Agree. I just wanted to show at least the one way found why/how it can happen, otherwise, with my tendency to give everyone the benefit of the doubt first, I would still be pulling my hair on this one. :)
> 
> There are some other insight to this, that IMV is worthy of implementation notes, but yes, I agree, the SMTP client MUST escape the dot lines if its not the true end of data marker.

This is not a new problem; I've occasionally seen clients that botched this ever since the mid-1980s.  (which is to say, ever since I first had anything to do with SMTP).

I'm not sure how implementation notes would help here.   If client implementors can't be bothered to read the spec, why would they read the implementation notes?

maybe we need to make the protocol more complicated so that they have to do so :)

Keith

Date: Tue, 16 Aug 2011 15:42:58 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

At 15:03 16-08-2011, Keith Moore wrote:
>sorry, I misunderstood what was being suggested.   if the server 
>gets DATA followed by a message (or a message fragment) followed by 
>CRLF.CRLF, it should accept the message (or fragment) and deliver 
>it.  no matter what else follows, or doesn't follow, during the same 
>SMTP session .

Yes.  The SMTP server mentioned that it is accepting the message (up 
to the end-of-data terminator.  Rejecting the message is not an 
option once the message has been accepted.

Regards,
-sm 

Date: Tue, 16 Aug 2011 18:29:01 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Paul Smith <[email-masked]>
CC: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Paul Smith wrote:

>> Is this something that is addressable?  Something the QP RFC needs to 
>> make a note off if not already there?
> 
> This is a broken sender, pure and simple. I think all you can do is what 
> you've been doing. Hopefully the sender will be fixed soon.
> 
> QP is irrelevant. The mail sender should be dot padding as the message 
> is sent. The encoding should make no difference.

Agree. I just wanted to show at least the one way found why/how it can 
happen, otherwise, with my tendency to give everyone the benefit of 
the doubt first, I would still be pulling my hair on this one. :)

There are some other insight to this, that IMV is worthy of 
implementation notes, but yes, I agree, the SMTP client MUST escape 
the dot lines if its not the true end of data marker.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 15:21:17 -0700
To: Ned Freed <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: Mail Data termination
CC: <[email-masked]>
Sender: [email-masked]

At 2:25 PM -0700 8/16/11, Ned Freed wrote:

>>  This is something that is so broken I think the best course is to
>>  reject it.  Hopefully the sender will notice and fix their bug.
>
>  Not sure rejecting is such a great idea...

Right -- I didn't actually mean "reject" but rather "not try and 
correct for".  There will be a valid (partial) message and a slew of 
invalid commands.

>  Another way to handle this on the receiving end is not to peek but 
> to do a hard
>  disconnect after some small number of bad commands are seen.

This seems like a reasonable approach.

>  The result will be a series of duplicate partial
>  messages for the receiver and a series of transmission failures for 
> the sender.
>  The duplicates will no doubt be annoying, but this has the advantage that it
>  doesn't cause damage when this particular set of symptoms are the result of a
>  different underlying problem.

And, hopefully, someone will notice the repeated failures and/or the 
repeated partial messages and fix the bug.

>  Meh. I think this may be one where trying to compensate for this on 
> the server end is a poor substitute for tracking down whoever is 
> doing this,
>  contacting them and urging them to fix the problem.

Agreed.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
razbliuto (ros-blee-OO-toe; Russian; noun): the feeling a person
retains for someone he or she once loved.

From: [email-masked]
Cc: Keith Moore <[email-masked]>, Randall Gellens <[email-masked]>, Hector Santos <[email-masked]>, [email-masked]
Date: Tue, 16 Aug 2011 15:19:57 -0700 (PDT)
Subject: Re: Mail Data termination
To: Keith Moore <[email-masked]>
Sender: [email-masked]

> On Aug 16, 2011, at 2:56 PM, Keith Moore wrote:

> > On Aug 16, 2011, at 2:46 PM, Randall Gellens wrote:
> >
> >> This is something that is so broken I think the best course is to reject it.
> >
> > +1

> sorry, I misunderstood what was being suggested.   if the server gets DATA
> followed by a message (or a message fragment) followed by CRLF.CRLF, it should
> accept the message (or fragment) and deliver it.  no matter what else follows,
> or doesn't follow, during the same SMTP session .

+1

				Ned

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 16 Aug 2011 15:09:13 -0700
Subject: RE: Mail Data termination
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Keith Moore
> Sent: Tuesday, August 16, 2011 3:04 PM
> To: Keith Moore
> Cc: Randall Gellens; Hector Santos; [email-masked]
> Subject: Re: Mail Data termination
> 
> sorry, I misunderstood what was being suggested.   if the server gets
> DATA followed by a message (or a message fragment) followed by
> CRLF.CRLF, it should accept the message (or fragment) and deliver it.
> no matter what else follows, or doesn't follow, during the same SMTP
> session .

+1.  Bonus points for sending forensic information upstream, or to the MLM developer, so whoever it is can fix the problem.

It's possible I also got the same message fragments, but I'd need to know which ones they were to go check.

Subject: Re: Mail Data termination
From: Keith Moore <[email-masked]>
Date: Tue, 16 Aug 2011 18:03:46 -0400
Cc: Randall Gellens <[email-masked]>, Hector Santos <[email-masked]>, <[email-masked]>
To: Keith Moore <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 2:56 PM, Keith Moore wrote:

> On Aug 16, 2011, at 2:46 PM, Randall Gellens wrote:
> 
>> This is something that is so broken I think the best course is to reject it. 
> 
> +1

sorry, I misunderstood what was being suggested.   if the server gets DATA followed by a message (or a message fragment) followed by CRLF.CRLF, it should accept the message (or fragment) and deliver it.  no matter what else follows, or doesn't follow, during the same SMTP session .

Subject: Re: Mail Data termination
From: Keith Moore <[email-masked]>
Date: Tue, 16 Aug 2011 18:02:27 -0400
Cc: [email-masked]
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 4:32 PM, Hector Santos wrote:

> 
> Randall Gellens wrote:
>> At 1:07 PM -0400 8/16/11, Hector Santos wrote:
>>> I've been seeing (maybe because I'm looking) more transactions from one of more particular senders that include a <CRLF>.<CRLF> end of data terminator in the body but continue with additional text after the DATA has been accepted.
>>> 
>>> Investigating it seems to be something we can't reliably address short of looking ahead of the receiver buffers to nullify these invalid <CRLF>.<CRLF>.
>> This is something that is so broken I think the best course is to reject it.  Hopefully the sender will notice and fix their bug.
> 
> Agree.
> 
> The problem is that a 250 was already issued once the <CRLF>.<CRLF> is detected so the only current possible SMTP "allowed" action is to not deliver (complete the transaction) due to a NO QUIT command cancellation which is what happened in our server setup.  I never saw these IETF messages.

no QUIT is a lousy reason to not deliver a message.  

Keith

Date: Tue, 16 Aug 2011 23:56:54 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

I've seen this twice. First, when SCO misconfigured sendmail and shipped 
it to some customers around 1998, then, when some spammer fubared his 
SMTP client sometime between 2003 and 2008 and misspammed his victims.

IMO so rare that these clients may safely be ignored.

Arnt

From: [email-masked]
Cc: [email-masked]
Date: Tue, 16 Aug 2011 14:25:39 -0700 (PDT)
Subject: Re: Mail Data termination
To: Randall Gellens <[email-masked]>
Sender: [email-masked]

> At 1:07 PM -0400 8/16/11, Hector Santos wrote:

> >  I've been seeing (maybe because I'm looking) more transactions from
> > one of more particular senders that include a <CRLF>.<CRLF> end of
> > data terminator in the body but continue with additional text after
> > the DATA has been accepted.
> >
> >  Investigating it seems to be something we can't reliably address
> > short of looking ahead of the receiver buffers to nullify these
> > invalid <CRLF>.<CRLF>.

> This is something that is so broken I think the best course is to
> reject it.  Hopefully the sender will notice and fix their bug.

Not sure rejecting is such a great idea... The problem is that unless the
on-the-fly encoding or whatever is causing a <CRLF>.<CRLF> in the output stream
that isn't a terminator is also doing something else that's invalid and which
can be checked, the mail transaction is legitimate and complete at the point in
the dialog where that terminator was received. So in order to detect that there
is a problem, the receiver has to peek at the  input stream and see if there's
anything waiting beyond the transaction. And if there is it could just as
easily be a problem like, say, the sender not implementing pipelining properly.

The peek may also not be reliable - it's entirely possible that the sender
sends out one line at a time. (Yes, that's inefficient, but considering
the brokenness here would you be surprised if the client is also inefficient?)

It's even possible for some problem with a subsequent message in the stream
to present as invalid command data having nothing to do with the previous,
correct message.

Another way to handle this on the receiving end is not to peek but to do a hard
disconnect after some small number of bad commands are seen. This will likely
have the effect of causing the transaction to fail (from the sender's
perspective - but of course it will have succeeded with a partial message from
the receiver's perspective). The result will be a series of duplicate partial
messages for the receiver and a series of transmission failures for the sender.
The duplicates will no doubt be annoying, but this has the advantage that it
doesn't cause damage when this particular set of symptoms are the result of a
different underlying problem.

The required buffer peek may also be an issue. Servers that support pipelining
may already have code in place to do this (so command replies can be batched
up), in which case it may pretty trivial to implement. But if not ... fiddling
with your buffer management has it's own set of risks.

Meh. I think this may be one where trying to compensate for this on the 
server end is a poor substitute for tracking down whoever is doing this,
contacting them and urging them to fix the problem.

				Ned

Date: Tue, 16 Aug 2011 16:32:49 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Randall Gellens wrote:
> At 1:07 PM -0400 8/16/11, Hector Santos wrote:
> 
>>  I've been seeing (maybe because I'm looking) more transactions from 
>> one of more particular senders that include a <CRLF>.<CRLF> end of 
>> data terminator in the body but continue with additional text after 
>> the DATA has been accepted.
>>
>>  Investigating it seems to be something we can't reliably address 
>> short of looking ahead of the receiver buffers to nullify these 
>> invalid <CRLF>.<CRLF>.
> 
> This is something that is so broken I think the best course is to reject 
> it.  Hopefully the sender will notice and fix their bug.

Agree.

The problem is that a 250 was already issued once the <CRLF>.<CRLF> is 
detected so the only current possible SMTP "allowed" action is to not 
deliver (complete the transaction) due to a NO QUIT command 
cancellation which is what happened in our server setup.  I never saw 
these IETF messages.

Looking further for other similar senders (within the past 3-4 days 
logs) with this problem, I saw mainly two sender sources and each seem 
to have two important different modes:

- A sender that only needs the 250 server response for message 
completion, it will not  try to resend even if the server dropped the 
session with all the 500 responses, and

- A sender that sees 250, gets dropped due to all the 500 responses 
and it will retry with the same repeated NO QUIT message canceled result.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 21:08:49 +0200
From: Paul Smith <[email-masked]>
To: Hector Santos <[email-masked]>
CC: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

On 16/08/2011 19:07, Hector Santos wrote:
>
> I've been seeing (maybe because I'm looking) more transactions from 
> one of more particular senders that include a <CRLF>.<CRLF> end of 
> data terminator in the body but continue with additional text after 
> the DATA has been accepted.
>
> Investigating it seems to be something we can't reliably address short 
> of looking ahead of the receiver buffers to nullify these invalid 
> <CRLF>.<CRLF>.
>
> In once such message, it originated as a IETF mailing list message 
> submission and looking at the archive, the message contains a series 
> of super long single line non-wrapper paragraphs but it appeared 
> during the list server distribution, it was transmitting a QP 
> conversion.   Comparing the original archived message which what our 
> receiver was getting, prematurely getting a <CRLF>.<CRLF>, accepting 
> the message and the all the remaining data ending up as 500 command 
> unknowns, I can see what happen.
>
> In the last line our receiver got, the archived had the long unwrapped 
> paragraph with an ending like so:
>
>    blah blah, when normative language ought to have been used.<CR><LF>
>
> and during the QP conversion, it chopped it up into shorter lines:
>
>    blah blah, when normative language ought to have been used=<CR><LF>
>    .<CR><LF>
>
> So it cut it off at the sentence ending period and put it on its own 
> line and that is where the receiver saw the DATA terminator sequence 
> <CRLF>.<CRLF>.
>
> Is this something that is addressable?  Something the QP RFC needs to 
> make a note off if not already there?

This is a broken sender, pure and simple. I think all you can do is what 
you've been doing. Hopefully the sender will be fixed soon.

QP is irrelevant. The mail sender should be dot padding as the message 
is sent. The encoding should make no difference.

Subject: Re: Mail Data termination
From: Keith Moore <[email-masked]>
Date: Tue, 16 Aug 2011 14:56:09 -0400
Cc: Hector Santos <[email-masked]>, <[email-masked]>
To: Randall Gellens <[email-masked]>
Sender: [email-masked]

On Aug 16, 2011, at 2:46 PM, Randall Gellens wrote:

> This is something that is so broken I think the best course is to reject it. 

+1

Date: Tue, 16 Aug 2011 11:46:53 -0700
To: Hector Santos <[email-masked]>, <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: Mail Data termination
Sender: [email-masked]

At 1:07 PM -0400 8/16/11, Hector Santos wrote:

>  I've been seeing (maybe because I'm looking) more transactions from 
> one of more particular senders that include a <CRLF>.<CRLF> end of 
> data terminator in the body but continue with additional text after 
> the DATA has been accepted.
>
>  Investigating it seems to be something we can't reliably address 
> short of looking ahead of the receiver buffers to nullify these 
> invalid <CRLF>.<CRLF>.

This is something that is so broken I think the best course is to 
reject it.  Hopefully the sender will notice and fix their bug.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
Power corrupts; Powerpoint corrupts absolutely.
            --Vint Cert

Date: Tue, 16 Aug 2011 14:24:02 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Mail Data termination
Sender: [email-masked]

Hector Santos wrote:
> 
> In the last line our receiver got, the archived [message] had the long 
> unwrapped  paragraph with an ending like so:
> 
>    blah blah, when normative language ought to have been used.<CR><LF>
> 
> and during the QP conversion, it chopped it up into shorter lines:
> 
>    blah blah, when normative language ought to have been used=<CR><LF>
>    .<CR><LF>
> 
> So it cut it off at the sentence ending period and put it on its own 
> line and that is where the receiver saw the DATA terminator sequence 
> <CRLF>.<CRLF>.
> 
> Is this something that is addressable?  Something the QP RFC needs to 
> make a note off if not already there?

One other question I forgot to ask as a possible solution is whether 
there is some receiver "expected" responsibility during the DATA state 
to transparently undo the QP conversion line by line so that it can 
detect lines like the above and see its actually one line and not a 
<CR><LF>.<CR><LF> sequence?

That would sound to me to be unreasonable to expect.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 16 Aug 2011 13:07:41 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Mail Data termination
Sender: [email-masked]

I've been seeing (maybe because I'm looking) more transactions from 
one of more particular senders that include a <CRLF>.<CRLF> end of 
data terminator in the body but continue with additional text after 
the DATA has been accepted.

Investigating it seems to be something we can't reliably address short 
of looking ahead of the receiver buffers to nullify these invalid 
<CRLF>.<CRLF>.

In once such message, it originated as a IETF mailing list message 
submission and looking at the archive, the message contains a series 
of super long single line non-wrapper paragraphs but it appeared 
during the list server distribution, it was transmitting a QP 
conversion.   Comparing the original archived message which what our 
receiver was getting, prematurely getting a <CRLF>.<CRLF>, accepting 
the message and the all the remaining data ending up as 500 command 
unknowns, I can see what happen.

In the last line our receiver got, the archived had the long unwrapped 
paragraph with an ending like so:

    blah blah, when normative language ought to have been used.<CR><LF>

and during the QP conversion, it chopped it up into shorter lines:

    blah blah, when normative language ought to have been used=<CR><LF>
    .<CR><LF>

So it cut it off at the sentence ending period and put it on its own 
line and that is where the receiver saw the DATA terminator sequence 
<CRLF>.<CRLF>.

Is this something that is addressable?  Something the QP RFC needs to 
make a note off if not already there?

When I first was seeing these I chalked it up as a stupid sender just 
blasting commands and data ignoring server responses as it it was 
behaving in pipelining mode.

But now with DKIM logs showing failure, it was highlighted with me 
personally when I saw the IETF message and going back a few days, 
there was several of these.

The good news I think, it was the only with submissions by the same 
person but overall it seems to be a QP conversion issue by the list 
server.

I am posting wondering is there could be something more that can be 
done, like maybe looking at incoming circular buffers to address this 
in a reliable fashion.   Sounds like that is more trouble than it is 
worth in trying to save what appears to be a broken behavior by the 
list server QP conversion process.

Any thoughts, comments?

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 11 Aug 2011 12:58:55 +0100
From: Alexey Melnikov <[email-masked]>
To: Bill McQuillan <[email-masked]>
CC: SMTP Discussion <[email-masked]>, ken carlberg <[email-masked]>
Subject: Re: Comments on: draft-melnikov-smtp-priority-02
Sender: [email-masked]

Hi Bill,
Sorry for the slow response.

Bill McQuillan wrote:

>On Fri, 2011-07-22, Alexey Melnikov wrote:
>  
>
>>ken carlberg wrote:
>>    
>>
>>>On Jul 13, 2011, at 3:03 AM, Bill McQuillan wrote:
>>>  
>>>      
>>>
>>>>>A value of 0 indicates an email from a client/network not
>>>>>supporting priorities or intended to be sent to such a server/
>>>>>network; this is the same as not specifying the PRIORITY
>>>>>parameter.
>>>>>      
>>>>>          
>>>>>
>>>>The possibility that I might want to send a priority 0 (normal)
>>>>message FROM a conforming system TO a conforming system seems not
>>>>to be considered.
>>>>    
>>>>        
>>>>
>>>ok.  It will be easy to add text and stating up front that a value of zero, by default, indicates "normal".  And then add the existing caveats.  And by "normal", I assume that we are in agreement that this describes the best effort service model that exists today as if the proposed extension never existed.
>>>  
>>>      
>>>
>>I've tried to clarify this by removing "or intended to be sent", as this
>>is not relevant here. My current text reads:
>>    
>>
>>      A value of 0 indicates an email from a client/network not
>>      supporting priorities, which is the same as not specifying the
>>PRIORITY parameter,
>>      i.e. this value will cause the standard SMTP behaviour in absence
>>of this extension.
>>    
>>
>>Is this better?
>>    
>>
>
>How about:
>
>A value of 0 indicates an email which is to be handled at the
>                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>normal priority level or is from a client/network not supporting
>^^^^^^^^^^^^^^^^^^^^^^^^^^^
>priorities, which is the same as not specifying the PRIORITY
>parameter, i.e. this value will cause the standard SMTP behaviour
>in absence of this extension.
>
To me "which is to be handled at the normal priority level" is the same 
as "not specifying the PRIORITY parameter", that is why I still prefer 
my version quoted above.