Subject: RE: RSET command - possible security loophole
From: [email-masked]
To: "Murray S. Kucherawy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Date: Tue, 31 May 2011 21:05:03 -0700 (PDT)
List-Id: <ietf-smtp.imc.org>

> > -----Original Message-----
> > From: Ned Freed [mailto:[email-masked]]
> > Sent: Tuesday, May 31, 2011 3:29 PM
> > To: Murray S. Kucherawy
> > Cc: [email-masked]
> > Subject: RE: RSET command - possible security loophole
> >
> > Authentication (AUTH) and session security (STARTTLS) are the obvious examples,
> > but it is also very common for implementations to support proprietary
> > mechanisms to allow setting of various session attributes (including but not
> > limited to the real IP address of the client). Various sorts of proxies are the
> > usual use-case for this stuff.

> Sorry, right, AUTH and STARTTLS of course.  My point is that once you hit
> MAIL and thus begin message-specific operations rather than
> connection-specific ones, I can't think of any SMTP commands that go back
> and potentially alter session state in any obvious way that I can think of.

Goodness, I certainly hope not. We discussed the possibility of allowing
someone to authenticate back when SMTP AUTH was first being specified. There
were a couple of people who wanted to allow this sort of thing:

    C: MAIL FROM:<whoever@wherever>
    S: 250 Go ahead, you unauthenticated thing you
    C: RCPT TO:<highmucketymuck@wherever>
    S: 559 Authorization failure, need to authenatice to send to this address
    C: AUTH blah blah blah
    S: 250 OK, you're authorized now
    C: RCPT TO:<highmucketymuck@wherever>
    S: 250 OK, you're allowed to do that now

I protested vehemently that the resulting security semantics would be a HUGE
mess. Just as one example, supppose the AUTH would have impacted the semantics
of the previous part of the transaction in some way, e.g., a recipient address
that looks successful but got silently blackholed because of the
lack of authorization.

No, the only way for this not to degenerate into madness is to disallow
session-changing stuff during a mail transaction.

> What am I missing?

Stated this way, nothing.

				Ned

From: [email-masked]
Cc: [email-masked], [email-masked]
Date: Tue, 31 May 2011 20:57:14 -0700 (PDT)
Subject: Re: UNAUTH
To: John Levine <[email-masked]>
Sender: [email-masked]

> > The place it might have value would be high traffic webmail
> > servers, where a single application serves a large number of
> > users. The ability to keep a connection pool open might be
> > helpful. TCP setup and teardown isn't that expensive compared
> > to authentication, but TLS setup might be.

> Seems to me that RFC 4954 provides the the AUTH=mailbox parameter
> on MAIL FROM for just this scenario.

AUTH= is the correct mechanism if there's an overarching authorizing
relationship, that is, the server supports some sort of authentication that
authorizes the client to use some set of AUTH= values.

But the AUTH= mechanism isn't useful for ad-hoc send-on-behalf-of systems. The
necessary relationship to make it work isn't there.

Another issue with AUTH is that the specification really doesn't nail down the
semantics sufficiently to insure interoperability, but that's really a separate
issue for another day.

				Ned

From: [email-masked]
Cc: SMTP Interest Group <[email-masked]>
Date: Tue, 31 May 2011 20:53:29 -0700 (PDT)
Subject: Re: UNAUTH
To: Steve Atkins <[email-masked]>
Sender: [email-masked]

> On May 31, 2011, at 6:12 PM, John Levine wrote:

> >
> >> I think defining an
> >> UNAUTHENTICATE/LOGOUT/whatever-you-want-to-call-it extension would be
> >> a great idea, but it's definitely a separate effort from any of this
> >
> > I understand what it would do, but I'm not sure I understand what
> > problem it solves.  If you want to drop your auth credentials, you
> > can always quit and call back.  Yes, that's slower, but does it
> > happen often enough to merit asking people to add yet another hack
> > to their mail servers?

> The place it might have value would be high traffic webmail
> servers, where a single application serves a large number of
> users. The ability to keep a connection pool open might be
> helpful. TCP setup and teardown isn't that expensive compared
> to authentication, but TLS setup might be.

The difficult in getting a working connection can also be a factor.

> I'm not convinced it'd be useful, but it's not obviously useless.

To be fair, it's only useful if some of the larger providers implement it.
Statistically speaking the smaller providers don't come up often enough for
it to be worth it.

				Ned

VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org
Date: 31 May 2011 23:55:26 -0400
From: "John R Levine" <[email-masked]>
To: "Ned Freed" <[email-masked]>
Cc: "IETF SMTP list" <[email-masked]>
Subject: Re: UNAUTH
Cleverness: None detected
Sender: [email-masked]

> Send on behalf schemes are the main target audience here. Given the
> concentration of email recipients on over-the-top providers and the larger
> ISPs, being able to unauthenticate and reauthenticate without having to get a
> new connection to some very busy systems going can result in significant
> savings in latency.

Oh, OK.  Why wouldn't you use the AUTH= parameter on MAIL FROM?

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

From: [email-masked]
Cc: [email-masked], [email-masked]
Date: Tue, 31 May 2011 20:50:33 -0700 (PDT)
Subject: Re: UNAUTH
To: John Levine <[email-masked]>
Sender: [email-masked]

> >I think defining an
> >UNAUTHENTICATE/LOGOUT/whatever-you-want-to-call-it extension would be
> >a great idea, but it's definitely a separate effort from any of this

> I understand what it would do, but I'm not sure I understand what
> problem it solves.

Send on behalf schemes are the main target audience here. Given the
concentration of email recipients on over-the-top providers and the larger
ISPs, being able to unauthenticate and reauthenticate without having to get a
new connection to some very busy systems going can result in significant
savings in latency.

> If you want to drop your auth credentials, you
> can always quit and call back.  Yes, that's slower, but does it
> happen often enough to merit asking people to add yet another hack
> to their mail servers?

On some systems yes, it does.

				Ned

Subject: Re: UNAUTH
From: Steve Atkins <[email-masked]>
Date: Tue, 31 May 2011 19:01:09 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On May 31, 2011, at 6:56 PM, John Levine wrote:

>> The place it might have value would be high traffic webmail
>> servers, where a single application serves a large number of
>> users. The ability to keep a connection pool open might be
>> helpful. TCP setup and teardown isn't that expensive compared
>> to authentication, but TLS setup might be.
> 
> Seems to me that RFC 4954 provides the the AUTH=mailbox parameter
> on MAIL FROM for just this scenario.

So it does. Not so much authentication as an assertion that
the client has already authenticated the user - which is going to
be at least as good, if not better, for the vast majority of webmail
setups.

Cheers,
  Steve

Date: 1 Jun 2011 01:56:21 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: UNAUTH
Organization: 
Sender: [email-masked]

>The place it might have value would be high traffic webmail
>servers, where a single application serves a large number of
>users. The ability to keep a connection pool open might be
>helpful. TCP setup and teardown isn't that expensive compared
>to authentication, but TLS setup might be.

Seems to me that RFC 4954 provides the the AUTH=mailbox parameter
on MAIL FROM for just this scenario.

R's,
John

Date: Tue, 31 May 2011 21:43:03 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "Murray S. Kucherawy" <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: Ned Freed [mailto:[email-masked]]
>> Sent: Tuesday, May 31, 2011 3:29 PM
>> To: Murray S. Kucherawy
>> Cc: [email-masked]
>> Subject: RE: RSET command - possible security loophole
>>
>> Authentication (AUTH) and session security (STARTTLS) are the obvious examples,
>> but it is also very common for implementations to support proprietary
>> mechanisms to allow setting of various session attributes (including but not
>> limited to the real IP address of the client). Various sorts of proxies are the
>> usual use-case for this stuff.
> 
> Sorry, right, AUTH and STARTTLS of course.  My point is that once you hit MAIL 
> and thus begin message-specific operations rather than connection-specific ones, 
> I can't think of any SMTP commands that go back and potentially alter session 
> state in any obvious way that I can think of.
> 
> What am I missing?

Well, define "message-specific" because MAIL just gives you the 
envelope return path address and we already know that 5321.Mail may 
not be same as the 5322.From.

"message-specific" can just be the RFC5322 payload.  Maybe you are 
pointing out that among the three commands:

    MAIL
    RCPT
    DATA

for a given transaction, we already make persistent reply code 
decisions for the same MAIL and RCPT values repeated in all same 
session transactions before even getting to the DATA state.

Consider this timing sequence

   T1   C: MAIL FROM:<FOO>
        ** CALL GREYLIST(IP) result: FAIL=450
   T2   S: 450 Greylist Rejected, try again later
   T3   C: MAIL FROM:<FOO>
        ** CALL GREYLIST(IP) result: PASS
   T4   S: 250 OK

where T3 = T2 + 4 mins and the client waits there for 4 mins assuming 
that might be enough of a block time without the the 5 min standard 
IDLE timeout. But then again, he can issue a NOOP every minute to 
reset the server idle timeout counter, keep the session alive and wait 
10 minutes!

I think its "valid" from the SMTP technical standpoint - he did wait 
enough to satisfy the block time. Normally, they disconnect and try 
later. But it would be sort of dumb to waste resources to wait.

So IMO, the issue is more about trying other transactions after DATA 
and how the server, in general, can be informed that new SMTP 
transactions would be disabled.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Subject: Re: UNAUTH
From: Steve Atkins <[email-masked]>
Date: Tue, 31 May 2011 18:21:34 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On May 31, 2011, at 6:12 PM, John Levine wrote:

> 
>> I think defining an
>> UNAUTHENTICATE/LOGOUT/whatever-you-want-to-call-it extension would be
>> a great idea, but it's definitely a separate effort from any of this
> 
> I understand what it would do, but I'm not sure I understand what
> problem it solves.  If you want to drop your auth credentials, you
> can always quit and call back.  Yes, that's slower, but does it
> happen often enough to merit asking people to add yet another hack
> to their mail servers?

The place it might have value would be high traffic webmail
servers, where a single application serves a large number of
users. The ability to keep a connection pool open might be
helpful. TCP setup and teardown isn't that expensive compared
to authentication, but TLS setup might be.

I'm not convinced it'd be useful, but it's not obviously useless.

Cheers,
  Steve

Date: 1 Jun 2011 01:12:42 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: UNAUTH
Organization: 
Sender: [email-masked]

>I think defining an
>UNAUTHENTICATE/LOGOUT/whatever-you-want-to-call-it extension would be
>a great idea, but it's definitely a separate effort from any of this

I understand what it would do, but I'm not sure I understand what
problem it solves.  If you want to drop your auth credentials, you
can always quit and call back.  Yes, that's slower, but does it
happen often enough to merit asking people to add yet another hack
to their mail servers?

R's,
John

Date: Tue, 31 May 2011 21:06:00 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "Murray S. Kucherawy" <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Murray S. Kucherawy wrote:

>> The place where "backwards" comes in is the place where I think
>> Hector has uncovered a problem, even though it is not the
>> problem his initial note was about.   I think that, as we are
>> checking extensions that  set data that needs to be carried from
>> one command to others, we should be checking to be sure that
>> action is either explicitly bound to either session or
>> transaction state or that new states are clearly defined.
> 
> Given that the model of SMTP in general involves some connection-specific
> stuff (accept(), 220, EHLO, 250) followed by zero or more message-specific 
> actions (MAIL, RCPT, DATA, RSET) and some things that have to do with 
> neither (RSET, NOOP, HELP, VRFY, EXPN), I'm at a loss to understand how 
> there's some blending of semantics here.

Other than mechanical errors and state to state flow allowance and 
restrictions, it is not SMTP that selects more "abstract" reply codes. 
These are based on local system implementation of policies. They 
define or direct what negative or positive reply codes are issued in 
order to drive the client/server state machine.

In general, each "mail transaction" (MAIL, RCPT, DATA) under the same 
section are viewed independent of each other.  It could be a batching 
idea where multiple messages for the same target host are queued and 
you deal with them with one or more outbound attempts

But putting aside AUTH or IP based authentication for the moment, 
because I strongly believe most systems bypass SMTP level filtering or 
restrictions when we have an authenticated system, this is more about 
the unauthenticated, unknown, unsolicited messages coming in where 
these "filters" come into play, like SPF, DNSRBL and GreyListing.  For 
our system, this overhead is skipped for authenticated sections.

So it comes down to something like this:

    S: 220 localhost.com ESMTP ready
    C: EHLO mail.example.com
    S: 250 localhost.com, Hello mail.example.com
    C: MAIL FROM:<[email-masked]>
    S: 250 OK
    C: RCPT TO:<[email-masked]>
    S: 250 OK
    C: DATA
    S: 354 OK
    ** data transferred **
    S: 451 Greylisted policy rejection (transactions disabled)
    C: RSET
    S: 250 ok, but not really
    C: MAIL FROM:<[email-masked]>
    S: 450 Sessions Transactions Are Disabled
    C: EHLO mail.example.com
    S: 550 Sessions Transactions Are Disabled
    C: QUIT
    S: 221 closing connection <CLICK>

The question/issue/idea is whether there should be persistent "Global 
Session" condition that is now applied to the channel or current 
session for any new transaction attempts.  So assuming this is 
practically reasonable, for the current commands that can begin the 
process of starting a new transaction:

RSET

    5321 does not allowed for negative reply codes for RSET so it has to
    be a 250.

MAIL FROM:

    This might be debatable, but IMV, it should be a temporary rejection
    and/or follow the DATA reply code.   The main idea I think is we
    don't want to tell the client his MAIL FROM address is not 
acceptable,
    unless it really is.

EHLO

    5321 allows for a 550 and I think it would be appropriate here to
    use a permanent rejection code.

> Are there properties of the session state, other than implementation-specific
> ones, that might ever be altered by commands after EHLO?  I can't think of 
> any.  

As noted above, SMTP does not make these reply code decisions for 
anyone. It only provide the mechanism for the state machine.  It is 
always local policy based with implementations helping to operators to 
leverage how a state machine will flow.

> Past EHLO, everything is either neutral or is a message-specific 
> instruction of some kind.  If that's the case, then declaring that RSET 
> only ever resets message state seems like where the RFC should leave it.

In my code for RSET, and I took a look at Sendmail and Exim, they are 
just clearing, for the most part, MAIL, RCPT and DATA information. And 
they all use a single function that RSET, MAIL and EHLO calls.

So in reality, RSET/MAIL/EHLO would do the same thing. The main point 
here is whether new transactions MAY be disabled for the same session 
based on a condition occurring from an previous transaction, and 
really, I think its DATA because we already do it with MAIL and RCPT. 
  Repeating the same MAIL and RCPT that was rejected once already is 
expected to be rejected again.  So if they can't get pass MAIL or 
RCPT, they will never get to DATA anyway.  So this is more above the 
final result with completing DATA.

Anyway, my opinion.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 31 May 2011 16:24:50 -0700
Subject: RE: RSET command - possible security loophole
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: Ned Freed [mailto:[email-masked]]
> Sent: Tuesday, May 31, 2011 3:29 PM
> To: Murray S. Kucherawy
> Cc: [email-masked]
> Subject: RE: RSET command - possible security loophole
> 
> Authentication (AUTH) and session security (STARTTLS) are the obvious examples,
> but it is also very common for implementations to support proprietary
> mechanisms to allow setting of various session attributes (including but not
> limited to the real IP address of the client). Various sorts of proxies are the
> usual use-case for this stuff.

Sorry, right, AUTH and STARTTLS of course.  My point is that once you hit MAIL and thus begin message-specific operations rather than connection-specific ones, I can't think of any SMTP commands that go back and potentially alter session state in any obvious way that I can think of.

What am I missing?

Date: Tue, 31 May 2011 18:42:38 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

SM wrote:

> Hi Hector,
> At 12:58 31-05-2011, Hector Santos wrote:
>> In regards to authentication, in our code, for IP based (Relay Tables, 
>> POPB4SMTP), it can not change the authentication based on IP.  For 
>> ESMTP AUTH, once the session authenticated, per RFC2554, section 4, it 
>> will not allow another AUTH to be issued with a 503 response.
> 
> Is your implementation compliant with RFC 2554 or RFC 4954?

Our server is compliant with RFC2554 and related AUTH docs in regards 
with RFC3207 (TLS) and RFC4409 (SUBMIT), i.e. important sequence to 
follow.

I don't recall using the update RFC4954 nor searching "4954" in the 
source code show any specific references which is normally how we 
annotate RFC update related protocol logic changes.

A quick perusal seems to show it combined many of the related docs. 
i.e. TLS, SUBMIT.

It seems the same restriction in 2554 was carried over to 4954 in 
section 4:

    Restrictions:
        After an AUTH command has been successfully completed, no more
        AUTH commands may be issued in the same session.  After a
        successful AUTH command completes, a server MUST reject any
        further AUTH commands with a 503 reply.

        The AUTH command is not permitted during a mail transaction.
        An AUTH command issued during a mail transaction MUST be
        rejected with a 503 reply.

So a RSET, imv, must not alter this authentication condition because 
the client has a protocol design restriction (thus not coded to 
re-authenticate) and the server will not allow it anyway.  Our server 
will issue an AUTH response:

       503 Invalid AUTH request. Already Authenticated

Same applies with IP based because there is no way for the client to 
change the IP during the same session.

Like Keith indicated, we don't have a "logout" feature.

Anyway, I don't quite see the complication with authentication and 
"reset" because there is nothing in SMTP AFAIK that currently allows it.

On a related note, there is one change our SUBMIT server does in 
regards to authentication and the growth of NATS usage.  This was 
noted to Klensin and Resnick to relax the EHLO checking requirements 
until the AUTH state is determined.

Since 4409 requires AUTHentication, it can delay the EHLO checking 
because depending on the MUAs being used on a private network LAN 
going over the wire with a NAT public address, there will be potential 
of an IP mismatch.

For example:

     Connecting client IP: 99.3.111.22  <---- NAT public address
     S: 220 host.example.com ESMTP Server ready
     E: EHLO [192.168.1.101]            <---- IP where MUA is on LAN

This would be an SMTP violation enforceable under port 587 (RFC4409) 
operations.

So our SMTP server has an option (on by default) that when port 587 is 
active, it will relax this EHLO IP literal filter check until the 
pending and required AUTH state is determined.

That was the workaround until the popular Thunderbird MUA was updated 
and TBIRD now offers MTA options to set the EHLO client domain name or 
IP literal since its not easy to detect what will be the NAT public 
address. Since it was a quick fix, no user UI was added and it must be 
done manually by editing the config file under TOOLS | OPTIONS | ADVANCED.

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: [email-masked]
Cc: "[email-masked]" <[email-masked]>
Date: Tue, 31 May 2011 15:29:09 -0700 (PDT)
Subject: RE: RSET command - possible security loophole
To: "Murray S. Kucherawy" <[email-masked]>
Sender: [email-masked]

> Are there properties of the session state, other than
> implementation-specific ones, that might ever be altered by commands
> after EHLO?  I can't think of any.

Authentication (AUTH) and session security (STARTTLS) are the obvious examples,
but it is also very common for implementations to support proprietary
mechanisms to allow setting of various session attributes (including but not
limited to the real IP address of the client). Various sorts of proxies are the
usual use-case for this stuff.

I'm not a fan of proxies personally, mostly because proper implementation seems
to be a much bigger challenge than anyone cares to admit, but pretending they
don't exist and such commands haven't been implemented would be silly.

> Past EHLO, everything is either neutral or is a message-specific instruction
> of some kind.  If that's the case, then declaring that RSET only ever resets
> message state seems like where the RFC should leave it.

Sorry, that's not even close to correct.

				Ned

Date: Tue, 31 May 2011 18:09:22 -0400
From: John C Klensin <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Keith Moore <[email-masked]>, Hector Santos <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--On Tuesday, May 31, 2011 14:48 -0700 Ned Freed
<[email-masked]> wrote:

>> > With respect to authentication, it's arguable that there
>> > should be some way to "logout" without ending the current
>> > TCP session.   But that would have to be a separate SMTP
>> > extension.
> 
>> Right.   Or part of a particular authentication extension.
> 
> I think defining an
> UNAUTHENTICATE/LOGOUT/whatever-you-want-to-call-it extension
> would be a great idea, but it's definitely a separate effort
> from any of this

Absolutely.  If I wasn't clear about that "separate effort", I
apologize.

  john

From: [email-masked]
Cc: Keith Moore <[email-masked]>, Hector Santos <[email-masked]>, [email-masked]
Date: Tue, 31 May 2011 14:48:15 -0700 (PDT)
Subject: Re: RSET command - possible security loophole
To: John C Klensin <[email-masked]>
Sender: [email-masked]

> --On Tuesday, May 31, 2011 11:19 -0400 Keith Moore
> <[email-masked]> wrote:

> > My impression is that the behavior of RSET has never been
> > defined with respect to authentication, and that the Right
> > Thing for RSET to do with respect to authentication is
> > absolutely nothing.
> >
> > A separate question is whether RSET affects anything in an
> > SMTP extension.  It's easy to say that RSET doesn't affect
> > extensions, but I doubt that's the right tack.  RSET should
> > definitely affect CHUNKING, for instance.  But I think RSET
> > should only affect the current message being sent.

> Keith,

> I think you are right, but that your formulation above is
> backwards.  As I read 5321, the semantics of RSET are completely
> clear and actually quite precise: It clears the mail transaction
> state and all the data that go with it; it does not clear the
> session state.  It is also clear that an extension could modify
> those semantics or even add arguments to RSET.  Possibly not a
> good idea, but very clearly possible in terms of how 5321 (and
> its predecessors) define the extension model.

Very nicely put. I agree 100%.

> The place where "backwards" comes in is the place where I think
> Hector has uncovered a problem, even though it is not the
> problem his initial note was about.   I think that, as we are
> checking extensions that  set data that needs to be carried from
> one command to others, we should be checking to be sure that
> action is either explicitly bound to either session or
> transaction state or that new states are clearly defined.

Well, to be fair, I think it's pretty obvious in most cases whether a given
extension involves session (e.g., AUTH), transaction (e.g., DSN), or no state
at all (e.g., PIPELINING). But having said that, it certainly would not hurt to
be specific about this, and may help.

> Such
> a clear definition would properly include a statement about what
> RSET (and EHLO and completion of the DATA command) does to the
> state and/or data.   That is a requirement on the relevant
> extension specification.

Sounds like a good plan moving forward.

> We may not have been clear enough
> about that requirement to be diligent and consistent about
> insisting on it -- in retrospect, it might even have been useful
> to put something about state or data retention in the template.
> But, at least modulo the possibility of tuning the template, it
> isn't a 5321 problem because 5321 does exactly what it should do
> -- defines the behavior in the absence of extensions and
> specifies that extensions can change that behavior.  As far as
> the existing extensions are concerned, the problem is much more
> likely "incomplete spec" than, e.g., "error in protocol".

+1

> > With respect to authentication, it's arguable that there
> > should be some way to "logout" without ending the current TCP
> > session.   But that would have to be a separate SMTP extension.

> Right.   Or part of a particular authentication extension.

I think defining an UNAUTHENTICATE/LOGOUT/whatever-you-want-to-call-it
extension would be a great idea, but it's definitely a separate effort from
any of this

				Ned

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Tue, 31 May 2011 14:45:35 -0700
Subject: RE: RSET command - possible security loophole
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John C Klensin
> Sent: Tuesday, May 31, 2011 12:44 PM
> To: Keith Moore
> Cc: Hector Santos; [email-masked]
> Subject: Re: RSET command - possible security loophole
> 
> The place where "backwards" comes in is the place where I think
> Hector has uncovered a problem, even though it is not the
> problem his initial note was about.   I think that, as we are
> checking extensions that  set data that needs to be carried from
> one command to others, we should be checking to be sure that
> action is either explicitly bound to either session or
> transaction state or that new states are clearly defined.

Given that the model of SMTP in general involves some connection-specific stuff (accept(), 220, EHLO, 250) followed by zero or more message-specific actions (MAIL, RCPT, DATA, RSET) and some things that have to do with neither (RSET, NOOP, HELP, VRFY, EXPN), I'm at a loss to understand how there's some blending of semantics here.

Are there properties of the session state, other than implementation-specific ones, that might ever be altered by commands after EHLO?  I can't think of any.  Past EHLO, everything is either neutral or is a message-specific instruction of some kind.  If that's the case, then declaring that RSET only ever resets message state seems like where the RFC should leave it.

-MSK

Date: Tue, 31 May 2011 14:06:41 -0700
To: Hector Santos <[email-masked]>
From: SM <[email-masked]>
Subject: Re: RSET command - possible security loophole
Cc: [email-masked]
Sender: [email-masked]

Hi Hector,
At 12:58 31-05-2011, Hector Santos wrote:
>In regards to authentication, in our code, for IP based (Relay 
>Tables, POPB4SMTP), it can not change the authentication based on 
>IP.  For ESMTP AUTH, once the session authenticated, per RFC2554, 
>section 4, it will not allow another AUTH to be issued with a 503 response.

Is your implementation compliant with RFC 2554 or RFC 4954?

Regards,
-sm 

Date: Tue, 31 May 2011 15:58:57 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Keith Moore wrote:
> My impression is that the behavior of RSET has never been defined 
> with respect to authentication, and that the Right Thing for RSET to 
> do with respect to authentication is absolutely nothing.
> 
> A separate question is whether RSET affects anything in an SMTP 
> extension.  It's easy to say that RSET doesn't affect extensions, but 
> I doubt that's the right tack.  RSET should definitely affect CHUNKING, 
> for instance.  But I think RSET should only affect the current message 
> being sent.
> 
> With respect to authentication, it's arguable that there should be some 
> way to "logout" without ending the current TCP session.   But that would 
> have to be a separate SMTP extension.

Good points.

In regards to authentication, in our code, for IP based (Relay Tables, 
POPB4SMTP), it can not change the authentication based on IP.  For 
ESMTP AUTH, once the session authenticated, per RFC2554, section 4, it 
will not allow another AUTH to be issued with a 503 response.

While I wasn't particular going in that direction with an 
authentication RSET relationship between two or more transactions, it 
was interesting to note how the interpretation of the text provided 
could mean different things.

I guess one question might be where/when does a Mail Transaction begin 
and start.     That seems to be:

    MAIL FROM:  - Start of Mail Transaction
    DATA EOD    - End of Mail Transaction

RSET, in general, clears everything about the "transaction" (MAIL, 
RCPT, DATA) at least our server does and taking a quick look at Exim 
and Sendmail code, pretty much the same thing, and since there is a 
specific order EHLO, MAIL, RCPT and DATA,  EHLO and MAIL can start a 
new transaction as well.

My post was about remembering a transaction "security" data point that 
had occurred in a previous failed transaction (most likely as a system 
wide policy generated at DATA before EOD) and its possible 
applicability in subsequent (same session) transactions.  RFC5321 made 
an effort to clarity policy related reply codes like 450, 550:

    DATA
      I: 354 -> data -> S: 250
                        E: 552, 554, 451, 452
                        E: 450, 550 (rejections for policy reasons)
      E: 503, 554

as oppose to 2821:

    DATA
       I: 354 -> data -> S: 250
                         E: 552, 554, 451, 452
       E: 451, 554, 503

So that is my only focus here.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 31 May 2011 15:43:45 -0400
From: John C Klensin <[email-masked]>
To: Keith Moore <[email-masked]>
cc: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--On Tuesday, May 31, 2011 11:19 -0400 Keith Moore
<[email-masked]> wrote:

> My impression is that the behavior of RSET has never been
> defined with respect to authentication, and that the Right
> Thing for RSET to do with respect to authentication is
> absolutely nothing.
> 
> A separate question is whether RSET affects anything in an
> SMTP extension.  It's easy to say that RSET doesn't affect
> extensions, but I doubt that's the right tack.  RSET should
> definitely affect CHUNKING, for instance.  But I think RSET
> should only affect the current message being sent.

Keith,

I think you are right, but that your formulation above is
backwards.  As I read 5321, the semantics of RSET are completely
clear and actually quite precise: It clears the mail transaction
state and all the data that go with it; it does not clear the
session state.  It is also clear that an extension could modify
those semantics or even add arguments to RSET.  Possibly not a
good idea, but very clearly possible in terms of how 5321 (and
its predecessors) define the extension model.

The place where "backwards" comes in is the place where I think
Hector has uncovered a problem, even though it is not the
problem his initial note was about.   I think that, as we are
checking extensions that  set data that needs to be carried from
one command to others, we should be checking to be sure that
action is either explicitly bound to either session or
transaction state or that new states are clearly defined.  Such
a clear definition would properly include a statement about what
RSET (and EHLO and completion of the DATA command) does to the
state and/or data.   That is a requirement on the relevant
extension specification.  We may not have been clear enough
about that requirement to be diligent and consistent about
insisting on it -- in retrospect, it might even have been useful
to put something about state or data retention in the template.
But, at least modulo the possibility of tuning the template, it
isn't a 5321 problem because 5321 does exactly what it should do
-- defines the behavior in the absence of extensions and
specifies that extensions can change that behavior.  As far as
the existing extensions are concerned, the problem is much more
likely "incomplete spec" than, e.g., "error in protocol".

> With respect to authentication, it's arguable that there
> should be some way to "logout" without ending the current TCP
> session.   But that would have to be a separate SMTP extension.

Right.   Or part of a particular authentication extension.

   john

Subject: Re: RSET command - possible security loophole
From: Keith Moore <[email-masked]>
Date: Tue, 31 May 2011 11:19:35 -0400
Cc: Hector Santos <[email-masked]>, [email-masked]
To: John C Klensin <[email-masked]>
Sender: [email-masked]

My impression is that the behavior of RSET has never been defined with respect to authentication, and that the Right Thing for RSET to do with respect to authentication is absolutely nothing.

A separate question is whether RSET affects anything in an SMTP extension.  It's easy to say that RSET doesn't affect extensions, but I doubt that's the right tack.  RSET should definitely affect CHUNKING, for instance.  But I think RSET should only affect the current message being sent.

With respect to authentication, it's arguable that there should be some way to "logout" without ending the current TCP session.   But that would have to be a separate SMTP extension.

Keith

Date: Tue, 31 May 2011 15:59:23 +0100
From: Tony Finch <[email-masked]>
To: John Levine <[email-masked]>
cc: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

John Levine <[email-masked]> wrote:
>
> >I have observed a "security loophole" where by spammers are
> >increasingly using RSET to avoid new local policy-based anti-spam
> >technologies such as DNSRBL, SPF, Greylisting, etc, at the SMTP level
> >and this is because the transaction "state" tables during the same
> >session were cleared.
>
> I've never seen that, but if it happens, it's a bug in the mail
> server, not a bug in the spec.
>
> No change to 5321 bis, please.

Right. It's the transaction state that is cleared. If the MTA keeps
per-connection state then that needs to be retained e.g. no point
re-doing DNS lookups.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Rockall, Malin, Hebrides: South 5 to 7, occasionally gale 8 at first in
Rockall and Malin, veering west or northwest 4 or 5, then backing southwest 5
or 6 later. Rough or very rough. Occasional rain. Moderate or good,
occasionally poor.

Subject: Re: RSET command - possible security loophole
From: Dave Crocker <[email-masked]>
Date: Mon, 30 May 2011 23:04:12 -0700
To: [email-masked]
CC: [email-masked]
Sender: [email-masked]

------AXIN34H7PN8U1BAFNWQ1PHHFFGARXN
Content-Type: text/plain;
 charset=UTF-8
Content-Transfer-Encoding: 8bit

+1

d/
--
Dave Crocker
bbiw.net

[email-masked] wrote:

> >I have observed a "security loophole" where by spammers are
> >increasingly using RSET to avoid new local policy-based anti-spam
> >technologies such as DNSRBL, SPF, Greylisting, etc, at the SMTP level
> >and this is because the transaction "state" tables during the same
> >session were cleared.

> I've never seen that, but if it happens, it's a bug in the mail
> server, not a bug in the spec.

> No change to 5321 bis, please.

+1

				Ned

------AXIN34H7PN8U1BAFNWQ1PHHFFGARXN
Content-Type: text/html;
 charset=utf-8
Content-Transfer-Encoding: 8bit

<html><head></head><body>+1<br>
<br>
d/<br>
--<br>
Dave Crocker<br>
<a href="http://bbiw.net">bbiw.net</a><br><br><div class="gmail_quote">[email-masked] wrote:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<pre style="white-space: pre-wrap; word-wrap:break-word; font-family: monospace"><br /><br />&gt; &gt;I have observed a "security loophole" where by spammers are<br />&gt; &gt;increasingly using RSET to avoid new local policy-based anti-spam<br />&gt; &gt;technologies such as DNSRBL, SPF, Greylisting, etc, at the SMTP level<br />&gt; &gt;and this is because the transaction "state" tables during the same<br />&gt; &gt;session were cleared.<br /><br />&gt; I've never seen that, but if it happens, it's a bug in the mail<br />&gt; server, not a bug in the spec.<br /><br />&gt; No change to 5321 bis, please.<br /><br />+1<br /><br />				Ned<br /></pre></blockquote></div></body></html>
------AXIN34H7PN8U1BAFNWQ1PHHFFGARXN--

Date: Tue, 31 May 2011 01:18:55 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

John C Klensin wrote:
>> I think it is one of those items to write/jot down in a "Keep
>> In Mind" todo list so that when a RFC5321bis is ever pursued,
>> we could bring it up, debate and address it among other things
>> that might be considered for a bis.
> 
> You could certainly do that.  The only advantage of errata is
> that they get tracked and not lost.  

Ok, but I was not begging or voluntary for one neither :).

> But, again, I don't think
> there is a 5321 issue here at all and my suggestion was about
> errata to the various extension documents, not to 5321. 

I agree there is no 5321 issue.  I like RFC5321 because it does 
provide great operational insights unlike other documents and I can 
only hope it stays that way.

> Even if 5321 were changed to insert an "authenticated state", 
> with RSET affecting the mail transaction state only, extensions would
> still be obligated to explain how much state RSET changed.  

I was not even going there and I apologize if I expressed it 
incorrectly.  I meant there are some points to discussed, but I that 
wasn't the issue with this topic.

> But, IMO, we wouldn't want to make that change because there are
> actually two different authentication models as far as the state
> table is concerned:
>   
>   S: 220...   (connection state)
>   C: EHLO (starting session state)
>     ...
>   C: MAIL FROM< (starting mail transaction state)
>     ...
>   C: <Sending mailbox authentication> (starting type 1
> authentication state)
>   ...
> 
> and
>   
>   S: 220...   (connection state)
>   C: EHLO (starting session state)
>     ...
>    C: <Sending session/ client system authentication> (starting
> type 2 authentication state)
>     ...
>  C: MAIL FROM< (starting mail transaction state)
>     ...

Yes, ESMTP AUTH, etc, or just plain old Allow Relay IP at the connect 
level.

I wasn't going there and I guess because, from my SMTP design 
standpoint, once you authenticated, the SMTP level filter crap doesn't 
apply.  We never wanted to add this stuff anyway and its only done 
after recognizing (like the world too) the massive abuse with 
anonymous, unsolicited, non-authenticated sessions.

I did not mean to imply RSET will change that authenticated condition 
which ever way that was determined:

      Allow IP Relay Tables
      POP B4 SMTP
      ESMTP AUTH

etc.

> Would one want the same behavior from RSET for both
> arrangements?  Probably not, although it could be specified
> either way.  

Not for the traditional authentication methods. This is information 
that must be preserved, but it isn't the issue at hand and now I can 
see how RSET is not the only issue in regards to the topic in hand. 
See below.

>> The reality is, as I presume you may agree at some level, SMTP
>> today is not a pure "stand alone" application. It is highly
>> integrated and "feature rich" with security related policy
>> concepts playing a greater role.  The flexible reply codes and
>> guidelines to follow it, allow for any kind of future state
>> machine to work very well.
> 
> Yes.  But, with the current definition and model that "future
> state machine" has to be specified as part of whatever
> extensions cause it to be appropriate or necessary.   There
> isn't any magic that changes the SMTP state machine itself.  One
> could revise the model itself, but the result wouldn't be 821/
> 2821/ 5321 SMTP any more.

I agree that SMTP is perfect. :)  And now I can see how not just RSET 
but other commands apply to this "security" topic.  Once a "security 
policy-based" condition is determined, there is reasonable logic to 
disallow any further continuance short of dropping the client.

In other words, using the new RFC5321 explicit "enhanced" description 
for 450, 550 used for "policy reasons", it could mean no further state 
machine advancement or resets will be allowed, i.e, no transactions 
will be accepted during the current session.

>> ...
>> As Derek pointed out, there are different transactions with
>> RSET and SMTP was designed to handle it independently
>> (correctly IMV).  I don't believe its a matter of stupidity or
>> one suggesting a spec not having the foresight. Its simple
>> more about current realizations about new SMTP integration
>> needs (which I personally believe applies to most, if not all,
>> operations) and I just was noting as general as I can, there
>> might be additional specific "security" related exceptions to
>> the RSET semantics.
> 
> We may just need to agree to disagree here, but, from my point
> of view, the SMTP semantics of RSET is quite clear and just
> fine.  The only relevant SMTP state is "mail transaction" and
> that is what RSET resets (it does not, for example, reset the
> server capability announcement returned in the reply to EHLO.
> If some extension comes along and specified additional state(s),
> it is obligated to specify what happens to those states in the
> presence of RSET.  If it doesn't do so, that spec is defective
> (not a problem with 5321 or new semantics for RSET).  If it
> specifies that RSET discards, e.g., the authentication
> information (perhaps by making the extension state part of the
> mail transaction state) then my claim is that starting a new
> mail transaction without noticing that authentication
> information is no longer valid, then I repeat my belief that
> would be a dumb (or at least careless) implementation, not much
> different in practice from accepting the sequence:
> 
>    EHLO MAIL RCPT DATA . RSET RCPT
> 
> and assuming that the second RCPT could reuse the data from the
> first (only) MAIL command despite the RSET.  I can imagine
> someone trying to justify that on the grounds of robustness, but
> the risks are such that I'd think it would be dumb (as well as
> fairly clearly non-conformant).
> 
>    john

Yes, and its unbelievable how this strayed off course. I did not mean 
to imply RSET cleared authentication information, especially if its 
traditional IP based authentication - can't reset the IP (under the 
same session)! :)

The issue here was a DATA (or any state for that matter) rejection 
based on local security policy and the RSET (or even EHLO, MAIL) 
clearing information about "a security condition" from the last 
transaction in the same session now possibly lost when/where a new 
transaction begins.

I guess in principle, once that "condition" is determine (by whatever 
means), it applies to any subsequent new transaction is what I am 
describing.

And yes, this really has nothing to do with SMTP itself, its rather 
perfect and starting a new transaction is well defined and has worked 
perfectly for us.... except now we have new conditions based on 
external information at the SMTP level and it may mean the state 
machine logic changes at the implementation level (not the spec).  At 
least, that is how I see it and having insights are good, not bad.

thanks

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 30 May 2011 23:52:20 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Glenn Anderson <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

John C Klensin wrote:

>> Note I said per
>> spec, because there are systems that will put the mail in
>> delivery motion once that final DATA <CRLF>.<CRLF> is received
>> and prior to an pending client command.
> 
> Which would be a perfectly valid and conforming interpretation
> of SMTP.  RSET is not specified as causing some action that
> occurred earlier to be undone or aborted.  FTP was designed to
> be somewhat asynchronous; SMTP is not, pipelining
> notwithstanding.  
> 
>> Is this wrong?
> 
> Yes, I think so.

Well, maybe I stated too much for this gotcha crowd. Our SMTP server 
is behaving correctly, but I can certainly see the dividing difference.

If your system is an like many old systems, and our was one of them 
for optimization reasons, it will immediately signal a router after a 
successful positive reception of DATA <CRLF>.<CRLF> to begin the 
delivery process, then absolutely, the MAIL, RCPT and DATA is 
immediately "cleared" and effectively, RSET is like NOOP because there 
is nothing to be cleared.

However, and we had discussions about this here, to be 100% compliant 
with RFC2821 and thus RFC5321, there is a "No QUIT/MAIL" cancellation 
requirement per section 4.1.1.10 (QUIT):

    If the connection is closed prematurely due to violations
    of the above or system or network failure, the server MUST cancel any
    pending transaction, but not undo any previously completed
    transaction, and generally MUST act as if the command or transaction
    in progress had received a temporary error (i.e., a 4yz response).

Under a RFC5321 compliant "No Quit/Mail" cancellation implementation, 
after completing the DATA state, the server is waiting for a pending 
RSET, MAIL or QUIT command:

    QUIT - complete transaction, if any
    MAIL - complete transaction, if any
           perform a "reset"
    RSET - cancel any pending DATA transaction delivery,
           perform a "reset"
    drop - cancel any pending DATA transaction delivery

We added this support in 2008 as a local policy option 
(EnableNoQuitCancel) which will alter your SMTP state flow, your 
optimization and now you MUST follow RSET vs QUIT/MAIL correctly. 
RSET (after DATA) aborts the transaction, QUIT/MAIL (after DATA) does 
not. RSET is not an NOOP at this point.

If you have an old school server (an optimized one), then the mail is 
immediately processed for delivery and the cancellation or reset 
logics doesn't apply - its was already reset.

Am I still wrong?

The issue with the topic, is that there *could be* "security policy" 
related concerns where due to a RSET (after DATA) does not "memorized" 
the last DATA rejection state and it may apply.

I agree that SMTP is working properly with the way RSET is defined and 
I am not advocating any immediately errata or bis change.  But I do 
believe it is security insight for SMTP in the modern age.  Just it 
down or not, I'm not going to start a Errata, if someone else things 
its worthy, go for it.  I am just making a note of it.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 30 May 2011 22:02:20 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, Glenn Anderson <[email-masked]>
cc: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--On Monday, May 30, 2011 20:16 -0400 Hector Santos
<[email-masked]> wrote:

> 
> Glenn Anderson wrote:
>> At 5:10 pm -0400 30/5/2011, Hector Santos wrote:
>>> So basically, on the first DATA attempt our data filter
>>> returned a  FAIL=451 which was a greylist filter script, the
>>> transaction and state  was cleared with RSET. Then during
>>> the 2nd attempt the client was able  to get by the initial
>>> rejection.
>> 
>> If you read a bit further in to section 4.1.1.5 it says:
>>    It is
>>    effectively equivalent to a NOOP (i.e., it has no effect)
>>    if issued immediately after EHLO, before EHLO is issued in
>>    the session, after an end of data indicator has been sent
>>    and acknowledged, or immediately before a QUIT.
>> 
>> If your server is adhering to the standard, you would still
>> have the  same problem even if the spammer left out the RSET,
>> or replaced it with  a NOOP, so this isn't a problem with
>> RSET.
>> 
>> Glenn.
> 
> Hi Glenn,
> 
> Hmmm,  the NOOP is truly a "NO OPeration" command which is a
> carry on from the FTP model (which SMTP is based on).  Its an
> old school "keep alive" concept to avoid idle timeouts.

Regardless of its origins, questions about when you would want
to use it, how old the school might be, etc., NOOP is specified
in 5321 and means what it says there, not whatever one might
speculatively derive from speculation about is origins.

> I would not interpret RSET as not an "NO OPeration" concept -
> it will literally reset, delete, purge, etc, stuff already
> temporarily recorded by the server for the current session
> pending a QUIT or MAIL command to activate the the final
> acceptance (per spec) for the mail delivery.

Please read what Glenn wrote and then try reading 5321,
especially Section 4.1.1.5, again.    There is a "mail
transaction" state, which is initiated by a successful MAIL
command and normally terminated by a successful "DATA" command
(acceptance of the DATA-terminating CRLF.CRLF).  It can also be
terminated, less normally, by QUIT, EHLO, or RSET.  RSET
terminates mail transactions.  Sending RSET when there is no
mail transaction in progress is not harmful (that text is there
partially to prevent someone from responding to a useless RSET
with "503 Command out sequence"), but it is also useless, as
useless as NOOP. 

Contrary to your comment above, RSET does not clear "stuff...
recorded for the current session" because it doesn't clear
session state.  If it did clear ("reset, delete, purge, etc.)
session state, then one would have to send EHLO after RSET to
reestablish client system identification and the extension
offers.  RSET can be followed by MAIL, VRFY, EHLO, or, given
extensions, other commands that might establish either mail
transaction state or some other state.  It is not required to be
followed by EHLO to rebuild the session state.

>   Not I said per
> spec, because there are systems that will put the mail in
> delivery motion once that final DATA <CRLF>.<CRLF> is received
> and prior to an pending client command.

Which would be a perfectly valid and conforming interpretation
of SMTP.  RSET is not specified as causing some action that
occurred earlier to be undone or aborted.  FTP was designed to
be somewhat asynchronous; SMTP is not, pipelining
notwithstanding.  

> Is this wrong?

Yes, I think so.

    john

From: [email-masked]
Cc: [email-masked]
Date: Mon, 30 May 2011 18:44:21 -0700 (PDT)
Subject: Re: RSET command - possible security loophole
To: John Levine <[email-masked]>
Sender: [email-masked]

> >I have observed a "security loophole" where by spammers are
> >increasingly using RSET to avoid new local policy-based anti-spam
> >technologies such as DNSRBL, SPF, Greylisting, etc, at the SMTP level
> >and this is because the transaction "state" tables during the same
> >session were cleared.

> I've never seen that, but if it happens, it's a bug in the mail
> server, not a bug in the spec.

> No change to 5321 bis, please.

+1

				Ned

Date: Mon, 30 May 2011 21:38:25 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>
cc: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--On Monday, May 30, 2011 19:33 -0400 Hector Santos
<[email-masked]> wrote:

> 
> John,
> 
> I don't think an Errata is necessary unless other people made
> aware of this begin checking their logs and realized its also
> happening to them as well.
> 
> I think it is one of those items to write/jot down in a "Keep
> In Mind" todo list so that when a RFC5321bis is ever pursued,
> we could bring it up, debate and address it among other things
> that might be considered for a bis.

You could certainly do that.  The only advantage of errata is
that they get tracked and not lost.  But, again, I don't think
there is a 5321 issue here at all and my suggestion was about
errata to the various extension documents, not to 5321.  Even if
5321 were changed to insert an "authenticated state", with RSET
affecting the mail transaction state only, extensions would
still be obligated to explain how much state RSET changed.  But,
IMO, we wouldn't want to make that change because there are
actually two different authentication models as far as the state
table is concerned:
  
  S: 220...   (connection state)
  C: EHLO (starting session state)
    ...
  C: MAIL FROM< (starting mail transaction state)
    ...
  C: <Sending mailbox authentication> (starting type 1
authentication state)
  ...

and
  
  S: 220...   (connection state)
  C: EHLO (starting session state)
    ...
   C: <Sending session/ client system authentication> (starting
type 2 authentication state)
    ...
 C: MAIL FROM< (starting mail transaction state)
    ...

Would one want the same behavior from RSET for both
arrangements?  Probably not, although it could be specified
either way.  Note that preserving the authentication information
across RSET in the first case would either make
   C: EHLO
     ...
   C: MAIL FROM:<[email-masked]>
     ...
   C: RCPT/ DATA/ . / RSET
     ...
   C: MAIL FROM:<[email-masked]>
impossible or actually would create a security hole.  On the
other hand, if all that is being done is to authenticate the
sending system and authorize it to send mail (just mail, not
mail on behalf of a particular user), then letting the
authorization carry across mail transactions makes perfect sense.

> The reality is, as I presume you may agree at some level, SMTP
> today is not a pure "stand alone" application. It is highly
> integrated and "feature rich" with security related policy
> concepts playing a greater role.  The flexible reply codes and
> guidelines to follow it, allow for any kind of future state
> machine to work very well.

Yes.  But, with the current definition and model that "future
state machine" has to be specified as part of whatever
extensions cause it to be appropriate or necessary.   There
isn't any magic that changes the SMTP state machine itself.  One
could revise the model itself, but the result wouldn't be 821/
2821/ 5321 SMTP any more.

>...
> As Derek pointed out, there are different transactions with
> RSET and SMTP was designed to handle it independently
> (correctly IMV).  I don't believe its a matter of stupidity or
> one suggesting a spec not having the foresight. Its simple
> more about current realizations about new SMTP integration
> needs (which I personally believe applies to most, if not all,
> operations) and I just was noting as general as I can, there
> might be additional specific "security" related exceptions to
> the RSET semantics.

We may just need to agree to disagree here, but, from my point
of view, the SMTP semantics of RSET is quite clear and just
fine.  The only relevant SMTP state is "mail transaction" and
that is what RSET resets (it does not, for example, reset the
server capability announcement returned in the reply to EHLO.
If some extension comes along and specified additional state(s),
it is obligated to specify what happens to those states in the
presence of RSET.  If it doesn't do so, that spec is defective
(not a problem with 5321 or new semantics for RSET).  If it
specifies that RSET discards, e.g., the authentication
information (perhaps by making the extension state part of the
mail transaction state) then my claim is that starting a new
mail transaction without noticing that authentication
information is no longer valid, then I repeat my belief that
would be a dumb (or at least careless) implementation, not much
different in practice from accepting the sequence:

   EHLO MAIL RCPT DATA . RSET RCPT

and assuming that the second RCPT could reuse the data from the
first (only) MAIL command despite the RSET.  I can imagine
someone trying to justify that on the grounds of robustness, but
the risks are such that I'd think it would be dumb (as well as
fairly clearly non-conformant).

   john

Date: Mon, 30 May 2011 20:16:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Glenn Anderson <[email-masked]>
CC: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Glenn Anderson wrote:
> At 5:10 pm -0400 30/5/2011, Hector Santos wrote:
>> So basically, on the first DATA attempt our data filter returned a 
>> FAIL=451 which was a greylist filter script, the transaction and state 
>> was cleared with RSET. Then during the 2nd attempt the client was able 
>> to get by the initial rejection.
> 
> If you read a bit further in to section 4.1.1.5 it says:
>    It is
>    effectively equivalent to a NOOP (i.e., it has no effect) if issued
>    immediately after EHLO, before EHLO is issued in the session, after
>    an end of data indicator has been sent and acknowledged, or
>    immediately before a QUIT.
> 
> If your server is adhering to the standard, you would still have the 
> same problem even if the spammer left out the RSET, or replaced it with 
> a NOOP, so this isn't a problem with RSET.
> 
> Glenn.

Hi Glenn,

Hmmm,  the NOOP is truly a "NO OPeration" command which is a carry on 
from the FTP model (which SMTP is based on).  Its an old school "keep 
alive" concept to avoid idle timeouts.

I would not interpret RSET as not an "NO OPeration" concept - it will 
literally reset, delete, purge, etc, stuff already temporarily 
recorded by the server for the current session pending a QUIT or MAIL 
command to activate the the final acceptance (per spec) for the mail 
delivery.   Not I said per spec, because there are systems that will 
put the mail in delivery motion once that final DATA <CRLF>.<CRLF> is 
received and prior to an pending client command.

Is this wrong?

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 30 May 2011 19:54:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "Derek J. Balling" <[email-masked]>
CC: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Derek J. Balling wrote:

> That interpretation opens up the following situation as problematic:
> 
>> Mail from: foo
>> rcpt to: bar
> [blah blah blah]
>> rset
>> mail from baz
>> rcpt to mat
> 
> because in that interpretation, 'foo's attempt would never make it 
> into any sort of permanent state.
> 
> My reading of the RFC would be much closer to THAT being the special 
> case that needs to be made (that you CAN stash and set aside for 
> later processing some state-data received prior to a RSET). But that's 
> not really all that much of a logistic stretch to make.
> 
> Just my $0.02 worth.

In fact, the sender did retry the first 451 rejected transaction 5 
minutes later and it was greylist accepted.

That lead me to think about the idea of having session state point 
rejection information passed to an external process/shim called at 
RSET to clear whatever was externally recording.  An overkill 
solution, but a thought. :)

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 30 May 2011 16:35:15 -0700
To: Hector Santos <[email-masked]>
From: SM <[email-masked]>
Subject: Re: RSET command - possible security loophole 
Cc: [email-masked]
Sender: [email-masked]

Hi Hector,
At 14:10 30-05-2011, Hector Santos wrote:
>But in general, for RFC5321, this "security" information should be 
>conveyed in regards to the RSET command.  Perhaps:
>
>    Any stored sender, recipients, and mail data MUST be discarded, and all
>    buffers and state tables cleared which the possible exception of local
>    policy transactional session security state information that may need
>    to be retained for the current SMTP session.

Please note that there has been a pre-evaluation of RFC 5321 
[1].  There is also an IETF WG [2] which is chartered to work on RFC 
5321.  I recommend reading the charter carefully as it spells out the 
scope of changes if 5321 is processed by the WG.

Regards,
-sm

1. http://tools.ietf.org/html/draft-ietf-yam-5321bis-smtp-pre-evaluation-05
2. http://www.ietf.org/dyn/wg/charter/yam-charter 

Date: Mon, 30 May 2011 19:33:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

John,

I don't think an Errata is necessary unless other people made aware of 
this begin checking their logs and realized its also happening to them 
as well.

I think it is one of those items to write/jot down in a "Keep In Mind" 
todo list so that when a RFC5321bis is ever pursued, we could bring it 
up, debate and address it among other things that might be considered 
for a bis.

The reality is, as I presume you may agree at some level, SMTP today 
is not a pure "stand alone" application. It is highly integrated and 
"feature rich" with security related policy concepts playing a greater 
role.  The flexible reply codes and guidelines to follow it, allow for 
any kind of future state machine to work very well.

We saw the RSET issue for awhile now and it was marked down a 
something to do for our next update. I wasn't quite sure how to best 
address it in general, but specifically from our SMTP level filters 
standpoint, the changes/solution we needed to do was clear - remember. 
That effort was now being done this week and I thought maybe I should 
make a note of it for the group as "food for thought."

As Derek pointed out, there are different transactions with RSET and 
SMTP was designed to handle it independently (correctly IMV).  I don't 
believe its a matter of stupidity or one suggesting a spec not having 
the foresight. Its simple more about current realizations about new 
SMTP integration needs (which I personally believe applies to most, if 
not all, operations) and I just was noting as general as I can, there 
might be additional specific "security" related exceptions to the RSET 
semantics.

-- 
Sincerely

Hector Santos
http://www.santronics.com

John C Klensin wrote:
> Hector,
> 
> Disclaimer: I'm predisposed to avoid changes, especially
> substantive ones, to RFC 5321 if possible; that bias might be
> affecting my opinion.  On the other hand, I might have the same
> opinion without it... and, in this case, I'm pretty sure I do.
> 
> It seems to me that you've just made a case for modifications to
> the documents that specify these "policy-based anti-spam
> technologies" and the associated extensions, not to 5321.  The
> extensions could reasonably say that they affect the state table
> and/or modify the correct behavior of RSET.  They could specify
> either that the critical information not be discarded or that
> the SMTP server not continue after a RSET until the
> authentication is repeated (or give the server implementer the
> choice).  They could also contain Security Considerations
> material that discusses this issue.
> 
> In that context, the most I can see doing to 5321 would be to
> extend Section 2.2.3 so that the last sentence of the first
> paragraph would read, e.g.,
> 
> 	"In particular, extensions can change the minimum limits
> 	specified in Section 4.5.3, can change the ASCII
> 	character set requirement as mentioned above, can
> 	introduce some optional modes of message handling, or
> 	can change aspects of the state table processing
> 	specified in this document."
> 
> But I think a modification to 5321 is unnecessary, first because
> the first two sentences of that paragraph read 
> 
> 	"Extensions that change fairly basic properties of SMTP
> 	operation are permitted. The text in other sections of
> 	this document must be understood in that context."
> 	That ought to be quite clear and does not require us to
> 	list every case.  
> 
> Second, doing authentication or authorization tests at the
> beginning of a session and then both discarding the resulting
> information and not requiring that the authentication and
> authorization steps be repeated is, IMO and in technical terms,
> just a dumb implementation.  Per the text quoted above, we've
> told people that, if they use extensions, they have to interpret
> any relevant statements in 5321 in the light of those extensions
> and their effects.  It should be obvious that, if one wants to
> be dependent on some set of security procedures, then executing
> commands without those procedures being in effect is unwise (or
> worse).
> 
> To say that even more bluntly, I don't think 5321 needs to
> provide "additional process insight".  No additional words in
> 5321 will prevent implementers who don't feel a need to think
> from their own carelessness and/or stupidity.
> 
> If you think that needs to be more clear in the relevant
> extension documents, I recommend that you start generating
> errata.
> 
>     best,
>      john
> 
> 
> --On Monday, May 30, 2011 17:10 -0400 Hector Santos
> <[email-masked]> wrote:
> 
>> In RFC5321, section 4.1.1.5 it says:
>>
>>     4.1.1.5.  RESET (RSET)
>>
>>     This command specifies that the current mail transaction
>> will be
>>     aborted.  Any stored sender, recipients, and mail data
>> MUST be
>>     discarded, and all buffers and state tables cleared.
>>
>> I have observed a "security loophole" where by spammers are
>> increasingly using RSET to avoid new local policy-based
>> anti-spam technologies such as DNSRBL, SPF, Greylisting, etc,
>> at the SMTP level and this is because the transaction "state"
>> tables during the same session were cleared. Since these
>> external functional generally just has one output, false/true
>> and/or reply code (45x/55x), the additional process insight to
>> retain the rejection state is not available.
>>
>> It seems we need an exception clause for the RSET command to
>> close this "security" loophole or clarification if the above
>> MUST apples to the state tables.
>> ...
> 
> 
> 

Date: Tue, 31 May 2011 11:31:02 +1200
To: [email-masked]
From: Glenn Anderson <[email-masked]>
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

--============_-905324231==_ma============
Content-Type: text/plain; charset="us-ascii" ; format="flowed"

At 5:10 pm -0400 30/5/2011, Hector Santos wrote:
>So basically, on the first DATA attempt our data filter returned a 
>FAIL=451 which was a greylist filter script, the transaction and 
>state was cleared with RSET. Then during the 2nd attempt the client 
>was able to get by the initial rejection.

If you read a bit further in to section 4.1.1.5 it says:
    It is
    effectively equivalent to a NOOP (i.e., it has no effect) if issued
    immediately after EHLO, before EHLO is issued in the session, after
    an end of data indicator has been sent and acknowledged, or
    immediately before a QUIT.

If your server is adhering to the standard, you would still have the 
same problem even if the spammer left out the RSET, or replaced it 
with a NOOP, so this isn't a problem with RSET.

Glenn.
--============_-905324231==_ma============
Content-Type: text/html; charset="us-ascii"

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<html><head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
 --></style><title>Re: RSET command - possible security
loophole</title></head><body>
<div>At 5:10 pm -0400 30/5/2011, Hector Santos wrote:</div>
<blockquote type="cite" cite>So basically, on the first DATA attempt
our data filter returned a FAIL=451 which was a greylist filter
script, the transaction and state was cleared with RSET. Then during
the 2nd attempt the client was able to get by the initial
rejection.</blockquote>
<div><br></div>
<div>If you read a bit further in to section 4.1.1.5 it says:</div>
<div>&nbsp;&nbsp; It is</div>
<div>&nbsp;&nbsp; effectively equivalent to a NOOP (i.e., it has no
effect) if issued</div>
<div>&nbsp;&nbsp; immediately after EHLO, before EHLO is issued in the
session,<b> after</b></div>
<div><b>&nbsp;&nbsp; an end of data indicator has been sent and
acknowledged</b>, or</div>
<div>&nbsp;&nbsp; immediately before a QUIT.</div>
<div><br></div>
<div>If your server is adhering to the standard, you would still have
the same problem even if the spammer left out the RSET, or replaced it
with a NOOP, so this isn't a problem with RSET.</div>
<div><br></div>
<div>Glenn.</div>
</body>
</html>
--============_-905324231==_ma============--

Date: 30 May 2011 23:08:50 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Subject: Re: RSET command - possible security loophole
Organization: 
Sender: [email-masked]

>I have observed a "security loophole" where by spammers are
>increasingly using RSET to avoid new local policy-based anti-spam
>technologies such as DNSRBL, SPF, Greylisting, etc, at the SMTP level
>and this is because the transaction "state" tables during the same
>session were cleared.

I've never seen that, but if it happens, it's a bug in the mail
server, not a bug in the spec.

No change to 5321 bis, please.

R's,
John

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Mon, 30 May 2011 15:26:39 -0700
Subject: RE: RSET command - possible security loophole
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of John C Klensin
> Sent: Monday, May 30, 2011 3:03 PM
> To: Hector Santos; [email-masked]
> Subject: Re: RSET command - possible security loophole
> 
> [...]
> To say that even more bluntly, I don't think 5321 needs to
> provide "additional process insight".  No additional words in
> 5321 will prevent implementers who don't feel a need to think
> from their own carelessness and/or stupidity.
> 
> If you think that needs to be more clear in the relevant
> extension documents, I recommend that you start generating
> errata.

It doesn't appear I was privy to the original message, but given the part of it that was quoted, +1 to John's reply.

-MSK

Subject: Re: RSET command - possible security loophole 
From: "Derek J. Balling" <[email-masked]>
Date: Mon, 30 May 2011 18:14:55 -0400
Cc: [email-masked]
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On May 30, 2011, at 5:10 PM, Hector Santos wrote:
>   This command specifies that the current mail transaction will be
>   aborted.  Any stored sender, recipients, and mail data MUST be
>   discarded, and all buffers and state tables cleared.

The counter-argument I'd make, interpretation-wise, is that the receiving-MTA must revert to the same data-state it was in prior to connection, where, in your example, <[email-masked]> has never made a message-attempt before. Thus, on the second attempt, it should be treated the same as the first, because by RSET'ing the sending-MTA cleared the state. Only on a QUIT or connection-drop should the receiving MTA then decide what to do with its state data (such as greylisted senders that made attempts during that connection).

That interpretation opens up the following situation as problematic:

> Mail from: foo
> rcpt to: bar
[blah blah blah]
> rset
> mail from baz
> rcpt to mat

because in that interpretation, 'foo's attempt would never make it into any sort of permanent state.

My reading of the RFC would be much closer to THAT being the special case that needs to be made (that you CAN stash and set aside for later processing some state-data received prior to a RSET). But that's not really all that much of a logistic stretch to make.

Just my $0.02 worth.

Cheers,
D

Date: Mon, 30 May 2011 18:03:23 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: RSET command - possible security loophole
Sender: [email-masked]

Hector,

Disclaimer: I'm predisposed to avoid changes, especially
substantive ones, to RFC 5321 if possible; that bias might be
affecting my opinion.  On the other hand, I might have the same
opinion without it... and, in this case, I'm pretty sure I do.

It seems to me that you've just made a case for modifications to
the documents that specify these "policy-based anti-spam
technologies" and the associated extensions, not to 5321.  The
extensions could reasonably say that they affect the state table
and/or modify the correct behavior of RSET.  They could specify
either that the critical information not be discarded or that
the SMTP server not continue after a RSET until the
authentication is repeated (or give the server implementer the
choice).  They could also contain Security Considerations
material that discusses this issue.

In that context, the most I can see doing to 5321 would be to
extend Section 2.2.3 so that the last sentence of the first
paragraph would read, e.g.,

	"In particular, extensions can change the minimum limits
	specified in Section 4.5.3, can change the ASCII
	character set requirement as mentioned above, can
	introduce some optional modes of message handling, or
	can change aspects of the state table processing
	specified in this document."

But I think a modification to 5321 is unnecessary, first because
the first two sentences of that paragraph read 

	"Extensions that change fairly basic properties of SMTP
	operation are permitted. The text in other sections of
	this document must be understood in that context."
	That ought to be quite clear and does not require us to
	list every case.  

Second, doing authentication or authorization tests at the
beginning of a session and then both discarding the resulting
information and not requiring that the authentication and
authorization steps be repeated is, IMO and in technical terms,
just a dumb implementation.  Per the text quoted above, we've
told people that, if they use extensions, they have to interpret
any relevant statements in 5321 in the light of those extensions
and their effects.  It should be obvious that, if one wants to
be dependent on some set of security procedures, then executing
commands without those procedures being in effect is unwise (or
worse).

To say that even more bluntly, I don't think 5321 needs to
provide "additional process insight".  No additional words in
5321 will prevent implementers who don't feel a need to think
from their own carelessness and/or stupidity.

If you think that needs to be more clear in the relevant
extension documents, I recommend that you start generating
errata.

    best,
     john

--On Monday, May 30, 2011 17:10 -0400 Hector Santos
<[email-masked]> wrote:

> 
> In RFC5321, section 4.1.1.5 it says:
> 
>     4.1.1.5.  RESET (RSET)
> 
>     This command specifies that the current mail transaction
> will be
>     aborted.  Any stored sender, recipients, and mail data
> MUST be
>     discarded, and all buffers and state tables cleared.
> 
> I have observed a "security loophole" where by spammers are
> increasingly using RSET to avoid new local policy-based
> anti-spam technologies such as DNSRBL, SPF, Greylisting, etc,
> at the SMTP level and this is because the transaction "state"
> tables during the same session were cleared. Since these
> external functional generally just has one output, false/true
> and/or reply code (45x/55x), the additional process insight to
> retain the rejection state is not available.
> 
> It seems we need an exception clause for the RSET command to
> close this "security" loophole or clarification if the above
> MUST apples to the state tables.
>...

Date: Mon, 30 May 2011 17:10:00 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: RSET command - possible security loophole 
Sender: [email-masked]

In RFC5321, section 4.1.1.5 it says:

    4.1.1.5.  RESET (RSET)

    This command specifies that the current mail transaction will be
    aborted.  Any stored sender, recipients, and mail data MUST be
    discarded, and all buffers and state tables cleared.

I have observed a "security loophole" where by spammers are 
increasingly using RSET to avoid new local policy-based anti-spam 
technologies such as DNSRBL, SPF, Greylisting, etc, at the SMTP level 
and this is because the transaction "state" tables during the same 
session were cleared. Since these external functional generally just 
has one output, false/true and/or reply code (45x/55x), the additional 
process insight to retain the rejection state is not available.

It seems we need an exception clause for the RSET command to close 
this "security" loophole or clarification if the above MUST apples to 
the state tables.

Here is an example of such a transaction:

**************************************************************************
Wildcat! SMTP Server v6.3.453.5
SMTP log started at Mon, 30 May 2011  16:19:40
Connection Time: 20110530 16:19:40  cid: 000079BD tid: 000008FC
Client IP: 64.187.117.40:27208 (unknown)
16:19:40 S: 220 winserver.com Wildcat! ESMTP Server v6.3.453.5 ready
16:19:40 C: EHLO mail.greatsunshines.com
16:19:40 S: 250-winserver.com, Pleased to meet you.
16:19:40 S: 250-SIZE 10240000
16:19:40 S: 250-ETRN
16:19:40 S: 250-AUTH CRAM-MD5 LOGIN PLAIN
16:19:40 S: 250-AUTH=LOGIN
16:19:40 S: 250 HELP
16:19:40 C: MAIL FROM:<[email-masked]>
16:19:40 S: 250 <[email-masked]>... Sender validation 
pending.
16:19:41 C: RCPT TO:<[email-masked]>
16:19:41 ** WCX Process: wcsap  ret: PASS
16:19:41 S: 250 <[email-masked]>... Recipient ok
16:19:41 C: DATA
16:19:41 S: 354 Start mail input; end with <CRLF>.<CRLF>
16:19:41 ** end of data received. (bytes: 2902) (141.7 K/s)
16:19:42 ** WCX Process: SmtpFilterHookLoader  ret: FAIL=451
16:19:42 S: 451 Please try again later (TempFail)
16:19:42 C: RSET
16:19:42 S: 250 Reset State #1
16:19:42 C: MAIL FROM:<[email-masked]>
16:19:42 S: 250 <[email-masked]>... Sender validation 
pending.
16:19:42 C: RCPT TO:<[email-masked]>
16:19:42 ** WCX Process: wcsap  ret: PASS
16:19:42 S: 250 <[email-masked]>... Recipient ok
16:19:42 C: DATA
16:19:42 S: 354 Start mail input; end with <CRLF>.<CRLF>
16:19:42 ** end of data received. (bytes: 11638) (141.0 K/s)
16:19:46 ** WCX Process: SmtpFilterHookLoader  ret: PASS
16:19:46 S: 250 Message received for delivery. (bytes: 11638)
16:19:46 C: QUIT
16:19:46 ** message copied to router prespool
16:19:46 ** Router signaled to begin handling new message
16:19:46 S: 221 closing connection
16:19:46 ** Completed

So basically, on the first DATA attempt our data filter returned a 
FAIL=451 which was a greylist filter script, the transaction and state 
was cleared with RSET. Then during the 2nd attempt the client was able 
to get by the initial rejection.

Now, this is an implementation issue since our greylist script does 
have the information, such as session client id (CID: 000079BD) passed 
to both DATA filter script calls (SmtpFilterHookLoader) and the 
greylist filter should not have allowed a second attempt for the same 
session to bypass its initial temp fail result.

But in general, for RFC5321, this "security" information should be 
conveyed in regards to the RSET command.  Perhaps:

    Any stored sender, recipients, and mail data MUST be discarded, 
and all
    buffers and state tables cleared which the possible exception of local
    policy transactional session security state information that may need
    to be retained for the current SMTP session.

Comments?

If anything, this might be good info for other operators and 
implementors of greylist and other SMTP level filtering methods to 
check to see if spammers are using RSET to get pass an initial reject.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 25 May 2011 11:10:32 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: Leading Dots - Disregard
Sender: [email-masked]

Disregard, we updated our beta server DATA block receiver and the code 
to undot the leading dot (per section 4.5.2 ) was left out.

Hector Santos wrote:
> Note: Arggh, resending because the example lines were getting extra 
> dots, so I am adding a leading space before each line!
> 
>  must be missing something here.  I noticed two MUAs, Outlook and
> Thunderbird adding extra dots for lines I would not expect it would.
> 
> For testing, I sent a simple text message with these five lines only:
> 
>  . one
>  .. two
>  ... three
>  .... four
>  ..... five
> 
> At my receiver, a socket read DATA block dump showed the MUAs sending
> an extra dot on each line:
> 
>  .. one
>  ... two
>  .... three
>  ..... four
>  ...... five
>  .                <-- expected <CRLF>.<CRLF> terminator
> 
> Why?  Is that correct?
> 
> What caught my eye was an intermediary also adding extra dots, in this
> case, a message I set to a list had a line beginning with:
> 
>      ... blah, blah
> 
> But it showed up here with two extra dots:
> 
>      ..... blah, blah
> 
> So when I begin testing why, I noticed the MUA was adding one extra
> and the intermediary adding one extra itself.
> 
> Is this normal?
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 25 May 2011 10:42:39 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Leading Dots
Sender: [email-masked]

Note: Arggh, resending because the example lines were getting extra 
dots, so I am adding a leading space before each line!

  must be missing something here.  I noticed two MUAs, Outlook and
Thunderbird adding extra dots for lines I would not expect it would.

For testing, I sent a simple text message with these five lines only:

  . one
  .. two
  ... three
  .... four
  ..... five

At my receiver, a socket read DATA block dump showed the MUAs sending
an extra dot on each line:

  .. one
  ... two
  .... three
  ..... four
  ...... five
  .                <-- expected <CRLF>.<CRLF> terminator

Why?  Is that correct?

What caught my eye was an intermediary also adding extra dots, in this
case, a message I set to a list had a line beginning with:

      ... blah, blah

But it showed up here with two extra dots:

      ..... blah, blah

So when I begin testing why, I noticed the MUA was adding one extra
and the intermediary adding one extra itself.

Is this normal?

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 25 May 2011 10:39:49 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Leading Dots
Sender: [email-masked]

I must be missing something here.  I noticed two MUAs, Outlook and 
Thunderbird adding extra dots for lines I would not expect it would.

For testing, I sent a simple text message with these five lines only:

.. one
... two
.... three
..... four
...... five

At my receiver, a socket read DATA block dump showed the MUAs sending 
an extra dot on each line:

... one
.... two
..... three
...... four
....... five
..                <-- expected <CRLF>.<CRLF> terminator

Why?  Is that correct?

What caught my eye was an intermediary also adding extra dots, in this 
case, a message I set to a list had a line beginning with:

      ... blah, blah

But it showed up here with two extra dots:

      ..... blah, blah

So when I begin testing why, I noticed the MUA was adding one extra 
and the intermediary adding one extra itself.

Is this normal?

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 24 May 2011 08:40:15 +0200
From: Alessandro Vesely <[email-masked]>
To: ietf-smtp list <[email-masked]>
Subject: test (please discard this message)
Sender: [email-masked]

sorry if I bother you with this
this message contains two lines

Date: Mon, 23 May 2011 19:39:19 +0300
From: "A. Rothman" <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
CC: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

If there's no further input, I'll go for the errata. Murray - can I take 
you up on your offer to help with the process?

To summarize:

1. Update the paragraph in section 4.1.2 regarding the use of the 
backslash to explicitly mention it applies only when used in a 
quoted-string.

2. Section 3.5.2 claims the responses from a VRFY or EXPN request must 
include the mailbox name using a "<local-part@domain>" construction - 
this both contradicts section 3.5.1 (which allows a response without the 
pointed brackets), and doesn't make sense on it's own since a 252 
response might not even have anything to return.

and optionally:

3. Seciton 3.5.1 can specify that in particular, using pointed brackets 
without quotes (as is used in MAIL and RCPT commands) is not valid here, 
since this seems to be a common mistake and the wording leaves room for 
confusion (although the ABNF is correct).

-Amichai

Date: Mon, 23 May 2011 09:40:32 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: Alessandro Vesely <[email-masked]>, "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Sun, May 22, 2011 at 10:37 AM, Keith Moore
<[email-masked]> wrote:
>
> On May 22, 2011, at 10:27 AM, Alessandro Vesely wrote:
>
>> On 20/May/11 16:38, Keith Moore wrote:
>>> On May 20, 2011, at 10:32 AM, Jeff Macdonald wrote:
>>>
>>>> Ok, I'll put aside that the receiver is ultimately the one responsible
>>>> for using any 3rd party system. What would the Subject be called? 3rd
>>>> party system? I'll assign x.9.z to those that fit that criteria.
>>>
>>> offhand, I don't have a better name in mind.
>>
>> Hm... "3rd" looks much like a numeric entry in its own right.  How
>> about "external reputation system" or anything more descriptive?
>
> Well, one could always spell it "third party system".    But I think "external reputation system" is probably fine too.

Yes, I like that too.

-- 
Jeff Macdonald
Ayer, MA

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Sun, 22 May 2011 10:37:34 -0400
Cc: "[email-masked]" <[email-masked]>
To: Alessandro Vesely <[email-masked]>
Sender: [email-masked]

On May 22, 2011, at 10:27 AM, Alessandro Vesely wrote:

> On 20/May/11 16:38, Keith Moore wrote:
>> On May 20, 2011, at 10:32 AM, Jeff Macdonald wrote:
>> 
>>> Ok, I'll put aside that the receiver is ultimately the one responsible
>>> for using any 3rd party system. What would the Subject be called? 3rd
>>> party system? I'll assign x.9.z to those that fit that criteria.
>> 
>> offhand, I don't have a better name in mind.
> 
> Hm... "3rd" looks much like a numeric entry in its own right.  How
> about "external reputation system" or anything more descriptive?

Well, one could always spell it "third party system".    But I think "external reputation system" is probably fine too.

Keith

Date: Sun, 22 May 2011 16:27:06 +0200
From: Alessandro Vesely <[email-masked]>
To: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

On 20/May/11 16:38, Keith Moore wrote:
> On May 20, 2011, at 10:32 AM, Jeff Macdonald wrote:
> 
>> Ok, I'll put aside that the receiver is ultimately the one responsible
>> for using any 3rd party system. What would the Subject be called? 3rd
>> party system? I'll assign x.9.z to those that fit that criteria.
> 
> offhand, I don't have a better name in mind.

Hm... "3rd" looks much like a numeric entry in its own right.  How
about "external reputation system" or anything more descriptive?

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Fri, 20 May 2011 10:38:17 -0400
Cc: "Rolf E. Sonneveld" <[email-masked]>, "[email-masked]" <[email-masked]>
To: Jeff Macdonald <[email-masked]>
Sender: [email-masked]

On May 20, 2011, at 10:32 AM, Jeff Macdonald wrote:

> On Fri, May 20, 2011 at 8:05 AM, Keith Moore <[email-masked]> wrote:
>> On May 19, 2011, at 5:29 PM, Jeff Macdonald wrote:
>> 
>>> On Thu, May 19, 2011 at 4:33 PM, Keith Moore <[email-masked]> wrote:
>>>> Because doing so will conflate different sources of the problem.  I don't
>>>> want to overload recipient-specified policy, recipient domain specified
>>>> policy, and blacklist-specified policy.   Those need to be kept separate,
>>>> because they need different fixes.   And it makes more sense to have a
>>>> separate 'subject' for each of these, than to try to distinguish the
>>>> different sources of these problems using different 'detail' codes.
>>> 
>>> So my thinking when I wrote the draft was the "source" was the
>>> anti-spam system. It seems to follow the spirit of the other subject
>>> codes. Like x.4.z means the source is the "Network". When a sender saw
>>> a x.8.z, he would understand the recipient's anti-spam system was
>>> involved. The detail code would tell him what was actually wrong.
>> 
>> Having a subject code for the recipient's anti-spam system is all well and good.  But if the message was rejected not because of the recipient's anti-spam system, but rather, because of a 3rd party blacklist or reputation server, I think there should be a separate subject code for that, because the likely way to fix the problem is going to be to via that blacklist or reputation server rather than via the recipient.
> 
> Ok, I'll put aside that the receiver is ultimately the one responsible
> for using any 3rd party system. What would the Subject be called? 3rd
> party system? I'll assign x.9.z to those that fit that criteria.

offhand, I don't have a better name in mind.

thanks,

Keith

Date: Fri, 20 May 2011 10:32:04 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: "Rolf E. Sonneveld" <[email-masked]>, "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Fri, May 20, 2011 at 8:05 AM, Keith Moore <[email-masked]> wrote:
> On May 19, 2011, at 5:29 PM, Jeff Macdonald wrote:
>
>> On Thu, May 19, 2011 at 4:33 PM, Keith Moore <[email-masked]> wrote:
>>> Because doing so will conflate different sources of the problem.  I don't
>>> want to overload recipient-specified policy, recipient domain specified
>>> policy, and blacklist-specified policy.   Those need to be kept separate,
>>> because they need different fixes.   And it makes more sense to have a
>>> separate 'subject' for each of these, than to try to distinguish the
>>> different sources of these problems using different 'detail' codes.
>>
>> So my thinking when I wrote the draft was the "source" was the
>> anti-spam system. It seems to follow the spirit of the other subject
>> codes. Like x.4.z means the source is the "Network". When a sender saw
>> a x.8.z, he would understand the recipient's anti-spam system was
>> involved. The detail code would tell him what was actually wrong.
>
> Having a subject code for the recipient's anti-spam system is all well and good.  But if the message was rejected not because of the recipient's anti-spam system, but rather, because of a 3rd party blacklist or reputation server, I think there should be a separate subject code for that, because the likely way to fix the problem is going to be to via that blacklist or reputation server rather than via the recipient.

Ok, I'll put aside that the receiver is ultimately the one responsible
for using any 3rd party system. What would the Subject be called? 3rd
party system? I'll assign x.9.z to those that fit that criteria.

-- 
Jeff Macdonald
Ayer, MA

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Fri, 20 May 2011 08:05:53 -0400
Cc: "Rolf E. Sonneveld" <[email-masked]>, "[email-masked]" <[email-masked]>
To: Jeff Macdonald <[email-masked]>
Sender: [email-masked]

On May 19, 2011, at 5:29 PM, Jeff Macdonald wrote:

> On Thu, May 19, 2011 at 4:33 PM, Keith Moore <[email-masked]> wrote:
>> Because doing so will conflate different sources of the problem.  I don't
>> want to overload recipient-specified policy, recipient domain specified
>> policy, and blacklist-specified policy.   Those need to be kept separate,
>> because they need different fixes.   And it makes more sense to have a
>> separate 'subject' for each of these, than to try to distinguish the
>> different sources of these problems using different 'detail' codes.
> 
> So my thinking when I wrote the draft was the "source" was the
> anti-spam system. It seems to follow the spirit of the other subject
> codes. Like x.4.z means the source is the "Network". When a sender saw
> a x.8.z, he would understand the recipient's anti-spam system was
> involved. The detail code would tell him what was actually wrong.

Having a subject code for the recipient's anti-spam system is all well and good.  But if the message was rejected not because of the recipient's anti-spam system, but rather, because of a 3rd party blacklist or reputation server, I think there should be a separate subject code for that, because the likely way to fix the problem is going to be to via that blacklist or reputation server rather than via the recipient.

Also, there's a difference between an anti-spam system that the recipient maintains, and one that the recipient domain maintains on behalf of all of its recipients.   Though I doubt that a lot of detail is needed for recipient-specified criteria.

Of course this begs the question of what subject code should be used when a message is rejected due to a combination of criteria - some based on the message itself, and some based on information obtained externally from a blacklist or reputation server.  And I guess in that case the subject code needs to point to whatever party aggregated that information and made the reject decision.

Keith

Date: Thu, 19 May 2011 17:29:35 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: "Rolf E. Sonneveld" <[email-masked]>, "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Thu, May 19, 2011 at 4:33 PM, Keith Moore <[email-masked]> wrote:
> Because doing so will conflate different sources of the problem.  I don't
> want to overload recipient-specified policy, recipient domain specified
> policy, and blacklist-specified policy.   Those need to be kept separate,
> because they need different fixes.   And it makes more sense to have a
> separate 'subject' for each of these, than to try to distinguish the
> different sources of these problems using different 'detail' codes.

So my thinking when I wrote the draft was the "source" was the
anti-spam system. It seems to follow the spirit of the other subject
codes. Like x.4.z means the source is the "Network". When a sender saw
a x.8.z, he would understand the recipient's anti-spam system was
involved. The detail code would tell him what was actually wrong.

I'm sure there will be other methods for determining spam. I really
don't think we need to break down each of those into separate subject
codes.

-- 
Jeff Macdonald
Ayer, MA

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Thu, 19 May 2011 16:33:48 -0400
Cc: Jeff Macdonald <[email-masked]>, "[email-masked]" <[email-masked]>
To: "Rolf E. Sonneveld" <[email-masked]>
Sender: [email-masked]

--Apple-Mail-156--1034969964
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On May 19, 2011, at 3:43 PM, Rolf E. Sonneveld wrote:

> On 5/19/11 5:44 PM, Jeff Macdonald wrote:
>>=20
>> On Wed, May 11, 2011 at 7:43 AM, Keith Moore =
<[email-masked]> wrote:
>>> I'm not following this thread closely, but I thought I'd say =
something about extended status codes.  Part of the idea of extended =
status codes is that you should be able to determine the likely source =
of the problem by looking at the second facet of the status code.  Or to =
put it another way, the second facet of the status code is supposed to =
indicate _who_ probably needs to fix the problem (e.g. the sender, the =
MSA, a relay, the delivery agent, etc.)
>=20
> =46rom the SMTP transaction point of view there are only two parties =
involved:
>=20
> the SMTP client and
> the SMTP server
Yes, but we're not talking about a code that changes any action to be =
taken by an SMTP agent here.   The only thing that SMTP does with =
respect to enhanced status codes is (if the server supports the =
appropriate SMTP extension) to convey the enhanced status code from the =
server to the client.   The SMTP client looks only at the 3-digit reply =
code, not at the enhanced status code.  Furthermore many enhanced status =
codes aren't passed in SMTP messages at all, but rather in DSNs.   The =
real purpose of the enhanced status code is to report to the sender of =
the message (envelope MAIL FROM or return-path) the specific nature of =
any errors, in a language-independent fashion.

=46rom the sender's point-of-view, there are many places where things =
can go wrong, e.g:
- the sender may have caused the error, perhaps by mistyping a recipient =
address,
- there might be an error in the formatting of the message that prevents =
some intermediary or delivery agent from processing the message, (e.g. a =
mail gateway or a spam filter that looks at MIME body-parts) (note that =
such an error can be caused either by bugs in the sender's MUA or by =
bugs in some sort of gratuitous translator downstream)
- there might be an error in the sender's MUA configuration (e.g. =
incorrect domain name, username, or password for MSA)
- there might be an error in DNS or some other server used to govern =
routing of the message
- there might be an error in mailer configuration in some relay between =
the sender's MSA and the recipient's final delivery agent (e.g. when a =
MX record points to an MTA that doesn't know what to do with mail for =
that domain)
- there might be a resource limitation in some relay (e.g. out of disk =
space), a hardware failure (can't read spool file), or network failure =
preventing an MTA from reaching any next hop
- the message might not meet criteria imposed by the recipient's mail =
domain (e.g. message size)
- the message might not meet criteria specified by the actual recipient =
(e.g. SIEVE filtering)
- the message might not meet criteria specified by some party with no =
direct responsibility to the sender or recipient (e.g. reputation =
servers, blackhole lists)

Each of these requires a different kind of fix, and probably, a =
different party to fix it.

The enhanced status codes aren't perfect, and cannot be.  It will always =
be possible for some conditions to have multiple potential sources.   =
e.g. if an MX record points to an MTA that doesn't know what to do with =
mail for that domain, is the problem in DNS or the MTA configuration? It =
could be either.   But if you're in a position of trying to determine =
why someone's mail isn't working, and how to fix it, it helps if the =
error indications you're getting back make as clear a distinction as =
possible between these different kinds of errors.  So for better or =
worse, the codes are organized in such a way as to attempt to =
distinguish errors according to the likely source of the error.
> So if the intention is to enable systems/people to determine who needs =
to take action to fix something, we only need two 'subject' codes: it's =
either the client that has to fix something, or the server.

See above.

> The server can delegate part of its policy determination process to =
other systems that provide input to it, but on the receiver side the =
SMTP server remains primary responsible entity for the transaction. If =
the SMTP server encounters problems in e.g. DNS blacklist lookups, or =
PTR lookups etc. it can use the 'detail' part of the enhanced status =
code (3rd facet) to indicate what the problem was (room for 1,000 =
different codes). Providing information about a 3rd party in the =
'subject' part of the ESC creates complexity: there is virtually no end =
to the number of parties that can be involved in a single transaction; =
apart from a 3rd party there can be a 4th party and possibly even a 5th =
party and so on. To give two examples:
>=20
> A user at company A (1st party) is sending mail to a user at company B =
(2nd party). Company B is using a hosted AS/AV service, like MessageLabs =
(3rd party) for sending and receiving mail. MessageLabs in turn, enables =
the customer to use one or more public DNS blacklists (4th party) within =
it's Dashboard function. Is it important for the MTA of Company A to =
'know' about the way the infrastructure of company B is designed and =
should Mlabs reflect part of this in the ESC it returns?
>=20
> Company A sends a newsletter mailing on behalf of Company B using a =
Company B sender address. A receiving system at Company C, performs a =
Call Back Verificiation (CBV) to the mail server of Company B. Company B =
in turn is using a DNS Blacklist and finds that the IP address of =
Company B is listed. What ESC code should Company C use in its =
communication with Company B during the SMTP session?
I agree that the number of parties that can be involved in a filtering =
decision is (sadly) unbounded.  But I still think it's useful to =
distinguish between:
"This message was bounced due to configuration of a recipient's MTA" (in =
which case you want to try to get the recipient's MSP to fix the =
problem), vs.=20
"This message was bounced due to criteria provided by a blacklist" (in =
which case you want to try to get the blacklist to fix the problem). =20
In the latter case, going to the recipient's MSP isn't likely to fix the =
problem any time soon.  The MSP might reconsider using the blacklist if =
they get enough complaints about it, but it will take a long time.  =
Meanwhile, they're probably just going to refer people to the blacklist.

Note that I never said I wanted to provide information about _which_ 3rd =
party caused the message to be rejected in the 2nd facet / subject of =
the ESC.   I just think that there should be a single ESC subject code =
reserved for rejections due to 3rd party criteria.

> There is a myriad of possible topologies and interactions here. Let's =
keep it simple: if (repeat: IF) the intent of ESCs is to make clear who =
should take action, use two different subjects, X if the client needs to =
take action and Y if the problem is on the server side. The 'Detail' in =
the ESC can then be used to distinguish between different types of =
problems.

That's true only from a microscopic view involving a single client and a =
single server.   Reality is a lot more complex than that.=20

> As Jeff already mentioned, history has learned us:
>=20
>> So it seems the intent and implementation have actually drifted since
>> Keith came up with the codes.
>=20
> As RFC3463 is widely used it seems to me it's too late to use ESCs to =
indicate which side of the SMTP connection needs to take action.

ESCs were never intended, and should not be used, to dictate any =
behavior for SMTP agents.  They were, and are, intended for senders.

And of course, some amount of drift is inevitable.  But it still makes =
more sense to try to preserve the structure of enhanced status codes =
than to just discard it.

> Why not simply use the x.7.y policy related codes and use the 'y' =
detail to differentiate between the various situations? If we want, we =
can reserve a range within 'y' for spam related reasons, like =
(2|4|5).7.[100-199]. But IMO _documenting_ the various codes in an RFC =
is more important than the actual codes we will use for this purpose.

Because doing so will conflate different sources of the problem.  I =
don't want to overload recipient-specified policy, recipient domain =
specified policy, and blacklist-specified policy.   Those need to be =
kept separate, because they need different fixes.   And it makes more =
sense to have a separate 'subject' for each of these, than to try to =
distinguish the different sources of these problems using different =
'detail' codes.

Keith

--Apple-Mail-156--1034969964
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On May 19, 2011, at 3:43 PM, Rolf E. Sonneveld =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite">
<div bgcolor=3D"#ffffff" text=3D"#000000">
    On 5/19/11 5:44 PM, Jeff Macdonald wrote:
    <blockquote =
cite=3D"mid:BANLkTi=3DONWMKsjOb_ob54G79A=[email-masked]" =
type=3D"cite">
      <pre wrap=3D"">On Wed, May 11, 2011 at 7:43 AM, Keith Moore <a =
class=3D"moz-txt-link-rfc2396E" =
href=3D"mailto:[email-masked]">&lt;[email-masked]&=
gt;</a> wrote:
</pre>
      <blockquote type=3D"cite">
        <pre wrap=3D"">I'm not following this thread closely, but I =
thought I'd say something about extended status codes. &nbsp;Part of the =
idea of extended status codes is that you should be able to determine =
the likely source of the problem by looking at the second facet of the =
status code. &nbsp;Or to put it another way, the second facet of the =
status code is supposed to indicate _who_ probably needs to fix the =
problem (e.g. the sender, the MSA, a relay, the delivery agent, etc.)
</pre>
      </blockquote>
    </blockquote>
    <br>
    =46rom the SMTP transaction point of view there are only two parties
    involved:<br>
    <br>
    <ol>
      <li>the SMTP client and</li>
      <li>the SMTP server</li></ol></div></blockquote>Yes, but we're not =
talking about a code that changes any action to be taken by an SMTP =
agent here. &nbsp; The only thing that SMTP does with respect to =
enhanced status codes is (if the server supports the appropriate SMTP =
extension) to convey the enhanced status code from the server to the =
client. &nbsp; The SMTP client looks only at the 3-digit reply code, not =
at the enhanced status code. &nbsp;Furthermore many enhanced status =
codes aren't passed in SMTP messages at all, but rather in DSNs. =
&nbsp;&nbsp;The real purpose of the enhanced status code is to report to =
the <i>sender</i> of the message (envelope MAIL FROM or return-path) the =
specific nature of any errors, in a language-independent =
fashion.</div><div><br></div><div>=46rom the sender's point-of-view, =
there are many places where things can go wrong, e.g:</div><div>- the =
sender may have caused the error, perhaps by mistyping a recipient =
address,</div><div>- there might be an error in the formatting of the =
message that prevents some intermediary or delivery agent from =
processing the message, (e.g. a mail gateway or a spam filter that looks =
at MIME body-parts) (note that such an error can be caused either by =
bugs in the sender's MUA or by bugs in some sort of gratuitous =
translator downstream)</div><div>- there might be an error in the =
sender's MUA configuration (e.g. incorrect domain name, username, or =
password for MSA)</div><div>- there might be an error in DNS or some =
other server used to govern routing of the message</div><div>- there =
might be an error in mailer configuration in some relay between the =
sender's MSA and the recipient's final delivery agent (e.g. when a MX =
record points to an MTA that doesn't know what to do with mail for that =
domain)</div><div>- there might be a resource limitation in some relay =
(e.g. out of disk space), a hardware failure (can't read spool file), or =
network failure preventing an MTA from reaching any next hop</div><div>- =
the message might not meet criteria imposed by the recipient's mail =
domain (e.g. message size)</div><div>- the message might not meet =
criteria specified by the actual recipient (e.g. SIEVE =
filtering)</div><div>- the message might not meet criteria specified by =
some party with no direct responsibility to the sender or recipient =
(e.g. reputation servers, blackhole lists)</div><div><br></div><div>Each =
of these requires a different kind of fix, and probably, a different =
party to fix it.</div><div><br></div><div>The enhanced status codes =
aren't perfect, and cannot be. &nbsp;It will always be possible for some =
conditions to have multiple potential sources. &nbsp; e.g. if an MX =
record points to an MTA that doesn't know what to do with mail for that =
domain, is the problem in DNS or the MTA configuration? It could be =
either. &nbsp; But if you're in a position of trying to determine why =
someone's mail isn't working, and how to fix it, it helps if the error =
indications you're getting back make as clear a distinction as possible =
between these different kinds of errors. &nbsp;So for better or worse, =
the codes are organized in such a way as to attempt to distinguish =
errors according to the likely source of the =
error.</div><div><blockquote type=3D"cite"><div bgcolor=3D"#ffffff" =
text=3D"#000000"><ol>
    </ol>
   =20
    So if the intention is to enable systems/people to determine who
    needs to take action to fix something, we only need two 'subject'
    codes: it's either the client that has to fix something, or the
    server. </div></blockquote><div><br></div><div>See =
above.</div><div><br></div><blockquote type=3D"cite"><div =
bgcolor=3D"#ffffff" text=3D"#000000">The server can delegate part of its =
policy determination
    process to other systems that provide input to it, but on the
    receiver side the SMTP server remains primary responsible entity for
    the transaction. If the SMTP server encounters problems in e.g. DNS
    blacklist lookups, or PTR lookups etc. it can use the 'detail' part
    of the enhanced status code (3rd facet) to indicate what the problem
    was (room for 1,000 different codes). Providing information about a
    3rd party in the 'subject' part of the ESC creates complexity: there
    is virtually no end to the number of parties that can be involved in
    a single transaction; apart from a 3rd party there can be a 4th
    party and possibly even a 5th party and so on. To give two =
examples:<br>
    <br>
    <ul>
      <li>A user at company A (1st party) is sending mail to a user at
        company B (2nd party). Company B is using a hosted AS/AV
        service, like MessageLabs (3rd party) for sending and receiving
        mail. MessageLabs in turn, enables the customer to use one or
        more public DNS blacklists (4th party) within it's Dashboard
        function. Is it important for the MTA of Company A to 'know'
        about the way the infrastructure of company B is designed and
        should Mlabs reflect part of this in the ESC it returns?<br>
        <br>
      </li>
      <li>Company A sends a newsletter mailing on behalf of Company B
        using a Company B sender address. A receiving system at Company
        C, performs a Call Back Verificiation (CBV) to the mail server
        of Company B. Company B in turn is using a DNS Blacklist and
        finds that the IP address of Company B is listed. What ESC code
        should Company C use in its communication with Company B during
        the SMTP session?<br></li></ul></div></blockquote><div>I agree =
that the number of parties that can be involved in a filtering decision =
is (sadly) unbounded. &nbsp;But I still think it's useful to distinguish =
between:</div><div><ul class=3D"MailOutline"><li>"This message was =
bounced due to configuration of a recipient's MTA" (in which case you =
want to try to get the recipient's MSP to fix the problem), =
vs.&nbsp;</li><li>"This message was bounced due to criteria provided by =
a blacklist" (in which case you want to try to get the blacklist to fix =
the problem). &nbsp;</li></ul><div>In the latter case, going to the =
recipient's MSP isn't likely to fix the problem any time soon. &nbsp;The =
MSP might reconsider using the blacklist if they get enough complaints =
about it, but it will take a long time. &nbsp;Meanwhile, they're =
probably just going to refer people to the =
blacklist.</div><div><br></div><div>Note that I never said I wanted to =
provide information about _which_ 3rd party caused the message to be =
rejected in the 2nd facet / subject of the ESC. &nbsp; I just think that =
there should be a single ESC subject code reserved for rejections due to =
3rd party criteria.</div><div><br></div></div><blockquote =
type=3D"cite"><div bgcolor=3D"#ffffff" text=3D"#000000">
    There is a myriad of possible topologies and interactions here.
    Let's keep it simple: if (repeat: IF) the intent of ESCs is to make
    clear who should take action, use two different subjects, X if the
    client needs to take action and Y if the problem is on the server
    side. The 'Detail' in the ESC can then be used to distinguish
    between different types of =
problems.<br></div></blockquote><div><br></div>That's true only from a =
microscopic view involving a single client and a single server. &nbsp; =
Reality is a lot more complex than =
that.&nbsp;</div><div><br></div><div><blockquote type=3D"cite"><div =
bgcolor=3D"#ffffff" text=3D"#000000">
   =20
    As Jeff already mentioned, history has learned us:<br>
    <br>
    <blockquote type=3D"cite">
      <pre wrap=3D"">So it seems the intent and implementation have =
actually drifted since
Keith came up with the codes.</pre>
    </blockquote>
    <br>
    As RFC3463 is widely used it seems to me it's too late to use ESCs
    to indicate which side of the SMTP connection needs to take =
action.<br></div></blockquote><div><br></div>ESCs were never intended, =
and should not be used, to dictate any behavior for SMTP agents. =
&nbsp;They were, and are, intended for =
senders.</div><div><br></div><div>And of course, some amount of drift is =
inevitable. &nbsp;But it still makes more sense to try to preserve the =
structure of enhanced status codes than to just discard =
it.</div><div><br><blockquote type=3D"cite"><div bgcolor=3D"#ffffff" =
text=3D"#000000">
    Why not simply use the x.7.y policy related codes and use the 'y'
    detail to differentiate between the various situations? If we want,
    we can reserve a range within 'y' for spam related reasons, like
    (2|4|5).7.[100-199]. But IMO _documenting_ the various codes in an
    RFC is more important than the actual codes we will use for this
    purpose.<br>
  </div>

</blockquote><br></div><div>Because doing so will conflate different =
sources of the problem. &nbsp;I don't want to overload =
recipient-specified policy, recipient domain specified policy, and =
blacklist-specified policy. &nbsp; Those need to be kept separate, =
because they need different fixes. &nbsp; And it makes more sense to =
have a separate 'subject' for each of these, than to try to distinguish =
the different sources of these problems using different 'detail' =
codes.</div><div><br></div><div>Keith</div><div><br></div></body></html>=

--Apple-Mail-156--1034969964--

Date: Thu, 19 May 2011 21:43:02 +0200
From: "Rolf E. Sonneveld" <[email-masked]>
To: Jeff Macdonald <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

This is a multi-part message in MIME format.

--Boundary_(ID_grF0aWES+zmhdPmVFNXK6Q)
Content-type: text/plain; CHARSET=US-ASCII; format=flowed
Content-transfer-encoding: 7BIT

On 5/19/11 5:44 PM, Jeff Macdonald wrote:
> On Wed, May 11, 2011 at 7:43 AM, Keith Moore<[email-masked]>  wrote:
>> I'm not following this thread closely, but I thought I'd say something about extended status codes.  Part of the idea of extended status codes is that you should be able to determine the likely source of the problem by looking at the second facet of the status code.  Or to put it another way, the second facet of the status code is supposed to indicate _who_ probably needs to fix the problem (e.g. the sender, the MSA, a relay, the delivery agent, etc.)

 From the SMTP transaction point of view there are only two parties 
involved:

   1. the SMTP client and
   2. the SMTP server

So if the intention is to enable systems/people to determine who needs 
to take action to fix something, we only need two 'subject' codes: it's 
either the client that has to fix something, or the server. The server 
can delegate part of its policy determination process to other systems 
that provide input to it, but on the receiver side the SMTP server 
remains primary responsible entity for the transaction. If the SMTP 
server encounters problems in e.g. DNS blacklist lookups, or PTR lookups 
etc. it can use the 'detail' part of the enhanced status code (3rd 
facet) to indicate what the problem was (room for 1,000 different 
codes). Providing information about a 3rd party in the 'subject' part of 
the ESC creates complexity: there is virtually no end to the number of 
parties that can be involved in a single transaction; apart from a 3rd 
party there can be a 4th party and possibly even a 5th party and so on. 
To give two examples:

    * A user at company A (1st party) is sending mail to a user at
      company B (2nd party). Company B is using a hosted AS/AV service,
      like MessageLabs (3rd party) for sending and receiving mail.
      MessageLabs in turn, enables the customer to use one or more
      public DNS blacklists (4th party) within it's Dashboard function.
      Is it important for the MTA of Company A to 'know' about the way
      the infrastructure of company B is designed and should Mlabs
      reflect part of this in the ESC it returns?

    * Company A sends a newsletter mailing on behalf of Company B using
      a Company B sender address. A receiving system at Company C,
      performs a Call Back Verificiation (CBV) to the mail server of
      Company B. Company B in turn is using a DNS Blacklist and finds
      that the IP address of Company B is listed. What ESC code should
      Company C use in its communication with Company B during the SMTP
      session?

There is a myriad of possible topologies and interactions here. Let's 
keep it simple: if (repeat: IF) the intent of ESCs is to make clear who 
should take action, use two different subjects, X if the client needs to 
take action and Y if the problem is on the server side. The 'Detail' in 
the ESC can then be used to distinguish between different types of problems.

As Jeff already mentioned, history has learned us:

> So it seems the intent and implementation have actually drifted since
> Keith came up with the codes.

As RFC3463 is widely used it seems to me it's too late to use ESCs to 
indicate which side of the SMTP connection needs to take action.

Why not simply use the x.7.y policy related codes and use the 'y' detail 
to differentiate between the various situations? If we want, we can 
reserve a range within 'y' for spam related reasons, like 
(2|4|5).7.[100-199]. But IMO _documenting_ the various codes in an RFC 
is more important than the actual codes we will use for this purpose.

/rolf

--Boundary_(ID_grF0aWES+zmhdPmVFNXK6Q)
Content-type: text/html; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
    <title></title>
  </head>
  <body bgcolor="#ffffff" text="#000000">
    On 5/19/11 5:44 PM, Jeff Macdonald wrote:
    <blockquote
      cite="mid:BANLkTi=ONWMKsjOb_ob54G79A=[email-masked]"
      type="cite">
      <pre wrap="">
On Wed, May 11, 2011 at 7:43 AM, Keith Moore <a class="moz-txt-link-rfc2396E" href="mailto:[email-masked]">&lt;[email-masked]&gt;</a> wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="">I'm not following this thread closely, but I thought I'd say something about extended status codes. &nbsp;Part of the idea of extended status codes is that you should be able to determine the likely source of the problem by looking at the second facet of the status code. &nbsp;Or to put it another way, the second facet of the status code is supposed to indicate _who_ probably needs to fix the problem (e.g. the sender, the MSA, a relay, the delivery agent, etc.)
</pre>
      </blockquote>
    </blockquote>
    <br>
    From the SMTP transaction point of view there are only two parties
    involved:<br>
    <br>
    <ol>
      <li>the SMTP client and</li>
      <li>the SMTP server</li>
    </ol>
    <br>
    So if the intention is to enable systems/people to determine who
    needs to take action to fix something, we only need two 'subject'
    codes: it's either the client that has to fix something, or the
    server. The server can delegate part of its policy determination
    process to other systems that provide input to it, but on the
    receiver side the SMTP server remains primary responsible entity for
    the transaction. If the SMTP server encounters problems in e.g. DNS
    blacklist lookups, or PTR lookups etc. it can use the 'detail' part
    of the enhanced status code (3rd facet) to indicate what the problem
    was (room for 1,000 different codes). Providing information about a
    3rd party in the 'subject' part of the ESC creates complexity: there
    is virtually no end to the number of parties that can be involved in
    a single transaction; apart from a 3rd party there can be a 4th
    party and possibly even a 5th party and so on. To give two examples:<br>
    <br>
    <ul>
      <li>A user at company A (1st party) is sending mail to a user at
        company B (2nd party). Company B is using a hosted AS/AV
        service, like MessageLabs (3rd party) for sending and receiving
        mail. MessageLabs in turn, enables the customer to use one or
        more public DNS blacklists (4th party) within it's Dashboard
        function. Is it important for the MTA of Company A to 'know'
        about the way the infrastructure of company B is designed and
        should Mlabs reflect part of this in the ESC it returns?<br>
        <br>
      </li>
      <li>Company A sends a newsletter mailing on behalf of Company B
        using a Company B sender address. A receiving system at Company
        C, performs a Call Back Verificiation (CBV) to the mail server
        of Company B. Company B in turn is using a DNS Blacklist and
        finds that the IP address of Company B is listed. What ESC code
        should Company C use in its communication with Company B during
        the SMTP session?<br>
      </li>
    </ul>
    <br>
    There is a myriad of possible topologies and interactions here.
    Let's keep it simple: if (repeat: IF) the intent of ESCs is to make
    clear who should take action, use two different subjects, X if the
    client needs to take action and Y if the problem is on the server
    side. The 'Detail' in the ESC can then be used to distinguish
    between different types of problems.<br>
    <br>
    As Jeff already mentioned, history has learned us:<br>
    <br>
    <blockquote type="cite">
      <pre wrap="">So it seems the intent and implementation have actually drifted since
Keith came up with the codes.</pre>
    </blockquote>
    <br>
    As RFC3463 is widely used it seems to me it's too late to use ESCs
    to indicate which side of the SMTP connection needs to take action.<br>
    &nbsp;<br>
    Why not simply use the x.7.y policy related codes and use the 'y'
    detail to differentiate between the various situations? If we want,
    we can reserve a range within 'y' for spam related reasons, like
    (2|4|5).7.[100-199]. But IMO _documenting_ the various codes in an
    RFC is more important than the actual codes we will use for this
    purpose.<br>
    <br>
    /rolf<br>
  </body>
</html>

--Boundary_(ID_grF0aWES+zmhdPmVFNXK6Q)--

Date: Thu, 19 May 2011 12:20:38 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On Wed, May 11, 2011 at 2:44 PM, Steve Atkins <[email-masked]> wrote:
>
> On May 11, 2011, at 11:11 AM, Keith Moore wrote:
>
>> one of the major reasons for enhanced status codes was to provide a language-independent indication of the error.    I certainly don't mind URLs being used to supply more information (especially if done in multiple languages) but I don't think it obviates the need for the enhanced status codes.
>
> I don't think it meets that goal, as it doesn't provide enough information to do anything useful in response to those broad categories. Pretty much all of the suggested categories break down into two groups:
>
>  1. Something went wrong. Keep trying.
>  2. Something went wrong. Read the error message.
>
> t's only worthwhile for an SMTP receiver to send these codes or for an SMTP sender to parse them if they can be used to adjust the behaviour of the sender in a mutually beneficial way.
>
> The codes as defined are broad categories that are informational (as opposed to imperative, e.g. "Please don't send email to this address again", "Please don't send any further email to this address / domain until a human has intervened", "Please reduce the rate at which you are sending email to this MX") so they don't seem to communicate much to the senders automation in terms of "what behaviour you should change in response to this message".

I'm very puzzled. 1) and 2) above is just distilling messages into 5xx
and 4xx. It applies to RFC 3463 as well. But the codes I'm proposing
do communicate what needs to be changed. x.8.2 say "excessive volume".
It addresses the "reduce the rate at which you are sending email".

Is your comment directed at RFC 3463 in general? For instance, x.3.4
Message too big for system, implies that the sender can't send a
message that big. I don't think it needs to say "please reduce message
size for acceptance".

> A sysadmin who ignores the human readable message (whether due to language issues, poor logging or what have you) and relies solely on the extended response code is going to be in just the same situation as automated code - they're not going to have enough information to make a decision about what they should do in response without reading the human readable part of the message.
>
> That's why I'm wondering what the intended use of these codes by the email senders is.

If by senders you mean "Marketers", I would hope better list
management. But for everyone else, less phone calls to the receiver.

-- 
Jeff Macdonald
Ayer, MA

Date: Thu, 19 May 2011 11:44:58 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Wed, May 11, 2011 at 7:43 AM, Keith Moore <[email-masked]> wrote:
> I'm not following this thread closely, but I thought I'd say something about extended status codes.  Part of the idea of extended status codes is that you should be able to determine the likely source of the problem by looking at the second facet of the status code.  Or to put it another way, the second facet of the status code is supposed to indicate _who_ probably needs to fix the problem (e.g. the sender, the MSA, a relay, the delivery agent, etc.)

Thanks Keith for clarification. The RFC calls facet "Subject".

and is section 2 it does say "The second sub-code indicates the
probable source of any delivery anomalies...."

but, well, read on.

>
> When mail is blocked because of information from a party that is not actually involved in transmission (e.g. a blackhole list), it makes sense to have a separate 2nd facet code for this, because (in some sense) it's the blackhole list that has to fix the problem before similar messages can be delivered in the future.   But the list below mixes several different sources in the same facet code.
>

When I first read this, I thought you were saying there really should
be only 3 Subject codes, but looking at RFC 3463, that can't be what
you meant.  I see more than one subject can cover a who. But mixing
several sources into one Subject is currently done in RFC 3463. From
the bottom 2 (I've changed some letters that are present in the RFC to
avoid anti-spam systems from triggering):

x.0.z - Other or Undefined Status
 could be sender or receiver

x.1.z - Addressing Status
 states "_generally_ can be corrected by sender..."
 so who is "sender"

x.2.z - Mailbox Status
 "... _assumed_ to be under the control of the recipient"
 so who is "receiver"

x.3.z - Mail System Status
 "... _assumed_ to be under the general control of the destination ..."
 so who is "receiver"

x.4.z - Network and Routing Status
 "... _assumed_ to be under the control of the destination or
intermediate system ..."
 so who is "receiver" (but could be sender too)

x.5.z - Mail Delivery Protocol Status
 "... the full range of problems ..."
 so who could be either "sender or receiver"

x.6.z - Message Content or Media Status
 " ... media issues are under the control of both the sender and the
receiver ... "
 so who is "sender or receiver"

x.7.z - Security or Policy Status
 "Security policy status issues are assumed to under control of either
or both the sender and recipient"
 so who is "sender or receiver"

So it seems the intent and implementation have actually drifted since
Keith came up with the codes. I suppose x.8.z could be used for when
who is "sender" and x.9.z could be used for when who is "receiver" and
x.10.z for "3rd party". But the word "Subject" or even "who" really
seems to be "what".

-- 
Jeff Macdonald
Ayer, MA

Date: Sat, 14 May 2011 08:48:33 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Jeff Macdonald <[email-masked]>
CC: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Jeff, based on the thread in DKIM

    http://mipassoc.org/pipermail/ietf-dkim/2011q2/016476.html

You might wish to consider adding a new X.8.27 extended status code 
for ADSP related rejects:

    554 5.8.27 Message refused by ADSP From.Domain=<IDENTITY>

If acceptable, I'm sure Murray can help provide the description as it 
relates to DKIM-aware MLM using this 5.8.27 detection to avoid 
creating erroneous automated unsubscribing notifications to members 
with DKIM-ADSP ready mail host receivers.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Jeff Macdonald wrote:
> Hi Folks,
> 
> There is a slight update, where I added a code for greylisting. I've
> also updated the contact information to reflect my current job.
> 
> Draft can be found here:
> 
> https://datatracker.ietf.org/doc/draft-macdonald-antispam-registry/
> 
> Suggested values for SMTP Enhanced Status Codes for Anti-Spam Policy
>                   draft-macdonald-antispam-registry-02
> 
> Abstract
> 
>    This document establishes a set of extended SMTP policy codes for
>    anti-spam.  It seeks to provide additional codes for error texts that
>    currently use the extended SMTP error code 5.7.1.  The anti-spam
>    codes were determined by looking at error texts produced by major
>    ISPs and finding commonalities.  The result is a new set of error
>    texts with associated extended SMTP error codes.
> 
> 
> 
> There is an implementation at my former job:
> 
> $ telnet mail-router.e-dialog.com 25
> Trying 208.94.20.16...
> Connected to mail-router.e-dialog.com.
> Escape character is '^]'.
> 550 5.8.3 Listed on RBL Zen: See http://www.spamhaus.org/query/bl?ip=1.1.1.1
> 
> Here I tried to connect to the inbound MX at e-Dialog, from my home
> over Comcast cable. This is a residential connection so the listing
> makes sense (it is actually a PBL listing).
> I've changed the IP to keep spammers at bay.
> 
> 
> If you've implemented this draft, please let me know.
> 
> 

Date: Thu, 12 May 2011 15:22:56 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: "[email-masked] Group" <[email-masked]>
Subject: Secured User Communications
Sender: [email-masked]

Just a small note about what you noted here:

Keith Moore wrote:

>> Most of the time its for good reasons - product liability, 
>> PCI requirements, etc.  But even then, that hasn't prevented 
>> people in breaking security needs by adding wrappers into their 
>> new user accounts/change passwords to save the plain text somewhere 
>> before a server secured the password storage.
> 
> There is no longer any good reason for using plaintext passwords, 
> particularly .....

True, but most packages do offer option like

    [X] Plain text ONLY over SSL/TLS

I believe the AUTH related RFCs also state this security consideration.

For PCI though, it may not be enough (depending on a few factors) and 
you would need SSL + HTTP Digest with the technically optional NONCE 
required support.

> ... not for customers who have their own domains and can 
> select their own user agents.

In my business and product experience, the opposite is true like in an 
Intranet; host with its own access clients and/or with proprietary 
security and access controls.  Its one reason why we continue to exist 
as a business.  A good example is HIPAA Compliancy requirements for 
SSL or SHA2 secured encrypted communications over the internet - the 
exception?  Direct Dialup! So if a small business, mom and pop shop or 
even mid to large, etc, wanted to get a fast certification entry into 
the medical insurance provider business - you can come to us! :) 
Remember, its not just the host, but their client users device 
capabilities too (doctors, banks, etc).

The times we get request or queries regarding exposing secured 
passwords is when they are doing some form of integration which leads 
the growing industry wide consideration of a "Single Identity" 
authentication concept.

See NSTIC - National Strategy for Trusted Identities in Cyberspace

http://www.nist.gov/nstic

Of course, the key industry vendors pushing and this include the 
marketing vendors - you would buy that "Single Trusted Identity" from 
them.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 12 May 2011 12:05:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: Laura Atkins <[email-masked]>, "[email-masked] Group" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Keith Moore wrote:

>>> All I'm asking for in the context of the current discussion is for 
>>> enhanced status codes to distinguish "rejected because of a 
>>> blacklist or reputation service" codes, from codes that indicate a 
>>> failure in the signal path.
>>
>> That sounds more like an SPF issue, not RBL. no?
>
> it applies to both, though the codes for the two should be distinct 
> (as is proposed).
>
>> X.8.3 seems to cover RBL - "IP listed on RBL RBL-NAME"
>> X.8.12 seems to fit SPF - IDENTITY is an un-delegated IP
> 
> the issue is that the 2nd facet value of '8' is being used both 
> for conditions determined locally and conditions determined by 
> a 3rd party.

I had an earlier draft post (than the final one you had first 
responded to) that suggested a technical reason why this I-D should 
use subject classifications per RFC3463 due to this I-D having a limit 
of 100 details.  But I didn't want to suggest a drastic change

Per RFC3463, what we are calling the 2nd facet formally is the subject:

       status-code = class "." subject "." detail

       class = "2"/"4"/"5"
       subject = 1*3digit
       detail = 1*3digit

With the current classifications for subject:

       X.0.XXX Other or Undefined Status
       X.1.XXX Addressing Status
       X.2.XXX Mailbox Status
       X.3.XXX Mail System Status
       X.4.XXX Network and Routing Status
       X.5.XXX Mail Delivery Protocol Status
       X.6.XXX Message Content or Media Status
       X.7.XXX Security or Policy Status

So technically, this I-D could of used existing subjects allowing for 
more than 100 "Anti-Spam" related reasons (in total).

The I-D is suppose to update RFC3463 with a new specific subject:

       X.8.XXX Anti-Spam policy related

But this suggest a world of AVS issues is limited to 100 reasons. So 
far, the I-D suggest there are 26.

RFC3463 goes into example status details for each of the status 
subjects and many of the I-D anti-spam codes fits there too.  So yes, 
I think that many of the proposed status codes can easily fit within 
existing RFC3463 subjects, if not all or most.

Also, you can see this "conflict" with rev1 of the I-D having a new 
X.8.26 specifically for Greylisting and the current Harris 
recommendation and non-IETF BCP is 4.7.1 which fits the RFC3463 
X.7.XXX security or policy status subject.

So one has to ask what are the motivations for the I-D X.8.XXX status 
subject?

Here is what I think:

   1.0 - Consolidation:

      1.1 - Definition, registration, limited to 100
      1.2 - Easier to Trace/Search, i.e. logs, grep "4.8.26"

   2.0 - Automation:

      2.1 - Reporting  (helping/blaming someone, i.e. DSN)
      2.2 - Local Notification (security, signaling local network)

The I-D is trying to formally do 1.1, which is fine, with the intent 
to help with 2.1.

But you have two issues:

   Most of the (sensitive) reporting is for the wrong people, and
   most likely won't reach them (false bounce), and

   Jeff's ultimate question which started this thread:

         Why aren't people using it?

My view is its more than just IETF registration of a new special 
subject with 26 details limited to 100.  Today, it needs to provide 
how the special subject helps the current process beyond just 
reporting because it could be mostly false targeted reporting.  I like 
the Alert Status idea because this special subject would have a higher 
local network interest.   I don't think it will help to have something 
like:

     X.8.11 <IDENTITY> has been compromised
     Pass the Buck: USER?
                    LOCAL HOST?
                    WHO?

If you look at the description:

     It has been determined by the receiver (or a 3rd
     party the receiver is using) that the IDENTITY
     (an IP address or Domain) has been compromised.
     This is usually and indication that the machine
     hosting the IP is infected with a virus or is
     part of a Zombie network.  Administrators MAY
     include a URL for further details by appending
     the text with ": see <URL> for further details."

It seem to missed out that the IDENTITY could be the AUTH user.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 12 May 2011 08:37:33 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: SMTP Interest Group <[email-masked]>
Subject: Re: abnf for <atext> in RFC 5321?
Sender: [email-masked]

On 5/11/2011 3:06 PM, Ned Freed wrote:
> Exactly. I was a little surprised we don't have something about the use
> of ABNF in the conventions section. I fully support putting that in and moving
> the statement about where to find missing productions there as well.

In recent years, I've been coming to the view that a document's statements about 
imported definitions, rules, or the like should explicitly provide the details 
of what is being imported, and from where.

So a statement might be along the lines of:

      Terms used in this document that are defined in RFC AAAA:

         term1
         term2

      ABNF rules imported from RFC XXXX:

         rule1
         rule2

      ABNF rules imported from RFC YYYY:

         rule3
         rule4

This means that a search of the document, to find the definition of a term or 
rule or whatever will find this explicit text.  No guessing or additional 
searching will be required.

At the same time, it does not suffer the dangers of repeated the actual text of 
of the definition, with its potential for divergence.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Thu, 12 May 2011 07:10:20 -0400
Cc: Laura Atkins <[email-masked]>, "[email-masked] Group" <[email-masked]>
To: Hector Santos <[email-masked]>
Sender: [email-masked]

On May 12, 2011, at 12:38 AM, Hector Santos wrote:

> Keith Moore wrote:
> 
>> (But overall, the state of commercial
>> mail service providers seems to be pretty sad.  I still haven't found one, for instance, that will let me disable all use of unencrypted passwords.)
> 
> Pretty Sad because you can't do this? hmmmm

that's just one example that I find particularly irritating.    part of the issue is that too many MUAs will try using plaintext passwords (without asking) if password-over-SSL fails for whatever reason, thus exposing the password in cleartext.  this is a disaster on a typical wireless network.

> Most of the time its for good reasons - product liability, PCI requirements, etc.  But even then, that hasn't prevented people in breaking security needs by adding wrappers into their new user accounts/change passwords to save the plain text somewhere before a server secured the password storage.

There is no longer any good reason for using plaintext passwords, particularly not for customers who have their own domains and can select their own user agents.

>>> With all of that being said, I think this is out of scope for this particular protocol discussion.
>> I don't think that whether a domain can use blacklists is open to question.  We've always said that an MTA can reject any message for any reason.  
> 
> True, but it would be pretty sad if receivers reject mail without good reason. RBL exist because it provides a payoff to filter the abusive side of mail.  If it didn't, people won't use it.

I'm not going to argue about why RBL exists.

>> I personally think that rejecting messages based on source IP address (regardless of the reason) makes for a lousy heuristic, one which will get worse over time (especially in IPv4 with the increasing use of multi-layered NATs).   
> 
> The only issue I've see with NATs is with EHLO [ip-literal] enforcement. For example, (LAN/WAN/Wireless) private IP network client use EHLO [local-mach-ip] and the peer connection IP is that of the public IP NAT machine - an SMTP violation.

some ISPs are already NATting several customers behind a single address or address block, so that the source address of a TCP connection is no longer uniquely associated with only one customer at a time.  this will only get worse over time because IPv4 addresses have been exhausted.  one implication is that any attempt to associate a reputation with an IP address, even temporarily, becomes increasingly dubious.

>> All I'm asking for in the context of the current discussion is for enhanced status codes to distinguish "rejected because of a blacklist or reputation service" codes, from codes that indicate a failure in the signal path.
> 
> That sounds more like an SPF issue, not RBL. no?

it applies to both, though the codes for the two should be distinct (as is proposed).

> X.8.3 seems to cover RBL - "IP listed on RBL RBL-NAME"
> X.8.12 seems to fit SPF - IDENTITY is an un-delegated IP

the issue is that the 2nd facet value of '8' is being used both for conditions determined locally and conditions determined by a 3rd party.

Keith

Date: Thu, 12 May 2011 00:38:12 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: Laura Atkins <[email-masked]>, "[email-masked] Group" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Keith Moore wrote:

> (But overall, the state of commercial
> mail service providers seems to be pretty sad.  I still haven't found one, 
> for instance, that will let me disable all use of unencrypted passwords.)

Pretty Sad because you can't do this? hmmmm

Most of the time its for good reasons - product liability, PCI 
requirements, etc.  But even then, that hasn't prevented people in 
breaking security needs by adding wrappers into their new user 
accounts/change passwords to save the plain text somewhere before a 
server secured the password storage.

>> With all of that being said, I think this is out of scope for 
>> this particular protocol discussion.
> 
> I don't think that whether a domain can use blacklists is open to 
> question.  We've always said that an MTA can reject any message for 
> any reason.  

True, but it would be pretty sad if receivers reject mail without good 
reason. RBL exist because it provides a payoff to filter the abusive 
side of mail.  If it didn't, people won't use it.

> I personally think that rejecting messages based on source IP 
> address (regardless of the reason) makes for a lousy heuristic, 
> one which will get worse over time (especially in IPv4 with 
> the increasing use of multi-layered NATs).   

The only issue I've see with NATs is with EHLO [ip-literal] 
enforcement. For example, (LAN/WAN/Wireless) private IP network client 
use EHLO [local-mach-ip] and the peer connection IP is that of the 
public IP NAT machine - an SMTP violation.

> All I'm asking for in the context of the current discussion is 
> for enhanced status codes to distinguish "rejected because of a 
> blacklist or reputation service" codes, from codes that 
> indicate a failure in the signal path.

That sounds more like an SPF issue, not RBL. no?

X.8.3 seems to cover RBL - "IP listed on RBL RBL-NAME"
X.8.12 seems to fit SPF - IDENTITY is an un-delegated IP

-- 
Sincerely

Hector Santos
http://www.santronics.com

Cc: Laura Atkins <[email-masked]>, "[email-masked] Group" <[email-masked]>
From: Al Iverson <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Date: Wed, 11 May 2011 23:02:08 -0500
To: Keith Moore <[email-masked]>
Sender: [email-masked]

Sorry, is this on topic?

--
Al Iverson | Chicago, IL | (312) 725-0130
www.aliverson.com | twitter.com/aliverson

On May 11, 2011, at 10:31 PM, Keith Moore <[email-masked]> wrote:

> 
> 
> On May 11, 2011, at 7:53 PM, Laura Atkins wrote:
> 
>> 
>> 
>> On May 11, 2011, at 3:15 PM, John C Klensin wrote:
>> 
>>> I may be wrong, but I don't think either Keith or I are saying
>>> "don't use blacklists in your systems".  Keith, I think, comes
>>> closer than I do.   For all you know, I might even be using
>>> blacklists as part of a scoring system (and I'm not telling).
>>> What I think we are saying is:
>>> 
>>>    (i) Don't force me to use blacklists or otherwise force
>>>    them on me or my mail.
>> 
>> No one forces you to use a blacklist or any filtering. Yes, it may be difficult to find a commercial provider that doesn't run some sort of blacklist, whether internally or externally maintained. However, you always have the option to run your own mailserver and not check mail against any blacklists.  For instance, we don't want mail blocked due to blocklists so we run our own server.
> 
> It's pretty impractical for most people to run their own servers.  It takes a large amount of expertise.  Fortunately, there are still some commercial providers that don't run a blacklist (or give you the option to turn it off for your incoming mail).  (But overall, the state of commercial mail service providers seems to be pretty sad.  I still haven't found one, for instance, that will let me disable all use of unencrypted passwords.)
> 
>> With all of that being said, I think this is out of scope for this particular protocol discussion. 
> 
> I don't think that whether a domain can use blacklists is open to question.  We've always said that an MTA can reject any message for any reason.  I personally think that rejecting messages based on source IP address (regardless of the reason) makes for a lousy heuristic, one which will get worse over time (especially in IPv4 with the increasing use of multi-layered NATs).   The spammers have long since figured out how to get around IP-based filters.   But I won't pretend that we can talk people out of using them. 
> 
> All I'm asking for in the context of the current discussion is for enhanced status codes to distinguish "rejected because of a blacklist or reputation service" codes, from codes that indicate a failure in the signal path.
> 
> Keith
> 
> 

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 23:31:03 -0400
Cc: "[email-masked] Group" <[email-masked]>
To: Laura Atkins <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 7:53 PM, Laura Atkins wrote:

> 
> 
> On May 11, 2011, at 3:15 PM, John C Klensin wrote:
> 
>> I may be wrong, but I don't think either Keith or I are saying
>> "don't use blacklists in your systems".  Keith, I think, comes
>> closer than I do.   For all you know, I might even be using
>> blacklists as part of a scoring system (and I'm not telling).
>> What I think we are saying is:
>> 
>> 	(i) Don't force me to use blacklists or otherwise force
>> 	them on me or my mail.
> 
> No one forces you to use a blacklist or any filtering. Yes, it may be difficult to find a commercial provider that doesn't run some sort of blacklist, whether internally or externally maintained. However, you always have the option to run your own mailserver and not check mail against any blacklists.  For instance, we don't want mail blocked due to blocklists so we run our own server.

It's pretty impractical for most people to run their own servers.  It takes a large amount of expertise.  Fortunately, there are still some commercial providers that don't run a blacklist (or give you the option to turn it off for your incoming mail).  (But overall, the state of commercial mail service providers seems to be pretty sad.  I still haven't found one, for instance, that will let me disable all use of unencrypted passwords.)

> With all of that being said, I think this is out of scope for this particular protocol discussion. 

I don't think that whether a domain can use blacklists is open to question.  We've always said that an MTA can reject any message for any reason.  I personally think that rejecting messages based on source IP address (regardless of the reason) makes for a lousy heuristic, one which will get worse over time (especially in IPv4 with the increasing use of multi-layered NATs).   The spammers have long since figured out how to get around IP-based filters.   But I won't pretend that we can talk people out of using them. 

All I'm asking for in the context of the current discussion is for enhanced status codes to distinguish "rejected because of a blacklist or reputation service" codes, from codes that indicate a failure in the signal path.

Keith

Date: Wed, 11 May 2011 22:36:47 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Peter Bowen <[email-masked]>
CC: [email-masked]
Subject: Re: LOGIN mechanism
Sender: [email-masked]

Peter Bowen wrote:
> I realize that SASL mechanisms are not directly in the domain of this
> group, but I'm hoping I might get pointed the right way.
> 
> I've been working on implementing a framework for building SMTP
> servers in ruby (https://github.com/pzb/groat-smtpd).  One of the
> issues I've run into several times is interoperability of the LOGIN
> mechanism with various clients.  The latest problem is certain
> Microsoft SMTP implementations that send an initial client response,
> and have major issues when it is not accepted.  Is there any reliable
> specification for the LOGIN mechanism and/or how it is used with SMTP?
> 
> Thanks,
> Peter

Hi Peter,

Is there an specific example of the problem to get an idea of what you 
seeing?

The specification for SASL is pretty straight forward. The documents 
to read should be:

    RFC4422 Simple Authentication and Security Layer (SASL)
    RFC2554 SMTP Service Extension for Authentication
    RFC2487 SMTP Service Extension for Secure SMTP over TLS
    RFC4409 Message Submission for Mail

But there are a few something you need to know that you may come 
across outside the docs or not obvious.

One specifically for LOGIN method is to display it twice like so:

   C: EHLO client.domain.com
   S: 250-server.com, whats up!
   S: 250-SIZE 10240000
   S: 250-AUTH CRAM-MD5 LOGIN DIGEST-MD5
   S: 250-AUTH=LOGIN
   S: 250 HELP

Some older clients look for that AUTH=LOGIN.

Also, LOGIN can be done in two forms

     AUTH LOGIN [base64_username]
     AUTH LOGIN

So your server needs to be ready for optional data.

Also, for the handshaking exchange, due to various client behavior 
(mostly cell I found), some pump multiple lines and if the server need 
to properly parse it on a per line basis (i.e. the next line reading 
comes off an input cache)

Finally, if you are just looking for LOGIN, which should be the 
minimum method, its pretty simple.  Here is some (older version( C/C++ 
code for the AUTH command for the LOGIN method only.  I pulled out the 
newer multi SASL methods version that interacts with a backend server 
in the handshake.

BOOL TSMTPServer::AUTH(char *args)
{

     if (!IsServiceAvailable()) return TRUE;

     if (!CheckTLS())return TRUE;

     if (!IsExtendedSmtp) {
        Send("500 AUTH not enabled at server!\r\n");
        return TRUE;
     }

     if (authattempts > 3) {
        Send("503 Too many AUTH attempts!\r\n");
        return TRUE;
     }

     if (IsUserAuthorized) {
        Send("503 Invalid AUTH request. Already Authenticated\r\n");
        return TRUE;
     }

     // Support for NON-RFC AUTH=LOGIN
     // - Designed by NETSCAPE, SUPPORTED by MICROSOFT
     // - supported by at least NS4, NS4.5, OL4, OL5
     // - optional: AUTH LOGIN [base64_username]
     // - optional: AUTH LOGIN

     char buf[1024];
     ZeroMemory(&username,sizeof(username));
     ZeroMemory(&password,sizeof(password));

     if (!strnicmp(args, "LOGIN", 5)) {

         int received = 0;
         int sockerr  = 0;

         ZeroMemory(&buf,sizeof(buf));

         //----------------------------------------------------------
         // Get USERNAME
         //----------------------------------------------------------

         // Some clients will provide the BASE64 username as part
         // of the LOGIN argument. Otherwise, ask for the username.

         if (strlen(args) > 6) {
             strcpy(buf, args+6);
             UnBase64(username, buf);
         }
         else {

             Send("334 VXNlcm5hbWU6====\r\n");

             ZeroMemory(&buf,sizeof(buf));
             received = Control->GetBlock(buf,sizeof(buf)-1);
             sockerr  = GetLastError();
             if ((received <= 0) && (sockerr != 0)) {
               Send("454 Failure to receive AUTH USERNAME. Error 
%d\r\n",sockerr);
               return TRUE;
             }

             if (received <= 0) {
                Send("535 Authorization Failed, No Username!\r\n");
                authattempts++;
                return TRUE;
             }

             UnBase64(username, buf);

             // 450.3b7 - just in case it was encoded
             if (!strcmp(buf,"*") || !strcmp(username,"*")) {
                Send("501 Authorization cancellation accepted\r\n");
                authattempts++;
                return TRUE;
             }
             //

         } // user name

         //----------------------------------------------------------
         // Get PASSWORD
         //----------------------------------------------------------

         Send("334 UGFzc3dvcmQ6====\r\n");

         ZeroMemory(&buf,sizeof(buf));
         received = Control->GetBlock(buf,sizeof(buf)-1);
         sockerr  = GetLastError();
         if ((received <= 0) && (sockerr != 0)) {
           Send("454 Failure to receive AUTH PASSWORD. Error 
%d\r\n",sockerr);
           return TRUE;
         }

         if (received <= 0) {
            Send("535 Authorization Failed, No Password!\r\n");
            authattempts++;
            return TRUE;
         }

         TrimCrLf(buf);
         UnBase64(password, buf);

         if (!strcmp(buf,"*") || !strcmp(password,"*")) {
             Send("501 Authorization cancellation accepted\r\n");
             authattempts++;
             return TRUE;
         }

         IsUserAuthorized = YourServerSideLogin(username,password);

         if (!IsUserAuthorized)
         {
            int err = GetLastError();
            switch (err) {
              case WC_USER_NOT_FOUND:
                 Send("535 Authentication Failure: User \"%s\" 
Unknown!\r\n",username);
                 break;
              case WC_INCORRECT_PASSWORD:
                 Send("535 Authentication Failure: Invalid 
Password!\r\n");
                 break;
              case WC_ACCESS_DENIED:
                 Send("535 Authentication Failure: Email Access 
Denied!\r\n");
                 break;
              default:
                 Send("535 System Authentication Failure %8X!\r\n",err);
                 break;
            }
            authattempts++;
            InterlockedIncrement(&smtpstats.ConnectRejects);
            return TRUE;
         }
        Send("235 Authentication successful\r\n");
        return TRUE;
     }

     Send("504 Unrecognized authentication type: %s.\r\n",args);
     authattempts++;
     return TRUE;
}

Hope this helps

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 20:52:10 -0400
Subject: LOGIN mechanism
From: Peter Bowen <[email-masked]>
To: [email-masked]
Sender: [email-masked]

I realize that SASL mechanisms are not directly in the domain of this
group, but I'm hoping I might get pointed the right way.

I've been working on implementing a framework for building SMTP
servers in ruby (https://github.com/pzb/groat-smtpd).  One of the
issues I've run into several times is interoperability of the LOGIN
mechanism with various clients.  The latest problem is certain
Microsoft SMTP implementations that send an initial client response,
and have major issues when it is not accepted.  Is there any reliable
specification for the LOGIN mechanism and/or how it is used with SMTP?

Thanks,
Peter

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Laura Atkins <[email-masked]>
Date: Wed, 11 May 2011 16:53:39 -0700
Cc: "[email-masked] Group" <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 3:15 PM, John C Klensin wrote:

> I may be wrong, but I don't think either Keith or I are saying
> "don't use blacklists in your systems".  Keith, I think, comes
> closer than I do.   For all you know, I might even be using
> blacklists as part of a scoring system (and I'm not telling).
> What I think we are saying is:
> 
> 	(i) Don't force me to use blacklists or otherwise force
> 	them on me or my mail.

No one forces you to use a blacklist or any filtering. Yes, it may be difficult to find a commercial provider that doesn't run some sort of blacklist, whether internally or externally maintained. However, you always have the option to run your own mailserver and not check mail against any blacklists.  For instance, we don't want mail blocked due to blocklists so we run our own server.

With all of that being said, I think this is out of scope for this particular protocol discussion. 

laura 

-- 
Laura Atkins
Word to the Wise			"The Deliverability Experts!"
Direct: 650 678-3454		Fax: 650 249-1909
AIM: wttwlaura			YIM: wttw_laura
Delivery blog: <http://blog.wordtothewise.com/>

Date: Wed, 11 May 2011 19:15:43 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Murray S. Kucherawy wrote:
>> -----Original Message-----

>> It all sound pretty similar too me - the end result 
>> is the same.
> 
> I like the idea of having an indication in the status 
> code of which actor involved is the one most likely to 
> be able to solve the problem, with more granularity 
> than is there now.
> 
> I dislike the idea of some normative indication about how 
> loud to sound the alarm bell.
> 
> Is that more clear?

Nope, due to the inconsistency in the methodology here (and in other 
places). Pointing fingers is equally as loud and with an overall goal 
(blame game) quite often unrealistic in the real world.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 19:04:51 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

FWIW, Some interesting for RBL.

Starting in 2003, RBL was by far the highest in filtering consistently 
averaging around 60-70%, but then it began to drop:

   2006 to 40-50%
   2009 to 20-30%
   2011 to 10-15%!

What has consistently remained over 60-70% over the 8 years is Return 
Path CBV (Call Back Verification), as well consistently over the years 
of 50-60% for RCPT TO (invalid local recipients checks).

SPF was always low around 1% but slowly climbed to ~6% today.

With better measurements at the DATA level this year, I'm seeing ~30% 
content rejects based on large database of spam words checks

John C Klensin wrote:
> 
> 
> --On Wednesday, May 11, 2011 20:36 +0000 John Levine
> <[email-masked]> wrote:
> 
>>> But, as far as "harm" goes, non-delivery of messages that
>>> should have been delivered and instead vanished without a
>>> trace is, for me, qualitatively worse than any quantity of
>>> spam.
>> With a sufficient quantity of spam in your mailbox, you can't
>> even find the real mail so it might as well not have been
>> delivered.  BTDT.
> 
> John,
> 
> This may be just a difference in perspective, but I see
> tradeoffs between my risk tolerance for losing messages and my
> desire to avoid having my mailbox filled with noise.  If I get
> the virtual knob turned all the way to one side, I lose all of
> the messages that might possibly be spam, i.e., all of the
> messages unless I accept only signed messages from people whose
> ability to manage keys I trust.  If it turn it too far the other
> way, I will have a lot of trouble finding real messages because
> of the spam.  My choice, my problem.  And I assume that you
> wouldn't try to deny me  (or Keith) the right to make tradeoff
> decisions even if you think they are stupid (or that my concern
> about lost messages is excessive).
> 
> We hit a disconnect (I think), only when you (or others)
> encourage some mail providers to reject or, worse, drop, my
> legitimate mail (for the purposes of this discussion, assume
> non-commercial, small number of recipients, and desired/
> expected by the proposed recipient) because the sender did not
> meet some semi-arbitrary criterion established by that provider
> and not under the control of either the sender or (especially)
> the intended recipient.
> 
> My definition of "semi-arbitrary" perhaps also differs from
> yours.  I would include in that category any rule that depends
> on heuristics (which, by definition, will fail sometimes) and
> any classification system that is not clearly identified and
> against which the intended recipient of the message does not
> have practice and effective recourse if the owners of the
> classification system get things wrong.  
> 
> I may be wrong, but I don't think either Keith or I are saying
> "don't use blacklists in your systems".  Keith, I think, comes
> closer than I do.   For all you know, I might even be using
> blacklists as part of a scoring system (and I'm not telling).
> What I think we are saying is:
> 
> 	(i) Don't force me to use blacklists or otherwise force
> 	them on me or my mail.
> 	
> 	(ii) If someone does use a blacklist and legitimate mail
> 	is dropped, both the list compiler and the party who
> 	decided to use that particular list and how should be
> 	easily identified and accountable.
> 
> I don't see blacklists as being special in that regard.  YOu
> could substitute any technique for preventing some messages from
> showing up in my primary inbox for "blacklist" in the above two
> comments and I would almost certainly still agree.
> 
>   best,
>    john
> 
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Content-class: urn:content-classes:message
Subject: RE: slight update to draft-macdonald-antispam-registry
Date: Wed, 11 May 2011 18:58:47 -0400
From: "MH Michael Hammer (5304)" <[email-masked]>
To: "John C Klensin" <[email-masked]>, "John Levine" <[email-masked]>, <[email-masked]>
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:owner-ietf-
> [email-masked]] On Behalf Of John C Klensin
> Sent: Wednesday, May 11, 2011 6:16 PM
> To: John Levine; [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> 
> 
> 
> --On Wednesday, May 11, 2011 20:36 +0000 John Levine
> <[email-masked]> wrote:
> 
> >
> >> But, as far as "harm" goes, non-delivery of messages that
> >> should have been delivered and instead vanished without a
> >> trace is, for me, qualitatively worse than any quantity of
> >> spam.
> >
> > With a sufficient quantity of spam in your mailbox, you can't
> > even find the real mail so it might as well not have been
> > delivered.  BTDT.
> 
> John,
> 
> This may be just a difference in perspective, but I see
> tradeoffs between my risk tolerance for losing messages and my
> desire to avoid having my mailbox filled with noise.  If I get
> the virtual knob turned all the way to one side, I lose all of
> the messages that might possibly be spam, i.e., all of the
> messages unless I accept only signed messages from people whose
> ability to manage keys I trust.  If it turn it too far the other
> way, I will have a lot of trouble finding real messages because
> of the spam.  My choice, my problem.  And I assume that you
> wouldn't try to deny me  (or Keith) the right to make tradeoff
> decisions even if you think they are stupid (or that my concern
> about lost messages is excessive).
>

Reality check - It is NOT all about you... and I'm saying this as
someone who has primarily worked in the mail space for organizations
that are "sender side". Mailbox providers of whatever size or stripe
will make the decisions on handling mail that work for them (I have been
waiting for Mr. Levine to invoke King Canute). Individuals in most parts
of the world have a choice of mailbox providers. If a provider is
dropping what endusers perceive as legitimate email on a somewhat
consistent basis I would expect the endusers to find another provider or
raise a fuss. What each individual user chooses to do with mail directed
to them and reaching their individual control is their personal
business.

> We hit a disconnect (I think), only when you (or others)
> encourage some mail providers to reject or, worse, drop, my
> legitimate mail (for the purposes of this discussion, assume
> non-commercial, small number of recipients, and desired/
> expected by the proposed recipient) because the sender did not
> meet some semi-arbitrary criterion established by that provider
> and not under the control of either the sender or (especially)
> the intended recipient.
> 

What you consider legitimate someone else may consider to be problematic
or abusive. As I indicated above, individuals have a plethora of choices
when it comes to mailboxes and many of those choices are free.

> My definition of "semi-arbitrary" perhaps also differs from
> yours.  I would include in that category any rule that depends
> on heuristics (which, by definition, will fail sometimes) and
> any classification system that is not clearly identified and
> against which the intended recipient of the message does not
> have practice and effective recourse if the owners of the
> classification system get things wrong.
> 
> I may be wrong, but I don't think either Keith or I are saying
> "don't use blacklists in your systems".  Keith, I think, comes
> closer than I do.   For all you know, I might even be using
> blacklists as part of a scoring system (and I'm not telling).
> What I think we are saying is:
> 
> 	(i) Don't force me to use blacklists or otherwise force
> 	them on me or my mail.
> 

If it is going to their system then it is their choice. I recognize this
as a sender (for various domains that range from small amounts of mail
to large amounts of mail) and have very rarely encountered problems from
arbitrary mailbox provider behavior. When it has happened my experience
has been that the mailbox provider is interested in addressing the
issue. This is of course provided that they are not receiving complaints
from their enduser base about that mail.

> 	(ii) If someone does use a blacklist and legitimate mail
> 	is dropped, both the list compiler and the party who
> 	decided to use that particular list and how should be
> 	easily identified and accountable.
> 

Held accountable by whom? You as an emitter of mail? Perhaps we should
implement public floggings? You could of course refuse to send mail to
that system and try to get others to join you in the hopes that the
offensive behavior will be modified.

> I don't see blacklists as being special in that regard.  YOu
> could substitute any technique for preventing some messages from
> showing up in my primary inbox for "blacklist" in the above two
> comments and I would almost certainly still agree.
> 

Mailbox providers will do what mailbox providers choose to do. There are
things I can control (what I do as an emitter of mail) and some things I
can't control (what receivers choose to implement). 

The reality is that there are large numbers of mail emitters
figuratively and in reality knocking on the doors of postmaster teams
trying to get their mail accepted because "they are nice people" or
"people really really want my mail". The reality is that most senders
believe their mail to be much more important than the intended
recipients perceive it. 

Mike

Date: Wed, 11 May 2011 15:28:59 -0700
To: [email-masked], [email-masked]
From: SM <[email-masked]>
Subject: Re: abnf for <atext> in RFC 5321?
Cc: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

Hi Dave, Ned,
At 14:13 11-05-2011, Dave CROCKER wrote:
>yup.  it /is/ showing up as an issue in multiple venues.

I vaguely recall a discussion about this before.  I couldn't find a 
reference to it.  The text that Steve Atkins posted rings a bell.

May I suggest that you both repost your messages to the YAM mailing 
list so that this can be tracked as an issue?  It can be considered 
when the document is processed.

Regards,
-sm 

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Wed, 11 May 2011 15:36:04 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Wednesday, May 11, 2011 3:25 PM
> To: [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> You seem to reject the whole idea which is funny since this tangent
> thread started as a Moore response to the basic idea of the status
> code parts to direct actionable "blame" (client vs receiver) and you
> agreed and followed up:
> 
>     "Perhaps the proposal would benefit from making
>      such distinctions in the new codes it's registering."
> 
> It all sound pretty similar too me - the end result is the same.

I like the idea of having an indication in the status code of which actor involved is the one most likely to be able to solve the problem, with more granularity than is there now.

I dislike the idea of some normative indication about how loud to sound the alarm bell.

Is that more clear?

Date: Wed, 11 May 2011 18:30:58 -0400
From: John C Klensin <[email-masked]>
To: John Levine <[email-masked]>, [email-masked]
cc: [email-masked]
Subject: Re: abnf for <atext> in RFC 5321?
Sender: [email-masked]

--On Wednesday, May 11, 2011 21:53 +0000 John Levine
<[email-masked]> wrote:

>...
> RFC 5322 has the definition of atext, so there's no dangling
> reference.  This is at most an editorial issue, where it might
> have been a good idea to be more specific about what symbols
> are imported from where, so those of us doing a grep for
> "atext" would find it.

And, if we ever get around to doing 5321bis ("two maturity
levels" is, in some respects, an argument against doing so in
the absence of significant errors), explicit identification of
all production definitions are imported from other documents,
and which documents they are imported from, is already on the
list.

Importing those things from (now) 5322, rather than duplicating
them, was an explicit decision made during the DRUMS period.
One can, of course, debate whether that was the right decision.
But, it is, somewhat more specific identification of sources
than is provided by Section 4.1.2 would be useful to some people
and harmless to others (as long as we do it correctly).  It was
actually suggested during the IETF Last Call on 5321; we decided
to not do it at that stage because we were concerned that doing
so at that late point in the process would introduce errors.

    john

Date: Wed, 11 May 2011 18:25:02 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Murray S. Kucherawy wrote:

> This all talks about software design and operation.  The 
> proposal to which I'm responding said "notify an operator".  
> There's a gigantic difference between the two in terms of 
> what's appropriate for a standards track RFC and what is not.

Disagree.

First, there was no formal proposal here. It was a starter suggestion 
for an Alert Status idea, i.e. LOW, MED, HIGH, etc and I provided an 
example using one code with a HIGH alert and what that could mean 
based on severity level, i.e. Notify Local Operator.

You seem to reject the whole idea which is funny since this tangent 
thread started as a Moore response to the basic idea of the status 
code parts to direct actionable "blame" (client vs receiver) and you 
agreed and followed up:

    "Perhaps the proposal would benefit from making
     such distinctions in the new codes it's registering."

It all sound pretty similar too me - the end result is the same.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 18:15:39 -0400
From: John C Klensin <[email-masked]>
To: John Levine <[email-masked]>, [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

--On Wednesday, May 11, 2011 20:36 +0000 John Levine
<[email-masked]> wrote:

> 
>> But, as far as "harm" goes, non-delivery of messages that
>> should have been delivered and instead vanished without a
>> trace is, for me, qualitatively worse than any quantity of
>> spam.
> 
> With a sufficient quantity of spam in your mailbox, you can't
> even find the real mail so it might as well not have been
> delivered.  BTDT.

John,

This may be just a difference in perspective, but I see
tradeoffs between my risk tolerance for losing messages and my
desire to avoid having my mailbox filled with noise.  If I get
the virtual knob turned all the way to one side, I lose all of
the messages that might possibly be spam, i.e., all of the
messages unless I accept only signed messages from people whose
ability to manage keys I trust.  If it turn it too far the other
way, I will have a lot of trouble finding real messages because
of the spam.  My choice, my problem.  And I assume that you
wouldn't try to deny me  (or Keith) the right to make tradeoff
decisions even if you think they are stupid (or that my concern
about lost messages is excessive).

We hit a disconnect (I think), only when you (or others)
encourage some mail providers to reject or, worse, drop, my
legitimate mail (for the purposes of this discussion, assume
non-commercial, small number of recipients, and desired/
expected by the proposed recipient) because the sender did not
meet some semi-arbitrary criterion established by that provider
and not under the control of either the sender or (especially)
the intended recipient.

My definition of "semi-arbitrary" perhaps also differs from
yours.  I would include in that category any rule that depends
on heuristics (which, by definition, will fail sometimes) and
any classification system that is not clearly identified and
against which the intended recipient of the message does not
have practice and effective recourse if the owners of the
classification system get things wrong.  

I may be wrong, but I don't think either Keith or I are saying
"don't use blacklists in your systems".  Keith, I think, comes
closer than I do.   For all you know, I might even be using
blacklists as part of a scoring system (and I'm not telling).
What I think we are saying is:

	(i) Don't force me to use blacklists or otherwise force
	them on me or my mail.
	
	(ii) If someone does use a blacklist and legitimate mail
	is dropped, both the list compiler and the party who
	decided to use that particular list and how should be
	easily identified and accountable.

I don't see blacklists as being special in that regard.  YOu
could substitute any technique for preventing some messages from
showing up in my primary inbox for "blacklist" in the above two
comments and I would almost certainly still agree.

  best,
   john

From: [email-masked]
Cc: Ned Freed <[email-masked]>, "A. Rothman" <[email-masked]>, SMTP Interest Group <[email-masked]>
Date: Wed, 11 May 2011 15:06:26 -0700 (PDT)
Subject: Re: abnf for <atext> in RFC 5321?
To: Dave CROCKER <[email-masked]>
Sender: [email-masked]

> On 5/11/2011 2:23 PM, Ned Freed wrote:
> > Yep. The only thing that's unusual here is that these sorts of linkages are
> > usually declared in the conventions section of the document rather than waiting
> > until the "collected ABNF" section to say it.

> You mean, if only there were something like a Document Conventions section,
> early in the document, for containing meta-specification information like this?

> Alas, the section you /are/ referring to is rather more narrowly is 4.2.1, with
> a title Command Argument Syntax.  That's a bit obscure for finding an import of
> default definition-by-reference of ABNF rules.

Exactly. I was a little surprised we don't have something about the use
of ABNF in the conventions section. I fully support putting that in and moving
the statement about where to find missing productions there as well.

				Ned

Cc: IETF SMTP <[email-masked]>
Date: Wed, 11 May 2011 14:30:52 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
To: "A. Rothman" <[email-masked]>
Sender: [email-masked]

> Hi,

> I was hoping someone can shed some light on the proper syntax an SMTP
> server should use according to RFC 5321 in the following cases:

> 1. The syntax definition for commands such as VRFY and EXPN (e.g.
> section 4.1.1.6) is

> vrfy = "VRFY" SP String CRLF

> with the syntax of "String" defined in section 4.1.2 as:

> String         = Atom / Quoted-string

> however, section 3.5.1 states:

>     The character string arguments of the VRFY and EXPN commands cannot
>     be further restricted due to the variety of implementations of the
>     user name and mailbox list concepts.

I'm not seeing a conflict here. You appear to be reading this as saying that
there are not and cannot be restrictions on the argument syntax. I don't read
it that way. Rather, I read as saying that the syntax cannot be further
restricted from what the syntax currently allows, that is, an Atom or
Quoted-string. And I'm not seeing much justification for reading it any other
way.

Of course in pratice processors allow all sorts of stuff that doesn't
meet these syntax restrictions - addresses are particularly common - but
I'm doubtful there is a consensus to loosen the syntax further, with all that
implies.

> What is not entirely clear to me is which is true - is any string of
> characters valid (syntax-wise), or must it be a Quoted-string (in double
> quotes) if any non-atext characters appear in it. This is especially
> confusing having seen various examples online (and some discussions on
> this mailing list as well) where pointed brackets are included in the
> VRFY argument (using a Path syntax similar to the one defined for the
> MAIL FROM and RCPT TO commands), but with no quoting.

And there are processors that accept unadorned addresses too. But I'm not
seeing such examples in RFC 5321. It's hardly news that there's incorrect stuff
out on the 'net and people tend to be pretty careless when throwing examples
around in list mail too.

> 2. Section 4.1.2 defines the backslash-escaped character mechanism in
> quoted-pairSMTP, which is used only in a Quoted-string (within
> double-quotes), and does not mention such escaping outside of a
> Quoted-string. The following text section states:

>     Note that the backslash, "\", is a quote character, which is used to
>     indicate that the next character is to be used literally (instead of
>     its normal interpretation).  For example, "Joe\,Smith" indicates a
>     single nine-character user name string with the comma being the
>     fourth character of that string.

> So, it is unclear whether this paragraph applies only to the
> Quoted-strings defined above, or to any characters in any argument to
> any command, or only to mailboxes (discussed in the preceding paragraph)
> or some other definition of when it does and does not apply.

On the contrary, I think it's actually pretty clear from the context that this
is discussing the *semantics* (as opposed to the syntax) of backslash within
quoted strings. And this is an important point - it's important to understand
that

    "Joe Blow"

and 

    "Joe\ Blow"

are semantically the same.

That said, I would not object to changing the "Note that the backslash" to
"Note that backslashes in Quoted-String elements" or something similar. But I
don't think this rises to the level of an outright error.

> 3. Still regarding VRFY (and maybe also EXPN?) section 3.5.1 states:

>     If a normal (i.e., 250) response is returned,
>     the response MAY include the full name of the user and MUST include
>     the mailbox of the user.  It MUST be in either of the following
>     forms:

>        User Name<local-part@domain>
>        local-part@domain

> Whereas section 3.5.2 claims:

>     When normal (2yz or 551) responses are returned from a VRFY or EXPN
>     request, the reply MUST include the<Mailbox>  name using a
>     "<local-part@domain>" construction

This is actually kind of silly in and of itself - since the domain may have to
be determined by some sort of lookup operation on VRFY argument, how can it
possibly be appropriate to require that a 252 "I didn't do anything" response
include a domain? When our server is configured to return 252 responses to
VRFY, it most certainly violates this restriction because it simply echoes the
supplied argument.

> Notably, the second example in the former section does not comply with
> the latter section (as it contains no pointed brackets). Which is the
> correct form?

Well, when you have two conflicting but overlapping requirements, the
safest course is usually to restrict yourself to the overlapping case that
meets both. But this one does warrant an erratum.

				Ned

Date: 11 May 2011 21:53:20 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: abnf for <atext> in RFC 5321?
Organization: 
Sender: [email-masked]

>> I believe that there is a statement that any productions that
>> cannot be found are defined either in the base ABNF spec or in
>> 5322.

SM is (as usual) correct.  Section 4.1.2 of 5321 says:

   Terminals not defined in this document, such as ALPHA, DIGIT, SP,
   CR, LF, CRLF, are as defined in the "core" syntax in Section 6 of
   RFC 5234 [7] or in the message format syntax in RFC 5322 [4].

RFC 5322 has the definition of atext, so there's no dangling
reference.  This is at most an editorial issue, where it might have
been a good idea to be more specific about what symbols are imported
from where, so those of us doing a grep for "atext" would find it.

R's,
John

From: "Murray S. Kucherawy" <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Date: Wed, 11 May 2011 14:36:27 -0700
Subject: RE: abnf for <atext> in RFC 5321?
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
> Sent: Wednesday, May 11, 2011 2:24 PM
> To: A. Rothman
> Cc: [email-masked]; Dave CROCKER; SMTP Interest Group
> Subject: Re: abnf for <atext> in RFC 5321?
> 
> > I did not read the
> > whole of RFC 5222 recently, but a simple search found this definition
> > quickly and it seems pretty well-defined. Copying it over to RFC 5321
> > might be more convenient though.
> 
> Maybe, but these sorts of changes may not interact well with the ongoing EAI
> work. I'm therefore opposed to making this change - if a change needs to be
> made it would be to make the document linkage more prominent.

I'd prefer the more complete change, but I could settle for this.

(It just seems weird for the ABNF definition of something in RFCx to actually be found in RFCy where y>x...)

Date: Wed, 11 May 2011 14:36:02 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Ned Freed <[email-masked]>
CC: "A. Rothman" <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: Re: abnf for <atext> in RFC 5321?
Sender: [email-masked]

On 5/11/2011 2:23 PM, Ned Freed wrote:
> Yep. The only thing that's unusual here is that these sorts of linkages are
> usually declared in the conventions section of the document rather than waiting
> until the "collected ABNF" section to say it.

You mean, if only there were something like a Document Conventions section, 
early in the document, for containing meta-specification information like this?

Alas, the section you /are/ referring to is rather more narrowly is 4.2.1, with 
a title Command Argument Syntax.  That's a bit obscure for finding an import of 
default definition-by-reference of ABNF rules.

>> I did not read the
>> whole of RFC 5222 recently, but a simple search found this definition
>> quickly and it seems pretty well-defined. Copying it over to RFC 5321
>> might be more convenient though.
>
> Maybe, but these sorts of changes may not interact well with the ongoing EAI
> work. I'm therefore opposed to making this change - if a change needs to be
> made it would be to make the document linkage more prominent.

Indeed.

5322 and 5321 already suffer from having parallel definitions; we should seek 
fewer of these, not more.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: "Murray S. Kucherawy" <[email-masked]>
To: "A. Rothman" <[email-masked]>
CC: IETF SMTP <[email-masked]>
Date: Wed, 11 May 2011 14:34:07 -0700
Subject: RE: RFC 5321 VRFY and quoting syntax
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: A. Rothman [mailto:[email-masked]]
> Sent: Wednesday, May 11, 2011 1:26 PM
> To: Murray S. Kucherawy
> Cc: IETF SMTP
> Subject: Re: RFC 5321 VRFY and quoting syntax
> 
> This is not exactly an issue of conflict, but rather of leaving room for
> interpretation. The ABNF is indeed well-defined on it's own as you
> state, however when one reads that the argument 'cannot be further
> restricted' (is that before or after quoting?) and sees many examples
> out in the wild, most notably the brackets but also others (as in
> Hector's previous response which uses backslash escaping outside of a
> quoted-string), it raises doubt of whether 'cannot be further
> restricted' means 'you can do anything you want' or indeed still must
> abide by the rules of quoting. Having many examples violating the rules
> of quoting gives more weight to the first interpretation. Or, in more
> practical terms - since there are many common violations, the RFC
> wording might have some room for improvement in regards to being more
> explicit and preventing these violations.

It has worked for me, in such situations, to err on the side of implementing the ABNF and not the examples.  In my experience there's more attention paid to getting that right than there is to getting the examples right.  However, this might not be one of those occasions.  :-)

> If you believe the ABNF, that's correct. 

See above.  :-)

> If you believe the text in that
> paragraph... not so sure any more. Even in the example within the
> paragraph of "Joe\,Smith", it is unclear if the quotes are part of the
> actual argument, or just a way to embed the example within a paragraph
> of explanatory text. Here too, I think the fact that there are various
> violations out there which can legitimately point to this paragraph and
> say 'it says so in the RFC'... it would be better imho to update the
> spec's wording to be unambiguous, e.g. "Note that the backslash, "\",
> when used within a quoted string, is a quote character..."

I would recommend opening an errata item then.  They get reviewed and eventually assigned to be included in the next update, and as it happens there's a working group chartered to take care of those sorts of things.

> ok, I'll do that when this thread ends, if no one comes up with a
> counter-explanation.

Let me know if you need help navigating the process.

-MSK

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Wed, 11 May 2011 14:28:46 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Wednesday, May 11, 2011 1:04 PM
> To: [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> > We're talking about providing a protocol, and including in an
> > IANA registry (which is the direction this appeared to be going)
> > some prescribed operator action seems inappropriate to me.  Define
> > what the new code means, and leave it there.
> 
> This mindset is archaic and outdated in 2011 and beyond.  The world is
> too complex and yet the issues are widely known. The audience of this
> document is not going to be reading it with an anal document editor
> perspective or "did I follow the IETF protocol guessing rule" mindset.

This mindset conflates protocol specification with implementation and BCP.

> This is not odd. SMTP is filled with modern mail handling insights and
> it added a few very important ones in RFC5321:
> 
> New Last sentence in Section 6.2, 1st para:
> 
>     When the receiver-SMTP accepts a piece of mail (by sending a "250 OK"
>     message in response to DATA), it is accepting responsibility for
>     delivering or relaying the message.  It must take this responsibility
>     seriously.  It MUST NOT lose the message for frivolous reasons, such
>     as because the host later crashes or because of a predictable
>     resource shortage.  Some reasons that are not considered frivolous
>     are discussed in the next subsection and in Section 7.8.

This all talks about software design and operation.  The proposal to which I'm responding said "notify an operator".  There's a gigantic difference between the two in terms of what's appropriate for a standards track RFC and what is not.

From: [email-masked]
Cc: [email-masked], Dave CROCKER <[email-masked]>, SMTP Interest Group <[email-masked]>
Date: Wed, 11 May 2011 14:23:59 -0700 (PDT)
Subject: Re: abnf for <atext> in RFC 5321?
To: "A. Rothman" <[email-masked]>
Sender: [email-masked]

> The introduction paragraph of section 4.1.2 explicitly links to two
> other RFCs for the missing syntax definitions,

RFCs 5234 and 5322 specifically.

> and indeed the second one
> (RFC 5222) contains a proper definition of atext.

Yep. The only thing that's unusual here is that these sorts of linkages are
usually declared in the conventions section of the document rather than waiting
until the "collected ABNF" section to say it.

> I did not read the
> whole of RFC 5222 recently, but a simple search found this definition
> quickly and it seems pretty well-defined. Copying it over to RFC 5321
> might be more convenient though.

Maybe, but these sorts of changes may not interact well with the ongoing EAI
work. I'm therefore opposed to making this change - if a change needs to be
made it would be to make the document linkage more prominent.

				Ned

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 17:27:19 -0400
Cc: "IETF SMTP list" <[email-masked]>
To: "John R Levine" <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 5:16 PM, John R Levine wrote:

>> Your "experience" with what you think of as "real email systems" doesn't mean jack when it comes to my business.
> 
> That's nice.  So I expect it's fair to conclude that your business is equally irrlevant to the rest of the world's e-mail users.

Reality is that user experience of spam, and spam filtering, varies tremendously from one situation to another.     Broad generalizations aren't useful for predicting the experience of individual users.

Keith

Subject: Re: abnf for <atext> in RFC 5321?
From: Steve Atkins <[email-masked]>
Date: Wed, 11 May 2011 14:21:58 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 2:13 PM, Dave CROCKER wrote:

> 
> 
> 
> On 5/11/2011 2:04 PM, SM wrote:
>> Hi Dave,
>> At 13:15 11-05-2011, Dave CROCKER wrote:
>>> <atext> seems to be an undefined rule in RFC 5321.
>>> 
>>> This is being noticed in a couple of different fora, so it's worth
>>> resolving/fixing.
>> 
>> It is mentioned in a message to YAM (
>> http://www.ietf.org/mail-archive/web/yam/current/msg00542.html ) and also on
> 
> yup.  it /is/ showing up as an issue in multiple venues.
> 
> 
>> this mailing list. There was a reply (
>> http://www.imc.org/ietf-smtp/mail-archive/msg06161.html ).
> 
> that notes suggests:
> 
>> I believe that
>> there is a statement that any productions that cannot be found
>> are defined either in the base ABNF spec or in 5322.
> 
> But I can find no such 'default' statement in 5322 (or 2821).

5321 4.1.2 says, immediately prior to the only mention of atext:

Terminals not defined in
   this document, such as ALPHA, DIGIT, SP, CR, LF, CRLF, are as defined
   in the "core" syntax in Section 6 of  RFC 5234 [7] or in the message
   format syntax in RFC 5322 [4].

5322 defines atext in section 3.2.3.

Cheers,
  Steve

Date: Wed, 11 May 2011 17:21:31 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "A. Rothman" <[email-masked]>
CC: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

A. Rothman wrote:
>>>
>>> However, for outputs, it MUST be in angle bracket email form.
>>
>> Oh I see, the example is wrong! To match the text, it should be:
>>
>>     User Name
>>     User Name <local-part@domain>
>>
>> but it seems there would be three input forms:
>>
>>     User Name
>>     User Name <local-part@domain>
>>     local-part@domain
>>
>> +1 in principle for an errata!!
>>
> That's a separate ambiguity :-)
> Whichever of the above is correct, it still does not solve the conflict 
> with the other section that states 'the reply MUST include the 
> <Mailbox>  name using a "<local-part@domain>" construction'. If your 
> interpretation is correct, it should be worded "If the reply contains a 
> mailbox, it MUST include it using a "<local-part@domain>" construction" 
> or something like that (the format is conditioned on the inclusion of 
> the mailbox, rather than being an absolute requirement).
> 
> I guess I'm first trying to figure out what the correct interpretation 
> is, so that I can know what the errata should say.
> 
> -Amichai

I'll punt on the ABNF syntax issue, if any. I always have trouble 
reading those anyway.

But see 3.5.1 paragraph:

    "User name" is a fuzzy term and has been used deliberately.  An
    implementation of the VRFY or EXPN commands MUST include at least
    recognition of local mailboxes as "user names".  However, since
    current Internet practice often results in a single host handling
    mail for multiple domains, hosts, especially hosts that provide this
    functionality, SHOULD accept the "local-part@domain" form as a "user
    name"; hosts MAY also choose to recognize other strings as "user
    names".

Not looking at the ABNF, this all the prior text in the 3.5.1 section 
does sort of make sense. It depends on how you interpret User Name.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 17:16:45 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
CC: Arnt Gulbrandsen <[email-masked]>, John C Klensin <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

John C Klensin wrote:
> --On Wednesday, May 11, 2011 21:31 +0200 Arnt Gulbrandsen
> <[email-masked]> wrote:
> 
>> On 05/11/2011 08:04 PM, Keith Moore wrote:
>>> I've suffered far more harm from people blocking my incoming
>>> mail due to input from DNSBLs, than I've ever suffered from
>>> spammers.
>> How I envy you.
> 
> I think it depends a bit on how one measures "harm".  I tend to
> feel significantly harmed when legitimate messages (either ones
> I send or ones that are sent to me) are not delivered -- even
> one or two a month is an intolerable level.  By contrast, I see
> spam as an expensive waste of time and resources, regardless of
> the mechanism I use to deal with those messages (usually
> amounting to a tradeoff between time and resources).  But, as
> far as "harm" goes, non-delivery of messages that should have
> been delivered and instead vanished without a trace is, for me,
> qualitatively worse than any quantity of spam.

+1

> So, sadly, I'm inclined to agree with Keith.

Well, it depends on one's perspective.  If you are in the business of 
marketing and/or writing DMA or bulk mailing tools, then sure, you 
don't like rejections, especially when IPs are discovered to be black 
listed.

On the other hand, if you are not in the business and support this 
stuff for yourself and customers, its no fun to suddenly find yourself 
black listed on a particular list.   Fortunately, there are procedures 
to get off the list that was somehow added by some stupid user.  My 
network ISP also got some complaint from a customer still getting 
OPT-IN announcements but forget to unsubscribe.  The ISP threaten us 
to lost our network and we came back with Tortious Interference 
warning - problem solved.  So sure, no one likes it when its personal.

Overall I agree with Dave, the industry benefit as a whole has been 
recognized and it is what it is - it isn't like its going to go away. 
  Marketers or tools people for Marketers who believe that receivers 
should accept all mail and let the user decide is not the real world. 
  The issue is not just the potential abuse on users, but also 
potential for the federal protection against abuse on systems as well 
- receivers has rights too! All you have to prove $5000 (may be 
$25,000 today) in harm/lost and you can dial up the FBI to file a case!

-- 
Sincerely

Hector Santos
http://www.santronics.com

VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org
Date: 11 May 2011 17:16:45 -0400
From: "John R Levine" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: "IETF SMTP list" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Cleverness: None detected
Sender: [email-masked]

> Your "experience" with what you think of as "real email systems" doesn't 
> mean jack when it comes to my business.

That's nice.  So I expect it's fair to conclude that your business is 
equally irrlevant to the rest of the world's e-mail users.

I think we're done now.  Have a fabulous day.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: Wed, 11 May 2011 14:13:54 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: SM <[email-masked]>
CC: SMTP Interest Group <[email-masked]>
Subject: Re: abnf for <atext> in RFC 5321?
Sender: [email-masked]

On 5/11/2011 2:04 PM, SM wrote:
> Hi Dave,
> At 13:15 11-05-2011, Dave CROCKER wrote:
>> <atext> seems to be an undefined rule in RFC 5321.
>>
>> This is being noticed in a couple of different fora, so it's worth
>> resolving/fixing.
>
> It is mentioned in a message to YAM (
> http://www.ietf.org/mail-archive/web/yam/current/msg00542.html ) and also on

yup.  it /is/ showing up as an issue in multiple venues.

> this mailing list. There was a reply (
> http://www.imc.org/ietf-smtp/mail-archive/msg06161.html ).

that notes suggests:

> I believe that
> there is a statement that any productions that cannot be found
> are defined either in the base ABNF spec or in 5322.

But I can find no such 'default' statement in 5322 (or 2821).

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 17:12:04 -0400
Cc: "IETF SMTP list" <[email-masked]>
To: "John R Levine" <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 5:06 PM, John R Levine wrote:

>>> With a sufficient quantity of spam in your mailbox, you can't even
>>> find the real mail so it might as well not have been delivered.  BTDT.
>> 
>> False.  It just means that I get to choose the method of spam filtering rather than having my ISP choose it for me.
> 
> You're assuming that your spam filters will work better than the DNSBLs you dislike.  Experience with real mail systems (see papers at CEAS) strongly suggests that the error rate of a well run DNSBL is no worse and probably better than whatever you think you might like better.

Your "experience" with what you think of as "real email systems" doesn't mean jack when it comes to my business.  

Keith

Date: Wed, 11 May 2011 14:04:33 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: abnf for <atext> in RFC 5321?
Cc: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

Hi Dave,
At 13:15 11-05-2011, Dave CROCKER wrote:
><atext> seems to be an undefined rule in RFC 5321.
>
>This is being noticed in a couple of different fora, so it's worth 
>resolving/fixing.

It is mentioned in a message to YAM ( 
http://www.ietf.org/mail-archive/web/yam/current/msg00542.html ) and 
also on this mailing list.  There was a reply ( 
http://www.imc.org/ietf-smtp/mail-archive/msg06161.html ).

Regards,
-sm 

VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org
Date: 11 May 2011 17:06:30 -0400
From: "John R Levine" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: "IETF SMTP list" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Cleverness: None detected
Sender: [email-masked]

>> With a sufficient quantity of spam in your mailbox, you can't even
>> find the real mail so it might as well not have been delivered.  BTDT.
>
> False.  It just means that I get to choose the method of spam filtering rather than having my ISP choose it for me.

You're assuming that your spam filters will work better than the DNSBLs 
you dislike.  Experience with real mail systems (see papers at CEAS) 
strongly suggests that the error rate of a well run DNSBL is no worse and 
probably better than whatever you think you might like better.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

Date: 11 May 2011 20:59:15 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: abnf for <atext> in RFC 5321?
Organization: 
Sender: [email-masked]

> <atext> seems to be an undefined rule in RFC 5321.

Its undefinition is unchanged from 2821.

> This is being noticed in a couple of different fora, so it's worth
  resolving/fixing.

The definition in 5322, which is the same as 2822, seems appropriate.

Since it took ten years for anyone to notice this error, I agree it
would be nice to fix it, eventually, but not urgently.  I would be
happy to file an erratum along these lines:

Section 4.1.2 refers to the ABNF symbol atext, but does not define it.
The intended definition is the same as in RFC 5322.  Hence Section
4.1.2 currently says:

   Atom           = 1*atext

but it should say:

   Atom           = 1*atext

   atext          =   ALPHA / DIGIT /     ; Printable US-ASCII
                       "!" / "#" /        ;  characters not including
                       "$" / "%" /        ;  specials.  Used for atoms.
                       "&" / "'" /
                       "*" / "+" /
                       "-" / "/" /
                       "=" / "?" /
                       "^" / "_" /
                       "`" / "{" /
                       "|" / "}" /
                       "~"

R's,
John

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 16:57:37 -0400
Cc: "[email-masked] Group" <[email-masked]>
To: Laura Atkins <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 4:32 PM, Laura Atkins wrote:

> On May 11, 2011, at 1:02 PM, John C Klensin wrote:
> 
>>> On 05/11/2011 08:04 PM, Keith Moore wrote:
>>>> I've suffered far more harm from people blocking my incoming
>>>> mail due to input from DNSBLs, than I've ever suffered from
>>>> spammers.
>>> 
>>> How I envy you.
>> 
>> I think it depends a bit on how one measures "harm".  I tend to
>> feel significantly harmed when legitimate messages (either ones
>> I send or ones that are sent to me) are not delivered -- even
>> one or two a month is an intolerable level.  
> 
> I've yet to find a filter, even the "read every message and filter manually" that doesn't occasionally mis-file a mail. For instance, my spam load is high enough that I lose one or two messages a month just because they get mixed up in the spam and manually deleted. 

That's happened to me too.   

This is the sort of thing that's going to vary from one recipient to another.  Not everyone gets the same amount, or kinds, of spam.  And not everyone's livelihood depends on being able to reliably receive (nearly) every piece of email.

Keith

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 16:55:31 -0400
Cc: [email-masked], [email-masked]
To: "John Levine" <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 4:36 PM, John Levine wrote:

>> But, as far as "harm" goes, non-delivery of messages that should
>> have been delivered and instead vanished without a trace is, for me,
>> qualitatively worse than any quantity of spam.
> 
> With a sufficient quantity of spam in your mailbox, you can't even
> find the real mail so it might as well not have been delivered.  BTDT.

False.  It just means that I get to choose the method of spam filtering rather than having my ISP choose it for me.

Keith

Date: Wed, 11 May 2011 23:52:58 +0300
From: "A. Rothman" <[email-masked]>
To: [email-masked]
CC: Dave CROCKER <[email-masked]>, SMTP Interest Group <[email-masked]>
Subject: Re: abnf for <atext> in RFC 5321?
Sender: [email-masked]

The introduction paragraph of section 4.1.2 explicitly links to two 
other RFCs for the missing syntax definitions, and indeed the second one 
(RFC 5222) contains a proper definition of atext. I did not read the 
whole of RFC 5222 recently, but a simple search found this definition 
quickly and it seems pretty well-defined. Copying it over to RFC 5321 
might be more convenient though.

-Amichai

On 05/11/2011 11:15 PM, Dave CROCKER wrote:

>
> Folks,
>
> <atext> seems to be an undefined rule in RFC 5321.
>
> This is being noticed in a couple of different fora, so it's worth 
> resolving/fixing.
>
> d/

Date: Wed, 11 May 2011 23:41:11 +0300
From: "A. Rothman" <[email-masked]>
To: Hector Santos <[email-masked]>
CC: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

On 05/11/2011 10:13 PM, Hector Santos wrote:

>
> Hector Santos wrote:
>>
>> Murray S. Kucherawy wrote:
>>>
>>> I would say the angle-bracket form is what would be expected, but 
>>> you're right, this looks like an ambiguity.  I would recommend 
>>> opening an errata item for it if that hasn't already been done.
>>
>> Section 3.5.1 seems pretty clear to me:
>>
>>    For the VRFY command, the string is a user name or a user name and
>>    domain (see below).....
>>
>>       User Name <local-part@domain>
>>       local-part@domain
>>
>> The former is one of many legal email address forms. In this one form 
>> allowed for this command, the angle-bracket is needed.  But if the 
>> user name is not going to be entered, it shows a 2nd input form where 
>> you don't need to use angle brackets.
>>
>> However, for outputs, it MUST be in angle bracket email form.
>
> Oh I see, the example is wrong! To match the text, it should be:
>
>     User Name
>     User Name <local-part@domain>
>
> but it seems there would be three input forms:
>
>     User Name
>     User Name <local-part@domain>
>     local-part@domain
>
> +1 in principle for an errata!!
>
That's a separate ambiguity :-)
Whichever of the above is correct, it still does not solve the conflict 
with the other section that states 'the reply MUST include the 
<Mailbox>  name using a "<local-part@domain>" construction'. If your 
interpretation is correct, it should be worded "If the reply contains a 
mailbox, it MUST include it using a "<local-part@domain>" construction" 
or something like that (the format is conditioned on the inclusion of 
the mailbox, rather than being an absolute requirement).

I guess I'm first trying to figure out what the correct interpretation 
is, so that I can know what the errata should say.

-Amichai

Date: 11 May 2011 20:36:39 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Organization: 
Sender: [email-masked]

> But, as far as "harm" goes, non-delivery of messages that should
> have been delivered and instead vanished without a trace is, for me,
> qualitatively worse than any quantity of spam.

With a sufficient quantity of spam in your mailbox, you can't even
find the real mail so it might as well not have been delivered.  BTDT.

R's,
John

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Laura Atkins <[email-masked]>
Date: Wed, 11 May 2011 13:32:44 -0700
To: "[email-masked] Group" <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 1:02 PM, John C Klensin wrote:

>> On 05/11/2011 08:04 PM, Keith Moore wrote:
>>> I've suffered far more harm from people blocking my incoming
>>> mail due to input from DNSBLs, than I've ever suffered from
>>> spammers.
>> 
>> How I envy you.
> 
> I think it depends a bit on how one measures "harm".  I tend to
> feel significantly harmed when legitimate messages (either ones
> I send or ones that are sent to me) are not delivered -- even
> one or two a month is an intolerable level.  

I've yet to find a filter, even the "read every message and filter manually" that doesn't occasionally mis-file a mail. For instance, my spam load is high enough that I lose one or two messages a month just because they get mixed up in the spam and manually deleted. 

This does seem to get worse with volume, though, so folks who only get a couple hundred spams a day may not ever find that to be a problem.

laura 

-- 
Laura Atkins
Word to the Wise			"The Deliverability Experts!"
Direct: 650 678-3454		Fax: 650 249-1909
AIM: wttwlaura			YIM: wttw_laura
Delivery blog: <http://blog.wordtothewise.com/>

Date: Wed, 11 May 2011 23:25:48 +0300
From: "A. Rothman" <[email-masked]>
To: "Murray S. Kucherawy" <[email-masked]>
CC: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

On 05/11/2011 08:47 PM, Murray S. Kucherawy wrote:

>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of A. Rothman
>> Sent: Monday, May 09, 2011 8:03 AM
>> To: IETF SMTP
>> Subject: RFC 5321 VRFY and quoting syntax
>>
>> 1. The syntax definition for commands such as VRFY and EXPN (e.g.
>> section 4.1.1.6) is
>>
>> vrfy = "VRFY" SP String CRLF
>>
>> with the syntax of "String" defined in section 4.1.2 as:
>>
>> String         = Atom / Quoted-string
>>
>> however, section 3.5.1 states:
>>
>>      The character string arguments of the VRFY and EXPN commands cannot
>>      be further restricted due to the variety of implementations of the
>>      user name and mailbox list concepts.
>>
>> What is not entirely clear to me is which is true - is any string of
>> characters valid (syntax-wise), or must it be a Quoted-string (in double
>> quotes) if any non-atext characters appear in it. This is especially
>> confusing having seen various examples online (and some discussions on
>> this mailing list as well) where pointed brackets are included in the
>> VRFY argument (using a Path syntax similar to the one defined for the
>> MAIL FROM and RCPT TO commands), but with no quoting.
> I'm missing how your first sentence here is a conflict.  It has to be a quoted-string if there are any non-atext characters, otherwise the quoting is not needed.
This is not exactly an issue of conflict, but rather of leaving room for 
interpretation. The ABNF is indeed well-defined on it's own as you 
state, however when one reads that the argument 'cannot be further 
restricted' (is that before or after quoting?) and sees many examples 
out in the wild, most notably the brackets but also others (as in 
Hector's previous response which uses backslash escaping outside of a 
quoted-string), it raises doubt of whether 'cannot be further 
restricted' means 'you can do anything you want' or indeed still must 
abide by the rules of quoting. Having many examples violating the rules 
of quoting gives more weight to the first interpretation. Or, in more 
practical terms - since there are many common violations, the RFC 
wording might have some room for improvement in regards to being more 
explicit and preventing these violations.
> Also, a good point on the angle brackets.  The syntax seems to suggest they're not valid (or, indeed, need quoting themselves).  I'm sure it's commonly accepted this way since the most commonly used SMTP commands (MAIL and RCPT) require them but most implementations also usually tolerate their absence.
>
>> 2. Section 4.1.2 defines the backslash-escaped character mechanism in
>> quoted-pairSMTP, which is used only in a Quoted-string (within
>> double-quotes), and does not mention such escaping outside of a
>> Quoted-string. The following text section states:
>>
>>      Note that the backslash, "\", is a quote character, which is used to
>>      indicate that the next character is to be used literally (instead of
>>      its normal interpretation).  For example, "Joe\,Smith" indicates a
>>      single nine-character user name string with the comma being the
>>      fourth character of that string.
>>
>> So, it is unclear whether this paragraph applies only to the
>> Quoted-strings defined above, or to any characters in any argument to
>> any command, or only to mailboxes (discussed in the preceding paragraph)
>> or some other definition of when it does and does not apply.
> It appears to apply only within quoted strings.  Its main function then would be to escape quotation marks within quoted strings, because everything else is not a character that separates tokens.
If you believe the ABNF, that's correct. If you believe the text in that 
paragraph... not so sure any more. Even in the example within the 
paragraph of "Joe\,Smith", it is unclear if the quotes are part of the 
actual argument, or just a way to embed the example within a paragraph 
of explanatory text. Here too, I think the fact that there are various 
violations out there which can legitimately point to this paragraph and 
say 'it says so in the RFC'... it would be better imho to update the 
spec's wording to be unambiguous, e.g. "Note that the backslash, "\", 
when used within a quoted string, is a quote character..."
>> 3. Still regarding VRFY (and maybe also EXPN?) section 3.5.1 states:
>>
>>      If a normal (i.e., 250) response is returned,
>>      the response MAY include the full name of the user and MUST include
>>      the mailbox of the user.  It MUST be in either of the following
>>      forms:
>>
>>         User Name<local-part@domain>
>>         local-part@domain
>>
>> Whereas section 3.5.2 claims:
>>
>>      When normal (2yz or 551) responses are returned from a VRFY or EXPN
>>      request, the reply MUST include the<Mailbox>   name using a
>>      "<local-part@domain>" construction
>>
>> Notably, the second example in the former section does not comply with
>> the latter section (as it contains no pointed brackets). Which is the
>> correct form?
> I would say the angle-bracket form is what would be expected, but you're right, this looks like an ambiguity.  I would recommend opening an errata item for it if that hasn't already been done.
ok, I'll do that when this thread ends, if no one comes up with a 
counter-explanation.
> -MSK
>
>
>

Date: Wed, 11 May 2011 13:15:27 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: SMTP Interest Group <[email-masked]>
Subject: abnf for <atext> in RFC 5321?
Sender: [email-masked]

Folks,

<atext> seems to be an undefined rule in RFC 5321.

This is being noticed in a couple of different fora, so it's worth resolving/fixing.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Wed, 11 May 2011 23:05:34 +0300
From: "A. Rothman" <[email-masked]>
To: Hector Santos <[email-masked]>
CC: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

It's a bit hard to follow which part here responds to which of the 3 
issues, but I'll try (within) :-)

On 05/11/2011 04:12 PM, Hector Santos wrote:

> I think that is it saying for INPUT, you can take either the angle 
> bracket or non-angle bracket form, but for OUTPUT it MUST be shown in 
> angle bracket form.  It is analogous to looking up someone by name in 
> a phone book - ultimately it is the single output form - telephone #
Both sections I quoted in #3 explicitly discuss the server's response 
(not input, or client request, if I understand your terminology 
correctly). They are also both MUSTs. One of them requires inclusion of 
the construction "<local-part@domain>" in any response, whereas the 
other one gives a response without the brackets as one of the two 
allowed forms - that's the conflict I'm seeing.

I'm all for the 'being lenient in input and strict with output' motto - 
but the spec must be unambiguous about the output part in order to do 
this, and preferably also discuss the possible inputs and preferred ones.
>
> Yes, I agree that in many areas of software, the term QUOTE can be 
> mixed up with the idea of ESCAPING.  In my opinion, ESCAPING is the 
> proper communications term as it used in many data communications 
> protocols.
>
>    http://www.google.com/search?q=define:character+escaping
Terminology might be ambiguous, but in this case it's not what bothers 
me - I'm sticking to the ABNF syntax notation definitions and the 
discussion of the use of backslash for its special purpose, regardless 
of what we call it :-)
>
> For VRFY, EXPN, the escaping "\" (SLASH) character is used to escape 
> the next character which could be a formal delimiting character to 
> separate multiple inputs.  For example:
>
>    VRFY   santos,hector,shane,[email-masked]
>
> This would be would be four "local account" user name changes,
>
>   santos
>   hector,
>   shane,
>   [email-masked]
>
> And depending on the host on how it looks up users, it be by some name 
> or even address.   Some systems (like ours) has three identities per 
> user account; A number ID, user name and email address. And the user 
> name may have an old school MHS format (first.last), and in addition 
> it may take into account the possible internal translations a system 
> may have like BOSS, PRESIDENT, SYSOP, SUPPORT etc.
>
> Versus:
>
>    VRFY  santos\,hector,shane,[email-masked]
>
> which would be three accounts to look up:
>
>    santos,hector
>    shane
>    [email-masked]
>
I think I understand your examples and why use of the backslash would be 
necessary. However, the part I find problematic, which shows up in your 
examples as well, is that according to the spec's ABNF this is invalid:

VRFY  santos\,hector,shane,[email-masked]

and this is the way it should be:

VRFY  "santos\,hector,shane,[email-masked]"

In other words, the issue is whether use of the backslash is valid 
anywhere, or only within a Quoted-string (as defined in ABNF). If your 
examples are indeed valid, then there's a bug in the ABNF part that 
needs to be fixed, or at the very least, the paragraph discussing the 
backslash should be explicit about it being allowed everywhere despite 
the ABNF limiting it just to quotes.

-Amichai

Date: Wed, 11 May 2011 16:04:14 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Murray S. Kucherawy wrote:

> I don't know what "standard set of recommendations" means in 
> this context.  

Sorry.

> We're talking about providing a protocol, and including in an 
> IANA registry (which is the direction this appeared to be going) 
> some prescribed operator action seems inappropriate to me.  Define 
> what the new code means, and leave it there.

This mindset is archaic and outdated in 2011 and beyond.  The world is 
too complex and yet the issues are widely known. The audience of this 
document is not going to be reading it with an anal document editor 
perspective or "did I follow the IETF protocol guessing rule" mindset.

Todays practitioners dealing with a high degree of mail integration 
and automation requirements need all the insights and recommendations 
they can get and when talking specifically about a special category of 
"Anti-Spam" extended reply codes, an idea that touches base with 
delivery reliability and security, having some form of AVS severity 
status will help provide implementator guidance at many levels of 
discipline.

This is not odd. SMTP is filled with modern mail handling insights and 
it added a few very important ones in RFC5321:

New Last sentence in Section 6.2, 1st para:

    When the receiver-SMTP accepts a piece of mail (by sending a "250 OK"
    message in response to DATA), it is accepting responsibility for
    delivering or relaying the message.  It must take this responsibility
    seriously.  It MUST NOT lose the message for frivolous reasons, such
    as because the host later crashes or because of a predictable
    resource shortage.  Some reasons that are not considered frivolous
    are discussed in the next subsection and in Section 7.8.

New section 6.2.  Unwanted, Unsolicited, and "Attack" Messages
New section 7.8.  Resistance to Attacks

6.2 changes many things with its modern recognition of new security 
needs and  the presence of new protocols such as SPF/SENDER-ID, ADSP 
and future augmented PAYLOAD security technology which allows for 
avoiding abusive bounce notifications.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 16:02:22 -0400
From: John C Klensin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

--On Wednesday, May 11, 2011 21:31 +0200 Arnt Gulbrandsen
<[email-masked]> wrote:

> 
> On 05/11/2011 08:04 PM, Keith Moore wrote:
>> I've suffered far more harm from people blocking my incoming
>> mail due to input from DNSBLs, than I've ever suffered from
>> spammers.
> 
> How I envy you.

I think it depends a bit on how one measures "harm".  I tend to
feel significantly harmed when legitimate messages (either ones
I send or ones that are sent to me) are not delivered -- even
one or two a month is an intolerable level.  By contrast, I see
spam as an expensive waste of time and resources, regardless of
the mechanism I use to deal with those messages (usually
amounting to a tradeoff between time and resources).  But, as
far as "harm" goes, non-delivery of messages that should have
been delivered and instead vanished without a trace is, for me,
qualitatively worse than any quantity of spam.

So, sadly, I'm inclined to agree with Keith.

    john

> 
> 

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 15:45:29 -0400
Cc: [email-masked]
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 3:31 PM, Arnt Gulbrandsen wrote:

> On 05/11/2011 08:04 PM, Keith Moore wrote:
>> I've suffered far more harm from people blocking my incoming mail due to input from DNSBLs, than I've ever suffered from spammers.
> 
> How I envy you.

I assume you don't envy me for losing business opportunities.  And I certainly don't envy those who have to fight spammers.

Keith

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 15:37:19 -0400
Cc: SMTP Interest Group <[email-masked]>
To: Steve Atkins <[email-masked]>
Sender: [email-masked]

--Apple-Mail-54-417924687
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On May 11, 2011, at 2:44 PM, Steve Atkins wrote:

> On May 11, 2011, at 11:11 AM, Keith Moore wrote:
>=20
>> On May 11, 2011, at 11:34 AM, Steve Atkins wrote:
>>=20
>>> On May 11, 2011, at 8:07 AM, Laura Atkins wrote:
>>>=20
>>>> On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:
>>>>=20
>>>>>> I'm not following this thread closely, but I thought I'd say =
something
>>>>>> about extended status codes.  Part of the idea of extended status =
codes
>>>>>> is that you should be able to determine the likely source of the
>>>>>> problem by looking at the second facet of the status code.  Or to =
put
>>>>>> it another way, the second facet of the status code is supposed =
to
>>>>>> indicate _who_ probably needs to fix the problem (e.g. the =
sender, the
>>>>>> MSA, a relay, the delivery agent, etc.)
>>>>>=20
>>>>> I haven't implemented this, but I have to say I really like this =
approach.  Perhaps the proposal would benefit from making such =
distinctions in the new codes it's registering.
>>>>=20
>>>>=20
>>>> If you're going to make the second facet of the status code be who =
probably needs to fix the problem, you could also make the third facet =
point to where they need to go to fix it.=20
>>>> It doesn't even need to be that detailed. 1 for this is internal =
(so you need to contact the recipient domain) and 2 for this is external =
(so you need to contact a third party). Then follow that with a URL =
pointing to the third party or the postmaster / internal troubleshooting =
pages.=20
>>>=20
>>> If there's a URL or human readable message that's all that's needed =
for someone to diagnose the delivery problem (which is how it's commonly =
done today). Adding additional computer readable information as an =
extended response code is only useful if it's intended to be handled =
entirely automatically - if you're going to put human eyeballs on it, =
the human readable response is all you need, once you've used the SMTP =
response (e.g. 5xx) to triage.
>>=20
>> one of the major reasons for enhanced status codes was to provide a =
language-independent indication of the error.    I certainly don't mind =
URLs being used to supply more information (especially if done in =
multiple languages) but I don't think it obviates the need for the =
enhanced status codes.
>=20
> I don't think it meets that goal, as it doesn't provide enough =
information to do anything useful in response to those broad categories. =
Pretty much all of the suggested categories break down into two groups:
>=20
>  1. Something went wrong. Keep trying.
>  2. Something went wrong. Read the error message.

The defined meaning of the enhanced status code should be a close =
approximation to the error message.=20

> t's only worthwhile for an SMTP receiver to send these codes or for an =
SMTP sender to parse them if they can be used to adjust the behaviour of =
the sender in a mutually beneficial way.
>=20
> The codes as defined are broad categories that are informational (as =
opposed to imperative, e.g. "Please don't send email to this address =
again", "Please don't send any further email to this address / domain =
until a human has intervened", "Please reduce the rate at which you are =
sending email to this MX") so they don't seem to communicate much to the =
senders automation in terms of "what behaviour you should change in =
response to this message".

They're not so much meant for "automation" - except perhaps for mailing =
lists - as for human consumption.

Admittedly, the codes weren't designed for a world where the sender has =
to argue with some blackhole list about whether his message should get =
through.  But even in the world where RFCs 1981-1894 were written, about =
half of all bounced messages were due to errors downstream of the sender =
- mostly DNS errors, MTA configuration errors, and a few recipient disk =
quota/space issues.   In general, the sender couldn't fix most of those =
problems directly, but it was still helpful to the sender to know where =
the problem was.

Today, there's still a very useful difference between telling the =
sender:=20
"your message didn't get delivered because the recipient address was not =
that of a valid recipient",  or
"your message didn't get delivered because it was too large", or
"your message didn't get delivered because we think it's spam based on =
the content", or=20
"your message didn't get delivered because blackhole list FOO accused =
your IP of generating spam", or
"your message didn't get delivered because your domain's DNS says that =
your IP is not authorized to send mail"

> A sysadmin who ignores the human readable message (whether due to =
language issues, poor logging or what have you) and relies solely on the =
extended response code is going to be in just the same situation as =
automated code - they're not going to have enough information to make a =
decision about what they should do in response without reading the human =
readable part of the message.

That might be true today, because today people are refusing delivery for =
a lot more reasons than anticipated at the time RFCs 1891-4 were =
written.  So extending the list of codes seems appropriate.
>=20
> That's why I'm wondering what the intended use of these codes by the =
email senders is.
>=20
> Cheers,
>  Steve
>=20

--Apple-Mail-54-417924687
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On May 11, 2011, at 2:44 PM, Steve Atkins =
wrote:</div><br><blockquote type=3D"cite"><div>On May 11, 2011, at 11:11 =
AM, Keith Moore wrote:<br><br><blockquote type=3D"cite">On May 11, 2011, =
at 11:34 AM, Steve Atkins wrote:<br></blockquote><blockquote =
type=3D"cite"><br></blockquote><blockquote type=3D"cite"><blockquote =
type=3D"cite">On May 11, 2011, at 8:07 AM, Laura Atkins =
wrote:<br></blockquote></blockquote><blockquote type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">On May =
11, 2011, at 7:01 AM, Murray S. Kucherawy =
wrote:<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">I'm =
not following this thread closely, but I thought I'd say =
something<br></blockquote></blockquote></blockquote></blockquote></blockqu=
ote><blockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">about =
extended status codes. &nbsp;Part of the idea of extended status =
codes<br></blockquote></blockquote></blockquote></blockquote></blockquote>=
<blockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">is =
that you should be able to determine the likely source of =
the<br></blockquote></blockquote></blockquote></blockquote></blockquote><b=
lockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">problem =
by looking at the second facet of the status code. &nbsp;Or to =
put<br></blockquote></blockquote></blockquote></blockquote></blockquote><b=
lockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">it =
another way, the second facet of the status code is supposed =
to<br></blockquote></blockquote></blockquote></blockquote></blockquote><bl=
ockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">indicate=
 _who_ probably needs to fix the problem (e.g. the sender, =
the<br></blockquote></blockquote></blockquote></blockquote></blockquote><b=
lockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">MSA, a =
relay, the delivery agent, =
etc.)<br></blockquote></blockquote></blockquote></blockquote></blockquote>=
<blockquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote></blockquote></blockquote><blo=
ckquote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite">I haven't implemented this, but =
I have to say I really like this approach. &nbsp;Perhaps the proposal =
would benefit from making such distinctions in the new codes it's =
registering.<br></blockquote></blockquote></blockquote></blockquote><block=
quote type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">If =
you're going to make the second facet of the status code be who probably =
needs to fix the problem, you could also make the third facet point to =
where they need to go to fix it. =
<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">It =
doesn't even need to be that detailed. 1 for this is internal (so you =
need to contact the recipient domain) and 2 for this is external (so you =
need to contact a third party). Then follow that with a URL pointing to =
the third party or the postmaster / internal troubleshooting pages. =
<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite">If there's a URL or human =
readable message that's all that's needed for someone to diagnose the =
delivery problem (which is how it's commonly done today). Adding =
additional computer readable information as an extended response code is =
only useful if it's intended to be handled entirely automatically - if =
you're going to put human eyeballs on it, the human readable response is =
all you need, once you've used the SMTP response (e.g. 5xx) to =
triage.<br></blockquote></blockquote><blockquote =
type=3D"cite"><br></blockquote><blockquote type=3D"cite">one of the =
major reasons for enhanced status codes was to provide a =
language-independent indication of the error. &nbsp;&nbsp;&nbsp;I =
certainly don't mind URLs being used to supply more information =
(especially if done in multiple languages) but I don't think it obviates =
the need for the enhanced status codes.<br></blockquote><br>I don't =
think it meets that goal, as it doesn't provide enough information to do =
anything useful in response to those broad categories. Pretty much all =
of the suggested categories break down into two groups:<br><br> &nbsp;1. =
Something went wrong. Keep trying.<br> &nbsp;2. Something went wrong. =
Read the error message.<br></div></blockquote><div><br></div><div>The =
defined meaning of the enhanced status code should be a close =
approximation to the error =
message.&nbsp;</div></div><div><br><blockquote type=3D"cite"><div>t's =
only worthwhile for an SMTP receiver to send these codes or for an SMTP =
sender to parse them if they can be used to adjust the behaviour of the =
sender in a mutually beneficial way.<br><br>The codes as defined are =
broad categories that are informational (as opposed to imperative, e.g. =
"Please don't send email to this address again", "Please don't send any =
further email to this address / domain until a human has intervened", =
"Please reduce the rate at which you are sending email to this MX") so =
they don't seem to communicate much to the senders automation in terms =
of "what behaviour you should change in response to this =
message".<br></div></blockquote><div><br></div>They're not so much meant =
for "automation" - except perhaps for mailing lists - as for human =
consumption.</div><div><br></div><div>Admittedly, the codes weren't =
designed for a world where the sender has to argue with some blackhole =
list about whether his message should get through. &nbsp;But even in the =
world where RFCs 1981-1894 were written, about half of all bounced =
messages were due to errors downstream of the sender - mostly DNS =
errors, MTA configuration errors, and a few recipient disk quota/space =
issues. &nbsp; In general, the sender couldn't fix most of those =
problems directly, but it was still helpful to the sender to know where =
the problem was.</div><div><br></div><div>Today, there's still a very =
useful difference between telling the sender:&nbsp;</div><div><ul =
class=3D"MailOutline"><li>"your message didn't get delivered because the =
recipient address was not that of a valid recipient", =
&nbsp;or</li><li>"your message didn't get delivered because it was too =
large", or</li><li>"your message didn't get delivered because we think =
it's spam based on the content", or&nbsp;</li><li>"your message didn't =
get delivered because blackhole list FOO accused your IP of generating =
spam", or</li><li>"your message didn't get delivered because your =
domain's DNS says that your IP is not authorized to send =
mail"</li></ul></div><div><br><blockquote type=3D"cite"><div>A sysadmin =
who ignores the human readable message (whether due to language issues, =
poor logging or what have you) and relies solely on the extended =
response code is going to be in just the same situation as automated =
code - they're not going to have enough information to make a decision =
about what they should do in response without reading the human readable =
part of the message.<br></div></blockquote><div><br></div>That might be =
true today, because today people are refusing delivery for a lot more =
reasons than anticipated at the time RFCs 1891-4 were written. &nbsp;So =
extending the list of codes seems appropriate.<br><blockquote =
type=3D"cite"><div><br>That's why I'm wondering what the intended use of =
these codes by the email senders is.<br><br>Cheers,<br> =
&nbsp;Steve<br><br></div></blockquote></div><br></body></html>=

--Apple-Mail-54-417924687--

Date: Wed, 11 May 2011 21:31:55 +0200
From: Arnt Gulbrandsen <[email-masked]>
Organization: Me, http://arnt.gulbrandsen.priv.no
To: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

On 05/11/2011 08:04 PM, Keith Moore wrote:
> I've suffered far more harm from people blocking my incoming mail due to input from DNSBLs, than I've ever suffered from spammers.

How I envy you.

Date: Wed, 11 May 2011 15:13:19 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

Hector Santos wrote:
> 
> Murray S. Kucherawy wrote:
>>
>> I would say the angle-bracket form is what would be expected, but 
>> you're right, this looks like an ambiguity.  I would recommend opening 
>> an errata item for it if that hasn't already been done.
> 
> Section 3.5.1 seems pretty clear to me:
> 
>    For the VRFY command, the string is a user name or a user name and
>    domain (see below).....
> 
>       User Name <local-part@domain>
>       local-part@domain
> 
> The former is one of many legal email address forms. In this one form 
> allowed for this command, the angle-bracket is needed.  But if the user 
> name is not going to be entered, it shows a 2nd input form where you 
> don't need to use angle brackets.
> 
> However, for outputs, it MUST be in angle bracket email form.

Oh I see, the example is wrong! To match the text, it should be:

     User Name
     User Name <local-part@domain>

but it seems there would be three input forms:

     User Name
     User Name <local-part@domain>
     local-part@domain

+1 in principle for an errata!!

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: IETF SMTP <[email-masked]>
Date: Wed, 11 May 2011 11:52:37 -0700
Subject: RE: RFC 5321 VRFY and quoting syntax
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Wednesday, May 11, 2011 11:38 AM
> To: IETF SMTP
> Subject: Re: RFC 5321 VRFY and quoting syntax
> 
> Murray S. Kucherawy wrote:
> > I would say the angle-bracket form is what would be expected,
> > but you're right, this looks like an ambiguity.  I would
> > recommend opening an errata item for it if that hasn't already
> > been done.
> 
> Section 3.5.1 seems pretty clear to me:
> [...]

His point is that the ABNF seems to disagree.  I think he's right.

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Wed, 11 May 2011 11:51:12 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Wednesday, May 11, 2011 11:10 AM
> To: [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> If was to become a standard, it would be a standard set of
> recommendations, not a standard for enforcement and for product
> oriented implementations who are in the business of providing options
> and features, not subjective locked out conclusions, it helps to have
> something some have a need for it.

I don't know what "standard set of recommendations" means in this context.  We're talking about providing a protocol, and including in an IANA registry (which is the direction this appeared to be going) some prescribed operator action seems inappropriate to me.  Define what the new code means, and leave it there.

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Steve Atkins <[email-masked]>
Date: Wed, 11 May 2011 11:44:06 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 11:11 AM, Keith Moore wrote:

> On May 11, 2011, at 11:34 AM, Steve Atkins wrote:
> 
>> On May 11, 2011, at 8:07 AM, Laura Atkins wrote:
>> 
>>> On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:
>>> 
>>>>> I'm not following this thread closely, but I thought I'd say something
>>>>> about extended status codes.  Part of the idea of extended status codes
>>>>> is that you should be able to determine the likely source of the
>>>>> problem by looking at the second facet of the status code.  Or to put
>>>>> it another way, the second facet of the status code is supposed to
>>>>> indicate _who_ probably needs to fix the problem (e.g. the sender, the
>>>>> MSA, a relay, the delivery agent, etc.)
>>>> 
>>>> I haven't implemented this, but I have to say I really like this approach.  Perhaps the proposal would benefit from making such distinctions in the new codes it's registering.
>>> 
>>> 
>>> If you're going to make the second facet of the status code be who probably needs to fix the problem, you could also make the third facet point to where they need to go to fix it. 
>>> It doesn't even need to be that detailed. 1 for this is internal (so you need to contact the recipient domain) and 2 for this is external (so you need to contact a third party). Then follow that with a URL pointing to the third party or the postmaster / internal troubleshooting pages. 
>> 
>> If there's a URL or human readable message that's all that's needed for someone to diagnose the delivery problem (which is how it's commonly done today). Adding additional computer readable information as an extended response code is only useful if it's intended to be handled entirely automatically - if you're going to put human eyeballs on it, the human readable response is all you need, once you've used the SMTP response (e.g. 5xx) to triage.
> 
> one of the major reasons for enhanced status codes was to provide a language-independent indication of the error.    I certainly don't mind URLs being used to supply more information (especially if done in multiple languages) but I don't think it obviates the need for the enhanced status codes.

I don't think it meets that goal, as it doesn't provide enough information to do anything useful in response to those broad categories. Pretty much all of the suggested categories break down into two groups:

  1. Something went wrong. Keep trying.
  2. Something went wrong. Read the error message.

t's only worthwhile for an SMTP receiver to send these codes or for an SMTP sender to parse them if they can be used to adjust the behaviour of the sender in a mutually beneficial way.

The codes as defined are broad categories that are informational (as opposed to imperative, e.g. "Please don't send email to this address again", "Please don't send any further email to this address / domain until a human has intervened", "Please reduce the rate at which you are sending email to this MX") so they don't seem to communicate much to the senders automation in terms of "what behaviour you should change in response to this message".

A sysadmin who ignores the human readable message (whether due to language issues, poor logging or what have you) and relies solely on the extended response code is going to be in just the same situation as automated code - they're not going to have enough information to make a decision about what they should do in response without reading the human readable part of the message.

That's why I'm wondering what the intended use of these codes by the email senders is.

Cheers,
  Steve

Date: Wed, 11 May 2011 14:38:28 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

Murray S. Kucherawy wrote:
> 
> I would say the angle-bracket form is what would be expected, 
> but you're right, this looks like an ambiguity.  I would 
> recommend opening an errata item for it if that hasn't already 
> been done.

Section 3.5.1 seems pretty clear to me:

    For the VRFY command, the string is a user name or a user name and
    domain (see below).....

       User Name <local-part@domain>
       local-part@domain

The former is one of many legal email address forms. In this one form 
allowed for this command, the angle-bracket is needed.  But if the 
user name is not going to be entered, it shows a 2nd input form where 
you don't need to use angle brackets.

However, for outputs, it MUST be in angle bracket email form.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 14:11:41 -0400
Cc: SMTP Interest Group <[email-masked]>
To: Steve Atkins <[email-masked]>
Sender: [email-masked]

--Apple-Mail-52-412787303
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On May 11, 2011, at 11:34 AM, Steve Atkins wrote:

> On May 11, 2011, at 8:07 AM, Laura Atkins wrote:
>=20
>> On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:
>>=20
>>>> I'm not following this thread closely, but I thought I'd say =
something
>>>> about extended status codes.  Part of the idea of extended status =
codes
>>>> is that you should be able to determine the likely source of the
>>>> problem by looking at the second facet of the status code.  Or to =
put
>>>> it another way, the second facet of the status code is supposed to
>>>> indicate _who_ probably needs to fix the problem (e.g. the sender, =
the
>>>> MSA, a relay, the delivery agent, etc.)
>>>=20
>>> I haven't implemented this, but I have to say I really like this =
approach.  Perhaps the proposal would benefit from making such =
distinctions in the new codes it's registering.
>>=20
>>=20
>> If you're going to make the second facet of the status code be who =
probably needs to fix the problem, you could also make the third facet =
point to where they need to go to fix it.=20
>> It doesn't even need to be that detailed. 1 for this is internal (so =
you need to contact the recipient domain) and 2 for this is external (so =
you need to contact a third party). Then follow that with a URL pointing =
to the third party or the postmaster / internal troubleshooting pages.=20=

>=20
> If there's a URL or human readable message that's all that's needed =
for someone to diagnose the delivery problem (which is how it's commonly =
done today). Adding additional computer readable information as an =
extended response code is only useful if it's intended to be handled =
entirely automatically - if you're going to put human eyeballs on it, =
the human readable response is all you need, once you've used the SMTP =
response (e.g. 5xx) to triage.

one of the major reasons for enhanced status codes was to provide a =
language-independent indication of the error.    I certainly don't mind =
URLs being used to supply more information (especially if done in =
multiple languages) but I don't think it obviates the need for the =
enhanced status codes.

Keith

--Apple-Mail-52-412787303
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On May 11, 2011, at 11:34 AM, Steve Atkins =
wrote:</div><br><blockquote type=3D"cite"><div>On May 11, 2011, at 8:07 =
AM, Laura Atkins wrote:<br><font class=3D"Apple-style-span" =
color=3D"#006312"><font class=3D"Apple-style-span" =
color=3D"#144FAE"><br></font></font><blockquote type=3D"cite">On May 11, =
2011, at 7:01 AM, Murray S. Kucherawy wrote:<br></blockquote><blockquote =
type=3D"cite"><br></blockquote><blockquote type=3D"cite"><blockquote =
type=3D"cite"><blockquote type=3D"cite">I'm not following this thread =
closely, but I thought I'd say =
something<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">about =
extended status codes. &nbsp;Part of the idea of extended status =
codes<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">is =
that you should be able to determine the likely source of =
the<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">problem =
by looking at the second facet of the status code. &nbsp;Or to =
put<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">it =
another way, the second facet of the status code is supposed =
to<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">indicate=
 _who_ probably needs to fix the problem (e.g. the sender, =
the<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite"><blockquote type=3D"cite">MSA, a =
relay, the delivery agent, =
etc.)<br></blockquote></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote =
type=3D"cite"><br></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite">I haven't implemented this, but =
I have to say I really like this approach. &nbsp;Perhaps the proposal =
would benefit from making such distinctions in the new codes it's =
registering.<br></blockquote></blockquote><blockquote =
type=3D"cite"><br></blockquote><blockquote =
type=3D"cite"><br></blockquote><blockquote type=3D"cite">If you're going =
to make the second facet of the status code be who probably needs to fix =
the problem, you could also make the third facet point to where they =
need to go to fix it. <br></blockquote><blockquote type=3D"cite">It =
doesn't even need to be that detailed. 1 for this is internal (so you =
need to contact the recipient domain) and 2 for this is external (so you =
need to contact a third party). Then follow that with a URL pointing to =
the third party or the postmaster / internal troubleshooting pages. =
<br></blockquote><br>If there's a URL or human readable message that's =
all that's needed for someone to diagnose the delivery problem (which is =
how it's commonly done today). Adding additional computer readable =
information as an extended response code is only useful if it's intended =
to be handled entirely automatically - if you're going to put human =
eyeballs on it, the human readable response is all you need, once you've =
used the SMTP response (e.g. 5xx) to =
triage.<br></div></blockquote><div><br></div>one of the major reasons =
for enhanced status codes was to provide a language-independent =
indication of the error. &nbsp; &nbsp;I certainly don't mind URLs being =
used to supply more information (especially if done in multiple =
languages) but I don't think it obviates the need for the enhanced =
status codes.</div><div><br></div><div>Keith<br></div><br></body></html>=

--Apple-Mail-52-412787303--

Date: Wed, 11 May 2011 14:09:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

If was to become a standard, it would be a standard set of 
recommendations, not a standard for enforcement and for product 
oriented implementations who are in the business of providing options 
and features, not subjective locked out conclusions, it helps to have 
something some have a need for it.

In fact, many smtp systems are already doing this.  Some results do 
trigger more alerts than just appending it in your general smtp.log 
file or session trace logs.

Its my opinion, no one needs to agree or be interested in it, that 
these anti-spam targeted responses can use the similar alert/severity 
insights on what is most smtp systems already do at some level, i.e. 
DoS attacks monitoring, or too many failed AUTH login attempts. For 
our system, an IP block notification is send to all hosting services, 
include FTP, TELNET, NNTP, etc.

Murray S. Kucherawy wrote:
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
>> Sent: Wednesday, May 11, 2011 9:25 AM
>> To: [email-masked]
>> Cc: [email-masked]
>> Subject: Re: slight update to draft-macdonald-antispam-registry
>>
>> For something labeled under "Anti-Spam" related, I like the idea of
>> building a consensus for something along the lines of a "Alert Status"
>> (i.e. LOW, MED, HIGH) that provides the BCP for these X.8.YYY codes:
>>
>>     Code:          5.8.11
>>     Sample Text:   IDENTITY has been compromised
>>     Associated Basic Status Code: 550
>>     Alert Status:  HIGH, Local Operator should be notified
>>
>> I mean, what will most receivers do when the receivers detect and
>> determine the need to issues these X.8.YYY codes?  There was a reason
>> for it and many of these are pretty serious where a) you don't want to
>> encourage a retry, thus 55x 5.8.YYY is issued and b) the local
>> operator may needs to notified rather than just log it.
> 
> I don't think it's a good idea to create a standard that attempts to provoke specific behavior in receivers (in this case, SMTP clients that receive these messages).  Otherwise, I can return 5.8.11 for anything that I think your operators should deal with right away.
> 
> It's sufficient to describe the problem in detail and let the operator decide what's critical and what isn't.
> 
> 
> 
> 

-- 
Sincerely

Hector Santos
http://www.santronics.com

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 14:07:45 -0400
Cc: [email-masked]
To: [email-masked]
Sender: [email-masked]

--Apple-Mail-51-412550853
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On May 11, 2011, at 11:19 AM, Dave CROCKER wrote:
>=20
> On 5/11/2011 6:51 AM, John Levine wrote:
>>> The problem I have with this argument is that blackhole lists, in my
>>> experience, cause a large number of legitimate messages to fail to =
be
>>> delivered.
>>=20
>> Having gone to a lot of MAAWG meetings, and talked to people who run
>> the mail systems at every large ISP in the country, and quite a few =
in
>> other countries, I can report that their experience with DNSBLs is
>> utterly unlike yours.  There are plenty of incompetently run DNSBLs,
>> but nobody uses them so they don't matter.
>=20
>=20
> Badly-run blacklists do cause problems.  But, then, few people choose =
to use bad lists.
>=20
> Blacklists are an integral part of real-world email operations today.
>=20
> When thinking about the scale and number of the major email service =
providers, it is not reasonable to suggest that they would continue that =
use were there a significant problem with false positives.

I'm not so worried about the major providers.   Most of them seem to be =
competent, these days.  But it was not always the case, and it might not =
always be the case in the future.  We should be careful about =
overgeneralizing from present circumstances.  The net changes =
dramatically every two years.

Keith

--Apple-Mail-51-412550853
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On May 11, 2011, at 11:19 AM, Dave CROCKER =
wrote:</div><blockquote type=3D"cite"><div><font =
class=3D"Apple-style-span" color=3D"#000000"><br></font>On 5/11/2011 =
6:51 AM, John Levine wrote:<br><blockquote type=3D"cite"><blockquote =
type=3D"cite">The problem I have with this argument is that blackhole =
lists, in my<br></blockquote></blockquote><blockquote =
type=3D"cite"><blockquote type=3D"cite">experience, cause a large number =
of legitimate messages to fail to =
be<br></blockquote></blockquote><blockquote type=3D"cite"><blockquote =
type=3D"cite">delivered.<br></blockquote></blockquote><blockquote =
type=3D"cite"><br></blockquote><blockquote type=3D"cite">Having gone to =
a lot of MAAWG meetings, and talked to people who =
run<br></blockquote><blockquote type=3D"cite">the mail systems at every =
large ISP in the country, and quite a few in<br></blockquote><blockquote =
type=3D"cite">other countries, I can report that their experience with =
DNSBLs is<br></blockquote><blockquote type=3D"cite">utterly unlike =
yours. &nbsp;There are plenty of incompetently run =
DNSBLs,<br></blockquote><blockquote type=3D"cite">but nobody uses them =
so they don't matter.<br></blockquote><br><br>Badly-run blacklists do =
cause problems. &nbsp;But, then, few people choose to use bad =
lists.<br><br>Blacklists are an integral part of real-world email =
operations today.<br><br>When thinking about the scale and number of the =
major email service providers, it is not reasonable to suggest that they =
would continue that use were there a significant problem with false =
positives.<br></div></blockquote></div><br><div>I'm not so worried about =
the major providers. &nbsp; Most of them seem to be competent, these =
days. &nbsp;But it was not always the case, and it might not always be =
the case in the future. &nbsp;We should be careful about =
overgeneralizing from present circumstances. &nbsp;The net changes =
dramatically every two =
years.</div><div><br></div><div>Keith</div><div><br></div><div><br></div><=
div><br></div></body></html>=

--Apple-Mail-51-412550853--

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 14:04:52 -0400
Cc: [email-masked]
To: John Levine <[email-masked]>
Sender: [email-masked]

John,

Assuming for the sake of argument that you're reporting the situation (mostly) accurately for major ISPs, I still think it's important that blacklists and reputation servers be explicitly cited when they're used to bounce a message.  

It's certainly not the case that "nobody" uses incompetently run DNSBLs, even if none of the major ISPs use them.   And it's not as if major ISPs handle all of the incoming mail.  The standards exist for everyone, not just major ISPs. 

I've suffered far more harm from people blocking my incoming mail due to input from DNSBLs, than I've ever suffered from spammers.    But it's not as if one is evil and the other isn't.  They're both evil.  Rather than trying to ignore one kind of evil while blocking another, some sort of balanced approach seems in order.

Keith

On May 11, 2011, at 9:51 AM, John Levine wrote:

> 
>> The problem I have with this argument is that blackhole lists, in my
>> experience, cause a large number of legitimate messages to fail to be
>> delivered.
> 
> Having gone to a lot of MAAWG meetings, and talked to people who run
> the mail systems at every large ISP in the country, and quite a few in
> other countries, I can report that their experience with DNSBLs is
> utterly unlike yours.  There are plenty of incompetently run DNSBLs,
> but nobody uses them so they don't matter.
> 
> I have my reservations about the wisdom or utility of providing fine
> grained reports to people whose mail you don't accept, but not because
> of error rates in DNSBLs.
> 
> R's,
> John
> 

From: "Murray S. Kucherawy" <[email-masked]>
To: "A. Rothman" <[email-masked]>, IETF SMTP <[email-masked]>
Date: Wed, 11 May 2011 10:47:08 -0700
Subject: RE: RFC 5321 VRFY and quoting syntax
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of A. Rothman
> Sent: Monday, May 09, 2011 8:03 AM
> To: IETF SMTP
> Subject: RFC 5321 VRFY and quoting syntax
> 
> 1. The syntax definition for commands such as VRFY and EXPN (e.g.
> section 4.1.1.6) is
> 
> vrfy = "VRFY" SP String CRLF
> 
> with the syntax of "String" defined in section 4.1.2 as:
> 
> String         = Atom / Quoted-string
> 
> however, section 3.5.1 states:
> 
>     The character string arguments of the VRFY and EXPN commands cannot
>     be further restricted due to the variety of implementations of the
>     user name and mailbox list concepts.
> 
> What is not entirely clear to me is which is true - is any string of
> characters valid (syntax-wise), or must it be a Quoted-string (in double
> quotes) if any non-atext characters appear in it. This is especially
> confusing having seen various examples online (and some discussions on
> this mailing list as well) where pointed brackets are included in the
> VRFY argument (using a Path syntax similar to the one defined for the
> MAIL FROM and RCPT TO commands), but with no quoting.

I'm missing how your first sentence here is a conflict.  It has to be a quoted-string if there are any non-atext characters, otherwise the quoting is not needed.

Also, a good point on the angle brackets.  The syntax seems to suggest they're not valid (or, indeed, need quoting themselves).  I'm sure it's commonly accepted this way since the most commonly used SMTP commands (MAIL and RCPT) require them but most implementations also usually tolerate their absence.

> 2. Section 4.1.2 defines the backslash-escaped character mechanism in
> quoted-pairSMTP, which is used only in a Quoted-string (within
> double-quotes), and does not mention such escaping outside of a
> Quoted-string. The following text section states:
> 
>     Note that the backslash, "\", is a quote character, which is used to
>     indicate that the next character is to be used literally (instead of
>     its normal interpretation).  For example, "Joe\,Smith" indicates a
>     single nine-character user name string with the comma being the
>     fourth character of that string.
> 
> So, it is unclear whether this paragraph applies only to the
> Quoted-strings defined above, or to any characters in any argument to
> any command, or only to mailboxes (discussed in the preceding paragraph)
> or some other definition of when it does and does not apply.

It appears to apply only within quoted strings.  Its main function then would be to escape quotation marks within quoted strings, because everything else is not a character that separates tokens.

> 3. Still regarding VRFY (and maybe also EXPN?) section 3.5.1 states:
> 
>     If a normal (i.e., 250) response is returned,
>     the response MAY include the full name of the user and MUST include
>     the mailbox of the user.  It MUST be in either of the following
>     forms:
> 
>        User Name<local-part@domain>
>        local-part@domain
> 
> Whereas section 3.5.2 claims:
> 
>     When normal (2yz or 551) responses are returned from a VRFY or EXPN
>     request, the reply MUST include the<Mailbox>  name using a
>     "<local-part@domain>" construction
> 
> Notably, the second example in the former section does not comply with
> the latter section (as it contains no pointed brackets). Which is the
> correct form?

I would say the angle-bracket form is what would be expected, but you're right, this looks like an ambiguity.  I would recommend opening an errata item for it if that hasn't already been done.

-MSK

Date: Wed, 11 May 2011 10:26:34 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Tim Kehres <[email-masked]>
CC: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

On 5/11/2011 9:45 AM, Tim Kehres wrote:
> Yes, I agree with your comments below. I thought I'd just throw in an alternate
> minority view. :-)

ack. tnx.

and it is probably good to have the archive include such points of what I will 
constructively call 'nuance'...

d/.
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Wed, 11 May 2011 10:20:53 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Wednesday, May 11, 2011 9:25 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> For something labeled under "Anti-Spam" related, I like the idea of
> building a consensus for something along the lines of a "Alert Status"
> (i.e. LOW, MED, HIGH) that provides the BCP for these X.8.YYY codes:
> 
>     Code:          5.8.11
>     Sample Text:   IDENTITY has been compromised
>     Associated Basic Status Code: 550
>     Alert Status:  HIGH, Local Operator should be notified
> 
> I mean, what will most receivers do when the receivers detect and
> determine the need to issues these X.8.YYY codes?  There was a reason
> for it and many of these are pretty serious where a) you don't want to
> encourage a retry, thus 55x 5.8.YYY is issued and b) the local
> operator may needs to notified rather than just log it.

I don't think it's a good idea to create a standard that attempts to provoke specific behavior in receivers (in this case, SMTP clients that receive these messages).  Otherwise, I can return 5.8.11 for anything that I think your operators should deal with right away.

It's sufficient to describe the problem in detail and let the operator decide what's critical and what isn't.

Date: Wed, 11 May 2011 13:11:06 -0400 (EDT)
From: Derek Diget <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

First, disclaimer...I have not read the draft, but please read on.

On May 11, 2011 at 08:34 -0700, Steve Atkins wrote:
=>On May 11, 2011, at 8:07 AM, Laura Atkins wrote:
=>> On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:
=>>>> I'm not following this thread closely, but I thought I'd say something
=>>>> about extended status codes.  Part of the idea of extended status codes
=>>>> is that you should be able to determine the likely source of the
=>>>> problem by looking at the second facet of the status code.  Or to put
=>>>> it another way, the second facet of the status code is supposed to
=>>>> indicate _who_ probably needs to fix the problem (e.g. the sender, the
=>>>> MSA, a relay, the delivery agent, etc.)
=>>> 
=>>> I haven't implemented this, but I have to say I really like this 
=>>> approach.  Perhaps the proposal would benefit from making such 
=>>> distinctions in the new codes it's registering.
=>> 
=>> If you're going to make the second facet of the status code be who 
=>> probably needs to fix the problem, you could also make the third 
=>> facet point to where they need to go to fix it.
=>>
=>> It doesn't even need to be that detailed. 1 for this is internal (so 
=>> you need to contact the recipient domain) and 2 for this is external 
=>> (so you need to contact a third party). Then follow that with a URL 
=>> pointing to the third party or the postmaster / internal 
=>> troubleshooting pages.
=>
=>If there's a URL or human readable message that's all that's needed 
=>for someone to diagnose the delivery problem (which is how it's 
=>commonly done today). Adding additional computer readable information 
=>as an extended response code is only useful if it's intended to be 
=>handled entirely automatically - if you're going to put human eyeballs 
=>on it, the human readable response is all you need, once you've used 
=>the SMTP response (e.g. 5xx) to triage.
=>
=>With the extended status codes as listed in this draft I don't see 
=>there being enough additional information provided to automatically 
=>alter the senders behaviour usefully in most cases, beyond the way 
=>they're already being signalled by the SMTP status codes. There's 
=>always going to need to be either a human looking at the response and 
=>making a decision - that most of the listed enhanced status codes 
=>include the phrase "see URL for further details" is a tacit admission 
=>of that - or more sophisticated automation that depends on many things 
=>other than the enhanced code.
=>
=>What is the intended use by senders of these codes (assuming that some 
=>significant subset of receivers were to implement them, and map their 
=>existing responses on to them as accurately as they can)?

Somewhat agree.  My feeling is that humans read the status code text as 
you say, but log parsers/report generators and/or queue watchers are 
looking at the status codes.  None of those can "fix" the problem(s), 
but they bring the problem to the attention of sites running them.  
Both as a sender and receiver.  (Note how this ties into Hector's...only 
good guys care/report, never bad guys. :)

-- 
***********************************************************************
Derek Diget                            Office of Information Technology
Western Michigan University - Kalamazoo  Michigan  USA - www.wmich.edu/
***********************************************************************

Date: Thu, 12 May 2011 00:45:47 +0800
From: Tim Kehres <[email-masked]>
To: [email-masked]
CC: Dave CROCKER <[email-masked]>, [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Dave,

Yes, I agree with your comments below.   I thought I'd just throw in an 
alternate minority view.   :-)  Over the years RBL's have caused much 
grief in the systems here.  My personal belief is that while they make 
an administrators life easier, they are bad for any business that can be 
hurt with false positives.   From my perspective that is any business 
that relies on email for their primary means of communications.

So for what we do here, my preference is to continue to rely on more 
deterministic methods for keeping the unwanted message volume down, 
while at the same time keeping the false positive rate at or near zero.

Best Regards,

-- Tim

On 5/12/2011 12:09 AM, Dave CROCKER wrote:
>
>
>
> On 5/11/2011 9:03 AM, Tim Kehres wrote:
>> I guess it all depends on what you consider a 'significant problem'. 
>> The metrics
>> used to make this determination will vary from business to business. 
>> For the
>> several sites that I run here (small in comparison), any false positives
>> represent a potential loss of business or a hit on business and we do 
>> everything
>> we can to avoid. Other mechanisms used in tandem seem to knock the 
>> spam levels
>> down to a level, at least for us, that are acceptable without the 
>> uncertainties
>> associated with RBL's.
>
>
> Hi, Tim.
>
> The issue is not that some folk comfortably choose not to use such 
> lists.  There are always such cases.
>
> The question is whether they are in widespread use (they are), whether 
> they are deemed essential by those users (it is) and whether they 
> cause the cited problem (apparently only minimally).
>
> False positives can be extremely damaging.  I've nearly lost some 
> consulting business because my filters were overly aggressive.  But 
> that said, the industry very clearly deems them generally a net positive.
>
> d/

Date: Wed, 11 May 2011 12:24:48 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
To: [email-masked]
Sender: [email-masked]

For something labeled under "Anti-Spam" related, I like the idea of 
building a consensus for something along the lines of a "Alert Status" 
(i.e. LOW, MED, HIGH) that provides the BCP for these X.8.YYY codes:

    Code:          5.8.11
    Sample Text:   IDENTITY has been compromised
    Associated Basic Status Code: 550
    Alert Status:  HIGH, Local Operator should be notified

I mean, what will most receivers do when the receivers detect and 
determine the need to issues these X.8.YYY codes?  There was a reason 
for it and many of these are pretty serious where a) you don't want to 
encourage a retry, thus 55x 5.8.YYY is issued and b) the local 
operator may needs to notified rather than just log it.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Laura Atkins wrote:
> 
> On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:
> 
>>> I'm not following this thread closely, but I thought I'd say something
>>> about extended status codes.  Part of the idea of extended status codes
>>> is that you should be able to determine the likely source of the
>>> problem by looking at the second facet of the status code.  Or to put
>>> it another way, the second facet of the status code is supposed to
>>> indicate _who_ probably needs to fix the problem (e.g. the sender, the
>>> MSA, a relay, the delivery agent, etc.)
>> I haven't implemented this, but I have to say I really like this approach.  Perhaps the proposal would benefit from making such distinctions in the new codes it's registering.
> 
> 
> If you're going to make the second facet of the status code be who probably needs to fix the problem, you could also make the third facet point to where they need to go to fix it. 
> It doesn't even need to be that detailed. 1 for this is internal (so you need to contact the recipient domain) and 2 for this is external (so you need to contact a third party). Then follow that with a URL pointing to the third party or the postmaster / internal troubleshooting pages. 
> 
> laura 
> 

Date: Wed, 11 May 2011 09:09:43 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: Tim Kehres <[email-masked]>
CC: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

On 5/11/2011 9:03 AM, Tim Kehres wrote:
> I guess it all depends on what you consider a 'significant problem'. The metrics
> used to make this determination will vary from business to business. For the
> several sites that I run here (small in comparison), any false positives
> represent a potential loss of business or a hit on business and we do everything
> we can to avoid. Other mechanisms used in tandem seem to knock the spam levels
> down to a level, at least for us, that are acceptable without the uncertainties
> associated with RBL's.

Hi, Tim.

The issue is not that some folk comfortably choose not to use such lists.  There 
are always such cases.

The question is whether they are in widespread use (they are), whether they are 
deemed essential by those users (it is) and whether they cause the cited problem 
(apparently only minimally).

False positives can be extremely damaging.  I've nearly lost some consulting 
business because my filters were overly aggressive.  But that said, the industry 
very clearly deems them generally a net positive.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Thu, 12 May 2011 00:03:35 +0800
From: Tim Kehres <[email-masked]>
To: [email-masked]
CC: Dave CROCKER <[email-masked]>, [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

I guess it all depends on what you consider a 'significant problem'.  
The metrics used to make this determination will vary from business to 
business.  For the several sites that I run here (small in comparison), 
any false positives represent a potential loss of business or a hit on 
business and we do everything we can to avoid.  Other mechanisms used in 
tandem seem to knock the spam levels down to a level, at least for us, 
that are acceptable without the uncertainties associated with RBL's.

Best Regards,

-- Tim

On 5/11/2011 11:19 PM, Dave CROCKER wrote:
>
>
>
> On 5/11/2011 6:51 AM, John Levine wrote:
>>> The problem I have with this argument is that blackhole lists, in my
>>> experience, cause a large number of legitimate messages to fail to be
>>> delivered.
>>
>> Having gone to a lot of MAAWG meetings, and talked to people who run
>> the mail systems at every large ISP in the country, and quite a few in
>> other countries, I can report that their experience with DNSBLs is
>> utterly unlike yours.  There are plenty of incompetently run DNSBLs,
>> but nobody uses them so they don't matter.
>
>
> Badly-run blacklists do cause problems.  But, then, few people choose 
> to use bad lists.
>
> Blacklists are an integral part of real-world email operations today.
>
> When thinking about the scale and number of the major email service 
> providers, it is not reasonable to suggest that they would continue 
> that use were there a significant problem with false positives.
>
> d/
>

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Steve Atkins <[email-masked]>
Date: Wed, 11 May 2011 08:34:24 -0700
To: SMTP Interest Group <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 8:07 AM, Laura Atkins wrote:

> 
> 
> On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:
> 
>>> I'm not following this thread closely, but I thought I'd say something
>>> about extended status codes.  Part of the idea of extended status codes
>>> is that you should be able to determine the likely source of the
>>> problem by looking at the second facet of the status code.  Or to put
>>> it another way, the second facet of the status code is supposed to
>>> indicate _who_ probably needs to fix the problem (e.g. the sender, the
>>> MSA, a relay, the delivery agent, etc.)
>> 
>> I haven't implemented this, but I have to say I really like this approach.  Perhaps the proposal would benefit from making such distinctions in the new codes it's registering.
> 
> 
> If you're going to make the second facet of the status code be who probably needs to fix the problem, you could also make the third facet point to where they need to go to fix it. 
> It doesn't even need to be that detailed. 1 for this is internal (so you need to contact the recipient domain) and 2 for this is external (so you need to contact a third party). Then follow that with a URL pointing to the third party or the postmaster / internal troubleshooting pages. 

If there's a URL or human readable message that's all that's needed for someone to diagnose the delivery problem (which is how it's commonly done today). Adding additional computer readable information as an extended response code is only useful if it's intended to be handled entirely automatically - if you're going to put human eyeballs on it, the human readable response is all you need, once you've used the SMTP response (e.g. 5xx) to triage.

With the extended status codes as listed in this draft I don't see there being enough additional information provided to automatically alter the senders behaviour usefully in most cases, beyond the way they're already being signalled by the SMTP status codes. There's always going to need to be either a human looking at the response and making a decision - that most of the listed enhanced status codes include the phrase "see URL for further details" is a tacit admission of that - or more sophisticated automation that depends on many things other than the enhanced code.

What is the intended use by senders of these codes (assuming that some significant subset of receivers were to implement them, and map their existing responses on to them as accurately as they can)?

Cheers,
  Steve

Date: Wed, 11 May 2011 08:19:50 -0700
From: Dave CROCKER <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

On 5/11/2011 6:51 AM, John Levine wrote:
>> The problem I have with this argument is that blackhole lists, in my
>> experience, cause a large number of legitimate messages to fail to be
>> delivered.
>
> Having gone to a lot of MAAWG meetings, and talked to people who run
> the mail systems at every large ISP in the country, and quite a few in
> other countries, I can report that their experience with DNSBLs is
> utterly unlike yours.  There are plenty of incompetently run DNSBLs,
> but nobody uses them so they don't matter.

Badly-run blacklists do cause problems.  But, then, few people choose to use bad 
lists.

Blacklists are an integral part of real-world email operations today.

When thinking about the scale and number of the major email service providers, 
it is not reasonable to suggest that they would continue that use were there a 
significant problem with false positives.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Laura Atkins <[email-masked]>
Date: Wed, 11 May 2011 08:07:07 -0700
Cc: "[email-masked]" <[email-masked]>
Sender: [email-masked]

On May 11, 2011, at 7:01 AM, Murray S. Kucherawy wrote:

>> I'm not following this thread closely, but I thought I'd say something
>> about extended status codes.  Part of the idea of extended status codes
>> is that you should be able to determine the likely source of the
>> problem by looking at the second facet of the status code.  Or to put
>> it another way, the second facet of the status code is supposed to
>> indicate _who_ probably needs to fix the problem (e.g. the sender, the
>> MSA, a relay, the delivery agent, etc.)
> 
> I haven't implemented this, but I have to say I really like this approach.  Perhaps the proposal would benefit from making such distinctions in the new codes it's registering.

If you're going to make the second facet of the status code be who probably needs to fix the problem, you could also make the third facet point to where they need to go to fix it. 
It doesn't even need to be that detailed. 1 for this is internal (so you need to contact the recipient domain) and 2 for this is external (so you need to contact a third party). Then follow that with a URL pointing to the third party or the postmaster / internal troubleshooting pages. 

laura 

-- 
Laura Atkins
Word to the Wise			"The Deliverability Experts!"
Direct: 650 678-3454		Fax: 650 249-1909
AIM: wttwlaura			YIM: wttw_laura
Delivery blog: <http://blog.wordtothewise.com/>

Date: Wed, 11 May 2011 10:44:54 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

John Levine wrote:
>> The problem I have with this argument is that blackhole lists, in my
>> experience, cause a large number of legitimate messages to fail to be
>> delivered.
> 
> Having gone to a lot of MAAWG meetings, and talked to people who run
> the mail systems at every large ISP in the country, and quite a few in
> other countries, I can report that their experience with DNSBLs is
> utterly unlike yours.  There are plenty of incompetently run DNSBLs,
> but nobody uses them so they don't matter.

Here's the thing, its much more popular at the smaller systems level 
and the larger systems were the late comers.  So its popularity is 
quite massive even without the larger system or trade group MAAWG 
endorsement.

Our default sites settings for DNSRBLs are:

   sbl.spamhaus.org
   list.dsbl.org
   bl.spamcop.net

In that initial order which as based efficiency and turn around time. 
  The algorithm monitors changes the order based on response rates. 
Higher overhead lookups moves the site to the bottom, better responses 
move it to the top of the query test.

> I have my reservations about the wisdom or utility of providing fine
> grained reports to people whose mail you don't accept, but not because
> of error rates in DNSBLs.

+1

-- 
Sincerely

Hector Santos
http://www.santronics.com

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Wed, 11 May 2011 07:01:08 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Keith Moore
> Sent: Wednesday, May 11, 2011 4:44 AM
> To: Hector Santos
> Cc: Jeff Macdonald; [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> I'm not following this thread closely, but I thought I'd say something
> about extended status codes.  Part of the idea of extended status codes
> is that you should be able to determine the likely source of the
> problem by looking at the second facet of the status code.  Or to put
> it another way, the second facet of the status code is supposed to
> indicate _who_ probably needs to fix the problem (e.g. the sender, the
> MSA, a relay, the delivery agent, etc.)

I haven't implemented this, but I have to say I really like this approach.  Perhaps the proposal would benefit from making such distinctions in the new codes it's registering.

Date: 11 May 2011 13:51:45 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Organization: 
Sender: [email-masked]

>The problem I have with this argument is that blackhole lists, in my
>experience, cause a large number of legitimate messages to fail to be
>delivered.

Having gone to a lot of MAAWG meetings, and talked to people who run
the mail systems at every large ISP in the country, and quite a few in
other countries, I can report that their experience with DNSBLs is
utterly unlike yours.  There are plenty of incompetently run DNSBLs,
but nobody uses them so they don't matter.

I have my reservations about the wisdom or utility of providing fine
grained reports to people whose mail you don't accept, but not because
of error rates in DNSBLs.

R's,
John

Date: Wed, 11 May 2011 09:51:23 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Keith Moore <[email-masked]>
CC: Jeff Macdonald <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

No argument from me here (and your other post). DNS based RBL has been 
a very controversial issue. But in this day of massive abuse, with not 
much else to go by, it has help, popular and yes, even hurt legitimate 
senders.

In my experience though, the good guys generally do make the next move 
to get it corrected. I never got a call from a bad guy. :)

I think you are right that many of these codes labeled as "Anti-Spam" 
Extended codes do fall under different categories some which are 
perfectly reasonable and may reflect a configuration issue, temporary 
DNS related issues.  Just the other day I got a DSN for a list member 
in our support group send after repeated exhausted attempted where he 
was the only one issuing a MAIL FROM response:

  C: MAIL FROM:<[email-masked]>
  S: 452 4.1.0 Policy violation. Your host 208.247.131.9 has
     no valid Reverse DNS. Please contact your ISP for further 
information.

when in fact it does PTR records.  So the member hosting system had 
some temporary DNS issue of course, the member informed and problem 
fixed.  Not quite the right codes IMO, a waste on our system retries, 
but ultimately resolved.

---

Keith Moore wrote:
> On May 10, 2011, at 2:55 PM, Hector Santos wrote:
>> Finally, this is a subject of opinions.  IMV, the I-D extended responses are focused for bad guys rejects, not good guys rejects except for the new X.8.26.
>>
>> So in general, for my own system, I don't particular like to give bad guys "clues" why their transaction failed.  For false positives, it can help the support process in a DSN, but for bad guys, its not like they will listen to them anyway or even get them or maybe redirected to someone else. They are going to blast away anyway.
>>
>> So if you can extract how these "bad guy" transaction rejection responses can help good guys solve a particular problem, more readers will consider it.
> 
> The problem I have with this argument is that blackhole lists, in my experience, cause a large number of legitimate messages to fail to be delivered.   I don't think it's reasonable to assume that just because a message fails some third party's arbitrary criteria, that the problem is with the sender (or purported sender) of the message.
> 
> In other words, sometimes the "bad guy" is the sender, and sometimes it's the blackhole list.
> 
> Keith

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 09:12:18 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: "A. Rothman" <[email-masked]>
CC: IETF SMTP <[email-masked]>
Subject: Re: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

I think that is it saying for INPUT, you can take either the angle 
bracket or non-angle bracket form, but for OUTPUT it MUST be shown in 
angle bracket form.  It is analogous to looking up someone by name in 
a phone book - ultimately it is the single output form - telephone #

Yes, I agree that in many areas of software, the term QUOTE can be 
mixed up with the idea of ESCAPING.  In my opinion, ESCAPING is the 
proper communications term as it used in many data communications 
protocols.

    http://www.google.com/search?q=define:character+escaping

For VRFY, EXPN, the escaping "\" (SLASH) character is used to escape 
the next character which could be a formal delimiting character to 
separate multiple inputs.  For example:

    VRFY   santos,hector,shane,[email-masked]

This would be would be four "local account" user name changes,

   santos
   hector,
   shane,
   [email-masked]

And depending on the host on how it looks up users, it be by some name 
or even address.   Some systems (like ours) has three identities per 
user account; A number ID, user name and email address. And the user 
name may have an old school MHS format (first.last), and in addition 
it may take into account the possible internal translations a system 
may have like BOSS, PRESIDENT, SYSOP, SUPPORT etc.

Versus:

    VRFY  santos\,hector,shane,[email-masked]

which would be three accounts to look up:

    santos,hector
    shane
    [email-masked]

-- 
Sincerely

Hector Santos
http://www.santronics.com

A. Rothman wrote:
> 
> Hi,
> 
> 
> I was hoping someone can shed some light on the proper syntax an SMTP 
> server should use according to RFC 5321 in the following cases:
> 
> 
> 1. The syntax definition for commands such as VRFY and EXPN (e.g. 
> section 4.1.1.6) is
> 
> vrfy = "VRFY" SP String CRLF
> 
> with the syntax of "String" defined in section 4.1.2 as:
> 
> String         = Atom / Quoted-string
> 
> however, section 3.5.1 states:
> 
>    The character string arguments of the VRFY and EXPN commands cannot
>    be further restricted due to the variety of implementations of the
>    user name and mailbox list concepts.
> 
> What is not entirely clear to me is which is true - is any string of 
> characters valid (syntax-wise), or must it be a Quoted-string (in double 
> quotes) if any non-atext characters appear in it. This is especially 
> confusing having seen various examples online (and some discussions on 
> this mailing list as well) where pointed brackets are included in the 
> VRFY argument (using a Path syntax similar to the one defined for the 
> MAIL FROM and RCPT TO commands), but with no quoting.
> 
> 2. Section 4.1.2 defines the backslash-escaped character mechanism in 
> quoted-pairSMTP, which is used only in a Quoted-string (within 
> double-quotes), and does not mention such escaping outside of a 
> Quoted-string. The following text section states:
> 
>    Note that the backslash, "\", is a quote character, which is used to
>    indicate that the next character is to be used literally (instead of
>    its normal interpretation).  For example, "Joe\,Smith" indicates a
>    single nine-character user name string with the comma being the
>    fourth character of that string.
> 
> So, it is unclear whether this paragraph applies only to the 
> Quoted-strings defined above, or to any characters in any argument to 
> any command, or only to mailboxes (discussed in the preceding paragraph) 
> or some other definition of when it does and does not apply.
> 
> 3. Still regarding VRFY (and maybe also EXPN?) section 3.5.1 states:
> 
>    If a normal (i.e., 250) response is returned,
>    the response MAY include the full name of the user and MUST include
>    the mailbox of the user.  It MUST be in either of the following
>    forms:
> 
>       User Name<local-part@domain>
>       local-part@domain
> 
> Whereas section 3.5.2 claims:
> 
> 
>    When normal (2yz or 551) responses are returned from a VRFY or EXPN
>    request, the reply MUST include the<Mailbox>  name using a
>    "<local-part@domain>" construction
> 
> Notably, the second example in the former section does not comply with 
> the latter section (as it contains no pointed brackets). Which is the 
> correct form?
> 
> 
> Any clarification would be greatly appreciated - thanks!
> 
> 
> Amichai
> 
> 
> 
> 
> 

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 08:32:45 -0400
Cc: Jeff Macdonald <[email-masked]>, "[email-masked]" <[email-masked]>
To: Hector Santos <[email-masked]>
Sender: [email-masked]

--Apple-Mail-49-392451084
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On May 10, 2011, at 2:55 PM, Hector Santos wrote:
>=20
> Finally, this is a subject of opinions.  IMV, the I-D extended =
responses are focused for bad guys rejects, not good guys rejects except =
for the new X.8.26.
>=20
> So in general, for my own system, I don't particular like to give bad =
guys "clues" why their transaction failed.  For false positives, it can =
help the support process in a DSN, but for bad guys, its not like they =
will listen to them anyway or even get them or maybe redirected to =
someone else. They are going to blast away anyway.
>=20
> So if you can extract how these "bad guy" transaction rejection =
responses can help good guys solve a particular problem, more readers =
will consider it.

The problem I have with this argument is that blackhole lists, in my =
experience, cause a large number of legitimate messages to fail to be =
delivered.   I don't think it's reasonable to assume that just because a =
message fails some third party's arbitrary criteria, that the problem is =
with the sender (or purported sender) of the message.

In other words, sometimes the "bad guy" is the sender, and sometimes =
it's the blackhole list.

Keith=

--Apple-Mail-49-392451084
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div><div>On May 10, 2011, at 2:55 PM, Hector Santos =
wrote:</div><blockquote type=3D"cite"><div><font =
class=3D"Apple-style-span" color=3D"#000000"><br></font>Finally, this is =
a subject of opinions. &nbsp;IMV, the I-D extended responses are focused =
for bad guys rejects, not good guys rejects except for the new =
X.8.26.<br><br>So in general, for my own system, I don't particular like =
to give bad guys "clues" why their transaction failed. &nbsp;For false =
positives, it can help the support process in a DSN, but for bad guys, =
its not like they will listen to them anyway or even get them or maybe =
redirected to someone else. They are going to blast away =
anyway.<br><br>So if you can extract how these "bad guy" transaction =
rejection responses can help good guys solve a particular problem, more =
readers will consider it.<br></div></blockquote></div><br><div>The =
problem I have with this argument is that blackhole lists, in my =
experience, cause a large number of legitimate messages to fail to be =
delivered. &nbsp; I don't think it's reasonable to assume that just =
because a message fails some third party's arbitrary criteria, that the =
problem is with the sender (or purported sender) of the =
message.</div><div><br></div><div>In other words, sometimes the "bad =
guy" is the sender, and sometimes it's the blackhole =
list.</div><div><br></div><div>Keith</div></body></html>=

--Apple-Mail-49-392451084--

Subject: Re: slight update to draft-macdonald-antispam-registry
From: Keith Moore <[email-masked]>
Date: Wed, 11 May 2011 07:43:46 -0400
Cc: Jeff Macdonald <[email-masked]>, "[email-masked]" <[email-masked]>
To: Hector Santos <[email-masked]>
Sender: [email-masked]

I'm not following this thread closely, but I thought I'd say something about extended status codes.  Part of the idea of extended status codes is that you should be able to determine the likely source of the problem by looking at the second facet of the status code.  Or to put it another way, the second facet of the status code is supposed to indicate _who_ probably needs to fix the problem (e.g. the sender, the MSA, a relay, the delivery agent, etc.)

When mail is blocked because of information from a party that is not actually involved in transmission (e.g. a blackhole list), it makes sense to have a separate 2nd facet code for this, because (in some sense) it's the blackhole list that has to fix the problem before similar messages can be delivered in the future.   But the list below mixes several different sources in the same facet code.  

Keith  (not the author of 1893, but I did come up with the scheme for numbering extended status codes)

On May 11, 2011, at 6:55 AM, Hector Santos wrote:

> 
> Hector Santos wrote:
> 
>> Also, one other category (Alert Status) perhaps to consider per code to provide insight on what codes are good candidate for internal reporting, feedback to operators, red "alert status", etc, so when the receiver detecting a reason for example x.8.11, it can provide a suggestion about the severity of the detection:
>>   Code:        | X.8.11
>>   Sample Text: | IDENTITY has been compromised
>>   Alert Status:| HIGH, Operator should be notified
> 
> BTW, this particular code, should be recommended to be a 5.8.11. I don't think we want to encourage a 4.8.11 for a detected compromised account.   How about others like this?  Hell, most of them :)
> 
> This is my fast, off the top of my head, opinion of the 55x vs 45x separation:
> 
> +-----------------------------------------------------------------------+
>   | Reply | Extended|       |
>   | Code  | Code    | Sample Response Text       |
> |-------+---------+-----------------------------------------------------|
>   | any   | X.8.0   | Undefined Policy detail       |
>   | 550   | 5.8.1   | Message refused by local policy       |
>   | any   | X.8.2   | Excessive volume       |
>   | 550   | 5.8.3   | IP listed on RBL RBL-NAME       |
>   | any   | X.8.4   | Too many invalid recipients       |
>   | 550   | 5.8.5   | Unacceptable content       |
>   | 550   | 5.8.6   | Suspected phishing attempt       |
>   | any   | X.8.7   | IDENTITY is dynamically blocked due to complaints   |
>   | 550   | 5.8.8   | IDENTITY is permanently blocked due to complaints   |
>   | any   | X.8.9   | Recipient not accepting messages from IDENTITY      |
>   | any   | X.8.10  | IDENTITY is a dynamic IP       |
>   | 550   | 5.8.11  | IDENTITY has been compromised       |
>   | any   | X.8.12  | IDENTITY is an un-delegated IP       |
>   | any   | X.8.13  | Reverse DNS lookup for IDENTITY failed       |
>   | any   | X.8.14  | IDENTITY is temporarily blocked       |
>   | 550   | 5.8.15  | IDENTITY is permanently blocked       |
>   | any   | X.8.16  | IDENTITY is an open relay       |
>   | 550   | 5.8.17  | Malformed content       |
>   | any   | X.8.18  | Recipients have complained about included content   |
>   | any   | X.8.19  | IDENTITY temporarily rate limited due to complaints.|
>   | any   | X.8.20  | IDENTITY temporarily rate limited       |
>   | any   | X.8.21  | IDENTITY-A is a non-authorized sender for IDENTITY-B|
>   | 550   | 5.8.22  | Message contains a virus       |
>   | 550   | 5.8.23  | IDENTITY has been permanently deferred       |
>   | 550   | 5.8.24  | messages from IDENTITY not accepted       |
>   | any   | X.8.25  | messages from local dynamic network not accepted    |
>   | 450   | X.8.26  | INDENTITY Greylisted, please retry later       |
> +-----------------------------------------------------------------------+
> 
> Any code description that says "permanent," according to SMTP should be 55x and the main idea behind permanent is to tell the client not the same transaction again which will repeat the same result. IOW, something has to change on the client side. Temporary means that something may change (i.e. policy, limits) on the receiver side for repeated attempts of the same transaction.
> 
> -- 
> Sincerely
> 
> Hector Santos
> http://www.santronics.com
> 
> 

Date: Wed, 11 May 2011 06:55:44 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Jeff Macdonald <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Hector Santos wrote:

> Also, one other category (Alert Status) perhaps to consider per code to 
> provide insight on what codes are good candidate for internal reporting, 
> feedback to operators, red "alert status", etc, so when the receiver 
> detecting a reason for example x.8.11, it can provide a suggestion about 
> the severity of the detection:
> 
>    Code:        | X.8.11
>    Sample Text: | IDENTITY has been compromised
>    Alert Status:| HIGH, Operator should be notified

BTW, this particular code, should be recommended to be a 5.8.11. I 
don't think we want to encourage a 4.8.11 for a detected compromised 
account.   How about others like this?  Hell, most of them :)

This is my fast, off the top of my head, opinion of the 55x vs 45x 
separation:

+-----------------------------------------------------------------------+
    | Reply | Extended| 
       |
    | Code  | Code    | Sample Response Text 
       |
 
|-------+---------+-----------------------------------------------------|
    | any   | X.8.0   | Undefined Policy detail 
       |
    | 550   | 5.8.1   | Message refused by local policy 
       |
    | any   | X.8.2   | Excessive volume 
       |
    | 550   | 5.8.3   | IP listed on RBL RBL-NAME 
       |
    | any   | X.8.4   | Too many invalid recipients 
       |
    | 550   | 5.8.5   | Unacceptable content 
       |
    | 550   | 5.8.6   | Suspected phishing attempt 
       |
    | any   | X.8.7   | IDENTITY is dynamically blocked due to 
complaints   |
    | 550   | 5.8.8   | IDENTITY is permanently blocked due to 
complaints   |
    | any   | X.8.9   | Recipient not accepting messages from IDENTITY 
      |
    | any   | X.8.10  | IDENTITY is a dynamic IP 
       |
    | 550   | 5.8.11  | IDENTITY has been compromised 
       |
    | any   | X.8.12  | IDENTITY is an un-delegated IP 
       |
    | any   | X.8.13  | Reverse DNS lookup for IDENTITY failed 
       |
    | any   | X.8.14  | IDENTITY is temporarily blocked 
       |
    | 550   | 5.8.15  | IDENTITY is permanently blocked 
       |
    | any   | X.8.16  | IDENTITY is an open relay 
       |
    | 550   | 5.8.17  | Malformed content 
       |
    | any   | X.8.18  | Recipients have complained about included 
content   |
    | any   | X.8.19  | IDENTITY temporarily rate limited due to 
complaints.|
    | any   | X.8.20  | IDENTITY temporarily rate limited 
       |
    | any   | X.8.21  | IDENTITY-A is a non-authorized sender for 
IDENTITY-B|
    | 550   | 5.8.22  | Message contains a virus 
       |
    | 550   | 5.8.23  | IDENTITY has been permanently deferred 
       |
    | 550   | 5.8.24  | messages from IDENTITY not accepted 
       |
    | any   | X.8.25  | messages from local dynamic network not 
accepted    |
    | 450   | X.8.26  | INDENTITY Greylisted, please retry later 
       |
 
+-----------------------------------------------------------------------+

Any code description that says "permanent," according to SMTP should 
be 55x and the main idea behind permanent is to tell the client not 
the same transaction again which will repeat the same result. IOW, 
something has to change on the client side. Temporary means that 
something may change (i.e. policy, limits) on the receiver side for 
repeated attempts of the same transaction.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Wed, 11 May 2011 06:15:20 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Jeff Macdonald <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Jeff Macdonald wrote:
> On Tue, May 10, 2011 at 2:55 PM, Hector Santos <[email-masked]> wrote:
>> So in general, for my own system, I don't particular like to give bad guys
>> "clues" why their transaction failed.  For false positives, it can help the
>> support process in a DSN, but for bad guys, its not like they will listen to
>> them anyway or even get them or maybe redirected to someone else. They are
>> going to blast away anyway.
>>
>> So if you can extract how these "bad guy" transaction rejection responses
>> can help good guys solve a particular problem, more readers will consider
>> it.
> 
> 
> I've heard that argument before, in which these help the bad guys. I
> honestly think the bad guys already know these details without having
> these codes. So yes, it would confirm what they already know. But in
> no way do I think this will empower the bad guys.

Well, we don't know either way but we have to presume its available 
for someone, somehow or not. You have low margin, high volume spammers 
using simple, low to no cost, low compliancy clients who will blast 
things regardless of reply codes and you have those who use proper 
clients who will naturally follow SMTP and also leverage feedback for 
their success reporting.  If you read many of the current DMA (Direct 
Marketing Association) and related trade groups blogs, eMarketing 
recommendations, you will read they have finally learned its better to 
prune out the email addresses that don't make it.

The difference with the new X.8.26 is that its not really in the same 
category with many of the others as it effects both bad and good.  You 
have many people in the WG like this one who were totally against GL 
and today you find them using it.  I was one of them that saw very 
quickly of its value.

> I do have another update pending incorporating the text that was
> suggested before. But it does seem that:
> 
> " assigning these codes helps troubleshoot problems and lower 
>   support costs by allowing sending administrators to resolve 
>   many problems themselves."
> 
> isn't verbose enough to get the point across to receivers.

Example of the "problems" and how it may lower support cost would be good.

There are some codes there that can help with configuration problems, 
like X.8.25, and others that might be related to compliancy, like 
x.8.4, .5, .16, .17.

Also, maybe related, yesterday we got a customer (fed/state/city 
politics newsletters agency) reporting a problem sending to AOL 
addresses with this DSN:

<[email-masked]>: host xxxx.xx.zzz.aol.com[#.#.#.#] said:
   552 5.3.4 E4.3 REQUESTED MAIL ACTION ABORTED: HEADERS EXCEED 32K 
BYTES OR
     RECORD TOO LONG (in reply to end of DATA command)

It turned out, so it seems, the To: line was huge with hundreds of 
continues lines filled with addresses.   That could also fall under x.8.5.

Also, one other category (Alert Status) perhaps to consider per code 
to provide insight on what codes are good candidate for internal 
reporting, feedback to operators, red "alert status", etc, so when the 
receiver detecting a reason for example x.8.11, it can provide a 
suggestion about the severity of the detection:

    Code:        | X.8.11
    Sample Text: | IDENTITY has been compromised
    Alert Status:| HIGH, Operator should be notified

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Tue, 10 May 2011 16:50:52 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Tue, May 10, 2011 at 2:55 PM, Hector Santos <[email-masked]> wrote:
> So in general, for my own system, I don't particular like to give bad guys
> "clues" why their transaction failed.  For false positives, it can help the
> support process in a DSN, but for bad guys, its not like they will listen to
> them anyway or even get them or maybe redirected to someone else. They are
> going to blast away anyway.
>
> So if you can extract how these "bad guy" transaction rejection responses
> can help good guys solve a particular problem, more readers will consider
> it.

I've heard that argument before, in which these help the bad guys. I
honestly think the bad guys already know these details without having
these codes. So yes, it would confirm what they already know. But in
no way do I think this will empower the bad guys.

I do have another update pending incorporating the text that was
suggested before. But it does seem that:

" assigning these codes helps troubleshoot problems and lower support costs by
   allowing sending administrators to resolve many problems themselves."

isn't verbose enough to get the point across to receivers.

-- 
Jeff Macdonald
Ayer, MA

Date: Tue, 10 May 2011 14:55:37 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Jeff Macdonald <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Jeff Macdonald wrote:

> If you've looked at the draft, and I mean those besides Murray, what
> has kept you from implementing it?

My opinion.

In general, like all things, the good ideas are the ones that solves a 
real problem.

The SMTP state machine is already well defined and it required reply 
codes to drive the client/server automation. Everything else is just 
"candy" like extended reply codes are optional, not required to drive 
the C/S protocol and  no one can depend on it solely. IOW, even modern 
clients must also support legacy server with pure SMTP responses.

So you have to provide a good reason why it should be considered.  Its 
a matter of technical writing style but I generally like to start with 
problem statement and then show why/how the proposed idea can help 
address the problem and maybe also add value (i.e. 2 for 1 benefit).

That is why I suggested the text about how it (specific extended codes 
like X.8.26) can help resolve some of the negativity behind 
GreyListing with advanced Retry methods, but as I mention above, for 
X.8.26, anyone with such an advanced retry method, it can't depend 
solely on this specific code. You got the original Harris 
recommendations of 451 and/or 4.7.1.

Finally, this is a subject of opinions.  IMV, the I-D extended 
responses are focused for bad guys rejects, not good guys rejects 
except for the new X.8.26.

So in general, for my own system, I don't particular like to give bad 
guys "clues" why their transaction failed.  For false positives, it 
can help the support process in a DSN, but for bad guys, its not like 
they will listen to them anyway or even get them or maybe redirected 
to someone else. They are going to blast away anyway.

So if you can extract how these "bad guy" transaction rejection 
responses can help good guys solve a particular problem, more readers 
will consider it.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Mon, 09 May 2011 18:02:39 +0300
From: "A. Rothman" <[email-masked]>
To: IETF SMTP <[email-masked]>
Subject: RFC 5321 VRFY and quoting syntax
Sender: [email-masked]

Hi,

I was hoping someone can shed some light on the proper syntax an SMTP 
server should use according to RFC 5321 in the following cases:

1. The syntax definition for commands such as VRFY and EXPN (e.g. 
section 4.1.1.6) is

vrfy = "VRFY" SP String CRLF

with the syntax of "String" defined in section 4.1.2 as:

String         = Atom / Quoted-string

however, section 3.5.1 states:

    The character string arguments of the VRFY and EXPN commands cannot
    be further restricted due to the variety of implementations of the
    user name and mailbox list concepts.

What is not entirely clear to me is which is true - is any string of 
characters valid (syntax-wise), or must it be a Quoted-string (in double 
quotes) if any non-atext characters appear in it. This is especially 
confusing having seen various examples online (and some discussions on 
this mailing list as well) where pointed brackets are included in the 
VRFY argument (using a Path syntax similar to the one defined for the 
MAIL FROM and RCPT TO commands), but with no quoting.

2. Section 4.1.2 defines the backslash-escaped character mechanism in 
quoted-pairSMTP, which is used only in a Quoted-string (within 
double-quotes), and does not mention such escaping outside of a 
Quoted-string. The following text section states:

    Note that the backslash, "\", is a quote character, which is used to
    indicate that the next character is to be used literally (instead of
    its normal interpretation).  For example, "Joe\,Smith" indicates a
    single nine-character user name string with the comma being the
    fourth character of that string.

So, it is unclear whether this paragraph applies only to the 
Quoted-strings defined above, or to any characters in any argument to 
any command, or only to mailboxes (discussed in the preceding paragraph) 
or some other definition of when it does and does not apply.

3. Still regarding VRFY (and maybe also EXPN?) section 3.5.1 states:

    If a normal (i.e., 250) response is returned,
    the response MAY include the full name of the user and MUST include
    the mailbox of the user.  It MUST be in either of the following
    forms:

       User Name<local-part@domain>
       local-part@domain

Whereas section 3.5.2 claims:

    When normal (2yz or 551) responses are returned from a VRFY or EXPN
    request, the reply MUST include the<Mailbox>  name using a
    "<local-part@domain>" construction

Notably, the second example in the former section does not comply with 
the latter section (as it contains no pointed brackets). Which is the 
correct form?

Any clarification would be greatly appreciated - thanks!

Amichai

Date: Mon, 9 May 2011 09:51:33 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Sun, May 8, 2011 at 9:47 PM, Murray S. Kucherawy <[email-masked]> wrote:
>
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Jeff Macdonald
>> Sent: Sunday, May 08, 2011 7:00 AM
>> To: [email-masked]
>> Subject: Re: slight update to draft-macdonald-antispam-registry
>>
>> > Some discussion from them about how they implemented it, what hurdles
>> > there were, what results they've seen -- you know, implementation
>> > report type stuff -- would be really interesting.
>>
>> I agree. Why not add some experiences of your own?
>
> I haven't implemented your draft, so I've none to share.  I'm just offering advice to head off the questions you're inevitably going to get down the line in the standards process.

So some drafts are lucky, in that all they are describing is what is
widely practiced. Makes it easy to fly through the standards process.

Other drafts are new ideas. Of course they'll take a while to get
through the process.

This draft is a mixture of stuff that is being practiced and a new
idea to create a new subject code for it.

So, an implementation report is probably a bit premature.

If you've looked at the draft, and I mean those besides Murray, what
has kept you from implementing it?

-- 
Jeff Macdonald
Ayer, MA

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sun, 8 May 2011 18:47:05 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Jeff Macdonald
> Sent: Sunday, May 08, 2011 7:00 AM
> To: [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> > Some discussion from them about how they implemented it, what hurdles
> > there were, what results they've seen -- you know, implementation
> > report type stuff -- would be really interesting.
> 
> I agree. Why not add some experiences of your own?

I haven't implemented your draft, so I've none to share.  I'm just offering advice to head off the questions you're inevitably going to get down the line in the standards process.

Date: Sun, 08 May 2011 11:16:26 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Jeff Macdonald <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Jeff Macdonald wrote:
> On Sun, May 8, 2011 at 3:23 AM, Murray S. Kucherawy <[email-masked]> 

>> Some discussion from them about how they implemented it, what 
>> hurdles there were, what results they've seen -- you know, 
>> implementation report type stuff -- would be really interesting.

> I agree. Why not add some experiences of your own?

I can tell you that our MTA has look for the following the last 7-8 
years of GL deployment:

    451
    4.7.1
    "grey" in response

to trigger a faster retry on the 2nd and 3rd attempts. It has 
drastically lower the bad connotations of using GL delivery issues in 
commercial environments.

In future updates, we will add 4.8.26 for checking but I am not sure 
if we will have it as a default for receivers to spit out yet.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Sun, 8 May 2011 10:00:03 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Sun, May 8, 2011 at 3:23 AM, Murray S. Kucherawy <[email-masked]> wrote:
>
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Jeff Macdonald
>> Sent: Friday, May 06, 2011 6:21 PM
>> To: [email-masked]
>> Subject: Re: slight update to draft-macdonald-antispam-registry
>>
>> > Are there any receiver implementations out there that do something
>> > with these codes?
>>
>> I've heard of one German ISP that said they plan too. I know one MTA
>> vendor that does. And an ESP.
>
> Some discussion from them about how they implemented it, what hurdles there were, what results they've seen -- you know, implementation report type stuff -- would be really interesting.

I agree. Why not add some experiences of your own?

-- 
Jeff Macdonald
Ayer, MA

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Date: Sun, 8 May 2011 00:23:23 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Jeff Macdonald
> Sent: Friday, May 06, 2011 6:21 PM
> To: [email-masked]
> Subject: Re: slight update to draft-macdonald-antispam-registry
> 
> > Are there any receiver implementations out there that do something
> > with these codes?
> 
> I've heard of one German ISP that said they plan too. I know one MTA
> vendor that does. And an ESP.

Some discussion from them about how they implemented it, what hurdles there were, what results they've seen -- you know, implementation report type stuff -- would be really interesting.

-MSK

Date: Fri, 6 May 2011 21:21:04 -0400
Subject: Re: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sender: [email-masked]

On Fri, May 6, 2011 at 7:25 PM, Murray S. Kucherawy <[email-masked]> wrote:
<snip>
> You might want to bring this up on the apps-discuss mailing list.

ok

> Are there any receiver implementations out there that do something with these codes?

I've heard of one German ISP that said they plan too. I know one MTA
vendor that does. And an ESP.

-- 
Jeff Macdonald
Ayer, MA

From: "Murray S. Kucherawy" <[email-masked]>
To: Jeff Macdonald <[email-masked]>, "[email-masked]" <[email-masked]>
Date: Fri, 6 May 2011 16:25:55 -0700
Subject: RE: slight update to draft-macdonald-antispam-registry
Accept-Language: en-US
acceptlanguage: en-US
Sender: [email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Jeff Macdonald
> Sent: Thursday, May 05, 2011 11:26 AM
> To: [email-masked]
> Subject: slight update to draft-macdonald-antispam-registry
> 
> Hi Folks,
> 
> There is a slight update, where I added a code for greylisting. I've
> also updated the contact information to reflect my current job.
> 
> Draft can be found here:
> 
> https://datatracker.ietf.org/doc/draft-macdonald-antispam-registry/

I spotted a couple of typos that I'll point out to you separately.

You might want to bring this up on the apps-discuss mailing list.

Are there any receiver implementations out there that do something with these codes?

Date: Fri, 06 May 2011 18:18:20 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: SM <[email-masked]>
CC: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

SM wrote:

> I sent a comment to one implementer a few years about suggested 
> switching to 450 as the code can be used to signify that it is a 
> temporarily blocked for policy reasons.  I know another implementation 
> that uses "450".  Refer to Section 4.2.2 of RFC 5321 for further 
> information.

I think you're right to use 450 based on the enhanced 5321 text.

But from 2003 and before RFC5321, Harris only had RFC2821 which didn't 
include the new "temporarily blocked" text.

The main thing is to help reduce Greylisting hurting rapid delivery 
using smarter retries than the recommended 30 mins and if there are 
Harris based systems out there, they might be looking for 451 as a 
trigger.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Fri, 06 May 2011 11:09:28 -0700
To: [email-masked]
From: SM <[email-masked]>
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

At 17:06 05-05-2011, Hector Santos wrote:
>So you can see there are many using the Harris document.  But you 
>see both 450 and 451 with the 4.7.1 which is probably reflective of 
>people, including myself, wondering if 450 was better based on what 
>RFC2821 had for example 45z text:
>
>   450  Requested mail action not taken: mailbox unavailable
>   451  Requested action aborted: error in processing
>   451  Requested action aborted: local error in processing
>   452  Requested action not taken: insufficient system storage

I sent a comment to one implementer a few years about suggested 
switching to 450 as the code can be used to signify that it is a 
temporarily blocked for policy reasons.  I know another 
implementation that uses "450".  Refer to Section 4.2.2 of RFC 5321 
for further information.

Regards,
-sm 

Date: Thu, 05 May 2011 20:06:30 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: Jeff Macdonald <[email-masked]>
CC: [email-masked]
Subject: Re: slight update to draft-macdonald-antispam-registry
Sender: [email-masked]

Jeff Macdonald wrote:

> Hi Folks,
> 
> There is a slight update, where I added a code for greylisting. I've
> also updated the contact information to reflect my current job.
> 
> Draft can be found here:
> 
> https://datatracker.ietf.org/doc/draft-macdonald-antispam-registry/
> 
> ...
> 
> If you've implemented this draft, please let me know.
> 

Jeff,

We started a major upgrade of our MTA and your previous I-D revs were 
already added part of the updated extended reply code text tables. 
I'll updated to rev2.

First some quick review typos found:

    X.8.4, typo "to"  change to "too"
    X.8.26, typo "INDENTIY"  change to "IDENTITY"

A few suggestions.

1 - Motivations for supporting this proposed standard.

With this rev2 update with greylisting info, you might want to add 
some motivations text to the introduction, for example after the 1st para:

    Extended reply codes, such as X.8.26, can help improve
    email communications under greylisted environments by
    using smarter reschedule retransmissions.

For example, our retry table has a retry rule specifically for 
Greylisting pushing up the few few retries, for example:

    if reply.attempts < 3 then
      if reply.code is 451 or
         reply.text has "4.7.1" or
         reply.text has "grey"
      then
         Reply.RetryMintutes(5)

2 - What is IDENTITY?

When we added the X.8.YYY entries, the immediate "scratch head" which 
was what macros (for the SMTP session process entities) to use for the 
IDENTITY in certain of the entries. It was not hard to figure out, but 
it might help to add a terminology/definition section to describe what 
IDENTITY is or might be.

   2. Terminology and Definitions

   2.1 IDENTITY

   The IDENTITY can be any value reflected within the extended
   reply code reason and response.  It may be directly related
   to the value being monitored or evaluated for anti-spam
   purposes [in each stage of the SMTP state machine].  Generally,
   the value is the SMTP client IP address or domain input from
   the client, such as the return path provided in MAIL FROM: state.

   Each extended reply code in section 4.1 provide further explicit
   insights to the identity values are being considered.

Or whatever you think is better as a general definition:)

3 - Suggestion to add Summary table

I think a summary section with easy to read table (i.e. in "one" page 
view) would greatly help readers simply pursuing the doc.  It will 
"sell" the proposal and increase consideration to implement. I would 
add it before the IANA Consideration section but also consider 
re-outlining the TOC and it as an introduction summary

    1.  Introduction
      1.1  Introducing the Anti-Spam Subject
      1.2  Summary of Enumerated Status Codes
    2.  Terminology and Definitions
    3.  Methodology of determining new detail codes
    4.  IANA Considerations
      4.1.  Enumerated Status Codes

1.2 Summary of Enumerated Status Codes

    The following table summarizes when extended codes are applicable for
    reply codes with example response text associated with the 
extended code.
    Please review the terminology section 2.1 for IDENTITY and the 
section 4.1
    IANA considerations to get detail information for each extended 
reply code.

              Table 1.0 - Anti-spam Extended Codes.
 
+-----------------------------------------------------------------------+
    | Reply | Extended| 
       |
    | Code  | Code    | Sample Response Text 
       |
 
|-------+---------+-----------------------------------------------------|
    | any   | X.8.0   | Undefined Policy detail 
       |
    | any   | X.8.1   | Message refused by local policy 
       |
    | any   | X.8.2   | Excessive volume 
       |
    | any   | X.8.3   | IP listed on RBL RBL-NAME 
       |
    | any   | X.8.4   | Too many invalid recipients 
       |
    | any   | X.8.5   | Unacceptable content 
       |
    | any   | X.8.6   | Suspected phishing attempt 
       |
    | any   | X.8.7   | IDENTITY is dynamically blocked due to 
complaints   |
    | any   | X.8.8   | IDENTITY is permanently blocked due to 
complaints   |
    | any   | X.8.9   | Recipient not accepting messages from IDENTIDY 
      |
    | any   | X.8.10  | IDENTITY is a dynamic IP 
       |
    | any   | X.8.11  | IDENTITY has been compromised 
       |
    | any   | X.8.12  | IDENTITY is an un-delegated IP 
       |
    | any   | X.8.13  | Reverse DNS lookup for IDENTITY failed 
       |
    | any   | X.8.14  | IDENTITY is temporarily blocked 
       |
    | any   | X.8.15  | IDENTITY is permanently blocked 
       |
    | any   | X.8.16  | IDENTITY is an open relay 
       |
    | any   | X.8.17  | Malformed content 
       |
    | any   | X.8.18  | Recipients have complained about included 
content   |
    | any   | X.8.19  | IDENTITY temporarily rate limited due to 
complaints.|
    | any   | X.8.20  | IDENTITY temporarily rate limited 
       |
    | any   | X.8.21  | IDENTITY-A is a non-authorized sender for 
IDENTITY-B|
    | any   | X.8.22  | Message contains a virus 
       |
    | any   | X.8.23  | IDENTITY has been permanently deferred 
       |
    | any   | X.8.24  | messages from IDENTITY not accepted 
       |
    | any   | X.8.25  | messages from local dynamic network not 
accepted    |
    | 45z   | X.8.26  | IDENTITY Greylisted, please retry later 
       |
 
+-----------------------------------------------------------------------+

Why use 45z for X.8.26?

For x.8.26, I was going to suggest changing that to 451, but its 
probably better to use 45z with some informative notes for using 
either 450 or 451.  But having an explicit 450 somewhat tells SMTP 
receivers to use 450 - period, and that may now what they wish to use.

The reason why is that there are already receivers using 451. Most, if 
not all, early greylisting adopters began using the "Official" 
GreyListing specs by the purported inventor:

    http://projects.puremagic.com/greylisting/whitepaper.html

This doc was often referenced in the support mailing list and its 
example for a response is:

    451 4.7.1 Please try again later

I did a quick search of today's SMTP outbound trace log for the word 
"grey" and found this unique grouping of responses:

   421 This server implements greylisting, please try again in # seconds
   450 4.2.0 <RCPT>: Recipient address rejected: Greylisted, see URL
   450 4.7.1 <RCPT>: Recipient address rejected: Greylisted for # minutes
   450 4.7.1 <RCPT>: Recipient address rejected: Policy Rejection-
                     Greylisted, please try later (see URL).
   450 4.7.1 <RCPT>: Recipient address rejected: Greylisted for # minutes
   450 4.7.1 <RCPT>: Recipient address rejected: GREYLISTING, please 
retry later
   450 4.7.1 <RCPT>: Recipient address rejected: Greylisted for # seconds
   451 4.7.1 Greylisting in action, please come back in HH:MM:SS
   451 4.7.1 Greylisting in action, please come back later
   451 Greylisted for # seconds
   451 Greylisted, please try again in # seconds
   451 Greylisting enabled, try again in # minutes

So you can see there are many using the Harris document.  But you see 
both 450 and 451 with the 4.7.1 which is probably reflective of 
people, including myself, wondering if 450 was better based on what 
RFC2821 had for example 45z text:

   450  Requested mail action not taken: mailbox unavailable
   451  Requested action aborted: error in processing
   451  Requested action aborted: local error in processing
   452  Requested action not taken: insufficient system storage

Just based on the text above, one can reasonable use 450 and 451.  But 
even though 451 for greylisted session wasn't really due to an error 
in processing, I went with it as a default because extended codes are 
optional.  I speculate that was what Harris was thinking too.

Hope this helps.

-- 
Sincerely

Hector Santos
http://www.santronics.com

Date: Thu, 5 May 2011 14:26:01 -0400
Subject: slight update to draft-macdonald-antispam-registry
From: Jeff Macdonald <[email-masked]>
To: [email-masked]
Sender: [email-masked]

Hi Folks,

There is a slight update, where I added a code for greylisting. I've
also updated the contact information to reflect my current job.

Draft can be found here:

https://datatracker.ietf.org/doc/draft-macdonald-antispam-registry/

Suggested values for SMTP Enhanced Status Codes for Anti-Spam Policy
                  draft-macdonald-antispam-registry-02

Abstract

   This document establishes a set of extended SMTP policy codes for
   anti-spam.  It seeks to provide additional codes for error texts that
   currently use the extended SMTP error code 5.7.1.  The anti-spam
   codes were determined by looking at error texts produced by major
   ISPs and finding commonalities.  The result is a new set of error
   texts with associated extended SMTP error codes.

There is an implementation at my former job:

$ telnet mail-router.e-dialog.com 25
Trying 208.94.20.16...
Connected to mail-router.e-dialog.com.
Escape character is '^]'.
550 5.8.3 Listed on RBL Zen: See http://www.spamhaus.org/query/bl?ip=1.1.1.1

Here I tried to connect to the inbound MX at e-Dialog, from my home
over Comcast cable. This is a residential connection so the listing
makes sense (it is actually a PBL listing).
I've changed the IP to keep spammers at bay.

If you've implemented this draft, please let me know.

-- 
Jeff Macdonald
Ayer, MA