Subject: Re: draft-ietf-sieve-refuse-reject-07.txt

From: Matthew Elvey <[email-masked]>
Date: Wed, 18 Jun 2008 11:22:11 -0700

On 5/29/08 9:22 AM, Ned Freed wrote:
> <Ned quote me but didn't include a quotation line; please do so in 
> future, Ned.>:
>
>
>> I disagree.  There is a loophole for an implementer to decide that 
>> what he or
>> she feels is preferred is to not bother fulfilling this requirement. 
>> It needs
>> to be closed.
>
> Then you really need to provide better evidence that such a loophole 
> exists -
> because I'm not seeing it.
That's funny.  You do see it.  You just don't realize it.  You drove the 
LMTP truck through the loophole!! (see LMTP discussion below)  My latest 
draft doesn't have the loophole that you've insisted LMTP 
implementations should be able to drive through!
>> I also support the addition of the text Ned proposed:
>
>> "the risk that these actions will generate blowback spam are
>> minimized but cannot be eliminated completely even in the case of 
>> ereject, so
>> caution is advised when using these actions to deal with messages 
>> determined to
>> be spam."
>
>> It can go at the end of 2.1.2.
>
> 2.1.2 discusses sending DSNs and is therefore entirely the wrong place 
> for
> this. It belongs at the end of 2.1.1, the section talking about 
> SMTP/LMTP level
> rejects.
Agreed.
>
>
>> and I don't see why
>>     Script generators SHOULD ensure that a rejection action being
>> shouldn't be
>>     Script generators MUST ensure that a rejection action being
>
> This cannot be a MUST, if for no other reason that such generators could
> easily be operating in an MUA content where ereject is not available.
Agreed.
>
>> also, let's change
>>     using the ereject action, as it is more suitable for such 
>> rejections.
>> to
>>     using the ereject action, as reject is unsuitable for such 
>> rejections.
>
> The problem with such claims is that the bar they effectively set is so
> high ereject cannot meet them either. I'm therefore opposed to this as 
> well.
I don't understand what you're saying here. Can you put it another way?
>
>> also, let's change
>>     support protocol level rejection, e.g. an MUA, and MUST be available
>> to
>>     support protocol level rejection, i.e. an MUA, and MUST be available
>
> It is entirely possible for there to be critters in the email universe 
> that act
> at the UA level but which aren't thought of as UAs. So what you are doing
> actually weakens the requirement by making it sound like it only 
> applies to
> MUAs specifically.
Have you thought of an example you can provide?

>> I still strongly support requiring a change to the default behaviour,
>> and feel that there is reluctance to change the current behaviour for
>> what was presented as nothing more than an unwillingness to explain what
>> we all agreed were net good reasons for the change.
>
> I fail to see continuing to impugn the motives for the position I have 
> taken is
> in any way helpful here. But in the interests of keeping this civil 
> I'm not
> going to respond further. 
I've said that the justification is poor.  It's not about you; it's not 
ad hominem, so please don't paint it as such.  The point is that the 
evidence/argument provided against chaning the default behaviour is weak.
>
>> And I'm not clear why this is needed; I don't think it's the case:
>> (However, the LMTP client may then have no choice
>>     but to generate a DSN to report the error, which may result in
>>     blowback.)
>> Where it appears to be the case, the LMTP client may just need to be
>> fixed.
>
> Fixed how? Suppose the LMTP client is on an ingress MTA. Messages come
> in from the Internet using SMTP and are then sent on using LMTP to affect
> final delivery.
I suspected you'd be bringing this case up again.
It needs to be fixed if it accepts the message before it has determined 
whether it should receive it.
(This discussion has already been hashed out; it's a bit frustrating to 
have to go through it again.)

>
> Suppose the Sieve implementation is operating as part of the LMTP 
> server. (This
> isn't how I'd ever do it, but it is nevertheless a perfectly legitimate
> implementation option.) A message comes in via SMTP and is queued for
> delivery. The LMTP client sends it to the LMTP server, which then runs
> the Sieve which then does an ereject. This is then translated into a
> 550 LMTP response.
>
> What is the MTA supposed to do now? 
You're asking me what to do once you've painted yourself into a corner.  
My answer is: DON'T paint yourself into a corner.

Do I sympathize with you for painting yourself in a corner?  Sure.  But 
if you insist on doing it in perpetuity, I don't.

Do the TCP or Ethernet specs say send data as fast as you please?  No; 
why should we say do what you please here just because there's a market 
for such harmful product? If you've written an ingress MTA to always 
accept mail before determining that the final delivery can be made, then 
you have reasons to paint yourself into a corner: laziness, defense of 
market niche, the system/software you've written works better that 
way...  Good enough reasons?  Not IMO.

> It has exactly two options available:
>
> (1) Silently discard the message.
> (2) Generate a DSN and send it to the MAIL FROM address.
>
> (1) is only appropriate if the MAIL FROM is emoty, NOTIFY=NEVER is in 
> effect,
> the MAIL FROM Is somehow known to be forged. That leaves generating a 
> DSN as the only viable alternative in most cases.
>
> Returning a 550 SMTP response is not possible for the simple reason 
> that the
> SMTP connection is long gone - in fact it could have taken place hours 
> or even
> days earlier.
> I will also note that the behavior of such an MTA, which not only 
> isn't the
> agent with the Sieve implementation, it likely will not even be aware 
> that
> Sieve is involved, is entirely out of scope for this working group. We 
> cannot
> impose requirements here even if we wanted to.
>
>> If there is indeed such a case, it needs to be specified.
>
> Specify what? 
Do what you did.
> The unfortunate reality is that once a message has been accepted
> by an ingress MTA the options for getting rid of it narrow down to 
> discarding it, with or without sending a notification.
> Now, I have long been a proponent of turning away as many messages as 
> possible
> while the connection is still there for reporting errors. But neither 
> this
> document nor this workging group are the places to make the case for 
> doing
> things this way.
>
>> Oh, and the acronyms MDA,  MTA and MUA are now used but not defined (as
>> Mail Delivery, Transfer and User Agents, respectively)
>> draft-crocker-email-arch-10.txt defines 'em, but it's been an I-D for as
>> long as this I-D, so I suggest we just spell 'em out on initial use.
>
> I agree the terms need to be defined but the right way to do is is by
> referencing the architecture document. The Sieve environment draft did 
> so and
> this has not proved to be a barrier to publication.
Ok.  Defined or just spelled out; either is fine with me.
>                 Ned

-Matthew

---

From: Ned Freed <[email-masked]>
Date: Wed, 18 Jun 2008 17:21:45 -0700 (PDT)

> implementations should be able to drive through!

I have never insisted on any such thing for LMTP. In fact I have from the start
of this debate back in 2006 pointed out the problems with treating LMTP as as a
case similar to SMTP.

My issue has always been that I'm against making an incompatible change to
_reject_, one that will render existing widely deployed implementations like
ours incompliant. And while we do use LMTP in our product, our Sieve
implementation operates at the SMTP level, not the LMTP level, so for me at
least there is no backwards compatibility issue with LMTP at all.

> I don't understand what you're saying here. Can you put it another way?

No, not really.

> Have you thought of an example you can provide?

List processors are the most obvious example of something seen as operating
with MUA authority that people don't think of as an MUA. use of basic Sieve in
conjunction with list processing seems like a pretty reasonasble fit to me.

> evidence/argument provided against chaning the default behaviour is weak.

Hundreds of sites at least. Thousands of machines. Many tens of millions of
users. All operating in full compliance with the original Sieve specification,
not using reject to deal with spam, and AFAIK not a demonstable source of
blowback spam.

Seems like a pretty good justification to me for not wanting to break
backwards compatibility.

Now, if you're after a specific use-case where this really matters, I can
provide that as well. The case we run into all the time is in places like Japan
where limiting rejection text to US-ASCII is widely seen as completely
unacceptable. These folks insist that when rejecting legitimate mail (not spam
blocking) the response go back in proper Japanese. Heck, they get really tense
if just the spacing gets screwed up for some reason.

(On a side note, I'm not even sure that extending SMTP to allow UTF-8 error
responses will be an acceptable alternative to DSNs or MDNs here. I have spent
literally dozens of hours dealing with Japanese format and presentation issues
in nondelivery notifications of various sorts.)

These folks will NOT be happy if all of a sudden the rejects they currently
have in their scripts all of a sudden start returning SMTP level responses
saying "generically offensive system response in i-default has replaced nice
Japanese response that probably managed to bow three times and be extremely 
apologetic and probably made it the recipients fault for not being able to
accept whatever the sender sent". Of course they won't complain - they don't do
that - but they may look elsewhere for their email software. And I happen to
know that there are several implementations out there using Sieve that have no
problem whatsoever ignoring the parts of the standards they find inconvenient.

> whether it should receive it.

But that's operating at the SMTP level, not the LMTP level. And the only
way to enforce that is to not allow LMTP server-side implementation of
ereject.

But if that's the consensus for what we want to do (and while I personally
don't object I've seen no consensus for such a change) then we need to come out
and say that explicitly, not try and eliminate the case by first including LMTP
as a possible place for ereject to be used but then trying to Venn-diagram it
out of existance with a bunch of complex overlapping requirements LMTP cannot
meet.

The approach you're advocating is a sure-fire recipe for confused implementors 
and broken implementations.

> have to go through it again.)

Well, that discussion apparently did not result in even a mention being added
of the likely consequences of LMTP server-side use of rereject, so apparently
it didn't stick.

> My answer is: DON'T paint yourself into a corner.

And the only way to avoid the corner (in the ereject case at least) is not
to operate at the LMTP server level.

More generally, however, there's no way to avoid this case. We go to a lot of
trouble to try and detect things like over quota conditions so we can reject at
the SMTP level. But this depends on back-channel communication to our MTA from
our store. If our MTA is used with someone else's LMTP server - as it sometimes
is - we have no way to perform a useful check at the SMTP level.

> if you insist on doing it in perpetuity, I don't.

The only thing I insist on in not to gratuitously make existing implementations
incompliant.

> for such harmful product?

This is an invidious comparison and shame on you for making it.  It would be
one thing if our implementation had violated the recommendations made by
existing standards and, say, implemented an MDN-generating reject when the
standard said not to and then encouraged its used for dealing with  spam (again
when the standard said such usage was inappropriate). BUt that's not what we
did. What we did was follow the standards quite carefully, including doing
everything we can to prevent our customers from using reject, with its
MDN-generating characteristics, from being used to deal with spam.

 If you've written an ingress MTA to always
> way...  Good enough reasons?  Not IMO.

And this, Matthew is an ad hominem attack - nothing more and nothing less -
made all the worse because it grossly mischaracterizes what our implementation
does and the reasons why it does it.

And at this point I'm done arguing about this. You, sir, either are an
anti-spam zealot or a very good imitation of one, and I see no point in
continuing to discuss this with you.

				Ned

---

From: Matthew Elvey <[email-masked]>
Date: Thu, 19 Jun 2008 12:43:19 -0700

This is how a conforming system implementation would work:

Step 1) SMTP client connects to receiving system, which consists of an 
ingress MTA sitting on the Internet  in front of (and protecting) an MDA 
(store) with which it communicates over LMTP.  The system (the MDA in 
particular) contains and processes Sieve scripts.

Step 2) SMTP client continues the SMTP conversation to the point that 
the system receives a message to one recipient and the character 
sequence "<CRLF>.<CRLF>".  (See 4.1.1.4 DATA (DATA) of RFC 2821)

Step 3) The system does NOT need to IMMEDIATELY reply,  instead, it may 
perform some processing, as RFC 2821 indicates.  RFC 2821 says 
specifically that the SMTP client SHOULD wait a minimum of at least 10 
minutes for the "250 OK" reply.  This processing should include:

Step 4) The MTA immediately connects to the MDA and attempts to pass on 
the message. If the MDA decides to 'ereject' the message, the MTA will 
pass along the message to the SMTP client by sending a failure message, 
instead of "250 OK".

I believe all or at least nearly all multi-component implementations can 
and should work this way.

Note: if the MTA tries to contact the MDA in step 4 and is unsuccessful, 
it can try again several times; it should have at least 10 minutes 
(that's a lot of milliseconds)  to get back to the SMTP client.   And 
even at that point, it can send a 4xx, and try again when the SMTP 
client tries again.

In other words, a conforming system cannot include an MTA that 
immediately accepts such a message, passes it to an MDA that processes 
it with a sieve script that decides to ereject it, and have that MDA 
blithely send out a DSN.

On 6/18/08 5:21 PM, Ned Freed wrote:
> <did it again.  :( >
> The approach you're advocating is a sure-fire recipe for confused 
> implementors and broken implementations.
>
Nope; see the 4 steps above.
> is - we have no way to perform a useful check at the SMTP level.
If you'd care to explain why an implementation can't conform to the 4 
steps I explain, I'm all ears; I have yet to hear such an explanation.  
(Other than the claim that the system isn't a system; the MTA and MDA 
are independent.)  My response is that if someone insists on looking at 
things that way, that's their right, but then their MDA should not be 
considered conforming.

Again, I've done nothing but argue with you about the difficulty and 
merits of conforming to proposed requirements.  I should expressed that 
differently, and for that I apologize: I could have instead said that 
changing an existing implementation to conform could be difficult and 
time consuming; I have said so before.  I'm sorry that you feel 
attacked, but I have not attacked you.

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 20 Jun 2008 10:46:51 +0200

> sequence "<CRLF>.<CRLF>".  (See 4.1.1.4 DATA (DATA) of RFC 2821)

"one recipient".  check.  do you have a document where best practice for
ensuring this is documented?  (please don't say "Everyone switch to
Qmail" :-)

> and should work this way.

I'm not aware of any open source systems where this is possible, so
please enlighten me.  I was going to extend Exim's callout mechanism to
do a full body callout so that it could be combined with Cyrus' LMTP,
but didn't get that far before switching to non-mail related employment.

-- 
regards,
Kjetil T.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 20 Jun 2008 12:15:35 +0200

Kjetil Torgrim Homme writes:
>> I believe all or at least nearly all multi-component implementations 
>> can and should work this way.
>
> please enlighten me.

I think it can be done with Postfix, but it's far from easy. Needs a bit 
of programming.

IMO the idea of demanding this kind of cooperation is overreach. It's 
not acceptable for a sieve extension to reach out beyond LMTP and make 
demands of the program that connects to the MDA via LMTP.

Please let's have this sieve extension published as an RFC. It's good. 
Exactly how good it is in practice is a QoS matter, not a sieve 
standards matter.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 20 Jun 2008 10:13:16 -0400

Hi folks,

--On June 20, 2008 12:15:35 PM +0200 Arnt Gulbrandsen 
<[email-masked]> wrote:

> Exactly how good it is in practice is a QoS matter, not a sieve standards
> matter.

As co-chair I am going to declare that we have WG consensus on the current 
draft, in spite of Matthew's comments. I don't see anyone else agreeing 
whole heartedly with his position. Therefore we are going to move ahead 
with submitting this draft to the IESG next week for IETF last call and 
IESG review.

-- 
Cyrus Daboo