Subject: RE: [Asrg] Ban the bounce; improved challenge-response systems

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Sun, 6 Apr 2003 05:30:58 -0700

> You also assume that mail is delivered to directly connected systems
> only.  What about, say, domains sitting behind UUCP or ETRN links
> wherein the relay box *HAS* no knowledge of accounts 
> configured on that
> domain as it has no organisational ties to it other than as an MX.

I think twenty years is more than enough time to phase out
an obsolete protocol. If stopping spam breaks UUCP then UUCP
breaks, don't give it a second's thought, that is a problem
for the UUCP diehards to solve, on their own.

I would start from a different premise, I would not 'ban the bounce'
instead I would observe that bounces are going to be much less
likely to get through in the future.

One approach we could take is to add in an authenticator token
into outgoing messages in such a way that it will be returned
in the bounce. That way an MTA can know if a bounce is legit
and not spam.

This is fairly easy to do with a standard secret & MAC 
protocol.

However that is only solving one part of the problem. The
bounce messages should carry the same info as SMTP error
codes and it should be possible for the MTA to process 
bounces in a transparent fashion without showing the bounce
message.

		Phill
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Chuq Von Rospach <[email-masked]>
Date: Sun, 6 Apr 2003 07:34:54 -0700

> for the UUCP diehards to solve, on their own.

Not every place is like that place you're at.

UUCP still has a fair usage in places in Europe, especially sweden. I 
deal with people there who are on uucp links all of the time. It also 
still has usage in less developed countries, and in places with really 
limited phone systems or none at all (scientific research groups using 
uucp over satphone, for instance).

It's rather easy to tell someoe else to change all of their stuff, 
though.

---

From: Jim Youll <[email-masked]>
Date: Sun, 6 Apr 2003 11:04:45 -0400

At 7:34 -0700 4/6/03, Chuq Von Rospach wrote:
>UUCP still has a fair usage in places in Europe, especially sweden. 
>I deal with people there who are on uucp links all of the time. It 
>also still has usage in less developed countries, and in places with 
>really limited phone systems or none at all (scientific research 
>groups using uucp over satphone, for instance).
>
>It's rather easy to tell someoe else to change all of their stuff, though.

just wondering aloud...

so UUCP is still in fair use but clearly a minority.

Is it really a good idea that all mail products should have to handle 
it - and design anti-spam or other things around it?

Might it make more sense from a design standpoint for UUCP endpoints 
to be gatewayed through something that talks current protocols on one 
side, and UUCP on the other? I have to think that most everyone who's 
using UUCP could probably find a host (probably owns one, perhaps at 
the home research institution that they contact via the satphone) 
that would serve this function. Even in the developing world, there 
are linkages to supportive "other" institutions that could help in a 
transitional time.

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 6 Apr 2003 09:16:19 -0600 (MDT)

> From: "Hallam-Baker, Phillip" <[email-masked]>

> ...

ETRN is part of STMP.  The reasons people continue to use ETRN and
UUCP have nothing to do with mere conservatism.  In the modern era,
they are too hard use (including keep running) compared with just
going with the flow.  They are used because in some situations they
are least painful solution.

More common examples of why relays are required and why they cannot
know whether an incoming message can be delivered during the SMTP
transaction are MX secondaries and corporate firewalls.  How can an
MX secondary or corporate firewall know not merely that the Rcpt_To
address is valid but that there will be no problems putting the message
into the mailbox?  "User unknown" is not the only 5yz SMTP error code
and not the only contents of a bounce.

> ...

I don't recall ever seeing a bounce that did not have all of the
information of the original STMP error code.  This fact falls out
of the common method of generating bounces.  Bounces are commonly
merely logs of the SMTP transaction up to and including the string
that came from the SMTP server including its error code.

Other than the retrying for 4yz error codes and the special case or
kludge to deal with the too many recipients bug in old SMTP servers,
the bounces are already handled exactly the same as SMTP error codes.
In both cases, the error code is shown to the originating end-user.  In
most cases, there is nothing that can be done except show the information
to the end-user so that the end-user can decide whether to send to some
other address (e.g. correct a typo), give up, or ask for help.

] From: Daniel Feenberg <[email-masked]>

] ...
] The proposal only suggests that hosts accepting mail from *stangers*
] should know if they are going to deliver that mail before they "OK" it at
] the completion of the mail acceptance process. RBLs do this, most content
] based screening does not, but could if it wanted too. MTAs don't usually
] support this very enthusiastically.
]
] No doubt that some mail architectures will not be able to do this. To
] reject mail for non-existant users the accepting MTA will have to have
] access to the user list, which is sometimes not available to the accepting
] MTA in the DMZ. That would have to change. The alternatives are to stop
] bouncing bad email-addresses or become a vehicle for spammers to abuse,
] neither of which is very attractive.

Before installing a solution, it is best to determine whether the
problem is significant.  No evidence has been offered to show that
bounces are a significant problem in the Internet in general.  The
Flowers.com court case, the laws against header forgery, or perhaps
other causes have significantly reduced the relative frequency with
which bounces cause problems for innocents.  That is not to say that
problems do not occur, but they've not been shown to be significant.

Getting rid of bounces would require all SMTP servers not only to know
that an address is valid, but to predict that there will be no problem
when the message is delivered.  The first will not happen in many
corporate environments.  There are security reasons that people find
compelling for keeping the DMZ SMTP servers ignorant, and "security"
trumps changes to fix things that haven't been a problem yet.  (Note that
I am not defending those security concern but only reporting them.)

More important, it is impossible for an SMTP server to know that
delivery will be successful.  Between end of the STMP transaction when
an MX secondary or DMZ SMTP server says "250 ok" and the seconds to
hours later when the server gets around to passing the message, the
user's mailbox could have overflowed, the user name could have been
terminated, or a computer could have crashed.

] We have gone to quite a bit of trouble to start doing Spamassassin
] screening in a Sendmail Milter, rather than in the local delivery agent,
] just for this reason. By refusing mail rather than dropping it on the
] floor we will give senders of false positives a rejection rather than
] risk losing the message entirely. 

There are additional, individually compelling reasons to filter in
the MTA instead of the MUA.  One compelling reason is that as anyone
who has read the postmaster mailbox for a modest to large domain knows,
bounces have long been unreliable.  Many bounces "double-bounce" into
postmaster mailboxes or just disappear into black holes.  As in most
situations, it is best to detect and report as early and directly as
possible.

Vernon Schryver    [email-masked]
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 6 Apr 2003 09:55:05 -0600 (MDT)

> From: Jim Youll <[email-masked]>

> side, and UUCP on the other?  ...

What happens when delivery fails at either end?  What if you send a
message from either the Internet side or the other side, and something
breaks?  Would you want your message to just disappear into a black hole
or do you want some error indication?  What if the problem is persistent
but intermittent?  Would you be happy with a mail system where some
of your messages disappear with traces?

Any error indication in such a case must traverse the entire path.
It must happen long after any of the the original SMTP or UUCP
transactions.  That's a description of a bounce.

UUCP is not significant, except that it provides clear and easily
understandable test cases for ill considered notions such as deprecating
STMP bounces.  For example, in this case, you could think of the
connection between the external and internal STMP servers of many
corporations as using UUCP and not be far wrong.  Whether internal
and external corporate SMTP servers or MX secondaries and primaries
are connected by UUCP or SMTP, the effects in this context are the same.

---

From: Chuq Von Rospach <[email-masked]>
Date: Sun, 6 Apr 2003 11:02:35 -0700

> it - and design anti-spam or other things around it?

and that's the right question: "throw it out" is unrealistic. "how do 
we make our systems work with it, or gateway to it" is a great 
question. answer? I don't know. but it needs to be figured out to make 
it work.

I'm not sure I've seen a suggestion here yet that considers 
transitional periods or gatewaying to non-conforming systems in their 
plans.

---

From: Jim Youll <[email-masked]>
Date: Sun, 6 Apr 2003 14:04:47 -0400

>of your messages disappear with traces?

I suppose that would be handled in a gateway. It could presumably
push bounces across itself, as well as regular messages, or even generate
them as appropriate...
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 6 Apr 2003 12:38:19 -0600 (MDT)

> >breaks? ...

> push bounces across itself, as well as regular messages, or even generate

How would that differ from the current situation?  All working UUCP-STMP
gateways that I've seen generate bounces when their own UUCP or SMTP
transactions fail.  Gateways treat their own bounces as well as bounces
from afar like other mail.  I can't think of an alternative that would
be as reliable, not to mention practical.

Again, bounces need to be generated for more than just "user unknown."
Unless you can cram the entire Internet into an old style BBS, there
will be relays.  As long as there are relays, you will have either
bounces or blackholes.  Without bounces, all remote failures are like
blackholes.  Who except spammers does not want to receive bounces for
their own mail?

---

From: Jim Youll <[email-masked]>
Date: Sun, 6 Apr 2003 14:51:54 -0400

>blackholes.  Who except spammers does not want to receive bounces for

Maybe I'm missing something.
I will try to answer based on signal- rather than content-analysis.

If there's a relatively very small number of hosts that must do some
protocol (UUCP) and if support for that protocol is holding up other things,
then it would seem to make sense to provide those hosts with a gateway option
that makes their world look just like it did before.

Presumably the number of such hosts is finite and also declining, such
that outbound problems from their gateways could be more easily contained
that attempts to "fix" an unlimited number of old-protocol hosts that never
even talk the protocol...

UUCP sites' inbound situation may be just the same as it ever had been, though
their gateways could presumably provide some new-protocol benefits in both
directions. As with any gateway scenario, information from the back-end may
need to move toward the gateway to make things work well.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Sun, 6 Apr 2003 13:25:07 -0600 (MDT)

> that makes their world look just like it did before.

For the general case of replacing one messaging protocol with another,
gateways are not just a good idea, but necessary.  I am not talking
about the necessity of gateways.  SMTP-UUCP gateways can be considered
classic examples of temporary gateways that always seem to last forever.

Instead I'm talking about bounces.  Assume you are on the new protocol
side of a gateway between the new, bounce-free, neo-SMTP Internet and
you want to send a message to a user on the old network.  The gateway
knows all valid UUCP (or whatever) addresses and so generates appropriate
neo-SMTP errors when presented with a bogus address.  (Never mind that
this is obviously grossly impractical.)  Thus, the gateway does not
generate bounces for most "spammer joe jobs."

Now consider these cases:

 1. between the time the gateway accepts your message and it is
   delivered to the distant mailbox, a host name is changed.  The
   change makes your original address invalid.

 2. the UUCP path between the gateway and the distant mailbox is ill
   and does not come up for more than 4 days.

 3. your message reaches the distant mailbox, but it turns out that
   the mailbox is too full and your message is rejected.

 4. your message reaches the distant mailbox, but is rejected by a
   spam filter.

Would you want some indication of any of those failures?  

If so, how would that failur indication differ from a classic SMTP
bounce as far as your MUA is concerned?

---

From: James Wilde <[email-masked]>
Date: 06 Apr 2003 21:49:08 +0200

Perhaps transitional periods, whether to legacy systems or UUCP and
other 'obsolete protocols' can be left till a little later in the
process we have embarked on.  Let's dream, for the present, about what a
world without spam would be like, and how it would be achieved when
every responsible person with a mail server has upgraded to Sendmail 9,
Postfix 5 or whatever versions it will be that include our suggestions.

Speaking personally I like the idea of dynamic whitelists coupled with
shared secrets and hmac but I'd have a hard job persuading my company to
install it.  How would such a system work for one of the many people who
wish to buy our products and send an email in response to a campaign? 
The campaign comes by snailmail on paper so there is no shared secret. 
Are we talking limited-time mailboxes here?

mvh/regards

James

---

From: J C Lawrence <[email-masked]>
Date: Sun, 06 Apr 2003 12:54:28 -0700

On Sun, 6 Apr 2003 05:30:58 -0700  
Phillip Hallam-Baker <Hallam-Baker> wrote:

>> wherein the relay box *HAS* no knowledge of accounts configured on
>> that domain as it has no organisational ties to it other than as an
>> MX.

> I think twenty years is more than enough time to phase out an obsolete
> protocol. If stopping spam breaks UUCP then UUCP breaks, don't give it
> a second's thought, that is a problem for the UUCP diehards to solve,
> on their own.

Large chunks of the 3rd world, as well as both hostile and remote
environments hang off UUCP links (ObNote: I did for years).  The fact
that this works and very cost effectively is something to be admired;
very much an example of a "suitable technology".  Should various
people's dreams about the 'net in space (colonies, stations, etc) be
realised I suspect something very like UUCP (and FTP by mail etc) will
resurface.  Assuming that only the directly small-latency connected are
or can be "interesting" seems painfully chauvinistic and
segregationalist.

> instead I would observe that bounces are going to be much less likely
> to get through in the future.

Ouch.  That would transfer SMTP from an occasionally and rarely
unreliable protocol to one that is both explicitly unreliable and
unauditable at the transport level.

I think one thing to realise here is that email is not bound to SMTP.
SMTP is merely a commonly used transport for mail that has certain
protocol holes and weaknesses that need to be fixed -- problems that are
shared and defined by any asynchronous store-and-forward message-based
protocol.  There are other transports that are widely used, and they
will be (increasingly) used as the net grows and extends into different
environments.  From our perspective I'd concentrate on two aspects:  

  1) Changing SMTP as a transport (not as a mail definition), needed to
  control the problem (without excluding other transports).

  2) Making any needed changes at the RFC 2822 level to support
  the necessary transport protocol features.

> One approach we could take is to add in an authenticator token into
> outgoing messages in such a way that it will be returned in the
> bounce. That way an MTA can know if a bounce is legit and not spam.

> This is fairly easy to do with a standard secret & MAC protocol.

> However that is only solving one part of the problem. The bounce
> messages should carry the same info as SMTP error codes and it should
> be possible for the MTA to process bounces in a transparent fashion
> without showing the bounce message.

Absolutely.  In the same line DSNs should, by now, be widely deployed
and a fundamental aspect of using email in the real world.  Sadly, error
handling is always the red headed step child and gets the transient
scraps of attention left over from adding ROI-generating features.

-- 
J C Lawrence                
---------(*)                Satan, oscillate my metallic sonatas. 
[email-masked]               He lived as a devil, eh?		  
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: J C Lawrence <[email-masked]>
Date: Sun, 06 Apr 2003 13:10:14 -0700

On Sun, 6 Apr 2003 11:04:45 -0400 

> so UUCP is still in fair use but clearly a minority.

A minority of traffic and addresses to be sure, however the total number
of humans behind such links can probably be assumed to be in the tens if
not hundreds of millions.  When numbers get large its easy to play the
percentage game while forgetting that very small percentages are still
large numbers.

> it - and design anti-spam or other things around it?

We are, in general, geeks, and so can safely deal with abstractions
which can't and shouldn't be exposed to JoeSixpack end-users.  

Email consists of two layers: payloads and transports.  

Payloads are currently defined by RFC 2822 and the various MIME RFCs
(plus other more minor bits like 2369 etc).  Transports carry payloads
about and are (primarily) at this point represented by SMTP and UUCP
(there are others) and are covered by RFCs 281, 976 and others that slip
my cranium at this point.

Its an easy and often tempting logical escape to try and conflate the
two abstractions into one, to consider the payload and the transport as
siamese twins that are ineluctably codependent.  However, that's not
current reality, and doesn't match (in general) with any of the standard
methods of protocol analysis and design.  Payload and transport are
fundamentally different -- this is one of the HUGE simplifications that
TCP/IP added to the networking arena.

Do you design your refrigerator for how many wheels are on the trucks
that will deliver it to your house, or do you design a refrigerator
which is a good fridge and which can be easily (generically) transported
to their consumers, and then design trucks which are good trucks and
which can easily transport generic payloads, like those fridges and
others to their consumers?

> the home research institution that they contact via the satphone) that
> would serve this function. Even in the developing world, there are
> linkages to supportive "other" institutions that could help in a
> transitional time.

That is precisely what happens now.  Necessarily from the DNS, MX, and
SMTP consistency point there needs to be a delivery target that acts as
the interface for mail routing.  Sometimes its a volunteer operation,
sometimes a professional service (for a while all of Congo hung off a
single 486-33 that was run out of a teen-agers back bedroom in Atlanta).

---

From: [email-masked]
Date: Sun, 6 Apr 2003 18:19:26 -0400

On Sun, Apr 06, 2003 at 09:16:19AM -0600, Vernon Schryver wrote

> transaction are MX secondaries and corporate firewalls.

  Is the corporate firewall a NAT machine or is it a dumb MTA that
relays to the "smart MTA" ?  If it's a dumb MTA, then why is it being
used ?

> How can an MX secondary or corporate firewall know not merely that
> the Rcpt_To address is valid but that there will be no problems
> putting the message into the mailbox?  "User unknown" is not the
> only 5yz SMTP error code and not the only contents of a bounce.

  Then fer-cryin-out-loud, hold off on the "OK" response for a few
seconds and wait till the email is accepted.  If it isn't, issue the
appropriate 5xx code after DATA:.

-- 
Walter Dnes <[email-masked]>
An infinite number of monkeys pounding away on keyboards will
eventually produce a report showing that Windows is more secure,
and has a lower TCO, than linux.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: J C Lawrence <[email-masked]>
Date: Sun, 06 Apr 2003 18:20:18 -0700

On Sun, 6 Apr 2003 18:19:26 -0400 

> appropriate 5xx code after DATA:.

And when the message turns out to be addressed to an alias or forward
which points outside that MX'es domain?  Or the system that handles mail
spooling (final delivery) is not the same as the recipient MX (almost
always the case for corporates), and reponse time from the spool machine
may exceed an hour (seen this quite commonly during mail bursts)?

Consider AOL's case for starters.  From what I can tell they have a
three-level setup, with a first line of simple receivers, a middle area
of message routers and analysers (eg where the silent discards happen),
and then a set of back end storage servers handling the LDA business.

---

From: Kee Hinckley <[email-masked]>
Date: Sun, 6 Apr 2003 21:53:34 -0400

>relays to the "smart MTA" ?  If it's a dumb MTA, then why is it being

- Scalability
- Distributed administration (you don't really think one department 
handles all email at a multi-national corporation--even if there's a 
single unified domain)?
- Gateways to mail systems using other protocols
- Spam filtering!
...
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: [email-masked] (Claus Färber)
Date: 07 Apr 2003 10:54:00 +0200

> which points outside that MX'es domain?

There's no reason why it could not work in this situation: You hold off  
the "OK" until you know that the message has been delivered, delivery  
has failed /or/ a timeout has been elasped.

> Or the system that handles mail spooling (final delivery) is not the
> same as the recipient MX (almost always the case for corporates), and
> reponse time from the spool machine may exceed an hour (seen this
> quite commonly during mail bursts)?

In this case you could configure your system to temporarily reject mail  
from outside until the spool machine responds timely again.

> and then a set of back end storage servers handling the LDA business.

Even this system should not receive mail faster than the analysers can  
handle it and storage servers can tuck it away. Each level can start to  
temporarily reject messages (or connection) until the next level can  
handle the mail.

Claus
-- 
http://www.faerber.muc.de/
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Chuq Von Rospach <[email-masked]>
Date: Mon, 7 Apr 2003 08:29:54 -0700

> temporarily reject messages (or connection) until the next level can

Doesn't work under many situations. I have a machine specifically set 
up as a incoming "sponge", to accept mail as fast as possible and hold 
it until it can be sent to a series of processing machines or scripts 
or mailboxes or other things.

To do what you want, I'd have to, among other things, move my 
processing machines from inside the firewall to a DMZ, where they 
connect directly to the net, which would make them much more vulnerable 
to cracking, and it would force me to move data that's currently behind 
the firewall system into a location where it's potentially accessible. 
To me, that's a non-starter. Sorry.

I think it also makes these systems more accessible to a dictionary 
attack, since now a spammer can directly query the account status via 
probing bounces -- and it's much more difficult to track that probe and 
stop it, since you've moved it out to the periphery, where a spammer 
who's poking at 8-10 incoming relays in sequence to do the dictionary 
attack would be very difficult to recognize and intercept on a real 
time basis.

Since since a lot of my incoming mail is very bursty (lots of short, 
intense peaks), I use the sponges to accept them as quickly as 
possible, and then process it in non-real time. this proposal would 
require my outgoing servers to accept and process the failures, and the 
requirement that I have to accept email in real time is not acceptable, 
some of my processing cycles can be delayed 10 days. Leaving mail on 
someone else's server until I get around to accepting it is a ban 
neighbor policy, I try to avoid that. Leaving email on someone else's 
server because I can't accept it until I'm ready to process it is a 
really bad idea.

This whole proposal doesn't seem to understand how and why large 
corporate e-mail systems work, and I'm not sure it buys anything useful 
on top of it. it asks companies to move their data into vulnerable 
situations, exactly the opposite of what we ought to be asking. Those 
stupid incoming relays are there for very good reasons: one is to 
accept mail to reduce the need for secondary MXes and other fallback 
situations, and to cut the need for mail retries and delivery delays 
(If my system is overloaded, currently, I can accept the mail into a 
sponge, and then retry every 10 minutes to speed delivery. If I have to 
push it back to someone else's system, their retry might by four hours, 
or six, or worse, causing that email to be horribly delayed. Bad 
thing). Second, those relays allow us to DMZ our mail systems, so that 
even if they are penetrated, no significant data is released. By 
forcing those systems to make these bounce decisions, you also force 
the data needed to make those decisions out from behind the firewall 
where it's vulnerable to attack or penetration, and that alone will 
make this proposal a non-starter for most large email installations.

---

From: Claus Assmann <[email-masked]>
Date: Mon, 7 Apr 2003 09:14:13 -0700

> has failed /or/ a timeout has been elasped.

Bad Idea.

RFC 2821 already requires a "fast" response to the final dot.

   To avoid receiving duplicate messages as the result of timeouts, a
   receiver-SMTP MUST seek to minimize the time required to respond to
   the final <CRLF>.<CRLF> end of data indicator.  See RFC 1047 [28] for
   a discussion of this problem.

What if the mail is expanded via an alias to hundreds of recipients?

What if only some of them fail?

---

From: Kee Hinckley <[email-masked]>
Date: Mon, 7 Apr 2003 12:27:06 -0400

>has failed /or/ a timeout has been elasped.

1. The whole point of having dumb SMTP front ends is for performance. 
If they have to sit there and wait until the back-end is done 
processing the email, you've kind of lost that benefit.
2. This assumes secondary MXs always know all of the legal email 
addresses. (I assume you aren't suggesting that a secondary MX wait 
to respond until the primary is back up again.)  We've had this 
discussion before.  Lot's of secondary MXs do not know what addresses 
are legal.
3. It's a major architectural change. It sounds very easy to say 
"don't return bounces", but clearly it requires rewriting major 
pieces of code in every MTA out there.

---

From: Daniel Feenberg <[email-masked]>
Date: Mon, 7 Apr 2003 12:48:18 -0400 (EDT)

> What if the mail is expanded via an alias to hundreds of recipients?

If the list is present, the message can be accepted. The various bounce
messages for the individual subscribers go to the list owner, not the
envelope from of someone posting to the list. The MTA can respond to the
list owner in any way mutually agreed upon. The idea is just not to
send DSNs to strangers.

---

From: Claus Assmann <[email-masked]>
Date: Mon, 7 Apr 2003 10:21:04 -0700

> list owner in any way mutually agreed upon. The idea is just not to

Please read RFC 2821:
3.10 Mailing Lists and Aliases

I didn't talk about mailing lists.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: J C Lawrence <[email-masked]>
Date: Mon, 07 Apr 2003 12:32:32 -0700

On 07 Apr 2003 10:54:00 +0200 
list-ietf-antispam  <[email-masked]> wrote:
>> On Sun, 6 Apr 2003 18:19:26 -0400 waltdnes <[email-masked]>
>> wrote:

> There's no reason why it could not work in this situation: You hold
> off the "OK" until you know that the message has been delivered,
> delivery has failed /or/ a timeout has been elasped.

Eeek!  So we have no secondary MXing structure and a requirement for a
continuous TCP session from the sender, bouncing thru intermittent MTAs
until it reaches a final destination MTA being required before a message
can be considered fully processed by any intervening node.  No.  Not
gonna happen.  Quite literally can't happen in more than a tiny
percentage of cases.

> In this case you could configure your system to temporarily reject
> mail from outside until the spool machine responds timely again.

So external systems engage in polling loops based on internal processing
criteria which they have no insight into?  Ouch.  You've just sent
delivery and receipt expense and overhead for an SMTP transaction into
overdrive.

---

From: J C Lawrence <[email-masked]>
Date: Mon, 07 Apr 2003 13:07:17 -0700

On Mon, 7 Apr 2003 12:48:18 -0400 (EDT) 
Daniel Feenberg <[email-masked]> wrote:
>> On Mon, Apr 07, 2003, Claus F=E4rber wrote: 

> If the list is present, the message can be accepted. 

Who said anything about a mailing list?  What about simple alias
expansion:

  addr1: addr2 addr3 addr4 addr5 addr6 addr7 

> The various bounce messages for the individual subscribers go to the
> list owner, not the envelope from of someone posting to the list. 

And in the above case with partial delivery success?

---

From: [email-masked] (Claus Färber)
Date: 08 Apr 2003 01:49:00 +0200

Kee Hinckley <[email-masked]> schrieb/wrote:
> to respond until the primary is back up again.)

If the secondary is down, it will have to accept the messages.

But secondaries are not only used when the primary MX is not available.
Spammers reportedly use them to circumvent problems with the primary.

> pieces of code in every MTA out there.

It's not saying "don't return bounces" but "dont't return bounces if you
can avoid it".
If you can't do it due to performance reasons, a network outage, or any  
other reason, then don't do it. But the MTAs should take all sensible  
measures to avoid having to send bounces.

Claus

---

From: [email-masked] (Claus Färber)
Date: 08 Apr 2003 01:37:00 +0200

>    a discussion of this problem.

Well, RFC 2821 is a bit misleading here. A "fast" response to the final  
dot is not really necessary to avoid duplicate messages.
The receiving MTA can take all the time it needs to process the message  
as long as it can still abort processing. Only when the message has been  
accepted irrevocably is it necessary to send the response as fast as  
possible.

It this example scenario, only the steps marked with "|" have to be as  
short as possible.

  Sender                Receiver                Other MTA
     -- open SMTP conn. --->
    <-- env. exchange ----->
     -- DATA -------------->
     -- <CRLF>.<CRLF> ----->
                             -- open SMTP conn. --->
                            <-- env. exchange ----->
                             -- DATA -------------->
                             -- <CRLF>.<CRLF> ----->
                                            (processes message)

|                                           (wrote message to spool,
|                                           => irrevocabe)
|                           <-- DATA resonse ------
|                  (=> irrevocable)
|   <-- DATA resonse ------

It should also be noted that many modern mailers (e.g. qmail) can  
deliver a message to the next host via SMTP much quicker than sendmail  
can write it into its queue.

Further, avoiding message duplication is not an absolute requirement. A  
slightly higher probability of duplicate messages might be acceptable if  
it helps to stop spam.

> What if the mail is expanded via an alias to hundreds of recipients?

An expansion counts as a delivery.

> What if only some of them fail?

This problem also occurs with multiple RCPT TOs. Then you have to send a  
positive response as soon as one delivery was successful (unless you are  
using a ESMTP extension that allows to send different status codes for  
different recipients).

Claus

---

From: Dave Crocker <[email-masked]>
Date: Mon, 7 Apr 2003 17:46:30 -0700

Claus,

CF> Well, RFC 2821 is a bit misleading here. A "fast" response to the final
CF> dot is not really necessary to avoid duplicate messages.

In fact, 25 years of experience are the basis for that language.

Delays in closing off transmission of the content very much DO lead to
duplicates. At every revision to Internet mail specifications, we have
pushed for less and less processing to be done interactively and more to
be deferred, for just this reason.

CF> The receiving MTA can take all the time it needs to process the message
CF> as long as it can still abort processing.

People forget that connections can terminate abnormally, such as through
a network outage. Robustness against network vagaries (delay, loss,
etc.) is one of the better indicators that a protocol is ready for
"internet prime time".

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 7 Apr 2003 20:45:32 -0600 (MDT)

> From: [email-masked] (=?ISO-8859-1?Q?Claus_F=E4rber?=)

> Spammers reportedly use them to circumvent problems with the primary.

There are far simpler ways to deal that spammer attack than to make
the secondary try to contact the primary to pass the message syncronously
through the primary.

 1. If you really think that passing the message synchronously through
    to the primary can work, then you don't need the secondary and should
    delete it.   In these days, MX secondaries often cause more problems
    than they fix, particularly for systems that deal with fewer than
    500,000 message/day.

 2. hide the secondary among duplicate RRs for the primary.  For example,
       srvr    IN   A     10.2.3.4
               IN   MX    10 srvr
               IN   MX    20 real-secondary
               IN   MX    20 srvr

> measures to avoid having to send bounces.

Nobody disagrees with that.  The problem is with uninformed and
incorrect claims incorrect that bounces are not needed, and with the
unworkable and inapplicable mechanisms to eliminate relaying.

A bigger issue is that no one has shown that bounces are a significant
spam problem.  The talk about fixing bounces amounts to individual
demonstrations of problem solving powers and has little or nothing to
do with solving the spam problem.

     .......

] From: Dave Crocker <[email-masked]>

] ...
] Delays in closing off transmission of the content very much DO lead to
] duplicates. ...

] CF> The receiving MTA can take all the time it needs to process the message
] CF> as long as it can still abort processing.
]
] People forget that connections can terminate abnormally ...

It might help to paint the duplicate problem explicitly:

   SMTP client                       SMTP server
1.   send("DATA\r\n"
          msg text
          "\r\n.\r\n")     -->
2.                                      recv(msg)
                                        (delay for spam filtering etc)
                 (network between client & server dies)
3.                        <--           send("250 OK")
4.                                      close()
5.         (gets error on recv())       (gets error on close())

At this point, the SMTP server has no idea whether the SMTP client heard
the "250 OK",  and so the SMTP server cannot know whether to deliver or
discard the message.  The SMTP client has no idea whether the server
sent "250 OK", "5yz reject", or crashed, and so the SMTP client cannot
know whether to send again.  No matter what the client and server decide,
there will be cases of duplicate or lost mesages.  The more time between
step #2 and step #3, the more likely this problem becomes.

---

From: Brad Templeton <[email-masked]>
Date: Mon, 7 Apr 2003 20:49:23 -0700

Rejoining the list after a week off for a conference (where I gave a nice
talk on spam issues, among other things) let me add that while I doubt
we can get rid of bounces -- no well designed software system drops traffic
on the floor without informing the interested parties (except if that is
part of the design, as in UDP) -- but we can make them easier to handle
for honest mailing lists.

This could be done with an authenticated errors-to header which provides
a common, authenticated point for all bounces to be sent to and collected,
so the mailing list manager can get one consolidated report of all problems,
ideally in a nice parseable mime multipart format or even an XML table.

Why is this useful in the fight against spam?  One of the key elements in
the fight against spam is sadly, the fight against bad spam-blockers.  People
who send mail need to know when their mail has been blocked so they can
fix the problem.   At the same time, sites don't wish to spend a lot of
resources sending out thousands of bounce replies to spams.

Thus if somebody who hosts a mailing list can provide an authenticated
address for delivery of combined reports -- with perhaps a different
token-containing address for each mailing they send out, or just a requirement
that the in-reply-to header be accurate on the errors -- they can get
reliable information on whether their mail is delivered, and work to fix
problems, all with low overhead to them and the net.

This has the advantage of being implementable in an incremental way.  Getting
a certified (not X.509, PLEASE!) error address is a voluntary act for list
managers interested in the most reliable lists.  Spammers are welcome to try
it too, but it just makes them easier to track down.

And of course, old MTAs will not send errors to the address, so you must
handle both, but slowly the volume would move to the consolidated format.

You still need bounces on individual mail, but you can eventually insist
that high volume mail use the high volume error reporting system.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Mon, 7 Apr 2003 23:38:17 -0600 (MDT)

> From: Brad Templeton <[email-masked]>

> that high volume mail use the high volume error reporting system.

Assume for the safe of argument without agreeing that the current
bounce scheme needs to be fixed (because I don't).  I'd rather delay
the effort to write and install future software implementing future
bounce protocols until after the installation of current software
implementing old protocols for lists, including support for RFC 2919
and RFC 2369.  This mailing list is among the minority of IETF/IRTF
lists that has List-Id headers.

My point is about priorities and that this should be about fixing spam. 
Perhaps bounces are a problem worth fixing despite the lack of
quantitative evidence.  If so, they are down the list of spam problems.
I think more pressing and relevant issues include:
  - defining "consent" and other terms of the art.
  - specifying the implied consent implied by having a public address.
  - listing the sorts of objectionable mail that this mailing list
     cares about.

We've spilled 10's of 1000's of words on solutions that even their
advocates must agree can't help this year or next.  Could there be a
moritorium on advocating wonderful solutions until the problem has
been sketched?  Do we really need to re-invent SMTP-TLS with shared
certs or PKI as I think I saw done today?  Maybe bounces from legitimate
mailing lists that might use a consolidated bounce format are a
significant problem, but are they a significant spam problem?

---

From: [email-masked] (Claus Färber)
Date: 08 Apr 2003 12:13:00 +0200

>                                         (delay for spam filtering etc)

                   (network dies, server crashes, whatever)

  2a.                                     (message written
                                          to spool)

> there will be cases of duplicate or lost mesages.

Well, you have to err on the safe side: As a client, unless you have  
positivly seen the "250 OK", you have to resend. As a server, you have  
to deliver the message if you sent the "250 OK".
So lost messages should not occur (although they reportedly do due to  
broken servers).

> The more time between step #2 and step #3, the more likely this
>  problem becomes.

If the problem occurs between #2 and #2a, the method above will not  
result in duplicates: The server has not taken over responsibility for  
the message and the client will just send it again.

Only between #2a and #3 is it impossible to avoid duplicates.

Claus

---

From: Daniel Feenberg <[email-masked]>
Date: Tue, 8 Apr 2003 07:24:50 -0400 (EDT)

> do with solving the spam problem.

I think the problem is rather that forged return addresses make people
reluctant to use otherwise attractive methods for spam control. A content
based spam detector that had a 1% false positive rate might be acceptable
if senders were informed that their message had been rejected, but not if
their message were dropped on the floor. After all, a dropped message (in
addition to being an RFC violation) could lead to social awkwardness or
financial loss much more readily than a rejected message. And we already
reject many messages for virus content, size, invalid address, etc -
people know to try another channel if they get a rejection. 

But a content based scanner can't DSN all its hits - most of the return
addresses are forged and many are of innocent third parties. So a
technique that makes sure the sender gets a notice (if the sender is
legitimate) without generating lots of notices to innocent third parties,
is an improvement over the current usual practice.

Restricting DSNs to the connecting host and its MXs is a reasonable
compromise along these lines. It is surely no worse than the current
nearly universal practice of content scanners of dropping suspected spam
on the floor. (I realize that there is always an option to deliver to a
spam folder, but I have never understood why that was an improvement over
ordinary delivery). 

It is true that a few DSNs would not find their way to legitimate senders
but if senders find that a problem, in the fullness of time, their mail
administrators may find ways to accomodate them. It would be up to them to
decide. We aren't talking about a legal statute, just what sites might do
to improve their service to their own users.

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 09:15:32 -0400

I have to ask this again and forgive me if it's a stupid question, but...

Is it _absolutely_ necessary to bounce back the entire message body, 
rather than just status and delivery data?

As a postmaster I can see message delivery details but not bodies and 
can diagnose just about anything. The idea of allowing a mail server 
to generate an entirely new message seems anachronistic at best, 
dangerous at worse. I do think most people keep an "outbox" today... 
10 years ago, maybe not, but please... even my cell phone stores the 
last many messages I've sent from it.

We process here a fair number of messages that are bounced in as 
their primary means of delivery. If message bodies weren't thrown 
back too, spammers could not exploit this back door.

---

From: "Jon Kyme" <[email-masked]>
Date: Tue, 08 Apr 2003 14:30:49 +0100

>(I realize that there is always an option to deliver to a

Because a UA (or some other mechanism) can age messages out of the "spam
folder" or cap it for size or something.

---

From: Chuq Von Rospach <[email-masked]>
Date: Tue, 8 Apr 2003 07:46:20 -0700

No, but I think it's important to return a full header at the minimum. 
It depends on who you think your audience for the bounce is. end users 
tend to want the whole thing so in case of a typo on sending they can 
get the message back adn send it again. programs that process bounces 
need enough info to process them properly.

Some folks return messages chopped off at some size, a useful 
compromise except it truncates MIME messages, which sometimes makes 
life interesting.

If I have a DSN that tells me what address it was originally sent to, 
I'm happy. If the DSN went through two sites that rewrote the addresses 
and now has no relevance to what it started out as, I need the full 
header to try to reconstruct what was broken along the way. If the 
header gets partially stripped (only minimal headers) I need the body 
to try to get around that. The more you muck with the message, the more 
message I need to avoid your mucking... (grin)

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 8 Apr 2003 09:04:28 -0600 (MDT)

> You have to add some steps:

The step you added has no effect and you added it in the wrong place.
For obvious practical reasons, most SMTP servers write the message to
a temporary file in a spool directory as it arrives.  Your step 2a in
practice happens during step 2 and is completed before the TCP ACK
for the "\r\n.\r\n" is received by the STMP client.

Or do I misunderstand what you mean by "write to spool"?  If you mean
"deliver to the user's mailbox" instead of "save in temporary spool file,"
then the step is still irrelevant, because network problems that happen
at any time after the "\r\n.\r\n" is received cannot be detected by
the SMTP server before the final close().

>   2a.                                     (message written to spool)

> Only between #2a and #3 is it impossible to avoid duplicates.

Not so.  If the network break any time after #2, the server will no
detect the problem until after the server sends "250 OK" and while
trying to close() the connection.  Only by sending "250 OK" will the
TCP state machine in the SMTP server start looking for any response
from the client.  The network can break at any time after the server
sent the TCP SYN to ACK the ""\r\n.\r\n" and the server will not know
until it calls close().

Making that duration longer makes network problems more likely and
so makes duplicates more likely.

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 11:07:10 -0400

Right. so Header != Body

If senders could just learn (I believe they can) to revisit the 
outbox and examine _that_ copy, then there is no reason whatever to 
clone message bodies and send them to FROM: addresses.

I think the idea of a bounce that contains the original message body 
has been the core problem and a vector for spammers... and could be 
solved by mandating a return-to-sender of full status data less any 
of the message body.

This would remedy a security/privacy hole as well w/r/t misdirected 
returns, admins and unintentional storage of transient stuff.*

- jim
-----
* The notorious follow-on topic/extension "there is no expectation of 
privacy in e-mail" is hereby decreed off-topic for this thread. 
Please don't go there or I'll have to call someone.

At 7:46 -0700 4/8/03, Chuq Von Rospach wrote:
>No, but I think it's important to return a full header at the 
>minimum. It depends on who you think your audience for the bounce 
>is. end users tend to want the whole thing so in case of a typo on 
>sending they can get the message back adn send it again. programs 
>that process bounces need enough info to process them properly.
>
>If I have a DSN that tells me what address it was originally sent 
>to, I'm happy. If the DSN went through two sites that rewrote the 
>addresses and now has no relevance to what it started out as, I need 
>the full header to try to reconstruct what was broken along the way. 
>If the header gets partially stripped (only minimal headers) I need 
>the body to try to get around that. The more you muck with the 
>message, the more message I need to avoid your mucking... (grin)
>
>
>>Is it _absolutely_ necessary to bounce back the entire message 
>>body, rather than just status and delivery data?

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 8 Apr 2003 09:32:38 -0600 (MDT)

> From: Daniel Feenberg <[email-masked]>

> > spam problem. ...

> their message were dropped on the floor. ...

1% false positive rates are accepted only by people who don't really
care about mail.  People who make their living on the net do not
tolerate false positive rates significantly higher than the underlying
error rate of the mail system or 0.1%, whichever is higher.  (15 or
20 years ago the net lost 1% or more of all messages, but like the
telephone system 50-75 years ago, expectations changed.)

> is an improvement over the current usual practice.

That assumes statements not in evidence and that I think are wrong:
  1. most return addresses of messages caught by spam filters are forged.
  2. many are of innocent third parties.
  3. "content based spam detectors" must use bounces to indicate false
      positives instead of STMP status codes.

There is limited evidence to disprove #1 and #2, but #3 is obviously
wrong.  Many content based spam detectors operate during the STMP
transaction and so do not themselves cause bounces.

> compromise along these lines. 

The problem with that notion is that it is impossible to define, not
to mention implement.  It makes no sense unless you assume more things
that at best have not been established:
  4. that STMP clients (mail senders) are also SMTP server (mail receivers)
  5. that relays are wrong or unneeded.

> administrators may find ways to accomodate them....

Waiting for the fullness of time for mail administrators to find a
solution to lost data is not a widely respected tactic for protocol
design, at least not in professional circles.

    .........

] From: Jim Youll <[email-masked]>
]
] I have to ask this again and forgive me if it's a stupid question, but...
]
] Is it _absolutely_ necessary to bounce back the entire message body, 
] rather than just status and delivery data?

Yes, it is absolutely necessary that enough of the message be bounced
so that sender can know which message bounced. 

] As a postmaster I can see message delivery details but not bodies and 
] can diagnose just about anything. 

Most mail senders care less about diagnosing the problem than knowing
which of their messages and to which addressee did not arrive.  

]                                   The idea of allowing a mail server 
] to generate an entirely new message seems anachronistic at best, 
] dangerous at worse. I do think most people keep an "outbox" today... 

An "outbox" containing the 50 messages you've sent in the last 5 days
does not help deal with a bounce until you know which message bounced.
You need to know which message nees to be retransmitted or otherwise
handled.  Even if the bounce contains all of the headers and you are
one in 10,000 who can read headers, you may not have enough information
to pick the original from your "outbox."  Only if your MUA generates
Message-IDs that are preserved by your MSA and all subsequent MTAs in
the path might you reliably figure it out.  The source of the bounce
cannot know if your MUA and MSA are sane.  What can it do except
send most of the SMTP transaction log?

---

From: Chuq Von Rospach <[email-masked]>
Date: Tue, 8 Apr 2003 08:45:22 -0700

> Right. so Header != Body

yes.

>
> If senders could just learn (I believe they can) to revisit the outbox

returning the body is mostly a leftover from days before most users had 
a gui client and an outbox. does AOL have an outbox? I don't remember.

> solved by mandating a return-to-sender of full status data less any of 
> the message body.

then the spammers will just put their message in an X-header. (grin, 
sort of)

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 11:53:53 -0400

At 8:45 -0700 4/8/03, Chuq Von Rospach wrote:
>returning the body is mostly a leftover from days before most users 
>had a gui client and an outbox. does AOL have an outbox? I don't 
>remember.

Yes. Everything does, even my phone, palm pilot, yada yada...
Right, so "leftover from days..." suggests it could be lost without 
horrible side effects, and a generally better world would ensue (re 
both spam and privacy)

>>I think the idea of a bounce that contains the original message 
>>body has been the core problem and a vector for spammers... and 
>>could be solved by mandating a return-to-sender of full status data 
>>less any of the message body.
>
>then the spammers will just put their message in an X-header. (grin, sort of)

but mail clients could be setup to not show headers, not 
display/load/interpret html/scripts in headers, etc... so sort of, 
yes, but it seems the overall information capacity available in this
exploit would seem to be greatly diminished by just saying that 
bounces are about delivery status, not about regenerating content.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Daniel Feenberg <[email-masked]>
Date: Tue, 8 Apr 2003 12:50:29 -0400 (EDT)

I think this thread is about run out, but I will comment on a couple of
items.

>   1. most return addresses of messages caught by spam filters are forged.

About half of spam in my inbox has a "hotmail" or "yahoo" return address. 
I wouldn't bounce a message to that address based on content, although
I realize that the address is unlikely to be active. 

>   2. many are of innocent third parties.

I don't know how many are innocent third parties. I get a few bounces
every week as an innocent third party myself, but I don't know if that is
typical. It may reflect the fact that most content based scanners don't
bounce mail. 

>       positives instead of STMP status codes.

I agree they don't have to. But none of the scanners presented at the MIT
spam conference could. I agree that it would be preferrable to do so, but
many here argue that it is not practical for large sites. I don't mean to
agree with them, but to accomodate their beliefs, in the interest of
obtaining agreement.

The only scanner I know of with this characteristic is "Canit", whose
author/publisher is a contributor to this list. 

> transaction and so do not themselves cause bounces.

They are to be praised. If I were King, I would allow only those scanners
to be used.

>   5. that relays are wrong or unneeded.

I understand that the connecting host may not wish to receive mail, and
ask only that if have an appropriate MX to receive mail on its behalf. If
you think that unreasonable, you should expand on why.

> design, at least not in professional circles.

You are not listening. I am suggesting an alternative to dropping messages
on the floor. I am not advocating dropping messages on the floor.

---

From: J C Lawrence <[email-masked]>
Date: Tue, 08 Apr 2003 10:10:30 -0700

On 08 Apr 2003 12:13:00 +0200 

> to deliver the message if you sent the "250 OK".  So lost messages
> should not occur (although they reportedly do due to broken servers).

It is impossible to get two nodes to have perfect knowledge that they
share the same data at the same instant over a non-instantaneous link.
In the case of SMTP this is realised by the server sending the 250 and
the client never receiving it (due to crash, lost connection, routing
error, whatever).  The server thinks it got the message and everything
is fine.  The client never got the ack, and so retries the delivery
later.  This is nothing to do with broken servers.

That said, a more common cause is broken servers mishandling of
rejections mid a RCPT-TO packet.

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 8 Apr 2003 11:24:32 -0600 (MDT)

> I realize that the address is unlikely to be active. 

What does that have to do with forgery?

> I don't know how many are innocent third parties. 

You previously said otherwise.

>                                                   I get a few bounces
> every week as an innocent third party myself, 

me too.

>          It may reflect the fact that most content based scanners don't
> bounce mail. 

That betrays another unsupported assumption, that the bounces you see
result from content based scanners.

> spam conference could.

So SpamAssassin and Brightmail were not presented at the MIT conference?
That might say something about the MIT conference, but nothing else I can
see.  (I know SpamAssassin can run during the SMTP transaction.  I'm
guessing Brightmail also can.)

>                        I agree that it would be preferrable to do so, but
> many here argue that it is not practical for large sites.  ...

Who has made that at least over-broad claim?  The DCC is a content
based scanner.  It works best and fastest during the SMTP transaction.
The fact that mode generates no bounces is incidental.

> you think that unreasonable, you should expand on why.

Many outfits use split mail senders and mail receivers for many good
and sufficient reasons.  Thus, it is impossible to send bounces back
to the IP address of the last SMTP client.

If instead you meant that bounces should go to an MX server of the
reverse DNS name of the last SMTP client, then you've destroyed the
utility of bounces for all mail that comes through an MX secondary.
What is an MX secondary supposed to do with a bounce from the primary?

---

From: J C Lawrence <[email-masked]>
Date: Tue, 08 Apr 2003 10:34:41 -0700

On Tue, 8 Apr 2003 08:45:22 -0700 

>> outbox

> had a gui client and an outbox. does AOL have an outbox? I don't

This tends to cause pain for me as I sent mail from the same address
from at least three physically discrete systems.  Admittedly I'm not the
common case, but when I receive a bounce at [email-masked] and the OutBox
copy is on my system at work/laptop/palmtop/other_device....

> then the spammers will just put their message in an X-header. (grin,

I tend to consider the concern about bounce payloads as a red herring.
I've seen a little (not much) deliberate use of bounces by spammers
(usually forgery of messages that look like bounces but aren't), but
more importantly most non-technical users (and that's the majority of
users) tell me that "bounces are confusing, techno-geeky gobbledegook".
Losing message bodies in a bounce would increase that perception and
make bounces even less useful to the vast majority of people who need to
see and understand those bounces (and already have problems with the
current state of affairs).

Error handling should be simple, obvious, and dumb.  Big red letters,
simple words, and language that a 3yr old can understand is the name of
the game.  The more we trim and curtail the level of helpful and
correlative data in bounces/error messages, the more we make them
opaque, unintelligible, and un-used by users, and therefore the more we
transform email into (effectively) being an unreliable transport:

  You never got my message?  Oh yeah, I got some confusing bounce
  message thing but I couldn't make sense of it.  
  
  Yeah, I know, those things are useless.  Just FAX/call me next time.

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 8 Apr 2003 11:52:42 -0400

>if senders were informed that their message had been rejected, but not if

As a counter-point, consider challenge-response systems, which are 
happy to send bounces to all senders, assumed spam or not.  Part of 
the difference might be that individuals probably don't have a big 
problem with bounces, but an ISP doing bounces is basically an 
indiscriminate weapon.  That's a good argument for a bulk bounce 
protocol.

>spam folder, but I have never understood why that was an improvement over

It's not ideal, but it is far faster to scan a folder of "probably 
spam" than to try and filter spam out of your real email.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Michael Marking <[email-masked]>
Date: Tue, 8 Apr 2003 11:34:30 -0700 (PDT)

- ------------------------------------------------------------------------

> transform email into (effectively) being an unreliable transport:

A bounce message should, in addition to ancillary information such as
headers, have two distinct components:

	a human-readable message explaining the reason for the bounce

	a machine-readable message with the necessary reason codes and
	other supporting information

Human readable messages won't always work by themselves. For example, if
the message is bounced with an explanation in English to a reader who
understands only Chinese, then the utility will be low.

If there were a standard syntax for a machine-readable bounce
explanation, then MUAs could convert those explanations into something
that makes sense in the context of the reader. It could automatically
say "Eudora" instead of "MUA", and say "XYZ's mail system" instead of
"relay ABC". There might even be a little competition among developers
as they strive to write the most sensible and intelligible
interpretations of the standard. The same thing has already been done
with e-mail addresses -- some users have never seen

	"john q. doe" <[email-masked]>

they only know address books, without any <>s, and otherwise get along
quite nicely.

The machine-readable message would, of course, facilitate automated
handling -- perhaps the mail system can deal with the problem, without
user intervention.

A similar more descriptive and well-defined set of SMTP error codes
would also help along those lines, but (maybe?) the length of the human
message would be more limited.

- ------------------------------------------------------------------------

---

From: J C Lawrence <[email-masked]>
Date: Tue, 08 Apr 2003 11:44:51 -0700

On Tue, 8 Apr 2003 11:34:30 -0700 (PDT) 
Michael Marking <[email-masked]> wrote:

> interpretations of the standard. 

Absolutely. Its something I've wanted to see for a long time:
MUA-specific localised etc renderings of bounces.  If sufficient
information were presented standard an MUA could provide a little
wizardish interface for bounces.  Say a dialog:

  Your messages to BUBBA about SUBJECT wasn't didn't make it..

  Click <here> to see the original message you sent that wasn't
  delivered.

  Looking at your mail history/address book it seems that you meant to
  send this message to <here> instead.  Click <this> button to send your
  message to that (corrected) address.

  Click <here> to review your message and resend it to a new or
  different address.

  Click <here> for more help or explanations on what happened to your
  message.

Given at least a little help from the DSN that wouldn't be that
difficult.

---

From: Brad Templeton <[email-masked]>
Date: Tue, 8 Apr 2003 12:43:48 -0700

> indiscriminate weapon.  That's a good argument for a bulk bounce 

Strictly speaking, a challenge/response tool does not send a "bounce" or
error message.  It sends a challenge, which is a new piece of mail, and
does not indicate the earlier message failed.  A proper c/r tool is holding
the mail in a spool, to be delivered once the response comes.   A "bounce"
means delivery failed, and the user may need to try again, or give up.

A C/R system MUST NOT challenge mailing list mail, for example, but a MTA
SHOULD send bounces/error reports on mailing list mail.

Challenges go to the author of the mail, as per the From/Reply-to, while
Bounces go to the envelope-from sender of the messages.

Now, one could define a set of practices for challenge/response, so that
the processing of challenges at the recipient end can be streamlined.

Though in most cases not automated of course as that would defeat the point
of some challnge systems, though "stamp" based systems would actually wish
to automate the response, especially if it were a CPU based stamp.

However, the key realization in all this discussion of bouncing is that
any anti-spam nature will cause some mail to be blocked, and since spam is
now over half of all mail, in fact it will cause _most_ mail to be blocked.

As such it is a very difficult task not to include some legit mail in that
pool

Though in most cases not automated of course as that would defeat the point
of some challnge systems, though "stamp" based systems would actually wish
to automate the response, especially if it were a CPU based stamp.

However, the key realization in all this discussion of bouncing is that
any anti-spam nature will cause some mail to be blocked, and since spam is
now over half of all mail, in fact it will cause _most_ mail to be blocked.

As such it is a very difficult task not to include some legit mail in that
blocked set.  Perhaps impossible.

Thus good error reporting on spam blockage is a MUST.  The current all too common
practice of dropping legitimate mail on the floor with nobody aware of it
is really not an acceptable design.

The people most likley to feel the pinch are mailing list operators, though
their pain per lost messages is less than that for person to person messages.
However, they will feel most of the brunt because they send the most legit
mail and their activites are technically very similar to spam, the only
difference being that people want to be on their lists.

It occurs to me that improved and more efficient error reporting lets us track
where our anti-spam solutions are failing us.

I would even think outside the box on error reporting.  For example, I could
imagine defining an errors-to style header which allows the specification of
a URL for http/https POST of error reports.  (An E-mail bounce to be sent
if the POST fails for whatever reason.)

This would actually be more efficient than using SMTP for this, and makes a
annoyance attack almost impossible since you can use a reserved word in the URL to
assure only people who wanted to get email errors get them.  (People can
still forge messages to get people unsubscribed however.)

I can also imagine big mail providers using this.  AOL for example could then
collect most bounces on messages sent by AOL members at an HTTP address, and
collate the so the user can, with a web interface, see their latest mail failures
at any time.  (Any new mail failure after the last browsing would normally
cause an E-mail indicating "there are new mail failures", but the volume in
the mailbox would be reduced for ones that come in bunches)
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 8 Apr 2003 15:07:48 -0400

>users) tell me that "bounces are confusing, techno-geeky gobbledegook".

Which is probably why spammers aren't using fake bounces.  I have 
lots of evidence that many users pay absolutely no attention to 
bounces.  They don't know what they are, they don't recognize where 
they came from, they've been told not to open messages from people 
they don't know--so they delete them.  Fundamentally they aren't 
going to be useful until the MUA can correctly identify them, prompt 
the user for a fix, and adjust the address book as well.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 8 Apr 2003 17:34:14 -0400

>means delivery failed, and the user may need to try again, or give up.

Not entirely true.  Many mail systems send notices that the message 
is still in the process of being delivered, but hasn't yet.  But 
let's avoid the word "bounce" for a second.

I don't see why a challenge/response system is any different than any 
other (non)-delivery notification.  It's a notification concerning 
the delivery of a message.  Sure, it requires some action.  But a 
vacation notice or a straight bounce also requires some action if you 
want to get the message through.

>SHOULD send bounces/error reports on mailing list mail.

Why do you say that?  We've all had the enjoyment of dealing with a 
C/R system that did just that to this list.  You clearly don't want 
the challenge going to each member of the mailing list.  It should go 
to the list owner, who can deal with it as they wish (probably by 
booting the subscriber--but that's up to them).
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Brad Templeton <[email-masked]>
Date: Tue, 8 Apr 2003 14:59:30 -0700

> let's avoid the word "bounce" for a second.

Well, there are error reports (message not delivered, you must send again
if you want it delivered) and status reports (message pending but not yet
completed, you need do nothing, you'll get an error if it fails) and challenges
(message pending but not delivered, you must do something to get it to be
delivered.)

Added to this could be "It's now been delivered" status reports for when
you got a prior status report or challenge, and an error which effectively
means "you took too long to respond to the challenge but it was delivered/
was not delivered."

(Yes, mail can be delivered if you don't resond to the challenge, in my
system I get a daily summary and pick any messages I want for delivery, but
I don't inform folks about this at present.)

> booting the subscriber--but that's up to them).

True, though because it is normal for mailing list management systems to
do automatic error handling -- usual technique being to count the errors
and remove the user after too many -- I doubt many would want to handle
actual challenges that need a human.   But it would be up to them as you
say.  But this means the challenge goes to the envelope-from, and might not
go to the human being who really needs to see it, no?

---

From: Chuq Von Rospach <[email-masked]>
Date: Tue, 8 Apr 2003 15:01:08 -0700

On Tuesday, April 8, 2003, at 02:34  PM, Kee Hinckley wrote:

> Not entirely true.  Many mail systems send notices that the message is 
> still in the process of being delivered, but hasn't yet.  But let's 
> avoid the word "bounce" for a second.

off-topic, but what an absolute waste of time, especially those that do 
it when the precedence headers tell them it's bulk and we don't CARE.

> C/R system that did just that to this list.

I think what brad is saying is that when you subscribe to a list, as a 
user, it's your responsibility to whitelist mail from that list through 
your c/r system. If you don't, and the c/r system challenges the mail 
list or users who post to it, the user is broken, not the list.

Our police is pretty simple here: if we get a challenge to any of our 
subscription systems, it's ignored, or treated as a bounce as 
appropriate. We will not respond to a challenge to open a port for a 
list. That's the user's responsibility. We feel the risk of responding 
and pissing off a user is so much worse than a user who does that to 
themselves and wonders why the subscription isn't working, so we'll let 
subscriptions lapse rather than respond to challenges for a list. 
(OTOH, I think that means we as list owners have a responsibility to 
explain this to users, so they know what needs to be done, and putting 
that in the FAQ and welcome messages is on my list to get done, 
hopefully this week).

---

From: [email-masked]
Date: Tue, 8 Apr 2003 20:33:47 -0400

On Tue, Apr 08, 2003 at 09:32:38AM -0600, Vernon Schryver wrote

> so that sender can know which message bounced.

  Unless I'm trying to mailbomb [email-masked] the Date: and
To: headers should be perfectly sufficient.

> ] As a postmaster I can see message delivery details but not bodies
> ] and can diagnose just about anything. 
>

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 20:59:17 -0400

I cannot imagine that what you present as a reason for sending back 
message bodies is a plausible or common scenario in 2003.

How many protocols do we have that - at the application layer - send 
back  the whole request when things don't work out? Imagine if you 
were to abort an FTP upload and got a copy of the whole file back "so 
you know which one failed"!

You're saying that sender must
.. send enough messages to ONE recipient
.. in the interval between the first transmission and the first return
.. in a setting where *some* of the messages are rejected and *some* 
are accepted
.. and the messages are so similar in character that the defective 
ones can't be discerned from a headers-only response message 
(including the Subject: line) ... (e.g. if I send a 400mb attachment 
that's rejected as "too large" I betcha I could figure it out from an 
error response and time stamp)

The more I think about this business of returning e-mails to the 
_apparent_ sender, the more completely wrong it seems. It is an 
anachronism and we should be considering putting an end to this 
practice.

---

From: Kee Hinckley <[email-masked]>
Date: Tue, 8 Apr 2003 20:50:34 -0400

>go to the human being who really needs to see it, no?

Yes.  But think how much fun list users have dealing with broken 
vacation programs, and multiply that by all the new C/R systems being 
developed.  I think it's preferable that the challenge end up getting 
sent to the list's bounce handler, and the sender removed, than the 
alternative.  So if you forget to whitelist, you don't get on the 
list.  That's pretty straightforward.

At 3:01 PM -0700 4/8/03, Chuq Von Rospach wrote:
>I think what brad is saying is that when you subscribe to a list, as 
>a user, it's your responsibility to whitelist mail from that list 
>through your c/r system. If you don't, and the c/r system challenges 
>the mail list or users who post to it, the user is broken, not the 
>list.

Right.  And what I'm saying is that the BCP should be that it sends 
to the list owner.  Then the only person to pay the cost of 
forgetting is the subscriber--not all the other list members.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: Chuq Von Rospach <[email-masked]>
Date: Tue, 8 Apr 2003 18:27:31 -0700

On Tuesday, April 8, 2003, at 05:50  PM, Kee Hinckley wrote:

> developed.

cleaning up after one of those today, in fact. in danish. appends a 
message to whatever is sent it, doesn't rate limit to any given 
address, and strips headers, so our no-loop protection is removed. I 
didn't catch it until we had about 30 of them, all > 3 megs in size, 
romping through the system.

Our server will be back to normal about midnight, I think. maybe.

running e-mail systems is so much fun. The only thing you can depend on 
is that standards will be ignored on a regular basis by people who do 
what they feel like doing. something to keep in mind as we draft new 
standards and expect everyone to follow them.

---

From: [email-masked]
Date: Tue, 8 Apr 2003 21:29:29 -0400

On Tue, Apr 08, 2003 at 12:43:48PM -0700, Brad Templeton wrote

> Strictly speaking, a challenge/response tool does not send a "bounce"
> or error message.  It sends a challenge, which is a new piece of
> mail, and does not indicate the earlier message failed.

  So we've got a different set of angels dancing on the pinhead.

> A proper c/r tool is holding the mail in a spool, to be delivered
> once the response comes.

  Can you spell D-O-S A-t-t-a-c-k ?

> A C/R system MUST NOT challenge mailing list mail, for example,
> but a MTA SHOULD send bounces/error reports on mailing list mail.
> 
> Challenges go to the author of the mail, as per the From/Reply-to,
> while Bounces go to the envelope-from sender of the messages.

  A whitelist needs only to know one (or more) of...
  - sending machine name(s)
  - envelope sender (MAIL FROM:)
  - if you're generous, an entire domain

  This should be simple enough to keep accessable to (not necessarily
physically on) the internet-facing MTA.  It'll have a public IP address
and (as seen from the inside of the firewall) an RFC1918 address, let's
say 192.168.1.1.

  Howsabout the following...

  Internet-facing MTA receives TCP handshake, MAIL FROM: and RCPT:.
  - It sends those data to a MySQL server at 192.168.1.2, which can
    respond to the MTA with any of the following, which the MTA then
    passes on to the sending MTA...
    250 OK
    450 Recipient's inbox is full. Try again later.
    550 Go away... and don't come back.
    950 See http://www.<destination ISP>.com/~userID/bypass.html for
temporary whitelisting info.

  OK, I pulled "950" out of a hat.  If it's already in use, assume
another number.  The sending MTA gets a 950 "challenge" (sorta like a
550, but without the "Don't come back" attitude).  The sending MTA
passes this on, eventually to the sender.  Mailing lists will have to
list on their web page the 3 items listed above.  If the list uses VERP,
its webpage should have a short form that accepts your email address and
generates the corresponding VERP, so the user knows what MAIL FROM: to
whitelist.  The change required to SMTP by this is a new set of response
codes.

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 8 Apr 2003 19:50:19 -0600 (MDT)

> you know which one failed"!

If you used FTP as much as you use mail and if your FTP commands
could take 5 days to fail, that would not sound so silly.
In other words, I suspect you never used UUCP for copying files.

> .. and the messages are so similar in character that the defective 

Yes.

> error response and time stamp)

Your seem to be assuming your situation is universal.  I bet we could
find someone who sends enough 400 MByte attachments that they could
not identify a bounce from only the headers added by their MUA.  I'd
ask people sending lab data or various graphics artists whether they're
careful about using unique subject lines.

Consider your traffic to this mailing list.  If you got a bounce next
week from calcite.rhyolite.com, how would you know which of the "courtesy
copies" you've sent me bounced?  Subject: lines would be useless.  If
your MUA generates Date: or Message-ID: lines and logs them in your
"outbox," you could figure it out, but could your grandmother?

> anachronism and we should be considering putting an end to this 

Fine!  I agree should be a better way.  Put an end to the mess.
Even I can sketch some obvious protocol bits what would do the technical
part of the job.  But you must accept the whole job and not just the
easy, netnews sidewalk superintendent parts of the job.

The easiest part of your job is figuring out and documenting the
protocol that will let a compliant MUA do the right thing when it
receives your new concise DSN.  It's also easy to look at the message
to decide whether to do an old or new and concise bounce.

You must also modify 3 or 4 MTAs (e.g. sendmail, exim, smtpd) to
generate the new DSN.  You'll need to fix 2 or 3 MUAs (e.g. Mozilla
and Outlook) to understand them.  But that's not the hard part.

An important but not hard part of the job is letting people at the
source of the message diagnose the problem.  People who've spent years
trying to guess why a distant machine run by idiots who can't look at
their own logs or do their own tests is intermittently rejecting mail
have more sympathy for the current, ad hoc, brute force, bandwidth
wasting, theoretically spammer exploitable scheme.

The hard part is getting a few 1,000,000 SMTP servers to install your
MTA patches and a few 100,000,000 people to install your new MUAs,
and to deal with the transition.  

My point is that this mailing list, like most anti-spam efforts, has
an awful lot of sidewalk supervising.

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 22:16:02 -0400

>In other words, I suspect you never used UUCP for copying files.

I forgot to say "hell no" to this!

No, I don't want the original back! I have one.
I also don't want some shattered version of it, or one with error messages
scribbled all over it.

I want status information. That's quite enough.

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 22:13:28 -0400

>wasting, theoretically spammer exploitable scheme.

Message body has nothing to do with this.

>and to deal with the transition.

That is not necessary. Just change the default behavior of sendmail
and when the next root exploit is found and everybody is forced to upgrade
within 24 hours, it'll be largely deployed.

>an awful lot of sidewalk supervising.

Ahhh no, but you've burned a lot of bytes building this strawman.

It seems the point on which we disagree is whether there is some
merit to regenerating entire new messages that contain full message
bodies.

I believe there is not, in most if not all cases, a compelling argument
for continuing the practice of sending back entire messages at the
application level.

Seriously, really, how common is the scenario you describe, and by what
other means would the correspondents have already solved the problem out \
of band?

The number of messages that are _not delivered_, and _not discernable by
the sender_ looking at everything but the body... this can't possibly be
a significant fraction of all e-mail.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Vernon Schryver <[email-masked]>
Date: Tue, 8 Apr 2003 20:47:38 -0600 (MDT)

> Message body has nothing to do with this.

As written, that is wrong, particularly in this age of body filters.
You're right that the body need not come from the remote system to
aid diagnosis.  Still, your new DSN format must make it completely
clear and unambiguous where the body fits in the transaction log.
Or your DSN system must recognize messages where any bounce should
include the message.

> within 24 hours, it'll be largely deployed.

That's a leisurely deployment schedule, since the last root exploit
of sendmail before the recent pair was six years ago.
(see http://search.cert.org/query.html?col=certadv&qt=sendmail )

Your "24 hours" won't be half over for at least six months.  Check
the Recived headers from this mailing list.  You'll find some contributors
are running 8.9.1a/8.9.1a.  Even odin is running 8.11.6/8.11.6

> for continuing the practice of sending back entire messages at the

When you're dealing with errors, hand waving doesn't justify not providing
sufficent information to deal with a significant minority of cases.

> other means would the correspondents have already solved the problem out 
> of band?
>

My unsupported claim is that this scenario is vastly more common than
spam disguised as bounces.  I bet it's more common that bogus bounces
resulting from forged spam headers.  As others have pointed out, most
users don't understand, and more important, don't care much about how
email works.

And yes, I agree that that users most commonly discard bounces unread.

---

From: Jim Youll <[email-masked]>
Date: Tue, 8 Apr 2003 23:00:45 -0400

>Or your DSN system must recognize messages where any bounce should

I did think about that. The common practice is not to return messages
that are rejected by filters, but to drop them somewhere e.g. a /spam
folder. Message body has - if not nothing, then very little - to do with it...

More common filters that scan for viruses definitely do not send the
message bodies back but rather, add an informational message. But there
is no reason the message can't be generated fresh FROM the destination MTA
TO the apparent sender. It need not be a literal "bounce" of the original,
but a reply to it... which leaves open options for cool MxA's to sync up
Message-IDs too.

I am not proposing a new DSN format.  Don't complicate things.

Sendmail already has this option, it's just not the default.
Matter of fact it can *ask* the other end not to send back message bodies
in bounces (not that the other end will pay attention).

>(see http://search.cert.org/query.html?col=certadv&qt=sendmail )

You take me too literally.

>users don't understand, and more important, don't care much about how

Using your own scenario - a hobbled MTA behind a UUCP feed
that is intermittent and error prone...

... by definition, those users, and that server, are BADLY equipped
to deal with full-message-body bounces and would be better served
by an efficient use of channel capacity. Returns of message bodies
is a relic that mimics physical message transmission. Bits don't need
to be sent back.

I am too. I hate receiving file attachments back in a bounce. Pointless
and actually, potentially dangerous.

It's an exercise that IS open to spammers (my servers definitely
handle messages that are delivered this way) with not much benefit
to the end user. If you want to make delivery status work right, continuing
to send extra junk at the application layer is just wrong.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: J C Lawrence <[email-masked]>
Date: Tue, 08 Apr 2003 22:27:07 -0700

On Tue, 8 Apr 2003 22:13:28 -0400 

> and when the next root exploit is found and everybody is forced to
> upgrade within 24 hours, it'll be largely deployed.

This is manifestly false.  Old versions of sendmail are rife in the real
world.

---

From: Jim Youll <[email-masked]>
Date: Wed, 9 Apr 2003 01:36:17 -0400

>This is manifestly false.  Old versions of sendmail are rife in the real

eh. don't take such things so literally.

this is not the sort of change that all must adopt in order to make a 
difference.

and yes there are many versions of sendmail in the wild, and many 
non-sendmails as well. But still the character of this suggestions is 
that of a change that can be made for private gain (internal bounces 
lose weight) that also yield public benefits. it is in that 
happy-space of individual incentive (even if slight) and public 
welfare.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: J C Lawrence <[email-masked]>
Date: Tue, 08 Apr 2003 22:57:24 -0700

On Wed, 9 Apr 2003 01:36:17 -0400 
Jim Youll <[email-masked]> wrote:
>> On Tue, 8 Apr 2003 22:13:28 -0400 Jim Youll <[email-masked]>
>> wrote:

> happy-space of individual incentive (even if slight) and public

Fair dinkum.  However this discussion of bounce contents is distractive,
unpurposed, and bears no relation to the purpose of this group.  Bounces
have yet to be shown as either a current spam vector of appreciable
size, or a likely future vector.

How about a little self-control and topic policing?  I know I'm no saint
there (glances guiltily toward his OutBox), but we can do better than
this.

---

From: Chuq Von Rospach <[email-masked]>
Date: Tue, 8 Apr 2003 23:02:51 -0700

> This is manifestly false.  Old versions of sendmail are rife in the 

also, if you go back and actually LOOK at the history of sendmail, 
there have only been two root exploits fixed going back to 8.9.3 in 
February of 1999, four years ago. In that time,  there have also been 2 
or 3 potential local DOS exploits fixed, a couple of those in cases not 
normally used by sendmail installations, and about half a dozen 
security fixes that work around broken things in OSes like Linux.

Lots of people dump on sendmail, but it's reputation for being a 
security dog got beaten out of it a while back. No, that's not as clean 
as having it being designed in from the front, but sendmail didn't have 
predecessors to learn from, either.

chuq (may the religious war begin....)

---

From: Matt Sergeant <[email-masked]>
Date: Wed, 9 Apr 2003 09:56:49 +0100

On Tuesday, Apr 8, 2003, at 20:43 Europe/London, Brad Templeton wrote:

> while

If this is the case, C/R systems will cause mail loops. I thought we'd 
learned from the vacation message problems that all automated responses 
should go to the envelope-from?

Matt.

---

From: [email-masked]
Date: Wed, 9 Apr 2003 18:11:29 -0400

On Wed, Apr 09, 2003 at 09:56:49AM +0100, Matt Sergeant wrote
> should go to the envelope-from?

  Practical question... (Theoretical types can stick their fingers in
their ears while I ask)  how many MTAs (besides Qmail with "Return-Path"
headers) make this info available to the MUA ?  And in those cases, how
many MUAs would know about it ?  The roblem with an end-user C/R system
is that the MUA usually only has the "From:" header to work with and Joe
Average (as opposed to the average subscriber to this list) doesn't have
a clue about the difference between "From:" and "MAIL FROM:"

---

From: J C Lawrence <[email-masked]>
Date: Wed, 09 Apr 2003 15:57:44 -0700

On Wed, 9 Apr 2003 18:11:29 -0400 

> their ears while I ask) how many MTAs (besides Qmail with
> "Return-Path" headers) make this info available to the MUA ?  

Exim can, but requires a non-default (at least under Debian/Linux)
configfile option.

> And in those cases, how many MUAs would know about it ?  

Proximal to zero.

> The roblem with an end-user C/R system is that the MUA usually only
> has the "From:" header to work with and Joe Average (as opposed to the
> average subscriber to this list) doesn't have a clue about the
> difference between "From:" and "MAIL FROM:"

True, unless the C?R system it set up server-side, or by someone
server-side proficient (which TMDA essentially requires).

---

From: wayne <[email-masked]>
Date: 09 Apr 2003 19:52:07 -0500

In <[email-masked]> J C Lawrence <[email-masked]> writes:

> On Wed, 9 Apr 2003 18:11:29 -0400 waltdnes  <[email-masked]> wrote:
> 

Are you sure that this is not the default behavior under Debian/Linux?
The copy of eximconfig on my "sid" version shows return_path_add
options on the appropriate transports.

> > And in those cases, how many MUAs would know about it ?  

Even if Debian/Linux add return-path headers by default, it wouldn't
change the answer.  However, I don't know enough about other mailers
to know if "proximal to zero" is right or not.

-wayne

---

From: J C Lawrence <[email-masked]>
Date: Wed, 09 Apr 2003 20:49:06 -0700

On 09 Apr 2003 19:52:07 -0500 

> options on the appropriate transports.

I should probably qualify my prior statement:

  I clearly recall adding the return-path items to my exim.conf under
  Debian/Linux roughly 3 years ago (and have comments in that file
  annotating that edit).  I haven't checked the current Debian exim
  package.

Thanks.

> to know if "proximal to zero" is right or not.

I'm not aware of any MUAs (possible exception of GNUs, as I'm not sure
if the config I used there was custom or default) which pay attention to
added envelope headers.

---

From: wayne <[email-masked]>
Date: 09 Apr 2003 23:26:48 -0500

> if the config I used there was custom or default) which pay attention to

uh, DUH!  I have no idea why I didn't mention this before.

I suspect there are quite a few MTAs that record the return path as a
header in the delivered email.  Or, at least, SpamAssassin lists a
bunch of variations on the header name in the EvalTests.pm code.  I
suspect that if this info was not commonly available, no one would
have spent the time to write the code and fix bugs in it.

-wayne

---

From: Vernon Schryver <[email-masked]>
Date: Wed, 9 Apr 2003 22:47:56 -0600 (MDT)

> From: J C Lawrence <[email-masked]>

> >>> "Return-Path" headers) make this info available to the MUA ?

Sendmail has added Return-Path for years, and the UNIX style From_
for decades.

> ...
> I'm not aware of any MUAs (possible exception of GNUs, ...

What does it mean for an MUA to know about the envelope sender?

The decades old BSD and SV `mail` programs know about the UNIX
style From_ header with its envelope sender value, and in fact require
it in mailboxes.  I suspect far newer MUAs such as Netscape when
running on UNIX-like systems are similar.
The old `mail` also knows about Return-Path in the sense of displaying
the header unless you tell recent versions to suppress it.

What about the decades old UNIX `vacation` program?  If it is an MUA,
then it certainly knows about the envelope sender, because that's
where it directs its messages.

---

From: [email-masked] (Claus Färber)
Date: 10 Apr 2003 13:48:00 +0200

> for the "\r\n.\r\n" is received by the STMP client.

No, that's a temporary file. The crucial point is where the server  
finally marks the message as accepted in its spool, i.e. where it  
logically where it to its verbatim spool. Before that point, it can  
abort and delete the temporary file.

In other scenarios, this point is:
. the message is written to the mailbox of the recipient (for example,
  it renames $MAILDIR/tmp/$MESSAGE to $MAILDIR/new/$MESSAGE).
. the server sends the final "\r\n.\r\n" as a client to the next SMTP
  server.

> trying to close() the connection.

That's a valid objection.

> Only by sending "250 OK" will the TCP state machine in the SMTP server
> start looking for any response from the client.

However, it might be possible to check for a connection abort and to  
abort the handling if that happens.

Claus

---

From: [email-masked] (Claus Färber)
Date: 10 Apr 2003 13:48:00 +0200

>> to deliver the message if you sent the "250 OK".  So lost messages
                                                       ~~~~~~

> later.

Maybe you should read what I've written: This situation can't lead to  
_lost_ messages.

> This is nothing to do with broken servers.

Only when the server (or client) is broken and does make wrong  
assumptions whether the other side feels responsible for the message can  
message loss occur.

Claus

---

From: J C Lawrence <[email-masked]>
Date: Thu, 10 Apr 2003 10:11:47 -0700

On 10 Apr 2003 13:48:00 +0200 

> abort the handling if that happens.

There's a race condition there.  Did the abort happen before or after
the far end received the 2250?  This is the core problem in protocol
design, and it is insolvable.  There is no way to establish perfect
knowledge of each other's state between two systems via messages over a
non-instantaneous link.  There are many things you can do to cut the
window of risk down, but it can't be zero.

---

From: J C Lawrence <[email-masked]>
Date: Thu, 10 Apr 2003 10:13:10 -0700

> Maybe you should read what I've written: This situation can't lead to

Agreed, my bad.  Apologies.  You can fault safe or you can fault unsafe.
The current design faults safe and thus allows dupes.

---

From: Kee Hinckley <[email-masked]>
Date: Thu, 10 Apr 2003 16:34:27 -0400

>their ears while I ask)  how many MTAs (besides Qmail with "Return-Path"

All MTAs are supposed to write Return-Path before the final delivery. 
CommuniGate and Sendmail certainly do.  I was unable to find anything 
in the specs indicating what should happen if there was already a 
Return-Path there (forgery attempt?), but it seems to me that the MTA 
should remove the old one (or rename it).  I don't know what common 
practice is though.  The only mailer I've seen that doesn't generate 
Return-Path is Qualcomm's Mac MTA--and that may have changed in newer 
versions.

>many MUAs would know about it ?  The roblem with an end-user C/R system

I don't believe there's any standard that specifies what to do with a 
Return-Path if you are an MUA.  Certainly most aren't going to 
display it.  Eudora can if you tell it to (I have it turned on). 
Outlook probably can.  I don't think OE lets you change what headers 
get displayed.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society