Subject: Re: e2e

From: Fred Baker <[email-masked]>
Date: Tue, 14 Aug 2007 14:21:59 -0700

On Jul 26, 2007, at 8:47 PM, Hallam-Baker, Phillip wrote:
> I don't think that I am misrepresenting the paper when I summarize  
> it as saying 'keep the complexity out of the network core'

I'm slogging through some old email, and choose to pick up on this.

Following Noel's rant (which is well written and highly correct), it  
is not well summarized that way. For example, quoting from the paper,  
"Performing a function at a low level may be more efficient, if the  
function can be performed with a minimum perturbation of the  
machinery already included in the low-level subsystem". So, for  
example, while we generally want retransmissions to run end to end,  
in an 802.11 network there is a clear benefit that can be gained at  
low cost in the RTS/CTS and retransmission behaviors of that system.

My precis would be: "in deciding where functionality should be  
placed, do so in the simplest, cheapest, and most reliable manner  
when considered in the context of the entire network. That is usually  
close to the edge."

Let's take a very specific algorithm. In the IP Internet, we do  
routing - BGP, OSPF, etc ad nauseum. Routing, as anyone who has spent  
much time with it will confirm, can be complex and results in large  
amounts of state maintained in the core. There are alternatives to  
doing one's routing in the core; consider IEEE 802.5 Source Routing  
for an example that occurred (occurs?) in thankfully-limited scopes.  
We could broadcast DNS requests throughout the Internet with trace- 
route-and-record options and have the target system reply using the  
generated source route. Or not... Sometimes, there is a clear case  
for complexity in the network, and state.

Let me mention also a different consideration, related to business  
and operational impact. Various kinds of malware wander around the  
network. One can often identify them by the way that they find new  
targets to attack - they probe for them using ARP scans, address  
scans, and port scans. We have some fairly simple approaches to using  
this against them, such as configuring a tunnel to a honeypot on some  
subset of the addresses on each LAN in our network (a so-called "grey  
net"), or announcing the address or domain name of our honeypot in a  
web page that we expect to be harvested. Honeypots, null routes  
announced in BGP, remediation networks, and grey networks are all  
examples of intelligence in the network that is *not* in the laptop  
it is protecting.

The end to end arguments that I am familiar with argue not for knee- 
jerk design-by-rote, but for the use of the mind. One wants to design  
systems that are relatively simple both to understand and to  
maintain, and to isolate for diagnosis. The arguments do not say that  
leaving all intelligence and functionality in the end system is the  
one true religion; they observe, however, that the trade-offs in the  
general case do lead one in that direction as a first intuition.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Lakshminath Dondeti <[email-masked]>
Date: Tue, 14 Aug 2007 22:59:35 -0700

I guess I'll jump in as well.  I was reading some of the related papers 
recently for a different reason including the ones on active networks 
(thank gods they are history) and whether that concept is in line with 
the e2e philosophy.

In any event, exploring one of your examples with the concepts in the 
paper in mind (perhaps I am using a verbatim application of the 
concepts) that the network may filter some (and that being the keyword) 
malware or suspicious traffic based on certain parameters is fine; but 
the point is that in the end, an application may have to determine what 
it accepts as legitimate traffic based on its own criteria.  Email junk 
filtering comes to mind as an example.  Trying to map that to one of the 
statements from the paper: "For the data communication system to go out 
of its way to be an extraordinary filter does not reduce the burden on 
the application program to filter as well."  In some sense it does 
reduce it, i.e., for most apps or users, the functionality provided by 
the network may be sufficient, but we get the idea.  Entities in the 
data communication system :), say the mail servers, do some filtering, 
but different email applications utilize different techniques to get the 
job done and some are adaptive based on user input etc.  I know there 
are efforts to do more and more in the mail servers, but the email 
applications are also getting more sophisticated over time.

Your point is well taken of course that there are no cut and dried rules.

For instance, I am not fully in tune with the arguments on security 
(secure data encapsulation to be more precise) in the paper.  The paper 
says that "data will be in the clear and thus vulnerable as it passes 
into the target node and is fanned out to the
target application. Third, the authenticity of the message must still be 
checked by the application."

That goes to the extent of saying that end-node to end-node protection 
is not sufficient and that the data must really protected all the way at 
the application layer.  I might in other contexts make the argument for 
security properties that do belong in the application layer (non 
repudiation comes to mind for instance), but there are security 
properties that we'd get through network layer security that we might 
not really get through application layer security.  I am also not sure I 
understand the thing about the authenticity of the message having to be 
checked by the application (do they mean that the data is vulnerable and 
that's why?).  I am also curious if some of this has to do with 
multi-user systems being popular back then.

Now that sounds like a rant ;).

regards,
Lakshminath

On 8/14/2007 2:21 PM, Fred Baker wrote:
>> I don't think that I am misrepresenting the paper when I summarize it 
>> as saying 'keep the complexity out of the network core'
> 
> Following Noel's rant (which is well written and highly correct), it is 
> not well summarized that way. For example, quoting from the paper, 
> function can be performed with a minimum perturbation of the machinery 
> already included in the low-level subsystem". So, for example, while we 
> generally want retransmissions to run end to end, in an 802.11 network 
> there is a clear benefit that can be gained at low cost in the RTS/CTS 
> and retransmission behaviors of that system.
> 
> My precis would be: "in deciding where functionality should be placed, 
> do so in the simplest, cheapest, and most reliable manner when 
> considered in the context of the entire network. That is usually close 
> to the edge."
> 
> Let's take a very specific algorithm. In the IP Internet, we do routing 
> - BGP, OSPF, etc ad nauseum. Routing, as anyone who has spent much time 
> with it will confirm, can be complex and results in large amounts of 
> state maintained in the core. There are alternatives to doing one's 
> routing in the core; consider IEEE 802.5 Source Routing for an example 
> that occurred (occurs?) in thankfully-limited scopes. We could broadcast 
> DNS requests throughout the Internet with trace-route-and-record options 
> and have the target system reply using the generated source route. Or 
> not... Sometimes, there is a clear case for complexity in the network, 
> and state.
> 
> Let me mention also a different consideration, related to business and 
> operational impact. Various kinds of malware wander around the network. 
> One can often identify them by the way that they find new targets to 
> attack - they probe for them using ARP scans, address scans, and port 
> scans. We have some fairly simple approaches to using this against them, 
> such as configuring a tunnel to a honeypot on some subset of the 
> addresses on each LAN in our network (a so-called "grey net"), or 
> announcing the address or domain name of our honeypot in a web page that 
> we expect to be harvested. Honeypots, null routes announced in BGP, 
> remediation networks, and grey networks are all examples of intelligence 
> in the network that is *not* in the laptop it is protecting.
> 
> The end to end arguments that I am familiar with argue not for knee-jerk 
> design-by-rote, but for the use of the mind. One wants to design systems 
> that are relatively simple both to understand and to maintain, and to 
> isolate for diagnosis. The arguments do not say that leaving all 
> intelligence and functionality in the end system is the one true 
> religion; they observe, however, that the trade-offs in the general case 
> do lead one in that direction as a first intuition.
>

---

From: Fred Baker <[email-masked]>
Date: Wed, 15 Aug 2007 12:16:20 -0700

On Aug 14, 2007, at 10:59 PM, Lakshminath Dondeti wrote:

> In any event, exploring one of your examples with the concepts in  
> the paper in mind (perhaps I am using a verbatim application of the  
> concepts) that the network may filter some (and that being the  
> keyword) malware or suspicious traffic based on certain parameters  
> is fine; but the point is that in the end, an application may have  
> to determine what it accepts as legitimate traffic based on its own  
> criteria.  Email junk filtering comes to mind as an example.   
> Trying to map that to one of the statements from the paper: "For  
> the data communication system to go out of its way to be an  
> extraordinary filter does not reduce the burden on the application  
> program to filter as well."  In some sense it does reduce it, i.e.,  
> for most apps or users, the functionality provided by the network  
> may be sufficient, but we get the idea.  Entities in the data  
> communication system :), say the mail servers, do some filtering,  
> but different email applications utilize different techniques to  
> get the job done and some are adaptive based on user input etc.  I  
> know there are efforts to do more and more in the mail servers, but  
> the email applications are also getting more sophisticated over time.

in that context, here's one that one could use to dramatically reduce  
spam intake.

There are companies that sell reputation services for things that  
send email. At one point I looked at the gigahunks of email stored on  
my laptop, and found that there were less than 700 predecessors to  
the first Cisco email hop (less than 700 systems outside Cisco that  
sent email to to [email-masked]) in my non-junk inboxes for 2003- 
present, but in the 30 days prior to my checking there were nearly  
5000 predecessors represented in my junk box. The overlap? 25 systems  
- likely from misclassified messages.

That suggests a simple approach - in one's firewall, null route the  
addresses reported by the reputation service as spam spews. It's a  
network layer solution to an application layer problem, yes, and it  
has all of the issues that reputation services have, and btw, you  
still want to run spamassasin-or-whatever on what comes through.  
Cisco IT tells me that it results in a dramatic reduction in spam,  
however, and saves them serious numbers of monetary units.

The communication system isn't being a filter, properly speaking - it  
is simply routing some traffic to black holes using standard routing  
technology. And it doesn't relieve the application of the burden of  
filtering. But it can help reduce the volume of crapola at the  
application.

---

From: Keith Moore <[email-masked]>
Date: Wed, 15 Aug 2007 16:38:57 -0400

> filtering. But it can help reduce the volume of crapola at the
...at the cost of dropping legitimate traffic.  the thing is, the set of
valid senders for you and the set of valid senders for everyone at cisco
is very different, and the latter set is much fuzzier.  and those
reputation services won't take responsibility for the mail that you lose
by trusting them, nor are they accountable to the senders either.

this is  not a way to make the network more robust.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 15 Aug 2007 13:44:09 -0700

> this is  not a way to make the network more robust.
Robust for what? Spammers? The simple fact of the matter is that the
alternative is to just shut down port 25 given the growth in both volume
and complexity to filter.  That ain't robust either. Dealing with false
positives is the cost of doing business on the internet these days. Welcome
to reality.

       Mike

---

From: Keith Moore <[email-masked]>
Date: Wed, 15 Aug 2007 16:45:56 -0400

> positives is the cost of doing business on the internet these days.
people who cite "reality" generally do so because they lack
justification for their statements.

find a way to make reputation servers accountable to both sender and
recipient, then we can talk about robustness.  until then, they're just
another form of DoS attack.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 15 Aug 2007 13:53:10 -0700

> justification for their statements.

The thing is, Keith, I don't lack justification. I've seen the numbers with
my own eyes in our own largish organization with an IT staff that's
super paranoid about about false positives (free clue: guess who some of
our customers undoubtedly are?)
> another form of DoS attack.

Attacking people in the field as "DoS attackers" who are just trying to
get by given the escalating onslaught is... something. In that Cluepon
short of a Happy Meal kind of way.

       Mike

---

From: Douglas Otis <[email-masked]>
Date: Wed, 15 Aug 2007 13:56:12 -0700

On Aug 15, 2007, at 12:16 PM, Fred Baker wrote:

> in that context, here's one that one could use to dramatically  
> reduce spam intake.
>
> The communication system isn't being a filter, properly speaking -  
> it is simply routing some traffic to black holes using standard  
> routing technology. And it doesn't relieve the application of the  
> burden of filtering. But it can help reduce the volume of crapola  
> at the application.

The early version of the "Realtime Black-hole list" published as a  
BGP feed worked in the same manner.  The current owners recognized  
the need to reduce the burden on an email filtering. : )

-Doug

---

From: Keith Moore <[email-masked]>
Date: Wed, 15 Aug 2007 17:06:05 -0400

> our customers undoubtedly are?)
and I've had more than my share of legitimate mail fail to be delivered
(in either direction) because of such measures.
you may consider that legitimate for your or cisco's purposes. whether
to throw away mail that can potentially be from customers is a business
decision that cisco can make.  that doesn't mean it's a reliable way to
run a network.

that and trusting current statistics to make decisions on future
architecture is always walking on thin ice.
> get by given the escalating onslaught is... something. 
what's that saying about the road to hell being paved by people with
good intentions?
I don't fault the intentions, I fault the logic.
as far as I can tell spam filters are about as much of a DoS attack on
email as spam is .

Keith

---

From: Douglas Otis <[email-masked]>
Date: Wed, 15 Aug 2007 14:59:56 -0700

On Aug 15, 2007, at 2:06 PM, Keith Moore wrote:

> and I've had more than my share of legitimate mail fail to be  
> delivered (in either direction) because of such measures. you may  
> consider that legitimate for your or cisco's purposes. whether to  
> throw away mail that can potentially be from customers is a  
> business decision that cisco can make.  that doesn't mean it's a  
> reliable way to run a network.

Keith,

More agree with you than you might expect.  To make email more  
robust, email delivery can change into being nothing more than a  
signaling protocol.  Instead of messages, the entire body of the  
message is replaced by an encoded link.

 From [email-masked] is derived from a link in the form of:

  https://<example.com>/<~local-part>/<double hash of content and time>

The double hash coupled with 404 error rate-limiting offers security,  
where web logs would be used to indicate whether a recipient received  
a message.  The SMTP server can accept all messages and not be  
concerned about DSNs that are no longer needed or used.  There would  
not be any cryptographic signatures of the message or path  
registrations to place a burden upon recipients when deciding where  
the message originated.  Security could be further enhanced by  
utilizing some version of Open-ID.

All messages can be accepted without costing much in the way of  
storage and without creating concerns about message disposition.  The  
responsibility of what gets tossed is directly controlled by the  
recipient.  The recipient can decide by using whatever reputation  
system they wish.  The integrity of the delivery system would depend  
upon feedback given the sender by the server publishing their  
messages.  For the transition, existing clients might depend upon the  
recipient to click on a message link, or to apply some automated  
reputation/selection process plugin.

The growing lack of DSNs would drive the remaining portion of email  
users into depending upon message links and message publishing rather  
than using SMTP to send whole messages.  Perhaps the current  
situation provides enough motivation to change.

-Doug

---

From: Tony Finch <[email-masked]>
Date: Wed, 15 Aug 2007 23:02:02 +0100

> (in either direction) because of such measures.

Of course, all reputation services are equally incompetent.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
IRISH SEA: SOUTHERLY, BACKING NORTHEASTERLY FOR A TIME, 3 OR 4. SLIGHT OR
MODERATE. SHOWERS. MODERATE OR GOOD, OCCASIONALLY POOR.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 16 Aug 2007 00:11:13 +0200

> positives is the cost of doing business on the internet these days.  

That means giving up on email because it's not useful if it's not  
very close to universal.

If only there were a standards organization that could design a  
messaging protocol that can work across the internet...

---

From: Keith Moore <[email-masked]>
Date: Wed, 15 Aug 2007 23:23:24 -0400

> Of course, all reputation services are equally incompetent.
without accountability, there's no good way to tell.

that, and incompetence is not the only problem.  malice is another one.

---

From: Greg Skinner <[email-masked]>
Date: Thu, 16 Aug 2007 03:36:08 +0000

> positives is the cost of doing business on the internet these days. Welcome

http://fm.vix.com/internet/security/superbugs.html

--gregbo

---

From: <[email-masked]>
Date: Thu, 16 Aug 2007 14:54:25 +0100

> Robust for what? Spammers? The simple fact of the matter is 
> that the alternative is to just shut down port 25 given the 
> growth in both volume and complexity to filter.

Finally, a sensible solution!

After all, why should any mail server operator accept incoming email
from another server without a prior mail peering agreement in place? The
email architecture of Simple Mail Transfer Protocol is completely broken
and has been largely superceded. Why not begin to turn off services on
the problem port? 

RFC 4409 split off the submission stream from the relay stream,
therefore one can reasonably assume that most of the non-spam port 25
traffic is relay. Some of that is customers of an ISP, who therefore
have an implicit or explicit mail peering agreement, relaying through
the ISP's mailserver. And some of it is anonymous relays delivering
messages to what appears to be the final destination.

If we get rid of the anonymity for relays delivering to final
destination, i.e. gmail.com sending a message for [email-masked] to an
aol.com mailserver, then most of the spam stream goes away. At that
point, we only have to worry about spam which subverts the submission
mechanism or some other weakness in the system. 

Anonymous relaying on port 25 is a known weakness. Why can't we close
this gap? Do we need a new port assignment for mail peer relaying? Do we
need to fix the SMTP-AUTH weaknesses? Do we need an upgrade to ESMTP
that is incompatible with previous SMTP versions to make a clear break
with the past?

Is there an informational RFC that describes the overall email
architecture that is provided by the various IETF protocols? Such a
document could help frame the discussion and identify weaknesses that
need further work such as the SPF/DKIM bandaids. We have gotten to the
present situation because various people doing work on the problem have
blinders on as that focus on one narrow area of the problem. We need to
describe the totality of the situation, point out what works fine or has
already been fixed, and suggest a way forward for fixing the problems. 

By the way, SPAM is not the problem. The problem is anonymity and lack
of accountability. I believe that the email architecture can be fixed so
that unknown people can still email me anonymously, and I won't be able
to identify them without cooperation from my ISP, but at the same time,
if these people are abusing the system, my ISP can follow a chain of
accountability to the sender, or to the point in the chain where
bilateral trust relationships have broken. In such a scenario, I am
confident that SPAM will still exist, but I am also confident that it
will no longer be such a firehose, just a dribble. SPAM is not a problem
in the email architecture. Lack of a solid chain of accountability
formed with bilateral trust agreements is the root problem with the
email architecture.

--Michael Dillon

---

From: Gunnar Lindberg <[email-masked]>
Date: Thu, 16 Aug 2007 16:44:28 +0200 (MEST)

In theory that isn't much of a problem. In fact you could even keep
SMTP for the transport and RFC822 for the message format.

Only accept incomming SMTP from hosts/networks you trust and/or have
some kind of formal agreement with. X.400 did roughly (exactly) that,
although it used a different transport. Ever heard of X.400 Spam :-)?

In practice it's of course less simple, since you will miss the baby
that went out with the water. And, noone will be prepared to pay the
cost of sending mail through such a regulated system. As is, we pay
via the time spent with the <Delete> button. Sorry, no free lunch.

	Gunnar Lindberg

>From [email-masked]  Thu Aug 16 00:13:07 2007
>In-Reply-To: <[email-masked]>
>References: <[email-masked]>	<[email-masked]>	<[email-masked]>	<[email-masked]>
>	<[email-masked]> <[email-masked]>
>Message-Id: <[email-masked]>
>From: Iljitsch van Beijnum <[email-masked]>
>Date: Thu, 16 Aug 2007 00:11:13 +0200
>To: Michael Thomas <[email-masked]>
>Cc: John Kristoff <[email-masked]>, Keith Moore <[email-masked]>,
>        Fred Baker <[email-masked]>, [email-masked]
>Subject: Re: e2e

---

From: Keith Moore <[email-masked]>
Date: Thu, 16 Aug 2007 10:56:14 -0400

[email-masked] wrote:
> from another server without a prior mail peering agreement in place?
in other words, why would you ever want to communicate with someone with
whom you had no prior arrangement?

just stick your head in a hole in the ground and be done with it.

Keith

---

From: Ted Hardie <[email-masked]>
Date: Thu, 16 Aug 2007 09:28:02 -0700

>just stick your head in a hole in the ground and be done with it.

As both of you know and understand, the email system was built to be an
any-to-any mesh.  That's not just a design goal.  Folks have operated a lot of
gateways, written a lot of 8-to-7 code, and jumped through a lot of hoops to make
sure that the network effect of email was a great as possible:  that there were as
few people left out of the email system as possible.

Spammers work with that design.  They send from anywhere (zombie hosts, poorly
policed webmail providers, domains and accounts that exist for only the briefest practical times) to anyone (dictionary attack-derived names, scraped addresses, etc). 

Many of the efforts to thwart spam work against the design of email,
and, as a result they tend to be both deeply painful and to have long term
effects long after their effectiveness at fighting spam is gone. 

There are messaging systems that do not have email's core design. 
Many IM systems, for example, don't assume you can send
a message to anyone; you have to be permitted first.  The problem is that
you need a way to communicate about the permission that doesn't use
that messaging:  you have to see the person, call them up, or send them email
to say "hey, I'm [email-masked], add me to your list".  You need an
any-to-any mesh *other* than the permission-based system, in other words,
for the permission-based system to avoid having a very limited network effect.
(Of course, if you wanted a limited network, that's not an issue).  You can move
the problem around, in other words, but as long as you retain that design goal,
you have the same problem.

I, for one, am not ready to abandon the design goal.  I think email was the first
"killer app" for the Internet, and it remains one of the most powerful initial
applications as Internet applications are made available in new arenas.  RIM's
success, for example, seems to me clearly due to bringing any-to-any email
to wireless users in ways others did not.  That does not mean that individuals
or organizations should not be able to reject email wholesale if they want; I
have recommended whitelist based systems for my nephews and nieces, to give
one trivial example.  But we should not change the core design; we need it.
It's really one of the most fundamental demonstrations of what the Internet
can do.
		best regards,
				Ted Hardie

---

From: Tony Finch <[email-masked]>
Date: Thu, 16 Aug 2007 18:07:17 +0100

On Thu, 16 Aug 2007, Gunnar Lindberg wrote:
> some kind of formal agreement with.

The difference between a whitelist model and a blacklist model is just
toggling a billion bits.

Tony.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 16 Aug 2007 11:20:51 -0700

On Aug 16, 2007, at 6:54 AM, <[email-masked]>  
<[email-masked]> wrote:

> Such a document could help frame the discussion and identify  
> weaknesses that need further work such as the SPF/DKIM bandaids. We  
> have gotten to the present situation because various people doing  
> work on the problem have blinders on as that focus on one narrow  
> area of the problem. We need to describe the totality of the  
> situation, point out what works fine or has already been fixed, and  
> suggest a way forward for fixing the problems.
>
> By the way, SPAM is not the problem. The problem is anonymity and  
> lack of accountability.

This should be stated just a bit differently.

Anonymity of SMTP clients and lack of accountability for SMTP  
client's access vetting is a major part of the problem.  CSV  
extensions would have made a difference and would have laid a  
foundation suitable for an IPv6 era.

Instead, SPF places DNS infrastructure in peril without identifying  
who administers the SMTP client.  This inability to authorize SMTP  
clients by name is not an oversight.  Path registration would have  
been safe using the SMTP client's name instead of expecting DNS to  
return _all_ IP addresses of _all_ systems operating on behalf of a  
domain.  Even with SPF, the receiver can't be sure whether an  
authorization is for a bounce address or some purported originator.   
To ensure the proprietary purported originator algorithm works,  
mailing lists and forwarders must now identify themselves as a  
purported originator using this algorithm.

DKIM, although extremely useful for filtering phish, avoids a means  
to authorize SMTP clients while insisting replay abuse is not a  
concern.  Again, this inability to authorize the SMTP client is not  
an oversight.  After all, SMTP is a store and forward protocol, and  
path registration impinges upon freedoms SMTP provides, right?  Of  
course, no provider wants to be identified to then deal with complaints.

Many providers filter messages prior to hitting a mailbox.  When not  
accepted, the message might be bounced as a DSN.  When not bounced, a  
suspect message is easily lost.  A sizable portion of spam is a  
spoofed bounce which just increases the chances a valid DSNs will be  
lost.  Here BATV might help, but only when domains are restricted to  
specific outbound servers.  A sizable portion of the spam is from  
large providers.  Their mail normally represents a mixture of good  
and bad.

Can accountability ever be forced upon 900 pound gorillas, or even an  
organization of 900 pound gorillas?  No.  Accountability will simply  
never happen, and gorilla solutions are likely to make the problem  
much worse.

SMTP needs to change how delivery integrity is maintained.  The  
delivery of a message needs to be controlled by the individual, and  
made separate from SMTP.  SMTP needs to change into offering a  
reference for the retrieval of messages.  Such a strategy ensures  
originating domain spoofing is less likely without burdening the  
recipient.  Not even a subject line should accompany the link  
comprised of a local-part and a double hash of the message and time- 
stamp.  Publishing could use HTTP/HTTPS and a back-end supported by a  
Lustre network FS, for example.  A message publishing system could  
provide notifications when a message is not fetched after some  
interval.  DSNs would not be relied upon, nor are DSNs currently that  
reliable anyway.

Of course this would not be practical or empowering without also  
offering individuals some tools.  Initially, the vetting could be  
done on behalf of individuals where their email could be pre-fetched  
and filtered.   However, Individuals will also be able to directly  
deal with the link vetting process themselves.  Individuals should  
have many choices in how their incoming email links are vetted.  Of  
course, the vetting process would depend upon the reputation of the  
publisher.

Any filtering that might happen after the message is fetched is  
responsibility of the individual or organization acting on behalf of  
the individual.  When a message contains malware, the individual or  
organization should report this to the authorities, but the bouncing  
of these message will have been made obsolete.

-Doug

---

From: SM <[email-masked]>
Date: Thu, 16 Aug 2007 12:30:47 -0700

>from another server without a prior mail peering agreement in place? The

Because that is how SMTP works.

>and has been largely superceded. Why not begin to turn off services on

We can if we don't want to receive any mail.

>mechanism or some other weakness in the system.

That would be SPF.

It would also restrict you to using a Return-path which is valid for 
the network from which you are sending the mail.  That may not be 
practical in some environments where you don't have an existing 
account and you can only send mail through the local mail server (no 
access to submission port on home server, e.g. outgoing TCP 587 
blocked or network unreachable).

>that is incompatible with previous SMTP versions to make a clear break

We could require authentication on port 25.  This would mean having 
prior agreement before exchanging mail between servers.  That was not 
how SMTP was designed to be and it would mean a clear break from the past.

A start would be restricting mail exchanges to Hotmail, Yahoo and 
Gmail.  That may sound acceptable to some people.  I am not 
suggesting doing that.

>architecture that is provided by the various IETF protocols? Such a

http://tools.ietf.org/html/draft-crocker-email-arch-09

>already been fixed, and suggest a way forward for fixing the problems.

See http://www.irtf.org/charter?gtype=rg&group=asrg

>formed with bilateral trust agreements is the root problem with the

Bilateral agreements mean a complete change of the email 
landscape.  Even then, I don't see how a small provider can afford 
blocking a large ESP.  Sure, we can make part of the spam problem go 
away temporarily by giving up some of the functionality email 
provides.  Most people may be willing to live with the restrictions 
until they experience the drawbacks first-hand.

Regards,
-sm

---

From: Douglas Otis <[email-masked]>
Date: Thu, 16 Aug 2007 14:36:31 -0700

On Aug 16, 2007, at 12:30 PM, SM wrote:

>> destination, i.e. gmail.com sending a message for [email-masked]  
>> to an aol.com mailserver, then most of the spam stream goes away.  
>> At that point, we only have to worry about spam which subverts the  
>> submission mechanism or some other weakness in the system.
>
> That would be SPF.

SPF will not restrict the Return-path!  Any SMTP client sending a  
message with a bounce address of <[email-masked]> will _not_ be  
considered unauthorized.  Regardless of where the SMTP client is  
located, <[email-masked]> will achieve the highest level of  
authorization.  White-listing is the only reasonable application for  
SPF, which is why '+all' is a good thing.   SPF is prone to shared  
servers, a multitude of authorization levels, and an uncertainty  
whether  authorization is intended for the PRA or Return-Path.

The resources of recipients who attempt to process SPF record-sets  
can be easily exploited.  SPF might be instrumental in staging DDoS  
attacks or in poisoning DNS, despite the use of BCP38 and ACLs on  
recursive resolvers.  By utilizing local-part macros, cached SPF  
records can issue tens or hundreds of new DNS transactions directed  
toward any victim domain's A, AAAA, or MX records when referenced by  
a message that could easily be part of a large spam campaign.  Curt  
timeouts in processing SPF scripts will circumvent congestion  
avoidance.  The attack will emerge from recipient's resolvers, and  
SMTP logs will be unlikely to offer a clue how it was achieved.  The  
bots will remain cloaked, and their full resources can remain  
dedicated to spamming while DoSing the heck out of their opposition.

> It would also restrict you to using a Return-path which is valid  
> for the network from which you are sending the mail.  That may not  
> be practical in some environments where you don't have an existing  
> account and you can only send mail through the local mail server  
> (no access to submission port on home server, e.g. outgoing TCP 587  

In addition, SPF may cause forwarded and mailing-list messages to be  
lost despite gaining access to secure submission.  SPF is an example  
of what not to do when confronting spam.  Don't create scripts that  
might be utilized by bad actors.  Those advocating SPF need to better  
consider the risks.  The benefits, assuming there are benefits, do  
not outweigh sizable risks to critical infrastructure.

-Doug

---

From: <[email-masked]>
Date: Thu, 16 Aug 2007 22:51:11 +0100

> As both of you know and understand, the email system was 
> built to be an any-to-any mesh.  That's not just a design 
> goal.  Folks have operated a lot of gateways, written a lot 
> of 8-to-7 code, and jumped through a lot of hoops to make 
> sure that the network effect of email was a great as 
> possible:  that there were as few people left out of the 
> email system as possible.

Funny thing is, that about 15 years ago, email was not an any-to-any
mesh. I recall several occasions where I advised people to relay email
off of aol.com or some other "centrally located" mail server to get
around the 30 hop-count limit. One case that I remember was a professor
in Peru who could not get email through to a colleage in Ukraine. In
Peru, he used UUCP relaying to get to an ISP in Lima. In the Ukraine, I
believe the only international connectivity was a 56k frame relay line
into Poland. And people didn't understand IP network design as well in
those days so unneccesarily high hop counts were common.

But at the same time that the mesh was not truly any-to-any from the
end-user perspective, it really was possible for any sender to send a
message to any receiver without making prior arrangement. I am not
suggesting that this be changed. What I am suggesting is that we revive
the old principle of relaying which worked so well 15 years ago, but in
an updated form where ONLY THE EMAIL RELAYING is done with prior
arrangement. When SPAM grew by leaps and bounds, organizations like
uu.net and aol.com turned off email relaying except for paying customers
(prior arrangement). I would like to see these type of organizations
extend that relaying to the core of the email mesh by making email
peering agreements.

In such a world, I would still be able to send a message to you anytime
I like without phoning you first. I would not need to know the path that
the email takes to get to you. You would not need to be registered in
any directory service. However, my email would not go directly to your
domain's mail server. It would go to the ISP or mail service operator
with which I have prior agreement, using the submit protocol. If they
have email peering with you, then they would send it directly to your
domain's mail server. If not, they will likely have mail peering with
some more central service, such as a larger ISP who will relay to you,
an ISP consortium that operates a mail relay service with wide peering
agreements, or some kind of commercial email relaying service. 

If you are upset about my email you can track it back to me via the
relay headers which are now harder to forge. There are a number of ways
this could be accomplished such as adding some kind of hash number to
the header that is then retained for a few days in case of complaints.
There are still ways that rogues can inject messages into the system.
One is to arrange peering under false pretences. But this will not last
long as too many complaints track back to the rogue. The other is to use
the ISP's submit port but this requires signing up for an account. If an
ISP's processes allow for a large number of account activations using
forged credit cards, then they will appear to their mail peers as a
rogue provider, and that will be incentive to keep some control over
this. For instance rate limiting email for new customers kind of like
the limited driving licences that some jurisdictions issue to under-21s.

A lot of this is vague and generalized because it is only a concept at
this point. But I hope that I have shown that a more structured email
architecture with a relay hierarchy built into it could still provide
much of the same capability that was intended by the original flat
any-to-any mesh concept.

I also expect that when this architecture is defined, whether it is done
inside or outside the IETF, it will also provide some hooks/protocols to
formally give reputation and filtering services a place in the
architecture. As I said, a next generation and more robust email
architecture will not eliminate SPAM. But it can reduce the volume of
SPAM, and it can provide support so that mail receivers can implement
more effective filtering of email. 

> Many of the efforts to thwart spam work against the design of 
> email, and, as a result they tend to be both deeply painful 
> and to have long term effects long after their effectiveness 
> at fighting spam is gone. 

I agree. And that is because SPAM was not known when the present
architecture was designed and attempts to resolve the problem focus too
narrowly on immediate problems, not on the larger architectural issues.
When you travel through a modern city, you can tell which highrises are
office buildings and which ones are apartment dwellings or hotels.
That's because the architects design them differently. If you built only
office buildings and then tried to convert them to dwellings, or vice
versa, you would find it to be deeply painful and have long terms
effects as well. The same thing happens when you try to convert 300
year-old buildings into any modern use. In London they often completely
destroy and remove the building except for the shell of external walls.
The new building then ties into the old walls in such a way that you
would hardly know that the interior is entirel new construction unless
you were there to watch the building process. I think that we can do
this with email. Many of the shops on Regent Street have gone through
this process if you are curious to see the results.

> I, for one, am not ready to abandon the design goal.  I think 
> email was the first "killer app" for the Internet, and it 
> remains one of the most powerful initial applications as 
> Internet applications are made available in new arenas.  

I think you are wrong about that. Many new Internet users now prefer IM
over email even if IM's permission system seems onerous to you. And many
use their mobile phone's SMS and ignore the Internet entirely. And a
very large number of people prefer to use HTTP for their email services
(Yahoo, Gmail). 

> RIM's success, for example, seems to me clearly due to 
> bringing any-to-any email to wireless users in ways others 
> did not. 

Others had done this i.e. Palm had such an email device and any Palm
device could be used with a phone handset that supported IR to do email.
The issue was one of packaging and marketing. And essentially, you are
wrong. RIM did not bring Internet email to the masses. Instead it
connected the masses to their corporate email systems while they were
not at their desk. 

In a way, RIM highlights the point that I am trying to make. We need our
email services to be mediated by an intermediary who can provide support
services that we cannot. In an age where everyone has become the system
administrator of their own network host, it is not feasible for the flat
email architecture to work. That's why it has been chipped away into the
bizarre collection of hacks that we now have. We need an architecture
that recognizes the role of intermediaries to relay email, arrange email
peering, cut off rogue sites, offer the use of filtering and recognition
services, and so on.

If we don't want to see the Internet reduced to applications-over-http,
then we have to address the issue of architecture aging. Email is an
area where it has been left too long, probably because everybody
believes that the problem is SPAM and that since SPAM is a social
problem a solution is outside the scope of the IETF. I believe that the
problem is that email is no longer pervasive and Internet email is
increasingly viewed as quaint, archaic and even BROKEN BEYOND REPAIR. We
can fix this by refreshing the architecture, identifying what works
well, and targeting the areas where new or revised protocols are needed.

--Michael Dillon

---

From: <[email-masked]>
Date: Thu, 16 Aug 2007 23:01:34 +0100

> http://tools.ietf.org/html/draft-crocker-email-arch-09

This is nice but a draft is not an RFC. Hopefully it will progress and
can then become a starting point for a discussion about creating a more
workable architecture.

> See http://www.irtf.org/charter?gtype=rg&group=asrg

Great. A web page last updated in 2003 and an email archive averaging 4
messages a month this year.

> landscape.  Even then, I don't see how a small provider can 
> afford blocking a large ESP.

Now you are criticising an architecture document which does not yet
exist. 

> Sure, we can make part of the 
> spam problem go away temporarily by giving up some of the 
> functionality email provides.  Most people may be willing to 
> live with the restrictions until they experience the 
> drawbacks first-hand.

I live with these drawbacks every day. But I also remember how Internet
email was 15 years ago and I wonder if we can't stop the slide into
obscurity and regain the features that we have lost.

--Michael Dillon

---

From: Keith Moore <[email-masked]>
Date: Thu, 16 Aug 2007 18:10:28 -0400

[email-masked] wrote:
> What I am suggesting is that we revive
> the old principle of relaying which worked so well 15 years ago
ah yes, I remember those good old days, when every site needed to have
experts to tell ordinary users which routes through the Byzantine email
system were likely to work, and also how to write a source-routed
recipient address that would successfully get your mail there and get
the return address rewritten so that the message could be replied to. 
if you were one of those experts, it made you feel valuable.  :)

I do think there might be some merit in having designated outgoing mail
servers for any particular domain, but SPF records are a disaster, and
DKIM is only a very marginal improvement over nothing. 

I also think there might be some merit in holding mail on the sender's
outgoing mail server until the receiver's MX is willing to deal with it,
thus pushing more of the costs toward the sender of a message.  (as in,
we'll accept your mail immediately if it's accompanied by a signed
statement saying that it's been spam and virus validated by a trusted
filter, or if you can provide some assurance from a trusted third-party
that it's been sent from a recognized individual who can be held
accountable for sending malicious email or spam.  otherwise, we're going
to defer it for awhile - say until our servers are less busy, and we're
going to rate-limit the unvalidated mail we accept from your domain.  
in the meantime you can just queue it on your end.)

But there's no value added by third-party mail relaying that I can see. 
It just adds cost and decreases reliability.

---

From: SM <[email-masked]>
Date: Thu, 16 Aug 2007 16:40:55 -0700

>recursive resolvers.  By utilizing local-part macros, cached SPF

I'll keep out of the debate about the advantages and disadvantages of SPF.

>messages a month this year.

A read of the archive may show that while everyone agrees that there 
is a problem, it's difficult to find consensus as to the solution.

>Now you are criticising an architecture document which does not yet

That was not the intent.

Regards,
-sm

---

From: Douglas Otis <[email-masked]>
Date: Thu, 16 Aug 2007 23:45:51 -0700

On Aug 16, 2007, at 3:10 PM, Keith Moore wrote:

> I also think there might be some merit in holding mail on the  
> sender's outgoing mail server until the receiver's MX is willing to  
> deal with it, thus pushing more of the costs toward the sender of a  
> message.

The concept of holding messages at the sender could be a basis for a  
change in SMTP for supporting this model, but at a message  
granularity.  The changes would be similar to that of BURL but would  
be Transfer by Reference or To Be Read, TBR.

The results seen after returning 4xx errors to SMTP clients is often  
the client will then retry frequently.  A better solution would be to  
devise an architecture that allows the message reference to always be  
accepted, but where actual message delivery occurs over a different  
channel.  This new mode of operation would occur when both sender and  
receiver support an transfer by reference.  Often more than 90% of  
messages received are undesired.  These messages will be either  
refused, placed into a junk folder, bounced, or deleted.  Many  
bounces will be deleted as well.  Actually transferring the message  
is sub-optimal when done within the SMTP sessions due to extremely  
high levels of abuse.  Accepting the message rather than a message  
reference also burdens the SMTP server with the duty of retaining its  
disposition and possibly issuing bounces.

A problem also plaguing email is falsified originating addresses.   
This has lead to rampant abuse where virtually little within a  
message is trusted as being valid.  While DKIM helps solve this  
problem, this solution demands additional resources of the  
recipient.  Whether DKIM will be able to withstand abuse may be in  
doubt, after all most email services are gratis.  Providers might opt  
for a cheaper solution.  Transfer by reference ensures the  
origination of the message is not falsified at substantially less  
cost for the inbound SMTP process.

To implement an transfer by reference, the MSA would create two  
hashes based upon the email message content.  The MSA then publishes  
the email message at a web-server where the combined hashes provide  
an access reference for security.  Rate-limiting 404 errors makes  
attempts at guessing a link futile.  If there is a desire to ensure  
even those running the recipient's MTA do not have access the  
message, a scheme like Open-ID could be employed.  When there are  
multiple recipients, a script on the web-server could request who is  
accessing the message.  Identifying who is fetching the message can  
also be automated by inserting the recipient's address into the link  
as follows:

MSGREF: http://[email-masked]/~moore/.mq/(sha1+sha256)

The web-server would track when an outbound message was posted, and  
whether it had been accessed by all intended recipients.  The web- 
server could generate a report of outstanding recipients that failed  
to fetch the message after some interval.  The web-server may attempt  
to retry sending the link to the recipient, or just notify the sender  
of a delivery failure.

Banks or financial institutions could use https to better secure  
message content, and to ensure customers they are at the expected web- 
site.  Any new MUA designed to handle transfer by reference should be  
able to annotate whether a publisher had been listed within their  
address book.  This feature should help avoid look-alike exploits.

Fetch messages from the reference link offers a positive confirmation  
of delivery.  From a commerce standpoint, this confirmation would be  
of significant value.  Those concerned about keeping their IP address  
private, should utilize an external proxy before accessing the web- 
sites.

When an MTA encounters a down-stream MTA or user-agent that does not  
support transfer by reference, it could fetch the message to remain  
compliant.  The cost of handling messages will is likely to be a  
primary motivation for transfer by reference, along with secure  
content, and a solid confirmation of delivery.

-Doug

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 17 Aug 2007 11:51:16 +0200

> blocked or network unreachable).

So what you're saying is that any mail server should be able to  
source messages from any mail user using any email address in any  
domain.

Sorry, that's not a reasonable desire in my opinion. If you find your  
path to your home mail server blocked, find yourself better  
connectivity rather than opening a gaping hole in the architecture.

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Aug 2007 08:56:32 -0400

Iljitsch van Beijnum wrote:
> So what you're saying is that any mail server should be able to source
> messages from any mail user using any email address in any domain.
>
> Sorry, that's not a reasonable desire in my opinion. 
what's unreasonable is trying to use IP addresses as authentication
tokens, or expecting applications to know about network topology in
order to route messages.

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Aug 2007 09:27:02 -0400

(changing the subject line since this has clearly drifted off
into a more specialized discussion).

Doug,

I'm not convinced of the merits of the general idea you are
pushing here, mostly because I still believe in the value of
mail systems with relay (or, if you prefer, store-and-forward)
capability.  If one requires that one be online to receive
actual messages, with end-to-end connectivity to the sender's
mail holding store, then sites with no direct connectivity to
the Internet are out of luck and sides with intermittent
connections or very long delay connections (think about a lot of
developing areas on earth as well as the interplanetary problem)
are in bad trouble.  

An important side-effect of the current mail model is that, if
the final delivery server is network-close to the intended
recipient, it doesn't make much difference to the recipient
whether mail flows into that server at gigabit bandwidth or at a
few kilobits/second or worse or it things flow only at 0300
local time.  The only connections that are really
network-performance-sensitive are between the sending MUA and
the submission server and between the final delivery MTA or
message store and the mail-retrieving user.  We've used those
properties to very good advantage in the past and, I believe,
will continue to use them to good advantage.

I share Dave Crocker's concern about deployment of replacements
for the current email model (see the article in the recent issue
of Internet Protocol Journal), but maybe it isn't impossible if
the need is significant enough and the solution doesn't make
other things worse (which I think yours may, but I haven't seen
enough of a proposal to evaluate).

One could preserve the relaying by turning message transmission
and delivery into a multiple-stage handshake:
	* Message notifying recipient of availability
	* Message from recipient to store asking for
	   transmission of the real message
	* Transmission of the real message
but that would be a fairly complex piece of protocol work, with
attacks possible at each step.

I note that, while one would want to update its non-existent
security mechanisms before trying to use it today, the
capability of sending a message structured as

   [...]
   Content-type: multipart/???; boundary="foo"
   
   --foo
   Content-type: text/plain

   Hey, Doug sent a message
   --foo
   Content-type: message/external-body
   [...]

has existed since the very first MIME specification and has
never been significantly used.  Maybe there is a message (sic)
in that for you; maybe not.

But the main point of this message is that, if you are serious
about this, generate an I-D that specifies all of the details,
analyzes the possible attacks and how you will repel them, and
so on.  IMO, you have passed the point at which talking about
what you might propose is helpful.   If the ideas have any
utility or applicability (even if that applicability isn't as
general as you believe), we need a real proposal not more
IETF-list traffic.

Just my opinion.

   john

--On Thursday, 16 August, 2007 23:45 -0700 Douglas Otis
<[email-masked]> wrote:

>>  sender's outgoing mail server until the receiver's MX is
>> willing to   deal with it, thus pushing more of the costs
>> toward the sender of a   message.
> 
> The concept of holding messages at the sender could be a basis
> for a change in SMTP for supporting this model, but at a
> message granularity.  The changes would be similar to that of
> BURL but would be Transfer by Reference or To Be Read, TBR.
>...

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 17 Aug 2007 15:50:48 +0200

>> source

> tokens, or expecting applications to know about network topology in

I agree that this suboptimal (but given that simple email users such  
as myself have so few options my server does take the source IP  
address into consideration when filtering at this point in time).

What we need to filter on are (of course) the identities of the  
servers and the originators of messages. But the former requires that  
servers don't trust random originators, hence my message, and that  
there is some way to make sure an originator is who he or she claims  
to be that works a bit better than simply taking the From: line at  
face value.

The counter argument is that spammers will simply use disposable  
identities. However, that at least makes sure that if I get a message  
from [email-masked] I know that it's actually from [email-masked] so  
whitelisting works and I don't get bombarded with bounces when a  
spammer uses my domain, and if we make getting an identity take  
enough time or money, spammers won't be able to get them fast enough  
to spam much before they're blacklisted.

Then again, misspelled fishing would be an order of magnitude harder  
if banks and retailers started using S/MIME, which is widely  
implemented today, but they can't be bothered, so it looks like  
protocol design isn't going to save the world any time soon.

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Aug 2007 10:09:03 -0400

--On Friday, 17 August, 2007 15:50 +0200 Iljitsch van Beijnum
<[email-masked]> wrote:

>...
> Then again, misspelled fishing would be an order of magnitude
> harder if banks and retailers started using S/MIME, which is
> widely implemented today, but they can't be bothered, so it
> looks like protocol design isn't going to save the world any
> time soon.

This is, IMO, an important point.   It seems to be easy
(relatively) to get large public providers of nearly-free email
to try new, and fairly weak, ideas like SPF or DKIM.
Individuals with small mail domains struggle along, resisting
being forced to either give up and join up with those large
providers and ideas that would inevitably make email costly to
them on a per-message basis.   But the key institutions that get
spoofed don't, in practice, seem to care.  S/MIME would work,
PGP would work, and so would any other reasonable method to
validate message source and integrity that depends on the
message and not the transport.  If the primary concern is
communications between a financial institution with which the
user already has an account (or equivalent relationship) and
that user, we don't even have the usual PKI problems: one can
deliver a sender key or cert out of band, validate it, and be
finished.

Not only will those institutions not bother with S/MIME or PGP,
but many of them won't support subaddresses (many reject
addresses containing "+", "/", or "=" as invalid, some even
reject "-").  But, while it gets no where near real
authentication, the ability to write a pair of rules that say 

	* if the message comes to "[email-masked]", and
	isn't from an address in the "bigbank.com" domain, it is
	trash and can be discarded, and
	
	* if the message appears to come from the "bigbank.com"
	and isn't addressed to [email-masked]", then it
	is trash and can be discarded.

turns out to be a powerful tool that is not easily defeated and
that does not require multiple handshakes between recipient and
putative sender.  But, if the financial institutions won't
support it and insist that email local parts consist only of
ASCII letters and digits, then its usefulness is limited.

That should, I think, make some predictions about the deployment
and effectiveness of anything really new and effective.  As with
certain types of credit card fraud, it appears to be cheaper for
the financial institutions to build the costs into their fee
structure and then just eat the losses, rather than making
significant investments in better systems or more inconveniences
that might drive customers away.

It is possible to infer from this that there just isn't enough
spam and phishing out there yet to be considered a problem --a
problem that needs to be solved, rather than one about which one
needs to make public statements and pass laws that are either
meaningless or not enforced-- by those who make and enforce laws
and policies.

     john

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Fri, 17 Aug 2007 11:54:45 -0400

On Fri, 17 Aug 2007 15:50:48 +0200
Iljitsch van Beijnum <[email-masked]> wrote:

> implemented today, 

S/MIME would be a fine start.  It also won't solve the problem until
someone develops a user interface that DTRT for naive users who don't
understand trust anchors, the difference between authentication and
authorization, or how to distinguish myfinancialcompany.com from
myfinancia1company.com when both have valid certificates.

		--Steve Bellovin, http://www.cs.columbia.edu/~smb

---

From: Keith Moore <[email-masked]>
Date: Fri, 17 Aug 2007 12:09:04 -0400

> , while it gets no where near real
> putative sender.
I've been using it for years, and entirely agree.

Keith

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 17 Aug 2007 13:10:41 -0400

The problem is incentive alignment. For example, for CNP (card not  
present) fraud, the merchant eats the loss, so the credit card  
company has limited incentive to make the system more secure. After  
all, they still get their cut even on charge-backs.

Same problem here: everybody might be better off with a more secure  
system, but the benefits don't occur until almost everyone uses the  
system, so nobody has an incentive to go first.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 17 Aug 2007 10:13:04 -0700

> developing areas on earth as well as the interplanetary problem)

The option will need to be negotiated at every stage.  It should be
rather easy to translate "transfer by reference" into the current
behavior at any point within the SMTP store-and-forward chain.  When the
MTA confronts a down stream MTA or MDA where the user is without access
to http, or not yet upgraded, then at this point the message would be
fetched.  Delivery reliability from that point forward needs to be high.
One might assume messages will have been vetted by publisher reputation
prior to fetching and that the message is near the last stage.

> will continue to use them to good advantage.

As have I.  The MDA could easily offer both modes to the user agent,
assuming the MDA has been upgraded to support transfer by reference.

> enough of a proposal to evaluate).

Yes, I read the article.  Unlike Dave, I don't see SPF/Sender-ID
helping, but instead path registration by address creates serious
problems.  I agree with Keith, the use of IP addresses as authentication
tokens needs to change, especially when there is a desire to introduce
the use of IPv6 with email.

In addition, the trust model does not work when vetting prior to the
critical acceptance of responsibility for message delivery occurs at the
MSA.  This operation might be by millions of small and large
organizations operating at a flat rate, free of charge, or gratis.  The
bad actors have access to essentially all MSAs.  Reasonable vetting will
not happen at this stage, although we might all agree that it should.

The MSA can not be trusted.  Transfer by reference permits every stage
subsequent to the MSA to apply their own vetting.  The reference can not
be falsified.  In most cases, less that 10% of the message will end up
being transferred.  A co-worker sent me his latest stats.  His MTA
currently finds 1% of the message received are not spam, where the
overall volume of spam has increased by more than a order of magnitude
in just the last year.  He is also someone currently blocking several AS
networks in his seemingly futile efforts in controlling MTA bandwidth. 

> 	* Transmission of the real message

What vetting can occur?  What element within a message can be trusted?
A message reference when accessed through a different channel can not be
falsified.  Even with DKIM, there is little that can be trusted.
Perhaps there might be exceptionally restricted MSAs shielded from
bad-actors, but this would not be the typical situation.

SMTP needs to change into a system that delivers an outside channel
message reference.  Outside channel message references can not be
spoofed.  Fetching the message though an outside channel provides
content security and a reliable indication the message was received.
SMTP normally working in a transfer by reference would be far more
reliable and far more suitable for commerce.

> attacks possible at each step.

Yes, and to what end?

> in that for you; maybe not.

Perhaps. It seems less is more with respect to what might be included
with the reference.  The vetting must be limited to the reputation of
the publisher and not include what some filter process might conclude. 

> general as you believe), we need a real proposal not more

I'd be happy to flesh-out a draft and bring it to some other
mailing-list. What would be the right venue? 

-Doug

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 17 Aug 2007 20:31:51 +0200

On 17-aug-2007, at 17:54, Steven M. Bellovin wrote:

> understand trust anchors,

Big yellow warning when S/MIME authentication fails in Apple's Mail  
is hard to miss even if you don't understand exactly what it is...

> or how to distinguish myfinancialcompany.com from

So I can register paypa1.com and then go to Verisign to get a  
certificate for that domain? If that's true, then I think the law  
makers in various jurisdictions have work to do.

The very simple idea of having a .bank TLD for financial institutions  
could also help a lot here.

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Fri, 17 Aug 2007 15:03:25 -0400

On Fri, 17 Aug 2007 20:31:51 +0200
Iljitsch van Beijnum <[email-masked]> wrote:

> > someone develops a user interface that DTRT for naive users who
> > don't understand trust anchors,

You'd be surprised what people will miss...  You also have to account
for people missing the presence of S/MIME, i.e., the bad guy just sends
the email without any protection and hopes folks don't notice.
> makers in various jurisdictions have work to do.

Given that paypa1.com was the very first phishing attack I saw, and
that there was a cert...  More recently, see
http://blog.washingtonpost.com/securityfix/2006/02/the_new_face_of_phishing_1.html
> could also help a lot here.
Same failure modes.

---

From: John Levine <[email-masked]>
Date: 17 Aug 2007 19:16:23 -0000

>that there was a cert... 

Really?  It belongs to ebay.

But you are quite right, getting an SSL cert is so easy these days that
it's not useful for much.  Maybe the green bar certs that are supposed
to be harder to get will help. maybe not.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Aug 2007 15:54:43 -0400

--On Friday, 17 August, 2007 15:03 -0400 "Steven M. Bellovin"
<[email-masked]> wrote:

>> The very simple idea of having a .bank TLD for financial
>> institutions could also help a lot here.
>> 
> Same failure modes.

Worse, unless the registry and its registrars were chartered
such that registering anything but the primary name of a
legitimate bank (with "legitimate" defined according to some
internationally- recognized authority model, which is a whole
other problem) would subject them to significant civil and
criminal penalties.

      john

---

From: SM <[email-masked]>
Date: Fri, 17 Aug 2007 15:13:31 -0700

>source messages from any mail user using any email address in any

That's what created the ubiquity of email.

>connectivity rather than opening a gaping hole in the architecture.

Finding better connectivity is not always an option especially when 
you are away from home.  Are we spoilt by "always-on" connectivity 
that we assume that all networks are reachable?

Regards,
-sm

---

From: SM <[email-masked]>
Date: Fri, 17 Aug 2007 16:18:34 -0700

>protocol design isn't going to save the world any time soon.

Right.

You picked a very apt subject line.  It's a simple protocol, it's 
easy to use and above all, it's cheap.  But when you are dealing with 
financial transactions, you're putting your customers at risk then as 
there is no way for them to determine the authenticity of the message.

>them on a per-message basis.   But the key institutions that get

Some people see using a large provider as better than running a mail 
server for a small domain.  After all, it's free.  But it may not be 
free tomorrow and it may turn into an expensive communication medium for them.

>deliver a sender key or cert out of band, validate it, and be

There are ways to validate the sender the first time you establish a 
contact.  Once that is done, you can use it to validate future 
communication you receive from that correspondent.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Sat, 18 Aug 2007 12:59:58 -0400

--On Friday, 17 August, 2007 16:18 -0700 SM <[email-masked]>
wrote:

> There are ways to validate the sender the first time you
> establish a contact.  Once that is done, you can use it to
> validate future communication you receive from that
> correspondent.

Sure.  As long as we all understand that there are tradeoffs
associated with each of these.  End-to-end signatures require
that both sender and recipient be able to manage keys (directly
or indirectly), but do not depend on every actor in the
transport chain being trustworthy, trusted, and willing to play.
Used carefully, they are also insensitive to forwarding,
resending, and  mid-transport rerouting.

Hop-by-hop transport-based solutions appear to be easier to
deploy --although there are some concerns about transitivity of
trust relationships and the ability of large mail providers to
force the smaller ones out, among other things-- and they
generally work much better when there is a direct connection
between the originating MSA and the final deliver MTA than when
relays are involved.   But they also tend to restrict services
somewhat.  

For example, there is a long, and IMO desirable, history of
people setting up "stable addresses" via a friend or institution
-- addresses that can remain constant even though the actual
mail address and mail store provided by a vendor or ISP may
change over time.  This reduces ISP or mail-provider lock-in and
seems like a good idea to many people.  However, suppose the
user Joe Blogs, maintains a permanent address at
[email-masked] and has, this month, a mailbox
at [email-masked].  Similarly J. Random Member has a
permanent mailbox at [email-masked] but a mail account
at [email-masked].  While some workarounds are
possible, it is not obvious mail is sent from
[email-masked] to [email-masked]
given that the message originates from the domain and address
space of postoffice.example.net, but that administration doesn't
know about, or have any relationship to, acm.org and hooya.com
doesn't have any information about routings via
forwarder.example.com.   Maybe we have to give that up --and
give in to the desire of those who run the large email services
to advertise themselves and lock users in -- but, from my point
of view, the techniques better have very high leverage on spam
and criminal enterprises in order to justify that.  Otherwise,
we just reduce the capabilities and attractiveness of the mail
system and increase the burdens on legitimate senders and
receivers without accomplishing much.  And, to me, that feels a
bit too much like just helping the bad guys win.

     john

---

From: Dave Crocker <[email-masked]>
Date: Sun, 19 Aug 2007 11:48:11 -0700

...
> force the smaller ones out, among other things-- and they

Given the poor history of actual deployment and use of authentication -- that 
IS what is being discussed, right? -- for email, I'm not sure where the 
"appear to be easier to deploy" comes from, unless it is the narrow 
consideration of the two popular path-based schemes, SPF and Sender-ID.  If 
so, the large-scale efficacy of them is either unclear or problematic, 
depending upon which skeptic is talking. My point is that "appear to be" 
requires constraining the consideration too much.

> relays are involved.   But they also tend to restrict services

In other words, hop-by-hop is easier, when there is only one hop?

>    Maybe we have to give that up --and

Right.  Or perhaps consider alternate techniques that do not impose this 
limitation?

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: John C Klensin <[email-masked]>
Date: Mon, 20 Aug 2007 11:06:03 -0400

--On Sunday, 19 August, 2007 11:48 -0700 Dave Crocker
<[email-masked]> wrote:

>> deploy --although there are some concerns about transitivity
>> of trust relationships and the ability of large mail
>> providers to force the smaller ones out, among other things--
>> and they
> 
> Given the poor history of actual deployment and use of
> authentication -- that IS what is being discussed, right? --
> for email, I'm not sure where the "appear to be easier to
> deploy" comes from, unless it is the narrow consideration of
> the two popular path-based schemes, SPF and Sender-ID.  If so,
> the large-scale efficacy of them is either unclear or
> problematic, depending upon which skeptic is talking. My point
> is that "appear to be" requires constraining the consideration
> too much.

If was precisely SPF and Sender-ID, and some similar approaches,
to which I was referring.    And I agree with the skepticism
and, based on my personal biases, could have said something a
lot stronger that "appear to be":  I was trying to be as neutral
as possible in that comment.   One may, of course, be able to
substitute "one hop in trust relationships" for "one hop in
transport", but it is not clear how much that changes things.
To avoid more of the confusion that comes from not giving
examples, while I believe that DKIM eliminates some of the
problems associated with SPF and Sender-ID, I don't see it as
eliminating all, or even most, of them _if it is used in the
same way_ (if it is used purely as a reputation service for
elevating the level of attention/ MUA priority given to some
messages, as you have suggested, I don't see it as even part of
this discussion).

But I don't think I understand how/why you believe it constrains
anything... it was just an observation that those techniques do
have skeptics, that there is some evidence that they don't work
as well (at least in simple form) as their most passionate
advocates would claim, and so on.

>> between the originating MSA and the final deliver MTA than
>> when relays are involved.   But they also tend to restrict
>> services somewhat.  
> 

In other words, too many of the proposals that keep kicking
around assume that there is a live, direct, online, path, using
stable and public IP addresses, between the set of systems under
close control of the receiver and those under close control of
the sender and that there are no gateways involved.  I note that
the absence of such control does not imply open relays or their
ilk although it does imply that, in some cases, not every system
in the path is guaranteed to be following exactly the same
rules. 

I was trying to suggest --obviously in too subtle a way-- that
the supposedly easy-to-deploy transport-based systems often
don't work well if there is more than one hop.  If that is
equivalent to what you say above, then so be it: the apparent
silliness of that statement may be just the point.

>> give in to the desire of those who run the large email
>> services to advertise themselves and lock users in -- but,
>> from my point of view, the techniques better have very high
>> leverage on spam and criminal enterprises in order to justify
>> that.  Otherwise,
> 
> Right.  Or perhaps consider alternate techniques that do not
> impose this limitation?

Sure.   You have any suggestions that don't require any of:

	* fundamental changes to how the mail system works,
	especially changes that would make things more difficult
	for legitimate users with slow, intermittent, or very
	expensive connectivity?
	
	* a PKI or equivalent with fairly strong assumptions
	about user ability to manage keys?
	
	* central control of the mail system, or control by a
	relatively small collection of collectively-selected, or
	regulator-selected or certified, providers?

	* treating a new correspondent address as inherently
	suspicious even if the correspondent is well-known and
	trusted?

--john

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Aug 2007 08:59:44 -0700

John C Klensin wrote:
> If was precisely SPF and Sender-ID, and some similar approaches, to which I
> was referring.    And I agree with the skepticism and, based on my personal
> biases, could have said something a lot stronger that "appear to be":  I
> was trying to be as neutral

(every now and again, i too experiment with being politic.)

> as possible in that comment.   One may, of course, be able to substitute
> "one hop in trust relationships" for "one hop in transport", but it is not
> clear how much that changes things.

Probably requires both constraints, to work, given the physical act of
registering source addresses.

> To avoid more of the confusion that comes from not giving examples, while I
> believe that DKIM eliminates some of the problems associated with SPF and
> Sender-ID, I don't see it as eliminating all, or even most, of them _if it
> is used in the

Reasoned discussion about tradeoffs and benefits is always helpful. This space
has not had much of that...

In any event, reasoned proponents of DKIM are clear that it has narrow intent
and narrow benefit.  While some of the constraints are already known, I
suspect most reasoned proponents would expect to discover additional ones.

> anything... it was just an observation that those techniques do have
> skeptics, that there is some evidence that they don't work

Interesting.  You were using "appear to" as a means of raising questions and I
was interpreting it to note the lack of sufficient experience to be able to
say "known to".  So we had pretty much opposite views of the term.  Sorry for
the misunderstanding.

By way of clarifying the point I was trying to make about path-registration
administrative overhead: It is easy to create the first record. By that
measure, the barrier to adoption for a 'sender' can be extremely low.  No new
software and maybe one or a few DNS records.  This is goodness.

But it misses the considerable long-term overhead, having to constantly
anticipate changes to the records, including registering all intermediaries,
such as outsourced sending engines and individual forwarders (such as
acm.org.).  In other words, the path-registration scheme requires upper-level
application work to be cognizant of email topology and to keep accurate
registration for it.

(The irony of this is that the lower network levels are pressing hard to
*de-couple* identification information from location information, and here we
have an email security scheme trying to marry them more closely.

> In other words, too many of the proposals that keep kicking around assume
> that there is a live, direct, online, path, using stable and public IP
> addresses, between the set of systems under close control of the receiver
> and those under close control of the sender and that there are no gateways
> involved.

Yup.

> * fundamental changes to how the mail system works, especially changes that
> would make things more difficult for legitimate users with slow,
> intermittent, or very expensive connectivity?  * a PKI or equivalent with
> fairly strong assumptions about user ability to manage keys?  * central
> control of the mail system, or control by a relatively small collection of
> collectively-selected, or regulator-selected or certified, providers?
> 
> * treating a new correspondent address as inherently suspicious even if the
> correspondent is well-known and trusted?

Sure.

d/

-- 

   Dave Crocker

---

From: SM <[email-masked]>
Date: Mon, 20 Aug 2007 14:41:08 -0700

>silliness of that statement may be just the point.

Transport-based systems may seem easy to deploy as you don't have to 
upgrade the MUAs out there.  The other argument that server-side 
technologies can gain faster adoption if you take into account that 
most of the users interact with this simple protocol through a web 
interface.  We still need to use a list to assess reputation when 
using transport-based systems.  If we have to pass the assessment to 
MUA, we still face that question of MUA upgrades.

>software and maybe one or a few DNS records.  This is goodness.

The barrier of adoption for some senders may be low but it may be a 
hurdle to others if we take into account geographical boundaries or 
size of business.  Unfortunately, we won't hear the voices of those 
facing the disconnect.

Regards,
-sm

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 20 Aug 2007 15:16:25 -0700

I have a slightly different take from John here.

My strong belief is that a proposal for a new protocol that does the same thing as SMTP but slightly better is a total non starter. No matter how much better the protocol is the cost of transition will dominate.

The only way that I see a new email infrastructure emerging is as a part of a more general infrastructure to support multi-modal communication, both synchronous and asynchronous, bilateral and multilateral, Instant Messaging, email, voice, video, network news all combined in one unified protocol. 

> -----Original Message-----
> From: John C Klensin [mailto:[email-masked]] 
> Sent: Friday, August 17, 2007 9:27 AM
> To: Douglas Otis; Keith Moore
> Cc: [email-masked]
> Subject: New models for email (Re: e2e)
> 
> mail systems with relay (or, if you prefer, 
> store-and-forward) capability.  If one requires that one be 
> online to receive actual messages, with end-to-end 
> connectivity to the sender's mail holding store, then sites 
> with no direct connectivity to the Internet are out of luck 
> and sides with intermittent connections or very long delay 
> connections (think about a lot of developing areas on earth 
> as well as the interplanetary problem) are in bad trouble.  
> 
> An important side-effect of the current mail model is that, 
> if the final delivery server is network-close to the intended 
> whether mail flows into that server at gigabit bandwidth or 
> at a few kilobits/second or worse or it things flow only at 
> 0300 local time.  The only connections that are really 
> I share Dave Crocker's concern about deployment of 
> replacements for the current email model (see the article in 
> the recent issue of Internet Protocol Journal), but maybe it 
> isn't impossible if the need is significant enough and the 
> solution doesn't make other things worse (which I think yours 
> may, but I haven't seen enough of a proposal to evaluate).
> 
> One could preserve the relaying by turning message 
> transmission and delivery into a multiple-stage handshake:
> but that would be a fairly complex piece of protocol work, 
> with attacks possible at each step.
> 
> never been significantly used.  Maybe there is a message 
> (sic) in that for you; maybe not.
> 
> But the main point of this message is that, if you are 
> serious about this, generate an I-D that specifies all of the 
> details, analyzes the possible attacks and how you will repel 
> them, and so on.  IMO, you have passed the point at which 
> talking about

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 20 Aug 2007 15:16:21 -0700

My appologies for not replying earlier.

Yes, Fred, you are entirely correct here and this was the original point that I raised in the plenary. David Clark is not the sort of person to propose a dogma. As you point out the original paper is couched in terms of 'here are some issues you might want to consider'.

The post you were responding to was on a secondary point where people were complaining that I mischaracterized 'functionality' as 'complexity'. To be honest it strikes me as a footling distinction without a difference, the type of thing that is raised when people have mistaken an engineering principle for an article of faith.

A more accurate precis would of course be something on the lines of "think hard about where you put complexity, it has often proved advantageous to keep application specific complexity out of the core'. We can argue over the wording but there is no real substitute for reading the paper.

Which is what prompted the original point I made in the plenary: when someone is using the end to end principle to slap down some engineering proposal they don't like I would at least like them to appear to have read the paper they are quoting as holy scripture.

There are relatively few engineering architectures that are intrinsically good or bad. The telephone network managed very well for almost a century with their architecture. The Internet has done well for a quarter century with almost the opposite architecture. The key is appropriateness. The telephone architecture was appropriate in the circumstances for which is was designed, the same for the Internet.

The Internet has changed and we need to recognize that there may be different circumstances that lead to the desirability of a different approach. My belief is that the most appropriate architecture for today's needs is a synthesis which recognizes both the need for the network core to be application neutral and the need for individual member networks of the Internet to exercise control over their own networks, both to protect the assets they place on their network and to protect the Internet from abuse originating from or relayed through their network.

> From: Fred Baker [mailto:[email-masked]] 
> Sent: Tuesday, August 14, 2007 5:22 PM
> To: Hallam-Baker, Phillip; John Kristoff
> Cc: [email-masked]
> Subject: Re: e2e
> 
> Following Noel's rant (which is well written and highly 
> correct), it is not well summarized that way. For example, 
> quoting from the paper, "Performing a function at a low level 
> may be more efficient, if the function can be performed with 
> a minimum perturbation of the machinery already included in 
> the low-level subsystem". So, for example, while we generally 
> want retransmissions to run end to end, in an 802.11 network 
> there is a clear benefit that can be gained at low cost in 
> the RTS/CTS and retransmission behaviors of that system.
> 
> My precis would be: "in deciding where functionality should 
> be placed, do so in the simplest, cheapest, and most reliable 
> manner when considered in the context of the entire network. 
> That is usually close to the edge."
> 
> Let's take a very specific algorithm. In the IP Internet, we 
> do routing - BGP, OSPF, etc ad nauseum. Routing, as anyone 
> who has spent much time with it will confirm, can be complex 
> and results in large amounts of state maintained in the core. 
> There are alternatives to doing one's routing in the core; 
> consider IEEE 802.5 Source Routing for an example that 
> occurred (occurs?) in thankfully-limited scopes.  
> We could broadcast DNS requests throughout the Internet with 
> trace- route-and-record options and have the target system 
> reply using the generated source route. Or not... Sometimes, 
> there is a clear case for complexity in the network, and state.
> 
> Let me mention also a different consideration, related to 
> business and operational impact. Various kinds of malware 
> wander around the network. One can often identify them by the 
> way that they find new targets to attack - they probe for 
> them using ARP scans, address scans, and port scans. We have 
> some fairly simple approaches to using this against them, 
> such as configuring a tunnel to a honeypot on some subset of 
> the addresses on each LAN in our network (a so-called "grey 
> net"), or announcing the address or domain name of our 
> honeypot in a web page that we expect to be harvested. 
> Honeypots, null routes announced in BGP, remediation 
> networks, and grey networks are all examples of intelligence 
> The end to end arguments that I am familiar with argue not 
> for knee- jerk design-by-rote, but for the use of the mind. 
> One wants to design systems that are relatively simple both 
> to understand and to maintain, and to isolate for diagnosis. 
> The arguments do not say that leaving all intelligence and 
> functionality in the end system is the one true religion; 
> they observe, however, that the trade-offs in the general 
> case do lead one in that direction as a first intuition.
>

---

From: Michael Thomas <[email-masked]>
Date: Mon, 20 Aug 2007 15:21:17 -0700

> The only way that I see a new email infrastructure emerging is as a part of a more general infrastructure to support multi-modal communication, both synchronous and asynchronous, bilateral and mult…

You mean SIP?

       Mike

---

From: John C Klensin <[email-masked]>
Date: Mon, 20 Aug 2007 18:27:17 -0400

--On Monday, 20 August, 2007 15:16 -0700 "Hallam-Baker, Phillip"
<[email-masked]> wrote:

> My strong belief is that a proposal for a new protocol that
> does the same thing as SMTP but slightly better is a total non
> starter. No matter how much better the protocol is the cost of
> transition will dominate.

And which part of that do you think I disagree with?

As a matter of principle, I believe an important one, I will not
say, to Doug or anyone else, "in spite of the fact that I
haven't seen a complete and coherent proposal, I think your idea
is too stupid to be worth discussion".    My guess is that, once
I see a complete proposal, I'm not going to be enthused about it
-- partially because of issues I've already identified-- but I
believe it is better for all of us if Doug actually generates a
proposal rather than tossing generalities and vague ideas at us
and having us toss generalities and vague ideas back.  And he
might just convince me that there is a useful area of
applicability for his ideas, after which we could quibble about
how broad that area was rather than arguing about the viability
of the protocol ideas.   I wouldn't plan on it, but I think
trying to keep an open mind is useful.

> The only way that I see a new email infrastructure emerging is
> as a part of a more general infrastructure to support
> multi-modal communication, both synchronous and asynchronous,
> bilateral and multilateral, Instant Messaging, email, voice,
> video, network news all combined in one unified protocol. 

Of course, such a protocol would have much greater odds of
success if it was also less complex than either SMTP or SIP,
provided absolutely reliable sender authentication and message
integrity that typical users could fully understand and
utilized, and also cured cancer.

     john

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 20 Aug 2007 15:47:00 -0700

The slightly different spin that I started from but did not manage to quite get to come out in the message was that I don't think that the successor to email will be designed as the successor to email.

I think that it will be an infrastructure that is designed to do something different and just happens to provide a more convenient or otherwise superior email capability while being seamlesly integrated into it. It will replace email by accident or not quite accident rather than design.

The same goes for the various proposals to replace the DNS with something 'better' by whatever metric people choose to apply (uses ASN.1 and will herald the return of OSI, uses angle brackets and will solve the worlds problems with its angle bracketty goodness, uses a proprietary patented protocol and will make the proponents who not coincidentally continue to exercise control over it rich, rich, rich). They will fail because they try too hard and they don't realise that their extrinsic objectives are incompatible with their public goal.

> Sent: Monday, August 20, 2007 6:27 PM
> To: Hallam-Baker, Phillip; Douglas Otis; Keith Moore
> Cc: [email-masked]
> Subject: RE: New models for email (Re: e2e)
> 
> 
> 
> does the 
> > same thing as SMTP but slightly better is a total non starter. No 
> > matter how much better the protocol is the cost of transition will 
> > dominate.
> 
> As a matter of principle, I believe an important one, I will 
> not say, to Doug or anyone else, "in spite of the fact that I 
> -- partially because of issues I've already identified-- but 
> I believe it is better for all of us if Doug actually 
> generates a proposal rather than tossing generalities and 
> vague ideas at us and having us toss generalities and vague 
> ideas back.  And he might just convince me that there is a 
> useful area of applicability for his ideas, after which we 
> could quibble about how broad that area was rather than 
> arguing about the viability
> > The only way that I see a new email infrastructure emerging is as a 
> > part of a more general infrastructure to support multi-modal 
> > communication, both synchronous and asynchronous, bilateral and 
> > multilateral, Instant Messaging, email, voice, video, 
> network news all 
> > combined in one unified protocol.
> 
> provided absolutely reliable sender authentication and 
> message integrity that typical users could fully understand 
> and utilized, and also cured cancer.
> 
>      john
> 
>

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Aug 2007 16:05:55 -0700

> utilized, and also cured cancer.

Well, this thread is entirely too timely for me to pass up the opportunity of 
citing:

"Is It Time to Replace SMTP?"

    tiny: <http://tinyurl.com/35m64e>
    full: 
<http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_10-2/102_smtp.html>

In the current issue of IPJ.

d/
-- 

   Dave Crocker

---

From: Fred Baker <[email-masked]>
Date: Mon, 20 Aug 2007 16:48:00 -0700

On Aug 20, 2007, at 3:16 PM, Hallam-Baker, Phillip wrote:
> Which is what prompted the original point I made in the plenary:  
> when someone is using the end to end principle to slap down some  
> engineering proposal they don't like I would at least like them to  
> appear to have read the paper they are quoting as holy scripture.

hear, hear...

> The Internet has changed and we need to recognize that there may be  
> different circumstances that lead to the desirability of a  
> different approach. My belief is that the most appropriate  
> architecture for today's needs is a synthesis which recognizes both  
> the need for the network core to be application neutral and the  
> need for individual member networks of the Internet to exercise  
> control over their own networks, both to protect the assets they  
> place on their network and to protect the Internet from abuse  
> originating from or relayed through their network.

The funny thing is that I'm not convinced that this is a change.

As I have said in this and other fora, there is an argument for  
functional complexity in the network, and routing is its poster  
child. Routing could be done from the edge and the network be unaware  
of how to route across it. It would be very difficult for a service  
provider to make any guarantees to his customers in such a case, but  
the obvious algorithms one might use are well known. We don't do that  
- we do indeed treat routing as an acceptable form of complexity that  
leaves significant state in the network - specifically because we  
find commensurate value in the functionality provided.

RFC 61 (August 1970), quoting a paper by L. Roberts and B. Wessler in  
the same year, observes that

    "A resource sharing computer network is defined to be a set of
    autonomous, independent computer systems, interconnected so as to
    permit each computer system to utilize all of the resources of each
    other computer system.  That is, a program running in one computer
    system should be able to call on the resources of the other computer
    systems much as it would normally call a subroutine."

Here, the author is clearly thinking of processes on computers. But  
just the following year RFC 164 reports that the Air Force wanted to  
build an autonomous network, in 1979 the SRI NIC commented (RFC 756)  
on the need for autonomous name servers in various networks (a line  
of reasoning that led to the present DNS a few years later), and in  
1983 RFC 820 reported the assignment of the first Autonomous System  
Number, used by RFC 827's EGP protocol for separating autonomous  
routing systems at the edge from the BBN-operated ARPANET.

 From my perspective, it has been at least 25 years, if not 37, since  
we recognized "both the need for the network core to be application  
neutral and the need for individual member networks of the Internet  
to exercise control over their own networks". What we have struggled  
with since is the recognition that while the core needs to be  
application neutral, it isn't necessarily service-neutral; the  
development of two QoS architectures, the traffic engineering in  
MPLS, and the amount of bickering that has gone around declaring such  
things to be inappropriate - ("just get enough bandwidth and all that  
goes away") - has been all about the assertion that the core should  
be application-neutral vs the assertion that it needs to be able to  
offer specific services.

I read in the paper this morning that the "Web" is running out of  
bandwidth. The article quoted Metcalf and others and their  
predictions of the death of the Internet, and noted that Bob finally  
blenderized his columns in front of an audience. My company figured  
in the article. So did the advent of video content, which requires  
significantly more bandwidth and significantly lower loss rates than  
more traditional voice services or TCP-based applications. Had the  
author had a clue, s/he would have said something about the Internet  
once again changing its fundamental service, from enabling terminal  
service, to moving files using FTP and Network News, to moving  
smaller files using SMTP and HTTP, to basic audio services, to peer- 
to-peer file sharing, and now to video. Not that the old goes away  
right away, but we add on and the old becomes less important.

What we need to do is figure out how to let the intelligent network  
core work cooperatively with the intelligent edge to let it do  
intelligent things. Right now, the core and the edge are ships in the  
night, passing and occasionally bumping into each other. No, we don't  
want unnecessary intelligence in the core. But, as with routing, I  
will argue that there is some that is constructive.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Aug 2007 17:03:50 -0700

Fred Baker wrote:
> functional complexity in the network, and routing is its poster child. 
...
> What we need to do is figure out how to let the intelligent network core 
> work cooperatively with the intelligent edge to let it do intelligent 
> things. Right now, the core and the edge are ships in the night, passing 
> and occasionally bumping into each other. 

Perhaps the requirement is to stop viewing the choices as adversarial and 
consider them in terms of potentially complementing each other.  The 
infrastructure is not a goal; it is there to support end-user capabilities. 
(To the folks actually using the Internet, infrastructure is overhead.)

At its very worst, e2e has never said that there should be no intelligence in 
the infrastructure, since as you note, routing has been there from the start 
and ain't exactly trivial.

What has been observed, however, is that it is better to keep a function out 
of the infrastructure, when possible, because it is both easier to adopt and 
easier to change, when it is at the edges.

Over time, what is outside the infrastructure often becomes part of it.  DNS 
is a good example, in terms of application-level utility.  In a different 
sense, this can even be true within application services.  MIME was carefully 
designed as an edge service.  No change to the mail transfer world was 
required. But try running a serious email service without MIME support today.

And that probably suggests the design philosophy:  Start be defining a service 
as strictly at the edges.  Preferably user hosts, but failing that, in edge 
networks.

If support is required deeper than that, fine.  But at least have a good 
understanding of why and try to make it as minimal as possible.

d/
-- 

   Dave Crocker

---

From: Michael Thomas <[email-masked]>
Date: Mon, 20 Aug 2007 17:17:27 -0700

Fred Baker wrote:
> night, passing and occasionally bumping into each other. 

Isn't that the essence of the problem though? That is, signaling between
trust domains is an inherently difficult thing to scale up? It seems to me
that even setting up static relationships within a semi-trusted domain is
pretty hard (say, so-called "Voice lans" or suchlike), but managing the
complexity of signaling just makes the eyes of operators to glaze over.

When I wrote that draft on rsvp vs. mobility vs. security ages ago, it
quickly became obvious that the combinatorics get quickly out of hand,
and that draft was just about the _theory_ of doing such a thing; gawd
help what the actual real life complexity is with different vendors, buggy
software, people trying to game such a system, etc, etc. Out of it I came
away with new appreciation for the power of Brute Force, and Ignorance
with respect to just throwing bandwidth at the problem.

So I guess I have to wonder whether ships in the night is such a bad
thing. One thing we do know is that reality has a tendency to distort
and magnify warts, even with seemingly "simple" applications. As
you add edge "tussles", the complexity seems to go through the
roof. Maybe these kinds of things can be done in small, centrally
marshaled  deployments, but for the whole internet?

       Mike, not advocating end-complexity-as-faith either

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Mon, 20 Aug 2007 21:10:01 -0400

Anyone who thinks that a new mail protocol that relies on users seeing
some "secure" or "trustworthy" indicator should read:

An Evaluation of Extended Validation and Picture-in-Picture Phishing
Attacks

Collin Jackson, Daniel R. Simon, Desney S. Tan, and Adam Barth, Proc.
USEC '07.

http://usablesecurity.org/papers/jackson.pdf

---

From: John Day <[email-masked]>
Date: Mon, 20 Aug 2007 21:46:31 -0400

>intelligent things. Right now, the core and the edge are ships in 
>the night, passing and occasionally bumping into each other. No, we 
>don't want unnecessary intelligence in the core. But, as with 
>routing, I will argue that there is some that is constructive.

Careful Fred, this begins to sound more like phone company hype of 
the 1980s for AIN, a return to the "beads on a string" model of data 
communications rather than the innovative ideas that lead to the 
Internet.

The thinking seems to be stuck in an "either-or."  It is too bad that 
those brilliant insights of 30 years ago were never pursued but 
became an object of reverence and stagnated.

Your remark about Robert and Bressler brings to mind an interesting 
observation I have noticed.  If you look at the work of Baran, 
Roberts, etc. those who were mature workers in the field before the 
"paradigm shift" they only seem to make half the shift.  (Not 
uncommon in the history of science).  They make the shift to packets 
but not the shift from determinism to non-determinism.  It seems to 
be left to those who come after to fully embrace the new model.  And 
much of the effort for the past 30 years has been between the 
deterministic supporters and the non-deterministic advocates.  But 
then it would seem that the new model is carried out not with the 
vision of founders but with a spirit of preservation by converts.

Maybe one of these days some of these young researchers will finally 
have the gumption to do what needs to be done.  Heck, When we were 
their age anyone our age didn't know squat and we were right!  But 
right now, they don't seem to be much of a challenge, if the best 
they can do is overlays and p2p.

Take care,
John

---

From: Dave Crocker <[email-masked]>
Date: Mon, 20 Aug 2007 20:43:09 -0700

Michael Thomas wrote:
>> multilateral, Instant Messaging, email, voice, video, network news all 

Probably not:

    <http://www.acmqueue.org/modules.php?name=Content&pa=showpage&pid=87>

d/
-- 

   Dave Crocker

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 21 Aug 2007 01:36:42 -0700

SIP plus an open identity infrastructure plus social networking, perhaps, maybe.

I think that SIP has the addressing part right, it supports an email address notation for starters, it uses the DNS. But it needs more.

In particular any successor messaging scheme has to have spam reduction, and nuisance call reduction built into the infrastructure. That means some form of buddy list concept but on an extended scale. I think that Tim Berners-Lee may be right that FOAF is the technology of choice here.

To pull it all together I think we need to integrate in the concept of identity infrastructure so that we can have a single identifier for both inbound and outbound identity. Outbound identity being what I claim when I authenticate to post on a blog, inbound identity when someone attempts to send me a contact message via whatever medium.

I like FOAF because it is a passive approach, I do not need to spend an inordinate amount of time asking other people to add me to their list and replying to requests from others. In FOAF links are allowed to be unidirectional. I can add John Klensin to my FOAF file and he can decide not to add me to his. Cf 404 not found, scruffy links work and turn out to be better than bidireactional maybe.

I think we could clean FOAF up a lot with a small amount of support protocol and linking it to the email address identifier, but the structure seems basically right. So for example when I add John to my FOAF file it might ping his FOAF server to let it know and maybe it would send me a ping in return if John changed his email address of his FOAF location in future. 

Maybe once a month or so John looks at the folk who have added him to their FOAF file and clicks on a few to add them to his FOAF file.

With such a capability most of the use cases for relay mail possibly, maybe evaporate, perhaps.

> From: Michael Thomas [mailto:[email-masked]] 
> Sent: Monday, August 20, 2007 6:21 PM
> To: Hallam-Baker, Phillip
> Cc: John C Klensin; Douglas Otis; Keith Moore; [email-masked]
> Subject: Re: New models for email (Re: e2e)
> 
> does the same thing as SMTP but slightly better is a total 
> non starter. No matter how much better the protocol is the 
> cost of transition will dominate.
> >
> > The only way that I see a new email infrastructure emerging 
> is as a part of a more general infrastructure to support

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 21 Aug 2007 11:09:07 +0200

> some "secure" or "trustworthy" indicator should read:

That's completely meaningless.

When you let small kids or dogs loose on the street, they will walk  
under cars because they don't understand what the cars are doing and  
that they won't be able to stop in time. The dogs will never  
understand these things, but generally, the kids grow up and learn to  
deal with traffic.

Today's internet users are 4-year old kids. The banks are parents  
that spoil their kids by bailing them out rather than making them  
feel the consequences of bad behavior. The banks actually like this,  
because this "service" enables them to keep pocketing a nice  
percentage of all credit card transactions.

At all levels, users, banks, certificate vendors and governments,  
things that could actually help the situation aren't done. So  
apparently there is no problem.

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 21 Aug 2007 02:10:09 -0700

> I read in the paper this morning that the "Web" is running 
> out of bandwidth. The article quoted Metcalf and others and 
> their predictions of the death of the Internet, and noted 
> that Bob finally blenderized his columns in front of an 
> audience. My company figured in the article. So did the 
> advent of video content, which requires significantly more 
> bandwidth and significantly lower loss rates than more 
> traditional voice services or TCP-based applications. Had the 
> author had a clue, s/he would have said something about the 
> Internet once again changing its fundamental service, from 
> enabling terminal service, to moving files using FTP and 
> Network News, to moving smaller files using SMTP and HTTP, to 
> basic audio services, to peer- to-peer file sharing, and now 
> to video. Not that the old goes away right away, but we add 
> on and the old becomes less important.

I think the author has it 180 degrees wrong.

The Internet is not running out of bandwidth, it is the improvement in bandwidth that is the driver here, the Internet is now within reach of supporting full definition video. The market demand for high definition is growing. So demand outstrips supply for a short while. The same thing happened in the early days of the Web, when embedded images were first plonked into Mosaic the Internet bandwidth was not sufficient to support them. The whole of CERN was sitting off a pipe thinner than some non-IETFers have running into their house (yes I know some folk have an OC 192 into their house, I remember when having a personal T1 was considered having won the bandwidth wars).

At the end of the day we can probably be confident that a saturation point for Internet bandwidth as far as user applications goes will come when we have enough bandwidth to saturate each of the five human senses. There might well be other uses beyond that of course but there will definitely be some breakpoint there.

The bigger challenge is how we put this bandwidth in the hands of a billion plus users without having collosal abuse issues. If the Internet pipe into your home has enough bandwidth to support live high definition video you have the ability to send one heck of a lot of spam.

The changes I see are not fundamental, Fred is entirely correct in pointing out that we have always had certain types of complexity in the core. The differences are subtler, of degree, of balance. The scales that tipped one way in 1980 tip in a different direction today.

Clark expected this of course, that is why the core message of the E2E paper might be summed up by J.J.Watson's mantra: Think!

---

From: <[email-masked]>
Date: Tue, 21 Aug 2007 10:53:43 +0100

> cost of transition will dominate.

Right! It is not the protocol that is at fault, it is the architecture
in which the protocol is used. I once lived in a house that had not been
designed by an architect. It started life as a granary, then was
converted to a tiny house by installing a kitchen sink and a bathroom.
Then the owner added a bedroom on one end, extending the gable roof.
Next the house was doubled by extending it to one side under a shed
roof. At this point the owner's alcoholism was affecting the quality of
the work. Various corners were cut such as a single-pane mobile home
window. Another bedroom was added to the other end of the house by
extending only the shed-roof section. No electrical outlets in this part
and only one tiny window. Finally, when the owner's alcoholism was well
advanced he filled in the gap beside the last bedroom with a hand-poured
concrete floor, and a roof and walls made with poles (with the bark
still on) and scrap plywood. This became the entrance foyer to the
house. Shortly thereafter, I bought the house, my first home purchase.

The Internet email architecture is a lot like this house, with add-ons
and patches. The latest round of SPF/DKIM stuff feels a lot like that
last room built with poles and whatever scrap was handy. I just feel
that given the lessons learned in scaling an email system to global
proportions, we could do better by taking an overall architectural view
to the problem, and coming up with a more robust architecture.

> video, network news all combined in one unified protocol. 

Wrong! This is a sure way to invite second-system effect. Read this
http://en.wikipedia.org/wiki/The_Mythical_Man-Month or better yet, read
the book itself. Sometimes it is better to leave things out of your
design than to include them all.

--Michael Dillon

---

From: Tony Finch <[email-masked]>
Date: Tue, 21 Aug 2007 20:04:59 +0100

> Sorry, that's not a reasonable desire in my opinion.

How else do you implement forwarding?

Tony.

---

From: Tony Finch <[email-masked]>
Date: Wed, 22 Aug 2007 00:10:02 +0100

On Thu, 16 Aug 2007, Keith Moore wrote:
> servers for any particular domain,

How do you intend to support forwarding? or do you have in mind something
like CSA not SPF?

> thus pushing more of the costs toward the sender of a message.

If I am not sure about the legitimacy of a message, it's better for me
to get as much information about it as possible (i.e. receive the whole
thing) so that my AI has the best chance of producing the right result.
The only case where delaying email might help is when I expect that more
information (e.g. from reputation services) might be available later. In
other cases, delays only harm legitimate email and don't hinder spammers
at all, since spammers have infinite computer resources.

Tony.

---

From: Tony Finch <[email-masked]>
Date: Wed, 22 Aug 2007 00:22:13 +0100

> 	* Transmission of the real message

I don't understand how it can be easier to do anti-spam checks in an
IM2000 design than in current email. The check has to be performed during
the first step otherwise you have already lost, but the notification has
fewer clues for an AI to work with.

Tony.

---

From: John C Klensin <[email-masked]>
Date: Tue, 21 Aug 2007 19:59:09 -0400

--On Wednesday, 22 August, 2007 00:22 +0100 Tony Finch
<[email-masked]> wrote:

> I don't understand how it can be easier to do anti-spam checks
> in an IM2000 design than in current email. The check has to be
> performed during the first step otherwise you have already
> lost, but the notification has fewer clues for an AI to work
> with.

FWIW, I agree completely.  For anti-spam purposes, the only
value of this sort of drill, IMO, would be to authenticate the
message sender.  While it doesn't have much appeal to me, there
is a fraction of the community for which "no incoming email at
all except by prior arrangement" makes sense.  For them, if the
receiving system had a whitelist of acceptable backward-pointing
addresses and could authenticate the legitimacy of those
addresses when a message arrived and reject any other messages,
those users would see no spam.  For the rest of us, there is
certainly a large fraction of today's spam that can be rejected
on the basis of bogus or inconsistent backward-pointing
addresses.  However, I believe that, if we were to make major
changes just to deal with such addresses, the spammers would
quickly clean up their acts and start using better ones (see
Paul Vixie's piece, cited some days ago).

Even for the more restricted community, the approach leaves some
open questions, e.g., 

	* Whether that community is important enough to justify
	mail protocol changes, and 
	
	* Whether the additional opportunities for attacks that
	are introduced by a multi-step process overwhelm any
	possible advantages from first authenticating the sender
	and then fetching the message content (presumably in an
	environment in which it is possible to authenticate the
	sender if the message content arrives in a single
	transaction).

Put differently, there is a community for whom a "we are not
willing to receive mail except from known, authenticated, and
trusted senders" solution would be sufficient to control what
that community considers spam.  That is largely because their
effective definition of spam requires neither "bulk" nor
"commercial" elements, only "sent by someone I haven't
pre-approved".  For that situation, and modulo the issues with
authentication mentioned in the article Steve Bellovin cited as
well as some other issues about how one authenticates, the
presumption is that one doesn't need to worry about scanning
message content for evidence of possible spam.

However, even that narrow an acceptance rule assumes that one's
trusted friends have not been turned into zombies.  It will not
prevent introduction of malware (by forwarding from careless
and/or zombie friends).  If one is concerned about malware, then
one needs to scan message content anyway (as I think you are
suggesting) and the appeal of what I understand of Doug's
proposal is reduced to saving the bandwidth for the messages
that can be rejected on the basis of backward-pointing addresses
at the cost of additional complexity, handshaking, etc.

I'm not convinced that is worth it --and strongly suspect it is
not-- but, if Doug believes otherwise, I'm looking forward to
seeing a proposal.

  best,
    john

---

From: Tony Finch <[email-masked]>
Date: Wed, 22 Aug 2007 01:11:19 +0100

On Tue, 21 Aug 2007, John C Klensin wrote:
>
> Put differently, there is a community for whom a "we are not willing to
> receive mail except from known, authenticated, and trusted senders"
> solution would be sufficient to control what that community considers
> spam.

Those people can use SMTP with authentication between relays - e.g. using
TLS (cf. XMPP), or SASL, or CSA, etc.

Tony.

---

From: Keith Moore <[email-masked]>
Date: Tue, 21 Aug 2007 20:28:24 -0400

> How do you intend to support forwarding? or do you have in mind something
I was making a general statement, not a detailed proposal.  Offhand I
think that forwarding should be traceable - that is, there should be
information in the message envelope detailing who or what forwarded the
message.  For a forwarded message, that person or process would need to
be associated with the outgoing mail server.  Note that I didn't say it
would be easy to fold that into the existing SMTP framework.
> at all, since spammers have infinite computer resources.
One advantage in having mail stay on the sender's side until delivery is
that it enables the sender to reliably determine the status of a message
at any time, and to reliably know whether a message were delivered,
refused, or bounced.  That feedback path would in turn provide
incentives for senders to supply reliable indications to recipients that
the message were legitimate (so as to encourage recipients' mail systems
to process the mail more quickly), and would also let senders know
whether such indicators were acceptable to recipients.

The purpose of keeping things on the sender side until delivery is not
to delay mail, so much as to make mail traceable.  Pushing the costs
toward the sender is a (mostly) useful side effect. 

Keith

---

From: Douglas Otis <[email-masked]>
Date: Tue, 21 Aug 2007 19:26:41 -0700

> not-- but, if Doug believes otherwise, I'm looking forward to  

I hope to have a draft ready in the near future.  When SMTP was  
developed, HTTP did not exist in its present state.  HTTP over a  
clustered file system is now able to function as a separate message  
channel for even the largest provider.  This separate channel can  
significantly lower the +90% levels of spam currently carried by  
SMTP, while also substantially moving the burdens toward the sender  
where it belongs.

Why change?  When a sender is suspected of spamming, the receiving  
MTA could limit exchanges to Transfer By Reference instead of issuing  
an ultimately more expensive refusal or message acceptance.  In the  
case of TBR, SMTP would be limited to just exchanging an HTTP message  
reference within the SMTP envelope.  This separate HTTP channel  
provides both delivery confirmation, and ensures the validity of the  
reference.

Whether the message is accessed depends solely upon the reputation of  
the HTTP publisher.  Any additional information could be abused and  
might require filtering.  Filtering burdens the recipient, where bad  
actors enjoy a significant advantage.  Not  relying upon filtering,  
finding an identifier independent of the IP address, excluding  
"friendly" information, and reducing the burden on the recipient are  
the general guidelines.

As the message reaches the MDA, the MDA would have the option to  
proxy access via BURL, fetch the message and revert to a normal SMTP  
form, or convert the reference into a pseudo message compatible with  
MUAs where recipients decide whether a message is to be accessed.   
TBR does not prevent spam, however it significantly lowers the  
recipient's burden and provides a simple means for avoiding unwanted  
or spoofed messages.  Details will follow in a draft.

-Doug

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Aug 2007 22:30:12 -0400

Ted Hardie wrote:
> any-to-any mesh.  That's not just a design goal. 

In looking at the thread before and after your note, I'm struck by the focus 
on technical issues, rather than human communication requirements.  Internet 
mail supports a wide array of human comm styles and needs.  The technology is 
useful for that breadth because it imposes as few limitations as possible.

When folks call for changes to the basic capabilities, they wave away segments 
of human communications.

We should be careful of doing that, let we really render email marginal.

In any event, when folks talk about making changes -- to add features, fight 
abuse, or whatever -- they should start with a non-technical discussion about 
what changes for the human experience of email.  And they should develop 
consensus for that change.

Once they've got that, we'll probably be able to figure out how to engineer it.

Until the consensus is developed, these sorts of threads are repetitious 
theoretical exercises, of no pragmatic benefit.

d/
-- 

   Dave Crocker

---

From: Keith Moore <[email-masked]>
Date: Tue, 21 Aug 2007 23:34:30 -0400

> In any event, when folks talk about making changes -- to add features,
> fight abuse, or whatever -- they should start with a non-technical
> discussion about what changes for the human experience of email.  And
> they should develop consensus for that change. 
I'm not sure that the discussion has to start there - but until that
part is addressed, there will be nagging questions about how the human
experience changes.  And it seems that failure to completely address
this topic has plagued most of the
proposals for anti-spam mechanisms, and severely hampered the
discussions about them.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Aug 2007 22:25:12 -0700

Tony Finch wrote:
> IM2000 design than in current email. 

It is always easier to create the perfect solution on paper, and then compare 
it to the imperfections of established practice.

Paper beats rock.

d/
-- 

   Dave Crocker

---

From: SM <[email-masked]>
Date: Tue, 21 Aug 2007 22:28:09 -0700

>provides both delivery confirmation, and ensures the validity of the

This would also apply to the case of unknown senders, I gather.  If 
so, we'll see more Transfer By Reference (TBR) messages.  We won't 
have the entire message to make a proper assessment.  The receiver 
could however use a reputation system or other means to tell whether 
he/she wants to accept the message from the sender.

>or spoofed messages.  Details will follow in a draft.

Your proposal pushes storage costs to the sender side.  It does 
introduce a delay in the communication which may put off some people 
who have the expectation of "IM".

Instead of using HTTP, the retrieval could be requested at the 
recipient's end by having the MUA send a message to the receiver's 
server.  That server would then initiate a transaction based on the 
sending address to request delivery of the complete 
message.  Compared to your HTTP channel method, there's much more 
delay here.  But we don't require a direct path to retrieve the 
message content.  There is another disadvantage.  The sending server 
cannot tell at the outset whether the final hop will only accept this 
type of transaction instead of a normal delivery for the sender/receiver pair.

The cost of deployment is high as the above requires servers and MUAs 
to be upgraded.  There's also the question of abuse which hasn't been explored.

Regards,
-sm

---

From: Dave Crocker <[email-masked]>
Date: Tue, 21 Aug 2007 22:53:48 -0700

John C Klensin wrote:
>...  For anti-spam purposes, the only
> message sender.  

Which "sender"?

Author of content?  Poster into transfer service?  Operator of originating 
host?  Operator of intermediary host, such as the last hop before receipt by 
the boundary MTA?

There are a few more roles we mush together under the term sender, but I 
suspect you mean one of the above.  Confusion in the term usually seems to be 
between author and originating operator, but often includes the outbound MTA 
operator.

d/
-- 

   Dave Crocker

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 22 Aug 2007 12:21:45 +0200

> How else do you implement forwarding?

Well, most people understand "forwarding" to be encapsulating the  
original message in a new message of your own in some way, so that it  
appears to come from the forwarder rather than the original sender.  
However, there's also something called "bounce" in some clients and  
"redirect" in others, where the original sender is preserved in the  
From: line. Note though that there are other fields that are less  
visible that tell you where the message was sent from, such as Sender:.

I have no problem breaking bounce/redirect. Yes, it has its uses, but  
so do open relays, which we don't have anymore. The current levels of  
mail abuse means that it's necessary to throw away a little baby with  
the bathwater to keep the toxic waste out of the bath.

---

From: Tony Finch <[email-masked]>
Date: Wed, 22 Aug 2007 16:31:29 +0100

On Wed, 22 Aug 2007, Iljitsch van Beijnum wrote:
>
> Well, most people understand "forwarding" to be encapsulating the original
> message in a new message of your own in some way, so that it appears to come
> from the forwarder rather than the original sender.

See RFC 2821 section 3.4 for the meaning I intended. RFC 2822 section
3.6.6 paragraph 5 has some helpful descriptions.

> I have no problem breaking bounce/redirect.

Anything that breaks that would break mailing lists.

Tony.

---

From: Keith Moore <[email-masked]>
Date: Wed, 22 Aug 2007 13:57:50 -0400

> the bathwater to keep the toxic waste out of the bath.
sorry, this is a nonstarter.  being able to set a forwarding address is
extremely important, and breaking Sender is indefensible.  (Sender is
useless unless it preserves the original identity of who or what sent
the message.)

---

From: Michael Thomas <[email-masked]>
Date: Wed, 22 Aug 2007 11:44:04 -0700

> useless unless it preserves the original identity of who or what sent
Yet, mangling forwarders which break signatures -- like this list -- are
indistinguishable from every day forgery. I agree that destroying
mailing list functionality is unacceptable, but we shouldn't kid ourselves
that there aren't unintended consequences from the original everybody-
trusts-each-other-group-hug assumptions present in the design.

       Mike

---

From: Keith Moore <[email-masked]>
Date: Wed, 22 Aug 2007 14:59:27 -0400

> trusts-each-other-group-hug assumptions present in the design.
who said anything about mailing lists?  forwarding and lists are
different beasts.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 22 Aug 2007 21:39:12 +0200

> useless unless it preserves the original identity of who or what sent

Obviously life is a bit more complex than "add PKI and signature to  
From: and stir". If I get a letter addressed to me from the tax  
people, I'll take it, regardless of how it happened to arrive. Junk  
mail on the other hand (the paper variety) will meet the inside of my  
paper bin without delay if I receive it through the mail, but if my  
boss gives it to me and asks me to look at it, I look at it.

With many of the current protocols, SMTP being a prime example, there  
are very many ways to do things that weren't foreseen, or at least  
not specifically accommodated, in the original protocol. That was  
great because it allowed innovation. Unfortunately, the most  
innovative people around these days are the spammers and fishers, so  
these days, when we come up with new protocols, we need to  
specifically allow everything that's good so implementations/users  
can reject everything else. I.e., a new mail protocol will have to  
address things like forwarding and mailinglists explicitly.

---

From: Keith Moore <[email-masked]>
Date: Wed, 22 Aug 2007 15:50:34 -0400

Iljitsch van Beijnum wrote:
> are very many ways to do things that weren't foreseen, or at least not
> specifically accommodated, in the original protocol. That was great
> because it allowed innovation. Unfortunately, the most innovative
> people around these days are the spammers and fishers, so these days,
> when we come up with new protocols, we need to specifically allow
> everything that's good so implementations/users can reject everything
> else. I.e., a new mail protocol will have to address things like
> forwarding and mailinglists explicitly.
I couldn't make sense of anything but the last sentence, but I agree
with the last sentence.

---

From: <[email-masked]>
Date: Wed, 22 Aug 2007 21:40:42 +0100

> I.e., a new mail protocol will have to 

Not protocol. A new email architecture will have to address all these
things explicitly, but the protocols may indeed be usable as is, or with
minor changes.

The point is that we need to examine the entire architecture first,
separate from any protocol details, and get clear and *EXPLICIT*
requirements. Then review existing technology/protocols to identify
gaps, then fill the gaps, and document how to transition to the new
architecture.

--Michael Dillon

---

From: John C Klensin <[email-masked]>
Date: Thu, 23 Aug 2007 06:33:18 -0400

--On Wednesday, 22 August, 2007 21:39 +0200 Iljitsch van Beijnum
<[email-masked]> wrote:

>...
> Unfortunately, the most innovative people around these days
> are the spammers and fishers, so these days, when we come up
> with new protocols, we need to specifically allow everything
> that's good so implementations/users can reject everything
> else. I.e., a new mail protocol will have to address things
> like forwarding and mailinglists explicitly.

I trust you understand that "anything not explicitly permitted
is prohibited" is equivalent to "there will be no innovation
from this point forward except by breaking the rules".

Not a network environment I want to live in.  YMMD.

Marginal and criminal elements are _always_ the most innovative
people around if there is profit in stretching the boundaries of
the rules.  In normal environments, the consequences of those
innovations are limited by effective legislation that
criminalizes sufficiently bad behavior and by enforcement and
punishment structures that create significant negative
incentives for that behavior.

Consider what would happen if there were no societal consensus
that breaking into houses and stealing things was a bad thing
and if the only laws permitted a given burglar to break into any
given house once, requiring only that he leave a note with an
address for notifications that he wasn't permitted to do it a
second time. Such a system would encourage house occupants to
make significant investments in repelling initial break-ins and
in an arms race between them and the burglar population about
break-in technologies.  Every time the home occupant built the
fence a half-meter higher and that appeared to have an effect,
the burglars would obtain ladders a meter longer.  If the
occupant decided to electrify the fences, the burglars would
obtain insulators and mechanisms to short the fences out.  It
would also encourage any burglar who wanted to expand her
business to create ways of hiding or changing identities so that
the prohibition on second and subsequent break-ins after
notification would not apply or be unenforceable.

Bingo!

>From that perspective, the real question here is how bad things
have to get before society decides to effectively criminalize
the behavior.  Technical measures that temporarily make things a
little less bad for some populations are the equivalent of the
half-meter increase in fence height and mostly serve to make the
bad guys more sophisticated, to reduce general quality of life,
and to give legislators and regulators excuses to continue to
delay action.

       john

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 23 Aug 2007 17:12:04 +0200

> from this point forward except by breaking the rules".

I disagree. This means that we need to make our protocols extensible  
from the start rather than come up with hacks to extend something  
that wasn't designed to be extensible. The most important part here  
is capability negotiation. However, this is a problem in store-and- 
forward communication.

> Not a network environment I want to live in.  YMMD.

You're already living there. Try using the RFC 1323 window scale  
option through a firewall. Despite the fact that this has been around  
for more than a decade and the firewall can easily see what's going  
on, a good percentage of them handle this incorrectly.

[...]

> It would also encourage any burglar who wanted to expand her

Sounds like today's internet to me: no law enforcement to speak of.

> and to give legislators and regulators excuses to continue to

I don't share your pessimism regarding technical solutions. In most  
cases, the technology isn't the weakest link, however. But  
technological measures can make crime less profitable and also help  
in target selection: I don't have to run faster than the lion, I just  
need to avoid being the slowest person running away from the lion.

A very big problem is that non-criminal organizations benefit from  
crime, so they're more interested in managing it rather than getting  
rid of it completely. For instance, if a bank sells you credit  
monitoring, they make money. If they implement better identity checks  
so that people can't take out loans using false credentials, that  
costs them money. (And customers that don't understand what the fuss  
is all about until they're a victim.)

---

From: Douglas Otis <[email-masked]>
Date: Thu, 23 Aug 2007 10:56:18 -0700

On Aug 23, 2007, at 3:33 AM, John C Klensin wrote:

> innovations are limited by effective legislation that criminalizes  
> sufficiently bad behavior and by enforcement and punishment  
> structures that create significant negative incentives for that  
> behavior.

Unfortunately, much of the undesired behavior happens under a guise  
of commerce.  CAN-SPAM illustrates marketing associations' influence,  
where those harmed now have no legal standing.  Any amount of e-mail  
referencing prior visits is not spam.  Any request to stop may not be  
immediate, and does not impair any number of affiliates.  Even  
replying "Unsubscribe" indicates the e-mail is being read, and makes  
the address a valuable commodity.

The "opt-out" fallacy extended into a "Do Not Solicit E-mail" list  
that was dropped after the fallacy of enforcing "opt-out" became  
apparent.  Unfortunately an inability to enforce "opt-out" did not  
change legislation into an "opt-in" model.  Prior to this  
legislation, the only model able to limit spam has been "Opt-In" when  
publishing bulk e-mail.  Is it really easier to ask for forgiveness  
than to ask for permission?

Legislators remain oblivious to practical considerations for curbing  
abuse and enforcement.  Unless employed by the government or an  
Internet provider, one has no standing to even seek enforcement or  
punishment.  Which problem were legislators attempting to solve, and  
for whom?

-Doug