Subject: Re: [Asrg] email pull (was RE: Authentication )

From: Kee Hinckley <[email-masked]>
Date: Tue, 1 Apr 2003 11:28:01 -0500

At 10:05 PM -0500 3/31/03, mathew wrote:
>The question is how the reliability of a pull-based system would 
>compare to the reliability of the current system. Don't forget that 
>although email pull would degrade my ability to receive mail from 
>users whose messaging servers are badly maintained, it would 
>*increase* my ability to *send* mail to those

No, because SMTP already takes care of that with store/forward and 
retry.  And besides, you still need to send them the notification 
that they have a message.

>>Different expectations of service, and different scale.  I get mail 
>>from far more servers than I surf in a given day.
>
>Maybe you should try an RSS feed reader. :-)

I have.

>>>The current system requires storing incoming e-mail for an 
>>>indefinite amount of time.
>>
>>Nope.  Only until the end user picks it up.
>
>Uh, that's an indefinite amount of time. I might not log in to 
>collect my mail for several months.
>
>In both the current setup and e-mail pull, the server holding the 
>mail can stop holding it once the recipient has downloaded it. 
>What's the difference?

The difference is that the receiver knows the rules.  Their ISP tells 
them what their limits are, how how long they will hold mail.  If 
they go on vacation they can make appropriate arrangements.  You've 
replaced that with a system where the limits are different with each 
of your correspondents, and furthermore, the sender now has to pay 
the storage costs if some of their recipients are on vacation.

>>No. You go online.  Get a list of different email systems, decide 
>>which ones you  want to fetch, and then go online again to fetch 
>>them.
>
>If the decision process is fast enough--and it probably will 
>be--then you just have a dual-threaded e-mail client.  Most offline 
>newsreaders work that way--download the headers, autoselect a bunch 
>of articles on the fly, and download those articles, then 
>disconnect. All in one cycle.

So pull is automatic with no user filtering, but...

>
>>And that points out the biggest flaw of the system.  Not only have you not
>
>Which is a big problem why? It's not going to chew up your 
>bandwidth, you can set retrieval timeouts as short as you want to 
>limit your dialup session to, and the CPU and disk overhead of 
>failing to make a connection is minimal. I'd make that payoff to get 
>rid of most spam.

what do you mean it won't chew up my bandwidth?  Spammer sends notice 
of email.  I automatically download the spam.  Where is this a win? 
You are assuming that the spam will dry up because they can't find a 
hosting provider?  I'll agree that it's valuable to try and drive 
them to that point, and that it will help.  But I don't think that a) 
that will be sufficient, and b) that a whole new system is necessary 
and c) that a system that makes spam detection harder is a good idea.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.
_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: mathew <[email-masked]>
Date: Tue, 1 Apr 2003 17:56:03 -0500

On Tuesday, Apr 1, 2003, at 11:28 US/Eastern, Kee Hinckley wrote:
> retry.

How does that deliver mail to a POP3 server that's down?

> And besides, you still need to send them the notification that they 
> have a message.

Not necessarily, if they're a regular correspondent of mine.

>> mail can stop holding it once the recipient has downloaded it. What's 
>> the difference?
>
> them what their limits are, how how long they will hold mail.  If they 
> go on vacation they can make appropriate arrangements.  You've 
> of your correspondents, and furthermore, the sender now has to pay the 
> storage costs if some of their recipients are on vacation.

Well, yeah. Making the sender pay the costs of the mail he sends is the 
whole point of the thing. That's one of the things that's really bad 
about spam, after all.

>> Which is a big problem why? It's not going to chew up your bandwidth, 
>> you can set retrieval timeouts as short as you want to limit your 
>> dialup session to, and the CPU and disk overhead of failing to make a 
>> connection is minimal. I'd make that payoff to get rid of most spam.
>

Putting back the context you deleted:

>> The spammer is sure to have an overloaded server, so now I'm not only 
>> going to try and fetch it--but try multiple times!
> Which is a big problem why? It's not going to chew up your bandwidth, 
> you can set retrieval timeouts as short as you want to limit your 
> dialup session to, and the CPU and disk overhead of failing to make a 
> connection is minimal. I'd make that payoff to get rid of most spam.

If you can't fetch it because the spammer has an overloaded server, 
which you admit he's sure to have, then you don't see it and it doesn't 
fill up your bandwidth.

If you're going to resort to tricks like deleting context to try and 
"win" the debate, then this discussion will end soon.

> hosting provider?

Uh... if they can't find anyone to host their messages for days at a 
time, and take the bandwidth hit of delivering those messages to their 
millions of of victims, then they won't be able to deliver their spam, 
yes. Seems pretty obvious to me.

>   I'll agree that it's valuable to try and drive them to that point, 
> and that it will help.  But I don't think that a) that will be 
> sufficient, and b) that a whole new system is necessary and c) that a 
> system that makes spam detection harder is a good idea.

Well, a) I'm not proposing that it's the only method needed for 
eliminating spam 100%; b) isn't really a criticism of this proposal, 
and I'll be more than happy to drop this idea if someone comes up with 
a better one; and c) has yet to be shown, or even argued.

Regarding c), it seems to me that all the present-day spam-detection 
methods can be used on the message as it's pulled from the source 
server, as additional lines of defense if the spammer *does* manage to 
find hosting. You can use all the present-day blacklist systems to 
blacklist providers of spam hosting for e-mail pull.

mathew

---

From: Kee Hinckley <[email-masked]>
Date: Wed, 2 Apr 2003 10:28:32 -0500

At 5:56 PM -0500 4/1/03, mathew wrote:
>>>compare to the reliability of the current system. Don't forget 
>>>that although email pull would degrade my ability to receive mail 
>>>from users whose messaging servers are badly maintained, it would 
>>No, because SMTP already takes care of that with store/forward and retry.
>

It holds it until the server comes up?  But that's pretty much a 
non-existant problem.  Delivery to a POP server is typically done by 
having the SMTP server either run a program, or stuff a file 
somewhere.  Neither requires the POP server to be running.

>Not necessarily, if they're a regular correspondent of mine.

Yes.  You've mentioned that and I've kind of ignored that.  The 
thought of everyone polling their all of their regularly 
correspondents is incredibly scary.  Currently incoming mail comes 
when it comes--very efficient.  The polling (the one inefficient 
piece) is limited in cost because it is a) local and b) gets lots of 
messages from lots of different locations.  In your model I have to 
poll lots of distant machines (not to mention remember how to find 
them, but I guess you'd make that implicit in the email address 
somehow).  Now how often do I do that?  I get mail from my father 
once or twice a week.  Should I poll once a week?  That would really 
annoy him when he sends me a support question.  In fact, if I don't 
responds to those within a few hours he generally calls.  Okay, so 
I'll poll him ever four hours.  So now his server goes from having 
two transactions per week with me to having 42 transactions per week. 
Now multiply that out by all your correspondents.  Not good.

>Well, yeah. Making the sender pay the costs of the mail he sends is 
>the whole point of the thing. That's one of the things that's really 
>bad about spam, after all.

No, actually that's not one of the things that is bad about spam. 
What's bad about spam is the cost to the receiver.  One of the things 
that might decrease the cost to the receiver is putting a cost on the 
sender.  That does *not* mean that what is wrong with the system is 
that the sender doesn't pay.

>>>The spammer is sure to have an overloaded server, so now I'm not 
>>>only going to try and fetch it--but try multiple times!
>>failing to make a connection is minimal. I'd make that payoff to 
>>get rid of most spam.
>
>which you admit he's sure to have, then you don't see it and it 
>doesn't fill up your bandwidth.

But my client sits there, for days on end, attempting to get that 
email, because it doesn't know whether it's from a spammer or from a 
flakey ISP.  It sits there tieing up resources, slowing down attempts 
to fetch other email....

>"win" the debate, then this discussion will end soon.

I did not, and do not, consider that that context changes anything.

>>  You are assuming that the spam will dry up because they can't find 
>>a hosting provider?
>
>time, and take the bandwidth hit of delivering those messages to 
>their millions of of victims, then they won't be able to deliver 
>their spam, yes. Seems pretty obvious to me.

Sorry, I was ambiguous.  Yes, it will dry up if they can't find a 
host.  However I see no reason why they won't be able to find a host.

I have issues with some of your further responses as well, but 
there's not real point.  To sum up my feelings on the proposal.

1. It doesn't definitely get rid of spam (as sender-pays might).  You 
agree to that.
2. It replaces the entire system.

I've already stated that I don't believe that any system that 
requires an all-new architecture, and provides negative incentive for 
adoption in the early stages, will have any chance of adoption.  One 
that has those problems *and* provides no assurance that it will stop 
spam is even less likely to be adopted.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: mathew <[email-masked]>
Date: Wed, 2 Apr 2003 18:21:04 -0500

> somewhere.  Neither requires the POP server to be running.

I meant "server" as in physical machine, not "daemon" as in piece of 
software. And the point is, both systems are equally useless if the 
software or the machine is broken, and both recover as soon as the 
software or machine goes up again.

> somehow).  Now how often do I do that?

It's a tradeoff, like most technical decisions. The automatic polling 
frequency could be anything from every second to never. It's a detail 
to be discussed--whether it's worth it, given the downtime expected 
from messaging servers. My guess is it won't be worth it, but you 
seemed to feel otherwise, that messaging servers would be too 
unreliable to depend on.

> What's bad about spam is the cost to the receiver.

There is going to be a cost to sending e-mail. Saying that the cost is 
not paid by the sender is the same as saying it's paid by the receiver, 
as someone has to pay for the network connection and disk space.

>   One of the things that might decrease the cost to the receiver is 
> putting a cost on the sender.  That does *not* mean that what is wrong 
> with the system is that the sender doesn't pay.

On the contrary, I think that that *is* one of the things that's wrong 
with the system. I think if the sender had to pay we'd see a lot less 
spam. After all, I get a lot less paper junk mail than I get e-mail 
spam.

> flakey ISP.

If you set your timeout to days, yes. And who would do that?

>   It sits there tieing up resources, slowing down attempts to fetch 
> other email....

What kind of resource load is an open idle TCP/IP connection on any 
remotely modern desktop computer?

> I've already stated that I don't believe that any system that requires 
> an all-new architecture, and provides negative incentive for adoption 
> in the early stages, will have any chance of adoption.

I stated that too. On March 12th I wrote:

>> However, I'm not going to write it up, because it has the same 
>> problem as Jason's: there's no clear route from here to there.  It 
>> requires e-mail to be sent via a fundamentally different protocol, 
>> rather than SMTP. There's no gain for early adopters, just expense 
>> and a limited audience.
>>
>> It strikes me that there might be a way to get such a thing rolled 
>> out, however. Just don't call it e-mail.

I don't see any method as *assuring* that spam will be stopped. I think 
that's an unreasonable criterion to set.

mathew

---

From: Kee Hinckley <[email-masked]>
Date: Thu, 3 Apr 2003 16:43:19 -0500

>software or machine goes up again.

In your system the two machines are controlled by two different 
entities and separated by a large amount of internet under control of 
neither of them.  That is inherently less reliable than two servers 
controlled by the same organization on their own network.

>>when it comes--very efficient.  The

>It's a tradeoff, like most technical decisions. The automatic 
>polling frequency could be anything from every second to never. It's 
>a detail to be

It's a tradeoff that the email "sender" has no control over.  It is 
controlled by the receiver, who will of course set the polling 
frequency as high as they can, since the cost to *them* is low.  You 
end up with the current volume problem in reverse.  The recipients 
are abusing the potential senders.

>>putting a cost on the sender.  That does *not* mean that what is 
>>wrong with the system is that the sender doesn't pay.
>
>On the contrary, I think that that *is* one of the things that's 
>wrong with the system. I think if the sender had to pay we'd see a 
>lot less spam. After all, I get a lot less paper junk mail than I 
>get e-mail spam.

I'm sure that if the sender had to pay we'd see a lot less spam.  I'm 
also certain that if the sender had to do thirty back flips before 
each email message we'd see less spam.  That doesn't mean that not 
requiring back flips is what's wrong with email.  I'm not trying to 
be flip here :-), just to point out that I don't see where not 
charging the sender more than they already pay for bandwidth is an 
inherent problem.  Charging could be a solution.   But it's not the 
case that someone made a mistake by not requiring it initially.

>>email, because it doesn't know whether it's from a spammer or from 
>>a flakey ISP.
>

Anyone who didn't want to lose email?

>remotely modern desktop computer?

One to each spammer machine, plus to each temporarily unavailable 
legitimate machine.  It's not a problem if the spammers really do go 
away and the network connections really are reliable.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: mathew <[email-masked]>
Date: Thu, 3 Apr 2003 22:06:59 -0500

> controlled by the same organization on their own network.

Umm, yes, and outside of the special case where you're sending e-mail 
across a corporate network, most e-mail today involves machines 
controlled by different entities separated by a large amount of 
internet under control of neither of them. So what is your point?

>> to be
> It's a tradeoff that the email "sender" has no control over.

Right. Just like right now, I have no control over when someone will 
receive this e-mail. They might poll never, or every five minutes.

>   It is controlled by the receiver, who will of course set the polling 
> frequency as high as they can, since the cost to *them* is low.

Someone (possibly you) already mentioned this as a problem, and I 
already pointed out how easy it is to do connection limiting. By IP 
would probably work, and if you require pollers to pass a secret which 
you send them in the first "I have mail for you" message, you could 
limit by recipient too.

>> lot less spam. After all, I get a lot less paper junk mail than I get 
>> e-mail spam.
>
> requiring back flips is what's wrong with email.

Well, many ISPs feel that the fact that the cost of spam is shifted to 
them rather than the sender *is* a big problem. Feel free to argue it 
out with them.

>   I'm not trying to be flip here :-), just to point out that I don't 
> see where not charging the sender more than they already pay for 
> bandwidth is an inherent problem.

Well, yes, if we could all be given infinite bandwidth and disk space 
for free, there'd be no need to make anyone pay the cost of anything. 
The problem is that in reality, we pay for bandwidth and disk space. 
Today, spam takes up more MB than my actual e-mail, and it's still 
growing. For the majority of people who are on dialup, getting 3MB of 
spam a day is a bandwidth problem.

>   Charging could be a solution.   But it's not the case that someone 
> made a mistake by not requiring it initially.

Oh, when SMTP was devised, it made perfect sense.

> Anyone who didn't want to lose email?

Tell me, what's your POP3 or IMAP timeout set to? What's your HTTP 
timeout set to?

Didn't think so.

mathew

---

From: Kee Hinckley <[email-masked]>
Date: Fri, 4 Apr 2003 02:04:59 -0500

At 10:06 PM -0500 4/3/03, mathew wrote:
>Umm, yes, and outside of the special case where you're sending 
>e-mail across a corporate network, most e-mail today involves 
>machines controlled by different entities separated by a large 
>amount of internet under control of neither of them. So what is your 
>point?

It's store and forward--so I don't see the network glitches.  On a 
pull system I do.

>>>polling frequency could be anything from every second to never. 
>>>It's a detail to be
>>

The issue is not when.  The issue is that I don't want every person 
who knows someone at my mail server polling my mail server every five 
minutes to see if there is something new.  That doesn't scale.

>>   It is controlled by the receiver, who will of course set the 
>>polling frequency as high as they can, since the cost to *them* is 
>>low.
>
>would probably

That doesn't limit the polls.  It just means that instead of aborting 
them with "no mail" I abort them with "come back later".  And if they 
are well behaved, maybe they'll listen to when I tell them to come 
back.  But remember, we're talking about my fetching email from a 
friend here.  You aren't going to tell me to come back in a 
week--even though that might be how often I get email from my friend. 
And I'm not going to set my poll to a week, because I'm not willing 
to have email from my friend arrive a week late.  So you are 
guaranteed that network traffic will be hire than in the current 
situation.

>>I'm sure that if the sender had to pay we'd see a lot less spam. 
>>I'm also certain that if the sender had to do thirty back flips 
>>before each email message we'd see less spam.  That doesn't mean 
>>that not requiring back flips is what's wrong with email.
>
>Well, many ISPs feel that the fact that the cost of spam is shifted 
>to them rather than the sender *is* a big problem. Feel free to 
>argue it out with them.

Many ISP feel that the fact that they are reimbursed for the cost of 
receiving email is a problem.  Who pays that cost is a separate 
issue.  But I give up--there's no point in continuing this argument.

>>>>email, because it doesn't know whether it's from a spammer or 
>>>>from a flakey ISP.
>>>

Wrong timeout.  We're talking about delivering email.  The correct 
question is, "What's the default timeout currently for most email 
servers before they give up trying to send email."  The typical 
answer is four days.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society

---

From: mathew <[email-masked]>
Date: Fri, 4 Apr 2003 18:44:47 -0500

> It's store and forward--so I don't see the network glitches.  On a 

OK, so you'd rather have mail silently fail to arrive, than know that 
there's an ongoing delivery problem. It didn't occur to me that that 
was what you were arguing for.

> back.

And if they're not, you ban them. Again, these problems have already 
been encountered and solved on the web.

>>>>>>> bandwidth, you can set retrieval timeouts as short as you want 
>>>>>>> to limit your dialup session to, and the CPU and disk overhead 
>>>>>>> of failing to make a connection is minimal. I'd make that payoff 
>>>>>>> to get rid of most spam.
>>>>>> If you can't fetch it because the spammer has an overloaded 
>>>>>> server, which you admit he's sure to have, then you don't see it 
>>>>>> and it doesn't fill up your bandwidth.
> Wrong timeout.  We're talking about delivering email.

Right, and if your POP server is down or overloaded, your mail won't 
get delivered, unless your POP timeout is really really high.

>   The correct question is, "What's the default timeout currently for 
> most email servers before they give up trying to send email."  The 
> typical answer is four days.

That's the limit on retries, not the connection timeout. If you think 
sendmail sits there with a connection waiting to open for four days, 
you're severely deluded.

Remember the context--you were complaining about your client having to 
keep your dialup session connected continuously for days at a time:

>>>>>>> bandwidth, you can set retrieval timeouts as short as you want 
>>>>>>> to limit your dialup session to, and the CPU and disk overhead 
>>>>>>> of failing to make a connection is minimal. I'd make that payoff 
>>>>>>> to get rid of most spam.
>>>>>> If you can't fetch it because the spammer has an overloaded 
>>>>>> server, which you admit he's sure to have, then you don't see it 
>>>>>> and it doesn't fill up your bandwidth.

The retry limit doesn't mean your client sits online for days polling 
and polling non-stop. A smart client will use exponential backoff. Even 
a dumb client will only poll as often as your current e-mail client 
polls for POP or IMAP mail.

mathew

---

From: Kee Hinckley <[email-masked]>
Date: Fri, 4 Apr 2003 22:10:08 -0500

I give up.  The current system has local polling and remote 
send-on-demand.  You  want to replace it with local send-on-demand 
and remote polling.  You don't see to see any problems with this.  I 
have no idea why it's not obvious to you, but I'm not going to put 
any more effort in it.
-- 
Kee Hinckley
http://www.messagefire.com/          Junk-Free Email Filtering
http://commons.somewhere.com/buzz/   Writings on Technology and Society