Subject: [testlist] Duplicate mails

From: "Tom.Petch" <[email-masked]>
Date: Sat, 29 Nov 2008 18:04:32 +0100

I have started getting duplicate mails from the asrg list.  I attach a pair and
they seem to be identical until it gets to the hop
from mail.ietf.org ([64.170.98.32]) to mk-inboundfilter-1-a-1.b2b.uk.tiscali.com
with ESMTP

whereupon one it stamped 18 Nov 2008 01:08:35
and the other 18 Nov 2008 01:08:19.

This pattern, of 15 to 20s apart, is typical.

It does not happen on all asrg posts, typically only on those from Barry Shein
and John Levine, both regular and completely innocent posters to the list.

Any ideas?

Tom Petch

Subject: Re: [Asrg] Email Postage (was Re: FeedBack loops)
From: Barry Shein <[email-masked]>
To: Anti-Spam Research Group - IRTF <[email-masked]>
Cc: [email-masked]
Date: Mon, 17 Nov 2008 20:02:16 -0500
List-Id: Anti-Spam Research Group - IRTF <asrg.irtf.org>

On November 18, 2008 at 00:27 [email-masked] (John Levine) wrote:
 > > > You know that double spending problem I keep whining about?  It kills
 > > > this model dead at any interesting message volume.  Nice try, though.
 > >
 > >Anyhow, again, based on that logic Amazon oughta start offering free
 > >shipping for all orders and just print their own US postage stamps!
 > >
 > >Whaddya think? Is it the trouble to do that which stops them? Or the
 > >fear of getting caught?
 > 
 > Both, I expect.  Oh, and Amazon does print its own postage stamps.

You know what I meant, we were talking about counterfeiting.

We (World) print our own postage stamps too! On a legal meter.

 > On the other hand, I would think it was painfully obvious to anyone
 > who thought about it for a few minutes that if spam were all sent by
 > large easily identifiable organizations such as Amazon, it would be a
 > lot easier to deal with.

Ya know, the "painfully obvious" rhetorical ploy is starting to sound
like teens who say "ya know".

What should be painfully obvious is that I've said about a dozen times
now that the goal here is to treat all bulk emailers alike in an
attempt to create an economic incentive.

That is, dissolve the very notion of spam. You can only really have
"spammers" as we commonly see them in a system which is fundamentally
broken.

That is, give stuff away for free and don't be surprised that you have
to institute rationing.

But, of course, the problem then is how do we pay for the ration
system and its enforcement if it's all free?

Most free (on a per-use basis) things require govt enforcement to
prevent abuse. For example, national parks or highways. Which means
taxation. But no one is offering to tax the internet and return those
taxes to an enforcement agency analogous to state highway patrols or
park rangers.

One analogy I gave earlier was how the music biz was very economically
incentivized to stop all kinds of piracy and threw a lot of firepower
(i.e., $$$) at it. And most of that well within the timeframe of the
spam problem.

But email abuse, email being free, remains mostly in the realm of
volunteers. And not many at that.

Here's a dichotomy:

How much money is spent by legitimate companies blocking incoming spam
versus getting their own email through the spam-laden cloud?

My guess would be "huh? through the spam-laden...um, nothing really,
we fire and forget mostly other than a quick scan to try to avoid
really common filtering triggers."

Take this group as another example. How much effort in the mechanics
of distributing this group is spent on avoiding incoming abuse, versus
evading outgoing (delivery) problems?

Well, I realize it's not zero due to the smallish size of the group,
there's some stuff about N bounces and you're nomail and mailman might
attract someone's attention.

But do you see the asymmetry I'm getting at?

Most of the effort is avoiding unwanted incoming.

Now take the music industry. They have both. They pursue piracy, but
they also pursue online vending like iTunes. They have an economy, one
supports the other.

   AGAIN: I am REALLY not inviting a flamefest regarding RIAA et al
   and their tactics. As I said earlier a lot of what they were after
   was egregious and justified if you believe anyone has a right to
   any money at all, "offshore" sites offering entire tens of
   thousands of albums for a dollar each because they were simply
   pirated. Some of their stuff was over the top. But at least they
   were trying and making some unmistakeable progress even if you're
   uncomfortable with it. Now compare and contrast with spam...

 > >So what could trying to think of a new approach possibly distract
 > >from?
 > 
 > New approaches would be great.  Recycling old ideas that are known not
 > to work and telling people to do something different when they point
 > that out, yet never having anything different to suggest is not so
 > great.

I guess I don't accept the "known not to work" part.

I think we've somehow gone from a period when most people weren't very
aware of spam so XYZ won't work to a period where the things which
wouldn't work are now just old-hat.

Let's be honest, thus far, NOTHING works.

Maybe we should have a "spam appreciation" day and urge everyone to
shut off all the filters for 24 hours and see how much difference it
makes really.

To the end-users? I'd guess their spam load would go up some but
they're pretty much deluged already.

ISPs and other centralized resource managers would probably suffer
more because a lot of what they block isn't going into anyone's mail
box ever, for example all those dictionary attacks and "no such user
floods". That'd strain bandwidth etc but ultimately have little effect
on end-users.

Anyhow, as far as I can tell the tone hasn't really changed from my
very first posting on the subject...it can't work...here's my straw
man reason why...etc etc etc. Then it sort of strayed to "ok, maybe
that's not why it won't work, but I know it won't work anyhow!"

And now we're heading straight to: I am losing patience making up
straw men as to why it won't work...IT WON'T WORK SO STFU OK?

Well, actually it has changed a little, I think some got tired enough
of the instaneous rejections and mob mentality that they began to
think about the possibility in a more positive way.

As the song goes, money makes the world go 'round.

-- 
        -Barry Shein

The World              | [email-masked]           | http://www.TheWorld.com
Purveyors to the Trade | Voice: 800-THE-WRLD        | Login: Nationwide
Software Tool & Die    | Public Access Internet     | SINCE 1989     *oo*
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

Subject: Re: [Asrg] Email Postage (was Re: FeedBack loops)
From: Barry Shein <[email-masked]>
To: Anti-Spam Research Group - IRTF <[email-masked]>
Cc: [email-masked]
Date: Mon, 17 Nov 2008 20:02:16 -0500
List-Id: Anti-Spam Research Group - IRTF <asrg.irtf.org>

On November 18, 2008 at 00:27 [email-masked] (John Levine) wrote:
 > > > You know that double spending problem I keep whining about?  It kills
 > > > this model dead at any interesting message volume.  Nice try, though.
 > >
 > >Anyhow, again, based on that logic Amazon oughta start offering free
 > >shipping for all orders and just print their own US postage stamps!
 > >
 > >Whaddya think? Is it the trouble to do that which stops them? Or the
 > >fear of getting caught?
 > 
 > Both, I expect.  Oh, and Amazon does print its own postage stamps.

You know what I meant, we were talking about counterfeiting.

We (World) print our own postage stamps too! On a legal meter.

 > On the other hand, I would think it was painfully obvious to anyone
 > who thought about it for a few minutes that if spam were all sent by
 > large easily identifiable organizations such as Amazon, it would be a
 > lot easier to deal with.

Ya know, the "painfully obvious" rhetorical ploy is starting to sound
like teens who say "ya know".

What should be painfully obvious is that I've said about a dozen times
now that the goal here is to treat all bulk emailers alike in an
attempt to create an economic incentive.

That is, dissolve the very notion of spam. You can only really have
"spammers" as we commonly see them in a system which is fundamentally
broken.

That is, give stuff away for free and don't be surprised that you have
to institute rationing.

But, of course, the problem then is how do we pay for the ration
system and its enforcement if it's all free?

Most free (on a per-use basis) things require govt enforcement to
prevent abuse. For example, national parks or highways. Which means
taxation. But no one is offering to tax the internet and return those
taxes to an enforcement agency analogous to state highway patrols or
park rangers.

One analogy I gave earlier was how the music biz was very economically
incentivized to stop all kinds of piracy and threw a lot of firepower
(i.e., $$$) at it. And most of that well within the timeframe of the
spam problem.

But email abuse, email being free, remains mostly in the realm of
volunteers. And not many at that.

Here's a dichotomy:

How much money is spent by legitimate companies blocking incoming spam
versus getting their own email through the spam-laden cloud?

My guess would be "huh? through the spam-laden...um, nothing really,
we fire and forget mostly other than a quick scan to try to avoid
really common filtering triggers."

Take this group as another example. How much effort in the mechanics
of distributing this group is spent on avoiding incoming abuse, versus
evading outgoing (delivery) problems?

Well, I realize it's not zero due to the smallish size of the group,
there's some stuff about N bounces and you're nomail and mailman might
attract someone's attention.

But do you see the asymmetry I'm getting at?

Most of the effort is avoiding unwanted incoming.

Now take the music industry. They have both. They pursue piracy, but
they also pursue online vending like iTunes. They have an economy, one
supports the other.

   AGAIN: I am REALLY not inviting a flamefest regarding RIAA et al
   and their tactics. As I said earlier a lot of what they were after
   was egregious and justified if you believe anyone has a right to
   any money at all, "offshore" sites offering entire tens of
   thousands of albums for a dollar each because they were simply
   pirated. Some of their stuff was over the top. But at least they
   were trying and making some unmistakeable progress even if you're
   uncomfortable with it. Now compare and contrast with spam...

 > >So what could trying to think of a new approach possibly distract
 > >from?
 > 
 > New approaches would be great.  Recycling old ideas that are known not
 > to work and telling people to do something different when they point
 > that out, yet never having anything different to suggest is not so
 > great.

I guess I don't accept the "known not to work" part.

I think we've somehow gone from a period when most people weren't very
aware of spam so XYZ won't work to a period where the things which
wouldn't work are now just old-hat.

Let's be honest, thus far, NOTHING works.

Maybe we should have a "spam appreciation" day and urge everyone to
shut off all the filters for 24 hours and see how much difference it
makes really.

To the end-users? I'd guess their spam load would go up some but
they're pretty much deluged already.

ISPs and other centralized resource managers would probably suffer
more because a lot of what they block isn't going into anyone's mail
box ever, for example all those dictionary attacks and "no such user
floods". That'd strain bandwidth etc but ultimately have little effect
on end-users.

Anyhow, as far as I can tell the tone hasn't really changed from my
very first posting on the subject...it can't work...here's my straw
man reason why...etc etc etc. Then it sort of strayed to "ok, maybe
that's not why it won't work, but I know it won't work anyhow!"

And now we're heading straight to: I am losing patience making up
straw men as to why it won't work...IT WON'T WORK SO STFU OK?

Well, actually it has changed a little, I think some got tired enough
of the instaneous rejections and mob mentality that they began to
think about the possibility in a more positive way.

As the song goes, money makes the world go 'round.

-- 
        -Barry Shein

The World              | [email-masked]           | http://www.TheWorld.com
Purveyors to the Trade | Voice: 800-THE-WRLD        | Login: Nationwide
Software Tool & Die    | Public Access Internet     | SINCE 1989     *oo*
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

_______________________________________________
Testlist mailing list
[email-masked]
https://mail.ietf.org/mailman/listinfo/testlist

---

From: James M Galvin <[email-masked]>
Date: Sat, 29 Nov 2008 15:03:15 -0500

The reason for the duplicate is because the message was sent twice.

The asrg list "moved" from ietf.org to irtf.org.  In real life the 
hosts are the same, i.e., one host with two names.

For a short while after the cut-over occurred there were some folks who 
were sending messages to both addresses:

    [email-masked]
    [email-masked]

Which is the case in the two messages you forwarded.

John Levine sent a note about this to the asrg list eventually, and the 
duplicates have trailed off as soon as folks stopped sending their 
messages to both addresses.

Double check your other messages to see if the duplicates are the 
result of the "duplicate" recipient.

Jim

-- On November 29, 2008 6:04:32 PM +0100 "Tom.Petch" 
<[email-masked]> wrote regarding [testlist] Duplicate mails --

> I have started getting duplicate mails from the asrg list.  I attach
> a pair and they seem to be identical until it gets to the hop
> from mail.ietf.org ([64.170.98.32]) to
> mk-inboundfilter-1-a-1.b2b.uk.tiscali.com with ESMTP
>
> It does not happen on all asrg posts, typically only on those from
> Barry Shein and John Levine, both regular and completely innocent
> posters to the list.
>
> Any ideas?
>
> Tom Petch

---

From: John Levine <[email-masked]>
Date: 29 Nov 2008 21:55:15 -0000

>    [email-masked]

Right.  I have a script that sets up the headers for outgoing list
mail, and it got confused by the switch from ietf to irtf.  Now I've
fixed it so you shouldn't be seeing dups from me any more.

On the other hand, I don't understand why mailman's dup detection isn't
catching them.  Both copies have the same message-ID, after all.

R's,
John

---

From: James M Galvin <[email-masked]>
Date: Sun, 30 Nov 2008 00:06:52 -0500

-- On November 29, 2008 9:55:15 PM +0000 John Levine <[email-masked]> 
wrote regarding Re: [testlist] Duplicate mails --

> On the other hand, I don't understand why mailman's dup detection
> isn't catching them.  Both copies have the same message-ID, after all.

I thought Mailman's dup detection was about not sending a subscriber a 
copy of the message if that same email address was in the To or Cc list 
for the message.

I've never actually looked at the code but that is the behavior I've 
seen.  I've never seen anything to suggest that Mailman checks the 
message-id.  Also, I've seen loops of messages with Mailman, although 
not too recently so perhaps the behavior has changed?

Jim

---

From: Glen <[email-masked]>
Date: Sun, 30 Nov 2008 13:18:11 -0800

Jim is correct.

Mailman seems to be woefully lacking in documentation - the curse of
many opensource projects, alas! - but the most relevant thing I found
was in the IETF's own archives:

http://www.ietf.org/mail-archive/web/ietf/current/msg37128.html

This matches what Jim says pretty directly.

Glen