Subject: [Asrg] ipv6 macro expansion example in SPF specification, DNS ranges...

From: David Nicol <[email-masked]>
Date: Fri, 21 Jan 2011 11:33:09 -0600

IPv6:
   %{ir}.%{v}._spf.%{d2}
   1.0.B.C.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.B.D.0.1.0.0.2.ip6._spf.example.com

cached NXDOMAIN suppressing subdomain lookup attempts seems like the
way to go, if one refers to a DNSWL in a SPF record.

---

From: John Levine <[email-masked]>
Date: 22 Jan 2011 02:11:39 -0000

>way to go, if one refers to a DNSWL in a SPF record.

Are you assuming that whatever does the SPF lookups remembers all of
its previous queries?

R's,
John

---

From: David Nicol <[email-masked]>
Date: Fri, 21 Jan 2011 21:43:36 -0600

that the cache would. The special strict NXDOMAIN cache for use by
DNSxL clients.

On Fri, Jan 21, 2011 at 8:11 PM, John Levine <[email-masked]> wrote:
> _______________________________________________
> Asrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/asrg
>

-- 
“The aeroplane is fatally defective. It is merely a toy—a sporting
play-thing.  It can never become commercially practical." -- Nikola
Tesla

---

From: John Levine <[email-masked]>
Date: 22 Jan 2011 04:29:17 -0000

>that the cache would. The special strict NXDOMAIN cache for use by

So you're expecting something other than a normal DNS cache that
exists now.

If you're doing a special purpose cache, why not do a smarter protocol
between the cache and the master server?  As far as the client's
concerned, once you need anything other than a vanilla DNS setup, it's
all new special purpose code anyway.

R's,
John

---

From: David Nicol <[email-masked]>
Date: Sat, 22 Jan 2011 19:51:00 -0600

> So you're expecting something other than a normal DNS cache that

Small patch to existing cache.

> all new special purpose code anyway.

How would one represent "e-mail may appear from sources appearing in
this whitelist" in an SPF record, with your b-tree proposal?
Popularize the b-tree protocol, then have a way to refer to it
included in SPFv3?

---

From: John Levine <[email-masked]>
Date: 23 Jan 2011 02:30:57 -0000

>Small patch to existing cache.

Hmmn.  As a proof of concept, how about making the appropriate changes
to BIND 9.7 and send us the patch file.  Or for that matter do it to
dnscache in djbdns, so you only have to look at 12K lines of code
rather than 400K lines.

>this whitelist" in an SPF record, with your b-tree proposal?

My suggestion is that if you find SPF to be useful and you want a
macro that means look up an address in a DNSxL, you should add a macro
that means look up this address in that DNSxL, rather than trying to
do it indirectly with %{ir} or whatever.  Since the installed base of
IPv6 whitelists and blacklists is nonexistent, the backward
compatibility problem is limited.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Mon, 24 Jan 2011 11:52:28 -0800

On 1/22/11 5:51 PM, David Nicol wrote:
> On Fri, Jan 21, 2011 at 10:29 PM, John Levine<[email-masked]>  wrote:
There was general consensus at MAAWG, SPF does not serve as a basis for 
mitigating spam.  Use of SPF was limited to determining which reputation 
services currently list an SPF authorized IP address when handling 
complaints, or whether the authorized IP address can be included in 
feedback.  This expects SPF records will resolve specific IP addresses.  
This is not necessarily true, nor is it always safe to assume a domain 
controls the authorized IP addresses.

When SMTP Authentication becomes possible with DANE resource records, 
this will remove a need to collect sets of DNS resource records 
representing both authorized v4 and v6 IP addresses.  Many records sets 
that only list v4 already have reached their "mechanism" limits.  DANE 
based authentication would provide a safer basis for directing feedback 
as well.  Unfortunately, it is not practical to list all possible v6 
addresses or domain names that might instigate abuse.  Defensive 
strategies must develop a generous list of "conforming" domains instead.

Since billions of new domains are used every day by malefactors, neither 
the possession of a domain, nor a v6 IP address, offers a safe basis for 
acceptance when screened with negative reputations unable to keep pace.  
Efforts that attempt to relate domains to IP addresses seen by targets 
to confirm legitimate use of a domain are also likely to prove 
problematic, which leaves cryptographic authentication. In addition, SPF 
may support a denial of service attack with its processing of encoded 
macros based upon email address local-parts.

Cryptographic authentication will not threaten DNS, and when based upon 
"conforming" domains, is unlikely to serve as a denial of service 
mechanism.  Of course, DNS already handles domain names effectively 
without a need to redesign the query mechanism to obtain which domains 
"conform" to rules of reasonable behavior.  It is also likely there will 
be a need to authorize other domains to better ensure acceptance of 
third-party transactions.

-Doug

---

From: Dotzero <[email-masked]>
Date: Mon, 24 Jan 2011 15:53:13 -0500

> controls the authorized IP addresses.

Doug, I'm not sure that you are correct when you state that there was
a consensus for what you describe above. There WAS a consensus that
many mailbox providers are not using SPF exactly the way that many
originally expected.

For example, many do not automatically use a hardfail as a basis to
reject mail or discard. The breakage on forwarded/relayed/vanity/lists
is in many cases too painful for both sending domains and mailbox
providers (think complaint handling/customer service costs).

Instead, many mailbox providers are considering SPF results in
conjunction with DKIM results in a combination whitelist/blacklist
approach. For example, if I get an SPF pass on a record that ends in
-all and/or I get a DKIM pass on a first party signature then I have a
high confidence level as to where the mail came from at either the
transport layer or the message layer. If I have both then I have an
even higher confidence level. This does not speak to the "goodness" or
badness" of the domain, only relationship to the domain.

On the other hand, if a domain is publishing a record ending in a -all
AND consistently DKIM signing (even in the absense of an ADSP record
although an ALL would be useful), we can have a high confidence level
of minimal false positives (that is, legitimate mail that is rejected
or discarded) if we throw away mail that fails both SPF and DKIM
checks.

This does not necessarily presume that a mailbox provider is looking
at external reputation services.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 24 Jan 2011 14:37:55 -0800

On 1/24/11 12:53 PM, Dotzero wrote:
> On Mon, Jan 24, 2011 at 2:52 PM, Douglas Otis<[email-masked]>  wrote:
Mike,

An SPF failure can not be trusted to be an indicator of spam.  DKIM 
signing is almost never assured, especially when handled by third-party 
services.  As such, these mechanisms failing alone or together still do 
not offer a safe basis for rejection.  Of course both passing means 
nothing as well.

-Doug

---

From: Dotzero <[email-masked]>
Date: Mon, 24 Jan 2011 21:14:59 -0500

On Mon, Jan 24, 2011 at 5:37 PM, Douglas Otis <[email-masked]> wrote:
> An SPF failure can not be trusted to be an indicator of spam.  DKIM signing
> is almost never assured, especially when handled by third-party services.
>  As such, these mechanisms failing alone or together still do not offer a
> safe basis for rejection.  Of course both passing means nothing as well.
>

Doug, there are plenty of people with real world operational
experience that would disagree with you.  You state that failing means
nothing and passing means nothing. If that is true, why are there a
significant number of implementers using this approach successfully?

---

From: Paul Ferguson <[email-masked]>
Date: Mon, 24 Jan 2011 18:21:11 -0800

> significant number of implementers using this approach successfully?

Hi Mike,

While I'm just catching up on this thread, I can also chime in that --
as far as I can tell, neither spf or dkim has had any impact on spam,
at least from what I can determine.

If that was the original intent of spf and/or dkim, then I would be
inclined to say that neither have properly addressed the spam problem.

$.02,

- ferg

-- 
"Fergie", a.k.a. Paul Ferguson
 Engineering Architecture for the Internet
 fergdawgster(at)gmail.com
 ferg's tech blog: http://fergdawg.blogspot.com/

---

From: John Levine <[email-masked]>
Date: 25 Jan 2011 03:07:07 -0000

>inclined to say that neither have properly addressed the spam problem.

I can't speak for the SPF crowd, but the goal of DKIM is to make it
easier to recognize mail from people you know, for some definition
of "people" and "know" which should make filtering more accurate.

It's also alleged to be useful to deter forgery, although I happen
to think that any forgery it deters is already covered under the
category of mail from people you don't know.

The only thing that's going to make the amount of spam drop is major
legal and technical advances that get rid of botnets.  Don't hold your
breath.

R's,
John

---

From: Dotzero <[email-masked]>
Date: Mon, 24 Jan 2011 22:10:02 -0500

> inclined to say that neither have properly addressed the spam problem.

Hey Paul,

Thanks for chiming in. As I pointed out in my original response to
Doug, I inicated that in and of themselves, SPF or DKIM
succeses/failures do not tell you about whether a message is SPAM.
What they do allow you to know is whether that message is tied to a
particular IP Address or Domain. The best description might be that
they are enabling technologies.

For domains that have good control over their mail flows (which
mailbox providers can determine for themselves without relying on 3rd
party reputation services), my experience and the experience of others
is that one can (for example) recommend to mailbox providers that they
can drop mail on a double fail (SPF/DKIM) with very little risk of
dropping legitimate mail. Because each of these approaches break in
tehir own ways, they are actually somewhat complimentary. This does
have the side effect of reducing SPAM purporting to be from those
domains at mailbox providers using this approach. There are other
permutations/combinations that allow varying degrees of whitelisting
(confidence) or skepticism that the message really did/did not come
from the purported domain.

Without naming any names, I am comfortable stating that a number of
financial organizations have been working on this sort of
implementation in conjunction with various large receivers with
successful outcomes. I know that folks from some of those
organizations lurk here (jump right in, the water's fine). The focus
is more on phishing attacks and less on generic spam.

Part of the issue that has been percolating for a while is that most
of the interaction has been through private peering arrangements
rather than through open standards that allow anyone to play. This
also means that the data isn't publicly available. I (and others) get
feeds on authentication failures from various receivers. Some of the
senders are comparing notes and we are seeing similar outcomes. Again,
mileage may vary based on the specific mail flows and the control
exercised by those controlling the domain.

Unfortunately, because the data is being shared under contractual
arrangements you just aren't going to see detailed data being
disclosed in public forums.  Particularly sensitive data would be data
on individual message authentication failures provided in ARF format.
The best I have seen is including "cleansed" headers (RFC5322 To and
Return Path redacted) with links from the message body included. At
least for me, if I remove from the dataset our own legitimate links (a
relatively small number), all I am left with is maliciousness. I guess
the crude way to put it is that nobody is accidently purporting to
send mail using our domains and accidently putting in links. (This
case would be described as DKIM None and SPF fail.) If you want to
hook up during RSA we can share notes. I could also make an
introduction to some folks (I know some of them you already know) who
could be in a position to make much more extensive data sets available
to you.

I will qualify my statements by pointing out that I play in the space
known as potentially highly abused brands. Mileage may vary
significantly for domains that allow their users to send mail from
whatever mail server is handy.

This of course does not address the issue of subversion of the
emitting domains systems which would be problematic under any
scenario.

To sum up, this approach works well for some domains but is not a
general solution to the problem of SPAM (or phishing). To the extent
someone suceeds in controlling abuse against domains their work, the
abusers simply move on to abuse another domain which is not as
aggressively defended.

I think the difference between what Doug is saying and what I am
saying is that he is only willing to accept a solution which is works
across the board.

 I'll take the wins where I can get them and accept that the abusers
will go commit their abuse somewhere else. From your perspective, this
approach may not be particularly useful because it might be considered
a wash (substitute one abused domain for another). For a mailbox
provider/ISP this approach is one more tool in their toolkit.

What can I say, Darwin was right.

I think we can also all agree that there are no magic bullets.

In closing, the reason I'm saying things on the list that are directed
to you is that I think it is useful for the wider group to understand
that data is out there even if the detailed data is not openly
available.

Mike

---

From: Douglas Otis <[email-masked]>
Date: Mon, 24 Jan 2011 19:24:49 -0800

> significant number of implementers using this approach successfully?
Defeating spam requires the reputation of SMTP clients be weighed for 
rejection or acceptance!  SPF failures say little about an SMTP client.  
DKIM failures also say little about the SMTP client because either 
mechanism MUST be allowed to fail to retain email delivery integrity.  
When failure of SPF or DKIM offers scant basis for judging an SMTP 
client, they are useless as a mitigation tool.  Ipso facto, their 
passing therefore provides little meaning as well, since mitigation must 
be based upon reliable mechanisms.  Overlapping results must be 
considered a mere distraction from what is needed to mitigate spam.  In 
rare cases, DKIM may play a role in preventing spoofing, but this can 
not be considered a significant component of spam mitigation.

-Doug

---

From: Dotzero <[email-masked]>
Date: Mon, 24 Jan 2011 22:55:00 -0500

On Mon, Jan 24, 2011 at 10:24 PM, Douglas Otis <[email-masked]> wrote:
>>> > signing
> rejection or acceptance!

Doug, could you share with us what reputation is? As far the mailbox
providers are concerned (that I have spoken with) it is reduced to
"What have you done to me today". Their systems don't care whether you
have had a good reputation for the past 2 years. If you start spewing
badness today and you start generating complaints today then you will
be throttled or blocked. If the time interval of reputation is that
short then reputation systems are not particularly useful as an
absolute requirement for defeating SPAM.

>SPF failures say little about an SMTP client.

They may or may not say something about the client in and of itself
but they do say something  about the relationship of the SMTP client
to the purported Mail From domain when a well constructed SPF record
ends in a -ALL

> mechanism MUST be allowed to fail to retain email delivery integrity.

A purist always brings a smile to my face. I know quite a few folks at
large receivers who would disagree with your "MUST". What they are
concerned about is ensuring the delivery integrity of mail their users
want. As far as delivery integrity of mail their users don't
want.....not so much. And that includes marketing emails their users
don't want. The issue for them is not generic email delivery integrity
but rather, how to find the sweet spot of
rejecting/blocking/discarding while minimizing (not eliminating every
single instance) performing such actions on mail their users actually
want.

>When
> failure of SPF or DKIM offers scant basis for judging an SMTP client, they
> are useless as a mitigation tool.  Ipso facto, their passing therefore
> provides little meaning as well, since mitigation must be based upon
> reliable mechanisms.

Reliability is not a generic. It is a measure. One mailbox provider
may choose a 99.99% reliability level and it's users may find that
acceptable. Another may choose 99.999% reliability and it's users
might find that acceptable. It may be that the mailbox provider
segregates various inbound mail streams in making these
determinations.

Here is a truth for you. Mailbox providers will determine for
themselves what works for them irrespective of any "MUST" you seek to
impose on them. They will use standards based approaches and they will
use non-standards based approaches. They will discard mail in a manner
that offends your sensibilities. At the end of the day it is not about
you....it is about them and their relationship with their customers.

>Overlapping results must be considered a mere
> distraction from what is needed to mitigate spam.  In rare cases, DKIM may
> play a role in preventing spoofing, but this can not be considered a
> significant component of spam mitigation.
>

"cannot" is a very strong word in the manner that you use it.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 25 Jan 2011 16:27:43 +0100

On 25/Jan/11 04:07, John Levine wrote:
> I can't speak for the SPF crowd

Why not?  SPF users are just a subset of SMTP users, and hence spam
victims, who thus deserve being considered by the ASRG.

> the goal of DKIM is to make it easier to recognize mail from people
> you know, for some definition of "people" and "know" which should
> make filtering more accurate.

The same is obviously true for SPF.  I concur they are both enabling
authentication technologies.

> legal and technical advances that get rid of botnets.  Don't hold your

Isn't that an overly renunciative approach to reputation systems?

---

From: Dotzero <[email-masked]>
Date: Tue, 25 Jan 2011 10:32:15 -0500

> legal and technical advances that get rid of botnets.  Don't hold your

The only thing that's going to make the amount of spam drop is heads
on pikes. Don't hold your breath.

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 25 Jan 2011 11:40:20 -0500

> inclined to say that neither have properly addressed the spam problem.

I emphatically concur: SPF and DKIM have no anti-spam value.

(And given that SPF was introduced with the thus-far unretracted claim:
	"Spam as a technical problem is solved by SPF."
I think it's quite fair to say that its original intent was anti-spam.)

As to whether has an impact on forgeries: that depends on what one
considers a "forgery".  My own view that is that both (as well as any
other similar schemes) *may* have some applicability on a local level,
but not at the scale of the entire Internet.  And that's not because of
any inherent issues with them: the problem is insoluble at the moment
no matter what technology is used, because of the underlying persistent
and systemic security issues.  But as I said, this depends on what one
considers a "forgery" and those using different definitions than mine
may find DKIM useful in that context.

---rsk

---

From: Douglas Otis <[email-masked]>
Date: Tue, 25 Jan 2011 11:29:48 -0800

On 1/24/11 7:55 PM, Dotzero wrote:
> > Defeating spam requires the reputation of SMTP clients be weighed
> > for rejection or acceptance!
>
>  Doug, could you share with us what reputation is?

Reputation Definition:  An assessment of an identifier that accurately 
reflects the behavior of the resource administrator.  For example, these 
identifiers might be the address of an SMTP client or the AS of the 
network provider.

SPF represents a poor choice.  SPF represents an unreliable 
authorization mechanism that also fails at isolating behaviors of 
resource administrators.  Those who attempt to make use of this 
mechanism often weaken the integrity of email delivery.

>  As far the mailbox providers are concerned (that I have spoken with)
>  it is reduced to "What have you done to me today".

Sorry, but this statement is vague, as is SPF.  No one knows how SPF 
might be used.  A PASS used as a basis to verify an unseen Sender-ID PRA 
header or Return-Path parameter can be highly deceptive and even place 
recipients at risk.  Malefactors can authorize anyone's resources, where 
macros contained in SPF can also obfuscate their breath.

>  Their systems don't care whether you have had a good reputation for
>  the past 2 years. If you start spewing badness today and you start
>  generating complaints today then you will be throttled or blocked.

SPF does not help with this problem, nor can DKIM be safely used in this 
manner, as it is designed to permit replays.  See Reputation Definition.

>  If the time interval of reputation is that short then reputation
>  systems are not particularly useful as an absolute requirement for
>  defeating SPAM.

Systems are often in transition from compromised to cleaned.  As such, 
effective SPAM mitigation MUST be responsive, either by actions of 
reputation services, or by resource administrators.  A responsive 
resource administrator represents by far the most effective actor in 
SPAM mitigation.

>  to the purported Mail From domain when a well constructed SPF record

An "-all" suffix means what?  Should MTAs refuse a message that fails a 
check that may require 111 transactions to verify?  An SMTP client 
issuing a message by properly carrying forward the Return-Path to ensure 
email delivery integrity may cause this check to fail and has done 
nothing wrong!

>  A purist always brings a smile to my face. I know quite a few folks
>  at large receivers who would disagree with your "MUST".

Are you saying that all SPF failures MAY be refused?

>  What they
>  are concerned about is ensuring the delivery integrity of mail their
>  users want. As far as delivery integrity of mail their users don't
>  don't want. The issue for them is not generic email delivery
>  integrity but rather, how to find the sweet spot of
>  rejecting/blocking/discarding while minimizing (not eliminating
>  every single instance) performing such actions on mail their users
>  actually want.
>
>  themselves what works for them irrespective of any "MUST" you seek
>  to impose on them. They will use standards based approaches and they
>  will use non-standards based approaches. They will discard mail in a
>  manner that offends your sensibilities. At the end of the day it is
>  not about you....it is about them and their relationship with their
>  customers.

Attempts to divine a recipient's desire has little to do with SPF.  Are 
you suggesting all failures be rejected, or all passes be accepted?  Any 
provider that hopes to meet the desires of their customers MUST look 
elsewhere to determine proper message handling.   For example, I have 
advocated use of a third-party authorization scheme to assist with this 
process.

-Doug

---

From: David Nicol <[email-masked]>
Date: Tue, 25 Jan 2011 15:02:18 -0600

On Tue, Jan 25, 2011 at 1:29 PM, Douglas Otis <[email-masked]> wrote:

> check that may require 111 transactions to verify?  An SMTP client issuing a
> message by properly carrying forward the Return-Path to ensure email
> delivery integrity may cause this check to fail and has done nothing wrong!

yeah, SPF presumes that everyone will play by the SPF rules, in which
case carrying forward a return path is no longer proper. (he said,
taking the bait.)

---

From: John Leslie <[email-masked]>
Date: Tue, 25 Jan 2011 16:14:57 -0500

Dotzero <[email-masked]> wrote:
>>> experience that would disagree with you. ?You state that failing means

   Lack of better alternatives...

> Doug, could you share with us what reputation is?

   Reputation (as the name implies) is a prediction of the likelihood of
near-future behavior.

> it is reduced to "What have you done to me today".

   Perhaps... if nothing better is available...

   The real question for a mailbox-provider is, "How much does the
mailbox customer want to see this?"

   For practical reasons, it's usually reduced to, "How much does my
typical mailbox customer want to see this?" And that value is arbitrarily
cloe to zero for email that's part of a spam run. So once you have
identified a spam rum, you're likely to want to reject it all; and once
the spam run is ended, you're likely willing to let email through.

> the past 2 years.

   As a practical matter, a 2-year-old good reputation doesn't say you
won't be compromised and start spewing spam.

> If you start spewing badness today and you start generating complaints
> today then you will be throttled or blocked.

   Reputation systems, in principle, could have one-second update cycles.
They could, in principle, differentiate the end of a single spam run on
a botted computer from the computer being no longer botted.

   Understandably, the "free" reputation services don't attempt that.
But that's a business model: not an inherent limitation.

> systems are not particularly useful as an absolute requirement for

   Day-old reputation information is still _somewhat_ useful. And, IMHO,
reputation _is_ an absolute requirement for defeating SPAM.

   We do need to add something equivalent to "incident in progress,"
where SPAM is likely to be interspersed with HAM, but there's reason
to believe the SPAM will be purged from retry queues (and how long
that should take).

   That's venturing into territory which presents challenges gathering
the information, absent what I call "vouching" services which are
given information about policies and procedures. The vouching services
(if trustworthy) can tell reputation services the estimated time to
repair a breach. (There are also a number of questions of trust, which
require customer relationships: for example, how much needs to be
sanitized in a SPAM report.)

   So, IMHO, _standalone_ reputation services are necessary but far
from sufficient for defeating SPAM.

--
John Leslie <[email-masked]>

---

From: Paul Ferguson <[email-masked]>
Date: Tue, 25 Jan 2011 13:16:54 -0800

>   Reputation (as the name implies) is a prediction of the likelihood of

...based on previously observed behavior.

- ferg

---

From: Dotzero <[email-masked]>
Date: Tue, 25 Jan 2011 20:06:03 -0500

> ...based on previously observed behavior.

So, what exactly does this mean when behavior suddenly changes? If a
domain or IP address (was well behaved yesterday) but begins spewing
badness today, what will your company do as an arbiter of whether mail
is accepted by your customers? Will you allow that domain or IP
address to spew badness? I highly doubt it. As some point once the
spewing has subsided you may (automatically or manually) again start
allowing traffic through from that domain or IP address. But that
isn't really reputation in the traditional sense of the word.

But that brings me back to my original question. If reputation doesn't
prevent a site from getting throttled or blocked when it goes bad,
what does reputation mean? It doesn't particularly protect the site
from the immediate consequences of going bad. It appears that the
responses are authoritative (this domain or IP is currently emitting
badness) rather than reputational (this site has a good reputation so
I will accept badness from it on the presumption they are going to
address it). I will grant that there may be some small slack cut based
on reputation but does it really extend that far?

Set aside whether you have personal contacts with someone at that
domain or ASN. That approach doesn't scale from either direction.

---

From: Paul Ferguson <[email-masked]>
Date: Tue, 25 Jan 2011 17:18:31 -0800

Hi Mike,

if you recall my presentation at ISOI 8 in Pasadena [1], I gave a
presentation (specifically) on domain reputation, followed by Eric
Ziegast of ISC on DNS RPZ [2].

One of the main points I was trying to make in that presentation is
that there should be specific distinctions made between reputation
"policy" and reputation "technical implementation". Each individual
organization -- both the ones that provide the "reputation", and the
ones that decide to "implement" it -- should determine for themselves
what their policies are. The technical implementation should, in my
opinion, should be "standardized" (for whatever your definition of
standardized may be), such that many different policies can be
implemented in the same fashion.

That's one reason I'm a big fan of DNS RPZ for implementing domain
reputation -- each implementing organization can choose for themselves
which reputation providers they choose to implement (e.g. Spamhaus,
SURBL, etc.).

Let's not get sucked in to a long drawn-out discussion of policy. It
generally spirals into a meaningless quagmire. :-)

- ferg

p.s. Of course, this doesn't necessarily address spam, per se, but
more broadly the distinction between policy & technical
implementation.

[1] http://www.isotf.org/isoi8.html
[2] http://www.isc.org/community/blog/201007/taking-back-dns-0

---

From: Steve Atkins <[email-masked]>
Date: Tue, 25 Jan 2011 17:31:05 -0800

On Jan 25, 2011, at 5:06 PM, Dotzero wrote:

> address to spew badness?

"badness" is hard to measure at scale once you've removed the
obvious botnet spew from your mailstream.

> I highly doubt it.

Depends. If it's sending a lot of wanted email, a spike of obviously
unwanted email and a middle ground of mail which you can't
decide about you're always going to want to deliver the wanted
email, and you're always going to want to block the obviously
unwanted email.

But the mail in the middle is harder to decide what to do about,
and that's where sender reputation helps.

> As some point once the
> what does reputation mean?

Reputation is, loosely, the past history of the sender from a decade
ago through to a second ago. It's not a simple integer (0=bad, 100=good)
however much people want to map it onto that. It includes, at least,
traffic volumes over time and fraction of email that was wanted vs
not wanted over time.

Comparing the past history of the sender (over a period of months)
to the current behaviour of the sender recently (minutes to hours) can
help you guess what the sender is up to, categorize them into one
of several "bins" automatically and treat mail from them appropriately.

If a sender has a history of not sending any email at all and you suddenly
see a lot of email from it then you can categorize it as "probably a
compromised end-user machine".

If it has a history of sending 1000 emails a day and it suddenly starts
spewing 100k then the change in behavior lets you categorize it
as "tiny smarthost with compromised system" and maybe block it outright.

But if a site has a history of sending you large volumes of wanted
email over a long period, and you suddenly see a spike of unwanted
email then you're likely to assume that it's a transient problem (bad
customer, perhaps) and that once you've notified them of the problem,
they'll fix it. Meanwhile you'll keep delivering most of the email from
them, on the assumption that it's mostly wanted.

They're all reasonable decisions. And they're things that can be
implemented as a set of business rules driven by, amongst other
things, the short term and long term history of the sender ("reputation")
in an automated way that doesn't require much per-sender configuration.
Plug in your policy, let it run, watch your statistics and tweak.

> It doesn't particularly protect the site

Yes. Filtering out "obvious" spam is easy. Recognizing "obvious"
1:1 email between regular correspondents isn't too hard.

Dealing with the big grey area in the middle is hard, and sender
reputation is about the only thing that gives you anything to base
delivery decisions on there, so for the big middle ground of email
good reputation will take you a long way.

Cheers,
  Steve

---

From: Dotzero <[email-masked]>
Date: Tue, 25 Jan 2011 20:49:32 -0500

> email, and you're always going to want to block the obviously

Wanting and doing are two different things. In many cases the
evaluator isn't going to get that granular. This is particularly true
where the unwanted mail is somehow malicious, not simply unwanted.

> and that's where sender reputation helps.

I'll grant you that about the middle. But that's not necessarily "badness".

> of several "bins" automatically and treat mail from them appropriately.

But the case that originally triggered the discussion is false
positives. That is a case of treating those messages inappropriately.

> compromised end-user machine".

A slightly different issue. For emitters of large volume mail streams,
the bad stream may still only be a percentage of the overall mail
stream.

> them on the assumption that it's mostly wanted.
>

I think you'll find (At least what I have seen) is that they will
throttle the overall stream (for example Yahoo! - 421 too many
complaints) or implement a temporary block (AOL), etc.

> Plug n your policy, let it run, watch your statistics and tweak.
>

I'm still radiating skepticism. While I agree that what you describe
is the optimal approach, I'm not sure it matches reality.

> 1:1 email between regular correspondents isn't too hard.

Various mailbox providers exhibit varying degrees of dealing with the
issue on a message basis rather than a stream basis. To the extent
that messages are diverted to a spam folder, we can infer the
confidence level that the receiver has in the capability of their
systems. That is, if they felt they could analyze individual messages
perfectly thent here wouldn't be a need for a spam folder. Ham would
get delierd and spam would go to /dev/null.

> good reputation will take you a long way.

Excellent response Steve (As usual) and I agree with you about the
middle. Remember the context of the discussion though. The triggering
question was whether one can make decisions directly based on SPF,
DKIM or a combination of the two. Doug stated that one could not. My
position is that you generally can (combinations)  for domains that
have good control over the mailstreams (particularly abused brands
such as financials).

---

From: John Leslie <[email-masked]>
Date: Tue, 25 Jan 2011 20:56:31 -0500

Dotzero <[email-masked]> wrote:
> So, what exactly does this mean when behavior suddenly changes?

   The reputation changes (almost as suddenly).

> If a domain or IP address (was well behaved yesterday) but begins
> spewing badness today, what will your company do as an arbiter of
> whether mail is accepted by your customers?

   If the badness is sufficient, they get a permanent error, until
I get around to clearing it.

   If the bandess is slight, they probably get a temporary error.
Ideally they fix the problem, clear the badness from their retry
queue, and everything works fine a few hours later.

   (OK, so more often they _don't_ clear the badness from their retry
queue, and their legitimate senders get SMTP warnings; but they stop
getting temporary errors after a week or so anyway.)

> Will you allow that domain or IP address to spew badness? I highly
> doubt it.

   In some cases, I _do_ allow local cable providers to spew badness,
but that's because they're local and my particular customers are
more upset about missing one good email than receiving 100 SPAMs.
This is real-world (TM), not academia...

> As some point once the spewing has subsided you may (automatically
> or manually) again start allowing traffic through from that domain
> or IP address.

   Automatically is much better!

> But that isn't really reputation in the traditional sense of the word.

   I suppose when you use a word, you are entitled to mean "just what I
choose it to mean -- neither more nor less." Nonetheless, I contend
this is, exactly, "prediction of the likelihood of near-future behavior."

   In the Internet, reputation can change in a few milliseconds.

> what does reputation mean?

   I can but guess you think of reputation as "prediction" based on
ancient history (several days ago, at least). But that's foolish in
an Internet where hardly anything is more than one second away.

> It doesn't particularly protect the site from the immediate
> consequences of going bad.

   Why should it?

   At most, I think it should give a "good" site more leeway in
cleaning up the badness. (But to do that well, we need communication
of what will be done and when...)

> It appears that the responses are authoritative (this domain or IP
> is currently emitting badness) rather than reputational (this site
> has a good reputation so I will accept badness from it on the
> presumption they are going to address it).

   Theres nothing "wrong" with the second kind of reputation service,
but neither is there much demand for it. Perhaps that's because
there's so little correlation between "we think they'll clean it up
RealSoonNow" and the actual responsiveness.

   Besides, why should any of us let through known badness even if
the prediction were true? Why shouldn't we instead keep giving
temporary errors until the badness is removed (or expired) from
their retry queue?

> I will grant that there may be some small slack cut based

   In a well-designed system, it could certainly extend to stopping
the temporary errors after a contracted number of hours to clean
things up.

   (And, BTW, yes, I have gone through postfix retry queues, removing
badness. It leaves me feeling _quite_ annoyed with the (ex-)customer.)

---

From: Douglas Otis <[email-masked]>
Date: Tue, 25 Jan 2011 19:08:02 -0800

On 1/25/11 5:49 PM, Dotzero wrote:
>
>  The triggering question was whether one can make decisions directly
>  based on SPF, DKIM or a combination of the two. Doug stated that one
>  could not. My position is that you generally can (combinations) for
>  domains that have good control over the mailstreams (particularly
>  abused brands such as financial).

Mike,

Correction.  To be clear, may I say one can use the position of stars in 
the sky when making decisions.

However, when making reasonable decisions about how well a provider is 
managing their mail resource, SPF pass or fail represents a poor 
measure.  SPF pass offers little upon which an administrator can be 
judged.  Malefactors are good at offering an SPF pass.  Providers that 
handle messages where a customer's SPF record ends up failing also does 
not necessarily mean the administrator has done anything wrong either.  
Instead, one must use identifiers that accurately track resources being 
managed by the administrator when assessing their stewardship, not how 
clever their customers are in deciding which SPF records to publish.  We 
both know customers are likely to get this wrong.  Your statements about 
some strange need to use "-all" suffixes in these records does not help 
the situation either.

Secondly, DKIM can be replayed and will not necessarily indicate the 
intended recipient of the message.  Blindly using DKIM as a basis to 
assert a signing domain is responsible for sending unsolicited bulk 
email may prove highly unfair when replayed by some malefactor.  When 
mitigating SPAM, both SPF and DKIM represent poor and often misused 
tools.  In addition, seldom are both DKIM and SPF required for message 
acceptance, which leaves either mechanism wide-open to abuse.  A 
requirement that both mechanisms pass would reduce the integrity of 
email delivery, and make recipients unhappy about their lost messages.

Use of cryptographic authentication of SMTP clients supported by DANE 
resource records can operate within IPv6, and then offer truly effective 
tools able to fairly aid in the mitigation of SPAM.  Use of third-party 
authorization would also permit policies able to cope with how email is 
normally used, without loosing track of the administrator's resource 
management.

-Doug

---

From: Dotzero <[email-masked]>
Date: Tue, 25 Jan 2011 23:00:49 -0500

On Tue, Jan 25, 2011 at 10:08 PM, Douglas Otis <[email-masked]> wrote:
> Correction.  To be clear, may I say one can use the position of stars in the
> sky when making decisions.
>

And if you are navigating at night, that would be a useful thing to do
- particularly if you have a sextant.

> managing their mail resource, SPF pass or fail represents a poor measure.

All SPF does is provide a "link" between the IP address/host and what
the domain (through a DNS record) indicates email for that domain is
emitted. As I have previously said, it is an enabler.

>  SPF pass offers little upon which an administrator can be judged.
>  Malefactors are good at offering an SPF pass.  Providers that handle
> messages where a customer's SPF record ends up failing also does not
> necessarily mean the administrator has done anything wrong either.

Context is everything. SPF pass means that the message (transport
layer) is tied to the domain through the DNS record. I'm going to do a
little flip flop here and say that if the behavior of the domain/IP
address is known then then an SPF pass can and does mean something
until the domain/IP address goes bad (the other discussion we have
been having). As far as an SPF fail, it depends on the domain, the
published record and the message itself to determine whether the
administrator has done anything wrong.

>Instead,
> one must use identifiers that accurately track resources being managed by
> the administrator when assessing their stewardship, not how clever their
> customers are in deciding which SPF records to publish.

I'm not clear on your distinction between administrators and customers.

> We both know
> customers are likely to get this wrong.  Your statements about some strange
> need to use "-all" suffixes in these records does not help the situation
> either.
>

Either you have a confidence in what you are doing and publishing or
you don't. I'll take a -all over a (naked) +all any day of the week.

> intended recipient of the message.  Blindly using DKIM as a basis to assert
> a signing domain is responsible for sending unsolicited bulk email may prove
> highly unfair when replayed by some malefactor.

I agree that there are some potential issues with replay attacks (for
example the double header issue). But as with your concerns about SPF
and multiple lookup DOS attacks, we haven't seen them. Thee are much
easier abuse vectors.

>When mitigating SPAM, both
> SPF and DKIM represent poor and often misused tools.

I'm not so sure about the "poor" part but I'll grant you the misused.
Is there any protocol on the internet that we can point to as not
having examples of poor implementations? One slight digression, as we
discuss DKIM, I am only commenting on 1st party signatures, not 3rd
party signatures.

>In addition, seldom
> are both DKIM and SPF required for message acceptance, which leaves either
> mechanism wide-open to abuse.  A requirement that both mechanisms pass would
> reduce the integrity of email delivery, and make recipients unhappy about
> their lost messages.
>

Don't think of it as a requirement that both pass. For mailstreams
that purport to be from a particular domain, what does a double pass
tell you with regard to the likelyhood that a message with a double
pass originated from the domain it purports to be (At both the
transport and message level).

> normally used, without loosing track of the administrator's resource

Let's just require everyone to use PKI and certs..... I'm not holding my breath.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 26 Jan 2011 13:32:30 +0100

>    Reputation (as the name implies) is a prediction of the likelihood of

Better yet, a prediction of the likelihood of a certain kind of
response to complaints and abuse reports.  Any host might be taken
down, hijacked, circumvented, or otherwise abused for sending spam,
what makes a difference is whether it has an abuse team who is willing
and capable of countering that.  _Reputation_ is a measure of such
will and capacity.

On 25/Jan/11 20:29, Douglas Otis wrote:
> Reputation Definition:  An assessment of an identifier that
> accurately reflects the behavior of the resource administrator.
> For example, these identifiers might be the address of an SMTP
> client or the AS of the network provider.

Let's not confuse authN and authZ:  Identifiers are related to the
former, assessments to the latter.  Doug's example alludes to the
problem of determining the worthier (more reputable) point in a hierarchy.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 26 Jan 2011 13:38:26 +0100

>    The reputation changes (almost as suddenly).

No, it shouldn't.  Although delays in fixing problems may affect it,
reputation measures the willingness and the ability to deal with them,
rather than the mere frequency at which problems occur.

---

From: Graeme Fowler <[email-masked]>
Date: Wed, 26 Jan 2011 12:57:32 +0000

> rather than the mere frequency at which problems occur.

I (and no doubt several others) beg to differ.

You may define "reputation" in that way, but that definition cannot be
automated.

At $workplace I use Exim to measure "reputation" - or "badness", or
possibly even "goodness" - as defined by a function of the rate of
invalid recipients, messages classified as spam, messages classified as
malware, unverifiable sender domains (per DNS lookups, not callbacks),
relay attempts, and a number of other indicators.
If that function returns a value under a certain threshold as compared
to the volume of "good" and "bad" mail arriving, that message is
deferred. Currently we don't reject - but we could change the threshold,
and do that instead.

There's no way for me to factor in human values such as "willingness" or
"ability" into that function; nor should I need to. If the data stream
from a given source is rubbish, I don't want it - and yes, that can hurt
legitimate messages (and has done so) but that's a chance we've been
willing to take.

So in my system, "reputation" is automated according to rates of hits on
certain criteria. This is just as valid a system as a manual one, and
should not be dismissed because it doesn't use human value judgements.

Graeme

---

From: John Leslie <[email-masked]>
Date: Wed, 26 Jan 2011 11:19:00 -0500

Alessandro Vesely <[email-masked]> wrote:
> response to complaints and abuse reports.

   I am sympathetic to that, but that too can change on short notice;
and many admins aren't volunteering to blindly accept the spam load
until that runs its course.

   Bot-herders, if they know that timeframe, could easily adjust their
spam-runs to finish before the deadline, leaving no actual useful action
for the admins to take.

   (And the time-frame to actually _clear_ a botted computer is insanely
long...)

   This is why I look to temporary errors as the appropriate action
during an incident.

> Any host might be taken down, hijacked, circumvented, or otherwise
> abused for sending spam, what makes a difference is whether it has
> an abuse team who is willing and capable of countering that.

   That does make _an_ important difference, but not one sufficient
for most admins to blindly accept the spam load during an incident.

> _Reputation_ is a measure of such will and capacity.

   "Your Network / Your Rules" applies, of course...

   But to me, "reputation" needs to suffer during the interim before
corrective action is completed.

   OTOH, in the absence of a useful abuse-team, reputation shold _stay_
bad for a rather long time. IMHO, of course...

   Which is why I consider "vouching" services so important.

---

From: Dotzero <[email-masked]>
Date: Wed, 26 Jan 2011 11:24:44 -0500

On Tue, Jan 25, 2011 at 8:56 PM, John Leslie <[email-masked]> wrote:
!
>   In the Internet, reputation can change in a few milliseconds.

And this goes straight to one of the big issues with any discussion
that uses the term "reputation". I'm going to disagree with you John
when you argue that everyone gets to define what reputation is in
their own way. If this were the case then any discussion of reputation
is meaningless. We need a common definition. Do you get to choose what
you mean when you say IP Address?

What you or anyone else gets to define for yourself are the attributes
that surround your use of the word reputation. That is, the
implementation that utilizes reputation.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 26 Jan 2011 17:33:15 +0100

> You may define "reputation" in that way, but that definition cannot be

Not easily, but possibly...

> possibly even "goodness" - as defined by a function of [behavior. ...]
> 

I'm not proposing to dismiss that.  However, I think "goodness" is a
better term than "reputation" for such function.  I propose to use the
term "reputation" for the value upon which one decides whether to send
an abuse report to the relevant entity.  (Contrast that with deferring
or rejecting mail from it.)

---

From: John Leslie <[email-masked]>
Date: Wed, 26 Jan 2011 11:52:39 -0500

Dotzero <[email-masked]> wrote:
> their own way.

   I really don't know how to stop them...

> If this were the case then any discussion of reputation is meaningless.

   If you believe that, I strongly urge you to stop trying.

> We need a common definition. Do you get to choose what you mean when
> you say IP Address?

   To the degree that some of us mean IPv4 only, yes. :^(

> implementation that utilizes reputation.

   IMHO, any Reputation service _necessarily_ gets to define what it
means by "reputation".

   (Otherwise, I see no reason to use any particular one.)

---

From: John Leslie <[email-masked]>
Date: Wed, 26 Jan 2011 12:01:24 -0500

Alessandro Vesely <[email-masked]> wrote:
> 
> I propose to use the term "reputation" for the value upon which one
> decides whether to send an abuse report to the relevant entity.

   For many of us, that's the null set.

   (I propose that abuse reports are useful when sent to a reputation
service by a customer of that service, and leave others to argue
whether they have any value beyond that. Thus I'm going to hang onto
the name "reputation service" for any such service.)

   Now, I'd be happy to use the term "vouching service" for something
determining whether abuse reports would be helpful.

   My experience, BTW, shows them to hardly ever be helpful (except to
a limited number of SMTP-sending domains, for which the occasion to
send one is rather rare).

> (Contrast that with deferring or rejecting mail from it.)

   I agree, it's a different animal.

   And I would agree, knowing when an abuse report will help _would_
be nice -- I'm just not holding my breath.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 26 Jan 2011 18:12:18 +0100

> for most admins to blindly accept the spam load during an incident.

Agreed.  Whether to accept, filter, or reject should also depend on
other factors.

>    "Your Network / Your Rules" applies, of course...

Yes, for the behavior.  "Reputation", as a term, does imply some
agreement among the people who define its value for the relevant
subject.  This is a value that we want to share.

> corrective action is completed.

Yes, if it takes too long.

> bad for a rather long time. IMHO, of course...

Yes, absolutely.

>    Which is why I consider "vouching" services so important.

I too.  However, I suddenly get struck dumb when I ask myself how
would I manage in order to vouch for someone, what feedback would I
need?  Possibly, _all_ of the people who interact with an entity
should contribute to the definition of its reputation.

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 26 Jan 2011 12:16:46 -0500

On 1/26/2011 11:24 AM, Dotzero wrote:
> On Tue, Jan 25, 2011 at 8:56 PM, John Leslie<[email-masked]>  wrote:
> !
>>

Everybody gets to define what a reputation means to them.  It does in 
the non-online world, the on-line world should be no different.  Other 
than that, you rely on the dictionary for what a reputation _is_, which 
leaves virtually infinite scope for details.

Reputation: A vague notion of something that may imply whether you want 
a given email or not, which could be based on, but not limited to, 
goodness, badness, neutralness, desirability, lawsuit-happiness, subject 
matter, shoe size or hair color in perhaps more than one not-necessarily 
orthogonal dimension, and generally not necessarily conformant with 
anyone else's notions, criteria or implementation.

> implementation that utilizes reputation.

There's a confusion between the term "reputation" as embodied by what we 
normally mean by what a 3rd party could theoretically supply, and that 
potentially very broad combination of things that's used to derive a 
consolidated "reputation" at the time that a singular implementation 
making a real-time decision about a given email will use.

My real-time decisions, for example, depend on a complex combination of 
3rd party yes/nos, 3rd party "maybes/maybe nots", "what it looks like", 
corporate policy, phone calls from management, rumors and lies, and just 
plain curmudgeonly curmudgeonness.

And we wouldn't have it any other way.

There's no point in discussing the latter, because people will do what 
they gotta do/feel like.  Nor should we constrain them.

Secondly, I don't think there's much point in proscribing what the 
former is based upon/means, because there are as many ways as there are 
3rd parties that it can be done and what a given "reputation" actually 
means precisely.  It will always be up to the user (the receiver) to 
decide what secret (or not-so-secret) incantations, sacrificial goats 
and transformations of such "reputations" that will go into the final 
decision.  The provider should explain what they're trying to impart 
(which may vary from one provider to another), but of necessity, the 
quality, coverage and semantics will differ between providers.

Just as they do, for example, between the SBL and XBL and PBL.  Each has 
different semantics, and recommended usage of the three _differ_.

Do we want to standardize reputation service protocols?  It'd be nice, 
but it may well be a complicated spec in order to preserve the semantic 
nuances of the provider's offerings.  Better to define a generic 
protocol that provides for the user being able to parameterize what a 
given reputation stream means to them (hopefully with the guidance of 
the provider).  You'll even need to be flexible in what the "entity this 
reputation is about" means.

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 26 Jan 2011 19:45:28 +0100

>    For many of us, that's the null set.

Alas.

> the name "reputation service" for any such service.)

That's a first step toward a constructive definition of "reputation".

Your latter proposal can be changed by blurring the "customer of"
membership, while enlarging a reputation service by adding more and
more customers.  For example, a site might host several virtual
reputation services by assigning higher factors to reports sent by
customers of many of them.  But then, are we going to also call
"reputation" the quality that one should have in order to become a
customer of those services?

Alternatively, we can imagine to send abuse reports to, say,
[email-masked], and have them forward them to the relevant ISP (or
RIR) who, in turn, will forward them to the relevant ADMD, and so on.
Then, we can imagine to optimize the resulting traffic by skipping
some of the initial hops and directly addressing a target with a
sufficiently high reputation.

Both approaches should result in the same thing: a global reputation
service.

> determining whether abuse reports would be helpful.

This may be a useful distinction.  I'd still consider that vouch=none
for a given domain should mean it has a pathologically low reputation.

---

From: Dotzero <[email-masked]>
Date: Wed, 26 Jan 2011 14:17:21 -0500

> for a given domain should mean it has a pathologically low reputation.

vouch=none might more appropriately mean that vouching isn't available
(for example, not enough data).
vouch=no might be more appropriate for a pathologically low reputation.

---

From: Douglas Otis <[email-masked]>
Date: Wed, 26 Jan 2011 14:56:06 -0800

Mike,

Actually, this is wrong.  While the default modifier for a mechanism is 
'+', when no mechanism matches and there is no 'all' suffix, the default 
for a naked record is '?' or neutral.  See section 4.7 of RFC4408.  It 
is not a matter of having confidence when you publish, it is how email 
for the domain in question is used.  As a general rule, asserting '-all' 
will result in the loss of some valid messages.  Secondly, a pass result 
is not modified by any 'all' suffix either.

-Doug

---

From: Douglas Otis <[email-masked]>
Date: Wed, 26 Jan 2011 14:56:11 -0800

On 1/26/11 9:16 AM, Chris Lewis wrote:
> than that, you rely on the dictionary for what a reputation _is_, 
> which leaves virtually infinite scope for details.

Chris,

Allow me to disagree.  As an industry, the goal should be to fairly 
assess identifiers that are tightly associated with accountable resource 
administrators.  In the case of an SMTP client, this could be an IP 
address known to be statically assigned, or a cryptographically verified 
host name of the server providing the service.  While it might not be 
practical to verify every connection using cryptography, the receiver 
can develop their own IP address ACLs based upon their view of 
connections from acceptable domains.  At this time, it is not practical, 
nor desirable, to base reputations on individual users of a resource 
being evaluated.  When there is abuse, administrators of that resource 
should be contacted, and be expected to correct the problem.  Since 
there is virtually an unlimited supply of domain names and now IP 
addresses, there is now a need to vet introductions of new domains 
(vouching).  IMHO, a good role for MAAWG and its members.

-Doug

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 27 Jan 2011 10:07:15 +0000

>    I really don't know how to stop them...

It's a linguistic issue. All words are defined by loose convention, and 
"reputation" is one of them. Sure, people can use "reputation" to mean 
anything they like, but some of those definitions are useful for 
communicating with others and some aren't.

But let's not forget that defining "reputation" means two things. First, 
the word "reputation" has a fairly tightly constrained meaning in the 
English language. Those who argue that reputation can mean anything 
probably aren't thinking of a new candidate entry for the Oxford English 
Dictionary.

But, second, the method of determining the reputation score that you're 
going to publish for any given identifier. That method can vary widely. 
Some operators may just viscerally hate the products or services that are 
associated with a given domain (vi4gra.com or microsoft.com), whereas 
others may have a carefully considered metric based on (say) feedback from 
tens of thousands of sites around the world. Those arguing that 
"reputation" can mean anything probably are referring to the methodology 
used to arrive at a reputation score.

We all know what a bad reputation *is*, but what we're really interested in 
is how a bad reputation *is acquired*.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

---

From: John Levine <[email-masked]>
Date: 27 Jan 2011 21:51:47 -0000

>is how a bad reputation *is acquired*.

Agreed.  I'm also interested in what one does with the info once you
have it.

R's,
John

---

From: Michael Thomas <[email-masked]>
Date: Thu, 27 Jan 2011 13:54:43 -0800

> Agreed.  I'm also interested in what one does with the info once you

Equally important is how a bad reputation is unacquired.

Mike

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 28 Jan 2011 10:17:18 +0100

On 27/Jan/11 22:54, Michael Thomas wrote:
>>> We all know what a bad reputation *is*, but what we're really
>>> interested in is how a bad reputation *is acquired*.
>>

We can do two sorts of things: share it, and use it for decision making.

> Equally important is how a bad reputation is unacquired.

Isn't that the same as acquiring good reputation?  But whom from, in
each case?

---

From: Jose-Marcio Martins da Cruz <[email-masked]>
Date: Fri, 28 Jan 2011 10:39:35 +0100

> We can do two sorts of things: share it, and use it for decision making.

I think an additionnal value could be attached to reputation - let me call it a confidence value. 
It's the same reasonning that in statistics - you can estimate a value of same variable but you 
should also estimate its confidence : how well your "estimation" represents the "real" value of that 
variable.

And this may be important in decision making.

Confidence value is something intuitive. One reason, *but not the only one*, is well known for all 
of us : some RBLs are more reliable than others. Another reason is the amount of data collected 
before assigning a bad reputation to someone.

> Isn't that the same as acquiring good reputation?  But whom from, in

No. Unacquaring a bad reputation may meant acquiring a neutral reputation.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 28 Jan 2011 11:11:50 +0000

--On 27 January 2011 21:51:47 +0000 John Levine <[email-masked]> wrote:

>> We all know what a bad reputation *is*, but what we're really interested
>> in  is how a bad reputation *is acquired*.
>

Well, yes. But the communication between a reputation service provider, and 
a reputation service consumer doesn't need to discuss that. The provider 
might give some hints about how they imagine the information being useful, 
but in the end it's up to the consumer to decide what to do with the 
information. For example, I think the "RFC ignorant" lists were supposed to 
be used as blacklists, but some people ended up using them as whitelists!

The key thing is that the consumer knows enough about the service that they 
can apply it to their own benefit.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 28 Jan 2011 11:17:37 +0000

--On 28 January 2011 10:17:18 +0100 Alessandro Vesely <[email-masked]> 
wrote:

> ____________

It's not quite the same. Reputation scores can be positive, negative or 
zero. And, they might not be simply trinary, but graded - say as 
percentages.

Unaquiring bad reputation might simply be a matter of going back to neutral 
reputation, at least for a probationary period. And, it could be gradual. 
For example, you might imagine that a domain's reputation would be badly 
hit by a system compromise, but gradually recovers over a period of a few 
days.

---

From: Michael Thomas <[email-masked]>
Date: Fri, 28 Jan 2011 07:23:16 -0800

> Isn't that the same as acquiring good reputation?  But whom from, in

 From what I've seen, they seem very asymmetric. Whereas acquiring a bad 
reputation is automated, unacquiring a 'bad' reputation seems to 
generally require human intervention, pleading, and in some cases what
amount to bribes. Considering that many of these reputation systems are 
hair triggered out of necessity, the amount of false positives is bound 
to be pretty high. So how to get back out of a blackhole is a whole lot
more arcane than falling into one.

Worse is that in some cases you simply cannot because the black hole's 
operators are uninterested, unavailable, unable, or in some cases just 
crazy.  Remember that it takes nothing to set up shop as a purveyor of 
blackholes. So there are zillions of them each vying to be the super 
massive blackhole. Some of them are really good, some of them are
really shady. And from what I can tell, there are a lot of mail operators
who don't seem to realize or care that there are quality differences.

Maybe we need a blackhole of blackholes.

Mike

---

From: Esa Laitinen <[email-masked]>
Date: Fri, 28 Jan 2011 16:53:57 +0100

On Fri, Jan 28, 2011 at 4:23 PM, Michael Thomas <[email-masked]> wrote:

> reputation is automated, unacquiring a 'bad' reputation seems to generally
> require human intervention, pleading, and in some cases what
> hair triggered out of necessity, the amount of false positives is bound to
> be pretty high. So how to get back out of a blackhole is a whole lot

True. This actually should motivate the mail senders to take care in their
e-mail hygiene so that they won't fall into that hole in the first place.
Not necessarily a bad thing, IMHO.

And this is due to the fact that the offenders don't change their behaviour
very often, or if they do, it doesn't often last very long.

> blackholes. So there are zillions of them each vying to be the super massive
> blackhole. Some of them are really good, some of them are
It doesn't really hurt much if somebody is in "Crazy Al's left-handers block
list": badly managed services will lose their users pretty quickly.

-- 
Mr. Esa Laitinen
Tel. +41 76 200 2870
skype/yahoo: reunaesa
Blog: http://happiloppuuahistaa.blogspot.com

---

From: Michael Thomas <[email-masked]>
Date: Fri, 28 Jan 2011 08:18:52 -0800

On 01/28/2011 07:53 AM, Esa Laitinen wrote:
>
> On Fri, Jan 28, 2011 at 4:23 PM, Michael Thomas <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     >From what I've seen, they seem very asymmetric. Whereas acquiring
>     a bad reputation is automated, unacquiring a 'bad' reputation
>     seems to generally require human intervention, pleading, and in
>     some cases what
>     amount to bribes. Considering that many of these reputation
>     systems are hair triggered out of necessity, the amount of false
>     positives is bound to be pretty high. So how to get back out of a
>     blackhole is a whole lot
> True. This actually should motivate the mail senders to take care in 
> their e-mail hygiene so that they won't fall into that hole in the 
> first place. Not necessarily a bad thing, IMHO.

While mail hygiene is a good thing, you can get sucked into
blackholes for perfectly innocent reasons. The easiest is that
you inherit address space that was in a blackhole. Getting
out of the blackholes of the usual suspects isn't that hard,
but like i say it's definitely not automated and can be very
shady itself.

>
> It doesn't really hurt much if somebody is in "Crazy Al's left-handers 
> block list": badly managed services will lose their users pretty quickly.

But not quickly enough. All it takes is a few clueless companies
or isp's to use badly run blackholes to make them a nuisance.  Worst
of all, is when they claim you're on some blackhole but don't tell you
which one, and it's not one of the ones that the various blackhole
aggregator/test sites knows about. I've been having problems with
some Swiss ISP that does this.

But it's the whole process that lacks any standardization or best
common practices as far as I can tell. If you're a legitimate sender,
you really don't want to waste your time sending stuff that readers
don't want to see. But the feedback loop is either nonexistent, very
murky or hasn't scaled down to where medium to small (read: have
day jobs doing other things than email delivery) can wire them up.

It seems to me that improving that situation would dramatically
help the false positive rate as well as the problem of accidentally
falling into blackholes of the clueful blackhole operators.

Mike

---

From: "Chris Lewis" <[email-masked]>
Date: Fri, 28 Jan 2011 12:13:53 -0500

> probably aren't thinking of a new candidate entry for the Oxford English

No, the existing dictionary definition may be "constrained" in one 
sense, but not necessarily constrained well enough for communication via 
a computer protocol.  Computers don't find human language easy after all.

Some write as if it can be adequately be represented as a single "good", 
"bad" or "neutral" indicator, or some scale between the three.

But can it?  Even the dictionary indicates it can't.

Here's one dictionary definition:

reputation [ˌrɛpjʊˈteɪʃən]
n
1. the estimation in which a person or thing is generally held; opinion
2. a high opinion generally held about a person or thing; esteem
3. notoriety or fame, esp for some specified characteristic
have a reputation to be known or notorious, esp for promiscuity, 
excessive drinking, or the like

Note the "specified characteristic".  The reputation may not necessarily 
be in the good-bad dimension.  Or the good-bad may depend on the context 
in which the question is asked.  Eg: could you rely on that same 
good-bad in the context of children versus adults?

Do we _really_ believe that there's a single reputation dimension in our 
context, or at least, that we can "get by" with just one?

I don't think so.

> used to arrive at a reputation score.

That speaks more to reputation of reputation instantiations in some 
sense - which is probably out of scope.  But it also does imply that a 
reputation service may well be measuring different but still valid 
things, and it would be best that the reputation service be able to tell 
the questioner what it actually means.

> We all know what a bad reputation *is*,

Kinda like porn, "we know it when we see it"? ;-)

Seriously, but do we?  Entity A and entity B may both have validly "bad" 
reputations, but for entirely different reasons, even within the same 
reputation service, in such a way that they need to be interpreted 
differently.  A reputation service needs to be able to impart at least 
some notion of context/reason so that a receiver can make an intelligent 
choice to respect/disregard/amplify/deprecate or whatever.

I don't think a simple yes/no/dunno or even a scale is adequate.

Spamhaus Zen, for example, is multi-dimensional.  Even its grossly 
simplistic expressive power is greater than I see the road we're 
treading here.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 29 Jan 2011 12:49:58 +0100

> Here's one dictionary definition:

Let me recall there's a proposed corollary to Godwin's law
http://www.irtf.org/mail-archive/web/asrg/current/msg15449.html

> excessive drinking, or the like

At any rate, please note the "generally held" part, which seems quite
essential to me.  In facts, in the off-line world, people say phrases
like "despite her reputation, IME she is <experienced quality>".
Thus, reputation would connote the shared facets of the experience.

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 29 Jan 2011 13:31:01 +0100

On 28/Jan/11 17:18, Michael Thomas wrote:
>> It doesn't really hurt much if somebody is in "Crazy Al's
>> left-handers block list": badly managed services will lose their
>> users pretty quickly.
> 

We (improperly) call reputation the judgment of self appointed block
list operators.  Spamhaus has a fame of being very good at it, but
people who've been erroneously bitten don't agree.

> day jobs doing other things than email delivery) can wire them up.

Hm..  Actually, there are some papers about how to run DNSBLs.
However, the semantics of a list is what it is: the unappealable
opinion of its operators.  Would it be better if it were the result of
a poll, say, or the average opinion of thousands of mailboxes operators?

> falling into blackholes of the clueful blackhole operators.

That would imply a radical change in attitude, from individual
survival tactics to free cooperative strategies.

---

From: John Leslie <[email-masked]>
Date: Sat, 29 Jan 2011 08:37:09 -0500

> http://www.irtf.org/mail-archive/web/asrg/current/msg15449.html
] 
] I think that as soon as you start quoting the dictionary, you've lost
] the argument.

   I thoroughly disagree. Perhaps quoting the dictionary "starts a
different" argument, but citing a scholar's opinion on derivation and
usage gives us something to focus on. IMHO, this aids the process of
convergence.

] In any technical field, one needs jargon as a shorthand to refer to
] agreed concepts.

   Perhaps there is such a need during development -- to refer to a
concept succinctly -- but it's fatal to standards-setting to use
poorly-defined words, and IMHO it's at best "sloppy" in documenting
research.

] When you start quoting the dictionary, you're arguing that some word
] has only its conventional meaning

   That would be _misquoting_ a dictionary. I presume competent writers
to not need to "misquote".

] Words like responsible, accredit, certify, are evolving into jargon,
] but we haven't yet come to consensus on what if any jargon meaning
] they have.

   If there's _no_ consensus, using them to describe research is a
bad idea. (In standard and law-writing, it's OK to include your own
definitions. In describing research, such re-definitions confuse
the reader.)

] The dictionary doesn't help, so leave it out of this discussion, please.

   I thoroughly disagree!

>> reputation [??r??pj????te??????n]
>> n

   For starters, that's three rather different meanings... I'd argue
that the third is the one closest to the concept we're discussing.

> essential to me...

   ... but it goes with the first definition, not the third...

> Thus, reputation would connote the shared facets of the experience.

   I can't think of an existent "reputation service" which aims for that.
Instead, they aim at "notoriety", and try to make early calls of that.

====

   Obviously, there are folks posting here who may be fixed on the first
or second defitions above. I believe quoting the dictionary may be the
best way to clarify the confusion of what we're actually discussing...

---

From: "Chris Lewis" <[email-masked]>
Date: Sat, 29 Jan 2011 11:14:25 -0500

On 1/29/2011 8:37 AM, John Leslie wrote:
> Alessandro Vesely<[email-masked]>  wrote:
>>

I'd ordinarily agree with Alessandro, which is why I don't recall ever 
quoting dictionaries before in decades of discussions like this, but he 
misses the point that I was directly responding to a comment that 
"Reputation" has a single dictionary meaning that we should rely upon 
for our work here, but as I attempted and you also pointed out 
explicitly, there are _three_.

Indeed, before we can start using a word as technical jargon, we have to 
understand what the jargon _means_.  I'm not sure we do yet.

And secondly, more to the point I elaborated on somewhat more, the third 
meaning is, I think, more in tune with what I think we want to do.  The 
point being that reputation isn't a single-dimensional 
"good-neutral-bad" trinary or scalar thing and we should be considering 
whether it's usefully multi-dimensional.  That makes a huge difference 
in what such a thing needs to be able to express.

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 30 Jan 2011 15:26:00 +0100

>    ... but it goes with the first definition, not the third...

The first two of them, actually.  "Known or notorious" is obviously
implicit in "generally held", so the third definition seems to just
drop the requirement that the general public should be willing to hold
the underlying judgment.  (Compare this with off-line world's news
services.)

I don't see a real difference among those definitions, because even if
we take 1 or 2, we want reputation records to be usable also by people
who never had direct experience with the relevant domain/IP.  For such
users, definition 3 has to apply.

> Instead, they aim at "notoriety", and try to make early calls of that.

I have difficulties thinking of an existent "reputation service" at
all :-)  Indeed, much of the confusion on this term may stem from the
fact that no impeccably designed reputation services exist yet.

By example, are DNSBLs reputation services?

Is Spamhaus DWL a reputation service?

Is Spamassassin AutoWhitelist, running at a given MTA, a reputation
service for the users of such MTA?

If there are well established uses of the term "reputation" we cannot
change its technical meaning.  We may use adjectives to disambiguate,
when needed.

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 31 Jan 2011 15:12:08 +0000

--On 28 January 2011 12:13:53 -0500 Chris Lewis <[email-masked]> wrote:

> No, the existing dictionary definition may be "constrained" in one sense,
> but not necessarily constrained well enough for communication via a
> computer protocol.  Computers don't find human language easy after all.

True, but that's not the point. What matters is how one aqcuires 
reputation. That is, the listing policy for the reputation service. That's 
what needs to be communicated when someone is considering whether to 
subscribe to a reputation service. Computer communication isn't relevant 
here.

> "bad" or "neutral" indicator, or some scale between the three.

That depends on what you mean by adequate. And that depends on what you're 
doing with the listing. If you're using it in -say- spamassassin, then you 
want to apply a score. A trinary result may be helpful, and the consumer of 
the service will weight that score accordingly. More useful would be -say- 
a positive or negative score out of 10 or 100 (more granualarity probably 
would not be meaningful). Again, the consumer would want to weight that 
score according to their judgement of the reliablity of the provider.

<snip>

> Seriously, but do we?

Yes, but different people, as you say, will differ as to whether it's 
deserved.

> Entity A and entity B may both have validly "bad"
> simplistic expressive power is greater than I see the road we're treading
> here.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 31 Jan 2011 10:55:32 -0800

> By example, are DNSBLs reputation services?
An important aspect of mitigation services is being missed.  This would 
be the IDENTIFIER being assessed, and whether it directly associates 
with an administrator of a service being rated for stewardship.  After 
all, momentary abuse events are often forgiven when the administrator 
can be observed responding.  Only administrators are able to impact the 
level of abuse seen on the Internet.  Without administrator's crucial 
efforts, mitigation services would also fail, in addition to the 
services they defend.

-Doug