Subject: IP-based reputation services vs. DNSBL (long)

From: Keith Moore <[email-masked]>
Date: Sun, 09 Nov 2008 13:28:30 -0500

Trying to sum up the situation here:

1. Several people have argued (somewhat convincingly) that:

- Source IP address reputation services can be valuable tools for
blocking spam,

- Such services can be better at identifying spam than per-message
content analysis,

- Such services can (and sometimes do) provide better feedback to the
sender, and thus better accountability, than per-message content
analysis that occurs later in the signal path, and

- At least some such services have very low false positive rates.

It's important to keep these in mind, as they appear to make a
compelling case for some kind of standardized reputation service.

2. Several people have also related experiences of valid messages being
blocked by such reputation services, and of the difficulty of routing
around them and getting their reputations corrected.

This strongly argues for such reputation services to have strict
accountability, and for there to be a clear, well-defined means by which
senders can correct erroneous reputations, and to get vital messages
delivered in spite of erroneous reputations.

3. An informal protocol for reporting reputations using DNS has been in
use for several years, and such use has become widespread.  An IRTF
group (ASRG) began a useful effort to document this protocol.

4. At some point ASRG decided that the protocol should be on the IETF
standards track and has requested such.

--

This process that produced this proposal reminds me of several patterns
I've seen come up often in IETF.

1. The first pattern is when an author or group gets confused between
the goal of writing an informational document to describe existing
practice, and the goal of writing a standards-track document that
describes desirable practice.

Offhand, I cannot recall a single experience when this has produced
satisfactory results.  The two goals are very much in conflict.  When
describing existing practice, it is very tempting to add extensions that
seem to be desirable, but which in fact are not part of existing
practice.   This is misleading at best, and at worst can cause
interoperability problems (my favorite example of this is RFC 1179).
OTOH, when writing standards that are based on existing practice, it is
very tempting to include existing practice (for the sake of
"compatibility") within the scope of permitted behavior, even when that
practice is dubious for one reason or another - poor security, say, or
poor scalability.

Over the years I've formed the opinion that the only reasonable way to
build a standard from existing practice was to first commit to
accurately documenting existing practice and publishing that document as
Informational.   Part of that process should be to document known
limitations of that existing practice.  Only after that document is
completed should there be an attempt to design a standard that is
compatible with existing practice.  That effort needs to be treated as a
design effort and subject to all of the usual vetting.   Furthermore it
needs to be understood that serious flaws from the original protocol
must not be included in the standard for the sake of backward compatibility.

2. The second pattern is when people insist that a widely deployed
protocol is inherently deserving of standardization, without further
vetting or changes, merely because it is widely deployed.

The simplest way of responding to this is that this is irrelevant under
our process.  The RFC 2026 criteria for proposed standard status require
both "rough consensus" of the IETF community and "no known technical
omissions".

The fact that something is widely deployed may be a indication of its
utility, but is not an indication of technical soundness.  Nor is it an
indication of consensus of the IETF technical community.

3. The third pattern is when a closed industry group, or an open group
that is not chartered to develop a standard protocol, insists that its
product merits standardization by IETF because it has gained consensus
of that group.

The problem with this is that the group did not attract the wide range
of interests that would normally attend an IETF standardization effort,
nor did the group operate under processes designed to ensure fairness
and openness.  Such efforts can be considered in the IETF process as
individual submissions, but they need a great deal of scrutiny, as this
tactic is often used as an "end run" around IETF.  My experience with
individual submissions is that simple, noncontroversial proposals
affecting a small number of parties rarely present serious obstacles to
standardization, but proposals that affect a large number of parties are
much more difficult, and with good reason.

The main point to be made here is that the consensus of an external
group means nothing in terms of either IETF consensus or judgment of
technical soundness.  In particular, external groups often have a much
narrower view of protocol requirements than IETF does.

All of these patterns are associated with delays in accepting a
standard.  They are also associated with poorer quality standards.

--

Back to the proposal at hand.  We have a proposal on the table for a
service which seems to have significant value if implemented properly.
And yet the proposal omits many of the details that appear to be
required for the service to have value, deferring those to a later "BCP"
document.  So there's an argument that the proposal as currently written
is incomplete, and standardization of the protocol is premature at best.

It is worth repeating that just because the notion of a reputation
service has value, and such services are widely used, does not imply
that using IP addresses as identifiers or the DNS protocol as a means of
transmitting reputation are technically sound.  There is reason to doubt
both of these assumptions, and there is no evidence that these design
questions have been given due consideration and resolved - as our
process would normally require.

At the same time, the process that produced this proposal has at least
three traits of processes that have been known to cause problems in the
past.  This should if nothing else give us pause, and counsel us to
proceed with caution.

So my recommendation to ASRG (and this document's authors) is as follows:

1. Abandon the effort to publish draft-irtf-asrg-dnsbl as a standards
track document, and instead commit to publishing it as an Informational
document as an accurate description of existing practice.  Be sure to
document observed limitations of existing practice.

2. Ask IETF to charter a working group tasked with developing a protocol
 for communicating email sender reputation.   The group can consider
DNSBL as a possible solution but should not be bound by a requirement to
be compatible with it, or to use DNS at all.

Keith

_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: John Leslie <[email-masked]>
Date: Mon, 10 Nov 2008 15:38:10 -0500

I find myself in complete agreement with Keith's major points:

Keith Moore <[email-masked]> wrote:
>...

   I might add that we don't need to standardize anything if we're
happy with what we already have.

>...

   Many "ordinary" folks are abandoning email rather than even _try_
to fix such problems.
 
> group (ASRG) began a useful effort to document this protocol.

   Such an effort is clearly useful for research purposes, and should
also be useful for any future attempts at standardization.

> standards track and has requested such.

   This is where we went wrong.

   Well, actually we went wrong quite a while ago, when a prior IESG
decided not to have a WG considering the spam problem in general. I
can't entirely blame the folks who have latched onto IRTF's ASRG in
the absence of an appropriate IESG forum.

   (And now we're carrying out a flame-war here -- a clear indication
IMHO that we need an IETF (not IRTF) list to move this discussion to.)

> describes desirable practice.

   This is human nature. IETF has developed protections againt this
(which do not require flame-wars). We should use them.

> vetting or changes, merely because it is widely deployed.

   This is commercial nature. IETF could use better protections against
this...

> product merits standardization by IETF because it has gained consensus

   This is not necessarily bad. But IESG (usually) tries to avoid the
situation getting this far -- by giving widespread notice of a WG charter
and encouraging cross-area review _before_ IETF last-call.

> Such efforts can be considered in the IETF process as individual
> submissions, but they need a great deal of scrutiny...

   I entirely agree, even though the necessary scrutiny is easy to
misinterpret as personal attacks by folks who "don't understand the
situation".

> narrower view of protocol requirements than IETF does.

   This is important! It's worth reading again.

> document observed limitations of existing practice.

   This is really easy to do, and wouldn't cause any delays.

> be compatible with it, or to use DNS at all.

   Lisa and Chris have stated that they're open to consider chartering
new WG if there seems to be consensus on a charter.

   What about it, folks?

--
John Leslie <[email-masked]>
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Lawrence Rosen" <[email-masked]>
Date: Mon, 10 Nov 2008 13:04:35 -0800

> new WG if there seems to be consensus on a charter.

As one of the people who objected when the previous spam WG was under way, I
now support this proposal to form a new WG to address the technical problem.

I hope the charter, unlike the previous one, will require the development of
a protocol for communicating email sender reputation that can be implemented
in email products without known patent encumbrances that are incompatible
with open source software. Email is simply too important to allow otherwise.

Best,

/Larry

Lawrence Rosen
Rosenlaw & Einschlag, a technology law firm (www.rosenlaw.com)
3001 King Ranch Road, Ukiah, CA 95482
707-485-1242 * cell: 707-478-8932 * fax: 707-485-1243
Skype: LawrenceRosen
Author of "Open Source Licensing: Software Freedom and 
                Intellectual Property Law" (Prentice Hall 2004)

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> John Leslie
> Sent: Monday, November 10, 2008 12:38 PM
> To: Keith Moore
> Cc: IETF
> Subject: Re: IP-based reputation services vs. DNSBL (long)
>

---

From: John Levine <[email-masked]>
Date: 10 Nov 2008 21:37:39 -0000

>I hope the charter, unlike the previous one, will require the
>development of a protocol for communicating email sender reputation
>that can be implemented in email products without known patent
>encumbrances that are incompatible with open source software. Email
>is simply too important to allow otherwise.

Not to belabor the totally painfully obvious, but DNSBLs are a
protocol for communicating email sender reputation that are
implemented in open source software without patent encumbrances and
have been for a deacade.

What would be the point of yet another WG to reinvent this wheel?

R's,
John

---

From: "Lawrence Rosen" <[email-masked]>
Date: Mon, 10 Nov 2008 16:15:27 -0800

> implemented in open source software without patent encumbrances and

Wonderful! /Larry

> From: John Levine [mailto:[email-masked]]
> Sent: Monday, November 10, 2008 1:38 PM
> To: [email-masked]
> Cc: [email-masked]

---

From: Eliot Lear <[email-masked]>
Date: Tue, 11 Nov 2008 09:17:43 +0100

> What would be the point of yet another WG to reinvent this wheel?

I tend to agree.  Here are a few questions for the IESG when considering 
this matter:

1.  Would declining to publish as a standard harm or hurt the 
community?  Would refusing to publish as a standard stop implementations 
or merely create potential interoperability issues that could lead to 
more legitimate messages being dropped?

2.  Does the IESG perceive that the creation of a working group would 
substantially change the content of the document in question?  Put 
another way, what would a working group consider doing differently?

3.  Would publishing on some other track serve a legitimate purpose, 
other than to duck the above two issues?

If the answer to the last question is "no", then I ask that the IESG 
properly address the first two.

Eliot

---

From: <[email-masked]>
Date: Tue, 11 Nov 2008 09:22:41 -0000

> Would refusing to publish as a standard stop 
> implementations or merely create potential interoperability 
> issues that could lead to more legitimate messages being dropped?

How would refusing to publish a document that is already public,
CREATE potential interoperability issues? The question is not
whether this information should be made public, because it already
has been and there is no reason to believe that an IETF refusal
would in any way prevent future publication of the information.

The heart of the question is whether or not this is work that
belongs in the IETF.

A big part of the issue is the fact that this draft glosses over
the security considerations of DNSBLs. If the draft had taken more
than three brief paragraphs to discuss these, then we would be 
having a different discussion.

DNSBLs are a temporary band-aid solution for a badly broken
Internet email architecture. They have provided the community
with an education but that doesn't mean that they should be
standardised by the IETF.

--Michael Dillon

---

From: Keith Moore <[email-masked]>
Date: Tue, 11 Nov 2008 06:17:28 -0500

> more legitimate messages being dropped?

How are either of these questions relevant to the criteria in RFC 2026?

> another way, what would a working group consider doing differently?

The working group could analyze the requirements of a reputation service
based on IP address, determine whether and how any newly discovered
requirements could be met using DNS, and fill in any details that are
missing from the informational specification that are needed for
reliable and robust interoperation _of the mail transport system_ when
using DNSxBLs.

Given the obvious use of this protocol as a means to influence mail
delivery decisions, I do not think it's appropriate to evaluate this
system as merely a protocol for carrying single bits of data independent
of their semantics.  It needs to be evaluated for its effect on mail
delivery.

Examples of details that need to be specified, at a minimum, include
TTLs (what should they be set to and why), security, TXT records (which
IMHO should be mandatory, along with a specification for how they're
returned in SMTP which should also be mandatory, and some specification
for what those TXT records say or point to).

I suspect there are additional details that also need to be specified.

My belief is that among the goals of standardizing this practice would
be:  To improve transparency and accountability of DNSBLs to end users
over what it is now, to improve security and reduce the potential for
use of DNSBLs for attack (including both DoS attack and thwarting of
spam blocking), to improve consistency in reporting between different
DNSBLs.

> other than to duck the above two issues?

Publishing the current document (with appropriate revisions) as
Informational would help avoid the conflict between accurately
documenting existing practice and observed experience with that practice
(which is entirely appropriate for Informational), and documenting
desirable practice (which is what standards are for).  Efforts to do
both at the same time always, in my experience, produce poor results.

Perhaps more obviously, publishing the current document as Informational
 allows documentation of existing practice in a short timeframe, and

> properly address the first two.

I hope the IESG will respect our established process, and recognize that
(a) there are indeed technical omissions in this document, and (b) the
document as currently written does not have rough consensus of the IETF
technical community, as shown already by the expressed concerns of
several contributors to this list.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Eliot Lear <[email-masked]>
Date: Tue, 11 Nov 2008 16:00:33 +0100

Keith,
> How are either of these questions relevant to the criteria in RFC 2026?

Since the request to advance in this case is for PS, there is a 
relatively low threshold to move the thing forward, as documented in 
Section 4 of RFC 2026.  Section 6 of RFC 2026 requires the IESG to use 
its collective judgment in reviewing requests for advancement.  I am 
attempting with both questions to provide guidance to the IESG as to how 
to how they might judge.

> reliable and robust interoperation _of the mail transport system_ when

That's all rather nebulous, Keith.  You then go on to cite examples 
which in fact are addressed very specifically in the draft, and so I 
suspect they are merely pretenses for what is really irking you, which 
you have stated repeatedly, first in your initial message on this thread:

> Under no circumstances should any kind of Blacklists or Whitelists be
> accepted by IETF as standards-track documents.  These lists are
> responsible for huge numbers of illegitimate delivery failures.  We
> should no more be standardizing such lists than to be standardizing spam.
>    

> spam blocking), to improve consistency in reporting between different

And what makes you think a technical standard would do that when it 
never has in the past?  Once again, I think you are overstating the role 
of this organization.  The choices a DNSBL makes to list someone are 
entirely its own.  The choice to use a given DNSBL is entirely that of 
the MTA administrator.  You cannot specify a standard to force a DNSBL 
to disgorge its logic.  When we have tried this in the past we ended up 
with superfluous information undermining confidence in any useful 
information that may be out there.  One need look no further than the 
value of old WKS records.

> several contributors to this list.

(a)  You complained about a lack of discussion on TTLs, which is in fact 
in the draft.  You complained about a lack of security considerations, 
which is in fact in the draft, and you complained about TXT records not 
being required, when in fact the draft makes use of a SHOULD.  It is not 
clear to me that a MUST meets the criteria listed in RFC 2119.

(b)  I count three or four very vocal opponents versus an enormous 
installed base.  But even were it more, I would hope the IESG wouldn't 
only use rough consensus to move forward, but also their judgment, for 
which we pay them so much ;-)  And that leads to my questions, which can 
be summed up as follows:

    * Are we better off with or without a proposed standard in this space?
    * Is this document the right document to base that standard on?
    * Is there sufficient benefit to be had by creating a working group?

For the record, my answer to these questions are Yes, Yes, and No.  I 
believe the onus is on those who would want a working group to come up 
with at least a plausible outcome that could improve the document.  Thus 
far I've heard no such thing, and there are risks for delaying 
publication, one of which would be calling into question the IETF's 
relevance.

Eliot

ps: for the astute, we are nearly back to a NAT argument.  The IETF 
version of Godwin's Law may apply.

---

From: Keith Moore <[email-masked]>
Date: Tue, 11 Nov 2008 10:33:34 -0500

>> should no more be standardizing such lists than to be standardizing spam.

To clarify, I've backed off of that extreme position after being
persuaded that a well-designed reputation protocol can increase
accountability and transparency for blacklisting services.  I feel like
if we put end-users into the feedback loop the system as a whole can
improve tremendously.

At the same time I think we've seen evidence in this discussion of a
fair amount of disconnect between operators and end-users - so am
absolutely convinced that such transparency and feedback are necessary
parts of a standard.

I am also convinced that the design of the reputation protocol has a
(perhaps subtle) effect on the reliability of a mail system that relies
on reputation services, and that this bears more examination.

> never has in the past? 

I strongly disagree that technical standards have never done this in the
past.

> Once again, I think you are overstating the role of this
> organization.  The choices a DNSBL makes to list someone are entirely
> its own.  The choice to use a given DNSBL is entirely that of the MTA
> administrator. 

I don't disagree with either of those statements.

> You cannot specify a standard to force a DNSBL to disgorge its logic. 

I'm not even sure what you mean by that, but I don't think I wish to do
any such thing.

> clear to me that a MUST meets the criteria listed in RFC 2119.

IMHO, The draft is woefully incomplete in both areas.  And these are
just examples of things that are wrong

> installed base.

The installed base is not the same thing as the IETF community.  If "the
installed base" wants to lend its reputation to support this document,
that's its own business.  But that's not the question at hand.  What is
being asked is for IETF to lend _its_ reputation to this document,
despite obvious technical flaws, deviations from our usual criteria in a
number of areas, and significant objection from within the IETF
community.  This is being requested on the basis of vague assertions of
some "installed base" by a small number of persons claiming to represent
that "installed base" - claims which, I might add, are fairly dubious
and which are irrelevant according to our established process.

At any rate, the purpose of IETF is not to recognize "the installed
base", the purpose of IETF is to specify what it believes will make the
Internet work well.

> But even were it more, I would hope the IESG wouldn't only use rough
> consensus to move forward, but also their judgment, for which we pay
> them so much ;-) And that leads to my questions, which can be summed
> up as follows:

> For the record, my answer to these questions are Yes, Yes, and No.

I disagree.  But more importantly, those answers are not relevant to the
question at hand - which is whether the current version of the current
document meets the criteria we've established for Proposed Standard.
And we've already seen plenty of evidence presented that it does not -
in either the technical quality or rough consensus categories.

A separate question to be addressed by IESG is how to further this work.
My personal opinion is that the best way to proceed is to first work
toward publishing this document of existing practice as Informational
and then form a WG to determine what kind of protocol (this or another
one) would be suitable for a standard.  I believe this gives us the best
possible result - quick publication of a document that will improve
interoperability of what is being done today, to be followed by a
standards effort that should "raise the bar" and produce an improved
reputation service in the future.  (do you really believe that DNSBLs in
their current form have no room for improvement?)

But there are lots of possible ways to proceed, and I'm sure IESG will
carefully consider that question.

Keith

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 11 Nov 2008 07:39:08 -0800

> now support this proposal to form a new WG to address the technical problem.

Can folks who want a working group explain what technical work is needed for the 
current specification?

What deficiencies need to be remedied and what enhancements need to be made?

Please explain these requirements in terms of community need as indicated by 
those who will use the output.

I suspect that the closer your responses are to text that should be in a 
charter, the more helpful it will be for justifying a working group.

Thanks.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 11 Nov 2008 07:41:19 -0800

> I tend to agree.  Here are a few questions for the IESG when considering 

I strongly urge community and IESG folk to think carefully about the questions 
Eliot is asking and to post responses to them.

d/
-- 

   Dave Crocker

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 11 Nov 2008 07:42:45 -0800

[email-masked] wrote:
> Internet email architecture. 

DNS-based reputation lists have been in production use for at least 11 years? 
There is no industry move to reduce their use.

By what metric does this qualify as "temporary"?

d/

-- 

   Dave Crocker

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 11 Nov 2008 07:55:41 -0800

Well, we have a critical dependency on a star that is going to run out of hydrogen at some point...

________________________________

From: [email-masked] on behalf of Dave CROCKER
Sent: Tue 11/11/2008 10:42 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: IP-based reputation services vs. DNSBL (long)

d/

--

   Dave Crocker

---

From: Sam Hartman <[email-masked]>
Date: Tue, 11 Nov 2008 11:12:42 -0500

Keith, I find myself in complete agreement with your message.  I
particularly like the fact that you took the time to go through a
complicated reasoning process in a slow, clear manner so that your
readers could determine whether they agree with your reasoning and if
not, where they disagree.

---

From: TS Glassey <[email-masked]>
Date: Tue, 11 Nov 2008 08:16:47 -0800

I have serious concerns with doing ANYTHING with the DNSBL entity 
because of the damage that it may do to our sponsors...

The IETF operates Standards not third party services, and so somehow 
this seems inappropriate.

Todd Glassey

Keith Moore wrote:
> ------------------------------------------------------------------------
>
>
> No virus found in this incoming message.
> Checked by AVG - http://www.avg.com 
> Version: 8.0.175 / Virus Database: 270.9.0/1779 - Release Date: 11/10/2008 7:53 AM
>
>

---

From: Tony Finch <[email-masked]>
Date: Tue, 11 Nov 2008 17:15:16 +0000

> process would normally require.

Could you give us a reference to an explanation of why the DNS might not
be a sound choice of protocol for reputation services?

This protocol can use domain names as lookup keys as well as IP addresses.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
ROCKALL MALIN HEBRIDES BAILEY: NORTHERLY OR NORTHWESTERLY 6 TO GALE 8
DECREASING 4 OR 5, BACKING SOUTHWESTERLY 5 OR 6 IN ROCKALL AND BAILEY LATER.
ROUGH OR VERY ROUGH. RAIN OR SQUALLY SHOWERS. MODERATE OR GOOD.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Tue, 11 Nov 2008 12:35:32 -0500

> be a sound choice of protocol for reputation services?

My concerns are along the following lines:

1. suitability of the DNS data and query model.  right now this protocol
essentially communicates one bit of information to be used in a decision
- i.e. whether the address or domain name is good or bad.  I suspect
that more information is needed to really make a good reputation
service.  I have doubts about whether DNS lends itself to including that
much information.  (It also bugs me that multiple queries are required
to  get the information necessary to perform a bounce/relay decision and
to bounce a message... granted that this is a flaw in the DNS protocol
design but using DNS for unintended purposes exacerbates the problem)

2. this is a very different use case than DNS was intended to address,
and implicit and explicit assumptions in the DNS design might not hold
for this case.  e.g. assumptions about how long information remains
valid and how widely it should be replicated.

3. security.  it might be that mechanisms already defined or used with
DNS (DNSSEC, source port randomization) are adequate, but I'd like to
see more analysis done.

4. effects of DNS caching.  if a host is removed from a blacklist it
should arguably be removed from all caches instantly, but DNS isn't
designed to facilitate that.  again, blacklisting someone else's IP
address is a different use case from managing the addresses associated
with your own hosts.  If you manage your own hosts' DNS RRs then you can
 tailor your operations to accommodate changes in DNS records, and you
can (at least in theory) adjust your TTLs in advance of any changes to
minimize the potential for disruption when you do need to change things.

5. slippery slope.  DNS is a vital service, and one that is very
difficult to replace.  It needs to remain focused on a narrow goal.  The
more we overload DNS, the more we threaten to add complexity that will
make it more fragile.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Tue, 11 Nov 2008 12:41:46 -0500

Dave CROCKER wrote:
> DNS-based reputation lists have been in production use for at least 11
> years? There is no industry move to reduce their use.

and yet spam remains a problem.  the problem will be addressed in one of
two ways: either better mechanisms for dealing with spam will be
developed, or people will stop using email.  (indeed, we're seeing both
trends - so it's hard to know which will dominate.)

either way, the current use of DNSBLs seems likely to be temporary.

what's the justification for carving them in stone, so to speak?  given
that you seem to accept that they're desirable, why not try to improve them?

I'll note that the "industry" does tend to paint itself into a corner
from time to time.  Sometimes innovation has to come from outside the
industry.

Keith

---

From: TS Glassey <[email-masked]>
Date: Tue, 11 Nov 2008 09:55:06 -0800

> design but using DNS for unintended purposes exacerbates the problem)
I agree - these Identity Protocols need to be stronger meaning that 
UDP/IP transports are by their very capability, not the right animal IMHO.
> valid and how widely it should be replicated.
Agreed. And again - the design of UDP based DNS is unacceptable for this.
> DNS (DNSSEC, source port randomization) are adequate, but I'd like to
For moving DNS resolutions possibly but NO credible security expert has 
been willing to sign off on the trust-model that DNSSEC uses...
> designed to facilitate that.  

The use of the term "SHOULD" here has legal implications - since many of 
these hosts were put into the BL's by Address Spoofing they were in fact 
NOT where the offensive SPAM was coming from and so placing those hosts 
there when the real issue is the refusal of the EMAIL Admin to do proper 
Header Filtration and Validation creates a direct liability.

I assure you RBL.ORG doesnt have enough money to pay a lawyer for a 
damage claim against it for its actions in 'putting in place a method 
and process specifically to prevent the addresses misrepresented in 
fraudulent headers, to the RBL.

> again, blacklisting someone else's IP
> with your own hosts.  

YEP!
> If you manage your own hosts' DNS RRs then you can

---

From: Matthias Leisi <[email-masked]>
Date: Tue, 11 Nov 2008 19:09:51 +0100

> - i.e. whether the address or domain name is good or bad.  I suspect

This is wrong. For todays DNSxLs, many queries return multiple bits of
information (eg to indicate a match on multiple lists, or varying levels
of trustworthiness). For some examples, have a look at
"25_dnsbl_tests.cf" of your nearest SpamAssassin installation.

> to bounce a message... granted that this is a flaw in the DNS protocol

This is also wrong for the reason given above.

> DNS (DNSSEC, source port randomization) are adequate, but I'd like to

Spammers have a high interest to circumvent / subvert / hack / hijack /
destroy propagation mechanisms of DNSxLs. In all the many years, they
have failed. I would conclude that the security of the mechanism is not
so bad.

> should arguably be removed from all caches instantly, but DNS isn't

Chapter 4 of the draft is dedicated to this chapter.

> more we overload DNS, the more we threaten to add complexity that will

A single nameserver of dnswl.org has a DNS traffic of roughly 10 GByte
per day, and I would guess that is a lot smaller than what a well-known
blacklist has. So far, the extensive use of DNS does not seem to have
destabilized DNS.

Keith, I fail to see a convincing argument in this discussion.

-- Matthias

---

From: Matthias Leisi <[email-masked]>
Date: Tue, 11 Nov 2008 19:14:20 +0100

> Header Filtration and Validation creates a direct liability.

I'm sorry to jump in once more. The paragraph above is simply incorrect.
A DNSBL that would be fooled by "address spoofing" would not provide
much value. What do you mean by "address spoofing"? Falsified "From:"
lines?

-- Matthias

---

From: Eliot Lear <[email-masked]>
Date: Tue, 11 Nov 2008 19:35:07 +0100

> standardised by the IETF.

DNSBLs are over 10 years old and are NOT going away any time soon, and 
so any notion of temporary must be placed into geological context for it 
to make sense.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: TS Glassey <[email-masked]>
Date: Tue, 11 Nov 2008 10:42:04 -0800

Matthias
Any DNS BL Listing process where those listings are based on complaints 
would create this.

The issue is that if SPAM HEADERS can have the source addresses forged 
then the DNS Blocking systems which were listed in those forged headers 
need to take that into account. So far as I can tell they dont.

Todd Glassey

---

From: "Chris Lewis" <[email-masked]>
Date: Tue, 11 Nov 2008 16:47:59 -0500

TS Glassey wrote:
> would create this. [spoofed IPs in DNSBLs]

Few DNSBL listing processes rely on "complaints" as you put it.
Certainly, none of the popular ones use them extensively, and most
refuse them.  Eg: the CBL explicitly refuses contributions of complaints.

Most DNSBL listing processes rely _only_ on the peer address of the
connection (either direct, or by header insertion by their own trusted
systems).  No-one has yet come up with a spam-economy-practical
mechanism for spoofing source IP in TCP/IP (SMTP) sessions.  There has
been much research on the topic, and it all seems to indicate that there
isn't one.  I'll refer you to papers by Steven Bellovin, Marcus Leech
and others.

[UDP packet source IPs are trivially forgeable.  But you can't send
email by UDP packets.  TCP/IP source IP is forgeable, but only at
extremely high effort levels - few spammers would be satisfied with a
throughput rate of a few spams per week (at most) per bot that works
only against some destinations, when the return rate is measured in the
single digits per million spams.  If TCP/IP source spoofing were to
become a spammer-practical method, the Internet has vastly bigger
problems than flakey email.]

The two most effective DNSBLs of all (CBL & PBL, both part of Spamhaus
Zen) don't look at headers at all.  The former takes its IPs directly
from the TCP/IP stack of the MTA receiving the email (eg:
getpeername()), and the latter is a policy assertion, largely by the
verified owner of the IP ranges in question.  IP spoofing is effectively
impossible in one, and irrelevant to the second.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Wed, 12 Nov 2008 08:26:21 -0800

Agree with your conclusion but your statement is not quite accurate.
 
Some spammers have in fact developed schemes that involve spoofing the source IP address of TCP sessions, but only where both IP addresses were under spammer control.
 
What some spammers used to do when dialup connections were still common and broadband rare is that they would use a dialup session as the purported source of the packets but really send the bulk of the message from a high speed connection. The dialup connection telling the high speed connection which sequence numbers to employ.
 
I don't know if it is still widely used but when is was being used the disruption caused to the network was cosiderably higher than for normal spam as you can expect.

From: [email-masked] on behalf of Chris Lewis
Sent: Tue 11/11/2008 4:47 PM
Cc: IETF
Subject: Re: IP-based reputation services vs. DNSBL (long)

---

From: "Chris Lewis" <[email-masked]>
Date: Wed, 12 Nov 2008 12:59:19 -0500

> Agree with your conclusion but your statement is not quite accurate.

I know that.  I had composed a footnote outlining split-routing in my
original email, but removed it because it would confuse the issue
precisely for the reasons you yourself outline below, without
invalidating anything I said.

Since you raise it, I'll elaborate.

You said:

> Some spammers have in fact developed schemes that involve spoofing the
> source IP address of TCP sessions, but only where both IP addresses were
> under spammer control.

<details of split-routing elided>.  Precisely.  Both IP addresses are
under spammer control, the listing still has precisely the desired
effect (blocking the undesirable email), and FPs are no more likely than
if the "alleged source" really was the true source.

Now if there were plausible circumstances where, say, infesting a real
MTA with split-routing malware was any more likely or preferable than
infesting a real MTA with non-split-routing spamware, it could be of
concern.  But there isn't.

Split-routing requires very intimate and synchronous real-time
bidirectional communications between both IP addresses of the split.  I
believe that most if not all implementations were where both IPs were on
the same machine, both IPs were "legitimately held" by the spammer (not
by infection).  That scenario is no longer feasible at typical BOT volumes.

> I don't know if it is still widely used but when is was being used the
> disruption caused to the network was cosiderably higher than for normal
> spam as you can expect.

It is not widely used any longer.  At best, the technique is of value in
very limited circumstances ("effectively free/super cheap accounts with
associated real IPs", eg: AOL "first month free" disks).  Secondly, it
also requires there to be a more "valuable" asset (the true source, a
paid high bandwidth account) that the spammer is trying to protect.

With BOTs, there is no "paid high bandwidth account" to protect, or at
least, not by these means - all parts of the spam-sending exercise, and
much of the immediately visible portions of the web content delivery are
expendable (eg: domain tasting, expendible intermediate proxies, etc).
Secondly, the volumes that spammers need to send to get any return are
too high for this technique to be feasible.

At its height, a number of people noted that the only known mass-use of
this technique was via AOL.  The CBL once listed over 20,000 AOL "ipt"
(end-user IPs) that were also outbound port 25 blocked by AOL.  Yes,
there was some confusion (including that of AOL admin staff) of how this
was possible, but _not_ because it was interfering with legitimate email
- their non-spamming customers _couldn't_ send email that way anyway.
However, once they understood what the issue was, the fix (block inbound
port 25) was accomplished in very short order.  If anything, the
juxtaposition of a CBL listing of the port 25-blocked AOL "split-end"
<grin> greatly speeded up resolution of the entire issue - certainly a
lot easier than whack-a-mole of thousands of individual accounts.

We've seen this situation a few times since then in a small way (none in
the last year or longer), but established practise is tending to
eliminate this as spammer-economy-viable (eg: see MAAWG Port 25
considerations document).  In fact, I think the AOL experience is one of
the prompters of that document's existance.

Aside from unusual situations like AOL (back then), it is no longer a
useful technique for spammers to exert effort on, and even if it was,
the distinction is moot, because the effect/consequences of a DNSBL
listing of the back-haul of a split-routed connection is identical to a
non-split-route connection.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: John Levine <[email-masked]>
Date: 12 Nov 2008 21:34:37 -0000

>What some spammers used to do when dialup connections were still
>common and broadband rare is that they would use a dialup session as
>the purported source of the packets but really send the bulk of the
>message from a high speed connection. The dialup connection telling
>the high speed connection which sequence numbers to employ.

Spammers used to do that, but it didn't involve any address spoofing,
just routing games.

The bad guy had a T1 and a dialup into the same box.  It used the IP
of the dialup its traffic but sent outbound packets over the T1,
getting return packets via the dialup.  Since spamming involves a lot
more outbound than inbound traffic, this still let them use most of
the T1.  When the dialup ISP noticed and cancelled the dialup account,
they'd just switch to another one, typically using a stack of free
trial AOL disks.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://www.johnlevine.com, ex-Mayor
"More Wiener schnitzel, please", said Tom, revealingly.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Thu, 13 Nov 2008 07:10:07 -0800

To answer your question about how they got round port 25 blocking, my guess is that they sent the initial packet out on yet another connection that was unblocked.
 
I don't see how the timing issues are so great, this is at worst going to double the round trip time and double the lost pckets. 
 
I have seen something similar described recently in the context of a cyber-conflict type attack.

From: [email-masked] on behalf of Chris Lewis
Sent: Wed 11/12/2008 12:59 PM
Cc: IETF
Subject: Re: IP-based reputation services vs. DNSBL (long)

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 13 Nov 2008 15:52:07 -0500

Hallam-Baker, Phillip wrote:
> To answer your question about how they got round port 25 blocking, my
> guess is that they sent the initial packet out on yet another connection
> that was unblocked.

Actually, I answered that question - they didn't "get around port 25
blocking".  They never sent from the (say AOL dialup) side, only from
the high speed side.   "three way handshaking" emulation of what's
supposed to be "two way", but physically only two (not three) machines.
 Since they're on the same machine, the timing is not much of an issue.
 Got high speed spam emission, at the expense of burning (lots of) free
AOL low speed access dialup disks.  Especially if you pipelined (whether
the recipient said it was okay or not) multiple parallel SMTP streams.

[The recipient usually has no way of knowing whether you're really
waiting for it's SMTP command return codes or not.  Which is exemplified
by one particular type of HTTP proxy attack.  Arrange the entire sending
side's SMTP commands in one buffer (eg: a HTTP CONNECT proxy), and send
it all at once.  Works just fine if you don't care about errors.  Which
high volume spammers don't.]

> cyber-conflict type attack.

Potentially still useful technique, where the economies are different.
_______________________________________________
Ietf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Thu, 13 Nov 2008 13:37:06 -0800

Yes, there are many spammers who use systems that essentially send out the buffer in a single packet. Even if not doing one arm routing they use it for acceleration.
 
Which is why a somewhat effective spam pre-filter is to simply enforce proper SMTP protocol use and reject a connection if the other side has queued up RCPT and DATA commands in the HELO packet.

From: [email-masked] on behalf of Chris Lewis
Sent: Thu 11/13/2008 3:52 PM
Cc: IETF
Subject: Re: IP-based reputation services vs. DNSBL (long)