Subject: RE: On Extensibility in MARID Records

From: "Jim Lyon" <[email-masked]>
Date: Wed, 16 Jun 2004 14:46:09 -0700

Andy takes a stab at asserting that SPFish syntax is enough, writing:
> My first stab is:
>     +a:zzz/hdr-feedback-bulk=[email-masked]
> or
>     +a:zzz/feedback-policy="dns:feedback.ibm.com?class=TXT"

Of course, there are problems here. In his first try, he says (by
example) that you concatenate the various modifiers together, separated
by "-".  But the limits are immediately apparent: The "=[email-masked]"
really modifies "feedback", but is stuck after "bulk" which is unrelated
to "feedback".

Furthermore, someone pointed out that we'd never really want to receive
a reply message for *every* matching message. So a publisher would
probably also need to stick a "p=.001" (send feedback with probability 1
in 1000) somewhere in there, and it's not obvious where.  Andy's fist
example stops well short of showing that it's easy to extend SPF syntax
to get there.

His second example might be somewhat more promising.  I assume that it
means "go resolve the URL "dns:feedback.ibm.com?class=TXT", and you'll
find something that gives the feedback policy. Of course, it immediately
begs the question of which URL schemes are allowed (dns? http? gopher?
ftp? mailto?) and what the format  and language of the fetched document
is (XML? something else? TBD?)

To add more fuel to the fire of how hard it is to design a new language
with extensibility, nobody has yet noticed that Andy's examples, as well
as all of mine, are bogus:  SPF already reserves the '/' character for
other purposes.

-- Jim Lyon

---

From: Marshall Rose <[email-masked]>
Date: Wed, 16 Jun 2004 15:54:24 -0700

the co-chairs think this is a useful thread; in fact, we're going to 
place the following plan in front of the working group:

it is clear to the co-chairs that the "tipping point" with respect to 
the working group's deliberations revolve around the sufficiency of the 
SPF syntax to address near-term concerns. if the syntax is adequate, it 
is hard to argue for an alternative or future syntax; if the syntax is 
inadequate, then the SPF syntax is of transitory use.

in order to drive consensus on this issue, the chairs put forth this 
plan:

between now and sunday, june 20, 2004 23:59:59 us/pacific time, ALL 
replies to this message must give examples of scenarios for which the 
SPF syntax is insufficient. any requests for clarifications MUST be 
sent directly to the authors of a scenario, who may then post a 
clarification in reply to their own email.

starting monday, june 21, 2004 00:00:00 us/pacific time, the working 
group may reply to any of these scenario messages for the purpose of 
explaining how the SPF syntax is sufficient.

on thursday, june 24, 2004 23:59:59 us/pacific time, the chairs will 
gauge the group's consensus on the issue of the sufficiency of the SPF 
syntax.

based on the chair's reading, the working group will be presented with 
a plan for moving forward within the timeframe allotted by the charter.

so, to re-cap:

1. now until sunday evening: people post difficult examples

2. monday until thursday evening: people post explanations on dealing 
with the difficulty

3. friday: chairs announce plan for moving forward.

/mtr

---

From: Marshall Rose <[email-masked]>
Date: Wed, 16 Jun 2004 17:28:28 -0700

> 3. friday: chairs announce plan for moving forward.

a postscript:

obviously, the "difficult examples" must refer to reasonably useful 
extensions; if the definition of "useful" becomes contentious, the 
chairs will make a determination.

/mtr

---

From: Margaret Olson <[email-masked]>
Date: Wed, 16 Jun 2004 23:06:23 -0400

Here's are my first few examples:

Example1.com:
example1.com uses two providers: esp.com and personalmail.com .

Mail originating from personalmail.com is always conversational 
(corporate individual mailboxes) and rate limited to 100 messages a day 
by personalmail.com

Mail authored by example1.com originating from esp.com is rate limited 
to 4,000 messages a month by esp.com. This mail is bulk mail.

example1.com is a small b to b service business, as accredited by 
accreditors.com

Note: esp.com and personalmail.com set rate limits on a per customer 
basis; these limits are functions of the customer, not functions of the 
service that apply in an undifferentiated manner to all customers. I am 
taking it as given that esp.com and personalmail.com can write code to 
do lookups to check the veracity of their customer's statements on the 
MARID record.

Example2.com:
Example2.com is a new customer at both personalmail.com and esp.com. 
They have no accreditation or reputation, but each service rate limits 
them to 100 messages a day and 400 messages a month, with a total 
address count of 100. In other words, they can not correspond with more 
than 100 different recipients over the period of a month on either 
service.

Receivers side services can sum up the rate limits to calculate the 
total number of recipients and messages that can emanate from 
example2.com and decide whether it exceeds their policies on 
authenticated but unaccredited senders.

Example3.com:
Example3.com has limited internal controls and has decided to outsource 
the whole record management  and sending policy enforcement problem to 
maridRus.com. They know they send outbound mail through their inbound 
servers, but other than that they don't know how they send.

maridrus.com starts by publishing a record referencing the mx records. 
They need two kinds of feedback; abuse complaints, of which they want 
100%, and channels not listed in the example.com records, of which they 
want 1 in 1,000 messages.

Over time, example.com and maridRus.com conclude that esp1.com and 
esp2.com, in use by Departments A and B respectively, are authorized 
vendors, and that they don't need to duplicate complaint monitoring. 
They update the record such that complaints generated on mail on the 
esp1 or esp2 channel are  handled by the esps, and not copied to 
maridRus.com.

Example4.com
Example4.com is an esp bounce handling domain. This domain appears only 
in MAIL-FROM and never in SUBMITTER or the 2822 from address.

Example5.com:
[This is included because I think that over time it will be useful to 
everyone to have receiving as well as sending policies published, and 
having another syntax for that side of the policy equation doesn't make 
much sense to me]

Example5.com publishes their acceptance criteria as well their sending 
policies. They accept mail from authenticated but unaccredited senders 
who do not send more than 100 messages a day or 400 messages a month. 
Otherwise, they accept accreditations from accreditors.com (which 
presumably has pass-through arrangements with some number of other 
accreditors). example5.com requires confirmed opt in from everyone 
except senders accredited as financial institutions by 
verythoroughandexpensiveaccreditations.com. For financial institutions 
they require only prior business relationship, as defined by 
verythoroughandexpensiveaccreditations.com

This is used by esp.com to decide whether or not to send based on the 
characteristics of each of their customers, and to report back to their 
customers on what they need to do to get their mail delivered.

My real concern with SPF is the strong suspicion there is a lot we 
haven't thought of yet.
Margaret.

On Jun 16, 2004, at 6:54 PM, Marshall Rose wrote:

> the working group's deliberations revolve around the sufficiency of 
> the SPF syntax to address near-term concerns. if the syntax is 
> adequate, it is hard to argue for an alternative or future syntax; if 
> the syntax is inadequate, then the SPF syntax is of transitory use.
>

---

From: Jonathan Gardner <[email-masked]>
Date: Mon, 21 Jun 2004 11:41:12 -0700

I think a general summary for my response is that we have to decide a few 
issues.

(1) What will SenderID do? Will it be limited to just authentication, or 
will it be expandable in the future? And authentication of what exactly? 
The sending MTA, or the sending MUA, or what?

If we decide that Sender ID will only authenticate whether a particular MTA 
at an IP address is allowed to send messages for a domain, then Sender ID 
is sufficient.

(2) Also, how will we handle complicated and rare situations? Will we try to 
write a syntax that handles every imaginable case, or will we accept 
limitations to the syntax with the idea that the complicated and rare 
sitatuations should be modified?

My responses are embedded below.

> do lookups to check the veracity of their customer's statements on the

I believe that rate limiting should be beyond the scope of Sender ID. For 
one, it is better implemented on the sending MTA side. This would be an 
implementation detail for the MTA, and requires no external knowledge to 
either example1.com or any receiving MTA.
 
The other possibility is to implement rate limiting on the receiving MTA's 
end. But how can the receiving MTA know how many email messages were sent 
to other MTAs? How can a receiving MTA trust the number that other MTAs 
report to it? These difficulties would greatly complicate Sender ID, and so 
should be external to Sender ID.

Thus, no extensions to Sender ID should be created for rate limiting.

The question of accreditation is also posed. How does example1.com become 
accredited with accreditors.com? How does accreditors.com inform other MTAs 
of their accreditation? How do other MTAs decide what to do with the 
accreditation information? Despite any answers above, accreditation has 
nothing to do with authorization. Either an MTA is allowed to send email 
for a domain or it isn't, and accreditation doesn't affect that.

Thus, no extension to Sender ID should be created for accreditation either.

In summary, no extensions should be made to Sender ID to describe 
example1.com's situation.

> authenticated but unaccredited senders.

The problems with receiving MTAs summing up the number of emails sent for a 
particular domain are outlined above. Drawing from the conclusions above, 
no extensions to Sender ID should be made for this situation.

I believe that entering into a three-way contract like this is difficult at 
best. Rather, example2.com would draw up one contract with personalmail.com 
and another with esp.com. Besides the problems of getting three people to 
completely agree to anything, the technical problem of reporting the number 
of emails sent is not a trivial one.

> esp1 or esp2 channel are  handled by the esps, and not copied to

Complaint monitoring is a complicated example. How are we to inform 
receiving MTAs that they should send complaints? For what situations do 
they send complaints? How do we handle complaint messages to avoid a 
situation where complaint messages generate more complaint messages? Are 
receiving MTAs obligated to send complaints? How do we verify that 
complaint messages are not falsified? Is there a universal standard or do 
we have to negotiate a method of verification?

I believe that complaint messages are beyond the realm of authentication 
because of the difficulties outlined above. The problems above are almost 
larger than the original problem of authentication. If ISPs and ESPs want 
to partner and exchange complaint messages, then that is their right to do 
so. Encoding such an agreement in Sender ID is not appropriate.

Thus, no extensions to Sender ID are needed for this situation.

Note that rather than monitoring complaint messaging, a more effective 
solution would be to audit the email sending facilities of example4.com. 
After the audit, strict Sender ID records could be published. Any remaining 
email sending facilities will later be discovered via bounced messaging or 
human-generated complaints or other signs.

> in MAIL-FROM and never in SUBMITTER or the 2822 from address.

It is the responsibility of the sending MTAs to ensure that email is 
properly formatted. Sender ID, I believe, should only be concerned with 
authorization.

> customers on what they need to do to get their mail delivered.

Acceptance policies are beyond the original premise of Sender ID. I don't 
believe publishing such a policy is necessary. Here's my explanation why.

esp.com should send the messages. If example5.com accepts it but later sends 
a bounce mesage, esp.com should forward the bounced messages from 
example5.com to their customers. If example5.com refuses to accept the 
mesage, esp.com will return a bounced message to their customer with the 
error message. It is example5.com's responsibility to provide accurate and 
descriptive error and bounce messages.

- -- 
Jonathan M. Gardner
Mass Mail Systems Developer, Amazon.com
[email-masked]

---

From: Luis Bruno <[email-masked]>
Date: Mon, 21 Jun 2004 21:31:03 +0100

> at an IP address is allowed to send messages for a domain, then Sender ID 

Sufficient, but not necessary; in other words, overkill.

Paging Hector Santos: I couldn't get email directly to you; 550 Return Path
not verifiable after RCPT TO: (and postmaster@ didn't work :-) )

-- 
Luis Bruno                                UTM: 29T 629481E 4511776N 576m

---

From: Andrew Newton <[email-masked]>
Date: Mon, 21 Jun 2004 16:55:10 -0400

> explaining how the SPF syntax is sufficient.

Just a reminder:  responses on SPF syntax being sufficient are now 
welcome.

-andy

---

From: "Hector Santos" <[email-masked]>
Date: Mon, 21 Jun 2004 17:18:20 -0400

----- Original Message ----- 
From: "Luis Bruno" <[email-masked]>
To: "IETF MARID WG" <[email-masked]>
Sent: Monday, June 21, 2004 4:31 PM
Subject: Re: Drive Towards Consensus [was Re: On Extensibility in MARID
Records]

> not verifiable after RCPT TO: (and postmaster@ didn't work :-) )

Oh, I got something this morning directly from you?

----- Original Message ----- 
From: "Luis Bruno" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Sent: Monday, June 21, 2004 7:01 AM
Subject: Re: MARID Records and the standards process

And I replied to this.

Let me check the anti-spam logs.  Ok, your 7am message was validated
successfully via CBV:

20040621 07:04:31 -------------------------------------
20040621 07:04:31 version    : 1.62 / 1.54
20040621 07:04:31 calltype   : SMTP
20040621 07:04:31 state      : rcpt
20040621 07:04:31 srvdom     : winserver.com
20040621 07:04:31 srvip      : 208.247.131.9
20040621 07:04:31 cip        : 193.137.7.30
20040621 07:04:31 cdn        : republico.estv.ipv.pt
20040621 07:04:31 from       : <[email-masked]>
20040621 07:04:31 rcpt       : <[email-masked]>
20040621 07:04:31 ruid       : 228947
20040621 07:04:31 testorder  : FLT RBL SPF CEP CBV
20040621 07:04:31 sapfilter  : pass (time:62)
20040621 07:04:31 saprbl     : testing 30.7.137.193.sbl.spamhaus.org
20040621 07:04:33 saprbl     : testing 30.7.137.193.list.dsbl.org
20040621 07:04:34 saprbl     : testing 30.7.137.193.bl.spamcop.net
20040621 07:04:35 saprbl     : pass (time:3485)
20040621 07:04:40 sapspf     : none (time:4921)
20040621 07:04:40 sapcep     : test from=republico.estv.ipv.pt
20040621 07:04:44 sapcep     : none (time:4875)
20040621 07:04:46 sapcbv     : total mx records: 0
20040621 07:04:51 try domain : republico.estv.ipv.pt ip: 193.137.7.30
20040621 07:04:51 # connecting to 193.137.7.30
20040621 07:04:52 S: 220 republico.estv.ipv.pt ESMTP Exim 4.22 Mon, 21 Jun
2004 12:02:15 +0100
20040621 07:04:52 C: NOOP WCSAP v1.62 Wildcat! Sender Authentication
Protocol http://www.santronics.com
20040621 07:04:52 S: 250 OK
20040621 07:04:52 C: HELO mail.winserver.com
20040621 07:04:52 S: 250 republico.estv.ipv.pt Hello ntbbs.winserver.com
[208.247.131.9]
20040621 07:04:52 C: MAIL FROM: <>
20040621 07:04:53 S: 250 OK
20040621 07:04:53 C: RCPT TO: <[email-masked]>
20040621 07:04:53 S: 250 Accepted
20040621 07:04:53 C: RCPT TO: <[email-masked]>
20040621 07:04:53 S: 550 relay not permitted
20040621 07:04:53 C: QUIT
20040621 07:04:53 sapcbv     : 250
20040621 07:04:53 result     : accept (-1)
20040621 07:04:53 wcsap finish (22172 msecs)
20040621 07:06:17 -------------------------------------

Why no SPF record?  <g>

I sent a reply to you, and I see a 10am transaction from you which failed
due to your return domain failed.

A 451 response was issued to allow you to try again.  It was tried 2-3 more
times.

20040621 10:18:42 -------------------------------------
20040621 10:18:42 version    : 1.62 / 1.54
20040621 10:18:42 calltype   : SMTP
20040621 10:18:42 state      : rcpt
20040621 10:18:42 srvdom     : winserver.com
20040621 10:18:42 srvip      : 208.247.131.9
20040621 10:18:42 cip        : 193.137.7.30
20040621 10:18:42 cdn        : republico.estv.ipv.pt
20040621 10:18:42 from       : <[email-masked]>
20040621 10:18:42 rcpt       : <[email-masked]>
20040621 10:18:42 ruid       : 228947
20040621 10:18:42 testorder  : FLT RBL SPF CEP CBV
20040621 10:18:42 sapfilter  : pass (time:63)
20040621 10:18:42 saprbl     : testing 30.7.137.193.sbl.spamhaus.org
20040621 10:18:42 saprbl     : testing 30.7.137.193.list.dsbl.org
20040621 10:18:43 saprbl     : testing 30.7.137.193.bl.spamcop.net
20040621 10:18:49 saprbl     : pass (time:6906)
20040621 10:18:49 sapspf     : none (time:703)
20040621 10:18:49 sapcep     : test from=republico.estv.ipv.pt
20040621 10:18:50 sapcep     : none (time:1282)
20040621 10:19:00 sapcbv     : rejected - can not resolve
republico.estv.ipv.pt
20040621 10:19:00 result     : reject (0)
20040621 10:19:00 smtp code  : 450
20040621 10:19:00 reason     : Rejected by WCSAP CBV
20040621 10:19:00 wcsap finish (19094 msecs)
20040621 10:19:50 -----------------------------------

I just tried again manually and your domain still fails MX and A record
lookups:

d:\wc5beta>nslookup -query=mx republico.estv.ipv.pt

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to ns1.mia.bellsouth.net timed-out

d:\wc5beta>nslookup -query=a republico.estv.ipv.pt

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to ns1.mia.bellsouth.net timed-out

Lesson/Notes here learned?

Strict SMTP compliancy works for valid return addresses works.  Spammers
will not complain about False Positives. However, legitimate people will.
But I don't see the FAULT in the SMTP operation.  It did its job as it
suppose to behave with a strong enforcment of SMTP compliancy - meaning that
ADDRESS better be good!    By far, this approach as eliminate a majority of
the anonymous mail abuse.

When MARID is implemented, the 2821 portion of it will replace MCEP (SAPCEP)
logic above.  At some point, I hope it to replace SPF, but SPF will probably
not be removed with the initial implementation.

The MARID 2822 logic will be added AFTER the 2821 is validated.  Nothing
from I see in Microsoft MCEP logic will validate this type of transaction
with a high degree of trust.  That address better be good when it is
provided at MAIL FROM:

-- Hector

---

From: Margaret Olson <[email-masked]>
Date: Mon, 21 Jun 2004 19:50:38 -0400

> either example1.com or any receiving MTA.
If the receiving MTA wants to know if this small business sends at a 
rate consistent with a legitimate small business or at a rate 
consistent with a spammer, then the receiving MTA will want to know the 
rates. They will also want to know if the sending MTA accredited as 
enforcing sending limits; this is why you need to know things about 
both the channel and the sender where the sender and the channel are in 
different domains. In many cases, they will be in separate domains; 
there are millions of domains that do not run their own MTAs.

> should be external to Sender ID.
The rate limiting needs to occur on the sending side, but the receiver 
needs to know what it is.

It is true that we don't at this moment know the specifics of how the 
accreditation, reputation, and rate limiting will work. But these areas 
are emerging; the fact that we don't have a spec in hand is not a 
reason to dismiss the need for them. If anything it's an argument for 
flexibility.

Margaret

---

From: Roy Badami <[email-masked]>
Date: Tue, 22 Jun 2004 01:04:44 +0100

>>>>> "Margaret" == Margaret Olson <[email-masked]> writes:

    Margaret> The rate limiting needs to occur on the sending side,
    Margaret> but the receiver needs to know what it is.

But there's no need for the sender to publish the rate limiting
information, because there's little reason for you to trust them.  It
make more sense for the sender to simply refer to a rate limit policy
published by their ISP (which could be achieved by an SPF modifier).

The reciever would then query the ISPs rate limiting policy for this
domain and (assuming positive accreditation of the ISP) the recipient
would then believe it.

Obviously the ISP's rate limiting policy would have to specify the IP
addresses that they're responsible for (they can't rate limit mail you
send out via another source) but this is all information that should
be published by the ISP, not the sender.

So I don't believe it belongs in policies published by the sender, but
in a completely new type of policy published by ISPs who send mail on
behalf of their customers.

Though, as I said before, I'd like to see a move towards (return to)
sites originating (and being responsible for) their own mail rather
than relaying through their ISP...

   -roy

---

From: Ted Hardie <[email-masked]>
Date: Mon, 21 Jun 2004 18:20:14 -0700

Reading Mark's post and the series of replies, I believe it would be fair
to say that his vision of SPF/MARID is a record which allows a
domain to make assertions about its own infrastructure that an MTA
might be able usefully to check when deciding whether to accept
traffic which asserts a relationship to that domain.

The extensions of "accreditation, reputation, and rate limiting" which
Margaret describe fundamentally involve a 3rd party (since rate limiting
only looks really useful if audited).  If that is correct, the question
becomes: are these third party assertions something that need to be
tied to the individual MARID assertions, are they an independent
pointer within the record, or are they out of scope?

Let's assume for now that at least the pointers to them are in scope.

Imagine for a moment that MARID includes an extension that says:  these
other people also wish to make assertions about me (with the implication
that the domain knows about it and is willing to point to it).  Is it
possible to extend MARID to do that?  Sure.  Give those records a 
similar syntax
to MARID and it becomes a set of assertions made by two or more different
parties about the same domain's infrastructure. Silly states then require
meta-reputation to resolve, but that may not be a big deal. Local policy
can resolve those towards permissiveness or restriction as required.

Is it also possible to do that such that it could be attached to other
assertions, as a generic mechanism for per-attribute 
reputation/accreditation/certification?  I think so.  That would,
again, require the other domain to maintain some record that
allowed you to check that they did make such an assertion.   As long
as the syntax for the per-attribute pointers is reasonably broad,
they could be records associated with multiple protocols or domains.
This way forward would also allow the original domain owner to specify
for which attributes the other domains' views were acceptable to the domain
owner (not binding on the recipient, but potentially a useful data point).

But the usefulness of either goes back to the issue raised by Roy Badami--what
does this mean in the context of the extension processing?

The WG discussions so far seem to have gone down two processing paths:
one in which MARID processing iterates through separate assertions until
one assertion explicitly allows, disallows or marks unknown the 
message traffic;
the other in which all the assertions are taken together to create three sets:
allowed, disallowed, not known.   In the first approach, new assertions can
easily be defined to have a null affect.  If the working group takes that
approach, though, it seems fairly critical to define the generic 
"external pointer"
syntax now, so that the use of external pointers modifying base assertions
does not later render null assertions which would otherwise
allow or disallow traffic.  An alternate approach there is to specify that
modifiers of a certain type can be ignored such that the base processing
of the assertions can proceed if they are not understood.  That would mean that
the syntax of the "external pointer" could be left vague right now, as long
as it is known to be of that type.

Since the "set construction" mechanism presumes that you will process
all the assertions to construct the sets, it is somewhat easier to have
mechanisms which allow assertions to be about other assertions.  The
ability to handle nested assertions is also very handy here.  The
difficulty with this approach and 3rd party assertions is that the
extension mechanism must describe what happens when they cannot
be resolved, and the choice of "pretend it wasn't there" is not so
easy in the face of nesting and assertions about assertions.  A
common, practical approach at that point is to limit external assertions
to ones in which "pretend it wasn't there" processing does no
harm (viz. the GeoPriv limitation that all rules must add permissions,
so that the failure to resolve external reference cannot result
in disclosure of private information).  That can both be a serious
limitation and difficult to get right when the base assertion and
the 3rd party assertion about it are not coordinated.

Without stepping on the toes of the original question asked by the
chairs, I'd be interested in hearing whether anyone believes MARID
will need the "set construction" mechanism, or whether it can
go forward with the iterative processing method.  That may
tell us something about the extension mechanism choices and
what our core needs for syntax are.

				regards,
					Ted Hardie

---

From: wayne <[email-masked]>
Date: Mon, 21 Jun 2004 22:31:26 -0500

> the syntax is inadequate, then the SPF syntax is of transitory use.

I've read the proposed scenarios and the replies to them.  It is hard
to give an overall response without making bland assertions, but it is
hard to give point-by-point responses without getting lost in the
details.

So, in this post, I'll give my overall view of these situations,
details will follow in other posts.

People advocating XML have given scenarios which are mostly fleshed
out versions of what they have been saying all along.  This is very
good, and it also means that I wasn't surprised by anything.

Overall, I found none of the scenarios to give a compelling reason why
requiring support of XML would outweight its the negative effects.
(These negative effects of XML are caused by both technical and, uh,
"emotional" reasons.  It is tempting to dismiss the "emotional"
reasons, but we have to deal with the real world here.)

Briefly, all of the scenarios fall into one or more of the following
catagories:

* They can easily be handled directly with the current SPF syntax.

* They can easily be handled with an SPF modifier that points to where
  the information can be found.

* They are so far beyond the scope of MARID that they dilute and
  muddle the purpose.

* They are not useful because no one will believe unverifiable claims,
  such as "I send only 100 emails per day".

* They can best be handled other ways.

* They require email receivers to do work for the senders with no
  benefit for the receivers.  As a result, I don't see them being
  implemented by the receivers.

-wayne

---

From: wayne <[email-masked]>
Date: Tue, 22 Jun 2004 01:01:54 -0500

In <[email-masked]> Margaret Olson <[email-masked]> writes:

> (corporate individual mailboxes) and rate limited to 100 messages a
> day by personalmail.com
>
> basis; these limits are functions of the customer, not functions of
> the service that apply in an undifferentiated manner to all
> customers. I am taking it as given that esp.com and personalmail.com
> can write code to do lookups to check the veracity of their customer's
> statements on the MARID record.

With SPFv1, this would be expressed as:

"v=spf1 include:esp.com include:personalmail.com -all accred=%{d}.accreditors.com"

Rate limiting must be done via the MTAs from those other companies and
so that is where such information should be obtained.  Otherwise, such
information is not trustworth.

So, you could have an SPF record that says:

"v=spf1 include:esp.com include:personalmail.com -all
 rate=%{d}._rates.esp.com rate=%{d}._r.personalmail.com"

The receiver could then do a DNS lookup on example2.com.rates.esp.com,
and parse the TXT record, or use information from an A record or
something.

> Example2.com:
> Example2.com is a new customer at both personalmail.com and
> esp.com. They have no accreditation or reputation, but each service
> rate limits them to 100 messages a day and 400 messages a month, with
> a total address count of 100. In other words, they can not correspond
> with more than 100 different recipients over the period of a month on
> either service.
>

See above for example1.com.  I think it addresses this issue.

> Example3.com:
> Example3.com has limited internal controls and has decided to
> outsource the whole record management  and sending policy enforcement
> problem to maridRus.com. They know they send outbound mail through
> their inbound servers, but other than that they don't know how they
> send.
>
> maridrus.com starts by publishing a record referencing the mx
> records. They need two kinds of feedback; abuse complaints, of which
> they want 100%, and channels not listed in the example.com records, of
> which they want 1 in 1,000 messages.

Abuse complaints is a hard subject.  Many people do not want to send
abuse complaints to just anyone because spammers will listwash and/or
mark your email address as being live.

Right now, you have things like registering at abuse.net and
spamcop.net.  There is also the RP DNS record that can be used.  (RP is
the "Responsible Person" for the domain, see RFC1183 published in
1990.)  Yes, you could also add a "rp=<email address>" modifier to the
SPF record, but I'm not sure if that is a good idea.

As far as getting information on 0.1% of the messages, I'm not sure I
see that as being much more useful than tracking 100% of the
messages.  If you send X million emails, you are likely to get some
fraction of that X million DNS lookups anyway.  Using an SPF exists:
tracking mechanism, you can efficiently track 100% of the invalid
messages. 

You can use SPF right now to do sampling with records such as:

example3.com publishes:
"v=spf1 mx redirect=%{d}._customer.maridRus.com"

on example3.com._customer.maridRus.com, we have:

for 0.1% of the time:
"v=spf1 exists:CL.%{i}.FR.%{s}.HE.%{h}.null.%{d} ?all"

For the remaining time:
"v=spf1 ?all"

This can be done with a very simple script, right now, with SPF
without any changes at all.  You don't have to depend on the receiving
side SPF implementation understanding and obeying a new extension.
Depending on receiving systems will create a huge sampling bias.  You
can fine tune things so that the tracking SPF record is deployed more
at night than during the day because the spam volume tends to stay
much more constant, but your server load is lower.  You can deploy the
tracking SPF record right when you know you are sending a burst of
email.

You can do all sorts of stuff that is completely under the domain
owner's control and you can do it today.  It would take me about 15
minutes to create and test a script to do this on my name server.

If you want to get fancier, you can hack up a name server to give out
the tracking SPF record one out of 1000 times it is requested.  You
could decide by the domain requesting the SPF record what kind of
tracking you want done and have the name server respond accordingly.

Could you imagine the email policy description that would do something
like "send me information about invalid domain name usage, but only
around 08:00 UTC on the first monday of each month, and only if your
from domains x, y, or z, or in *.co.uk, but not ebay.co.uk".  You can
do this today with SPF and a little creativity with your name server.
You will almost certainly never be able to do this if you try adding
extensions, even if you use XML.

Basically, I'm saying that the right place to evaluate these decisions
is at the ESP, rather than forcing/depending on all the receivers in
the world to do the decision making for you.

> vendors, and that they don't need to duplicate complaint
> monitoring. They update the record such that complaints generated on
> mail on the esp1 or esp2 channel are  handled by the esps, and not
> copied to maridRus.com.

Ok, so on example3.com._customer.maridRus.com, we change to:
"v=spf1 include:esp1.com include:esp2.com ?all"

As far as the abuse reports go, I think it is an incredibly bad idea
to send email to some arbitrary email address based on information
published in a possibly mallicious DNS record.  That is one reason
that the RP DNS record has never found favor.  Moreover, many people
do not want to send abuse reports directly to people who may well be
spammers or phishers.

The only real solution to abuse complaints are register with
spamcop.net, abuse.net and/or send email to [email-masked].  If
example3.com doesn't want to deal with abuse complaints, it can
forward those on to someone else.

> Example4.com
> Example4.com is an esp bounce handling domain. This domain appears
> only in MAIL-FROM and never in SUBMITTER or the 2822 from address.

"v=spf1 +all 2822.from=never"

Now, this isn't a really good solution in either SPF-classic nor
SPF-ID, but the syntax is there.

> having another syntax for that side of the policy equation doesn't
> make much sense to me]
>
> who do not send more than 100 messages a day or 400 messages a
> month. Otherwise, they accept accreditations from accreditors.com
> (which presumably has pass-through arrangements with some number of
> other accreditors). example5.com requires confirmed opt in from
> everyone except senders accredited as financial institutions by
> characteristics of each of their customers, and to report back to
> their customers on what they need to do to get their mail delivered.

Example5.com publishes
"v=spf1 .... incoming-policy=http://www.%d/incoming-policy.xml"

I don't see how your paragraph above leverages much, if anything, of
the SPF semantics, and it won't fit into a DNS lookup.

> My real concern with SPF is the strong suspicion there is a lot we

I'm sure you are right.  I don't see anything that causes a need for
XML though.

-wayne