Subject: co-chair judgment of consensus related to last call period of 23-Aug-2004 to 10-Sept-2004

From: Andrew Newton <[email-masked]>
Date: Sat, 11 Sep 2004 13:59:06 -0400

The following is the judgment of the co-chairs relating to consensus 
within the MARID working group during the last call period of 
23-August-2004 through 10-September-2004.

1) On the issue of using a DNS name prefix, there is at least rough 
consensus that no prefix should be used.

2) On the issue of compliance with the use of the TXT record, the 
working group has at least rough consensus that TXT usage is acceptable 
for compliance and should not be specified as a configuration that will 
be non-compliant.  However, there is at least rough consensus that the 
use of the SPF-specific record type is more desirable than the use of a 
TXT record type.  It is the opinion of the co-chairs that the -protocol 
document clearly state that the usage of TXT records will most likely 
be deprecated by future protocol definition.

3) On the issue of ignoring patent claims, the working group has at 
least rough consensus that the patent claims should not be ignored.  
Additionally, there is at least rough consensus that the participants 
of the working group cannot accurately describe the specific claims of 
the patent application. This stems from the fact that the patent 
application is not publicly available.  Given this, it is the opinion 
of the co-chairs that MARID should not undertake work on alternate 
algorithms reasonably thought to be covered by the patent application.  
We do feel that future changes regarding the patent claim or its 
associated license could significantly change the consensus of the 
working group, and at such a time it would be appropriate to consider 
new work of this type.

4) On the issue of the ABNF in section 3.4.1 of -protocol and multiple 
scopes, there is at least rough consensus to allow the syntax and 
record structure to support multiple scopes.

With regard to items 3 and 4 above, it is also the opinion of the 
co-chairs that any attempt by the MARID working group to define any new 
scopes other than "mailfrom" and "pra" for the SPF syntax will at this 
time result in failure to find consensus within the working group.

The document authors have agreed to producing new drafts intended to 
meet the chartered work item, and a consensus call on them or the 
appropriate diffs will be forthcoming.  This work plan does not include 
scopes outside of "mail from" and "pra", and it is our opinion that no 
new work items of this type should be considered until MARID has 
successfully produced a first specification.

-MARID co-chairs

---

From: wayne <[email-masked]>
Date: Sat, 11 Sep 2004 14:42:27 -0500

In <[email-masked]> Andrew Newton <[email-masked]> writes:

> application is not publicly available.

Agreed.

>                                         Given this, it is the opinion

Uh, can you give an example of an alternative algorithm to the PRA
that has been discussed that might be "reasonably thought to be
covered by the PRA patent"?   I can't think of any.

As per RFC3668, which all participants of this mailing list are
supposed to comply with, if we start creating an alternative that does
happen to be covered by any IPR of any other participant, we will
learn about such encumbrance.

See:
http://www.ietf.org/NOTEWELL.html
http://www.ietf.org/rfc/rfc3668.txt

So, while I agree that we shouldn't try to create a standard that we
know is encumbered, isn't this pretty much what we have been trying to
do already?

> co-chairs that any attempt by the MARID working group to define any
> new scopes other than "mailfrom" and "pra" for the SPF syntax will at
> this time result in failure to find consensus within the working group.

It is the opinion of this participant that the pra scope will not
reach consensus due in large part to the patent mess, whether there is
a mailfrom scope or not.

It is the opinion of this participant that the mailfrom scope will not
reach consensus. unless it also allows for the helo scope to be
checked and thus be compatible with SPF-classic.  SPF-classic is the
current defacto standard and I think standardizing existing practice
is the easiest way to reach a consensus.

>                                         This work plan does not
> include scopes outside of "mail from" and "pra", and it is our opinion
> that no new work items of this type should be considered until MARID
> has successfully produced a first specification.

Given this work plan, I do not expect this working group to ever
produce a first specification.   *very heavy sigh*

-wayne

---

From: Hadmut Danisch <[email-masked]>
Date: Sat, 11 Sep 2004 21:58:00 +0200

> successfully produced a first specification.

Is there any (estimated) timeline?

Hadmut

---

From: Chuck Mead <[email-masked]>
Date: Sat, 11 Sep 2004 15:59:08 -0400

Andrew Newton wrote:

| 3) On the issue of ignoring patent claims, the working group has at
| least rough consensus that the patent claims should not be ignored.
| Additionally, there is at least rough consensus that the participants of
| the working group cannot accurately describe the specific claims of the
| patent application. This stems from the fact that the patent application
| is not publicly available.  Given this, it is the opinion of the
| co-chairs that MARID should not undertake work on alternate algorithms
| reasonably thought to be covered by the patent application.  We do feel
| that future changes regarding the patent claim or its associated license
| could significantly change the consensus of the working group, and at
| such a time it would be appropriate to consider new work of this type.

I have to ask because it appears to me that you have left this somewhat
nebulous.

Does this mean that the PRA (with it's encumbrance) is being accepted as
a part of the MARID specification?

If so I do not know what in the world you could have been reading to
draw the conclusion that there was "rough concensus" to go forward with it!

- --
[email-masked], head geek
http://moongroup.com

---

From: Guillaume Filion <[email-masked]>
Date: Sat, 11 Sep 2004 17:05:46 -0400

Andrew Newton wrote:
> [I]t is the opinion of the 
> co-chairs that MARID should not undertake work on alternate algorithms 
> reasonably thought to be covered by the patent application.  We do feel 
> that future changes regarding the patent claim or its associated license 
> could significantly change the consensus of the working group, and at 
> such a time it would be appropriate to consider new work of this type.

I'm not sure if I get this right, and by reading the replies it seems 
that we can't reach consensus of what you mean. hihihi

Reading point 3, I understand that the MARID co-chairs seek an 
alternate, encumbered algorithm to replace the PRA algorithm. Is this 
correct?

Regards,
GFK's
-- 
Guillaume Filion, ing. jr
Logidac Tech., Beaumont, Québec, Canada - http://logidac.com/
PGP Key and more: http://guillaume.filion.org/

---

From: Daniel Quinlan <[email-masked]>
Date: 11 Sep 2004 14:48:14 -0700

The IETF MARID WG co-chair wrote:

>> Given this, it is the opinion of the co-chairs that MARID should not
>> undertake work on alternate algorithms reasonably thought to be
>> covered by the patent application.

> know is encumbered, isn't this pretty much what we have been trying to

I would also like to hear the answers to Wayne's above questions.
(I agree with the point he is trying to make.)

> reach consensus due in large part to the patent mess, whether there is

Agreed.  If "mailfrom" is the only other scope, then I suspect Apache
SpamAssassin, Apache James, and perhaps other open source
implementations will wholly reject Sender ID.

Maybe that's what some people want.  I'm perplexed by this whole
direction.
 
> is the easiest way to reach a consensus.

Agreed, a "helo" scope should be explored.
 
> produce a first specification.   *very heavy sigh*

I disagree with this.  I think we might produce a specification that
will be largely ignored or succeed in handing a major part of the
internet to anyone who brings patented technology into IETF standards.
I don't mean to be inflammatory, but those are the only two *likely*
outcomes I can see for the current plan:

  - the specification doesn't happen or is ignored
  - email authentication is owned by (the patent holder) since (the
    patent holder) can change the revocable patent license in the future

It's *possible* the patent(s) will not be granted, but I prefer to avoid
relying on things outside of my control as much as possible.

Daniel

-- 
Daniel Quinlan
http://www.pathname.com/~quinlan/

---

From: "Mark C. Langston" <[email-masked]>
Date: Sat, 11 Sep 2004 15:01:03 -0700

> Agreed, a "helo" scope should be explored.

Given that we're treading down this path, I'm wondering if it wouldn't
be best to specify a range of scopes that cover the major elements of
information supplied in both RFC2821 RFC2822.  Then, if people wish to
combine certain scopes in particular ways to provide unique solutions,
and/or wish to patent these approaches, that's up to them.  The 
documents would become references for what should be checked, and how.

However, at that point, I think we'd be writing best practices for
RFC2821 and RFC2822 checking for various types of abuse: spam,
joe-jobbing, etc., as well as specifying mechanisms that aid in these
"atomic" information checks.

-- 
Mark C. Langston            GOSSiP Project          Sr. Unix SysAdmin
[email-masked]   http://sufficiently-advanced.net    [email-masked]
Systems & Network Admin      Distributed               SETI Institute
http://bitshift.org       E-mail Reputation       http://www.seti.org

---

From: "william(at)elan.net" <[email-masked]>
Date: Sat, 11 Sep 2004 16:14:35 -0700 (PDT)

> consensus that no prefix should be used.
I'm not sure there was rought consensus on this. I'll try to calculate 
number of posts for and against and post results, but unless I'm mistaken 
it was equal! I've privately made recomendation to co-chairs on this
issue that it would be appropriate to ask official opinion of dns expert 
(meaning WG tech advisor on dns issues) and in fact co-chairs privately
also agree that it is a good idea; it is unfortunete that they know 
decided the issue on their own without the expert opinion or clear
consensus from the WG.
 
> be deprecated by future protocol definition.
Yes, it seems that most opinions expressed were comparable to above.
There does seem rough consensus on this.

> least rough consensus that the patent claims should not be ignored.  
Actually I have not seen rough consensus that patent claims should not be 
ignored, but I don't think there was consensus that they should be ignored 
either (not until we can read the actual patent), so we're still in limbo.

> algorithms reasonably thought to be covered by the patent application.  
On this I STRONGLY disagree.  I believe there was in fact consensus that
we should work on alternative to PRA that is not covered by patent. 

I also note that in light of that we do not know what exactly patent said,
nobody can not make a judgement call that any alternative would be covered
by patent. In fact if we do work on alternative and it turns out to be 
covered by patent we'll know it because there would be another IPR 
disclosure as this is in fact appropriate way to describe patent claims.

I'll publicly say that I'll ignore chairs recomendation that we do not 
work on alternatives to PRA because I believe based on opinion expressed
by the large number of members of this WG that PRA would not reach consensus
of this WG to proceed to be a standard unless we provide an alternative that
does not have licensing limitations.

> working group, and at such a time it would be appropriate to consider 
Actually there was some consensus that changes to the license may lead to 
rough consensus on PRA, so at such time if changes are done we should 
consider it, but NOT the other way around!
 
> record structure to support multiple scopes.
Yes.
 
> time result in failure to find consensus within the working group.
I do not see how you can make this judgement without actually seeing what 
alternatives are.
 
> successfully produced a first specification.
I'm going to disagree and work on alternatives, because I believe this WG
will be unable to produce specification on PRA scope in its current form.

-- 
William Leibzon
Elan Networks
[email-masked]

---

From: "william(at)elan.net" <[email-masked]>
Date: Sat, 11 Sep 2004 16:31:15 -0700 (PDT)

It seems I made a mistake and did not notice that chairs did not say that 
we should not work alternatives AT ALL but said that we should not work 
on alternatives that WOULD BE covered by patent application.

The above is appropriate, however I note that chairs said that we should 
not work on alernatives that "reasonably thought to be covered by the 
patent application". My objection still stands that in itself is
meaningless because we'll know if alernative is covered when there 
would be another IPR disclosure as it relates to such possible alernative 
draft. Until there is one, we can not reasonable be expected to know what 
would be covered by patent application because we have not seen one!

> does not have licensing limitations. 

---
William Leibzon, Elan Networks:
 mailto: [email-masked]
Anti-Spam Research Worksite:
 http://www.elan.net/~william/asrg/

---

From: George Mitchell <[email-masked]>
Date: Sat, 11 Sep 2004 17:57:15 -0700 (PDT)

Congratulations to the chairs for your work on a contentious subject.
I think you mostly got it right.

> [...]

This is the troublesome part.  I think everyone agrees on "mailfrom",
but not on "pra".  Also, what happened to "helo"?

> successfully produced a first specification.
Despite my nitpicking, thanks for this sign of forward progress.
-- George Mitchell

---

From: Yakov Shafranovich <[email-masked]>
Date: Sat, 11 Sep 2004 21:30:14 -0400

Andrew Newton wrote:
> Additionally, there is at least rough consensus that the participants of 
> the working group cannot accurately describe the specific claims of the 
> patent application. This stems from the fact that the patent application 
> is not publicly available.  Given this, it is the opinion of the 

Are you referring to fetchmail and similar attempts?

...

> successfully produced a first specification.

Andy,

Does this mean that the PRA scope can proceed?

Yakov

---

From: "Eric A. Hall" <[email-masked]>
Date: Sun, 12 Sep 2004 12:24:14 -0400

> be deprecated by future protocol definition.

I don't think that's good enough and would offer an alternative:

 * specs sent to last call MUST include a new RRType code

 * an addendum or a sibling spec MAY describe coexistence mechs for
   a TXT RR for backwards compatibilty

This model ensures that future implementations cannot claim "compliance"
until they fix whatever bugs are keeping them from using the new RRType,
while also providing a support channel for legacy implementations.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: Dave Crocker <[email-masked]>
Date: Sun, 12 Sep 2004 10:06:12 -0700

Andrew,

AN> 2) On the issue of compliance with the use of the TXT record, the 
AN> working group has at least rough consensus that TXT usage is acceptable
AN> for compliance and should not be specified as a configuration that will
AN> be non-compliant.  However, there is at least rough consensus that the
AN> use of the SPF-specific record type is more desirable than the use of a
AN> TXT record type.  It is the opinion of the co-chairs that the -protocol
AN> document clearly state that the usage of TXT records will most likely
AN> be deprecated by future protocol definition.

Please clarify.

What, exactly, would be required to be supported and what would be
optional?

In particular, how will interoperability be assured at all stages of
deployment and use?

To be entirely pedantic, how will the specification ensure that the
client does not implement an incompatible capability from the server?

If client and server do no share at least one MUST, then there will be
non-interoperability.

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>

---

From: Peter Koch <[email-masked]>
Date: Sun, 12 Sep 2004 19:50:38 +0200

>    a TXT RR for backwards compatibilty

I fully agree. And the question of whether a prefix is necessary or not
is of course closely related to the choice of RR type. TXT would need
a prefix, a new type would not.
And regarding the "installed base": while I appreciate the efforts by the
proponents and the early adopters, recent counts for a certain larger
TLD show that there are only a few thousand TXT RRs with "spf" content,
often multiple of those existing in (below) the same 2nd level domain.
Less than 2% of TXT RRs are used for SPF.
This is of course a success for this project but does not constitute
an installed base cast in stone compared to what would be necessary to
talk about wide deployment. A6 seems to have been more popular before it
was redirected down the experiment path.

-Peter

---

From: Frank Ellermann <[email-masked]>
Date: Sun, 12 Sep 2004 22:34:21 +0200

George Mitchell wrote:

> I think everyone agrees on "mailfrom", but not on "pra".

Not sure about everyone, but certainly me.

>  Also, what happened to "helo"?

IIRC the idea was to discuss CSV after SPF, and adding it now
before this discussion could be confusing.  As soon as we have
completed spf2.0/mailfrom adding other scopes like PRA or HELO
is easy.  Maybe PRA is then unnecessary, and we find a simple
solution by matching the relevant headers against MAIL FROM:

If nothing matches, and if there's no Sender then it's probably
a case of "2476bis 8.1 MAY-be not", otherwise it's a phisher.

This could also solve the problem mentioned by Wayne and Anne:
| In this sense, neither protecting almost invisible things
| like the Return-Path: nor the Resent-From: header really
| satisify their concerns.
                            Bye, Frank

---

From: Daniel Quinlan <[email-masked]>
Date: 13 Sep 2004 00:57:14 -0700

> If client and server do no share at least one MUST, then there will be

Yes.  It would also be nice if the world could avoid doing duplicate DNS
queries for spf2 records.  I don't think a single person has disagreed
with that design goal and I noted this in my earlier comments, so I hope
that is addressed in the next revision of the specifications.  One or
the other forever or both with a pre-specified flag day in the
not-too-distant future.

Daniel

---

From: Chuck Mead <[email-masked]>
Date: Mon, 13 Sep 2004 13:20:29 -0400

Chuck Mead wrote:
|
| Andrew Newton wrote:
|
| | 3) On the issue of ignoring patent claims, the working group has at
| | least rough consensus that the patent claims should not be ignored.
| | Additionally, there is at least rough consensus that the participants of
| | the working group cannot accurately describe the specific claims of the
| | patent application. This stems from the fact that the patent application
| | is not publicly available.  Given this, it is the opinion of the
| | co-chairs that MARID should not undertake work on alternate algorithms
| | reasonably thought to be covered by the patent application.  We do feel
| | that future changes regarding the patent claim or its associated license
| | could significantly change the consensus of the working group, and at
| | such a time it would be appropriate to consider new work of this type.
|
| I have to ask because it appears to me that you have left this somewhat
| nebulous.
|
| Does this mean that the PRA (with it's encumbrance) is being accepted as
| a part of the MARID specification?

It would be nice if the co-chairs would be willing to answer this
question. Thanx.

- --
Chuck Mead <[email-masked]>
Instructor II (and resident Postfix bigot), GLS
Disclaimer: "It's Thursday and my name is Locutus of B0rk!"
Addendum: "Bwahahaha! Fire up the orbital mind-control lasers!"

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Sep 2004 11:24:30 -0700

Daniel,

>> If client and server do no share at least one MUST, then there will be

DQ> Yes.  It would also be nice if the world could avoid doing duplicate DNS
DQ> queries for spf2 records.

My concern is not about efficiency. It is about the fundamental ability
to get any successful interoperability.

If there is no core requirement for support of a particular record type,
then there will be clients that support only one record type and servers
that support only another.  This means that the clients will not be able
to access the information.

The instant you talk about "duplicate queries" you are assuming that the
client supports more than one type.  The question is whether they are
required to.  That is, do they have a MUST for each type?

DQ>  so I hope
DQ> that is addressed in the next revision of the specifications.  One or
DQ> the other forever or both with a pre-specified flag day in the
DQ> not-too-distant future.

For Internet-scale services there can be no such thing as a flag day.
There is not central authority to assert it.  Even if there were, there
is no way to coordinate that scale of operation.  Even centrally
controlled companies cannot do flag-days for large-scale deployment
changes.

d/

---

From: Daniel Quinlan <[email-masked]>
Date: Mon, 13 Sep 2004 12:13:53 -0700

Dave Crocker <[email-masked]> writes:

> My concern is not about efficiency. It is about the fundamental
> ability to get any successful interoperability.

I'm concerned about that as well, but efficiency is a concern if you're
trying to process mail as quickly as possible.  The limit of 10
redirects/includes is a bit worrisome too.

> If there is no core requirement for support of a particular record
> type, then there will be clients that support only one record type and
> servers that support only another.  This means that the clients will
> not be able to access the information.
>
> The instant you talk about "duplicate queries" you are assuming that
> the client supports more than one type.  The question is whether they
> are required to.  That is, do they have a MUST for each type?

There definitely needs to be a shared MUST for one type.  At that point,
though, how much point is there in even mentioning the other?

I absolutely don't want to see a MUST for two types for the receiver if
there's no schedule for phasing TXT out.

> There is not central authority to assert it.  Even if there were,
> there is no way to coordinate that scale of operation.  Even centrally

I agree it's not entirely practical, but as there is no easy or fast way
to deploy new RR types, we're limited to the non-practical.  Your point
is well made.  Maybe "flag day" is overstated, but a predefined
recommended phase-out schedule would be a good idea if the TXT record is
optional.

Daniel

---

From: wayne <[email-masked]>
Date: Mon, 13 Sep 2004 22:28:28 -0500

> Less than 2% of TXT RRs are used for SPF.

I seem to recall a survey of the .de TLD, but I can't find it.  (The
closest I found was your survey of DNS servers.)

If I recall correctly, the survey counted things a little bit
differently than the surveys that I've done, but the results were
reasonablely consistent when the differences were taken into account.

For example, you say that 2% of TXT records are used for SPF.  This
may well be true, but at the spot in the domain hierarchy where SPF
records are likely to be checked (the email domain names), I found 19%
back in May and 36% in my most recent survey.

Can you give me a pointer to this survey that you are referring to?
Can you explain the differences between the 2% you quote and the 36% I
found?

Across all domains, the total is certainly over 100,000 as the SPF
adoption roll is just under that number.  Many of the SPF records I
found in my survey aren't on the adoption roll since the adoption roll
is completely optional.  I would guess that there are somehwere
between 150,000 and 300,000 SPF records currently deployed.

-wayne

---

From: Jonathan de Boyne Pollard <[email-masked]>
Date: Mon, 13 Sep 2004 15:21:35 +0100

PK> [...] the question of whether a prefix is necessary or not is of
PK> course closely related to the choice of RR type. TXT would need
PK> a prefix, a new type would not.

A new resource record type would require a "prefix" for exactly the same
reasons that the "TXT" resource record type would.  The benefit of a
prefix in this instance is to reduce DNS response sizes, and it applies
both to abusing the "TXT" resource record type and to using a new
resource record type.

To see why this is, one needs to look at what DNS lookups SMTP Relay
clients and servers actually perform, and one needs to look at how the
"SRV" resource record type could have been defined but wasn't.

Even 3 years after the publication of RFC 2821, which obviated the
requirement that caused these to be done in the first place, many SMTP
Relay client softwares _still_ perform "ANY" DNS lookups instead of
separate "CNAME" and "MX" lookups (or, better yet in light of RFC 2821,
just "MX" lookups).  By adding SPF-style information to, say, the domain
name "aol.com.", the size of the response to an "ANY" query for that
domain name is increased (and hence so too is the likelyhood of
incurring the overhead of falling back to using DNS/TCP).

Note two important points here:

* Everyone thinks of the DNS lookups that SMTP Relay servers will be
performing, but forgets that (because so many softwares lag more than 3
years behind the times), SMTP Relay _clients_ will be affected too,
since they perform "ANY" lookups on the very same set of domain names
(i.e. those that occur in the domain name portion of an SMTP mailbox name).
* The actual type of the resource record is irrelevant.  A 128-octet
"TXT" resource record added to "aol.com." will have the same particular
effect of increasing the size of an "ANY" response as a 128-octet "FOO"
resource record would.

Now consider how "SRV" resource records could have been designed, but
weren't.  "SRV" resource records _could_ have been designed to
incorporate the protocol and the service names in their data portions

     example.org. IN SRV 86400 nicname tcp 0 0 43 whois.example.org.
     example.org. IN SRV 86400 ldap tcp 0 0 389 dir.example.org.
     example.org. IN SRV 86400 http tcp 0 0 80 www.example.org.

and service lookup would have involved performing a "SRV" lookup and
then scanning the resultant resource record set, which would have grown
with each new service defined and provided, for resource records with
names that matched the desired service.  (Do this design and its
concomitant problems seem somewhat familiar, boys and girls?)

Instead, "SRV" resource records were designed so that the service and
protocol names were encoded in the owner domain name,

     _nicname._tcp.example.org. IN SRV 86400 0 0 43 whois.example.org.
     _ldap._tcp.example.org. IN SRV 86400 0 0 389 dir.example.org.
     _http._tcp.example.org. IN SRV 86400 0 0 80 www.example.org.

and thus so that individual "SRV" resource record sets only covered
single services.  Thus, looking up the locator information for an
individual service doesn't present one with a whole slew of information
about services that one wasn't interested in, that one then has to wade
through picking out what one wants; and, also, the sizes of "SRV"
resource record sets don't grow as "SRV" lookup is defined for further
services.

In database terms (and without going into detail about the DNS database
schema): The primary key for the global, distributed, DNS database is an
<owner-name,class,type> tuple, and service lookup encodes an
<service-domain-name,class,type,protocol,service> search key into that
tuple, thereby making use of the DNS lookup mechanism to ensure that the
lookup yields only those data that are actually relevant.

Client-restriction data in fact uses a very similar
<client-domain-name,class,type,protocol,service> search key, which can
be encoded in the same way.  Ironically, this would have been clearer if
we didn't have "MX" resource records but had used "SRV" resource records
to find SMTP Relay servers.  The pairing of

     _smtp._tcp.example.org. IN SRV 86400 0 0 25 a.mx.example.org.

to locate the "example.org." SMTP Relay servers and

     _smtp._tcp.example.org. IN SPF 86400 "-all"

to publish the restrictions on the "example.org." SMTP Relay clients
would have seemed relatively straightforward.

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 14 Sep 2004 13:44:26 +0200

On Sep 11, 2004, at 19:59, Andrew Newton wrote:

> working group has at least rough consensus that TXT usage is 
> acceptable for compliance and should not be specified as a 
> configuration that will be non-compliant.  However, there is at least 
> rough consensus that the use of the SPF-specific record type is more 
> desirable than the use of a TXT record type.  It is the opinion of the 
> co-chairs that the -protocol document clearly state that the usage of 
> TXT records will most likely be deprecated by future protocol 
> definition.

Given this text it is not clear to me whether the -protocol document 
will specify any kind of use of TXT records.

My view is still that:

- TXT records are in use for SPF and friends already
- There is no need for the MARID wg to specify how to use TXT records
- Instead the MARID wg should specify a new RR Type, and specify how it 
is used

I.e. it is enough with one transition, from what some people use today, 
to what the MARID wg is specifying. Yes, it is ok for the -protocol 
document to talk about this transition, but...I don't like seeing 
"short term deployment of MARID things" because short-term doesn't 
exist on the Internet.

       Patrik

---

From: Andrew Newton <[email-masked]>
Date: Tue, 14 Sep 2004 09:16:27 -0400

On Sep 13, 2004, at 10:21 AM, Jonathan de Boyne Pollard wrote:

> just "MX" lookups).

I looked into this about a year ago and found that most MTAs did MX 
then A,AAAA.

-andy

---

From: Andrew Newton <[email-masked]>
Date: Tue, 14 Sep 2004 12:06:52 -0400

> Can you explain the differences between the 2% you quote and the 36% I

Attached you will find a survey that Marcos Sanz and I did of com, net, 
and de.
We did look at every domain but we did sample a statistically 
significant number of them.

I'll reserve my conclusions for now, save the fact that your numbers 
depend on where you look.

-andy

                                                 com/net           de
MX Reocrds                                      7,617,555         
1,164,17
MX Records With Address In Target                  63,797              
521
MX Records With Unresolvable Targets              358,369           
13,324
MX Records Pointing To Loopback Address           125,956              
949
MX Records At Zone Apex                         6,949,466         
1,119,31
Wildcard MX Record                                668,089           
44,861

Unique MX Record by IP Address                  2,264,858          
196,545

TXT Records                                       407,961           
17,850
Wildcard TXT Record                                25,669              
187
TXT Records at Zone Apex                          356,537           
17,474
CNAME'd TXT Records at Zone Apex                      248               
  4
TXT with SPF Content                              185,607            
2,304
TXT with Caller-ID Content                            137               
  2
TXT with _ep Prefix                                25,755              
189
TXT with _ep Prefix and No Caller-ID Content       25,624              
187
TXT with _ep Prefix and Caller-ID Content             131               
  2
TXT with _ep Prefix and SPF Content                24,368              
156
Wildcard TXT with SPF Content                      24,378              
156
Wildcard TXT with Caller-ID Content                     0               
  0
TXT at Zone Apex with SPF Content                 136,861            
1,992
TXT at Zone Apex with Caller-ID Content                 6               
  0

Average Size of All TXT Records                        39               
21
Avearge Size of All Wildcard TXT Records               12               
27
Average Size of All Apex TXT Recrods                   43               
21
Average Size of All SPF Records                        56               
27
Average Size of All Caller-ID Records                  82               
76
Largest TXT Record                                    423              
232
Largest SPF Record                                    386              
232
Largest Caller-ID Record                              249               
96

Total Number of Domains Scanned                 8,477,489          
736,725
Domains with A Record at Zone Apex              5,574,982          
503,652
Domains with A Record at Apex Pointing to Loopba      867              
269
Domains with AAAA Record at Zone Apex               2,284               
58
Domains with Address in MX Record                  62,513              
499
Domains with Unresolvable MX Target               226,673           
12,497
Domains with MX Pointing to Loopback Address       86,344              
915
Domains with MX Record at Zone Apex             4,697,344          
676,517
Domains with TXT Record at Zone Apex              329,707           
15,071
Domains with Wildcard MX Record                   437,982           
26,616
Domains with Wildcard TXT Record                   25,619              
186
Domains with Apex MX and Wildcard MX              418,547           
26,443
Domains with Apex TXT and Wildcard TXT             24,531              
181
Domains with SPF                                  138,603            
1,993
Domains with Caller-ID                                136               
  2
Domains with both SPF and Caller-ID                   107               
  1
Domains with SPF and Wildcard TXT                  24,378              
156
Domains with SPF in Wildcard TXT                   24,378              
156
Domains with SPF and Wildcard TXT but not Both          0               
  0
Domains with SPF and Other Content in Wildcard T        4               
  0
Domains with SPF at Zone Apex                     136,853            
1,991
Domains with SPF '+all'                                 7               
  0
Domains with SPF '-all'                            76,386              
571
Domains with SPF '?all'                            51,005              
105
Domains with SPF '~all'                            10,351               
96
Domains with Caller-ID and Wildcard TXT                26               
  0
Domains with Caller-ID in Wildcard TXT                  0               
  0
Domains with Caller-ID and Wildcard TXT but not        26               
  0
Domains with Caller-ID and Other Content in Wild        0               
  0
Domains with a valid A or MX at the Apex        6,104,870          
693,109
Domains with a valid A or MX at the Apex and SPF  130,464            
1,926
Domains with a valid A or MX at the Apex and Cal      121               
  2

Average Size of Wildcard TXT for Domains with Wi       12               
27
Average Size of Apex TXT for Domains with Apex T       43               
20

Total Number of Domains In Zone(s)              33,887,010        
5,896,72
Total Number of Domains Processed               8,477,489          
736,725

---

From: Andrew Newton <[email-masked]>
Date: Tue, 14 Sep 2004 12:45:58 -0400

> significant number of them.

Correction:
We did NOT look at every domain but we did sample a statistically 
significant number of them.

-andy

---

From: wayne <[email-masked]>
Date: Tue, 14 Sep 2004 12:54:17 -0500

In <[email-masked]> Andrew Newton <[email-masked]> writes:

> Attached you will find a survey that Marcos Sanz and I did of com,
> net, and de.

Thank you very much for posting this!  Very interesting.

A few questions/comments:

* can you tell me when this survey was conducted?

* for those looking closely at the numbers, it appears that
the last digit of the numbers for the de TLD were sometimes droppped.

* it appears that you survedy about 25% of com/net and 12.5% of de.

* scaling up the numbers, there appears to be about 750,000 SPF
records in com/net and 18,000 SPF records in de.

* you found SPF records for around 2% of the com/net domains, but
only 0.3% of the de domains.

* for com/net, SPF records account for 45% of all TXT records, for de
it is only 13%

> depend on where you look.

I'm sure this is very true.

-wayne

---

From: Andrew Newton <[email-masked]>
Date: Tue, 14 Sep 2004 14:36:52 -0400

> * can you tell me when this survey was conducted?

com/net was done July 5.  I believe de was done August 18.  I do have 
com/net numbers around mid-august, by the were not much different.  I 
just don't have them handy at the moment.

> the last digit of the numbers for the de TLD were sometimes droppped.

I'll attach a CSV (the other CSV) format.

> * it appears that you survedy about 25% of com/net and 12.5% of de.

I believe both are well above the number needed for statistical 
significance.

> I'm sure this is very true.

I was merely pointing out the differences between com/net and de.

-andy

---

From: Sam Varshavchik <[email-masked]>
Date: Tue, 14 Sep 2004 18:33:58 -0400

> I looked into this about a year ago and found that most MTAs did MX 

Generally, DNS servers will return a CNAME in response to a query for any RR 
for that hostname; and recursive resolvers will follow the CNAME chain.

The original client receives a reply that includes all the CNAMEs, and the 
MX RRs; provided that everything fits in the UDP packet.

---

From: David Woodhouse <[email-masked]>
Date: Tue, 14 Sep 2004 23:25:07 +0100

> successfully produced a first specification.

I strongly disagree with this opinion. I believe that it does not make
sense to have multiple scopes, and certainly not to plan to add even
more scopes later. I shall explain my reasoning:

The 'mail from' and 'pra' scopes require significant changes in current
practice, which would essentially require the entire Internet to
'upgrade' to conform. 

Setting aside the deployment problems posed by this, let us assume that
despite the fact that most of them haven't even discovered ESMTP yet,
the whole world _does_ actually manage to upgrade tomorrow; to perform
SRS and to add whatever we decide to use instead of the badly-chosen
'Resent-From:' header.

Once such an upgrade has occurred, the identities actually _checked_ by
either scope would be modified automatically by mail servers as the mail
is in transit -- each mail server could pick any arbitrary 'domain' to
put into those identities for checking, as long as the DNS records for
that 'domain' permit the IP address of the server in question. 

The identity which is being checked at each hop would no longer be
directly related to the original sender of the mail, but merely serves
as a verified identifier for the entity which controls the mail server
in question, and can be used to determine a level of trust for that
server.

Therefore, the 'mail from' and 'pra' scopes should be considered equal,
not as complementary forms of 'authentication'. Once the whole world has
upgraded, each scope provides merely an arbitrary handle by which to
classify the mail host which is submitting a given mail.

That is why it does not make sense to offer multiple scopes. One would
suffice, and it should be one which does not suffer potential IPR
problems and which does not require such a worldwide 'upgrade'. The HELO
identifier checked against an IP address, or a signature on TLS
certificates, or perhaps the SUBMITTER SMTP extension, would provide an
equally suitable identifier for the entity responsible for a given mail
server, without any of the technical difficulties.

Therefore, the working group should abandon the 'mail from' and 'pra'
scopes and seek a _single_ scope which serves the purpose of identifying
the entity responsible for a given mail server.

The problem of true authentication of senders is a separate one which
needs to be addressed by a true end-to-end method. To use a hop-by-hop
method based solely on IP addresses for such a task is inherently
insecure and is counter-productive due to the confusion it causes.

-- 
dwmw2

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 14 Sep 2004 16:54:35 -0700

I agree with Andy.

I thought we should have done this all along. The mail from and pra
mechanisms are merely means of interpreting and using the SPF data. The data
itself consists of no more and no less than a set of IP addresses
corresponding to approved outgoing edge servers which may or may not be
known (and specified) to be complete.

		Phill

> -----Original Message-----
> From: David Woodhouse [mailto:[email-masked]]
> Sent: Tuesday, September 14, 2004 6:25 PM
> To: Andrew Newton
> Cc: IETF MARID WG
> Subject: Re: co-chair judgment of consensus related to last 
> call period
> of 23-Aug-2004 to 10-Sept-2004
> 
> 
>

---

From: wayne <[email-masked]>
Date: Tue, 14 Sep 2004 20:35:50 -0500

[top quoting fixed, extra words deleted, and the Reference: header
restored.  All due to brain-dead mail software.]

In <[email-masked]> "Hallam-Baker, Phillip" <[email-masked]> writes:

> David Woodhouse [mailto:[email-masked]] wrote:
>> 
>> >                                         This work plan does not include 

No, apparently you agree with David Woodhouse.  It is easy to get
confused when you don't use proper quoting.

> I thought we should have done this all along.

Yes, I know.  This was discussed long ago.  The rough consensus was
against your opinion.  I thought that while there should be multiple
scopes, we could have one record that covered them all.  The rough
consensus was against my opinion.

Now, can we *PLEASE* stop re-raising old issues everything someone new
comes into this working group?

-wayne

---

From: David Woodhouse <[email-masked]>
Date: Wed, 15 Sep 2004 08:58:43 +0100

> restored.  All due to brain-dead mail software.]

Thanks for fixing that; it's amazing how poorly people will behave in
public fora. Personally, I configure my mailing lists to reject any mail
with 'Re:' in the Subject: header but neither References: nor
In-Reply-To: headers. Philip, you really ought to invest in some
non-broken mail software if you want to use it in public.

> "Hallam-Baker, Phillip" <[email-masked]> writes:

Maybe I have indeed missed the discussion on this precise topic, but I
suspect you might both have missed my point -- and in either case I
apologise. Philip might be used to upside-down email and have thought
that my first paragraph was in fact my conclusion, but I didn't expect
that of you so it's probably my fault for not being concise enough in
the first place.

I was not saying that we should ditch either pra or mail-from scope and
continue with only one or the two, which I know is an opinion which has
been voiced before. 

I was saying that we should ditch _both_ pra and mail-from scopes and
continue with something more appropriate.

The point was that the mail-from and pra scopes are _both_ only useful
for determining a trust level for a given mail server, by determining
whom it belongs to. The 'domain' we use becomes nothing more than an
arbitrary 'trust handle' which isn't even seen by most users anyway, as
it's hidden in the Return-Path: or Resent-From:^WForwarded-For: headers.

The mailfrom/pra scopes are not useful as end-to-end validation that a
given mail really did come from the person from whom it claims to come.
A mailfrom/pra 'PASS' is neither necessary nor sufficient to make that
judgement reliably. There are both false positives and false negatives
if one were to inadvisedly use it for such a purpose.

Yet the very nature of the scopes makes it easy for the naïve user or
admin to believe that they can be used that way. Therefore, it is
actually counterproductive -- because it encourages a mistaken belief
that this provides real end-to-end authentication when it cannot. 

My suggestion is that we should go forward with an entirely _different_
scope, not just unify on either mail-from or whatever the unencumbered
pra replacement would be.

There are many possibilities which would offer us some 'handle' on the
owner of the mail server in question, without all the problems which
both mail-from and pra scopes suffer.

-- 
dwmw2

---

From: Tony Finch <[email-masked]>
Date: Wed, 15 Sep 2004 17:28:22 +0100

> the entity responsible for a given mail server.

I agree. I also think that in this scenario the mechanism should be CSV
rather than SPF, because CSV is vastly more efficient and less abusive of
the DNS.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LYME REGIS TO LANDS END INCLUDING THE ISLES OF SCILLY: WEST OR NORTHWEST
BACKING SOUTHWEST 3 OR 4, INCREASING 5 TO 7 LATER. MAINLY FAIR. MAINLY GOOD.
SLIGHT OR MODERATE, BECOMING ROUGH IN THE WEST.

---

From: Rik van Riel <[email-masked]>
Date: Wed, 15 Sep 2004 12:51:45 -0400 (EDT)

> classify the mail host which is submitting a given mail.

Good point.  Multiple scopes only really seem to make sense
in a world where SPF isn't deployable ...

Rik
-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan

---

From: Rand Wacker <[email-masked]>
Date: Wed, 15 Sep 2004 10:24:19 -0700 (PDT)

> rather than SPF, because CSV is vastly more efficient and less abusive of

And less abusive of the forwarders.

My worry about the current direction to standardize a mailfrom scope is
that there hasn't been a lot of technical discussion about the implication
of changing the return paths by the group.

-Rand

---

From: David Woodhouse <[email-masked]>
Date: Wed, 15 Sep 2004 19:15:47 +0100

> of changing the return paths by the group.

The rewriting using SRS or the Resent-From:^WForwarded-For: header has
been bolted on almost as an afterthought, to patch up the bits where the
original assumptions of the mailfrom or pra scopes respectively don't
quite match reality.

You seem to be correct that there hasn't been much discussion about how
they affect the operation of the overall scheme. In fact I think there
hasn't even been a lot of _thought_ about the implications.

Having considered the implications myself, I believe that the mere
_possibility_ of SRS or the header addition means that each of the
mailfrom and pra scopes becomes nothing more than a way to determine the
trustworthiness of the _individual_ mail host which is submitting the
mail in question.

There are better ways to determine the trustworthiness of a mail host,
without the requirement for the whole world to be 'upgraded'.

-- 
dwmw2

---

From: Rand Wacker <[email-masked]>
Date: Wed, 15 Sep 2004 11:22:47 -0700 (PDT)

> trustworthiness of the _individual_ mail host which is submitting the

Right.  When you add SRS in to the equation, SPF Classic essentially
"degrades" in to CSV-like functionality.

-Rand

---

From: David Woodhouse <[email-masked]>
Date: Wed, 15 Sep 2004 20:37:20 +0100

> "degrades" in to CSV-like functionality.

That's twice you've said this of only 'mailfrom' scope, and it doesn't
seem to have been an accident.

It should be noted that the 'pra' scope, with the forwarders able to add
an arbitrary domain in whatever header is used in place of the 
'Resent-From:' header, has precisely the same characteristics.

-- 
dwmw2

---

From: Rand Wacker <[email-masked]>
Date: Wed, 15 Sep 2004 14:18:20 -0700 (PDT)

> 'Resent-From:' header, has precisely the same characteristics.

Not arbitrary, forwarders can only add domains they are authorized to add
in the Resent-From: header (if the receiver is checking the PRA).

The difference between adding headers and using SRS is that in order to
avoid blowing 64-byte localpart limit, SRS advocates that secondary
forwarding hops (anyone doing an SRS rewrite on an SRS0 address) remove
the interim hop information, so you really only do know the first hop a
message purported to take and the last hop it really did take.  By adding
Resent-From headers you can get some idea of the full path that a message
took, and if you trust different hops along the path to add the proper
checks then you can do some work to determine where a message came from
before it started along a chain you trust.

Of course, this is ludicrously complex, difficult to implement and manage,
and probably subject to creative spoofing somewhere.  If we want to admit
that none of the IP-based schemes can safely give us any assurance beyond
the most recent hop, then we should be working on systems that admit that
and don't try to reach beyond what they can really do by changing a
fundamentally important piece of envelope information like the bounce
address.

I personally believe that Sender ID gives a positive benefit in that its
going to allow people to positively identify a user-visible piece of
messages coming from certain sources for a large majority of real internet
traffic (direct site-to-site, no interim hops).  The IP-based solutions
won't allow many of us to outright reject forgeries at the border because
the risk of false positives introduced by forwarding and mail lists is
just too high, but they will allow us to remove uncertainty about the
legitimacy of messages that came directly to us.

To get end-to-end security we will have to use crypto, and I think
everyone here would agree with that.

-Rand

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 17 Sep 2004 13:32:23 +0200

On Tue, Sep 14, 2004 at 12:06:52PM -0400,
 Andrew Newton <[email-masked]> wrote 

> Attached you will find a survey that Marcos Sanz and I did of com, net, 

Here are the results for ".fr" :

Total Number of Domains In Zone(s):      267 657
Total Number of Domains Processed :       67 393
...
TXT Records                                 :         1 038
Wildcard TXT Record                         :            26 (  2,50%)
TXT Records at Zone Apex                    :           983 ( 94,70%)
CNAME'd TXT Records at Zone Apex            :             0 (  0,00%)
TXT with SPF Content                        :            89 (  8,57%)
TXT with Caller-ID Content                  :             3 (  0,29%)
TXT with _ep Prefix                         :            29 (  2,79%)
TXT with _ep Prefix and No Caller-ID Content:            26 (  2,50%)
TXT with _ep Prefix and Caller-ID Content   :             3 (  0,29%)
TXT with _ep Prefix and SPF Content         :             4 (  0,39%)
Wildcard TXT with SPF Content               :             4 (  0,39%)
Wildcard TXT with Caller-ID Content         :             0 (  0,00%)
TXT at Zone Apex with SPF Content           :            81 (  7,80%)
TXT at Zone Apex with Caller-ID Content     :             0 (  0,00%)
...
Domains with SPF                                        :           81 (  0,12%)
Domains with Caller-ID                                  :            3 (  0,00%)
Domains with both SPF and Caller-ID                     :            3 (  0,00%)
Domains with SPF and Wildcard TXT                       :            4 (  0,01%)
Domains with SPF in Wildcard TXT                        :            4 (  0,01%)
Domains with SPF and Wildcard TXT but not Both          :            0 (  0,00%)
Domains with SPF and Other Content in Wildcard TXT      :            0 (  0,00%)
Domains with SPF at Zone Apex                           :           81 (  0,12%)
Domains with SPF '+all'                                 :            0 (  0,00%)
Domains with SPF '-all'                                 :           28 (  0,04%)
Domains with SPF '?all'                                 :            6 (  0,01%)
Domains with SPF '~all'                                 :            3 (  0,00%)
Domains with Caller-ID and Wildcard TXT                 :            0 (  0,00%)
Domains with Caller-ID in Wildcard TXT                  :            0 (  0,00%)
Domains with Caller-ID and Wildcard TXT but not Both    :            0 (  0,00%)
Domains with Caller-ID and Other Content in Wildcard TXT:            0 (  0,00%)

---

From: Peter Koch <[email-masked]>
Date: Fri, 17 Sep 2004 19:45:45 +0200

wayne <[email-masked]> wrote:

{apologies for the delay}

> records are likely to be checked (the email domain names), I found 19%

email domain names as in '2nd level domain names' or 'owners of MX RRs'?
There's still some hierarchy in the DNS and as an example I found a single
zone with nearly 300 SPF RRs "v=spf1 -all" without an MX RR at the same owner.

> Can you give me a pointer to this survey that you are referring to?

It's based on the raw data collected during the RIPE DNS hostcount for the
DE toplevel domain.

> Can you explain the differences between the 2% you quote and the 36% I

The hostcount works by examining every zone at every level it can get hold of.
Let's take further discussion of the survey mechanics off list, please.

My point was that neither are TXT RRs mostly used for SPF nor is the
deployment broad enough to give a pressing need to stick with TXT RRs.
Dealing with transition in the spec is OK, but that should be done in an
appendix or an informational document.

-Peter

---

From: wayne <[email-masked]>
Date: Thu, 23 Sep 2004 20:46:11 -0500

> Attached you will find a survey that Marcos Sanz and I did of com,

I've had a couple of requests for more raw data about what TXT records
are out there and SPF records in particular.  So, I've finished up a
couple of surveys and lightly munged the data to keep spammers from
collecting domain names.  You can find the raw data at:

http://www.midwestcs.com/spf/surveys/

The "spammerlist" files are the domains found in a spammers list of
email addresses.  There were about 1.3 million unqiue domain names
used by the spammer.  The list was old when I got it about a year and
a half ago and the spammer never bothered to clean up obviously bogus
domain names, so the actual number of valid domain names is much less
than 1.3 million, maybe as few as 500k.  I have not bothered to clean
up the list because I use it for testing SPF stuff.  Even if the email
addresses have bogus domains, I expect spammers to send email claiming
to be from them so I need to make sure that my SPF system can deal
with them.

Files ending in "spf" are the unique SPF records found.  Files ending
in "txt" are summaries of the TXT records found.

Let me know if you have questions.

-wayne

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 24 Sep 2004 08:21:12 -0400

Based on these numbers, the item of interest was the growth rate:

Between 4/20 and 9/12, you had a growth rate of 68 new SPF spammers per day.

This is just with two states, so it would be interesting to see the
acceleration with more collection points.  I suggest a weekly or monthly
collection to see this pattern.

Nevertheless, overall, its not good to see this, especially, if that rate
does not go down over time.  However, fortunately, in my analysis, most
systems experience a segment of the spammer base.  In other words, I'm only
the basically the same systems or same pattern of systems using spoofs.

What I find extremely interesting is that these systems don't seem to be
learning from a programmatic standpoint, only administratively by adding the
domains.  In other words,  my rejects are pretty constant with the same
information, over and over and over again.  They might go away for a few
days or week, but soon enough they are back with the same type of
information that gets them rejected.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

---

From: wayne <[email-masked]>
Date: Fri, 24 Sep 2004 08:05:34 -0500

[top posting fixed]

In <000d01c4a231$3c5e3280$6401a8c0@hdev1> "Hector Santos" <[email-masked]> writes:

> "wayne" <[email-masked]> wrote:
>
>> used by the spammer.  [...]
>

The "spammerlist" is not a list of spammers, but a list of email
addresses used by at least one spammer.  Or, are saying that "SPF
spammers" are people who publish open SPF records or something?

-wayne

---

From: Andrew Newton <[email-masked]>
Date: Fri, 24 Sep 2004 11:36:48 -0400

> Let me know if you have questions.

No question, just a comment:  absolutely great stuff!

-andy