Subject: rough consensus and working code

From: wayne <[email-masked]>
Date: Tue, 15 Jun 2004 11:35:31 -0500

Yesterday in the Jabber session, Marshall raised the issue that there
really doesn't appear to be a rough consensus in this working group,
and people are not working toward a compromise solution.  In fact,
people appear to be hardening their positions.  If this working group
can't reach a rough consensus, then it is unlikely that the IESG will
agree with any RFCs we try to publish.

I have to agree with this assessment.  There doesn't appear to be a
rough consensus.  However, the long standing IETF mantra has, to the
best of my knowledge, been "rough consensus *AND* working code".  What
we lack with most proposals is working code.

I think the lack of a rough consensus is in large part due to the lack
of working code.  Working code quickly dispels both wishful-thinking
and FUD.  Working code is a different way of expressing a proposal so
disagreements and misunderstandings about a proposal can be cleared up
by looking at the code and then either the code or the proposal can be
fixed to make things clearer.

The devil is always in the details.  As long as we continue to
consider proposals that don't have working code, we allow people to
nit-pick proposals that are complete, while glossing over the problems
with proposals that exist on paper only.

Maybe I'm wrong, but I strongly suspect that the IESG is going to
require working implementations of any RFC we submit as standard track
RFCs.  At some point before we select one or more proposals, we have
to have a near-finalized document with working code, preferably with
at least two different people/organizations creating code from the
spec.

I see three basic possibilities:

1) We stop talking about proposals that don't have working code

2) People putting forward proposals need to start publish code ASAP

3) We change the schedule to allow for more time.

Back in March, I made the following post which I will repost here:
http://www.imc.org/ietf-mxcomp/mail-archive/msg00617.html

It is a long post but, sadly, it has worn well with time.  Only a few
things really need to be changed.  I posted it before the Caller-ID
proposal came around, so please change the "Gordon, Hadmut, Markus" to
"Harry, Jim, Bob".  Also, since this was posted months ago, you need
to change SPF email checking from "5-50 million messages per *month*"
to "5-50 million messages per *day*"

I think it is sad that very little has changed in the intervening
months with respect to working code.  Working code is not only very
important, but something that can't just be whipped up quickly.  (Code
can be written quickly, but it is rarly working code.)

The final paragraph almost makes me want to cry.

    * To: "IETF MXCOMP" <ietf-mxcomp@xxxxxxx>
    * Subject: Re: Ruminations
    * From: wayne <wayne@xxxxxxxxxxxxx>
    * Date: Mon, 29 Mar 2004 12:22:42 -0600

In <20040328035322.GA45563@xxxxxxxxx> Markus Stumpf <maex-lists-email-ietf-mxcomp@xxxxxxxxx> writes:

> On Sat, Mar 27, 2004 at 03:36:21PM -0600, Gordon Fecyk wrote:
>> It's happening right now.  Meng should pat himself on the back for a great
>> marketing campaign but ultimately, they decided.

Meng produced a lot of working code, and listened to what many other
people said.  SPF didn't gain it's current popularity by marketing.
It is the leading designated sender system because of the work of many
many people working on many different parts of the problem.  

> But it may take some time to se any results at all.
> Currently there is a big hype about adding SPF records. So what?
> It has no effect at all.

One of the things that was discussed last summer on the SPF lists is
how to solve the chicken-and-egg problem of no one publishing SPF
records if no one checks them, and no one checking them if there
aren't any published.

We realized that people would publish SPF records, just to make a
public statement by the domain owners that they want to protect their
brand names and good reputation.  This has benefits even if few people
are actually checking the SPF records.

With this in mind, we was decided that we needed to pay *very* close
attention to making it as easy as possible for people to publish SPF
records, and for those SPF records to require minimal maintenance.
Hence the creation of the 'mx', 'ptr' and 'include' mechanisms, and
the "SPF wizards" web pages which help people create SPF records.

> If I didn't, miss something, I'd like to see a big ISP or mail hoster
> that actually rejects emails because of SPF records. As soon as that is
> going to happen THEN the users will tell and then we'll see if that big
> entity is going to keep a counterstance. And that is what I still doubt,
> at least until all the current problems are resolved and the solution
> implemented in the MTAs.

The amount of email that is being checked with SPF is doubling ever
2-3 weeks and is currently on the order of 5-50 million messages per
month.  Yes, that is a very small percentage of all email, but the
rate of growth is almost scary.

The amount of email being checked via SPF has already meant that there
has been a stead stream of email forwarders and such that have
wandered into the the SPF mailing lists because SPF of the effect that
SPF has had on them.  Yes, many of them are not happy with the
situation or with the possible solutions such as SRS.  However, the
response has actually been better than I was expecting.  I'm seeing
quite a few forwarders adopting things like SRS in order to keep email
flowing smoothly.

So, the first goal of SPF was to make it easy to publish SPF records.
The next goal was to make it easy for mail admins to adopt SPF and SRS
by creating easy-to-install plugins to the major MTAs.  It was decided
that any time there was a choice between making things easier for
domain owners and mail admins or making the job of writing SPF
implementations easier, the domain owners and mail admins should win.

The result has been that the creation of high quality SPF
implementations has taken some time, and this has made it harder for
mail admins to start checking SPF records, especially for the "big
ISPs".  This situation is improving and I would say that SPF
checking will have had enough field testing and have high enough
quality/performance implementations that you will start seeing larger
ISPs checking SPF records by this summer, or maybe earlier.

This working group, however, appears to already be falling badly
behind on its own schedule, with no end of the debate about 'which
identities' in sight.  Instead we get to hear the sour-grapes moaning
from Gordon, Hadmut, Markus, et al who, over the last year, have
decided to put a lot more effort into talking than actually
implementing their ideas.  If you guys had actually been able to work
*with* people instead of promoting exclusively yourselves, either one
of your proposals would have become the de-facto standard that SPF has
become.  Maybe, if the IETF decides to be irrelevant, one of you will
get the honor of a de-jure standard, but I doubt that will happen
either.

Oh, and before you get all huffy about my comments showing that I
can't "work with people", remember that having a thick skin is an
important part of being able to work with people.  I've held my tounge
for a long time while various people have poked jabs at SPF.  If I can
deal with such jabs as long as there is something productive coming
with them.  The proof of the pudding is in the eating and there are
more people actually working together on SPF than all of the other
LMAP proposals, plus this working group combined.

Sadly, what I really expect to happen is over the next year or so,
this working group will try to come up with some sort of modified SPF
proposal with gratuitously incompatible changes.  Mean while, SPF will
continue to slowly evolve, adopting any really good suggestions that
are proposed and ignoring the bad ones, just like it has since last
summer.  After much debate, (2 years?) this working group will finally
realize that SPF is the de-facto standard and will finally decide to
throw out their work and adopt SPF.

*yawn*

I have code to write and bugs to fix in the SPF system.

-wayne

---

From: Andrew Newton <[email-masked]>
Date: Tue, 15 Jun 2004 13:26:41 -0400

> we lack with most proposals is working code.

I think implementations are a good thing, a very good thing.

However, RFC 2026 mentions that to get a Proposed Standard RFC 
implementations are not required (though they are encouraged).  To get 
a standard from Proposed Standard to Draft Standard, two separate 
interoperating implementations are required.

Our milestones explicitly says "PS" - for Proposed Standard.

-andy

---

From: John Levine <[email-masked]>
Date: 15 Jun 2004 17:48:22 -0000

>implementations are not required (though they are encouraged).

We really do need working code.  Publishing records is fine, but until
people start running a lot of mail through MARID authentication
checkers, we won't know whether it really scales or how easy it is for
bad guys to circumvent.  This doesn't mean that you have to use it for
live spam filtering (you'd be nuts to bounce mail that failed SPF, for
example) but at least log it and count what you find.

I know there's open source code for domain keys coming shortly.  What's
available for SPF, Caller ID, and the merged thing?

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
http://www.taugh.com

---

From: wayne <[email-masked]>
Date: Tue, 15 Jun 2004 13:02:04 -0500

> interoperating implementations are required.

*sigh*

Ok, for the second time in two weeks, I will quote the relevant
potions of RFC2026:

   Usually, neither implementation nor operational experience is
   required for the designation of a specification as a Proposed
   Standard.  However, such experience is highly desirable, and will
   usually represent a strong argument in favor of a Proposed Standard
   designation.

   The IESG may require implementation and/or operational experience
   prior to granting Proposed Standard status to a specification that
   materially affects the core Internet protocols or that specifies
   behavior that may have significant operational impact on the
   Internet.

I never said that we are *required* to have working code.  What I have
said is that I think that working code helps reach consensus and that
the IESG may likely consider MARID to "materially affect a core
Internet Protocol" and/or "have significant operational impact on the
Internet".

Personally, I think the IESG would be nuts not to require such testing
and that we would be nuts not to have all our ducks lined up in a row
before we go to the IESG asking for a PS RFC.  The requirements for
going from a PS to a DS are much higher, in particular needing two
implemenations of every feature and I suspect that the IESG will not
require that level of testing.

For a slightly longer discussion, see my earlier note at:
http://www.imc.org/ietf-mxcomp/mail-archive/msg01796.html

-wayne

---

From: "william(at)elan.net" <[email-masked]>
Date: Tue, 15 Jun 2004 11:13:33 -0700 (PDT)

> available for SPF, Caller ID, and the merged thing?

Its inappropriate to compare domain keys to MARID, they are not doing IETF 
WG effort. If anything you can compare it to individual/non-IETF-group effort
like SPF and in that case you see that there is substantial amount of 
working code available and they are major component of the MARID.

Considering SPF community, I think once standards here are formalized and 
these standards have community support (i.e. "consensus"), the code will
become available very soon.

---
William Leibzon
Elan Networks
[email-masked]

---

From: Rand Wacker <[email-masked]>
Date: Tue, 15 Jun 2004 11:29:46 -0700 (PDT)

On Tue, 15 Jun 2004, John Levine wrote:

> I know there's open source code for domain keys coming shortly.

^coming shortly^available for the past couple of weeks

We (Sendmail) released an open source implementation of a DomainKeys
milter just prior to the INBOX conference.  It plugs into Sendmail
through the milter protocol and can currently sign outgoing messages and
verify and mark incoming messages.  There's been quite a bit of feedback
so far and Murray has kicked out numerous incremental releases since then.
I'm sure someone will point out that there are questions about some of the
Yahoo licensing terms but those are being worked through as we type.

More info, code, and mailing lists can be found at:

	http://www.sendmail.net/dk-milter/

> What's available for SPF, Caller ID, and the merged thing?

We had also been working on a Caller-ID milter (as there were already a
few SPFv1 implementations out there), but haven't released it yet as to
wait and see what happens with the spec merger.

> example) but at least log it and count what you find.

One of the other things we've been working on is a test plan to evaluate
all of the major schemes against a number of different criteria.  We've
built a site to help coordinate community testing effort, and are still
working on some of the technical details.  I'm sure that this group has
enough brainpower and probably some good tools to help in evaluating all
of these solutions, we'd be happy to help in coordinating putting them to
use.  For more information on what we've done so far, please check out:

	http://sendmail.net/

-Rand

---

From: Ted Hardie <[email-masked]>
Date: Tue, 15 Jun 2004 11:40:43 -0700

>wayne wrote:

>require that level of testing.

As a different data point, note that the IESG did approve this working
group with a very tight timeline, knowing that there would be trade-offs
in that approval.  I have also discussed a number of the issues raised
in the group on the IESG's bi-weekly calls, so I hope there will be
few surprises to the IESG in the work.

Perhaps the best advice I can give you here is to worry about getting
the engineering right rather than worrying about the IESG.  The first
tends to take care of the second, and the second rarely takes care
of the first.
			regards,
				Ted Hardie

---

From: John Levine <[email-masked]>
Date: 15 Jun 2004 18:58:50 -0000

>Considering SPF community, I think once standards here are formalized
>and these standards have community support (i.e. "consensus"), the
>code will become available very soon.

I sure hope the code precedes formalization, because I have little
confidence that a paper design will stand up in practice.  As many
others have noted, e-mail and anti-spam is one of the most hostile
computing enviroments around, and if there's any exploitable chinks
in the armor, the bad guys will exploit them to the utmost.

Re SPF, I know about the SPF publishing wizard, and I've seen a couple
of libraries that decode SPF data, but how many people have actually
plugged them into their MTAs to see what happens?

Regards,

---

From: "Eric A. Hall" <[email-masked]>
Date: Tue, 15 Jun 2004 16:32:16 -0500

> plugged them into their MTAs to see what happens?

I did for a while (not doing so today, but that's incidental with
infrastructure issues).

SPF is useful when [1] there is a record, and [2] the mail comes from a
source that is clearly unauthorized for the associated domain. It is not
particularly useful in any other combination. For example, you cannot
*reliably* interpret the lack of a record as meaning anything, since many
organizations (including some of my big customers) will send mail from an
account in a division-specific domain, but will use a server in a parent
or sibling domain for relay purposes. Meanwhile, the mere existence of a
record and the use of an authorized server doesn't tell you much either,
since spammers can create records that point to their servers too.

So all in all, the subset of messages that meet the criteria are small,
but this is primarily due to the relatively small number of domains that
have records. I don't actually think that this will change much, since
even while more domains may get records, spammers will either use other
domains or will setup their own records.

But the whole value proposition of SPF and similar efforts is that I can
prevent *MY* domain from being used in various forgeries, so in that
regard it is still useful.

On the other hand, the cost of operating a server goes up somewhat due to
the increased query load, the fact that synchronous blocking lowers the
number of transfers per day that my machinery can handle, etc., while the
benefits mostly flow to other people.

SPF is worth publishing, but the value for checking is marginal and will
probably stay that way.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: "Eric A. Hall" <[email-masked]>
Date: Tue, 15 Jun 2004 18:11:11 -0500

On 6/15/2004 4:32 PM, Eric A. Hall wrote:

> SPF is worth publishing, but the value for checking is marginal and
> will probably stay that way.

ps--if the value for checking is proportional to the cost (which it is),
then the value goes down as:

  * record sizes require fallback processing

  * firewalls/bugs/etc break fallback processing

  * large numbers of large records make caching expensive/difficult

  * multiple RRs of the same type (but not the same meaning) require
    deeper analysis (= more processing, = more blocking)

I know some folks have said that they will actively work against large
RRs, but for me I'll just not implement the checker. Almost every mail
that was ever trapped with SPF was also trapped via other analysis (such
as spamassassin checks). Slightly improved heuristic trapping just isn't
worth the extra expense, especially at large scales.

---

From: "Jon Kyme" <[email-masked]>
Date: Wed, 16 Jun 2004 09:22:43 +0100

> > will probably stay that way.

I think this has been said about a million times before, but surely SPF
(etc) is, as things stand, only worth publishing if a sufficient proportion
of sites check it? If, as you suggest, the benefit in checking is low, the
benefit in publishing will remain small.

There are a few things that could change this relationship without the
proportion of checking sites suddenly going up.
1. Spammers move away from forged senders.
      Benefit in publishing falls
2. Receivers more strongly weight their local policy against 
   non-publishers
      Benefit of publishing increases
3. Publishers tighten their published policy i.e. for SPF move to
   -all (or as we call it in our office "f**k-all")
      Benefit of checking increases, which *may* increase the benefit
      of publishing.

I believe we'll see some of these processes in action over the next few
months. But given that we know precisely nothing about the relative
strength of the effects, it's simply not possible to say what the outcome
will be. Hence much SPF (and MARID) evangelism must be faith-based. Now
that's not bad in itself, but it shouldn't be confused with engineering.

Of course one thing that could really change the interactions here would be
some regulatory change affecting a large proportion of hosts of the MTS :-)

---

From: Matthew Elvey <[email-masked]>
Date: Sun, 20 Jun 2004 18:26:20 -0700

A perhaps redundant comment:

It would be nice if there was a rough consensus. 

On 6/15/04 9:35 AM, wayne sent forth electrons to convey:

> ...
Hmm.  I think Meng's Unified SPF plan is a great move in terms of this 
second goal of making things easy for mail admins.  I'm won over.
We still use EHLO to select the domain to authenticate; we ditch the PRA 
and SRS.  So our friends you were talking about DON'T HAVE TO DO 
ANYTHING!  The big German hoster just has to make sure that the EHLO its 
server sends out is a domain that has an SPF record that validates its 
IP (and reputation/accreditation). SRS doesn't need to be deployed!
In my proposal, most domains DON'T NEED  new DNS records AT ALL.
(And none of this RFrom stuff is necessary either.)  Near lightning-fast 
deployment is feasible.  And we're still providing and using M.A.R.I.D. 
effectively.
How cool is that?