Subject: Re: Obstacles between us and the finish line

From: wayne <[email-masked]>
Date: Thu, 02 Sep 2004 20:13:17 -0500

Back on July 1, I posted a message with the subject "Obstacles between
us and the finish line".  I posted an updated to it two weeks later.
I decided to go back and post another updated.

The first two messages message can be found here:
http://www.imc.org/ietf-mxcomp/mail-archive/msg02476.html
http://www.imc.org/ietf-mxcomp/mail-archive/msg02651.html

> Deadlines for this working group are rapidly approaching.  According
> to Andy's recent post, we have about 2 weeks (until 07/18) to finish
> things up.

For those who don't remember, 7/18 was the first deadline for MS to
fix the license.

> In my (oh so very humble) opinion, I see the following obstacles in
> the way:
>
> 1) Proposals must have solid I-Ds
>
> 2) Proposals must have working code
>
> 3) Proposals must have been tested on real world data
>
> 4) Proposals must have IP licensing terms that allow for use in both
>    commercial and GPLed MTAs.
>
> 5) Proposals must not have gratuitous incompatibilities with an
>    installed base.
>
>
> I have said from very early on that I think we may well need more than
> one RFC to handle the various identities.  To be quite honest, I don't
> care if we have a dozen overlapping proposals, I think the market can
> easily decide which are useful.
>
> As such, I see the following proposals that may be considered:
> SPF-classic, CSV, Sender-ID and Unified-SPF.
>
> It appears to DMP, FSV, and Caller-ID are no longer being pushed by
> anyone.
>
>
> I'm going to go over all the proposals in a sec, but I think I should
> make a few comments on the requirements.
>
> 1) solid I-Ds:  Proposals must have been gone over with many eyes,
> especially with developer's eyes when they are trying to implement the
> proposal.  I think it is far too late to do anything with a proposal
> other than use it to implement code and make adjustments based on that
> experience.
>
> 2) working code, and 3) testing, I think are pretty obvious.
>
> 4) IP licensing:  I'm not dogmatic about the GPL or commercial
> licenses, but there are significant MTAs and mail filters that fall
> under both kinds of licenses, and several others to boot.  I think
> that any proposal that has incompatible IP licensing needs to be
> dropped.

Sadly, in response to this post back in July, Ted Hardie (the IETF
Area Director for this WG), told us to stop worry about IP licensing
issues.  I believe the mess we are in right now is a direct result of
this.  Instead of getting the IP licensing issue resolved *much*
earlier on, we now have a crisis and a no-win situation.

> 5) Incompatibilities:  Proposals such as SPF have a significant install
> base and doing things like moving the location of the TXT records,
> dropping macros, or using a different RR, etc. make no sense to me and
> will likely run into a buzz saw with SPF adopters.
>
>
>
> Ok, so how do the current proposals stack up?
>
>
> ****   SPF-classic   ****
>
> 1) solid I-D:  Yes
> It was supposed to have been submitted as an experiemental RFC, but I
> don't know the status. 
>
> 2) working code:  Yes
> Lots of working code in fact.  New announcements of commercial
> products using SPF a couple of times a week.
>
> 3) Tested:  Yes
> It is being used to check millions of emails per day.
>
> 4) IP Licensing:  No problems
>
> 5) gratuitous incompatibilities:  None.
>
>
> I think SPF-classic should certainly be *one* of the proposals put
> forth by this working group.

I think it is really too bad that SPF-classic was not *one* of the
proposals.  It would have made our live much easier right now.

> ****   CSV   ****
>
> 1) solid I-D:  pretty good
> To the best of my knowledge, no one has tried to implement code based
> on it, but it appears to have been well reviewed by a fair number of
> people.  I think there is time to get implementation experience if the
> backers hurry.

Better I-Ds have been put forward, but I know of no change on the
implementation experience part.

>
> 2) working code:  none
>
> 3) testing:  none
>
> 4) IP Licensing:  No problems
>
> 5) gratuitous incompatibilities:  None.
>
>
> I think that CSV could be one of the proposals, if its backers quickly
> start creating working code and doing testing.

I think it is kind of funny that CSV has made it on the WG TODO list,
but neither SPF-classic nor Unified-SPF did.

> ****   Sender-ID   ****
>
> 1) solid I-D:  No
> I have looked again at the draft-ietf-marid-core-01.txt I-D, and
> frankly, it is not even close to being in a state worth commenting on.
> It is hugely incomplete and incompatible with SPF, which it tries to
> be compatbile with.  I can't see it becoming solid in the next couple
> of weeks.

The SenderID I-Ds have been much improved since July, but I think
there are still a lot of rough edges.

> 2) working code:  none
> In particular, the PRA and SUBMITTER are both on paper only.

There is now some working code for the PRA, but none for SUBMITTER.  

> 3) testing:  none
> I requested results of the PRA algorithm the first day that the
> Caller-ID spec was proposed to this working group.  I made a point of
> raising the issue at the interim meeting.  In the many months since
> then, nothing has been forth coming.  I am getting seriously worried
> about this.

Ok, a little small-scale testing has been done on the PRA now, but the
results have only shown that it doesn't work very well.

> 4) IP Licensing:  problems
> It appears that the Sender-ID's license is incompatible with the GPL.
> This could, in theory, be fixed very quickly, but the issues of IP
> disclosure and licensing problems has been known since before the
> interim meeting and nothing has been done.

Woah.  Here we are two months later and *STILL* nothing has been done.

> 5) gratuitous incompatibilities:  None.
> In theory, the only incompatiblity is that it tries to use existing
> SPF records for 2822 checks.  Since SPF records were published with
> the idea that they would be used for 2821.FROM and 2821.HELO, there
> could be problems.

Ah, gratuitous incompatibilities have been introduced with the change
of the version number.

> In practice, there are many incompatiblities with the current
> Sender-ID spec and the SPF spec.
>
>
> While Sender-ID appears to be the RFC-apparent for this WG, I have
> very serious doubts that it will be in any shape to be advanced before
> the deadlines.  I am beginning to think this proposal should be
> dropped as the backers of key parts appear to be moving way too
> slowly. 

Well, Sender-ID wasn't in shape to make the deadline of 7/18, so the
deadline was moved.  

> ****   Unified-SPF   ****
>
> 1) solid I-D:  fuzzy
> As Dave Crocker pointed out in a message earlier today, there is no
> Unified-SPF I-D.  This needs to be fixed ASAP or this proposal must be
> dropped.  Fortunately, the differences between SPF-classic and
> Unified-SPF are small enough that I think I-Ds could be written fairly
> quickly and yet remain pretty solid.
>
> 2) working code:  none
>
> 3) testing:  none
>
> 4) IP Licensing:  No problems
>
> 5) gratuitous incompatibilities:  None.
> The Unified-SPF proposal needs to make some changes to the SPF records
> in order to support the various scopes that it tries to cover.  This
> can be solved several ways.
>
>
> I think that Unified-SPF has a chance to be a solid proposal, but the
> I-Ds need to be written and the changes to the SPF-classic code needs
> to be tested.
>
>
>
> ****   DMP   ****
>
> Ok, I said DMP isn't being pushed by anyone, but I'll list it anyway
>
> 1) solid I-D:  Yes
>
> 2) working code:  Yes
>
> 3) testing:  Yes
>
> 4) IP Licensing:  No problems
>
> 5) gratuitous incompatibilities:  None.
>
>
> I think that DMP is in far better shape than most of the other
> proposals that appear to be much more likely to advanced by this
> working group.  *boggle*
>
>
> Right now, I would say that SPF-classic and DMP are the only proposals
> that are ready for RFC consideration.  For backers of other proposals:
> you have two weeks to finish.

I wish I could say we have come a long way, but I don't think we have.

-wayne

---

From: Douglas Otis <[email-masked]>
Date: Thu, 02 Sep 2004 19:50:13 -0700

On Thu, 2004-09-02 at 18:13, wayne wrote:

> implementation experience part.

Not extensive, but more than none.

> but neither SPF-classic nor Unified-SPF did.

Looking to the next step, CSV offers an extremely simple extension to
RFC2821 for authenticating the EHLO domain.  I see this as highly
important for reputation services, as identities obtained from either
Sender-ID or SPF are not suitable for basing reputation assertions
without running considerable risk when making assumptions of the
integrity of the mail channel.  

Once CSV is in place, to implement something extremely similar to SPF or
Sender-ID becomes just a trivial name list created using PTR records
much like DNA.  I was hoping to get some feedback before offering code. 
http://www.ietf.org/internet-drafts/draft-otis-marid-mpr-00.txt

I would not expect many to be satisfied with just an EHLO check.

-Doug

---

From: Graham Murray <[email-masked]>
Date: Fri, 03 Sep 2004 06:56:48 +0100

Douglas Otis <[email-masked]> writes:

> much like DNA.

Unfortunately this is not quite as trivial as it seems. I believe that
PTR records are not the easiest records for the domain owner
(especially the owner of small domains) to have control over.

---

From: Ted Hardie <[email-masked]>
Date: Fri, 3 Sep 2004 12:30:28 -0700

>earlier on, we now have a crisis and a no-win situation.

There's no reference to which post of mine is meant, but I
assume http://www.imc.org/ietf-mxcomp/mail-archive/msg02748.html
is the one to which this refers.

That post did not say "stop worrying about the licensing issues",
it said "Discuss licensing in terms of the engineering decisions"
or more fully:

>Comments based solely on the licensing terms without regard to the engineering
>choices they affect *do not speak to the question working groups need to
>decide*.   The sudden appearance of new working group participants
>after postings inviting them to comment is welcome *if they contribute to
>the engineering discussion*.  But if you are here to comment on licensing
>outside of the engineering context, you are wasting your electrons.

Several posters since that point have made very clear posts describing
the engineering choices and deployment mechanisms which would have to
change in the context of their reading of this license.  I again point
to Eric's post as one example; there are others.

Saying "I don't like this" means nothing.  Saying "I won't implement this"
or "I will deploy this" means something.  Saying "To do this I would have
to implement an external filter mechanism in my MTA and distribute 
the Sender-ID
filter separately under this license.  The cost of that external filter
mechanism development plus the filter development isn't worth
the spam reduction costs in my environment.  If someone contributed the code
for the external filter mechanism,  however, I would be willing to work
on code efficiency and provide a pointer to an external filter" gives
a lot more for the group to work with.

				Ted Hardie

---

From: Douglas Otis <[email-masked]>
Date: Fri, 03 Sep 2004 14:38:45 -0700

> (especially the owner of small domains) to have control over.

Review the MPR draft at:

This was not referring to a reverse DNS lookup.  This record type is
being used in much the same manner as with the DNA draft but using the
_mp._smtp.xxxx.xxx sub-domain.  This would be a forward lookup and
something easily within the control of the domain owner.

-Doug

---

From: Sam Varshavchik <[email-masked]>
Date: Fri, 03 Sep 2004 19:09:34 -0400

> or "I will deploy this" means something.

Right.  I'm afraid that this licensing issue has a major engineering impact. 
Engineering as in:  well, this licensing issue will prevent any engineering 
from occuring in the first place.

---

From: wayne <[email-masked]>
Date: Mon, 06 Sep 2004 01:12:42 -0500

> is the one to which this refers.

I'm sorry, I should have been clearer.  I was referring to your
earlier post:
http://www.imc.org/ietf-mxcomp/mail-archive/msg02660.html

> it said "Discuss licensing in terms of the engineering decisions"

In the post I was referring to, Shevek was describing the friction
caused by the use of new/unfamiliar licenses and how people often
picked familiar licenses instead.

In response you wrote, in part:

: I urge the working group to stop fixating on this and go back to reviewing
: and contributing to the specifications.  This list is intended to focus on
: the engineering needed to get its charter done, and we have enough distractions
: without raising this spectre from its well-deserved grave.

In neither post did you take steps to get the IPR license issue
resolved.  I'm afraid that the IPR issue have never been placed in the
grave and this spectre will continue to be raised as long as the
patent license terms conflict with so much of the existing email
infrastructure.

-wayne