Subject: RE: MARID Records and the standards process

From: "Jim Lyon" <[email-masked]>
Date: Sun, 20 Jun 2004 00:07:23 -0700

John Levine wrote (regarding a MARID record like
+indirect:msn.com/targetrep):
> If I were a spammer sending Ci@l1s spam through zombies on
> MSN and Comcast, I agree that this is the SPF record I would
> use.  But it is hard for me to imagine why anyone else would
> believe an attempt to piggyback on MSN and Comcast's reputation
> unless they already had other reasons to find me credible.

You get to piggyback on MSN's reputation exactly when you send mail
through their mail servers.  If MSN has a reputation for not sending
spam, and you get my domain's mail from MSN's servers, then you can be
assured it's not spam, because if I try to send spam, MSN will either
kick me off, rate-limit me, or fine me.  This is exactly why the
questions of authentication and reputation are intertwined.  You don't
get to piggyback on MSN's reputation if your mail doesn't go through
MSN's servers.

Regarding XML or not, John wrote:
> If experiments show that recipients can use big rich XML data
> to deal with spam significantly better, great.  At that point,
> it should be easy to send MARID 1.1 with XML along the track.
> But you have to do the work and show us the horse before you
> can get this cart moving.

Assuming that MARID 1.0 enjoys any success, then if we go down the route
you suggest there will be lots of MTAs and other programs out there that
understand SPF-style syntax, but not XML.  If we then define an XML
syntax, it would be a hard step for a domain to publish using it,
because their record becomes effectively invisible to all of the
then-deployed MTAs out there.

In short, discontinuous changes of format seldom happen.
Upward-compatible extensions happen all the time. SPF format does not
have sufficient extensibility. XML does.

-- jimbo

---

From: "Hector Santos" <[email-masked]>
Date: Sun, 20 Jun 2004 05:54:25 -0400

----- Original Message ----- 
From: "Jim Lyon" <[email-masked]>
To: "John R Levine" <[email-masked]>
Cc: "IETF-MXCOMP" <[email-masked]>
Sent: Sunday, June 20, 2004 3:07 AM
Subject: RE: MARID Records and the standards process

> have sufficient extensibility. XML does.

This might be true, but that is only because of the assumptions being used
for SPF format.

Microsoft has it written that XML can be broken up into multiple records?
Correct? So why can't this apply to SPF as well?

Sure, XML is ok and will support extensions.  But XML doesn't have a
copyright on flexible extension layouts or format.

Here is example of a simple format and extension concept that is flexibility
with extreme easy interpretation.  XML requires the reading of the entire
required for integrity, the following does not.

This is based on a unfinished proposal I was writing awhile back called
TIDAL  for "Transaction IDentification Authentication Logic"

Note: Verbosity of Records intentional for illustration purposes.  Also, I
did this long ago before I knew any idea of optimization for TXT vs RR. It
is the concept that is important here. Not the size or record type.

   DNS TXT record _tidal.example.com

IN  TXT  "TIDAL.VERSION=1.3"
IN  TXT  "TIDAL.SERVER.PRODUCT=WCSMTP"
IN  TXT  "TIDAL.SERVER.SID=2020-2929-AB2F-9323"
IN  TXT  "TIDAL.SERVER.CHALLENGE=A58F-2322-DE2F-232A"
IN  TXT  "TIDAL.SERVER.FLAGS="AUTHREG, NOSPAM, PTRREQ"
IN  TXT  "TIDAL.SERVER.HOURS.OPEN=9-17 EST, EMPLOYEE, VENDORS"
IN  TXT  "TIDAL.SERVER.HOURS.OPEN=20-23 EST,  EXTERNAL"
IN  TXT  "TIDAL.SERVER.HOURS.DAYS=Mon-Fri"
IN  TXT  "TIDAL.SERVER.RFC=3821"
IN  TXT  "TIDAL.CLIENT.ACCEPT.IP4=208.248.131.9"
IN  TXT  "TIDAL.CONTACT.ABUSE="[email-masked]"
IN  TXT  "TIDAL.UBE.GATEWAY="forspammers.examples.com"
IN  TXT  "TIDAL.MAIL.ATTACHMENTS.SIZE=NONE"
IN  TXT  "TIDAL.MAIL.ATTACHMENTS.RESTRICTED="DOC, JPG, BMP, XLS, RTF"
IN  TXT  "TIDAL.MAIL.RECIPIENT.MAX=10"

Minimum TIDAL records required (most systems)

TIDAL.VERSION=major.minor
TIDAL.CLIENT.ACCEPT.x=y

X=Y could be the standard SPF/MCEP concepts that we have today.

Since we are talking about possible extension, TIDAL was a proposal that
incorporates most of my mail hosting, transport and gateway design
experiences in the past 25+ years in multiple C/S and P2P mail networks and
as a early pioneer in BBS systems and offline mail systems.  In 1984, I
designed the 3rd Offline Mail product in the market place behind TAPCIS and
QWK.  I made the most money off the two since TAPCIS targeted CompusServe,
the original QWK targetted PCBOARD and Silver Xpress (my product) called
over 20+ BBS Online Mail Hosting Systems, including the top 3 multi-million
dollars BBS systems that was helping define the new wave of PC based
electronic communications.  Of all the mail products that was not under my
control was the direct ownership of a BBS product, but we have a product
that covered every other aspect of the mail market.  As these early markets
was evolving to the new Internet, in 1996 we brought the #1 BBS system at
the time newly designed for WIndows and the internet called WINSERVER
"Wildcat! Interactive Net Server."  It was atleast 5 years ahead of its time
and today with Microsoft .Net as the closest technology to what we have
today.  The point of this short resume is to show why I think the following
extensions is something I see as fitting the mold of current and future mail
design requirements.

o Living with SPAM.

It should go with saying SPAM is here to stay. The advertising and marketing
dollars behind it simply too big to be ignored.   The purpose of CANSPAM was
to offer a compromise between the capitalistic realisms of the market place
and the required correction required in the technology to minimize the
end-market concerns and abuse.

TIDAL offers a way for spammers to fit into the mail model by helping them
be legit. Yet, at the same time,  mail systems will like to avoid SPAM or
make it available during certain parts of the day.

The goal here is to help shift overhead to the spammer by having them lookup
the tidal records to determine what are server attribute and policies.
What can the spammer learn here?

TIDAL.SERVER.FLAGS="AUTHREG, NOSPAM, PTRREQ"
TIDAL.SERVER.HOURS.OPEN=9-17 EST, EMPLOYEE, VENDORS
TIDAL.SERVER.HOURS.OPEN=20-23 EST,  EXTERNAL
TIDAL.SERVER.HOURS.DAYS=Mon-Fri
TIDAL.SERVER.RFC=3821
TIDAL.UBE.GATEWAY="forspammers.examples.com"

The spammer can see what opens is the server open for unsolicated mail, and
he can see that the server has provided a specific gateway host to use.  He
can also see whether the host allows spam and the type of authentication
required or checks done to allow mail sending.

Of course, these are extensions.  You don't need to support them, but if
wanted to added more server attribute flags or extensions you can easily do.
For example, Microsoft may want to introduce a way for spammers to contact
them for registration:

TIDAL.UBE.REGISTRATION=http://spammer.registration.microsoft.com

o Operational Hours

Spammer or not, companies would love to be able to define how their mail
servers are being used.  This will reduce scalarbility requirements and
cost.  A Tidal Server can define the hours of operations based on the type
of transaction:

TIDAL.SERVER.HOURS.OPEN=9-17 EST, EMPLOYEE, VENDORS
TIDAL.SERVER.HOURS.OPEN=20-23 EST,  EXTERNAL
TIDAL.SERVER.HOURS.DAYS=Mon-Fri

This basically says the servers are down on the weekends so clients should
even try to send mail over the weekend.  It also says that only employees
and vendors can send mail during business hours and that a 3 hour window is
available for anonymous transactions.

Why will clients support this?  Because they won't be able to send mail
otherwise.  Clients will support it because the efficient of their operation
is improved allowing them schedule mail at the appropiate time.

o Acceptable Mail

Tidal Servers can define server attributes defining the type of acceptable
mail, for example:

TIDAL.MAIL.ATTACHMENTS.SIZE=NONE
TIDAL.MAIL.ATTACHMENTS.RESTRICTED="DOC, JPG, BMP, XLS, RTF"
TIDAL.MAIL.RECIPIENT.MAX=10

In this case, the TIDAL server has no limits on size but it will reject mail
containing DOC, JPB, BMP, XLS or RTF context.  It also specifies a limit on
the number of recepients allowed.

Possible MAIL attributes extensions might be Microsoft defining the maximum
number of messages allowed per IP

TIDAL.MAIL.IPLIMIT=100
TIDAL.MAIL.IPLIMIT.HOURLY=5
TIDAL.MAIL.IPLIMIT.DAILY=10

That basically says that the sender has 100 days since its initial message
to send a maximum of 1000 messages but can only do so 10 per day and 5 per
hour.

You might even want to add an expiration per IP

TIDAL.MAIL.IP.EXPIRE=20040701

etc.

o Sender Authethication

Of course, the main goal in all this is to authenticate the sender.
Besides the fact that there is a certain level of  "implied authorization"
with a client supporting tidal records,  at the most basic level, we can
have the standard concepts used by SPF/MCEP to define the tidal records:

TIDAL.CLIENT.ACCEPT.MX
TIDAL.CLIENT.ACCEPT.IP4
TIDAL.CLIENT.ACCEPT.IP6
TIDAL.CLIENT.REJECT.MX
TIDAL.CLIENT.REJECT.IP4
TIDAL.CLIENT.REJECT.IP6

etc,

a possible extension for reporting can be isolated to just IP6 lookups:

TIDAL.CLIENT.REJECT.IP6.REPORT=xxxxxxxxxxxx

but TIDAL goes further to promote true Client/Server handshaking negotiation
ideas using product identification, serialization, hashing and challenge
handshaking response concepts.

TIDAL.SERVER.PRODUCT=WCSMTP
TIDAL.SERVER.SID=2020-2929-AB2F-9323
TIDAL.SERVER.CHALLENGE=A58F-2322-DE2F-232A

But of course, this is all beyond the scope of MARID.

Finally, if you have not noticed yet, the TIDAL format is compatible with
XML conversion with no lost in functionality.

<tidal version=1.2>
  <server>
    <product=WCSMTP/>
    <sid=2020-2929-AB2F-9323/>
    <challenge=A58F-2322-DE2F-232A/>
    <flags>
      <authreg=1/>
      <nospam=1/>
      <ptrreq=1/>
    </flags>
    <rfc=3821/>
  </server>
  <client>
    <accept>
      <ip4=208.248.131.9/>
    </accept>
  </client>
  <contact abuse="[email-masked]"/>
  <ube>
   <gateway="forspammers.examples.com"/>
  </ube>
  <mail>
   <attachments>
     <size=none/>
     <rejected="DOC, JPG, BMP, XLS, RTF"/>
     <recipient>
        <max=10/>
     </recipient>
   </attachments>
  </mail>
</tidal>

Jim, my main point is that you can be flexibilty with extensions using an
easy to write and read non-XML format.

With all this said,  XML has only one benefit - marketability.  Say XML,
right or wrong, people have a clue of what it is and what to expect.

I just want you to realize that XML and support XML extensions requires the
reading and parsing of the entire structure otherwise, some XML parsers,
including the Microsoft XML parser will return a "invalid/illegal XML
format" error.

A parsing logic for SPF (or TIDAL) does not required the reading of all the
records. Only what the client understands and supports.  This is important
because SPF allows for short circuiting.  XML may have trouble with this
important client conditional logic. XML requires the entire structure to be
syntaxically correct first because you can even begin using its rules.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

---

From: "John R Levine" <[email-masked]>
Date: 21 Jun 2004 02:25:36 -0400

> > use. ...

> kick me off, rate-limit me, or fine me. ...

I understand this argument, but it still strikes me as utterly
unpersuasive.  For one thing, this kind of piggybacking only makes sense
when networks don't have unauthorized users, which means no zombies, and I
doubt that any plan that only works when the zombie problem is solved will
be usable soon enough to be interesting.  If I were that C1@lis spammer
sending out tons of spam through WBW (an ISP with a dreadful reputation),
I'd publish SPF saying that I use MSN, Comcast, and WBW.  I might even
sign up for a few MSN and Comcast accounts and trickle out a little mail
through them.  What reputation do I get?  The max?  The min?  The median?
The one whose IPs a particular message came through?  That's in effect an
IP based reputation system, which I don't think has a lot of support in
MARID, given the modest enthusiasm I've seen for CSV.

At least as important, MSN and Comcast are going to have reputations that
say these are large consumer ISPs with 24/7 abuse desks and other facts
that don't trickle down to their customers.  They may or may not be
willing to vouch for various characteristics of some or all of the domains
that their customers use, but they get to decide, not you or me.

> In short, discontinuous changes of format seldom happen.

Indeed.  They only happen when they're useful.  You cited http as an
extensible format a little while ago, and I can't help but notice that
extensible http 1.0 is quite incompatible with non-extensible http 0.9,
yet everyone adapted because 1.0 has advantages that made the transition
worthwhile.

This still boils down to "it might be useful".  I have lots of swell
anti-spam ideas that might be useful, but I'm not going to tell people to
build standards around them until I have some experiece that demonstrates
how useful they are.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.