Subject: Why should I link a library thats larger than my ENTIRE MTA?

From: James Couzens <[email-masked]>
Date: Mon, 14 Jun 2004 15:53:21 -0700

Hello everyone.  Greetings and salutations to you all.

Since we are on the fine topic of XML, I would like to throw in my two
bits if you please.

Rather than quibbling about whether XML belongs in DNS or not, I would
like to raise a more important question.  Since I've found illustrations
to be useful in helping others to understand things, and I myself
personally benefit from them I would like to use one here, a visual one.

---- START VISUAL AID ----

[james@lore] / $ du -sh /var/qmail/bin
1010K   /var/qmail/bin

[james@lore] / $ ls -lah /usr/lib | grep libspf.so.1.0.1
-rwxr-xr-x   1 root root  65K Jun 14 15:40 libspf.so

Smashing!  1010K in qmail binaries, to be fair my binaries already
include my Qmail-SPF patch, as well as various other patches to bring it
in line with all the "hacks" made to SMTP and POP3 etc.. over the years.

[james@lore] / $ ls -lah /usr/lib | grep libxml2.a
-rwxr-xr-x    1 root root  1.3M Jun  4 18:42 libxml2.so.2.6.9

Lets pretend we're talking about money instead of bytes

Qmail ... $ 1,010     Qmail ... $ 1,010
libSPF .. $    65     libXML2 . $ 1,300
                      spfid?!     ?????
            -----                 -----
Total ... $ 1,075     Total ... $ 2,300

---- END VISUAL AID ----

Now can anyone see whats wrong with this picture, because *I* for one
certainly can.  So I am supposed to link a 1.3M library against my MTA,
___JUST___ to validate senders?  Can someone please explain this to me? 
I would ask why XML is even being considered but I'm not out looking to
start a flame, I simply would like to illustrate to everyone here that
there are some very deeply wrong things with XML, completely outside the
realm of publishing XML within DNS.

If this is not the appropriate time to be raising this point then I
apologize and please continue with the current discussion and I shall
attempt to resume this at another date and time.

Thank you all for reading.

Cheers,

James

-- 
James Couzens,
Programmer
-----------------------------------------------------------------
http://libspf.org -- ANSI C Sender Policy Framework library
http://libsrs.org -- ANSI C Sender Rewriting Scheme library
-----------------------------------------------------------------
PGP: http://gpg.mit.edu:11371/pks/lookup?op=get&search=0x6E0396B3

---

From: Aredridel <[email-masked]>
Date: Mon, 14 Jun 2004 17:50:35 -0600

> realm of publishing XML within DNS.

May I suggest a much more appropriate tool for the job? Expat, the XML
parser from James Clark, is a far smaller and simpler library. libxml is
a good general tool for dealing with large, complex XML related tasks.
For a simple record parser, expat is a much more natural interface.

A visual aid:

-rwxr-xr-x  1 root root 125K Jan 19 10:23 /usr/lib/libexpat.so.0.5.0
-rwxr-xr-x  1 root root 475K Mar 11 03:28 /usr/lib/libxml.so.1.8.17
-rwxr-xr-x  1 root root 957K Apr 19 04:41 /usr/lib/libxml2.so.2.6.9

So if your libraries are a bit larger than mine (debugging symbols,
likely), you're talking more like 200K. I think that's reasonable. If
not, in the tradition of qmail, a little patch doing just the neccesary
parsing and nothing more is likely to be quite a bit tighter. I think
the 20K tossed around earlier is quite likely to be accurate to well
less than an order of magnitude.  You won't get the ease of
extensibility, but I think that's a fair compromise.

Ari

---

From: Douglas Otis <[email-masked]>
Date: Mon, 14 Jun 2004 17:40:10 -0700

On Mon, 2004-06-14 at 15:53, James Couzens wrote:
<snip>
> realm of publishing XML within DNS.

XML needs more characters to encode data than an encoding scheme
specifically designed for CIDR information for outbound SMTP hosts. 
Something like BGP comes to mind.  By making the XML headers virtual,
standalone, and cast in stone by the standard, then it should not
require an XML parser for the records as the syntax should then be
fairly simple for basic scripts.

This is not likely to happen however, as there is a desire from some
vendors to leave this header open while claiming not to understand the
meaning of 'standalone' in the XML standards or accept a concept of
keeping this record definition a function of the standards process.  The
reason is to rapidly evolve this record, their stated goal, likely into
a largely proprietary format where the size of the record will grow well
beyond reasonable limits for DNS. Their previously stated limit for this
record was 2 K bytes and then changed to no limit. : (

In this regard of size, I was not happy to see a macro language added to
SPF.  Simple error messages as part of refusal or standardized header
notices could offer the same information.  These errors should be
enumerated to the degree needed and include related fields.

There is also a concern as to the number of queries needed and how many
times a record can be dereferenced.  If this becomes even moderately
used, it will seriously limit an ability to handle a mail stream with
the number of transactions.  Looking within the message is also based
upon trust.  To enable such trust, trust is forwarded to other entities,
or taken over by rewriting parameters as a means to claim trust.  As
proposed, there is no means to publish a comprehensive list of such
relationships or to detect breaks in this chain.  It would be better to
stop mail at its source rather than "perhaps" stopping a bounce.

If this even remotely puts a dent in the amount of mail abuse, expect
the system to be taken to its knees by the complexity allowed in the
standard.

-Doug

---

From: James Couzens <[email-masked]>
Date: Mon, 14 Jun 2004 18:21:45 -0700

> extensibility, but I think that's a fair compromise.

I do not believe it is appropriate to just write a small patch and
implement "just the necessary parsing", just as you can not do this with
SPF, you either implement all of it, or you don't bother at all.  This
can't be a hack-job.  You know as well as I do what all the click monkey
super admins are going to try to do with this, the next thing you know
you won't be able to parse anything.  And what if I don't want to loose
my precious extensibility?  (Rumour has it the SPF query language is
extensible btw).  

My point remains even with James Clarks's libexpat (mine is 175K
stripped btw) why should I have to link this hulking library, PLUS
another library which then has to interpret the XML?  I bet its very
feasible to get my SPF parser down to much less than 65K (which btw
includes its own DNS parsing as well).

XML just does not make any sense.  Its going to cause problems in DNS,
even the smallest library is still massive comparative to my SPF
parser.  All of this bloat for what?  

SPFv1 is extensible
SPFv1 fits in DNS
SPFv1 parser is only 65K (and could EASILY be smaller)
SPFv1 wasn't developed by a corporation and thus its motives are void of
dollar signs

Hrm, thats odd, SPFv1 does absolutely everything we need, without any of
the problems we don't.  XML has everyone's knickers in a knot and its
just a really really bad idea.  I thought we were trying to fix problems
here, not create more.  All I see is feature creep.

Furthermore, I would like to ask just what is with everyone and
"extensibility" here?  I thought we were trying to fix a problem?  Since
when does stopping forgery need to be extensible?  Why can't we just fix
this problem right the first time?  Seems to me this entire thing
reminds me of something.. can't quite put a finger on it... something
about releasing software with the mindset that should there have been
any oversight it can just be patched later?  ..something update..

If we are going to have to involve things other than originally aspired
to by the SPFv1 DRAFT (which coincidentally seems to be missing from the
table...) we ought to be doing so keeping the KISS principle in mind. 
Saying that we need to use XML because in 2020 something different might
need to be done is like saying I should go buy a Greyhound bus because I
might have some children in the future.

As was clearly iterated by Gordon Fecyk:

> Both XML and Unicode are convenient choices for authors whose
> employers use those technologies natively in their implementations. 
> These are not as convenient for anyone else.  I think we don't need to
> concern ourselves with making life convenient just for one vendor. 
> And even with that, there are other vendors who develop e-mail for the
> same vendor's platform that would not benefit from the same
> convenience.

What SPFv1 has proposed is suitable for ALL operating systems, and can
easily be implemented likely in any modern language, and there already
exist parsers many of which are cross platform, and there are patches
against all of the major MTA's including Exchange (albeit you'll have to
pay for this plugin).

Cheers,

James