Subject: external lists and spam protection

From: Dilyan Palauzov <[email-masked]>
Date: Tue, 02 Oct 2007 00:09:55 +0200

Hello,
     In the draft-melnikov-sieve-external-lists-01 I would like very  
much to have two more tests:

    Usage:  envelope [COMPARATOR] [ADDRESS-PART] [MATCH-TYPE] [:list]
            <envelope-part: string-list>
            <key-list: string-list>

where ADDRESS-PART can be :hostname and :hostip, near the usual :from  
and :to. The :hostname and :hostip are substituted with the hostname  
of the last host that sent the mail, and hostip is the IP of that  
host. The idea is, that the script can check if the mail comes from an  
unwanted host and reject it. The real application is, that the script  
can check in an DNS blacklist, if the IP of the sending host is  
present there and refuse the mail in this case. I think it will be  
useful, once sieve scripts are executed during the smtp connection is  
not closed - then every user could specify which DNS blacklists does  
s/he want to make use of.

   At the same time the :hostname and :hostip have no usage without  
the external lists, therefore I suggest they are included here.

   Question: what to do if an implementation has no access to the  
information, or if a message is somehow not sent over smtp?

   Question: Does somebody have an idea, if spammers use one and the  
same EHLO text to send mails, regardless of the sending host? In this  
way envelope :helo would be useful as well.

   By the way, where is the changelog of draft-ietf-sieve-notify-xmpp-05 ?

   Със здраве,
     Дилян

---

From: Ned Freed <[email-masked]>
Date: Mon, 08 Oct 2007 09:51:20 -0700 (PDT)

I'm glad you brought this up - I've been meaning to comment on this draft.

> s/he want to make use of.

I see what you're after here but to my mind this is a separate extension. Also,
it doesn't really fit under envelope - getting at the remote host and IP seems
to be a much better fit for the environment extension. Please see
draft-freed-sieve-environment-01.txt when it comes out (should be any time now)
and take a look at the remote-host and remote-ip environment items.

> the external lists, therefore I suggest they are included here.

I disagree. While they may be most useful when used with an external list, it's
not at all unreasonable to have a script that checks this information for other
reasons.

> information, or if a message is somehow not sent over smtp?

The environment extension deals with this issue by having the test fail
unconditionally. This makes it easy to test to see if an item is supported:

    if environment :matches "foo" "*" { code for when item foo is available }
    else { code for when it isn't }

Note that unlike comparators there is presently no way to specify a specific
environment item in a require clause or ihave test. I'm disinclined to provide
one since whether or not some enviroment items are applicable depends more on,
well, the current enviroment the script is executing in and not on the Sieve
implementation.

> way envelope :helo would be useful as well.

The HELO/EHLO parameter value is pretty much the definition of a boundary case.
Since it is presented before the transaction starts you could say it really
isn't part of the envelope. OTOH, it is almost always thought of as being part
of it.

My personal preference is to make getting the EHLO value an environment item as
well, but that's mostly because it avoid the need to write up yet another
envelope extension. But I could be convinced this isn't the way to go, which is
why I haven't added it to the environment draft yet.

Now, getting back to :list, I think this is a tremendously useful extension but
it needs a little work. My first issue with it is that :list is currently
defined as a new tagged argument orthogonal to any existing arguments. THis
seems wrong - I think :list should instead be defined as a new MATCH-TYPE. I
say this because it sems to me that :list only makes sense as a :is match - I
want to be able to define lists using, say a dynamic LDAP URL where the list
name simply gets shoved in to the URL somewhere.  I for one don't see any way
to support :contains for list names and :matches and :regex are truly scary.

My second issue is that the lists it accesses are inherently single-column. I'd
like to see the ability to look something up on a list and then get some
property information back about the list entry that matched. I'd also like to
have a way to get back information on what actually matched - remember that
many of our tests check multiple things even if a list of things to check isn't
specified.

One way to do this that doesn't require any draft changes is to fold the
property check into the list name. Suppose you have an external list of
addresses "foo" that associates a status of "red", "green" or "yellow" with
each address. This could be dealt with by defining "virtual" lists "foo-red",
"foo-green" and "foo-yellow". But this rapidly gets unwieldy and also can
result in scripts checking the same list more than once - not good if the list
check is expensive, which it definitely can be. I believe the better way to do
it is to have the test either have a way of specifying required propery values,
have a way to return property values, and preferably both. (Note that the thing
that matched can be thought of as a property.)

The obvious change is to change this so that :list takes the list name or names
as an argument and have the key-list match against a single returned property.
But this really grots up the more common single-column case and doesn't solve
the problem of getting the thing that matched back so I think it is a
nonstarter.

Another approach is to define a second tagged argument, say :property, that
takes as its argument the value that the matching list's "property" has  to
have. Or you can have :propert take two arguments specifying the property name
and value respectively if you want to support multiple properties. (I note in
passing that it is trivial to map either form into an LDAP filter.)

But while this is cleaner it still may result in scripts performing multiple
queries against the same list - not good. And it doesn't provide a way to
get back the item that matched - also not good.

Variables provide the obvious way to get information back from :list. Suppose
:list, like :match, were to set the ${0}, ${1} ... ${N} variables? ${0} would
be given the value of the item that matched, ${1} would be the first associated
property, ${2} would be its value, ${3} would be the second property name, and
so on. So now you could write something like:

    require ["extlists", "variables", "reject"];
    if address :domain :list "from" "known-domains" {
        set "domain" "${0}";
        set "status "${2}";
        if string "${status}" "bogus" {
          reject "From ${domain} is considered bogus";
        }
        elsif string "${status}" "good" { keep; }
    }

I prefer this last approach. The one drawback I see is that this does require
variables support, but I reallly don't think that's overly onerous.

>    By the way, where is the changelog of draft-ietf-sieve-notify-xmpp-05 ?

Sorry, can't help you with that.

				Ned