Subject: My message about notify looping issues and possible solutions

From: Ned Freed <[email-masked]>
Date: Mon, 03 Dec 2007 14:14:00 -0800 (PST)

Date: Tue, 06 Nov 2007 07:28:30 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Request for review of draft-ietf-sieve-notify-mailto
To: Alexey Melnikov <[email-masked]>
Cc: Ned Freed <[email-masked]>, Cyrus Daboo <[email-masked]>,
 Barry Leiba <[email-masked]>, Michael Haardt <[email-masked]>,
 Keith Moore <[email-masked]>
Message-hash: 66BB4143B8AAEAD5C2F3C59D7E3BBF4F
Original-recipient: rfc822;[email-masked]

> Can you please review the latest draft-ietf-sieve-notify-mailto, as Lisa
> is about to issue IETF LC on it.

> In particular I am looking for your feedback on 2 issues:

> 1). Is the text of mail loop prevention sufficient?

It has at least two bugs, one obvious, the other subtle.

First the obvious bug: THe text currently says that any Auto-submitted: field
prevents notifications from being generated. What about "Auto-submitted: No"?
Mind you, I doubt very much that generation of this field with this value will
occur often if at all (in fact one possibility is to simply deprecate the "no"
value at this point), but this is nevertheless an issue that needs to be fixed.
The obvious solution is to treat "no" and only "no" as the same as not having
an auto-submitted field.

I also have to wonder if it doesn't make sense to add a parameter to
Auto-submitted: that can be used to identify the email address of the owner of
the sieve generating the notification.

Incidentally, given that this document established an auto-submitted registry
Keith Moore probably should be alerted to this action and I'm cc'ing him on
this response. I don't think he follows the Sieve WG regularly.

Now the subtle bug: The current rules permit a case where an undetectable loop
can occur. Suppose I create a sieve that says:

   require "notify";
   notify :from "[email-masked]"
          :message "whatever"
          "mailto:[email-masked]";

This causes a notification to be sent unconditionally to an invalid address
whose validity cannot be checked immediately and with an envelope from pointing
back to the sieve owner. So in comes a triggering message, out goes a
notification. The notification then bounces and unless the DSN contains an
auto-submitted field it will generate another notification. (Note that the
current draft standard for DSNs, RFC 3464, preceeded RFC 3834 and hence does
not require an Auto-submitted field be included in the DSN. In fact RFC 3834
goes out of its way not to impose requirements on DSNs and MDNs so an
implementation can be RFC 3834 compliant without adding Auto-submitted to DSNs.

Since we cannot retroactively impose a requirement that DSNs and MDNs contain
an auto-submitted field (and even it would take too long to deploy) we have to
solve this some other way. I see two possibiities, one tricky, cute, but
unreliable, the other ugly but reliable:

(1) Require that when the envelope from on the triggering message is blank
    that the message be checked to see if it is a DSN/MDN and if it is look
    at the third part to see if it contains an auto-submitted header.

(2) Use the same trick we required on redirect: When the triggering message
    contains an empty envelope from require that any notifications it generates
    also have an empty envelope from.

I mention (1) only because of its cuteness - IMO it isn't going to be reliable
enough, if for no other reason than the widespread existance of DSNs and MDNs
using nonstandard formats. (Thank you qmail...) That leaves (2). I don't like
it much but I don't see any real alternative.

> 2). Something I remember from a previous conversation with you: are you
> happy with the requirement on the Notify action to preserve Received
> headers from the original message?

No I'm not. As currently defined it's quite simply wrong to do this. The
original message is what garnered those trace field values, not the
notification. Someone attempting to trace the message path using this
information could easily end up being very confused. It is also bound to trip
any loop detection logic that has been set to fairly tight tolerances. And
there's also the spam filter issue - one check I've seen made is to compare the
dates on the Received: fields with the Date: field. A Date: field that's in the
future relative to the first Received: field could raise flags and cause
problems. (Mind you, I don't see a reason to cater to broken filter criteria
but that's not sufficient justification for ignoring the issue completely.)

Unfortunately there are no really good ways to solve this problem. I see the
following possibilities:

(1) Rely on auto-submitted being preserved as a loop check and don't do the
    Received: line copy at all.

(2) Classify the Auto-submitted: field as a trace field and require that it
    appear above the copied Received: fields as a sort of boundary between
    the old set of Received: fields and the new.

(3) Same as (2) except make the trace field some newly defined thing.

(4) Same as (2) except use a specially formatted Received: field as the
    boundary.

(5) Make the Received: line copy a SHOULD rather than a MUST. Add some
    discussion that the separation between old and new Receieved: fields
    can usually be determined by comparing the dates with the those on the Date:
    field.

(6) Add the discussion mentioned in (5) but ignore the problem otherwise.

I think (2) is our best bet. It should not be hard for implementors to do and
it deals with the tracing problem, albeit in a somewhat fragile way (header
fields of different with different names can and do get reordered in some
cases). And while it doesn't solve the tight tolerance loop counter it does
offer various ways out, e.g., impose a loose limit on the total number of
Received: lines but a tighter limit on the number above the Auto-submitted:
line. (I already have some logic along these lines so this is the course of
action I'll probably bursue.) The same is true of date checks - filters could
limit their checks to the fields above the Auto-submitted: (of course this
requires a change in those filters).

(1) makes me somewhat nervous - people have had their hands slapped often
enough about preserving Received: fields but there's no similar history to
support Auto-submitted: field preservation. I also doubt it will make it past
the present IESG - I'm sure you know why. (6) is unacceptable to me, so much so
that if it is the path chosen I will be implementing notify with the Received:
line copy action disabled by default, regardless of whether or not it is deemed
to be mandatory in the document. (We have several sites that make heavy, heavy
use of notify and at least a couple of them do a lot of messaging tracing that
would be totally confused without a well-defined trace boundary. One of them
also operates in a regime where it is likely that trying to sort things by time
comparisons won't be reliable.) I can live with (3) or (4) but fail to see any
advantage over (2). I can also live with (5), of course, but like (1) I doubt
it will make it past the IESG.

A few more comments on the present draft:

THe text in section 2.7 about :from needs to say that security checks MAY be
applied to the :from values users specify. 

I see nothing in the security considerations section or anywhere else about
placing limits on the number of notify actions a sieve script can perform. Nor
is there any discussion of this in the base notify draft. Based on our recent
experience with the base specification, I'm pretty sure this is a mistake that
is going to guarantee a discuss from you-know-who. 

The obvious way to address this is to add some text either in this document or
in the base notify specification that's modelled on the discussion of redirect
in the base specification.

Finally, a nit: The example shows :importance being specified even
though the specification assigns no significance to :importance
when this method is used. I suggest removing it.

				Ned

---

From: Michael Haardt <[email-masked]>
Date: Tue, 04 Dec 2007 09:41:15 +0100

> [Sending a notification to an address that generates a bounce]

>     at the third part to see if it contains an auto-submitted header.

I'm afraid that's something for an ideal world.

> it much but I don't see any real alternative.

Indeed.

> information could easily end up being very confused.

Personally, I would be more confused on not having "Received:" from the
triggering message, but that may just be me.

> It is also bound to trip

I never thought of the date issue, and although I don't use a check like
that, "Date:" being in the future looks really ugly to me.

>     Received: line copy at all.

I am less afraid of trouble in the real world, but if that choice is
going to cause trouble with approval, it isn't one.

> (3) Same as (2) except make the trace field some newly defined thing.

Depending on the order of different fields relative to each other
sounds like asking for real trouble to me.

> (4) Same as (2) except use a specially formatted Received: field as the

I never saw "Received:" being reordered.  It's ugly, but anything apart
from (1) is, and to me it sounds best among all options.

> (6) Add the discussion mentioned in (5) but ignore the problem otherwise.

It is bad enough "Date:" is sometimes being "fixed" or accepted although
being wrong.  Knowing that and trying to assign relevance to it is not
an option to me.

Notifications are really a new kind of message, an odd mix somewhere
between redirected and vacation messages, so all this is not just relevant
to Sieve.  I consider it important to reach a consensus, no matter how
much work it may still take, because anything we screw up now probably
can't be fixed later.

Thanks a lot for your review!

Michael

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 4 Dec 2007 10:35:15 +0100

FWIW, I tried something else entirely, namely using RFC 4865 to handle looping.

Use HOLDFOR=120 if you've sent a notification in the same general 
direction in the recent past, and mail loops become annoyances rather 
than disasters. With a round-trip time of 2-4 minutes, it's difficult 
to pile up more than a three- or four-digit number of messages before 
the loop is discovered and fixed. A whole weekend is just 1000-2000 
messages.

Arnt