Subject: mostly open issues

From: Tim Showalter <[email-masked]>
Date: 11 Nov 1998 15:25:43 -0500

Here are the things I still believe are unresolved.

1.  Request: Implementations are required to decode header charsets. 

    (Chris and Harald have probably resolved this, but I need to figure
    out wording.)

2.  Request: Reject should be optional. 

    (I just want to know that this is consensus.)

3.  Request: forward adds Resent-* headers. 

    (I just want to know that this is consensus.)

4.  Request: Short-circuit evaluation should be either MAY or MUST, and
    must be discussed in case it matters in the future.

    (I have heard contradictory opinions on this.  Someone (I forget
    who) wanted it removed since it doesn't yet matter.  Chris (and
    probably others) want it in there because it will probably
    eventually matter.  I propose that it has to be discussed, is a MAY
    for implementations, and a MUST if extensions offer tests with side
    effects.)

I have a list of things I expect to change in -05 at
<http://www.contrib.andrew.cmu.edu/~tjs/sieve-changes.html>, which is
the list I posted last week with all the changes I know about.

Thanks!

Tim

-- 
Tim Showalter <[email-masked]>

---

From: Randall Gellens <[email-masked]>
Date: Wed, 11 Nov 1998 12:40:15 -0800

>     (I just want to know that this is consensus.)

I think Reject has to be optional, because some implementation 
environments can't support it (specifically, some environments either 
can't control the DSN generation, or can't alter the text used in it).

>     (I just want to know that this is consensus.)

My understanding of the DRUMS discussion on Resent-* headers is that 
they are not to be used for .forward style automatic, unconditional 
forwarding, but are supposed to be used for user-initiated forwarding. 
I say that Sieve falls into this latter category, and thus Resent-* 
headers SHOULD be generated.

>     must be discussed in case it matters in the future.

I've implemented them, but I suspect there may be implementation 
environments where this is hard to support, and so I don't think we can 
require it.

---

From: Tim Showalter <[email-masked]>
Date: 11 Nov 1998 15:58:27 -0500

> Date: Wed, 11 Nov 1998 12:40:15 -0800
> From: Randall Gellens <[email-masked]>

> headers SHOULD be generated.

Okay, I'm not sure if I agree with the first half, but I can accept the
second half in any case.

> environments where this is hard to support, and so I don't think we can 

Really?  Could you provide an example?  I can think of only one, and I
think it's rather absurd.

Chris, why shouldn't this be "SHOULD"?  I think SHOULD would be useful
since I assume short-circuit evaluation will be required by extensions
that specify tests with side effects, and it's useful from an efficiency
standpoint.

---

From: "Alan K. Stebbens" <[email-masked]>
Date: Wed, 11 Nov 1998 13:41:51 -0800

| 3.  Request: forward adds Resent-* headers.

This should be context dependent.  If the Sieve filter is run by the system
administrator, and forwarding is caused for site administrative reasons,
then the Resent headers probably should not be inserted.

On the other hand, if the Sieve filter is run by a user, causing a
forwarding, then the Resent headers should be added.

Perhaps a new action "resend", as a synonym to "forward", can be defined, to
cause the additional effect of inserting headers?

	resend  newaddress	# gets "Resent-*" headers added
	forward newaddress	# no "Resent-*" headers

--
Alan K. Stebbens <[email-masked]>

---

From: Tim Showalter <[email-masked]>
Date: 11 Nov 1998 17:04:08 -0500

> Reply-To: <[email-masked]>
> From: "Alan K. Stebbens" <[email-masked]>
> Date: Wed, 11 Nov 1998 13:41:51 -0800
> 

I don't understand the difference.  Typically Sieve scripts are run by
the system at time of mail delivery, which falls into neither of the
above cases.

I believe the right thing to do is to always add Resent headers, despite 
the possible efficiency gains of not doing so.  I believe the benefit of 
trace informaton outweighs the benefits of efficiency gains, especially
given that forward is going to cause mail loop headaches.

---

From: "Alan K. Stebbens" <[email-masked]>
Date: Wed, 11 Nov 1998 14:15:13 -0800

| > | 3.  Request: forward adds Resent-* headers.
| >
| > This should be context dependent.  If the Sieve filter is run
| by the system
| > administrator, and forwarding is caused for site administrative reasons,
| > then the Resent headers probably should not be inserted.
| >
| > On the other hand, if the Sieve filter is run by a user, causing a
| > forwarding, then the Resent headers should be added.
|
| I don't understand the difference.  Typically Sieve scripts are run by
| the system at time of mail delivery, which falls into neither of the
| above cases.

Many ISPs run filters on all incoming mail.  Sometimes, forwarding takes
place because of infrastructure changes, system changes, etc., on a
per-domain basis, especially in a managed messaging environment.  Resent
headers should not be inserted because of these system-wide, system-imposed
filtering rules.

On the other hand, when a user forwards a piece of mail, unconditionally or
not, Resent often should be added.

As I said, it depends on the context.

I think defining a "Resend" action with the additional semantics of
inserting headers is better than imposing the headers on "forward".

| I believe the right thing to do is to always add Resent headers, despite
| the possible efficiency gains of not doing so.  I believe the benefit of
| trace informaton outweighs the benefits of efficiency gains, especially
| given that forward is going to cause mail loop headaches.

I view Resent- headers as informational only.  Tracing should rely on
Received headers, not Resent headers.

---

From: Randall Gellens <[email-masked]>
Date: Wed, 11 Nov 1998 14:55:17 -0800

> think it's rather absurd.

No, I can't provide an example off-hand, but then I am not familiar 
with every environment in which someone might want to implement Sieve. 
Sure, C, AppleScript, and (I think) Perl can all provide this, but 
given that Sieve is likely to be implemented using whatever tools an 
existing mail server provides, I think we should be careful about the 
restrictions we impose.

---

From: Randall Gellens <[email-masked]>
Date: Wed, 11 Nov 1998 14:47:32 -0800

> headers should not be inserted because of these system-wide, system-imposed

I would not expect Sieve to be used in these cases.  Existing methods 
(for example, .forward) would be used.  Sieve is for conditional, 
message-based actions, during final delivery, on behalf of users.

---

From: Chris Newman <[email-masked]>
Date: Wed, 11 Nov 1998 15:53:24 -0800 (PST)

> Okay, I'm not sure if I agree with the first half, but I can accept the

Actually, I disagree with Randy here.  My take on the DRUMS decision was
that Resent-* headers are for User Agent manual resends.  If Sieve is
running on an IMAP server, then Resent-* headers shouldn't be generated,
IMHO.

> that specify tests with side effects, and it's useful from an efficiency

There is no justification to use SHOULD.  Either programmers can rely on
short-circuit behavior or they can't.  The former is MUST, the latter is
MAY.

		- Chris

---

From: "Alan K. Stebbens" <[email-masked]>
Date: Wed, 11 Nov 1998 17:12:21 -0800

| > Many ISPs run filters on all incoming mail.  Sometimes, forwarding takes
| > place because of infrastructure changes, system changes, etc., on a
| > per-domain basis, especially in a managed messaging environment.  Resent
| > headers should not be inserted because of these system-wide,
| system-imposed
| > filtering rules.
|
| I would not expect Sieve to be used in these cases.  Existing methods
| (for example, .forward) would be used.  Sieve is for conditional,
| message-based actions, during final delivery, on behalf of users.

An example: When one of several domains of users is moved from one mailer to
another, during the migration phase the former will forward mail to the
latter, on a system-wide basis.  Using .forward features, or other per-user
features is neither convenient nor very efficient.

Our product uses Sieve as the system-wide mail filter, specifically for SPAM
filtering, but it is useful to the system administrators for other purposes
as well.

In any case, my principal point was that the "forward" action should not
always insert "Resent-" headers.  I suggested that another action, "resend",
be defined to apply this semantic.

---

From: Randall Gellens <[email-masked]>
Date: Wed, 11 Nov 1998 17:20:43 -0800

At 3:53 PM -0800 11/11/98, Chris Newman wrote:

> My take on the DRUMS decision was

As I recall, we didn't discuss cases such as Sieve, only user-agent 
filtering and automatic, .forward style.  The consensus was to generate 
Resent-* on the former, not the latter.

So, where would Sieve fit in?  I say Sieve, being (a) under the direct 
control of the end-user, and (b) acting conditionally based on 
per-message criteria, is essentially an extension of the UA, and thus 
Resent-* headers should be generated.  By comparison, I consider Sieve 
to be equivalent to existing filtering in many clients, which often 
supports a forward action.

---

From: Randall Gellens <[email-masked]>
Date: Wed, 11 Nov 1998 17:27:45 -0800

> features is neither convenient nor very efficient.

The fact that you use Sieve instead of a .forward file, or a mail 
server configuration change, doesn't change the situation.  Your use of 
Sieve is fine for you, and I'm happy it works for you, but it isn't the 
target for Sieve.  Sieve is intended for user-directed actions during 
final delivery.

> filtering, but it is useful to the system administrators for other purposes

There are mail servers which permit sites to install filters or custom 
inspection and actions at two different points: message traversal, and 
message delivery.  Sieve operates at message delivery.  If you want to 
use the same syntax at message traversal, that's fine, but I don't 
think it should drive the syntax, especially when it can unnecessarily 
complicate things.

> be defined to apply this semantic.

I would object to that.  Having two different actions, with almost 
identical semantics, seems to me to be a bad idea.  It will lead to 
user confusion, and users will use the wrong one.

If your specific product wants to use Sieve at a stage earlier than 
final delivery, and wants to add product-specific extensions, for 
example, a relay action, that's a different matter.

---

From: "Alan K. Stebbens" <[email-masked]>
Date: Thu, 12 Nov 1998 00:42:48 -0800

| > In any case, my principal point was that the "forward" action should not
| > always insert "Resent-" headers.  I suggested that another
| action, "resend",
| > be defined to apply this semantic.
|
| I would object to that.  Having two different actions, with almost
| identical semantics, seems to me to be a bad idea.  It will lead to
| user confusion, and users will use the wrong one.

If you do not wish to add a new action, then please do not add unwanted
semantics to the existing action.  The "forward" action should do simply
that: forward email; it should not *also* insert headers.  If I want to
insert headers, I'll arrange to do so myself.

If the "forward" action always insert headers, then it cannot be used to
forward mail without modification.  There _are_ users that do not wish to
have additional headers inserted as a part of a forwarding.  So, there must
be an action that provides the "forwarding-without-modification" or the
Sieve rules become useless for this usage.

What is the issue with a "resend" action?  I do not think these definitions
will confuse users:

	Resend -- insert "Resent-" headers and then forward.
	Forward -- forward the message (without modification).

Given these definitions, users have a clear choice of behaviour.

---

From: Jack De Winter <[email-masked]>
Date: Thu, 12 Nov 1998 13:52:36 -0500

>> > environments where this is hard to support, and so I don't think we can 

Just bringing these up as it may affect things:

Not to make things more confusing here, but in 2.11 where it is
mentioned that left-to-right evaluation is to be used (implied that
there may be extensions that allow for multiple arguments), it does
not make any indication of anything akin to the ( and ) for expressions
in C.  

Also, Randall Gellens mentioned in one of his posts that he used ^r and ^f as
placeholders in his implementation.  Not slapping Randall's hands on
this subject, but shouldn't we have some SIEVE supplied method of doing
that instead of local implementation stuff?  Otherwise Randall's scripts
are not really going to be that portable after all.

regards,
Jack
-------------------------------------------------
Jack De Winter - Wildbear Consulting, Inc.
(519) 884-4498		http://www.wildbear.on.ca/homepages/jack/

---

From: Randall Gellens <[email-masked]>
Date: Thu, 12 Nov 1998 14:46:02 -0800

> are not really going to be that portable after all.

^r and ^f only make sense if MARK is added.  If the script can't add a 
header, then there is no need for expressions which expand to 
message-dependent values.

I'd love for MARK to be added, but I'm also happy with leaving it out 
of the base spec, and doing it as an extension.

---

From: Ned Freed <[email-masked]>
Date: Thu, 12 Nov 1998 22:05:54 -0800 (PST)

> running on an IMAP server, then Resent-* headers shouldn't be generated,

My take, arrived at independently, is that Chris is reading DRUMS correctly.

In addition, we've found that generation of Resent-* fields isn't a good idea.

We generated Resent-* fields as a result of automatic forwards for around 10
years. And throughout that time we had regular reports of problems with it --
some agents use Resent-* fields preferentially, often with poor results,
multiple sets of Resent-* fields are inherently problematic, various MTAs have
funny ideas about what constitutes a "correct" set of Resent-* fields and
bounce what they think of improperly formatted messages, and so on.

But we persevered because RFC822 certainly encourages this practice as part of
forwarding, and we felt that not documenting such forwarding actions was poor
practice.

But then along came DRUMS, which relgated these fields to the status of trace
information and for user agents. And so we decided to stop generating them to
match up to the standard-to-be.

And guess what? It seems there was downside to this. It seems that nobody
really wanted the fields to begin with, so while a bunch of problems went away,
no new ones cropped up.

So now I don't think these fields should be used for automatic forwards under
any circumstances.

> short-circuit behavior or they can't.  The former is MUST, the latter is

I don't think the existance of environments where short-circuit evaluation
isn't possible constitutes justification for making the language flexible in
this way. Either short-circuit evaluation is useful enough to standardize, in
which case it should be a MUST, or it isn't, in which case we should say that
you MUST NOT assume the language does it and leave it at that.

A MAY in such cases will encourage people to use short-circuits as a language
feature when they find it works. And this will lead to interoperability
problems when scripts are written assuming it.

My personal take is that short-circuit evaluation is useful enough to warrant a
MUST. But I'd rather have a statement saying scripts MUST NOT take advantage of
short-circuit evaluation than one saying implementations MAY do it.

				Ned

---

From: Steve Hole <[email-masked]>
Date: Fri, 13 Nov 1998 08:45:54 -0700

> In addition, we've found that generation of Resent-* fields isn't a good idea.

I'm inclined to agree with Ned.   Not because of his and Chris's reading of 
DRUMS, but because of the argument that the resent headers are generally not 
as useful as one would think.  The add complexity and potential ambiguity 
with little relevant benefit -- especially if a receiving UA is not supposed 
to use them for anything other than trace information. 

---  
Steve Hole                           
The Esys Corporation
Mailto:[email-masked]  
Phone:403-424-4922

---

From: Randall Gellens <[email-masked]>
Date: Fri, 13 Nov 1998 11:47:36 -0800

OK, you all have convinced me.  I think the spec should be silent on 
Resent-* headers.

---

From: Tim Showalter <[email-masked]>
Date: 17 Nov 1998 15:47:24 -0500

> Date: Thu, 12 Nov 1998 22:05:54 -0800 (PST)
> From: Ned Freed <[email-masked]>

> My personal take is that short-circuit evaluation is useful enough to
> warrant a MUST. But I'd rather have a statement saying scripts MUST
> NOT take advantage of short-circuit evaluation than one saying
> implementations MAY do it.

Okay, scripts can't currently take advantage of short-circuit
evaluation.

If an extension exists that makes it possible to use short-circuit
evaluation, could we have that extension change this assumption?

---

From: Tim Showalter <[email-masked]>
Date: 17 Nov 1998 15:50:53 -0500

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 18 Nov 1998 00:36:19 +0300

> evaluation, could we have that extension change this assumption?

I think yes. Why not?

--
Regards,
Alexey Melnikov
------------------------------------------
SMTP/POP3/IMAP4/ACAP servers creation team

Epsylon Technologies, Russia
 http://www.taxxi.com

Imap Development Kit (my own product)
http://194.87.43.111/homerus/mail/idk/index.htm

Fax (in San Diego, California): 1 (619) 8393837
------------------------------------------

---

From: Jack De Winter <[email-masked]>
Date: Tue, 17 Nov 1998 17:59:49 -0500

>> evaluation, could we have that extension change this assumption?

Well, one example of why not would be the following example:

if( check1 AND check2 ) {

(just to use some logical components that people are somewhat
familiar with, excuse the non-SIEVE grammar, dont have the spec
near me at the moment)

If a given UI understands that shortcutting is supported, then all
is moot.  If a given script parser/evaulator does not support it, then
all is moot.  What is left is the case of a p/e that supports it through
a given extension (or for that matter, a set of extensions that all
require it), and a UI that doesnt understand shortcutting, or at least
when the p/e will use it.

If all check1 and check2 ever are are comparisons (is a message
greater than X bytes and not from someone on my personal list), then
there is no problem.

The problem occurs if a subsequent extension to SIEVE would allow for
a comparison operation to modify something as a result.  An example would
be:

if( check1 AND keyring( "foobar" )) {

in this case, I am constructing a mythical extenstion called keyring that
allows the script to check to see if it is a PGP signed message, and if
it isnt on my personal keyring, to add it to a keyring "foobar" for
later inspection.

In this case, it is important to know if shortcutting is in place, as
if it is, keyring will never be executed if check1 is FALSE.  If we
always want keyring to be evaluated, we would have to put it first if
shortcutting is in place. Otherwise, if shortcutting is not in place,
we know everything will be evaluated, and in this case, any storage
actions of keyring will be performed.

Now, I have given a possible case "keyring" of something that checks for
PGP signed messages and keeps them on an arbitrary keyring (for later
inspection) if not already on a keyring.  This is something I would
myself consider possible and probable.

Are there other cases of extensions people are thinking of where part
of an expression may have an action taking place within it that
shortcutting evaluation would have an impact on?

regards,
Jack

---

From: Ned Freed <[email-masked]>
Date: Wed, 18 Nov 1998 00:20:57 -0800 (PST)

> evaluation, could we have that extension change this assumption?

I suppose so in theory, but I really don't much care for the idea. 

I'm sorry, but if short circuit evaluation is useful then it should be a
requirement, and if not it should not even be an extension. I don't buy the
argument that this is problematic because some languages don't support
short-circuit evaluation. Just because you don't have short-circuit in some
language doesn't mean that you cannot write a sieve interpreter in that
language that does have short circuit characteristics. The two are largely
unrelated unless you're using a direct translation approach, in which case
other constraints are likely to be a lot more onerous.

Indeed, my sieve implementation is a case in point -- it does short circuit
evaluation and it is written in a language that doesn't itself support the
concept. And my using a language with this concept would not have altered the
implementation task in any significant way.

				Ned

---

From: Tim Showalter <[email-masked]>
Date: 18 Nov 1998 13:40:37 -0500

> Date: Wed, 18 Nov 1998 00:20:57 -0800 (PST)

> requirement, and if not it should not even be an extension.

The problem I have is that with the existing language, you can't tell
the difference between an implementation that doesn't suppot
short-circuit evaluation and one that does by its behavior, so I don't
want to specify it.

> I don't buy the argument that this is problematic because some
> languages don't support short-circuit evaluation. [...]

Just for the record, I buy this.

If someone could suggest real wording for this, I'd be happy to take it
and just get it over with.  (And real soon, too.)

---

From: Randall Gellens <[email-masked]>
Date: Fri, 20 Nov 1998 17:02:48 -0800

> other constraints are likely to be a lot more onerous.

True.  I withdraw my concern regarding mandatory short circuit 
evaluation.  (After reading Ned's message I realized that my own 
implementation would still support short-circuit evaluations even if 
the underlying language didn't.)

How about something like:

ANYOF evaluation MUST stop as soon as one element evaluates to TRUE. 
That is, Sieve implementations are REQUIRED to support short-circuit 
evaluations.

(With the same for ALLOF, except TRUE replaced by FALSE.)

---

From: Ned Freed <[email-masked]>
Date: Fri, 20 Nov 1998 18:59:28 -0800 (PST)

> (With the same for ALLOF, except TRUE replaced by FALSE.)

Sounds fine to me.

				Ned