Subject: List of expected changes for sieve 05 draft

From: Tim Showalter <[email-masked]>
Date: 6 Nov 1998 16:40:13 -0500

I hope to have an 05 draft out real soon now.

Here are the list of things that I know of (and remember) that have been
requested as changes in the next draft.  They're marked as follows:

* "Consensus" indicates something that I believe is consensus, and will change.
* "Bug" indicates something that's just wrong.
* "Request" indicates something that someone has asked for, and will be
  changed if I get confirmation that it's consensus.
* "Question" indicates something that I'm not sure how I'm supposed to word
  it, and would appreciate some more information.

1.  Concensus: Short circuit evaluation is a SHOULD, not a MUST.

2.  Question: Should short-circuit evaluation be as is, a MAY, or not
    discussed at all?

3.  Concensus: i;ascii-casemap is required.

4.  Question: Should i;ascii-casemap be the default?

5.  Consensus: Fileinto is optional; discussion of local mail folders and POP3 
    should be removed.

6.  Request: A non-present comparator is considered to be basically a syntax
    error.

7.  Request: Implementations are required to decode header charsets.

8.  Bug: Section 7, remove reference to "support" and "reply".

9.  Bug: Remove all XXXs (section 2.3, 2.11.1, for instance) and fix all
    examples.

10. Bug: Header name compares are always case-insensitive.
     
11. Bug: 4.2: example is incorrect.  ("From" should be "To"; contains should
    be :contains)

12. Consensus: What happens when an error occurs should be dropped; it should
    not be part of the specification, and is just a quality-of-implementation
    issue.

13. Request: Reject should be optional.

14. Consensus: There should be two namespaces for extension names, one "vnd.", 
    one everything else, just like several other protocols.

I am way overdue for posting the reasons that I made some of the changes to
the draft.  Here they are.  (Some, not all, of these were made in the long
delay between 03 and 04, and were ok with the circa 03 list participants and
not okay with the influx of people after 04.)

1.  "Matches" should be just another comparator, in my opinion.
    (This allows easy addition of regular expressions or globs.)
    I'm not real attached to this.

2.  Tagged arguments: Some time ago, tagged arguments were proposed and
    accepted.  I strongly prefer them because they allow keywords to be taken
    out of the grammar, reducing the number of special cases, and allowing
    extensions to be added without changes to the grammar.

2a. Can we keep tagged arguments?

2b. Arbitrarily, I allowed tagged arguments to be in any order and freely
    interspersed in commands.  This was probably a mistake, but I wanted to
    preserve wordings like this:

       header "From" :contains "tjs"

    What do people want?  The dominant stated opinion is that they should only 
    happen after the keyword, and that's fine with me.

3.  Elsif was added because it removes an ambiguity in the grammar, making
    "else" yet another command.  This is discussed in anything that talks
    about writing C parsers; "else if" is an imbiguity, and many languages
    have an "elsif" as a result.  I have a fairly strong preference for this.

    Note that "else if" is illegal in 04.

4.  The formal grammar was a major change from 03, and I have strong
    reservations against changing it back.  03 had a grammar that was
    over-specified, and under-permissive.  Every extension required a grammar
    change; this seems likely to lead to a horrible grammar when all is said
    and done.

    This has probably left considerable ambiguity in some other places in the
    draft, but I don't beleive that's a good reason to switch back to the
    previous grammar.

-- 
Tim Showalter <[email-masked]>

---

From: "Barry Leiba" <[email-masked]>
Date: Fri, 6 Nov 1998 21:56:26 -0500

> 4.  Question: Should i;ascii-casemap be the default?

I vote for that.

> 7.  Request: Implementations are required to decode header charsets.

Since these requests came from comments by me, I should state clearly
that I agree with these two.

> 13. Request: Reject should be optional.

At one level I agree with this, but at another level (and this applies
to optional "fileinto" also) I think that making the basic features 
optional hurts interopreability -- that is, it hurts portability of
scripts.  That said, I understand *why* people want to make "fileinto"
and "reject" optional.  I want to be able to test "envelope-sender"
too, and we have that in our implementation... but I understand why
people don't want it as a required function.

> 1.  "Matches" should be just another comparator, in my opinion.

Sort of.  The problem with that is that it introduces a whole set
of comparators: matches-ascii, matches-ascii-casemap, and so on,
one for each comparator that's used for "contains".  In some sense,
though, it's not a problem: "contains xxx" is basically just
"matches *xxx*", but for the fact that "contains x*x" is *not*
the same as "matches *x*x*".  If we put in an escape character for
the wildcards, then we can get rid of "contains" and "is", and 
just use "matches" for everything:
   :is "abc" ...becomes... :matches "abc"
   :contains "abc" ...becomes... :matches "*abc*"
   :is "a*c" ...becomes... :matches "a\*c"
   :contains "a*c" ...becomes... :matches "*a\*c*"

I guess I'm not really stating much of an opinion on this; I think
it's fine the way it is, with "is", "contains", and "matches".  But
if we're going to make a change here, I suppose we should think about
this.

> 2a. Can we keep tagged arguments?

Sure, why not?

> 2b. Arbitrarily, I allowed tagged arguments to be in any order and freely
...
>     What do people want?  The dominant stated opinion is that they should 
>     only happen after the keyword, and that's fine with me.

Yes, I'm of that opinion.

Barry Leiba, Multimedia Messaging  ([email-masked])
http://www.research.ibm.com/people/l/leiba

---

From: Chris Newman <[email-masked]>
Date: Mon, 09 Nov 1998 15:52:27 -0800 (PST)

> 2.  Question: Should short-circuit evaluation be as is, a MAY, or not

Short-circuit evaluation should either be a MUST or a MAY.  It needs to be
mentioned, so people know whether or not they can rely on it when writing
code.

> 3.  Concensus: i;ascii-casemap is required.

Yup.

> 4.  Question: Should i;ascii-casemap be the default?

I'm not sure.  How does this interact with the local-part being
case-sensitive and the domain-part being case-insensitive?  Could it lead
to confusion?

> 6.  Request: A non-present comparator is considered to be basically a syntax

Sounds reasonable.

> 7.  Request: Implementations are required to decode header charsets.

I could live with this.  A lighter-weight alternative would be:
Implementations are required to (a) treat all non-ASCII characters in a
script as a syntax error or (b) decode MIME header encoding.  That's the
"you don't have to do it, but if you do it, do it right" approach.

> 10. Bug: Header name compares are always case-insensitive.

Why is that a bug?  Header names are US-ASCII case insensitive in RFC 822.

		- Chris

---

From: Tim Showalter <[email-masked]>
Date: 9 Nov 1998 22:34:20 -0500

> Date: Mon, 09 Nov 1998 15:52:27 -0800 (PST)
> From: Chris Newman <[email-masked]>

> case-sensitive and the domain-part being case-insensitive?  Could it lead

Hm.  Perhaps we should just canonicalize all domain names to
(lower/upper)case for compares, then provide primitives (i.e., not
"header") for looking at to/from/cc like IMAP does for looking at parsed
address headers.  (That's simpler than anything else I can think of.)
Then we can make casemap the default.

Making i;ascii the default is more or less a lose for similar arguments,
so I'd rather not do that.

> Why is that a bug?  Header names are US-ASCII case insensitive in RFC 822.

Er, that's not necessarily what the draft says, but it should be.  I
don't think it's clear int he draft.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 10 Nov 1998 10:21:56 +0100

>"you don't have to do it, but if you do it, do it right" approach.

Decoding to what?

3 possibilities:

- Decode to octet string (simple, but dangerous)
- Decode to UTF-8 (requires universal charset translation)
- Decode if you recognize the charset, otherwise leave alone, which
  leaves a *slight* incompatibility problem with server upgrades.
 
I'm fine with the decoding being to an octet string, and the expected
charset being implicit in the comparator function, but one needs
to *somehow* get access to the charset name(s?) to be able to detect the
case where "shit happens".
Something like this:

  if subject contains "I ordered a Räksmörgås" then
      if matched charset is iso-8859-1 then
          do something
      else
          don't dare to do something
      fi
  fi

Syntax wildly inventive...haven't read -04, I'm afraid....

Note that Räksmörgås will be represented in UTF-8 in the script while
the script is being moved around. UTF-8 can't represent the octets
of 8859-1 without an escaping mechanism, and a layman user would go
bonkers if asked to use one.

And simply declaring matching on anything that isn't English illegal
forever is Not An Option.

We have a problem.

(Räksmörgås is Swedish for a shrimp open-faced sandwich; popular for
testing because it contains the 3 most important special Swedish letters...)

                     Harald

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

---

From: Chris Newman <[email-masked]>
Date: Tue, 10 Nov 1998 09:53:30 -0800 (PST)

There are three issues to address:

(1) Which charsets are permitted in the scripting language (currently
only UTF-8).

(2) Which charsets constitute minimal support for RFC 2049 in incoming
message headers.

(3) What to do when charsets from (1) & (2) mismatch.

I have a strong preference to keep (1) unchanged -- UTF-8 only.  Allowing
a script to contain embedded data in multiple charsets makes script
viewing and composition _much_ more complex as well as making the
cross-charset comparison problem even worse.

As for (2), I'd say if RFC 2049 decoding is done, support for UTF-8,
ISO-8859-1, and the ASCII subset of the ISO-8859-* charsets should be the
minimum required.  UTF-8 is required by RFC 2277 and is easy since it
matches the charset for the scripting language.  ISO-8859-1 is easy since
it's a proper subset of UTF-8.  And the other rule (at least the ASCII
subset of ISO-8859-*) comes directly from RFC 2049. 

In the interest of getting Sieve deployed faster, it may be desirable to
permit implementations which don't support RFC 2049 to be compliant
(possibly under a SHOULD support RFC 2049 clause).  When RFC 2049 isn't
supported, I'd say that a comparision string in the script with 8-bit
content MUST fail to match (we have to hold the line on just-send-8 in
headers now so we can allow UTF-8 in headers down the road).

(3) is nasty and creates the behavior change you noted as servers are
upgraded to support more charsets.  As long as (1) is fixed at UTF-8, that
effectively requires translation to UTF-8 to do comparisons.

The alternative would be to allow scripts to embed octet strings labelled
with a charset for comparison purposes.  While that would make Japanese or
Chinese localization easier, it makes the international problem harder in
addition to the other drawbacks mentioned above.

		- Chris

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 10 Nov 1998 20:50:03 +0100

>cross-charset comparison problem even worse.

Agreed.

>subset of ISO-8859-*) comes directly from RFC 2049. 

Makes sense.

>headers now so we can allow UTF-8 in headers down the road).

Agreeed.
Do you have anything like a "feature test macro" in the language now?
I'm thinking of something like

   require charset iso-2022-jp

to let a script say that unless the server is able to decode this
charset into UTF-8 for comparision, the script should go Boink at once
instead of behaving randomly.

>addition to the other drawbacks mentioned above.

OK, let's require that "all actions have the same result as if all
character data was converted to UTF-8 before comparisions are done".
And 2049 strings in unknown charsets don't match anything at all.

Seems to make sense to me.

                  Harald

---

From: Tim Showalter <[email-masked]>
Date: 10 Nov 1998 16:29:13 -0500

> From: "Barry Leiba" <[email-masked]>
> Date: Fri, 6 Nov 1998 21:56:26 -0500

> and "reject" optional.

Reject, fileinto, and envelope-sender commands all have problems that
not all systems can actually implement them, so I think we're pretty
much screwed here, although perhaps reject and envelope-sender need to
be added to the draft.

> I want to be able to test "envelope-sender" too, and we have that in
> our implementation...

What does the command look like?  If you've got a working syntax, we
might as well use it.

> one for each comparator that's used for "contains".

Okay.  I withdraw this, and will put :matches back into the next draft.
That's clearly better than having comparators all over the place.

I think that's probably the general opinion, too.

I think having :changes, :is, and :matches is useful, but we could do it 
all with :matches if people really wanted to.

> > 2a. Can we keep tagged arguments?

There was a complaint about the change from "is" to ":is", but I'm
rather attached to :is for reasons I stated.  

> Yes, I'm of that opinion.

Ok, I'll plan on changing things this way.

---

From: Randall Gellens <[email-masked]>
Date: Tue, 10 Nov 1998 15:30:04 -0800

> What does the command look like?  If you've got a working syntax, we

My Sieve implementation uses "envelope.rcpt" and "envelope.from" as tests.

I've also implemented "Mark" as an action which adds a header to the 
message; "^f" and "^r" are expanded to the envelope return-path and 
recipient, respectively.

For example (from my test Sieve script):

	if envelope.rcpt contains "+"
		mark "x-detail: ^r";

---

From: Tim Showalter <[email-masked]>
Date: 11 Nov 1998 00:36:56 -0500

> Date: Tue, 10 Nov 1998 15:30:04 -0800
> From: Randall Gellens <[email-masked]>
 
> My Sieve implementation uses "envelope.rcpt" and "envelope.from" as tests.

Do we need to add . to the list of legal characters in an identifier?
I'd rather not.

> 	if envelope.rcpt contains "+"

Ok.  So what does x-detail signify?

Tim

---

From: "Barry Leiba" <[email-masked]>
Date: Wed, 11 Nov 1998 13:20:35 -0500

> What does the command look like?  If you've got a working syntax, we

} elseif  envelopesender :contains :comparator "i;ascii-casemap" ["ietf-sasl"]  {

I actually like Randy's "envelope.rcpt" and "envelope.from".  If you want
to avoid the ".", we could make it like "header":

} elseif envelope ["from"] :contains :comparator "i;ascii-casemap" ["ietf-sasl"]  {

...which also allows '["from", "rcpt"]' for more flexibility.

---

From: Randall Gellens <[email-masked]>
Date: Wed, 11 Nov 1998 11:42:39 -0800

> ...which also allows '["from", "rcpt"]' for more flexibility.

I like using the same form for envelope and header.

---

From: Ned Freed <[email-masked]>
Date: Wed, 11 Nov 1998 11:47:53 -0800 (PST)

> ...which also allows '["from", "rcpt"]' for more flexibility.

What a great idea! I really like this approach, even though it isn't what
I implemented.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 11 Nov 1998 23:34:07 +0300

> 2.  Question: Should short-circuit evaluation be as is, a MAY, or not

I think that short-circuit evaluation should be MUST (may be some future extensions
will depend on it) or not discussed at all.

> so I'd rather not do that.

Adding primitives for dealing with addresses seems good.

Then i;ascii-casemap should be the default.

Sounds reasonable.

> 13. Request: Reject should be optional.

Will it stay in the main document?

> 2a. Can we keep tagged arguments?

No doubt.

>     happen after the keyword, and that's fine with me.

I prefer to have tagged arguments only after keyword.

Regards,
Alexey Melnikov
------------------------------------------
SMTP/POP3/IMAP4/ACAP servers creation team

Epsylon Technologies, Russia
 http://www.taxxi.com

Imap Development Kit (my own product)
http://194.87.43.111/homerus/mail/idk/index.htm

Fax (San Diego, California): 16198393837
------------------------------------------

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 11 Nov 1998 23:45:04 +0300

> What a great idea! I really like this approach, even though it isn't what

I like this approach too, because future extensions may add different components to
envelope without changing grammar.

BTW, I think that operations with envelope are quite frequent, so they should be
standardized.

Cheers,
Alexey Melnikov

Fax (in San Diego, California): 1 (619) 8393837
------------------------------------------

---

From: Tim Showalter <[email-masked]>
Date: 18 Nov 1998 13:54:44 -0500

> Date: Wed, 11 Nov 1998 11:47:53 -0800 (PST)
> From: Ned Freed <[email-masked]>

> What a great idea! I really like this approach, even though it isn't what

I'm writing this now and it occurs to me that it might be clearer if
that's "from" and "to" instead of "from" and "rcpt".

This is a nice approach, though.

---

From: "Barry Leiba" <[email-masked]>
Date: Wed, 18 Nov 1998 14:33:25 -0500

Agreed.

---

From: Tony Hansen <[email-masked]>
Date: Wed, 18 Nov 1998 14:39:49 -0500

> This is a nice approach, though.

How about "mail-from" and "rcpt-to" to make them more consistent with
the smtp envelope verbs?

	Tony Hansen

---

From: Tim Showalter <[email-masked]>
Date: 18 Nov 1998 15:32:25 -0500

> Date: Wed, 18 Nov 1998 14:39:49 -0500
> From: Tony Hansen <[email-masked]>

> How about "mail-from" and "rcpt-to" to make them more consistent with

I don't see a lot of merit in making them similar to the SMTP verbs.
Even really clueful users (ones capable and willing to write raw Sieve
scripts) don't know much about SMTP, and making them type "rcpt" doesn't 
seem like the right thing to me.

---

From: Matthew Wall <[email-masked]>
Date: Wed, 18 Nov 1998 17:19:01 -0400

the entity known as Tim Showalter <[email-masked]> wrote:

> I don't see a lot of merit in making them similar to the SMTP verbs

well, except there is virtue in being consistent with _something_, and one
might make the point that the SMTP-savvy are going to be well-represented
among those looking at the syntax.

Still, I'd prefer 'from' and 'to' for simplicity's sake, even if it's
technically a bit less explicit. Why bother typing the extra chars, eh?

- matt <[email-masked]>

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 19 Nov 1998 01:13:52 +0300

> How about "mail-from" and "rcpt-to" to make them more consistent with

IMHO, this is better, because it is less confusing.

--
Best Regads,
Alexey Melnikov

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 18 Nov 1998 15:23:23 -0700

> seem like the right thing to me.

Well, this (hopefully :-) clueful user finds mail-from and rcpt-to to
be much more intiutive than the alternatives.

--lyndon

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 19 Nov 1998 01:25:38 +0300

> - matt <[email-masked]>

Why not just use "f" and "t". They are even shorter :-)

---

From: Ned Freed <[email-masked]>
Date: Wed, 18 Nov 1998 18:27:24 -0800 (PST)

> How about "mail-from" and "rcpt-to" to make them more consistent with

To my mind this makes things less consistent, not more. MAIL FROM can have many
parameters, the FROM address being just one of them. And RCPT TO has, if
anything, even more. So to me mail-from identifies a bunch of stuff and so does
rcpt-to.

But since the set of parameters to MAIL FROM and RCPT TO is currently disjoint,
and is likely to remain that way, it seems more consistent to me to go by
parameter name, with the "special" parameter names FROM and TO for the
special, initial parameters.

I also _really_ don't like the use of minus in these names. I thought this came
up some time ago and it was decided that we wouldn't use minus in names. Of
course this can simply be cured by using underscores instead.

				Ned