Subject: NULL vs. ""

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 25 May 2005 21:42:43 +0200

Hi,

in a sieve script (with variable and/or other extensions), can you think 
of a use where a null string works differently from an empty string, in 
a useful way?

One relevant null string could be the value of an absent header field. 
The first of these messages has a null subject, the second an empty 
subject.

---
From: [email-masked]
Date: 24 May 2005 19:52:09 +0100

Body text here
---
From: [email-masked]
Subject:
Date: 24 May 2005 19:52:09 +0100

Body text here
---

I know it's possible to distinguish by looking at whether there is a 
header field at all. But that won't do. I need an example showing how 
differentiating the values of those two header fields could be useful 
and desirable.

If no such example exists, and I've tried really hard to find it, that's 
fine too ;) Then I can say that NULL and "" should be the same.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 27 May 2005 10:47:21 +0200

NULL and "" should be the same.

Arnt

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 27 May 2005 13:29:58 +0200

> NULL and "" should be the same.

by NULL I assume you mean "undefined" or "not present".

it's hard to make a general comment, but it's clear that is not
generally true in Sieve today.  are you proposing to change the
semantics of the header test?

   If a header listed in the header-names argument exists, it contains
   the null key ("").  However, if the named header is not present, it
   does not contain the null key.  So if a message contained the header

           X-Caffeine: C8H10N4O2

   these tests on that header evaluate as follows:

           header :is ["X-Caffeine"] [""]         => false
           header :contains ["X-Caffeine"] [""]   => true

on the other hand, in the variables extension it is impossible to
distunguish between an unset variable and a variable set to "".
-- 
Kjetil T.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 27 May 2005 14:49:46 +0200

> by NULL I assume you mean "undefined" or "not present".

I assume so too, but it doesn't necessarily have to be so ;)

If you'll look at draft-newman-comparator (currently expired) you'll see 
that "NULL strings" are mentioned in section 4.6, the behaviour of 
comparing, equality testing and substring testing on NULL strings is 
defined, but a NULL string itself is defined only implicitly as "a 
string that contains no characters and is not the same as an empty 
string".

In the context of sieve, that would seem to encompass the value of an 
absent header field, the value of a variable that is initialized from 
such a header field, etc.

> semantics of the header test?

I think I'm not.

I agree it's not generally true, but I'm not sure whether it's completely true.

I want to either kill NULL strings in the comparator draft or provide a 
raison d'etre in the form of a good example. Either is fine with me. 
And I want to be compatible with the established use of comparators.

>    the null key ("").  However, if the named header is not present, 
>    it does not contain the null key.  So if a message contained the 
>    header
>
>            X-Caffeine: C8H10N4O2
>

There are no null strings here, only empty strings, so I don't think 
it's relevant.

> distunguish between an unset variable and a variable set to "".

And that sounds very much in tune with sieve...

To me it sounds as if requiring a distinction between NULL (whatever it 
is) and "" is a little too much of an imposition. Better to kill the 
NULL concept in comparators.

Arnt

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 27 May 2005 16:05:40 +0200

> string that contains no characters and is not the same as an empty 

aha, thanks for the pointer for where you're coming from.

> such a header field, etc.

yes.

> > are you proposing to change the semantics of the header test?
> 
> I think I'm not.
> 

the example wasn't very relevant here, since the header exists.
consider instead:

   header :contains "X-Absent" ""

the LHS of the comparison is the value of lookup_header("X-Absent"), and
it returns a NULL value (do not confuse with "null key" :-).  it seems
to me that the comparator draft needs to support NULL values to be able
to support Sieve semantics.
-- 
Kjetil T.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 27 May 2005 16:28:11 +0200

> the example wasn't very relevant here, since the header exists.

And since "" is a empty string, not an NULL string. The comparator text 
states that NULL strings aren't substrings of anything, so :contains 
["X-Caffeine"] [:null] should evaluate to false (if :null were valid).

> the LHS of the comparison is the value of lookup_header("X-Absent"), 
> and it returns a NULL value (do not confuse with "null key" :-).  it 
> seems to me that the comparator draft needs to support NULL values to 
> be able to support Sieve semantics.

Well, can you show some example where it actually makes a difference, 
and where the difference is useful? It has to be some collation 
operation where it's sensible and understandable that NULL (TBD) is 
distinct from "". The draft offers two properties that seem relevant to 
sieve:

1. NULL is equal only to NULL. The example could use :is.

2. NULL is not a substring of anything. The example could rely on 
:contains to return false if the second argument is NULL.

I can't construct such an example, and I really don't want to take the 
draft forward with a feature that's incompatible with some running code 
and whose utility I cannot explain.

Arnt

---

From: Philip Guenther <[email-masked]>
Date: Fri, 27 May 2005 12:32:46 -0700

>["X-Caffeine"] [:null] should evaluate to false (if :null were valid).

Right.  To avoid confusion going forward, I've updated the base-spec to
refer to the zero-length string ("") as "empty" instead of "null".

Hmm, perhaps the third text paragraph of section 5.7 ("Test Header")
should say that if a named header is not present, then its value is the
NULL string (...with verbiage to keep people from confusing that with
the empty string).  Indeed, the behavior of matches against missing
headers could be construed as underspecified currently.  Sure, a missing
header doesn't contain the empty string, but does is compare equal to
zero under the i;ascii-numeric comparator?

>:contains to return false if the second argument is NULL.

Neither of these can be expressed in sieve, as sieve has no way to
express a NULL second argument to a comparison.  The reverse of the
first of those ("NULL !:is non-NULL") is arguably used when the header
test is given a header name that isn't present.

(I can also imagine an implementation that treated
	address :domain "To" "example.com"
as using a NULL string for the domain when the address in the To field
was syntactically invalid by lack of a domain part, ala:
	To: John Smith <smith>
...but how invalid inputs are handled is beyond the scope of the spec.)

>and whose utility I cannot explain.

The alternative is for every application that uses collations and
combines the comparison with a key lookup (i.e., anything that rolls an
email header value lookup into the operation) to describe how the "key
not found" case is handled.  The -03 rev of the collation registry draft
let both application protocols and collations override the default
handling of NULL strings; dropping them would effectively make that
_just_ application specified.

Philip Guenther

---

From: Tim Showalter <[email-masked]>
Date: Fri, 27 May 2005 13:02:01 -0700

> ["X-Caffeine"] [:null] should evaluate to false (if :null were valid).

I deeply regret using the words "null string" and apologize for my 
ignorance.

"null" should be changed to "empty" everywhere in 3028bis, and we can 
avoid this confusion.

I'm not sure what a NULL string means in the context of Sieve, 822, or 
at all, actually.

Tim

---

From: Ned Freed <[email-masked]>
Date: Fri, 27 May 2005 13:34:51 -0700 (PDT)

> I'm not sure what a NULL string means in the context of Sieve, 822, or

It's nice to know I'm not the only one who is confused by all this.

				Ned

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 30 May 2005 14:31:47 +0100

> I'm not sure what a NULL string means in the context of Sieve, 822, or at 
> all, actually.

I suppose we are saying it's the value of an absent header, not just an 
empty header.

The discussion reminds me of when I was trying to write gui to say "when 
header X is empty" and then was surprised how many iterations of script I 
went through until I got exactly what I wanted.  These were both wrong in 
terms of the concept I was trying to communicate to users:

if header :is "Cc" ""  // ie if there is a header that is blank
if not exists "Cc"        // ie if the header does not exist

I eventually ended up with:

if not header :regex "Cc" "."    // If there is no Cc header that contains 
any character

Which felt pretty clumsy for what seems to be a pretty obvious use for 
Sieve.  It also requires regex, which isn't necesarily available.  The 
alternative being:

if anyof (header :is "Cc" "", not exists "Cc")

But a NULL string concept doesn't really help much here either, as it 
basically offers the same as what we have with the exists test, but it just 
allows you do use variables instead of the exists test directly.  Perhaps 
most of the time you would actually want to treat absent headers the same as 
empty headers in terms of processing, and if you really wanted to make the 
distinction then you could just use exists?

What might be nice is if we could say:

if header :is "Cc" NULL  // ie if the header is absent or empty

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 30 May 2005 17:42:24 +0200

On Mon, 2005-05-30 at 14:31 +0100, Nigel Swinson wrote:
> header X is empty" [...] I eventually ended up with:
> 
> Sieve.

can you expound a little on this obvious use?  I don't see the use case,
myself, and am therefore skeptical to the need for a new reserved word
NULL.
-- 
Kjetil T.

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 30 May 2005 17:10:18 +0100

> myself, and am therefore skeptical to the need for a new reserved word

Well the user is trying to say "where there is no information in XXXX 
headers".  They don't care if the the header is absent, or is empty, they 
just care that it contains no data and want to switch on both in the same 
manner.  So I could imagine script wanting to say:

"If there are no Cc addresses"
"If there is no subject"
"If there is no spam score header"
"If there is no virus score header information"
"If the priority of the message is not specified"

Headers with no value to them sound pretty useless, but of course some 
headers are required to be there to meet the standards, and a header with no 
value isn't prohibted by the standards, so a sensible filter really should 
accommodate them.  It feels as though sieve as it stands leaves the user to 
write a couple of faulty scripts before they finally write their anyof(not 
exists, is empty) test.

Of course this issue can be abstracted away and solved in the gui, I just 
thought it was an issue worth bringing up in the context of this NULL/"" 
discussion, and wondered if it was worth us providing an easier solution for 
writing these filters.

Nigel

---

From: Ned Freed <[email-masked]>
Date: Mon, 30 May 2005 09:21:27 -0700 (PDT)

> Sieve.  It also requires regex, which isn't necesarily available.

Regex isn't necessary here. While I fail to see a use case for this particular
test, the first way to code this as a single test that came to mind was:

  if not header :matches "Cc" "*?*

Seems very straightforward to me.

> Thealternative being:

> if anyof (header :is "Cc" "", not exists "Cc")

This is actually clearer to read IMO, and is probably how I would code it if
I ever wanted such a test.

> if header :is "Cc" NULL  // ie if the header is absent or empty

The semantics of this are not at all obvious from inspection. Given that it is
both unnecessary and IMO much less readable, I see no justificatio for adding
it and lots of justification for keeping it out.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Mon, 30 May 2005 09:29:50 -0700 (PDT)

> headers".

Careful here. The simple presence of a field can be used to convey information.
Bcc: is the obvious example - now that recipient fields are optional it can be
used to indicate bcc copies were sent. RFC 2822 calls out this usage
specifically, as a matter of fact. I'm not sure I would ever write
a script that tested bcc this way, however, given how many transport
agents summarily (and incorrectly) drop bcc fields from messages in transport.

I have also seen an empty keywords field used to indicate "we checked for
applicable keywords but didn't find any", although RFC 2822 appears to disallow
this usage.

> They don't care if the the header is absent, or is empty, they

> Headers with no value to them sound pretty useless,

Not necessarily. See above.

> but of course some

I'm again failing to see the real use case. Of course you can write users want
this or that behavior, but wanting something doesn't mean it is actually
useful. About the closest thing here is the "no subject" test you cite - I
could sort of see wanting to treat an empty subject the same way as as no
subject. Recipient field tests that go beyond things testing for explicit
presence of a particular address have always struck me as extremely risky given
the vagarities of how clients construct these fields and how they get mangled
by the transport infrastructure. As for score tests, this is probably a case
where you want to distinguish between no field and an invalid field (an empty
field is presumably invalid).

> discussion, and wondered if it was worth us providing an easier solution for

IMO the proposed solution adds complexity without adding value.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 30 May 2005 18:55:47 +0200

> I ever wanted such a test.

the alternative would be to redefine the semantics of header :is so that
header :is "Cc" "" is true even when Cc is non-existent.  those who
wishes to check for existance still have the exists test.  this shifts
the need for "awkward" code over to those who write careful code in the
first place.

I'm not sure if such a change to Sieve can be made without the script
explicitly asking for it.  it seems pretty innocuous to me, but the
gains are pretty slim, too.
-- 
Kjetil T.

---

From: Ned Freed <[email-masked]>
Date: Mon, 30 May 2005 10:17:27 -0700 (PDT)

> the need for "awkward" code over to those who write careful code in the

it is much too late to make such a change without it being a new capability.

> gains are pretty slim, too.

One way you complicate the test for an empty field. The other way you
complicate the test for a missing-or-empty field. The use cases for either one
are not particularly compelling, and given the large cost associated with each
new capability, leaving things alone is the obvious right answer.

				Ned

---

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 31 May 2005 07:21:13 -0400

> if anyof (header :is "Cc" "", not exists "Cc")

Given that the language definition has no way to express a NULL, I don't
see that as a bad way to state the test.  It's clearer to me than the
regex solution (and the equivalent :matches solution).

> if header :is "Cc" NULL  // ie if the header is absent or empty

really wouldn't work either.  A NULL would still have to be different
from the empty string, so you'd have to say:

    if header :is "Cc" [ NULL, "" ]  // if the header is absent or empty

which is indeed slightly tighter and clearer than the 'anyof' test.
Personally I'm not against having a NULL reserved keyword that
represents a missing string value, but it would probably involve a lot
of changes to existing implementations.  I doubt it would be disruptive
to existing sieve scripts though-- only to implementations, thus it
would have to be done via an extension and not a 3028bis addition, and
probably would be quite a bit of an uphill battle.

I think a NULL test of some sort could be useful with variables,
but that's another road that's already been paved...

mm

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 31 May 2005 15:57:21 +0200

> but that's another road that's already been paved...

not sure what that expression means, but the document hasn't been
approved yet, so changes can still happen.

the test introduced with variables is "string", so there is no way to
access the value of a variable except inside a string.  likewise for
setting a variable, you (currently) have to provide a string.  having
"${unset_variable}" extrapolate to a special value rather than a string
is IMO ugly and counter-intuitive.

to support NULL values with variables, I see two alternatives:

        a) a new test which takes a variable _name_ as its argument.  we
        also need a NULL token which can be used with that test and with
        SET.
        
        b) a new test to check explicitly for unset values.  a new
        action UNSET which can only set variables to the NULL value.

I don't really like a NULL/UNSET token, it opens to many issues.  I
wouldn't mind adding alternative b) to the spec, though, it may be
useful in some cases.
-- 
Kjetil T.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 31 May 2005 16:16:42 +0200

Kjetil Torgrim Homme writes:
> I wouldn't mind adding alternative b) to the spec, though, it may be 
> useful in some cases.

I'd say no. Noone has asked for it yet.

Nigel needed something to do with emptiness or nullness, but I didn't 
really understand what. Doesn't seem as though he wanted b), though.

Arnt

---

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 31 May 2005 10:32:42 -0400

> approved yet, so changes can still happen.

Nothing sinister..  just an acknowledgement that people would like to
get on with finalizing that draft and not enter into new tangents in it.

> is IMO ugly and counter-intuitive.

Probably so.  There could conceivably be different kinds of syntaxes to
do different kinds of expansions, though; I think that notion has been
raised before.  e.g. one could borrow from 'sh' syntax, which has:

    ${var:alt}

where the text "alt" is used if 'var' does not exist as a variable.
Or some other kind of expansion controls, e.g. other tagged options
in "set" (along the lines of :length).  

> wouldn't mind adding alternative b) to the spec, though, it may be

Seems rational to me, but then again there's that thing about delaying
the draft with more reviews.

mm

---

From: Ned Freed <[email-masked]>
Date: Tue, 31 May 2005 08:23:33 -0700 (PDT)

> approved yet, so changes can still happen.

The variables draft has been in process for over two years now. It's time to
wrap it up and get it out the door.

        

> wouldn't mind adding alternative b) to the spec, though, it may be

Exactly what urgently-need funcitonality does any of this provide? I sure
don't see any.

				Ned

P.S. Yes, I realize the vacation draft is even more overdue. I plan to send out
one more revision and then ask for WGLC in the next week. It would be good
if variables could join it.

Cc: Mark E Mallett <[email-masked]>, [email-masked]
Date: Tue, 31 May 2005 08:23:33 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NULL vs. ""
To: Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

> approved yet, so changes can still happen.

The variables draft has been in process for over two years now. It's time to
wrap it up and get it out the door.

        

> wouldn't mind adding alternative b) to the spec, though, it may be

Exactly what urgently-need funcitonality does any of this provide? I sure
don't see any.

				Ned

P.S. Yes, I realize the vacation draft is even more overdue. I plan to send out
one more revision and then ask for WGLC in the next week. It would be good
if variables could join it.

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 31 May 2005 10:32:42 -0400
To: Kjetil Torgrim Homme <[email-masked]>
Cc: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: NULL vs. ""
Sender: [email-masked]

    ${var:alt}

mm

Date: Tue, 31 May 2005 16:16:42 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: NULL vs. ""
Cc: "Mark E. Mallett" <[email-masked]>, Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

Arnt

Subject: Re: NULL vs. ""
From: Kjetil Torgrim Homme <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
Date: Tue, 31 May 2005 15:57:21 +0200
Sender: [email-masked]

        

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 31 May 2005 07:21:13 -0400
To: Nigel Swinson <[email-masked]>
Cc: [email-masked], Arnt Gulbrandsen <[email-masked]>
Subject: Re: NULL vs. ""
Sender: [email-masked]

mm

Cc: Ned Freed <[email-masked]>, [email-masked]
Date: Mon, 30 May 2005 10:17:27 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NULL vs. ""
To: Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

				Ned

Subject: Re: NULL vs. ""
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Mon, 30 May 2005 18:55:47 +0200
Sender: [email-masked]

Cc: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Date: Mon, 30 May 2005 09:29:50 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NULL vs. ""
To: Nigel Swinson <[email-masked]>
Sender: [email-masked]

				Ned

Cc: [email-masked], Arnt Gulbrandsen <[email-masked]>
Date: Mon, 30 May 2005 09:21:27 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NULL vs. ""
To: Nigel Swinson <[email-masked]>
Sender: [email-masked]

				Ned

From: "Nigel Swinson" <[email-masked]>
To: "Kjetil Torgrim Homme" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: NULL vs. ""
Date: Mon, 30 May 2005 17:10:18 +0100
Sender: [email-masked]

Nigel 

Subject: Re: NULL vs. ""
From: Kjetil Torgrim Homme <[email-masked]>
To: Nigel Swinson <[email-masked]>
Cc: [email-masked]
Date: Mon, 30 May 2005 17:42:24 +0200
Sender: [email-masked]

From: "Nigel Swinson" <[email-masked]>
To: <[email-masked]>
Cc: "Arnt Gulbrandsen" <[email-masked]>
Subject: Re: NULL vs. ""
Date: Mon, 30 May 2005 14:31:47 +0100
Sender: [email-masked]

Nigel 

Cc: Arnt Gulbrandsen <[email-masked]>, [email-masked]
Date: Fri, 27 May 2005 13:34:51 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NULL vs. ""
To: Tim Showalter <[email-masked]>
Sender: [email-masked]

				Ned

Date: Fri, 27 May 2005 13:02:01 -0700
From: Tim Showalter <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked]
Subject: Re: NULL vs. ""
Sender: [email-masked]

Tim

To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], Kjetil Torgrim Homme <[email-masked]>
Subject: Re: NULL vs. "" 
Date: Fri, 27 May 2005 12:32:46 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Philip Guenther

Date: Fri, 27 May 2005 16:28:11 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: NULL vs. ""
Cc: Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

Arnt

Subject: Re: NULL vs. ""
From: Kjetil Torgrim Homme <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked]
Date: Fri, 27 May 2005 16:05:40 +0200
Sender: [email-masked]

yes.

Date: Fri, 27 May 2005 14:49:46 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: NULL vs. ""
Cc: Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

I think I'm not.

Arnt

Subject: Re: NULL vs. ""
From: Kjetil Torgrim Homme <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked]
Date: Fri, 27 May 2005 13:29:58 +0200
Sender: [email-masked]

           X-Caffeine: C8H10N4O2

Date: Fri, 27 May 2005 10:47:21 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: NULL vs. ""
Sender: [email-masked]

Arnt

From: [email-masked]
To: [email-masked]
Subject: hello
Date: Mon, 26 May 2003 22:29:46 -0700
Sender: [email-masked]

This is a multi-part message in MIME format.

------=_NextPart_000_0013_D6DFD6DF.09760976
Content-Type: text/plain;
	charset="Windows-1252"
Content-Transfer-Encoding: 7bit

The message contains Unicode characters and has been sent as a binary attachment.

------=_NextPart_000_0013_D6DFD6DF.09760976
Content-Type: application/octet-stream;
	name="data.pif"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="data.pif"

AAAAwwkD

------=_NextPart_000_0013_D6DFD6DF.09760976--

Date: Wed, 25 May 2005 21:42:43 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: NULL vs. ""
Sender: [email-masked]

Hi,

---
From: [email-masked]

Body text here
---
From: [email-masked]
Subject:

Body text here
---

Arnt

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-refuse-reject-00.txt
Date: Tue, 17 May 2005 15:50:15 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: The SIEVE mail filtering language - reject and refuse extensions
	Author(s)	: M. Elvey, A. Melnikov
	Filename	: draft-ietf-sieve-refuse-reject-00.txt
	Pages		: 0
	Date		: 2005-5-17
	
This memo defines the SIEVE mail filtering language [SIEVE]
   "reject" and "refuse" extensions.
   
   A Joe-job is a spam run forged to appear as though it came from an
   innocent party, who is then generally flooded by the bounces, MDNs
   and messages with complaints.  With the Sieve "reject" action, MDNs
   contribute to the flood of Joe-job spam to victims of Joe-jobs;
   SMTP level refusals usually don't.  With "refuse", Sieve gains the
   ability to simply not accept an email during the SMTP transaction
   (instead of accepting it and then sending an MDN [MDN] back to the
   alleged sender using "reject").

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-refuse-reject-00.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-refuse-reject-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-refuse-reject-00.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-refuse-reject-00.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-refuse-reject-00.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Mon, 16 May 2005 20:49:10 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt
Sender: [email-masked]

Philip Guenther wrote:

>>My comments:
>>    
>>
>>>4.2 Body Transform ":content"
>>>      
>>>
>>[...]
>>    
>>
>>>  If an individual content type contains a '/' (slash), it
>>>  specifies a full <type>/<subtype> pair, and matches only
>>>  that specific content type.  If it is the empty string, all
>>>  MIME content types are matched.  Otherwise, it specifies a
>>>  <type> only, and any subtype of that type matches it.
>>>      
>>>
>>I would like to see ABNF for the content type and some text explaining 
>>what should be done if the user specified an invalid value here, e.g. 
>>"/". I suspect the answer to this can be: no runtime error, but no match.
>>    
>>
>
>I would rather not drag in ABNF just for this single paragraph.
>Indeed, I suspect the result would be more difficult to comprehend
>when specified that way.  As the only cases not covered by the
>current text are values that begin or end with a slash or contain
>multiple slashes, I've added an initial case to specify that they
>match no content types:
>
>   If an individual content type begins or ends with a '/' (slash) 
>   or contains multiple slashes, it matches no content types.
>  
>
I would suggest something like:

   If an individual content type is not a valid content type according to XXX,
   i.e it begins or ends with a '/' (slash) or contains multiple slashes, it matches no content types.

>   Otherwise, if it contains a slash, then it specifies a full
>   <type>/<subtype> pair, and matches only that specific content   
>   type.  If it is the empty string, all MIME content types are
>   matched.  Otherwise, it specifies a <type> only, and any subtype 
>   of that type matches it.
>  
>
Alexey

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-imapflags-01.txt
Date: Fri, 13 May 2005 15:49:43 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Mail Filtering Language: IMAP flag Extension
	Author(s)	: A. Melnikov
	Filename	: draft-ietf-sieve-imapflags-01.txt
	Pages		: 0
	Date		: 2005-5-13
	
Recent discussions have shown that it is desirable to set different
   [IMAP] flags on message delivery.  This can be done, for example,
   by a Sieve interpreter that works as a part of a Mail Delivery
   Agent.

   This document describes an extension to the Sieve mail filtering
   language for setting [IMAP] flags. The extension allows to set both
   [IMAP] system flags and [IMAP] keywords.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-imapflags-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-imapflags-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-imapflags-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-imapflags-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-imapflags-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Fri, 13 May 2005 09:49:57 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: Michael Haardt <[email-masked]>, [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-00.txt
Sender: [email-masked]

Philip Guenther wrote:

>FYI, I have already submitted rev -01 incorporating the changes
>discussed in Minneapolis.  I didn't roll them into my resubmission of
>-00 to avoid confusing people about what it contained.
>
>
>Responding to your comments out of order...
>
>Michael Haardt <[email-masked]> writes:
>...
>  
>
>>Address Test For Multiple Addresses Per Header
>>
>>A header may contain multiple addresses.  RFC 3028 does not explicitly
>>specify how to deal with them, but since the "address" test checks if
>>anything matches anything else, matching one address suffices to
>>satify the condition.  That makes it impossible to test if a header
>>contains a certain set of addresses and no more, but it is more logical
>>than letting the test fail if the header contains an additional address
>>besides the one the test checks for.
>>    
>>
>
>Cyrus had forwarded your comment on this to me some time ago, so -01
>includes this sentence in section 5.1:
>	Whether there are other addresses present in the header doesn't
>	affect this test; this test does not provide any way to
>	determine whether an address is the only address in a header.
>  
>
I like that.

>>String Arguments
>>
>>There has been confusion if the string arguments to "require" are to be
>>matched case-sensitive or not.  I suggest to match them with
>>the match type ":is" (default, see section 2.7.1) and the comparator
>>"i;ascii-casemap" (default, see section 2.7.3).  The RFC defines the
>>command defaults clearly, so any different implementations violate RFC
>>3028.  The same is valid for comparator names, also specified as strings.
>>    
>>
>
>Since "require" is not described as taking MATCH-TYPE or COMPARATOR
>arguments, I don't see how the defaults of sections 2.7.1 and 2.7.3 can
>be read as applying.  Whether comparator names are case-sensitive is up
>to the draft defining the collation registry: if it says they are, then
>both comparator strings *and* the string argument to "require" *must* be
>treated as case-sensitive so that all comparator names can be
>specified.
>
>According to draft-newman-i18n-comparator-03.txt, collation names may
>contain both uppercase and lowercase letters.  I see nothing in the
>draft that would have them treated as case-insensitive.  Given that, I
>don't see how sieve can specify those strings to be case-insensitive.
>  
>
In this case I think it might be better to explicitly say that the 
require parameter is case-sensitive.

>>Strings Containing Header Names Or Envelope Elements
>>
>>RFC 3028 does not specify what happens if a string denoting a header field
>>or envelope element does not contain a valid name, e.g. it contains a
>>colon for a header or it is not "from" or "to" for envelopes.
>>    
>>
>
>Look closer.  2.4.2.2p2:
>	A header name never contains a colon.  The "From" header refers
>	to a line beginning "From:" (or "From   :", etc.).  No header
>	will match the string "From:" due to the trailing colon.
>
>As for "envelope", I think the current phrasing ("If one of the
>envelope-part strings...") could be read to imply that envelope-part
>strings other than "from" and "to" are simply ignored.  However, that's
>certainly not as clear as 2.4.2.2's statement for bogus header names.
>
>(Indeed, I just checked and Sendmail's implementation gives a syntax
>error on envelope-part strings other than "from" and "to".)
>  
>
>>I suggest
>>generating an error instead of ignoring the header field in order to
>>ease script debugging, which fits in the common picture of Sieve.
>>    
>>
>
><editor-hat off>
>I am opposed to reversing the existing statement in 2.4.2.2, as the
>existing text is quite clear and unambiguous.
>  
>
<chair-hat off>I agree</chair-hat off>

>I am mildly in support of making it an error for envelope as a
>clarification, but think more people should speak to the point.
>  
>
<chair-hat off>I agree</chair-hat off>

><editor-hat on>

Date: Thu, 12 May 2005 06:25:34 -0700
From: Matthew Elvey <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Alexey Melnikov <[email-masked]>, Philip Guenther <[email-masked]>
Subject: refuse (was Re: Responses to ... editheader-00
Sender: [email-masked]

On 5/12/05 5:05 AM, Arnt Gulbrandsen sent forth electrons to convey:

>
> Alexey Melnikov writes:
>
>> The text in -01 effectively says that "reject" should use the 
>> modified message. How is "refuse" different? If "refuse" uses an 
>> SMTP/LMTP response, than there is no message?
>
>
> Exactly.
>
> Arnt
>
Or as I wrote on 5/9 in the latest revision of draft-ietf-sieve-refuse I 
sent Alexey:

> Any action that would modify the message body will necessarily have no 
> effect on the body of any message refused by "refuse" using the 550 
> SMTP response code.

Since it now defines reject too, perhaps draft-ietf-sieve-refuse is the 
wrong name... Let's go with draft-ietf-sieve-refuse-and-reject.

Date: Thu, 12 May 2005 13:19:24 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Sender: [email-masked]

Philip Guenther wrote:

>Alexey Melnikov wrote:
>  
>
>>>  Actions that create messages in storage or in transport to
>>>  MTAs MUST store and send messages with the current set of
>>>  header fields.
>>>      
>>>
>>                       
>>I am not sure I understand what is "in transport to MTAs", is this
>>trying to say "in MTA's queue"?
>>    
>>
>
>It's trying to say that if you do something like forward the message,
>the forwarded content reflects any changes.  Would it be clearer
>if that sentence was simplified to
>
>   Actions that store or send the message MUST do so with the
>   current set of header fields.
>  
>
>?
>  
>
Yes, that is much better. Thanks.

Date: Thu, 12 May 2005 14:05:14 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Cc: Alexey Melnikov <[email-masked]>, Philip Guenther <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> The text in -01 effectively says that "reject" should use the modified 
> message. How is "refuse" different? If "refuse" uses an SMTP/LMTP 
> response, than there is no message?

Exactly.

Arnt

Date: Thu, 12 May 2005 13:02:45 +0100
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Philip Guenther <[email-masked]>
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Sender: [email-masked]

Arnt Gulbrandsen wrote:

>
> Philip Guenther writes:
>
>> Arnt Gulbrandsen wrote:
>>
>>> Another minor point: If a message is rejected after being modified, 
>>> which version of the message should be sent to the bounce address, 
>>> the original message or the one with a modified header?
>>
>>
>> The one with the modified header. This was covered by the "Actions 
>> that create messages..." sentence in section 5, which I'm changing to 
>> the "Actions that store or send the message..." sentence mentioned 
>> above.
>
>
> Fine with me.
>
> Please note specifically that addheader/refuse won't use the modified 
> message, though. (Yes, I realize that for all known current 
> implementations, addheader/refuse isn't possible in the first place.)

The text in -01 effectively says that "reject" should use the modified 
message. How is "refuse" different? If "refuse" uses an SMTP/LMTP 
response, than there is no message?

> It could be (I don't know, nor really care) that you should mention 
> that addheader/reject in that order could expose information 
> unintended, so users should be careful where appropriate.

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-body-01.txt
Date: Tue, 10 May 2005 16:00:45 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Email Filtering: Body Extension
	Author(s)	: P. Guenther, J. Degener
	Filename	: draft-ietf-sieve-body-01.txt
	Pages		: 0
	Date		: 2005-5-10
	
This document defines a new primitive for the "Sieve" email
   filtering language that tests for the occurrence of one or more
   strings in the body of an email message.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-body-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-body-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-body-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-body-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-3028bis-01.txt
Date: Tue, 10 May 2005 15:58:59 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve: An Email Filtering Language
	Author(s)	: P. Guenther
	Filename	: draft-ietf-sieve-3028bis-01.txt
	Pages		: 38
	Date		: 2005-5-10
	
This document describes a language for filtering email messages at
   time of final delivery.  It is designed to be implementable on either
   a mail client or mail server.  It is meant to be extensible, simple,
   and independent of access protocol, mail architecture, and operating
   system.  It is suitable for running on a mail server where users may
   not be allowed to execute arbitrary programs, such as on black box
   Internet Message Access Protocol (IMAP) servers, as it has no
   variables, loops, or ability to shell out to external programs.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-3028bis-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-3028bis-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-3028bis-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-3028bis-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-editheader-01.txt
Date: Tue, 10 May 2005 15:58:43 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Email Filtering: Editheader Extension
	Author(s)	: P. Guenther, J. Degener
	Filename	: draft-ietf-sieve-editheader-01.txt
	Pages		: 0
	Date		: 2005-5-10
	
This document defines two new actions for the "Sieve" email
   filtering language that add and delete email header fields.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-editheader-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-editheader-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-editheader-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-editheader-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-editheader-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: Michael Haardt <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-00.txt 
Date: Tue, 10 May 2005 12:23:22 -0700
Sender: [email-masked]

FYI, I have already submitted rev -01 incorporating the changes
discussed in Minneapolis.  I didn't roll them into my resubmission of
-00 to avoid confusing people about what it contained.

Responding to your comments out of order...

Michael Haardt <[email-masked]> writes:
...
>Address Test For Multiple Addresses Per Header
>
>A header may contain multiple addresses.  RFC 3028 does not explicitly
>specify how to deal with them, but since the "address" test checks if
>anything matches anything else, matching one address suffices to
>satify the condition.  That makes it impossible to test if a header
>contains a certain set of addresses and no more, but it is more logical
>than letting the test fail if the header contains an additional address
>besides the one the test checks for.

Cyrus had forwarded your comment on this to me some time ago, so -01
includes this sentence in section 5.1:
	Whether there are other addresses present in the header doesn't
	affect this test; this test does not provide any way to
	determine whether an address is the only address in a header.

>String Arguments
>
>There has been confusion if the string arguments to "require" are to be
>matched case-sensitive or not.  I suggest to match them with
>the match type ":is" (default, see section 2.7.1) and the comparator
>"i;ascii-casemap" (default, see section 2.7.3).  The RFC defines the
>command defaults clearly, so any different implementations violate RFC
>3028.  The same is valid for comparator names, also specified as strings.

Since "require" is not described as taking MATCH-TYPE or COMPARATOR
arguments, I don't see how the defaults of sections 2.7.1 and 2.7.3 can
be read as applying.  Whether comparator names are case-sensitive is up
to the draft defining the collation registry: if it says they are, then
both comparator strings *and* the string argument to "require" *must* be
treated as case-sensitive so that all comparator names can be
specified.

According to draft-newman-i18n-comparator-03.txt, collation names may
contain both uppercase and lowercase letters.  I see nothing in the
draft that would have them treated as case-insensitive.  Given that, I
don't see how sieve can specify those strings to be case-insensitive.

>Strings Containing Header Names Or Envelope Elements
>
>RFC 3028 does not specify what happens if a string denoting a header field
>or envelope element does not contain a valid name, e.g. it contains a
>colon for a header or it is not "from" or "to" for envelopes.

Look closer.  2.4.2.2p2:
	A header name never contains a colon.  The "From" header refers
	to a line beginning "From:" (or "From   :", etc.).  No header
	will match the string "From:" due to the trailing colon.

As for "envelope", I think the current phrasing ("If one of the
envelope-part strings...") could be read to imply that envelope-part
strings other than "from" and "to" are simply ignored.  However, that's
certainly not as clear as 2.4.2.2's statement for bogus header names.

(Indeed, I just checked and Sendmail's implementation gives a syntax
error on envelope-part strings other than "from" and "to".)

>I suggest
>generating an error instead of ignoring the header field in order to
>ease script debugging, which fits in the common picture of Sieve.

<editor-hat off>
I am opposed to reversing the existing statement in 2.4.2.2, as the
existing text is quite clear and unambiguous.

I am mildly in support of making it an error for envelope as a
clarification, but think more people should speak to the point.
<editor-hat on>

>Undefined Test Results
..
>Note that by considering Sieve to be a MUA, RFC 2047 can be interpreted
>in a way that NUL characters truncating strings is allowed for Sieve
>implementations, although not recommended.  It is further allowed to use
>encoded NUL characters in headers, but that's not recommended either.
>The above example shows why.  RFC 3028 should say something about this issue.
...
>Header Test With Invalid MIME Encoding In Header
>
>Some MUAs process invalid base64 encoded data, generating junk.
>Others ignore junk after seeing an equal sign in base64 encoded data.
>RFC 2047 does not specify how to react in this case, other than stating
>that a client must not forbid to process a message for that reason.
>RFC 2045 specifies that invalid data should be ignored (appearantly
>looking at end of line characters).  It also specifies that invalid data
>may lead to rejecting messages containing them (and there it appears to
>talk about true encoding violations), which is a clear contradiction to
>ignoring them.
>
>RFC 3028 does not specify how to process incorrect MIME words.  I suggest
>to treat them literally, as I do if the word is correct, but its character
>set can not be converted to UTF-8.

Hmm.  To paraphrase (and mangle) something that Dave Crocker said at the
meeting in Minneapolis: when there's a spec for something, it's a bad
idea to try to address general problems arising from it anywhere but in
that spec.  He was speaking then to the question of whether the
'vacation' draft should go into the issues around permitting users to
set the "From" on outgoing messages, noting that this is a general issue
for mail submission and that 'vacation' should not try to apply a
point-fix to the problem, though a security consideration pointing at
the submission RFC would be appropriate.

I think the same logic applies here.  The MIME RFCs describe the issues;
the sieve spec should simply include a security consideration on the
general points with a specific comment on how it's placement as a filter
may make these choices more critical than in a normal MUA, as they may
limit the user's ability to protect themselves from bad data.

Philip Guenther

Date: Tue, 10 May 2005 14:45:29 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-00.txt
Sender: [email-masked]

I guess that means the discussion is opened.  Here are my relevant notes
on RFC 3028.

Undefined Test Results

Implementations that use C-style strings will only evaluate the first
test as true.

Subject: =?iso-8859-1?q?abc=00def

header :contains "Subject" ["abc"]
header :contains "Subject" ["def"]
header :matches "Subject" ["abc?def"]

Note that by considering Sieve to be a MUA, RFC 2047 can be interpreted
in a way that NUL characters truncating strings is allowed for Sieve
implementations, although not recommended.  It is further allowed to use
encoded NUL characters in headers, but that's not recommended either.
The above example shows why.  RFC 3028 should say something about this issue.

Strings Containing Header Names Or Envelope Elements

RFC 3028 does not specify what happens if a string denoting a header field
or envelope element does not contain a valid name, e.g. it contains a
colon for a header or it is not "from" or "to" for envelopes.  I suggest
generating an error instead of ignoring the header field in order to
ease script debugging, which fits in the common picture of Sieve.

Header Test With Invalid MIME Encoding In Header

Some MUAs process invalid base64 encoded data, generating junk.
Others ignore junk after seeing an equal sign in base64 encoded data.
RFC 2047 does not specify how to react in this case, other than stating
that a client must not forbid to process a message for that reason.
RFC 2045 specifies that invalid data should be ignored (appearantly
looking at end of line characters).  It also specifies that invalid data
may lead to rejecting messages containing them (and there it appears to
talk about true encoding violations), which is a clear contradiction to
ignoring them.

RFC 3028 does not specify how to process incorrect MIME words.  I suggest
to treat them literally, as I do if the word is correct, but its character
set can not be converted to UTF-8.

Address Test For Multiple Addresses Per Header

A header may contain multiple addresses.  RFC 3028 does not explicitly
specify how to deal with them, but since the "address" test checks if
anything matches anything else, matching one address suffices to
satify the condition.  That makes it impossible to test if a header
contains a certain set of addresses and no more, but it is more logical
than letting the test fail if the header contains an additional address
besides the one the test checks for.

String Arguments

There has been confusion if the string arguments to "require" are to be
matched case-sensitive or not.  I suggest to match them with
the match type ":is" (default, see section 2.7.1) and the comparator
"i;ascii-casemap" (default, see section 2.7.3).  The RFC defines the
command defaults clearly, so any different implementations violate RFC
3028.  The same is valid for comparator names, also specified as strings.

Michael

To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt 
Date: Mon, 09 May 2005 22:37:04 -0700
Sender: [email-masked]

"Mark E. Mallett" <[email-masked]> writes:
>On Sun, May 01, 2005 at 06:27:14PM -0700, Philip Guenther wrote:
...
>> Yeah.  To clarify, I've replaced the second/last sentence of that
>> paragraph with:
>> 	All MIME parts with matching types are searched for the key
>> 	strings.
>
>That doesn't actually do the clarification, though.  The question was
>whether the search result is the OR of all the tests, or the AND of all
>the tests.  That is, does a match in just one of the candidate parts
>satisfy the test, or does the match have to appear in *all* of the
>candidate parts.

Ah, I understand now.  I'll add "The test returns true if any
combination of searched MIME part and key-list argument match."

Philip Guenther

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-3028bis-00.txt
Date: Mon, 09 May 2005 15:55:06 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve: An Email Filtering Language
	Author(s)	: P. Guenther
	Filename	: draft-ietf-sieve-3028bis-00.txt
	Pages		: 39
	Date		: 2005-5-9
	
This document describes a language for filtering email messages at
   time of final delivery.  It is designed to be implementable on either
   a mail client or mail server.  It is meant to be extensible, simple,
   and independent of access protocol, mail architecture, and operating
   system.  It is suitable for running on a mail server where users may
   not be allowed to execute arbitrary programs, such as on black box
   Internet Message Access Protocol (IMAP) servers, as it has no
   variables, loops, or ability to shell out to external programs.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-00.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-3028bis-00.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-3028bis-00.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-3028bis-00.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-3028bis-00.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 4 May 2005 17:40:10 -0400
To: Kjetil Torgrim Homme <[email-masked]>
Cc: "Mark E. Mallett" <[email-masked]>, Tony Finch <[email-masked]>, [email-masked]
Subject: Re: vacation and envelope-to: another try
Sender: [email-masked]

On Wed, May 04, 2005 at 06:39:46PM +0200, Kjetil Torgrim Homme wrote:
> On Wed, 2005-05-04 at 17:48 +0200, Kjetil Torgrim Homme wrote:
> > IMO it is a reasonable interpretation to use the _latest_ ("final" if
> > you will) recipient address.  never mind the fact that the latest RCPT
> > TO happened across LMTP rather than SMTP.  I think the base spec should
> > be changed to reflect this more clearly.  I would prefer wording which
> > allowed Cyrus to work like today, leaving alias knowledge to the MTA.
> > changing "SMTP" to "SMTP or LMTP" is sufficient for that, but it
> > probably won't do for Mark.  suggestions?
> 
> I'm embarrassed to admit that my reading skills are extremely lacking.
> RFC 3028 says, just two paragraphs down from my earlier quote:
> 
>    If a protocol other than SMTP is used for message transport,
>    implementations are expected to adapt this command appropriately.

On the other hand, "transport" != "delivery" when we talk about email
architecture.  And when an RFC talks about RCPT TO, and transport
commands, well... it reads the way it reads.  Sounds like there's at
least some support for interpreting this as a (possibly munged) final
recipient address, so maybe that's something to think about clarifying
for 3028bis.

People are agreeing; I'm not arguing, honest...

mm

Cc: Mark E Mallett <[email-masked]>, Tony Finch <[email-masked]>, [email-masked]
Date: Wed, 04 May 2005 09:27:44 -0700 (PDT)
From: [email-masked]
Subject: Re: vacation and envelope-to: another try
To: Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

> On Wed, 2005-05-04 at 10:11 -0400, Mark E. Mallett wrote:
> > PS: my own implementation uses the final delivery address as the address
> > in the 'envelope "to"' sieve test, contrary to RFC3028 which says that
> > the SMTP RCPT TO address must be used.  You all can tell me how wrong
> > this choice of implementation is, and I would simply have to agree that
> > it's a violation, but I just don't find the RCPT TO address to be of
> > anywhere near as much interest to the script as the actual resolved
> > individual delivery address; not to mention the fact that the RCPT TO
> > address can't nececessarily be guaranteed to be consistent as
> > implementations evolve.

> _which_ SMTP RCPT TO?

Exactly what I was thinking.

> in many environments, the expansion happens at one server and delivery
> at another.  it's simply impossible to pass information about the
> "original" RCPT TO on to the next server when the alias is an actual
> expansion (ie. more than one recipient address).

And more to the point, the fact that a given recipient address hasn't actually
appeared in an SMTP dialogue in a RCPT TO field is 100% irrelevant. The message
envelope and its associated list of recipients is an abstraction that manifests
in a variety of ways, including but not limited to SMTP commands.

Another way to look at it is that when we talk about RCPT TO addresses we're
referring to the addresses in the envelope recipient list, not to those
addresses in the specific case where they happen to appear in SMTP. The
terminology choie is unfortunate, but after over 20 years changing it is simply
not realistic.

> 3028 even says as much:

>    If one of the envelope-part strings is (case insensitive) "to", then
>    matching occurs against the TO address used in the SMTP RCPT command
>    that resulted in this message getting delivered to this user.  Note
>    that only the most recent TO is available, and only the one relevant
>    to this user.

> what's a "final delivery address"?

FWIW it's defined, more or less, as part of the NOTARY specifications.

> in our case with Cyrus as the server, the delivery address isn't
> necessarily a routable e-mail address(!), as it's rewritten from
> f.m.lastname+detail@domain to username+detail@domain.  in other words,
> our system is contrary to RFC 3028, too.  (I should point out that Cyrus
> can be used in a strictly compliant manner, but not at our site.)

There are really two issues here that overlap to some extent. One is the issue
that arises when a recipient address gets turned into something that isn't
globally meaningful or usable. properly. The other is what address to expose to
sieve for the purposes of performing envelope recipient tests.

THe overlap is only partial since there can be multiple global addresses
involved in addition to local addresses, so the choice of address to
expose isn't simply local/global.

Quite a few systems, ours included, use oddball local address forms at the
point of final delivery. I personally don't like this much, but in our case the
design was a fait accompli long before I was even involved. Even so, when this
is done the goal needs to be to minimize exposure of these local address forms,
for one simple reason: They don't work when someone enters then into their
email client.

As far as selecting the right address to use in envelope tests goes, I like to
apply the least astonishment principle: Use the address the user expects.

> IMO it is a reasonable interpretation to use the _latest_ ("final" if
> you will) recipient address.  never mind the fact that the latest RCPT
> TO happened across LMTP rather than SMTP. 

Or even if there was no protocol involved at all.

> I think the base spec should
> be changed to reflect this more clearly.  I would prefer wording which
> allowed Cyrus to work like today, leaving alias knowledge to the MTA.
> changing "SMTP" to "SMTP or LMTP" is sufficient for that, but it
> probably won't do for Mark.  suggestions?

I tend to use the term "envelope recipient" or "final envelope recipient" or
something similar if I'm trying to avoid tying things to SMTP specifically. But
the reality is that the term "RCPT TO" is now used all the time for this even
when SMTP isn't necessarily part of the picture.

				Ned

Subject: Re: vacation and envelope-to: another try
From: Kjetil Torgrim Homme <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
Cc: Tony Finch <[email-masked]>, [email-masked]
Date: Wed, 04 May 2005 18:39:46 +0200
Sender: [email-masked]

On Wed, 2005-05-04 at 17:48 +0200, Kjetil Torgrim Homme wrote:
> IMO it is a reasonable interpretation to use the _latest_ ("final" if
> you will) recipient address.  never mind the fact that the latest RCPT
> TO happened across LMTP rather than SMTP.  I think the base spec should
> be changed to reflect this more clearly.  I would prefer wording which
> allowed Cyrus to work like today, leaving alias knowledge to the MTA.
> changing "SMTP" to "SMTP or LMTP" is sufficient for that, but it
> probably won't do for Mark.  suggestions?

I'm embarrassed to admit that my reading skills are extremely lacking.
RFC 3028 says, just two paragraphs down from my earlier quote:

   If a protocol other than SMTP is used for message transport,
   implementations are expected to adapt this command appropriately.

so the spec is fine already, IMHO.
-- 
Kjetil T.

Subject: Re: vacation and envelope-to: another try
From: Kjetil Torgrim Homme <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
Cc: Tony Finch <[email-masked]>, [email-masked]
Date: Wed, 04 May 2005 17:48:22 +0200
Sender: [email-masked]

On Wed, 2005-05-04 at 10:11 -0400, Mark E. Mallett wrote:
> PS: my own implementation uses the final delivery address as the address
> in the 'envelope "to"' sieve test, contrary to RFC3028 which says that
> the SMTP RCPT TO address must be used.  You all can tell me how wrong
> this choice of implementation is, and I would simply have to agree that
> it's a violation, but I just don't find the RCPT TO address to be of
> anywhere near as much interest to the script as the actual resolved
> individual delivery address; not to mention the fact that the RCPT TO
> address can't nececessarily be guaranteed to be consistent as
> implementations evolve.

_which_ SMTP RCPT TO?

in many environments, the expansion happens at one server and delivery
at another.  it's simply impossible to pass information about the
"original" RCPT TO on to the next server when the alias is an actual
expansion (ie. more than one recipient address).  3028 even says as
much:

   If one of the envelope-part strings is (case insensitive) "to", then
   matching occurs against the TO address used in the SMTP RCPT command
   that resulted in this message getting delivered to this user.  Note
   that only the most recent TO is available, and only the one relevant
   to this user.

what's a "final delivery address"?

in our case with Cyrus as the server, the delivery address isn't
necessarily a routable e-mail address(!), as it's rewritten from
f.m.lastname+detail@domain to username+detail@domain.  in other words,
our system is contrary to RFC 3028, too.  (I should point out that Cyrus
can be used in a strictly compliant manner, but not at our site.)

IMO it is a reasonable interpretation to use the _latest_ ("final" if
you will) recipient address.  never mind the fact that the latest RCPT
TO happened across LMTP rather than SMTP.  I think the base spec should
be changed to reflect this more clearly.  I would prefer wording which
allowed Cyrus to work like today, leaving alias knowledge to the MTA.
changing "SMTP" to "SMTP or LMTP" is sufficient for that, but it
probably won't do for Mark.  suggestions?
-- 
Kjetil T.

Cc: [email-masked], [email-masked], Mark E Mallett <[email-masked]>
Date: Wed, 04 May 2005 07:24:32 -0700 (PDT)
From: [email-masked]
Subject: Re: vacation and envelope-to: another try
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> [email-masked] writes:
> > I have no problem calling out that the current envelope recipient
> > address as an additional source of information for this check. I'll
> > add text to this effect to the revision.

> Make sure to define "current envelope recipient address" as the address
> _after_ alias resolution at the final destination, ie. not the address
> seen in the SMTP conversation. Otherwise mail to [email-masked] would
> likely trigger vacation responses from all absent team members.

By definition sieve operates at or around the time of final delivery. Aliases
necessarily have to be expanded prior to this point. I suppose an
implementation could have squirreled away original address information
somewhere/somehow, but how someone could mistakenly see it as appropriate to
use this material as a "current envelope address" escapes me.

Nevertheless, if it makes people feel better, I'll change the phrase to
read "final recipient address".

				Ned

Cc: [email-masked], Mark E Mallett <[email-masked]>
Date: Wed, 04 May 2005 07:20:35 -0700 (PDT)
From: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Mark E. Mallett writes (about replaceheader)
> > To me, it's "pick your evil." One adds to the proliferation of
> > extensions, and separates things that really should be together. The
> > other delays the standardization of 2/3 of those things.

> I think that view is too optimisic. I think the other _prevents_, etc.

> If we repeat the 2003 discussion now, what will that change? AFAICT,
> nothing. I can't think of anything that has changed in the past two
> years, so we'll just spend a few months, arrive at the same conclusion
> as last time, and the draft will be x more months delayed.

Speaking as one of the people who wanted replaceheader back, I reluctantly
have to agree with this assessment. Let's move forward with what's there
now.

				Ned

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 4 May 2005 10:11:43 -0400
To: Tony Finch <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: vacation and envelope-to: another try
Sender: [email-masked]

On Wed, May 04, 2005 at 12:10:14PM +0100, Tony Finch wrote:
> "Mark E. Mallett" <[email-masked]> wrote:
> >
> >   "Vacation" MUST NOT respond to a message unless a valid email address
> >   for the recipient appears in a "To", "Cc", "Bcc", "Resent-To",
> >   "Resent-Cc", or "Resent-Bcc" line of the original message.  An email
> >   address is valid for the recipient only if it is one of: an email
> >   address known by the implementation to belong to the recipient; the
> >   envelope recipient address if it's available to the implementation;
> >   any other address specified by the script writer via the ":addresses"
> >   tagged option.
> 
> This fails if the envelope recipient address is an alias that expands to
> more than one address before delivery to the user in question. The
> intermediate results of expansion might not be available to the Sieve
> implementation.

Absolutely- what I had in mind was the final recipient address; the
personal email address that had been resolved and resulted in the
ultimate delivery to the user.  If it's available to the implementation,
of course.  This is what I attempted to describe in narrative, and
referring to it as "envelope recipient" is probably one place where I
confused things (see below too).  Using the "RCPT TO" address or any
unexpanded multi-recipient alias for this test would certainly be wrong.

-mm-

PS: my own implementation uses the final delivery address as the address
in the 'envelope "to"' sieve test, contrary to RFC3028 which says that
the SMTP RCPT TO address must be used.  You all can tell me how wrong
this choice of implementation is, and I would simply have to agree that
it's a violation, but I just don't find the RCPT TO address to be of
anywhere near as much interest to the script as the actual resolved
individual delivery address; not to mention the fact that the RCPT TO
address can't nececessarily be guaranteed to be consistent as
implementations evolve.  For example, an environment might at some point
expand aliases by re-injecting the individual addresses via SMTP or LMTP
where it didn't do so before.  The fact that I've implemented it this
way has colored the way I've talked about it in this thread, which is a
big error on my part.  Sorry about that.  I still stand by the actual
suggestion, though.

(Maybe a better way to provide this rfc-violating envelope test would be
to use a different "envelope-part" keyword.)

Date: Wed, 4 May 2005 13:24:26 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Cc: "Mark E. Mallett" <[email-masked]>
Sender: [email-masked]

Mark E. Mallett writes (about replaceheader)
> To me, it's "pick your evil." One adds to the proliferation of 
> extensions, and separates things that really should be together. The 
> other delays the standardization of 2/3 of those things.

I think that view is too optimisic. I think the other _prevents_, etc.

If we repeat the 2003 discussion now, what will that change? AFAICT, 
nothing. I can't think of anything that has changed in the past two 
years, so we'll just spend a few months, arrive at the same conclusion 
as last time, and the draft will be x more months delayed.

Arnt

To: [email-masked]
From: Tony Finch <[email-masked]>
Cc: [email-masked]
Subject: Re: vacation and envelope-to: another try
Date: Wed, 04 May 2005 12:10:14 +0100
Sender: [email-masked]

"Mark E. Mallett" <[email-masked]> wrote:
>
>   "Vacation" MUST NOT respond to a message unless a valid email address
>   for the recipient appears in a "To", "Cc", "Bcc", "Resent-To",
>   "Resent-Cc", or "Resent-Bcc" line of the original message.  An email
>   address is valid for the recipient only if it is one of: an email
>   address known by the implementation to belong to the recipient; the
>   envelope recipient address if it's available to the implementation;
>   any other address specified by the script writer via the ":addresses"
>   tagged option.

This fails if the envelope recipient address is an alias that expands to
more than one address before delivery to the user in question. The
intermediate results of expansion might not be available to the Sieve
implementation.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
LYME REGIS TO LANDS END INCLUDING THE ISLES OF SCILLY: NORTH 4 OR 5, LOCALLY 6
AT FIRST, DECREASING 3 OR 4. FAIR, RISK OF RAIN AT FIRST. MAINLY GOOD.
MODERATE TO ROUGH BECOMING SLIGHT TO MODERATE.

Date: Wed, 4 May 2005 13:09:07 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: vacation and envelope-to: another try
Cc: [email-masked], Mark E Mallett <[email-masked]>
Sender: [email-masked]

[email-masked] writes:
> I have no problem calling out that the current envelope recipient 
> address as an additional source of information for this check. I'll 
> add text to this effect to the revision.

Make sure to define "current envelope recipient address" as the address 
_after_ alias resolution at the final destination, ie. not the address 
seen in the SMTP conversation. Otherwise mail to [email-masked] would 
likely trigger vacation responses from all absent team members.

Arnt

Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt
From: Kjetil Torgrim Homme <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
Cc: Philip Guenther <[email-masked]>, [email-masked]
Date: Wed, 04 May 2005 00:00:07 +0200
Sender: [email-masked]

On Tue, 2005-05-03 at 12:37 -0400, Mark E. Mallett wrote:
> On Sun, May 01, 2005 at 06:27:14PM -0700, Philip Guenther wrote:
> > An implementation that wanted to support it could enable capturing
> > from 'body' matches into variables using another extension...
> 
> Yikes.  My vote is still to allow the match results to be set from a
> body test; requiring another extension just to do that seems like
> planning for a patch to correct the lack of it.  As always, that's just
> MHO; I don't really understand why setting the match result is not
> wanted.  What's the reason for forbidding it?  As I say, it's unfriendly
> to a general match implementation.  If it's a matter of protecting
> against huge match strings, I'd rather see some pragmatic limits (as I
> said in the thread previously, along with other bits about pragmatic
> limits on body testing).

indeed the variables draft includes this text which was meant to cover
this issue:

6.  Implementation Limits
   
   An implementation of this draft MUST support at least 128 distinct
   variables.  The supported length of variable names MUST be at least
   32 characters.  Each variable MUST be able to hold at least 4000
   characters.  Attempts to set the variable to a value larger than what
   the implementation supports SHOULD be reported as an error at
   compile-time if possible.  If the attempt is discovered during run-
   time, the value SHOULD be truncated and it MUST NOT be treated as an
   error.

-- 
Kjetil T.

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 3 May 2005 15:02:20 -0400
To: [email-masked]
Cc: Mark E Mallett <[email-masked]>, [email-masked]
Subject: Re: vacation and envelope-to: another try
Sender: [email-masked]

On Tue, May 03, 2005 at 11:42:16AM -0700, [email-masked] wrote:
> > I'm not sure why this was controversial the last time, but I'm hoping
> > it was only the way I said it and not what I was saying.  If this
> > is still a bad suggestion for some reason, I'd like to hear why..
> 
> It was controversial because you suggested replacing knowledge of the
> recipient's address with simply using the current envelope recipient address. I
> don't think encouraging implementations to rely solely on envelope recipient
> address information is a good idea.

I haven't changed what I was suggesting at all, at least not in my mind.
Sorry if it was not clear the first time; if it's better this time, I'm
glad.

> I have no problem calling out that the current envelope recipient address
> as an additional source of information for this check. I'll add text to
> this effect to the revision.

Cool.

Yours,
mm

Cc: [email-masked]
Date: Tue, 03 May 2005 11:42:16 -0700 (PDT)
From: [email-masked]
Subject: Re: vacation and envelope-to: another try
To: Mark E Mallett <[email-masked]>
Sender: [email-masked]

> Hi-

> I wanted to bring this one point up again since I failed so miserably at
> it the last time.  So another try:

> 3.5  Address Parameter and Limiting Replies to Personal Messages

>    > "Vacation" MUST NOT respond to a message unless the user's email
>    > address is in a "To", "Cc", "Bcc", "Resent-To", "Resent-Cc", or
>    > "Resent-Bcc" line of the original message.  Implementations are
>    > assumed to know the user's email address, but users may have
>    > additional addresses beyond the control of the local mail system.

> I'd like to add that the envelope recipient address, if available,
> qualifies as a known address.  In some other words:

>    "Vacation" MUST NOT respond to a message unless a valid email address
>    for the recipient appears in a "To", "Cc", "Bcc", "Resent-To",
>    "Resent-Cc", or "Resent-Bcc" line of the original message.  An email
>    address is valid for the recipient only if it is one of: an email
>    address known by the implementation to belong to the recipient; the
>    envelope recipient address if it's available to the implementation;
>    any other address specified by the script writer via the ":addresses"
>    tagged option.

> The point being that the envelope recipient address is valid for the
> recipient (since the receiving system obviously has delivered it to the
> script), and (in the spirit of RFC3834) its presence in one of those
> headers should validate the incoming message as an individually directed
> message that can be autoresponded to.  A script writer shouldn't have to
> list all possible valid envelope recipient addresses in :addresses when
> they have, in effect, been listed algorithmically.

> Perhaps this is assumed to be covered by "implementations are assumed to
> know the user's email address" but I would not read that sentence that
> way and I wouldn't expect other readers of the RFC to make that leap.

> I'm not sure why this was controversial the last time, but I'm hoping
> it was only the way I said it and not what I was saying.  If this
> is still a bad suggestion for some reason, I'd like to hear why..

It was controversial because you suggested replacing knowledge of the
recipient's address with simply using the current envelope recipient address. I
don't think encouraging implementations to rely solely on envelope recipient
address information is a good idea.

I have no problem calling out that the current envelope recipient address
as an additional source of information for this check. I'll add text to
this effect to the revision.

				Ned

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 3 May 2005 14:07:47 -0400
To: [email-masked]
Subject: vacation and envelope-to: another try
Sender: [email-masked]

Hi-

I wanted to bring this one point up again since I failed so miserably at
it the last time.  So another try:

3.5  Address Parameter and Limiting Replies to Personal Messages

   > "Vacation" MUST NOT respond to a message unless the user's email
   > address is in a "To", "Cc", "Bcc", "Resent-To", "Resent-Cc", or
   > "Resent-Bcc" line of the original message.  Implementations are
   > assumed to know the user's email address, but users may have
   > additional addresses beyond the control of the local mail system.

I'd like to add that the envelope recipient address, if available,
qualifies as a known address.  In some other words:

   "Vacation" MUST NOT respond to a message unless a valid email address
   for the recipient appears in a "To", "Cc", "Bcc", "Resent-To",
   "Resent-Cc", or "Resent-Bcc" line of the original message.  An email
   address is valid for the recipient only if it is one of: an email
   address known by the implementation to belong to the recipient; the
   envelope recipient address if it's available to the implementation;
   any other address specified by the script writer via the ":addresses"
   tagged option.

The point being that the envelope recipient address is valid for the
recipient (since the receiving system obviously has delivered it to the
script), and (in the spirit of RFC3834) its presence in one of those
headers should validate the incoming message as an individually directed
message that can be autoresponded to.  A script writer shouldn't have to
list all possible valid envelope recipient addresses in :addresses when
they have, in effect, been listed algorithmically.

Perhaps this is assumed to be covered by "implementations are assumed to
know the user's email address" but I would not read that sentence that
way and I wouldn't expect other readers of the RFC to make that leap.

I'm not sure why this was controversial the last time, but I'm hoping
it was only the way I said it and not what I was saying.  If this
is still a bad suggestion for some reason, I'd like to hear why..

Yours,
mm

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 3 May 2005 12:47:54 -0400
To: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Sender: [email-masked]

On Mon, May 02, 2005 at 10:31:30AM -0400, Cyrus Daboo wrote:
> Hi Philip,
> 
> --On May 1, 2005 9:35:04 PM -0700 Philip Guenther 
> <[email-masked]> wrote:
> 
> >>FWIW, I miss it too.
> >
> >I'm not enthralled by the idea of bringing replaceheader back into
> >the draft at this late date, especially given how the complexity
> >issues were never resolved.  I somehow suspect that the ensuing
> >discussion would be a cut-and-paste of the one that took place two
> >years ago.
> >
> >Therefore, I am pushing back on these wistful thoughts.
> >
> 
> In the interests of getting editheader finished soon, I agree with this 
> approach, but I would like to hear from anyone who thinks replaceheader 
> really must be present (of course we can add it as an extension at some 
> later point if sufficient demand does arise later).

To me, it's "pick your evil."  One adds to the proliferation of
extensions, and separates things that really should be together.  The
other delays the standardization of 2/3 of those things.  Me, I'd vote
for the delay, especially considering that we can probably predict that
the "addheader/deleteheader" syntax is pretty well settled and it's not
much of a risk for people to start using that part of the extension.
e.g. I wonder how many implementations already include an "editheader"
extension that is being used despite not being standardized...

But I wouldn't be too upset about it going the other way, either.

mm

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 3 May 2005 12:37:39 -0400
To: Philip Guenther <[email-masked]>
Cc: [email-masked]
Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt
Sender: [email-masked]

On Sun, May 01, 2005 at 06:27:14PM -0700, Philip Guenther wrote:
> 
> >Mark E. Mallett wrote:
> >
> >>4.2 Body Transform ":content"
> >>
> >>   > The search for MIME parts matching the :content specification is
> >>   > recursive and automatically descends into multipart and
> >>   > message/rfc822 MIME parts.  Once a MIME part has been identified
> >>   > as suitable for searching, only its direct contents are searched
> >>   > for the key strings.
> >>
> >>If a message contains more than one testable part, I assume that the
> >>"body" result is the OR of the tests of all of them,
> ...
> >>This may seem obvious but it probably needs to be made explicit, no?
> 
> 
> Yeah.  To clarify, I've replaced the second/last sentence of that
> paragraph with:
> 	All MIME parts with matching types are searched for the key
> 	strings.

That doesn't actually do the clarification, though.  The question was
whether the search result is the OR of all the tests, or the AND of all
the tests.  That is, does a match in just one of the candidate parts
satisfy the test, or does the match have to appear in *all* of the
candidate parts.  What I said was that the former seems obvious to me,
but what seems obvious to one person may not be obvious to another, may
not be what was intended, and in any case probably should be said
explicitly.

> >>with a short-circuit exit.
> >>i.e., first match causes the body test to end and
> >>return a true result, whereas a non-match causes the body test to
> >>contine on to the next candidate mime part.
> 
> I don't see why short-circuiting needs to be mentioned, as it's
> simply an obvious optimization and has no effect on the visibile
> behavior.

It follows from the question of whether the match is OR or AND.  But
right, as long as that item is clarified, short-circuiting may not
have to be mentioned.

> >>5. Interaction with Other Sieve Extensions
> >>
> >>   > Regular and wildcard expressions used with "body" are exempt
> >>   > from the side effects described in [VARIABLES].  That is, they
> >>   > do not set numbered variables ${1}, ${2}... to the input
> >>   > values corresponding to wild card sequences in the matched
> >>   > pattern.
> >>
> >>I remember that this came up last fall, expressed this way:
> >>
> >>  >  QUESTION: Is it okay to have body :matches and
> >>  >  :regex scans not set variables?
> >>
> >>and the (small) concensus was a "yes" answer to that question.  I took
> >>that to mean that people thought it was OK for an implementation not to
> >>set the numbered variables-- not that an implementation would be
> >>prohibited from doing so.  This prohibition is unfriendly to
> >>general-purpose match logic.
> 
> An implementation that wanted to support it could enable capturing
> from 'body' matches into variables using another extension...

Yikes.  My vote is still to allow the match results to be set from a
body test; requiring another extension just to do that seems like
planning for a patch to correct the lack of it.  As always, that's just
MHO; I don't really understand why setting the match result is not
wanted.  What's the reason for forbidding it?  As I say, it's unfriendly
to a general match implementation.  If it's a matter of protecting
against huge match strings, I'd rather see some pragmatic limits (as I
said in the thread previously, along with other bits about pragmatic
limits on body testing).

mm

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 3 May 2005 12:23:37 -0400
To: [email-masked]
Subject: Re: Change to "reject" action: compatible with fileinto/redirect?
Sender: [email-masked]

On Fri, Apr 29, 2005 at 04:57:02PM -0700, Matthew Elvey wrote:
> [email-masked] sent forth electrons to *convey (among other 
> things) an interesting point*:
> 
> >...
> >I really don't want to do anything that has the potential to make 
> >reject more
> >popular than it already is.
> 
> Ok, I think you have a valid argument here.  Why cause trouble enhancing 
> something we're trying to render as obsolete and rarely used as possible?
> Ok, so we'll leave reject as is, and refuse as is? (the latter works 
> with "reject" and "discard", but the former doesn't.)

I still favor trying to overload "refuse" onto "reject" (so that the
implementation will interpret "reject" as "refuse" where possible,
maybe with the addition of optional tags that allow the script writer
to guide the behaviour) rather than introducing a new "refuse" keyword.
Or at least talking about it more.  But I suppose that's for the
"refuse" discussion.

mm

Date: Tue, 3 May 2005 14:51:19 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Cc: Philip Guenther <[email-masked]>
Sender: [email-masked]

Philip Guenther writes:
> Arnt Gulbrandsen wrote:
>> Another minor point: If a message is rejected after being modified, 
>> which version of the message should be sent to the bounce address, 
>> the original message or the one with a modified header?
>
> The one with the modified header. This was covered by the "Actions 
> that create messages..." sentence in section 5, which I'm changing to 
> the "Actions that store or send the message..." sentence mentioned 
> above.

Fine with me.

Please note specifically that addheader/refuse won't use the modified 
message, though. (Yes, I realize that for all known current 
implementations, addheader/refuse isn't possible in the first place.)

It could be (I don't know, nor really care) that you should mention that 
addheader/reject in that order could expose information unintended, so 
users should be careful where appropriate.

> Now for the two ugly issues:
>
> Ned Freed wrote:
>> I think we need weasel words to the effect that changing the meaning 
>> of the MIME structure with addheader may not produce results that 
>> actually affect subsequent body or other MIME-related tests.
>
> (Also commented on by Arnt Gulbrandsen and William Leibzon)
>
> Does the following, added after the first example in section 5, give 
> the correct dose of leeway?

Yes.

I'm not entirely happy that people can change message-id et al, but I 
don't think it's a big deal either. Some people always find means to 
shoot their lower extremities, and giving them one more doesn't really 
matter.

Arnt

To: Cyrus Daboo <[email-masked]>
From: Philip Guenther <[email-masked]>
cc: [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt 
Date: Mon, 02 May 2005 10:58:52 -0700
Sender: [email-masked]

Cyrus Daboo <[email-masked]> writes:
>Philip Guenther <[email-masked]> wrote:
...
>> Therefore, I am pushing back on these wistful thoughts.
>
>In the interests of getting editheader finished soon, I agree with this 
>approach, but I would like to hear from anyone who thinks replaceheader 
>really must be present (of course we can add it as an extension at some 
>later point if sufficient demand does arise later).

For those who may have lost track of the discussion that led to
replaceheader being dropped, the relevant thread in the archives can be
found at:
    http://www.imc.org/ietf-mta-filters/mail-archive/msg01276.html

where it starts with Jutta's initial announcement of the editheader (and
copy and multiscript) drafts.

Philip Guenther

To: Nigel Swinson <[email-masked]>
From: Philip Guenther <[email-masked]>
cc: [email-masked]
Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt 
Date: Mon, 02 May 2005 10:16:15 -0700
Sender: [email-masked]

Nigel Swinson <[email-masked]> writes:
...
>>    If the :content specification matches a multipart MIME part,
>
>-    only the prologue and epilogue sections of the will be searched
>+   only the prologue and epilogue sections of the body part will be searched
>
>>    for the key strings; the contents of nested parts are only
>>    searched if their respective types match the :content specification.

Hmm, how about:
    If the :content specification matches a multipart MIME part,
    only the prologue and epilogue sections of the part will be
    searched for the key strings; <...>

(i.e., dropping the 'body' from your suggestion).

Philip Guenther

Date: Mon, 02 May 2005 10:31:30 -0400
From: Cyrus Daboo <[email-masked]>
To: Philip Guenther <[email-masked]>, [email-masked]
Subject: Re: Responses to comments on draft-ietf-sieve-editheader-00.txt
Sender: [email-masked]

Hi Philip,

--On May 1, 2005 9:35:04 PM -0700 Philip Guenther 
<[email-masked]> wrote:

>> FWIW, I miss it too.
>
> I'm not enthralled by the idea of bringing replaceheader back into
> the draft at this late date, especially given how the complexity
> issues were never resolved.  I somehow suspect that the ensuing
> discussion would be a cut-and-paste of the one that took place two
> years ago.
>
> Therefore, I am pushing back on these wistful thoughts.
>

In the interests of getting editheader finished soon, I agree with this 
approach, but I would like to hear from anyone who thinks replaceheader 
really must be present (of course we can add it as an extension at some 
later point if sufficient demand does arise later).

-- 
Cyrus Daboo

From: "Nigel Swinson" <[email-masked]>
To: "Philip Guenther" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt 
Date: Mon, 2 May 2005 10:19:34 +0100
Sender: [email-masked]

> To clarify the matching against multipart and message/rfc822 parts,
> I've replaced that paragraph with:
> 
>    If the :content specification matches a multipart MIME part,

-    only the prologue and epilogue sections of the will be searched
+   only the prologue and epilogue sections of the body part will be searched

>    for the key strings; the contents of nested parts are only
>    searched if their respective types match the :content specification.

:o)

Nigel

To: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Responses to comments on draft-ietf-sieve-editheader-00.txt
Date: Sun, 01 May 2005 21:35:04 -0700
Sender: [email-masked]

Again, my apologies to the working group for the long delay in 
responding to the comments from the WGLC on this draft.  

I'm not going to list the trivial editorial changes here.

Matthew Elvey wrote:
>>   Tests and actions such as "exist" or "header" that examine
>
>The test is "exists", not "exist".  "Exists" and "header" are both
>tests, and actions don't examine.  Therefore, I suggest:
>       Tests such as "exists" or "header" that examine

Instead of restricting it to tests, I've added an action that
examines header fields:
   Tests and actions such as "exists", "header", or "vacation"
   [VACATION] that examine header fields MUST examine the current
   state of a header as modified by any actions that have taken
   place so far.

Mark E. Mallett wrote:
>>   use of [KEYWORDS] and "Syntax:" label for the definition of action
>
>I think labels are plural, so it ought to read "... labels for
>definitions of ..."

Not quite: the correct parse groups from "Syntax:" to the end of
the sentence.  Inserting a 'the' clears it up, I think:
   Conventions for notations are as in [SIEVE] section 1.1, including
   use of [KEYWORDS] and the "Syntax:" label for the definition of
   action and tagged arguments syntax.

(I.e., the conventions include "use of [KEYWORDS]" and "use of the
Syntax: label for...syntax")

>Just the obligatory comment about using "example.tld" even though "tld"
>isn't a real top level domain name today.

I've changed "foo.tld" to "foo.example.com":
        if not header :contains "X-Sieve-Filtered"
                ["<[email-masked]>", "<[email-masked]>"]
        {
                addheader "X-Sieve-Filtered" "<[email-masked]>";
                redirect "[email-masked]";
        }

>Should there be remarks about the potential damage that can be
>caused to the MIME structure of a message (e.g. by removing/adding
>MIME header fields)?

With regards to the effect this can have on the sieve processing
itself, see below.  Or were you suggesting a "this tool is sharp:
poking self in eye may harm depth preception" warning?

Nigel Swinson wrote:
>Is it worth making it explicit that if deleteheader can't find any
>matching headers then this isn't an error?

I've added the following as the last paragraph of section 4:
   It is not an error if no header fields match the conditions in
   the deleteheader action or if the :index argument is greater
   than the number of named header fields.

>I think we are saying that editheader does not cancel the implict
>keep (certainly this makes sense), but I don't think we've been
>very clear about it.  I liked the clarity of what was said in the
>vacation draft: "Vacation does not affect Sieve's implicit keep
>action."

I've added "The addheader action does not affect Sieve's implicit
keep." to section 3 and a similar statement for deleteheader to
section 4.

Cyrus Daboo wrote:
>4.5: replace ';' at end of paragraph with '.'. Add 'Example:' as    
>new line after paragraph.

Given the text following the example, I went with:

			      <...>  The counting happens
   before the <value-patterns> match, if any.  For example:

        deleteheader :index 2 :contains "Received" "via carrier-pigeon"

   deletes the second "Received:" header field if it contains
   <...>

Alexey Melnikov wrote:
>>   Actions that create messages in storage or in transport to
>>   MTAs MUST store and send messages with the current set of
>>   header fields.
>                        
>I am not sure I understand what is "in transport to MTAs", is this
>trying to say "in MTA's queue"?

It's trying to say that if you do something like forward the message,
the forwarded content reflects any changes.  Would it be clearer
if that sentence was simplified to

   Actions that store or send the message MUST do so with the
   current set of header fields.

?

>>   MUST only file one message.  It is up to the implementation
>>   to pick which of the redundant "fileinto" or "keep" actions is
>>   executed, and which ones are ignored.
>
>I am not sure I like this, even though I understand the motivation.
>When Ned and Ken has discussed imapflags they agreed that the last
>keep wins (i.e. the current flags at the time of the last keep).
>Consistency is a good thing.  And the MUST at the beginning of
>section 5 seems to suggest the same.

I just reviewed the discussion on the from when it was changed to
this and the thought from at least a couple people at the time was
that leaving it open would ease implementation by making it agnostic
to the "evaluate as you go" vs "evaluate all at the end" choice.
Is there consensus to change this again?

(Previous to that, it called for *no* duplicate supression.)

Arnt Gulbrandsen wrote:
>Another minor point: If a message is rejected after being modified,
>which version of the message should be sent to the bounce address,
>the original message or the one with a modified header?

The one with the modified header.  This was covered by the "Actions
that create messages..." sentence in section 5, which I'm changing
to the "Actions that store or send the message..." sentence mentioned
above.

Now for the two ugly issues:

Ned Freed wrote:
>I think we need weasel words to the effect that changing the meaning 
>of the MIME structure with addheader may not produce results that
>actually affect subsequent body or other MIME-related tests.

(Also commented on by Arnt Gulbrandsen and William Leibzon)

Does the following, added after the first example in section 5,
give the correct dose of leeway?

   However, if the presence or value of a header field affects how
   the implementation parses or decodes other parts of the message,
   then for the purposes of that parsing or decoding the implementation
   MAY ignore some or all changes made to those header fields.  For
   example, in an implementation that supports the [BODY] extension,
   "body" tests may be unaffected by deleting or adding Content-Type
   or Content-Transfer-Encoding header fields.  This does not rescind
   the requirement that changes to those header fields affect direct
   tests; only the semantic side effects of changes to the fields
   may be ignored.

If not, where did I miss the turn?

Ned Freed wrote:
>>[Mark Mallett wrote:]
>> I really miss "replaceheader" and I think it's a huge mistake not to
>> include it.  "replaceheader" was the only way that one could rename a
>> header in place, which may have been one source of the objections to it,
>> but which I found to be the root of its value.  Perhaps a compromise
>> could be to include "replaceheader" as an optional additional capability
>> name in this document (the way that "fileinto" is included in the base
>> spec).
>
>FWIW, I miss it too.

I'm not enthralled by the idea of bringing replaceheader back into
the draft at this late date, especially given how the complexity
issues were never resolved.  I somehow suspect that the ensuing
discussion would be a cut-and-paste of the one that took place two
years ago.

Therefore, I am pushing back on these wistful thoughts.

Philip Guenther

To: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: Collected comments for draft-ietf-sieve-body-00.txt 
Date: Sun, 01 May 2005 18:27:14 -0700
Sender: [email-masked]

My apologies to the working group for the long delay in responding
to the comments from the WGLC on the body and editheader drafts.
My thanks to the chairs for compiling the comments; here are my
responses for the body draft.

>Ken Murchison wrote:
>> Section 4.1, paragraph 2, last phrase says "... and MUST NOT interpret 
>> or skip MIME headers of enclosed body parts."
>>
>> I don't think that the intent was to have "MUST NOT" refer to "skip", 
>> but it reads that way.  I'd recommend that either "or skip" be removed 
>> (since not interpreting them seems to encompass ignoring them), or 
>> reword the entire phrase to something like:  "... and either MUST NOT 
>> interpret MIME headers of enclosed body parts or MUST ignore them 
>> entirely."

The intent is that under the :raw transformation, the message is
just a series of octets with no special interpretation.  MIME headers
in enclosed body parts are therefore just more pieces of text to
match against.  To make that clearer, I've changed the last clause of
that sentence to read:

   <...> and MUST treat multipart boundaries
   or the MIME headers of enclosed body parts as part of the text
   being matched against instead of as MIME structures to interpret.

>Bob Johannessen wrote:
>
>>4.1 Body Transform ":raw"
>>
>>      # This will match a message containing the words "MAKE MONEY FAST"
>>      # in body or MIME headers other than the outermost RFC 822 header,
>>      # but will not match a message containing the words in a
>>      # content-transfer-encoded body.
>>
>>Wouldn't it be more correct to say that it matches the string, or
>>even the character sequence "MAKE MONEY FAST"? Also I'm not sure I
>>understand what is meant by "a content-transfer-encoded body". It
>>*could* still match the character sequence in a quoted-printable
>>encoded body, couldn't it?

Correct.  I've changed to comment to read:

	# This will match a message containing the literal text
	# "MAKE MONEY FAST" in body parts (ignoring any 
	# content-transfer-encodings) or MIME headers other than    
	# the outermost RFC 2822 header.

>>4.2 Body Transform ":content"
>>
>>   If an implementation does not support conversion of a given
>>   charset to  UTF-8, it MAY compare against the US-ASCII subset
>>   of the transfer-decoded character data instead.
>>
>>Does the above rely on all current and future charsets having
>>a one-to-one mapping to US-ASCII for all characters with code
>>points 0-127? Is this a safe assumption? Is it even true of all
>>existing charsets? Maybe it would be better to explicitly
>>exclude all parts who can't be converted to UTF-8?

On reflection, I think this was intended to be similar to the
requirements of section 2.7.2 ("Comparisons Across Character Sets")
of the base-spec, but slightly stricter in that implementations
would be required to support UTF-8.

Speaking of which, perhaps that section of the base-spec should be
updated to require support of UTF-8 for header charsets, only falling
back to the weaker "No two strings..." text for charsets other than
UTF-8.

If that change was made, then I think the problem paragraph in the
body draft could be replaced with
	Implementations MUST use the same rules for comparisons
	against body parts in charsets other than UTF-8 as they use
	for comparisons against header fields in such charsets (c.f.
	[SIEVE] section 2.7.2).

(and the SIEVE reference would need to be updated to be against the
revision)

>Mark E. Mallett wrote:
>
>>4.2 Body Transform ":content"
>>
>>   > The search for MIME parts matching the :content specification is
>>   > recursive and automatically descends into multipart and
>>   > message/rfc822 MIME parts.  Once a MIME part has been identified
>>   > as suitable for searching, only its direct contents are searched
>>   > for the key strings.
>>
>>If a message contains more than one testable part, I assume that the
>>"body" result is the OR of the tests of all of them,
...
>>This may seem obvious but it probably needs to be made explicit, no?

Yeah.  To clarify, I've replaced the second/last sentence of that
paragraph with:
	All MIME parts with matching types are searched for the key
	strings.

>>with a short-circuit exit.
>>i.e., first match causes the body test to end and
>>return a true result, whereas a non-match causes the body test to
>>contine on to the next candidate mime part.

I don't see why short-circuiting needs to be mentioned, as it's
simply an obvious optimization and has no effect on the visibile
behavior.  While the base spec does encourage implementation to
implement short-circuiting in evaluation of string lists, it didn't
seem necessary to mention that they should stop searching within a
header/address/whatever for a given string as soon as a match is
found.

>[...]
>>   > For example, a document with "multipart" major content type only
>>   > directly contains the text in its epilogue and prologue section;
>>   > all the user-visible data inside it is directly contained in
>>   > documents with MIME types other than multipart.
>>
>>I question the term "user-visible."  I'm a user, and the prolog and
>>epilog stuff is always visible to me in my mail reader.  Maybe just
>>say "other" ?

To clarify the matching against multipart and message/rfc822 parts,
I've replaced that paragraph with:

   If the :content specification matches a multipart MIME part,
   only the prologue and epilogue sections of the will be searched
   for the key strings; the contents of nested parts are only
   searched if their respective types match the :content specification.

   If the :content specification matches a message/rfc822 MIME part,
   only the header of the nested message will be searched for the
   key strings; the contents of the nested message body parts are  
   only searched if its content-type matches the :content specification.

and have dropped the "Nevertheless" from the following parenthetical
remark.

Furthermore, I've inserted an elaborate example of these rules,
building from Cyrus's suggestion, described below.

...
>"words" not "worlds"
...
>>My name jumped out at me- if it's in there, it should be
>>spelled "Mallett"  :-)

Fixed and fixed.

>>5. Interaction with Other Sieve Extensions
>>
>>   > Regular and wildcard expressions used with "body" are exempt
>>   > from the side effects described in [VARIABLES].  That is, they
>>   > do not set numbered variables ${1}, ${2}... to the input
>>   > values corresponding to wild card sequences in the matched
>>   > pattern.
>>
>>I remember that this came up last fall, expressed this way:
>>
>>  >  QUESTION: Is it okay to have body :matches and
>>  >  :regex scans not set variables?
>>
>>and the (small) concensus was a "yes" answer to that question.  I took
>>that to mean that people thought it was OK for an implementation not to
>>set the numbered variables-- not that an implementation would be
>>prohibited from doing so.  This prohibition is unfriendly to
>>general-purpose match logic.

An implementation that wanted to support it could enable capturing
from 'body' matches into variables using another extension...

>>Also, if it is a prohibition, shouldn't
>>"MUST NOT" appear there?

   Regular and wildcard expressions used with "body" are exempt  
   from the side effects described in [VARIABLES].  That is, they
   MUST NOT set numbered variables ${1}, ${2}... to the input values
   corresponding to wild card sequences in the matched pattern.
   However, if the extension is present, variable references in the   
   key strings or content type strings are evaluated as described
   in the draft.

(That takes into account a suggestion from Nigel Swinson as well).

>Nigel Swinson wrote:
>>7. Security Considerations
>>I suggest:
>>-   replacement for a virus or spam filtering system.
>>+  replacement for a spam, virus or other security related filtering system.

Done.

>Cyrus Daboo wrote:
>> --On March 28, 2005 15:36:30 +0100 Nigel Swinson 
<...comments on the lack of clarity in matching multipart
types and a suggestion of an example...>

I've inserted the following example into section 4.2:
-----
   Example:
        From: Whomever
        To: Someone
        Date: Whenever
        Subject: whatever
        Content-Type: multipart/mixed; boundary=outer

     &  This is a multi-part message in MIME format.
     &
        --outer
        Content-Type: multipart/alternative; boundary=inner

     &  This is a nested multi-part message in MIME format.
     &
        --inner
        Content-Type: text/plain; charset="us-ascii"

     $  Hello
     $
        --inner
        Content-Type: text/html; charset="us-ascii"

     %  <html><body>Hello</body></html>
     %
        --inner--
     &
     &  This is the end of the inner MIME multipart.
     &
        --outer
        Content-Type: message/rfc822

     !  From: Someone Else
     !  Subject: hello request

     $  Please say Hello
     $
        --outer--
     &
     &  This is the end of the outer MIME multipart.

   In the above example, the '&', '$' and '%' characters at the
   start of a line are used to illustrate what portions of the
   example message are used in tests:

   - the lines starting with '&' are the ones that are tested when
     a 'body :content "multipart" :contains "MIME"'
     test is executed.

   - the lines starting with '$' are the ones that are tested when
     a 'body :content "text/plain" :contains "Hello"' test is
     executed.

   - the lines starting with '%' are the ones that are tested when
     a 'body :content "text/html" :contains "Hello"' test is executed.
   - the lines starting with '$' or '%' are the ones that are tested
     when a 'body :content "text" :contains "Hello"' test is executed.

   - the lines starting with '!' are the ones that are tested when
     a 'body :content "message/rfc822" :contains "Hello"' test is
     executed.
----

>Cyrus Daboo wrote:
...
>> Header: Fix alignment of 'Philip Guenther'
>> 1.3 : 'specifies it syntax' -> 'specifies its syntax'
>> 2.2 : 'with extension' -> 'with the extension'
>> 3.1 : reformat syntax
>> 3.4 : 'all "body" tests fail' -> 'all "body" tests return false'
>> 4.2 : reformat syntax
>> 4.2 : the term 'document' is used to refer to a MIME 'part', I would 
>>       prefer using 'part' in all cases.
>> Appendix B: missing reference [REGEX]

All done

>> 4.2p6 : 'decoded to prior' -> 'decoded prior'

Done.  I've added text to require support for the 7bit, 8bit, and
binary transfer encodings so that the MAY only applies to
not-yet-standardized encodings:
   MIME parts encoded in "quoted-printable" or "base64" content
   transfer encodings MUST be decoded prior to the match.  MIME
   parts in "7bit", "8bit", "binary" content transfer encodings
   MUST be matched as they are.  MIME parts in content transfer
   encodings other than those MAY be decoded, omitted from the test,
   or processed as raw data.

>> 4.3 : just for completeness add an example.

Added:
   Example:
        require ["body", "fileinto"];

        # Save messages mentioning the project schedule in the
        # project/schedule folder.
        if body :text :contains "project schedule" {
                fileinto "project/schedule";
        }

>My comments:
>
> >4.2 Body Transform ":content"
>[...]
> >   If an individual content type contains a '/' (slash), it
> >   specifies a full <type>/<subtype> pair, and matches only
> >   that specific content type.  If it is the empty string, all
> >   MIME content types are matched.  Otherwise, it specifies a
> >   <type> only, and any subtype of that type matches it.
>
>I would like to see ABNF for the content type and some text explaining 
>what should be done if the user specified an invalid value here, e.g. 
>"/". I suspect the answer to this can be: no runtime error, but no match.

I would rather not drag in ABNF just for this single paragraph.
Indeed, I suspect the result would be more difficult to comprehend
when specified that way.  As the only cases not covered by the
current text are values that begin or end with a slash or contain
multiple slashes, I've added an initial case to specify that they
match no content types:

   If an individual content type begins or ends with a '/' (slash) 
   or contains multiple slashes, it matches no content types.  
   Otherwise, if it contains a slash, then it specifies a full
   <type>/<subtype> pair, and matches only that specific content   
   type.  If it is the empty string, all MIME content types are
   matched.  Otherwise, it specifies a <type> only, and any subtype 
   of that type matches it.

At this point, I think there is only one unresolved issue: what is
required for charset conversion when using :content?  As stated
above, my preference would be to update the base spec revision's
secion 2.7.2 to require support for UTF-8, and then simply refer
to that in section 4.2 of the body I-D.

Opinions?

Philip Guenther