Subject: Proposed agenda for the Sieve WG meeting in Paris

From: Alexey Melnikov <[email-masked]>
Date: Tue, 12 Jul 2005 18:54:58 +0100

Alexey Melnikov wrote:

> I've requested a 2 hours slot at Paris IETF. I haven't seen the agenda 
> yet.
> Can I please ask editors to reply directly to me and indicate if the 
> are planning to come, not planning to come or still deciding.

Hi everyone,
Due to lack of any suggestions I came up with the following agenda 
(please comment!):

Agenda

- Introduction (blue sheets, scribe etc) (1 min)
- Agenda Bashing (4 mins)
- Base spec discussion (WGLC comments) (25 mins)
- Vacation draft (WGLC comments) (20 mins)
- Variables draft (issues brought up by security review) (15 mins)
- Body test draft (WGLC comments) (5 mins)
- Edit header draft (WGLC comments) (5 mins)
- IMAP Flags (10 mins)
- Reject draft (5 mins)
- Milestones Updates (10 mins)
- Other business (20 mins)

Total: 120 minutes.

========
Notes:
1). The following drafts will not be discussed, untill their authors 
want to bring up some issues:
- Relation 3431bis
- Subaddress 3598bis
- Spamtestbis
- Regex draft

2). Anybody knows fate of the following 2 drafts?
- Notify draft
- MIME part tests

Alexey,
SIEVE WG co-chair.

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 13 Jul 2005 08:58:23 -0400

Hi Alexey,

> - Other business (20 mins)

Stick spamtest onto the list as well.

-- 
Cyrus Daboo

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 14 Jul 2005 15:17:13 +0100

Updated as per comments from Cyrus and Philip:

Agenda

- Introduction (blue sheets, scribe etc) (1 min)
- Agenda Bashing (4 mins)
- Base spec discussion (WGLC comments) (25 mins)
- Vacation draft (WGLC comments) (20 mins)
- Variables draft (issues brought up by security review) (15 mins)
- Body test draft (WGLC comments) (5 mins)
- Edit header draft (WGLC comments) (5 mins)
- Spamtest draft (5 mins)
- IMAP Flags (10 mins)
- Reject draft (5 mins)
- Date and Index Extensions (draft-freed-sieve-date-index-00.txt) (10 
minutes)
- Milestones Updates (10 mins)
- Other business (5 mins)

Total: 120 minutes.

---

From: Matthew Elvey <[email-masked]>
Date: Sun, 31 Jul 2005 22:18:16 -0700

On 7/14/05 7:17 AM, Alexey Melnikov sent forth electrons to convey:
> ...
Anything you have planned?  Let's set a last call period to end within a 
few weeks.

Current draft:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-refuse-reject-00.txt

I think this draft is ready for a WG last call, assuming the 3028bis 
last call goes well. We started drafting this in 2003. Let's get on with 
it. There have been no issues on-list with the latest draft published; 
this should draw out any final feedback, and is an opportunity to ask 
for positive experiences (e.g. implementors) as well. 
http://wiki.fastmail.fm/index.php/SieveExtensionsSupportMatrix shows 1 
working implementation and 2 more in the works. We're ready to go for 
RFC status, IMHO.

> ...

http://ns.ietf.org/ietf/05aug/sieve.txt:

Tuesday, August 2 at 1815-1945 equals:

http://timeanddate.com/worldclock/fixedtime.html?month=8&day=2&year=2005&hour=18&min=15&sec=0&p1=195

Where will the online chat be?  I plan to login...
Also, that's only a 90 minute slot, not 120...  we'll be OK?

Date: Sun, 31 Jul 2005 22:18:16 -0700
From: Matthew Elvey <[email-masked]>
To: Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Paris, Sieve refuse-reject draft
Sender: [email-masked]

On 7/14/05 7:17 AM, Alexey Melnikov sent forth electrons to convey:
> ...
Anything you have planned?  Let's set a last call period to end within a 
few weeks.

Current draft:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-refuse-reject-00.txt

I think this draft is ready for a WG last call, assuming the 3028bis 
last call goes well. We started drafting this in 2003. Let's get on with 
it. There have been no issues on-list with the latest draft published; 
this should draw out any final feedback, and is an opportunity to ask 
for positive experiences (e.g. implementors) as well. 
http://wiki.fastmail.fm/index.php/SieveExtensionsSupportMatrix shows 1 
working implementation and 2 more in the works. We're ready to go for 
RFC status, IMHO.

> ...

http://ns.ietf.org/ietf/05aug/sieve.txt:

Tuesday, August 2 at 1815-1945 equals:

http://timeanddate.com/worldclock/fixedtime.html?month=8&day=2&year=2005&hour=18&min=15&sec=0&p1=195

Where will the online chat be?  I plan to login...
Also, that's only a 90 minute slot, not 120...  we'll be OK?

Date: Sun, 31 Jul 2005 12:43:47 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Re: Working Group Last Call: draft-ietf-sieve-rfc3598bis-00.txt  (Subaddress)
Sender: [email-masked]

Alexey Melnikov wrote:

> Correction:
>
> Alexey Melnikov wrote:
>
>> I would like to draw your attention to the following draft:
>>
>> <http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt 
>> <http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-00.txt>>
>
> This should be just 
> "<http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt>". 
>
>
>> A two week working group last call of this document starts on 14th 
>> July 2005 (tomorrow) and ends on 28th July 2005 at 6 pm EST.
>>
>> Please review this document and send issues to the list or directly 
>> to the author(s).
>>
>> NB If you do review this document, but have no issues or comments, 
>> please send both co-chairs (or the list) an email to indicate you 
>> have looked at it. This will allow the chairs to gauge the scope of 
>> reviews of this document to aid in our determination on whether to 
>> send it to the IESG. Thanks.
>
The WG Last Call for draft-ietf-sieve-rfc3598bis-00.txt has ended. Can I 
please ask all people who have reviewed the document to send me an 
email. Thanks!

I've seen 2 minor editorial suggestions and a request to describe 
subaddressing structure (left hand side prefix versa suffix) in more 
details. So I think the document is in a good shape, but a new revision 
is needed. I hope we can get away without another WG Last Call.

Alexey
SIEVE WG Co-chair

Cc: [email-masked]
To: Andrei Dumitrescu <[email-masked]>
Date: Thu, 28 Jul 2005 08:56:54 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: ":matches" issue
Sender: [email-masked]

> How should the following test be evaluated when matched against a
> From:user4@localdomain header ?

> if header :matches "from"  "us?"

This test should fail.

> ":matches" should behave like ":is" or like ":contains" ?

:is. You can always add leading and trailing *'s if you want it to work like
:contains. There's even an example showing this in RFC 3028. (Removing implicit
leading and trailing *'s would be a lot harder, which is why it needs to work
this way...)

				Ned

To: [email-masked]
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: ":matches" issue 
Date: Thu, 28 Jul 2005 08:44:47 -0700
Sender: [email-masked]

Andrei Dumitrescu <[email-masked]> writes:
>How should the following test be evaluated when matched against a
>From:user4@localdomain header ?
>
>if header :matches "from"  "us?"
>
>":matches" should behave like ":is" or like ":contains" ?

It should fail; :matches behaves like :is.  If behavior like :contains
is desired, the user should include a star at both ends of the pattern.

This was clarified in the 3028bis I-D in rev -01 or so, where the
description of :matches in section 2.7.1 starts:
   The ":matches" match type specifies a wildcard match using the
   characters "*" and "?"; the entire value must be matched.  <...>

Philip Guenther

Date: Thu, 28 Jul 2005 17:54:00 +0300
From: Andrei Dumitrescu <[email-masked]>
Reply-To: [email-masked]
Organization: Gecad Technologies
To: [email-masked]
Subject: ":matches" issue
OpenPGP: id=C292A9DF
Sender: [email-masked]

How should the following test be evaluated when matched against a
From:user4@localdomain header ?

if header :matches "from"  "us?"

":matches" should behave like ":is" or like ":contains" ?

- --
Andrei DUMITRESCU
Software Test Engineer / AXIGEN
mailto: [email-masked]
Tel: +40-21-303 2080 Fax: +40-21-303 2081
http://www.axigen.com

- ------------------------------------------------
~ This message is confidential. It may also be privileged or otherwise
~ protected by work product immunity or other legal rules. If you have
~ received it by mistake please let us know by reply and then delete it
~ from your system; you should not copy the message or disclose its
~ contents to anyone.

To: Michael Haardt <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-04.txt 
Date: Tue, 26 Jul 2005 21:16:29 -0700
Sender: [email-masked]

Michael Haardt <[email-masked]> writes:
>On Thu, Jul 21, 2005 at 10:02:47AM -0700, Tim Showalter wrote:
...
>> You're right.  The word "Syntax" should be replaced by "Usage" or 
>> something.  I don't think "Semantics" is right, either.
>
>"Usage" sounds much better indeed.

Done for rev -05...

Philip Guenther

From: "Mark E. Mallett" <[email-masked]>
Date: Mon, 25 Jul 2005 17:25:03 -0400
To: Philip Guenther <[email-masked]>
Cc: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

On Mon, Jul 25, 2005 at 01:42:11PM -0700, Philip Guenther wrote:
> "Mark E. Mallett" <[email-masked]> writes:
> >On Thu, Jul 21, 2005 at 12:28:45AM +0200, Kjetil Torgrim Homme wrote:
> >> Modifier ":quoteregex"
> >> 
> >>         Every character with special meaning for :regex (".", "*", "?"
> >>         etc.) is prefixed with "\" in the expansion.  This modifier is
> >>         only available when the "regex" extension is in effect.
> >
> >Do we want to more specifically say something like "Ensures that "
> >[ ...every character with special meaning is prefixed with "\" ... ],
> >i.e. if a character is already quoted, we don't want another quote
> >character to be added (otherwise the quoted character would become
> >unquoted, a literal quote character would be added, etc).
> 
> No!  :quotewildcard/:quoteregex must escape any backslash characters
> that are already present so that the resulting pattern or regexp
> will match them literally.

Right.  Check.  A slip of the brain.

mm

Cc: [email-masked]
To: Mark E Mallett <[email-masked]>
Date: Mon, 25 Jul 2005 13:56:11 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

> [ attaching this comment to a random message in the thread..]

> On Thu, Jul 21, 2005 at 12:28:45AM +0200, Kjetil Torgrim Homme wrote:
> >
> > Modifier ":quoteregex"
> >
> >         Every character with special meaning for :regex (".", "*", "?"
> >         etc.) is prefixed with "\" in the expansion.  This modifier is
> >         only available when the "regex" extension is in effect.

> Do we want to more specifically say something like "Ensures that "
> [ ...every character with special meaning is prefixed with "\" ... ],
> i.e. if a character is already quoted, we don't want another quote
> character to be added

Absolutely not. We want all characters quoted, including quoting
characters. If you don't do this a literal sequence like "\*" ends
up matching a "*", not "\*" as it should. We want "\*" to become
"\\\*" so it matches "\*".

> (otherwise the quoted character would become
> unquoted, a literal quote character would be added, etc).

That's not what happens. See above.

				Ned

To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt 
Date: Mon, 25 Jul 2005 13:42:11 -0700
Sender: [email-masked]

"Mark E. Mallett" <[email-masked]> writes:
>On Thu, Jul 21, 2005 at 12:28:45AM +0200, Kjetil Torgrim Homme wrote:
>> Modifier ":quoteregex"
>> 
>>         Every character with special meaning for :regex (".", "*", "?"
>>         etc.) is prefixed with "\" in the expansion.  This modifier is
>>         only available when the "regex" extension is in effect.
>
>Do we want to more specifically say something like "Ensures that "
>[ ...every character with special meaning is prefixed with "\" ... ],
>i.e. if a character is already quoted, we don't want another quote
>character to be added (otherwise the quoted character would become
>unquoted, a literal quote character would be added, etc).

No!  :quotewildcard/:quoteregex must escape any backslash characters
that are already present so that the resulting pattern or regexp
will match them literally.  For example, the wildcard pattern for
matching the literal text
	foo\*bar
is
	foo\\\*bar
and _not_
	foo\*bar

Ergo, something like
	set :quotawildcard "b" text:
	foo\*bar
	.
must have the same result as
	set "b" text:
	foo\\\*bar
	.

(I used a multiline literal to avoid confusing the issue with the
extra round of backslash escaping that quoted-strings under go.)

Philip Guenther

From: "Mark E. Mallett" <[email-masked]>
Date: Mon, 25 Jul 2005 15:37:45 -0400
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

[ attaching this comment to a random message in the thread..]

On Thu, Jul 21, 2005 at 12:28:45AM +0200, Kjetil Torgrim Homme wrote:
> 
> Modifier ":quoteregex"
> 
>         Every character with special meaning for :regex (".", "*", "?"
>         etc.) is prefixed with "\" in the expansion.  This modifier is
>         only available when the "regex" extension is in effect.

Do we want to more specifically say something like "Ensures that "
[ ...every character with special meaning is prefixed with "\" ... ],
i.e. if a character is already quoted, we don't want another quote
character to be added (otherwise the quoted character would become
unquoted, a literal quote character would be added, etc).

Ditto for :quotewildcard

BTW, I also prefer ":quotematches" to ":quotewildcard" so that the
keyword is more reflective of the match-type name.

mm

Cc: Ned Freed <[email-masked]>, Kjetil Torgrim Homme <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Sat, 23 Jul 2005 20:34:17 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

> Ned Freed <[email-masked]> writes:
> ...
> ...
> >I prefer to have the draft name and date in there, so the ones I have
> >look like this:
> >
> >   [I-D.ietf-sieve-3028bis]
> >              Philip, P. and T. Tim, "Sieve: An Email Filtering
> >              Language", draft-ietf-sieve-3028bis-04 (work in progress),
> >              July 2005, <http://www.ietf.org/internet-drafts/
> >              draft-ietf-sieve-3028bis-04.txt>.

> Umm, please make that:
>    [I-D.ietf-sieve-3028bis]
>               Guenther, P. and T. Showalter, "Sieve: An Email Filtering
>               Language", draft-ietf-sieve-3028bis-04 (work in progress),
>               July 2005, <http://www.ietf.org/internet-drafts/
>               draft-ietf-sieve-3028bis-04.txt>.

Sorry about that; pure carelesness on my part. Fixed now.

> As for the XML, the surname attribute should have the author's family
> name, not his or her personal (given) name.  I would assume that the
> fullname attribute should contain both personal and family name, in
> whatever order is appropriate for the author's culture, but that's just
> a guess.

That's correct.

				Ned

To: Ned Freed <[email-masked]>
Cc: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt 
Date: Sat, 23 Jul 2005 20:03:10 -0700
Sender: [email-masked]

Ned Freed <[email-masked]> writes:
...
...
>I prefer to have the draft name and date in there, so the ones I have
>look like this:
>
>   [I-D.ietf-sieve-3028bis]
>              Philip, P. and T. Tim, "Sieve: An Email Filtering
>              Language", draft-ietf-sieve-3028bis-04 (work in progress),
>              July 2005, <http://www.ietf.org/internet-drafts/
>              draft-ietf-sieve-3028bis-04.txt>.

Umm, please make that:
   [I-D.ietf-sieve-3028bis]
              Guenther, P. and T. Showalter, "Sieve: An Email Filtering
              Language", draft-ietf-sieve-3028bis-04 (work in progress),
              July 2005, <http://www.ietf.org/internet-drafts/
              draft-ietf-sieve-3028bis-04.txt>.

As for the XML, the surname attribute should have the author's family
name, not his or her personal (given) name.  I would assume that the
fullname attribute should contain both personal and family name, in
whatever order is appropriate for the author's culture, but that's just
a guess.

Philip Guenther

Cc: Ned Freed <[email-masked]>, [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 23 Jul 2005 18:42:26 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

> On Wed, 2005-07-20 at 16:00 -0700, Ned Freed wrote:
> > > > On an unrelated point, the [SIEVE] reference needs to be changed to the
> > > > replacement I-D to dodge the "no side-effects in tests" restriction.

> > > well, 3028bis must be published first, anyway, so I'll update the
> > > reference when its done?

> > Might as well update it now as it is likely that the variables draft
> > will be approved before 3028bis comes out as an RFC. The RFC Editor
> > understands how to handle such dependencies and will update the
> > reference for you.

> okay, but I don't know what such a reference should look like.  my best
> guess:

> [SIEVE]
> Guenther, P., "Sieve: An Email Filtering Language", Work in Progress

I prefer to have the draft name and date in there, so the ones I have look like
this:

   [I-D.ietf-sieve-3028bis]
              Philip, P. and T. Tim, "Sieve: An Email Filtering
              Language", draft-ietf-sieve-3028bis-04 (work in progress),
              July 2005, <http://www.ietf.org/internet-drafts/
              draft-ietf-sieve-3028bis-04.txt>.

   [I-D.ietf-sieve-variables]
              Homme, K., "Sieve Mail Filtering Language: Variables
              Extension", draft-ietf-sieve-variables-04 (work in
              progress), July 2005, <http://www.ietf.org/
              internet-drafts/draft-ietf-sieve-variables-04.txt>.

In case anyone cares, the way to get this output from xml2rfc is with
something like:

<reference anchor='I-D.ietf-sieve-3028bis'
           target='http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-04.txt'>
<front>
<title>Sieve: An Email Filtering Language</title>
<author initials='P' surname='Philip' fullname='Guenther'>
    <organization>Sendmail, Inc.</organization>
</author>
<author initials='T' surname='Tim' fullname='Showalter'>
    <organization/>
</author>
<date month='July' day='18' year='2005' />
</front>
<seriesInfo name='Internet-Draft' value='draft-ietf-sieve-3028bis-04' />
</reference>

<reference anchor='I-D.ietf-sieve-variables'
           target='http://www.ietf.org/internet-drafts/draft-ietf-sieve-variables-04.txt'>
<front>
<title>Sieve Mail Filtering Language: Variables Extension</title>
<author initials='K' surname='Homme' fullname='Kjetil Homme'>
    <organization>University of Oslo</organization>
</author>
<date month='July' day='15' year='2005' />
</front>
<seriesInfo name='Internet-Draft' value='draft-ietf-sieve-variables-04' />
</reference>

On a side note, I'm seeing an increasing tendency to try and avoid referencing
internet-drafts even in cases where such a reference is clearly appropriate,
and I cannot say I like it. While I understand the desire to avoid unnecessary
document dependencies in the RFC Editor queue, IMO the risk of the wrong
reference slipping into the final document far outweighs the delays
dependencies can cause. If there's a problem where documents are blocked
unnecessarily even after dependencies - and I've seen some indications that
such a problem does in fact exist - we need to deal with that problem rather
than gloss over it in this way.

> > I have one additional suggestion. Thinking about it, I don't like the name
> > choices here. Specifically, ":quotewildcard" refers to the characters being
> > quoted and not the context they are being quoted for, while ":quoteregex"
> > refers to the context they are being quoted for and not the characters being
> > quoted. How about changing ":quotewildcard" to ":quotematches" so both
> > names refer to the context?

> :quotematches was the original suggested name, and it seems logical in
> tandem with :quoteregex, but it sounds awfully misleading on its own.

>    SET :quotematches "address" "[${1}]"

> it's tempting to think that the modifier will only affect the match
> variable, IMHO.

Not to me, but since it appears to be to others I'll let this one drop.

				Ned

content-class: urn:content-classes:message
Subject: Out of Office AutoReply: <Possible SPAM> hello
Date: Sat, 23 Jul 2005 19:39:57 +0300
From: "Romascanu, Dan \(Dan\)" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

This is a multi-part message in MIME format.

------_=_NextPart_001_01C58FA5.290ED7E6
Content-Type: text/plain;
	charset="windows-1255"
Content-Transfer-Encoding: quoted-printable

I am out-of-office, on a business trip, untill July 23, 2005, and may be =
able to read your e-mail only after this date. If you need to contact me =
urgently, please call the mobile phone number +1-917-957-0270.

Regards,

Dan

------_=_NextPart_001_01C58FA5.290ED7E6
Content-Type: text/html;
	charset="windows-1255"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dwindows-1255">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
6.0.6603.0">
<TITLE>Out of Office AutoReply: &lt;Possible SPAM&gt; hello</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/plain format -->

<P><FONT SIZE=3D2>I am out-of-office, on a business trip, untill July =
23, 2005, and may be able to read your e-mail only after this date. If =
you need to contact me urgently, please call the mobile phone number =
+1-917-957-0270.<BR>
<BR>
Regards,<BR>
<BR>
Dan<BR>
</FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C58FA5.290ED7E6--

Date: Fri, 22 Jul 2005 10:34:46 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-04.txt
Sender: [email-masked]

On Thu, Jul 21, 2005 at 02:43:13PM -0700, Ned Freed wrote:
> As to the idea of actually using formal grammar for this sort of thing, been
> there, done that, bad idea. Having been on the receiving end of questions 
> for
> RFC 1521 issues, I can tell you that using subgrammars (or whatever the 
> correct
> term for this is) creates about 10X more confusion than clarity.

Too bad, but I guess a paragraph like

  All Sieve commands, including extensions, MUST be words of the following
  generic grammar with the start symbol "start".  They SHOULD be specified   
  using a specific grammar, though.

is not going to help most people a lot, despite specifying *exactly*
the same as RFC 3028.

> The simple fact of the matter is that people just aren't that good at
> extracting usage information from ABNF. THis isn't to say we should not 
> have a
> precise definition of the overall syntax in ABNF - implementors at the very
> least need it - but using ABNF to provide usage information' goes too far. 

I built a full ABNF for my implementation to move as much as possible
from semantic analysis to syntactic analysis, but that is just one
possible approach.  I append my implementation notes, but they are
not meant as suggested change to the draft, but to illustrate the step
between RFC 3028 and actual code.  The { } specification is pretty
weak and just "worked for me".

I am pretty sure others use the generic ABNF for syntactic analysis,
following straightforward RFC 3028.  That is well possible and it does
not need a specific grammar.

> The
> form we use now is simple, easy to read, and gets the job done.

Once "Syntax" is replaced by "Usage", it is fine with me and I shall
agree to the next released draft.

Michael
----------------------------------------------------------------------
The grammar is specified in ABNF with two extensions to describe tagged
arguments that can be reordered and grammar extensions: { } denotes a
sequence of symbols that may appear in any order.  Example:

  options = a b c
  start =  { options }

is equivalent to:

  start =  ( a b c ) / ( a c b ) / ( b a c ) / ( b c a ) / ( c a b ) / ( c b a )

The symbol =) is used to append to a rule:

  start =  a
  start =) b

is equivalent to

  start =  a b

All Sieve commands, including extensions, MUST be words of the following
generic grammar with the start symbol "start".  They SHOULD be specified
using a specific grammar, though.

   argument        = string-list / number / tag
   arguments       = *argument [test / test-list]
   block           = "{" commands "}"
   commands        = *command
   string          = quoted-string / multi-line
   string-list     = "[" string *("," string) "]" / string
   test            = identifier arguments
   test-list       = "(" test *("," test) ")"
   command         = identifier arguments ( ";" / block )
   start           = command

The basic Sieve commands are specified using the following grammar, which
language is a subset of the generic grammar above.  The start symbol is
"start".

  address-part     =  ":localpart" / ":domain" / ":all"
  comparator       =  ":comparator" string
  match-type       =  ":is" / ":contains" / ":matches"
  string           =  quoted-string / multi-line
  string-list      =  "[" string *("," string) "]" / string
  address-test     =  "address" { [address-part] [comparator] [match-type] }
                      string-list string-list
  test-list        =  "(" test *("," test) ")"
  allof-test       =  "allof" test-list
  anyof-test       =  "anyof" test-list
  exists-test      =  "exists" string-list
  false-test       =  "false"
  true=test        =  "true"
  header-test      =  "header" { [comparator] [match-type] }
                      string-list string-list
  not-test         =  "not" test
  relop            =  ":over" / ":under"
  size-test        =  "size" relop number
  block            =  "{" commands "}"
  if-command       =  "if" test block *( "elsif" test block ) [ "else" block ]
  stop-command     =  "stop" { stop-options } ";"
  stop-options     =
  keep-command     =  "keep" { keep-options } ";"
  keep-options     =
  discard-command  =  "discard" { discard-options } ";"
  discard-options  =
  redirect-command =  "redirect" { redirect-options } string ";"
  redirect-options =
  require-command  =  "require" { require-options } string-list ";"
  require-options  =
  test             =  address-test / allof-test / anyof-test / exists-test
                      / false-test / true-test / header-test / not-test
                      / size-test
  command          =  if-command / stop-command / keep-command
                      / discard-command / redirect-command
  commands         =  *command
  start            =  *require-command commands

The extensions "envelope" and "fileinto" are specified using the following
grammar extension.

  envelope-test    =  "envelope" { [comparator] [address-part] [match-type] }
                      string-list string-list
  test             =/ envelope-test

  fileinto-command =  "fileinto" { fileinto-options } string ";"
  fileinto-options =
  command          =/ fileinto-command

The extension "copy" is specified as:

  fileinto-options =) ":copy"
  redirect-options =) ":copy"

And so on ...

Date: Fri, 22 Jul 2005 10:03:53 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-04.txt
Sender: [email-masked]

On Thu, Jul 21, 2005 at 10:02:47AM -0700, Tim Showalter wrote:
> Michael Haardt wrote:
> >Section 1.1:
> >
> >   Each section on a command (test, action, or control) has a line
> >   labeled "Syntax:".  This line describes the syntax of the command,
> >   including its name and its arguments.
> >
> >If this line described syntax, its contents would be listed in the
> >grammar.  There is a number of RFCs that list specific rules throughout
> >the document, explaining them, and all of them are listed together in
> >the full grammar.  That's _not_ the case for Sieve.
> 
> You're right.  The word "Syntax" should be replaced by "Usage" or 
> something.  I don't think "Semantics" is right, either.

"Usage" sounds much better indeed.

Michael

Cc: Michael Haardt <[email-masked]>, [email-masked]
To: Tim Showalter <[email-masked]>
Date: Thu, 21 Jul 2005 14:43:13 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-04.txt
Sender: [email-masked]

> Michael Haardt wrote:
> > Section 1.1:
> >
> >    Each section on a command (test, action, or control) has a line
> >    labeled "Syntax:".  This line describes the syntax of the command,
> >    including its name and its arguments.
> >
> > If this line described syntax, its contents would be listed in the
> > grammar.  There is a number of RFCs that list specific rules throughout
> > the document, explaining them, and all of them are listed together in
> > the full grammar.  That's _not_ the case for Sieve.

> You're right.  The word "Syntax" should be replaced by "Usage" or
> something.  I don't think "Semantics" is right, either.

"Usage:" would be my choice - it matches what's commonly used in many other
sorts of documents for this stuff. I'm going to switch the vacation and
date/index drafts to use "Usage:" for now.

As to the idea of actually using formal grammar for this sort of thing, been
there, done that, bad idea. Having been on the receiving end of questions for
RFC 1521 issues, I can tell you that using subgrammars (or whatever the correct
term for this is) creates about 10X more confusion than clarity.

The simple fact of the matter is that people just aren't that good at
extracting usage information from ABNF. THis isn't to say we should not have a
precise definition of the overall syntax in ABNF - implementors at the very
least need it - but using ABNF to provide usage information' goes too far. The
form we use now is simple, easy to read, and gets the job done.

				Ned

Date: Thu, 21 Jul 2005 10:02:47 -0700
From: Tim Showalter <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-04.txt
Sender: [email-masked]

Michael Haardt wrote:
> Section 1.1:
> 
>    Each section on a command (test, action, or control) has a line
>    labeled "Syntax:".  This line describes the syntax of the command,
>    including its name and its arguments.
> 
> If this line described syntax, its contents would be listed in the
> grammar.  There is a number of RFCs that list specific rules throughout
> the document, explaining them, and all of them are listed together in
> the full grammar.  That's _not_ the case for Sieve.

You're right.  The word "Syntax" should be replaced by "Usage" or 
something.  I don't think "Semantics" is right, either.

Tim

Date: Thu, 21 Jul 2005 14:52:28 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

Philip Guenther wrote:

>>>The lack of a full list of regex specials is a bit awkward here, but I
>>>guess I can live with it.
>>>      
>>>
>>there is no ABNF in [REGEX] to refer to, and a full list will be prone
>>to inaccuracies.  not that I expect any changes to the syntax allowed by
>>it, but ...
>>    
>>
>
>That would be an argument for moving :quoteregex to the regex draft.
>
I agree.

 >I don't see any wording that limits support for :quotaregex to when the
 >regex extension is also required, so even implementations that don't
 >support regex will need to support :quoteregex.

Right, I don't think this should be required for a bare "variables" 
implementation.

 >To me, that means that this document needs to specify exactly what its 
behavior is.  (Yeah, you
 >could make [REGEX] a normative reference, but ... ick.)

To me ":quoteregex" as written says that REGEX is a normative reference.

I want to repeat and maybe rephrase something I've said to Kjetil 
off-list: I don't think the variables draft should depend on documents 
that might take undetermined amount of time to complete.

Alexey

-- 
Alexey Melnikov
__________________________________________
Isode M-Box Message Store developer
http://www.isode.com/products/m-box.html

IETF standard related pages:
http://www.melnikov.ca/mel/devel/Links.html

Personal Home Page: http://www.melnikov.ca
__________________________________________

To: [email-masked]
Subject: Re: I-D ACTION:draft-ietf-sieve-3028bis-04.txt
From: Michael Haardt <[email-masked]>
Date: Thu, 21 Jul 2005 10:27:48 +0200
Sender: [email-masked]

Hello,

now there is only one open issue left on my list:

Section 1.1:

   Each section on a command (test, action, or control) has a line
   labeled "Syntax:".  This line describes the syntax of the command,
   including its name and its arguments.

If this line described syntax, its contents would be listed in the
grammar.  There is a number of RFCs that list specific rules throughout
the document, explaining them, and all of them are listed together in
the full grammar.  That's _not_ the case for Sieve.

RFC 3028 confuses syntax and semantics sometimes.  It uses a generic
grammar as syntax for commands and tests and performs many checks during
semantic analysis.  Syntax is specified by grammar rules, semantics by
natural language.  Section 1.1 should either say "Semantics" or there
should be two grammars: A generic grammar (the one currently given)
and a specific grammar.  The latter could be written by introducing a
new structure for sets of elements that may appear in any order.

Michael

Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Thu, 21 Jul 2005 01:52:52 +0200
Sender: [email-masked]

On Wed, 2005-07-20 at 16:00 -0700, Ned Freed wrote:
> > > On an unrelated point, the [SIEVE] reference needs to be changed to the
> > > replacement I-D to dodge the "no side-effects in tests" restriction.
> 
> > well, 3028bis must be published first, anyway, so I'll update the
> > reference when its done?
> 
> Might as well update it now as it is likely that the variables draft
> will be approved before 3028bis comes out as an RFC. The RFC Editor
> understands how to handle such dependencies and will update the
> reference for you.

okay, but I don't know what such a reference should look like.  my best
guess:

[SIEVE]
Guenther, P., "Sieve: An Email Filtering Language", Work in Progress

> I have one additional suggestion. Thinking about it, I don't like the name
> choices here. Specifically, ":quotewildcard" refers to the characters being 
> quoted and not the context they are being quoted for, while ":quoteregex"
> refers to the context they are being quoted for and not the characters being
> quoted. How about changing ":quotewildcard" to ":quotematches" so both
> names refer to the context?

:quotematches was the original suggested name, and it seems logical in
tandem with :quoteregex, but it sounds awfully misleading on its own.

   SET :quotematches "address" "[${1}]"

it's tempting to think that the modifier will only affect the match
variable, IMHO.

the argument to :regex _is_ a "regexp"
the argument to :matches isn't a "match", that's the result, but it
_contains_ wildcards.

one alternative is :quotepattern, but I don't think that's any
clearer. :quotematchespattern is too long...

-- 
Kjetil T.

Cc: Philip Guenther <[email-masked]>, [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 20 Jul 2005 16:00:05 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

> On Wed, 2005-07-20 at 14:54 -0700, Philip Guenther wrote:
> > Kjetil Torgrim Homme <[email-masked]> writes:
> > >there is no ABNF in [REGEX] to refer to, and a full list will be prone
> > >to inaccuracies.  not that I expect any changes to the syntax allowed by
> > >it, but ...
> >
> > That would be an argument for moving :quoteregex to the regex draft.  I
> > don't see any wording that limits support for :quotaregex to when the
> > regex extension is also required, so even implementations that don't
> > support regex will need to support :quoteregex.

> ouch, you're right.  suggestion:

> Modifier ":quoteregex"

>         Every character with special meaning for :regex (".", "*", "?"
>         etc.) is prefixed with "\" in the expansion.  This modifier is
>         only available when the "regex" extension is in effect.

Seems reasonable.

> > To me, that means that
> > this document needs to specify exactly what its behavior is.  (Yeah, you
> > could make [REGEX] a normative reference, but ... ick.)
> >
> > (Hmm, what if the regex extension is revised to support not just the
> > normal ("extended") POSIX regexps but also the old "basic" regexps?  The
> > quoting rules for the two are different.  (Basic regexps are _not_ a
> > subset of extended regexps for functionality, though the GNU
> > implementations have blurred that fact by extending them both.))

> :quoteregex specifically works for :regex (and the added sentence above
> locks the behavioural description to [REGEX]), if an :basicregex is
> introduced, there will be no quoting modifier for it.

> > Anyway, if :quoteregex isn't moved, then the minimal list of characters
> > that are escaped by :quoteregex is:
> >
> > \ . [ ^ $ ( ) { * + ? |

> that's just too ugly a duplication of specification.  if we need to add
> that, I'd rather split out the :regex bits.

> > On an unrelated point, the [SIEVE] reference needs to be changed to the
> > replacement I-D to dodge the "no side-effects in tests" restriction.

> well, 3028bis must be published first, anyway, so I'll update the
> reference when its done?

Might as well update it now as it is likely that the variables draft
will be approved before 3028bis comes out as an RFC. The RFC Editor
understands how to handle such dependencies and will update the
reference for you.

I have one additional suggestion. Thinking about it, I don't like the name
choices here. Specifically, ":quotewildcard" refers to the characters being 
quoted and not the context they are being quoted for, while ":quoteregex"
refers to the context they are being quoted for and not the characters being
quoted. How about changing ":quotewildcard" to ":quotematches" so both
names refer to the context?

				Ned

Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: [email-masked]
Date: Thu, 21 Jul 2005 00:28:45 +0200
Sender: [email-masked]

On Wed, 2005-07-20 at 14:54 -0700, Philip Guenther wrote:
> Kjetil Torgrim Homme <[email-masked]> writes:
> >there is no ABNF in [REGEX] to refer to, and a full list will be prone
> >to inaccuracies.  not that I expect any changes to the syntax allowed by
> >it, but ...
> 
> That would be an argument for moving :quoteregex to the regex draft.  I
> don't see any wording that limits support for :quotaregex to when the
> regex extension is also required, so even implementations that don't
> support regex will need to support :quoteregex.

ouch, you're right.  suggestion:

Modifier ":quoteregex"

        Every character with special meaning for :regex (".", "*", "?"
        etc.) is prefixed with "\" in the expansion.  This modifier is
        only available when the "regex" extension is in effect.

> To me, that means that
> this document needs to specify exactly what its behavior is.  (Yeah, you
> could make [REGEX] a normative reference, but ... ick.)
> 
> (Hmm, what if the regex extension is revised to support not just the
> normal ("extended") POSIX regexps but also the old "basic" regexps?  The
> quoting rules for the two are different.  (Basic regexps are _not_ a
> subset of extended regexps for functionality, though the GNU
> implementations have blurred that fact by extending them both.))

:quoteregex specifically works for :regex (and the added sentence above
locks the behavioural description to [REGEX]), if an :basicregex is
introduced, there will be no quoting modifier for it.

> Anyway, if :quoteregex isn't moved, then the minimal list of characters
> that are escaped by :quoteregex is:
> 
> \ . [ ^ $ ( ) { * + ? |

that's just too ugly a duplication of specification.  if we need to add
that, I'd rather split out the :regex bits.

> On an unrelated point, the [SIEVE] reference needs to be changed to the
> replacement I-D to dodge the "no side-effects in tests" restriction.

well, 3028bis must be published first, anyway, so I'll update the
reference when its done?
-- 
Kjetil T.

To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt 
Date: Wed, 20 Jul 2005 14:54:50 -0700
Sender: [email-masked]

Kjetil Torgrim Homme <[email-masked]> writes:
>On Wed, 2005-07-20 at 10:51 -0700, Ned Freed wrote:
...
>> 4.1.2.2.  Modifier ":quoteregex"
>> 
>>    Every character with special meaning for :regex (".", "*", "?" etc.)
>>    is prefixed with "
>> 
>> I assume you meant '"\"' and not '"'. Looks like you need to apply
>> some quoting to the draft source...
>
>yeah, I always forget that \ must be written \e in roff.
>
>> The lack of a full list of regex specials is a bit awkward here, but I
>> guess I can live with it.
>
>there is no ABNF in [REGEX] to refer to, and a full list will be prone
>to inaccuracies.  not that I expect any changes to the syntax allowed by
>it, but ...

That would be an argument for moving :quoteregex to the regex draft.  I
don't see any wording that limits support for :quotaregex to when the
regex extension is also required, so even implementations that don't
support regex will need to support :quoteregex.  To me, that means that
this document needs to specify exactly what its behavior is.  (Yeah, you
could make [REGEX] a normative reference, but ... ick.)

(Hmm, what if the regex extension is revised to support not just the
normal ("extended") POSIX regexps but also the old "basic" regexps?  The
quoting rules for the two are different.  (Basic regexps are _not_ a
subset of extended regexps for functionality, though the GNU
implementations have blurred that fact by extending them both.))

Anyway, if :quoteregex isn't moved, then the minimal list of characters
that are escaped by :quoteregex is:

\ . [ ^ $ ( ) { * + ? |

On an unrelated point, the [SIEVE] reference needs to be changed to the
replacement I-D to dodge the "no side-effects in tests" restriction.

Philip Guenther

Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Wed, 20 Jul 2005 21:15:10 +0200
Sender: [email-masked]

On Wed, 2005-07-20 at 10:51 -0700, Ned Freed wrote:
> Basic idea looks good, however, a couple of typos have crept in. First, the
> syntax at the beginning of 4.1 fails to list the two new modifiers.

doh!  thanks.

> Second, the plain text draft now says:
> 
> 4.1.2.1.  Modifier ":quotewildcard"
> 
>    Every character with special meaning for :matches ("*", "?" and " is
>    prefixed with "

doh! it was supposed to say:

        Every character with special meaning for :matches ("*", "?" and
        "\") is prefixed with "\" in the expansion.

> 4.1.2.2.  Modifier ":quoteregex"
> 
>    Every character with special meaning for :regex (".", "*", "?" etc.)
>    is prefixed with "
> 
> I assume you meant '"\"' and not '"'. Looks like you need to apply some quoting
> to the draft source...

yeah, I always forget that \ must be written \e in roff.

> The lack of a full list of regex specials is a bit awkward here, but I
> guess I can live with it.

there is no ABNF in [REGEX] to refer to, and a full list will be prone
to inaccuracies.  not that I expect any changes to the syntax allowed by
it, but ...

> Finally, the section ends with "Using two or more modifiers of the same
> precedence is a syntax error.". This isn't quite right: These aren't syntax
> errors. How about saying "It is an error to use two or more modifiers of the
> same precedence in a single SET operation" or something similar.

thanks.  I used "command" rather than "operation" to avoid using new
terminology.

I also downgraded refering to an unknown namespace to just an "error"
rather than a "syntax error".

> > I did not remove the text about interaction with regex yet.  I feel the
> > text belongs in the variables draft, so if it is procedurally possible,
> > I prefer it stays there rather than move into the regex draft.
> 
> I also would like to keep the text here if possible.

I guess we'll decide this in Paris?
-- 
Kjetil T.

Cc: [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 20 Jul 2005 10:51:42 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
Sender: [email-masked]

> On Fri, 2005-07-15 at 10:50 -0400, [email-masked] wrote:
> > A URL for this Internet-Draft is:
> > http://www.ietf.org/internet-drafts/draft-ietf-sieve-variables-04.txt

> the changes are:

> * added :quotewildcard and :quoteregex mofiefiers to SET.

Basic idea looks good, however, a couple of typos have crept in. First, the
syntax at the begining of 4.1 fails to list the two new modifiers. Second, the
plain text draft now says:

4.1.2.1.  Modifier ":quotewildcard"

   Every character with special meaning for :matches ("*", "?" and " is
   prefixed with "

4.1.2.2.  Modifier ":quoteregex"

   Every character with special meaning for :regex (".", "*", "?" etc.)
   is prefixed with "

I assume you meant '"\"' and not '"'. Looks like you need to apply some quoting
to the draft source...

The lack of a full list of regex specials is a bit awkward here, but I
guess I can live with it.

Finally, the section ends with "Using two or more modifiers of the same
precedence is a syntax error.". This isn't quite right: These aren't syntax
errors. How about saying "It is an error to use two or more modifiers of the
same precedence in a single SET operation" or something similar.

> * added this paragraph to security considerations:

>         Since values stored by SET exceeding implementation limits are
>         silently truncated, it's not appropriate to store large
>         structures with security implications in variables.

> I did not remove the text about interaction with regex yet.  I feel the
> text belongs in the variables draft, so if it is procedurally possible,
> I prefer it stays there rather than move into the regex draft.

I also would like to keep the text here if possible.

				Ned

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-spamtestbis-01.txt 
Date: Tue, 19 Jul 2005 15:50:01 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: SIEVE Email Filtering: Spamtest and Virustest Extensions
	Author(s)	: C. Daboo
	Filename	: draft-ietf-sieve-spamtestbis-01.txt
	Pages		: 10
	Date		: 2005-7-19
	
The SIEVE email filtering language "spamtest", "spamtestplus" and
   "virustest" extensions permit users to use simple, portable commands
   for spam and virus tests on email messages.  Each extension provides
   a new test using matches against numeric 'scores'.  It is the
   responsibility of the underlying SIEVE implementation to do the
   actual checks that result in values returned by the tests.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-spamtestbis-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-spamtestbis-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-spamtestbis-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-spamtestbis-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-spamtestbis-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Tue, 19 Jul 2005 12:29:29 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Working Group Last Call: draft-ietf-sieve-3028bis-04.txt (Sieve base)
Sender: [email-masked]

I would like to draw your attention to the following draft:

<http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-04.txt>

A two week working group last call of this document starts today, 19th 
July 2005 and ends on 2nd August 2005 at 6 pm EST.

Please review this document and send issues to the list or direct to the 
author(s).

NB If you do review this document, but have no issues or comments, 
please send both co-chairs (or the list) an email to indicate you have 
looked at it. This will allow the chairs to gauge the scope of reviews 
of this document to aid in our determination on whether to send it to 
the IESG. Thanks.

--
Alexey Melnikov
SIEVE WG Co-chair

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-3028bis-04.txt 
Date: Mon, 18 Jul 2005 18:50:02 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve: An Email Filtering Language
	Author(s)	: T. Showalter, P. Guenther
	Filename	: draft-ietf-sieve-3028bis-04.txt
	Pages		: 37
	Date		: 2005-7-18
	
This document describes a language for filtering email messages at
   time of final delivery.  It is designed to be implementable on either
   a mail client or mail server.  It is meant to be extensible, simple,
   and independent of access protocol, mail architecture, and operating
   system.  It is suitable for running on a mail server where users may
   not be allowed to execute arbitrary programs, such as on black box
   Internet Message Access Protocol (IMAP) servers, as it has no
   variables, loops, or ability to shell out to external programs.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-04.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-3028bis-04.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-3028bis-04.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-3028bis-04.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-3028bis-04.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Sat, 16 Jul 2005 15:13:37 +0100
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: [email-masked]
Subject: Re: NUL handling and security considerations [was: Re: My open issues with RFC3028bis]
Sender: [email-masked]

Ned Freed wrote:

>>>I may be stretching it too far here, but AFAIK, there are implementations
>>>that truncate strings, thus corrupting test results.  Trying to label them
>>>non-conforming probably won't succeed, but we should not silently ignore
>>>this problem.
>>>      
>>>
>>I guess there are two choices:
>>A) Require correct handling of NUL
>>B) Strongly prefer correct handling of NUL and warn about the dangers of
>>   not doing so in the security considerations
>>    
>>
>I have no major problem with A but I think B is a better choice. FWIW, the
>implementation I work on has no problem handling NULs, but I worry that
>this will make many other implementations non-conforming.
>
I suspect that Cyrus Sieve doesn't handle encoded NULs properly. So I 
would prefer B.

Subject: Re: I-D ACTION:draft-ietf-sieve-variables-04.txt
From: Kjetil Torgrim Homme <[email-masked]>
To: [email-masked]
Date: Sat, 16 Jul 2005 16:09:36 +0200
Sender: [email-masked]

On Fri, 2005-07-15 at 10:50 -0400, [email-masked] wrote:
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-ietf-sieve-variables-04.txt

the changes are:

* added :quotewildcard and :quoteregex mofiefiers to SET.
* added this paragraph to security considerations:

        Since values stored by SET exceeding implementation limits are
        silently truncated, it's not appropriate to store large
        structures with security implications in variables.

I did not remove the text about interaction with regex yet.  I feel the
text belongs in the variables draft, so if it is procedurally possible,
I prefer it stays there rather than move into the regex draft.
-- 
Kjetil T.

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-body-02.txt 
Date: Fri, 15 Jul 2005 15:50:02 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Email Filtering: Body Extension
	Author(s)	: P. Guenther, J. Degener
	Filename	: draft-ietf-sieve-body-02.txt
	Pages		: 0
	Date		: 2005-7-15
	
This document defines a new primitive for the "Sieve" email
   filtering language that tests for the occurrence of one or more
   strings in the body of an email message.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-02.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-body-02.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-body-02.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-body-02.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-body-02.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-variables-04.txt 
Date: Fri, 15 Jul 2005 10:50:01 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Mail Filtering Language: Variables Extension
	Author(s)	: K. Homme
	Filename	: draft-ietf-sieve-variables-04.txt
	Pages		: 16
	Date		: 2005-7-15
	
In advanced filtering rule sets, it is useful to keep state or
   configuration details across rules.  This extension changes the
   interpretation of strings, adds an action to store data in variables,
   and supplies a new test so that the value of a string can be
   examined.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-variables-04.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-variables-04.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-variables-04.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-variables-04.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-variables-04.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Fri, 15 Jul 2005 13:39:10 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

Philip Guenther wrote:

>>In case you ask
>>how easy it would be to upgrade existing Sieve implementations to support
>>"auth", if nothing else to return an empty string: I will implement that
>>in Exim these days.  Cyrus already has it and one other implementation,
>>too.
>>    
>>
>Umm, what's the point in implementing it to always return the empty
>string?  IMHO, an implementation should only advertise and support
>envelope-auth if it implements and is configured to support SMTP-AUTH.
>  
>
I agree.

Date: Fri, 15 Jul 2005 13:31:24 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: Alexandros Vellis <[email-masked]>, [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

Philip Guenther wrote:

>Anyway, for now I've amended the first paragraph of section 5.4 to read:
>
>   The "envelope" test is true if the specified part of the SMTP (or
>   equivalent) envelope matches the specified key.  This specification
>   defines the interpretation of the (case insensitive) "from" and "to"
>   envelope-parts.  Additional envelope-parts may be defined by other
>   extensions; implementations SHOULD consider unknown envelope parts an
>   error.
>
>
>Perhaps I'm misreading the work/pain implied by the SMTP-AUTH reference.
>Do those with more experience think it'll be easy?
>  
>
I personally think that it would be quite difficult to move the whole 
chain of SMTP-AUTH references (SASL (2222bis), ...) to Draft. So a 
separate document would be better.

To: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
From: Michael Haardt <[email-masked]>
Date: Fri, 15 Jul 2005 12:19:32 +0200
Sender: [email-masked]

> SMTP-AUTH is currently at proposed; while Rob Siemborski has started the
> ball rolling on revising it (draft-siemborski-rfc2554bis-04.txt), its
> normative references form a pretty hefty dependency tree.  I personally
> believe we'll be able to advance the sieve base spec to draft before
> SMTP-AUTH gets there.

I see.  So basically it's the decision of getting the Sieve base
specification out the door soon and have an envelope-auth extension,
or possibly wait a long time just due to dependencies, plus being unable
to advance any of the Sieve extensions, because they reference the Sieve
base extension?

Sounds like Usenet hierarchy modification decisions. :-/

> Umm, what's the point in implementing it to always return the empty
> string?  IMHO, an implementation should only advertise and support
> envelope-auth if it implements and is configured to support SMTP-AUTH.

Right now, we appear to have three kinds of implementations:

o  Anything but "to" and "from" returns an error.
o  Anything but "to" and "from" is being ignored.
o  Anything but "to", "from" and "auth" is being ignored.

RFC 3028 appears to allow all three, because it simply does not specify
what happens if anything but "to" and "from" is used, and that is not
good.

Your suggestion puts a SHOULD on the first option, which does not cause
the other options to violate the spec.  It just specifies previously
unspecified behaviour.  It may as well define what "auth" does (apart
from the formal reasons above).

> If the AUTH= parameter wasn't used, shouldn't the envelope "auth" part
> never match and _not_ be the empty string, like an absent header used
> with the "address" test?

Correct me, if I am wrong, but as I see it

  MAIL FROM: <addr-spec> AUTH=<>

means the same as

  MAIL FROM: <addr-spec>

although the first example formally specifies the authenticated sender.
MTAs are in fact free not to trust AUTH=, acting as if AUTH=<> or no
AUTH parameter was being passed.  Treating all three ways as setting
the authenticated sender to the empty string makes sense to me.

If an implementation always returns the empty string, it means it never
trusts the authenticated sender information.  That is perfectly legal,
so implementing "auth", should the base specification require to do so,
is very easy.  I don't think anybody would in fact do that, but it is
an easy upgrade path for existing implementations for being compliant
with the new spec.

Michael

To: Michael Haardt <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: 3028bis open issue #2: unknown envelope-part names 
Date: Fri, 15 Jul 2005 02:35:32 -0700
Sender: [email-masked]

Michael Haardt <[email-masked]> writes:
>On Thu, Jul 14, 2005 at 08:00:11PM -0700, Philip Guenther wrote:
...
>> Perhaps I'm misreading the work/pain implied by the SMTP-AUTH reference.
>> Do those with more experience think it'll be easy?
>
>I am not sure I understand what you are talking about.  <...>

I was referring to the procedural issues it creates.  A protocol cannot
advance any further on the standard track (proposed->draft->full) than
the protocols/documents for which it has normative references.

SMTP-AUTH is currently at proposed; while Rob Siemborski has started the
ball rolling on revising it (draft-siemborski-rfc2554bis-04.txt), its
normative references form a pretty hefty dependency tree.  I personally
believe we'll be able to advance the sieve base spec to draft before
SMTP-AUTH gets there.

>In case you ask
>how easy it would be to upgrade existing Sieve implementations to support
>"auth", if nothing else to return an empty string: I will implement that
>in Exim these days.  Cyrus already has it and one other implementation,
>too.

Umm, what's the point in implementing it to always return the empty
string?  IMHO, an implementation should only advertise and support
envelope-auth if it implements and is configured to support SMTP-AUTH.

>I don't think there are disagreements on its specification.  I understood
>it that way: For SMTP, it is the AUTH parameter to MAIL FROM, in case
>the mail system trusts that information (RFC 2554, section 5).  For other
>protocols, it is the authenticated sender address.  If no authenticated
>sender address is available, it is the empty string.  Feel free to correct
>me, if I am wrong.

If the AUTH= parameter wasn't used, shouldn't the envelope "auth" part
never match and _not_ be the empty string, like an absent header used
with the "address" test?

Philip Guenther

Date: Fri, 15 Jul 2005 10:51:11 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

On Thu, Jul 14, 2005 at 08:00:11PM -0700, Philip Guenther wrote:
> Anyway, for now I've amended the first paragraph of section 5.4 to read:
> 
>    The "envelope" test is true if the specified part of the SMTP (or
>    equivalent) envelope matches the specified key.  This specification
>    defines the interpretation of the (case insensitive) "from" and "to"
>    envelope-parts.  Additional envelope-parts may be defined by other
>    extensions; implementations SHOULD consider unknown envelope parts an
>    error.

Good, that closes this long-standing issue.

> Perhaps I'm misreading the work/pain implied by the SMTP-AUTH reference.
> Do those with more experience think it'll be easy?

I am not sure I understand what you are talking about.  In case you ask
how easy it would be to upgrade existing Sieve implementations to support
"auth", if nothing else to return an empty string: I will implement that
in Exim these days.  Cyrus already has it and one other implementation,
too.

I don't think there are disagreements on its specification.  I understood
it that way: For SMTP, it is the AUTH parameter to MAIL FROM, in case
the mail system trusts that information (RFC 2554, section 5).  For other
protocols, it is the authenticated sender address.  If no authenticated
sender address is available, it is the empty string.  Feel free to correct
me, if I am wrong.

Michael

From: Philip Guenther <[email-masked]>
To: [email-masked]
Subject: Re: 3028bis open issue #3: require 2047 decoding? 
Date: Thu, 14 Jul 2005 20:32:16 -0700
Sender: [email-masked]

I wrote:
...
>Since an implementation _could_ consider its "local convention" to be
>"omit", removing the test doesn't really ban anything but does make the
>idea less obvious.

That should say "removing the _text_", of course...

To: Alexandros Vellis <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: 3028bis open issue #2: unknown envelope-part names 
Date: Thu, 14 Jul 2005 20:00:11 -0700
Sender: [email-masked]

Alexandros Vellis <[email-masked]> writes:
....
>Based on the above comments, with which I wholeheartedly agree, I would
>like to propose actually adding the envelope-auth capability in the base
>spec. It is only a matter of one more paragraph clarifying the "auth"
>value to the envelope argument.

Unfortunately, it's a bit more than that: I'm fairly certain that the
SMTP-AUTH reference would need to be normative ("...must be read to
understand or implement...").  Rather than involving 3028bis in the
tangle that moving any sort of authentication protocol along the
standard track appears to be, I suggest the envelope-auth extension be
defined in a separate document.

Anyway, for now I've amended the first paragraph of section 5.4 to read:

   The "envelope" test is true if the specified part of the SMTP (or
   equivalent) envelope matches the specified key.  This specification
   defines the interpretation of the (case insensitive) "from" and "to"
   envelope-parts.  Additional envelope-parts may be defined by other
   extensions; implementations SHOULD consider unknown envelope parts an
   error.

Perhaps I'm misreading the work/pain implied by the SMTP-AUTH reference.
Do those with more experience think it'll be easy?

Philip Guenther

To: Ned Freed <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: 3028bis open issue #3: require 2047 decoding? 
Date: Thu, 14 Jul 2005 19:42:10 -0700
Sender: [email-masked]

Ned Freed <[email-masked]> writes:
><Barry wrote>
>> Definitely first on the list.  I actually think this *is* best practice,
>> despite Ned's having said he doesn't think there is a best practice on
>> this, and, in particular, I'd like to eliminate "omitted".
>
>I certainly could live with omitting "omitted".

Since an implementation _could_ consider its "local convention" to be
"omit", removing the test doesn't really ban anything but does make the
idea less obvious.  I've removed it for the next rev.

Philip Guenther

Cc: Barry Leiba <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Thu, 14 Jul 2005 18:57:15 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NULL vs. ""
Sender: [email-masked]

> [cleaning my mailbox of unanswered suggestions...]

> Barry Leiba <[email-masked]> writes:
> >> On Mon, 2005-05-30 at 09:21 -0700, Ned Freed wrote:
> >>>>if anyof (header :is "Cc" "", not exists "Cc")
> >>>
> >>>This is actually clearer to read IMO, and is probably how I would code it if
> >>>I ever wanted such a test.
> >
> >Indeed.  And so why don't we wrap up this too-long discussion this way?:
> >1. *Document* that string comparisons against headers MUST NOT match
> >absent headers.

> I've updated 5.7p3, second sentence to now read:
>     However, if the named header is not present, it does not match any
>     key, including the empty key.

Look good to me.

> >2. *Document* that the preferred way to test for a header that is either
> >empty or absent is as Ned wrote above.

> I've added as a new paragraph at the end of 5.7:
>    The preferred way to test whether a given header is either empty or
>    absent is to combine an "exists" test and a "header" test:

>            anyof (header :is "Cc" "", not exists "Cc")

WFM as well.

				Ned

To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: My open issues with RFC3028bis 
Date: Thu, 14 Jul 2005 18:51:24 -0700
Sender: [email-masked]

"Mark E. Mallett" <[email-masked]> writes:
>On Fri, Jul 01, 2005 at 05:58:04PM -0700, Philip Guenther wrote:
>> Works for me.  2.4.2.2 now reads:
>>    Similarly, synactically invalid header names cause the same result as
>>    syntactically valid header names that are not present in the message.
>>    In particular, an implementation MUST NOT cause an error for
>>    synactically invalid header names.
>
>Would it be reasonable to add that says this refers specifically to
>header tests (e.g. "in header tests, ...")?  Otherwise something like
>the 'editheader' extension will conflict with this prohibition, as
>editheader says (or, at least, implies, and probably should say) that
>bad header names are to cause an error.

Good point.  I've added "in tests" to the end of that sentence.

Philip Guenther

To: Ned Freed <[email-masked]>
Cc: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: security review of variables 
Date: Thu, 14 Jul 2005 18:39:04 -0700
Sender: [email-masked]

Regarding trucation of values in sieve variables...

Ned Freed <[email-masked]> writes:
...
>It seems to me the useful alternative would be to have a way of testing to see
>if truncation has occured. This way you could code tests to see if the
>dataa being processed is abnormally large and respond accordingly.

For comparison, procmail had a configurable limit on the length of
variable expansions and a means to test whether that limit has been
reached.  To the best of my knowledge, the mechanism for the test (the
PROCMAIL_OVERFLOW variable) has never been used in a real world script
to modify the script's behavior.

>But even this is of extremely marginal utility, for two reasons. First, string
>sizes can already be checked, and it probably makes more sense to see if
>something is abnormally long rather than checking to see if it has bumped up
>against a truncation limit. Second, the availability of such a mechanism does
>not mean it will actually be used, and my guess is that any such feature (and
>this includes the namespace stuff) will see little if any actual use. And
>rarely used features are security risks in and of themselves.

I completely agree with all of the above.

Philip Guenther

To: Barry Leiba <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: NULL vs. "" 
Date: Thu, 14 Jul 2005 18:24:37 -0700
Sender: [email-masked]

[cleaning my mailbox of unanswered suggestions...]

Barry Leiba <[email-masked]> writes:
>> On Mon, 2005-05-30 at 09:21 -0700, Ned Freed wrote:
>>>>if anyof (header :is "Cc" "", not exists "Cc")
>>>
>>>This is actually clearer to read IMO, and is probably how I would code it if
>>>I ever wanted such a test.
>
>Indeed.  And so why don't we wrap up this too-long discussion this way?:
>1. *Document* that string comparisons against headers MUST NOT match 
>absent headers.

I've updated 5.7p3, second sentence to now read:
    However, if the named header is not present, it does not match any
    key, including the empty key.

>2. *Document* that the preferred way to test for a header that is either 
>empty or absent is as Ned wrote above.

I've added as a new paragraph at the end of 5.7:
   The preferred way to test whether a given header is either empty or
   absent is to combine an "exists" test and a "header" test:

           anyof (header :is "Cc" "", not exists "Cc")

Philip Guenther

Date: Thu, 14 Jul 2005 15:17:13 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Re: Proposed agenda for the Sieve WG meeting in Paris
Sender: [email-masked]

Agenda

Total: 120 minutes.

To: [email-masked]
Subject: Re: Working Group Last Call: draft-ietf-sieve-rfc3598bis-00.txt (Subaddress)
From: Michael Haardt <[email-masked]>
Date: Wed, 13 Jul 2005 23:58:20 +0200
Sender: [email-masked]

> > I would like to draw your attention to the following draft:
> > 
> > <http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt 
> > 
> > NB If you do review this document, but have no issues or comments, 
> > please send both co-chairs (or the list) an email to indicate you have 
> > looked at it. This will allow the chairs to gauge the scope of reviews 
> > of this document to aid in our determination on whether to send it to 
> > the IESG. Thanks.

There was a longer discussion about the draft specifying how subaddresses
are encoded, and AFAIK, the result was that the encoding should be left
to the mail system and using the subaddress extension on anything but
envelope from may give bogus results, because applying local encoding
rules for foreign addresses may be wrong.

I miss that change in the draft.  Our system uses "--" as separator,
other systems use prefixes, not suffixes.  Exim offers subaddressing with
any encoding that can be expressed by its string expansion operators for
some years now, so there is at least one major MTA where the discussed
model does not fit.

Changes from draft-ietf-sieve-3028bis-01.txt
 5. Refer to the zero-length string ("") as "empty" instead of
    "null"

So draft-ietf-sieve-rfc3598bis-00 should use "empty" as well.

Sorry for responding so late,

Michael

To: [email-masked]
Subject: Re: vacation stuff
From: Michael Haardt <[email-masked]>
Date: Wed, 13 Jul 2005 23:42:05 +0200
Sender: [email-masked]

> This now sounds like you're talking about limiting how many handles a given
> script can specify. But I still don't see the point, since a given script can
> only have one handle apply per incoming message, so the overriding limit
> is still the number of remembered responses.

Rereading the draft:

   NOTE: One way to implement the necessary mechanism here is to store a
   hash of either the current handle and the recipient or, if no handle
   is provided, a hash of the vacation action parameters specifying the
   message content and the recipient.  If a script is changed,
   implementations MAY reset the records of who has been responded to
   and when they have been responded to.

   Implementations are free to limit the number of remembered responses,
   provided the limit is no less than 1000.  When limiting the number of
   tracked responses, implementations SHOULD discard the oldest ones
   first.

And since it talks about storing all responses in a large hash table,
this is probably not 1000 per handle, but 1000 in total.  Perhaps we
should say that to be entirely clear.

I use one table per handle, throwing tables away when their handles are
no longer active, and thought it were 1000 entries per table, but the
number of tables was not limited.  When I suggested the limit of 1000,
indeed I meant 1000 per table and never read that part again after it
was introduced.  Having at least 1000 active handles sounds a little
much to me, but I could live with that, in case nobody wants to put an
additional limit to the minimum number of handles.

Michael

Cc: [email-masked]
To: Michael Haardt <[email-masked]>
Date: Wed, 13 Jul 2005 10:13:11 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: vacation stuff
Sender: [email-masked]

(getting back to this now - sorry for the delay)

> > > I fail to see any real added risk here. Consider: Vacation can be executed at
> > > most once per incoming message, so the maximum number of database entries the
> > > vacation action can produce per incoming message is one. So, if the intent is
> > > to fill up the database, the simplest way to do it is set up regular vacation
> > > without a handle setting and send in a bunch of messages from different
> > > sources. Note. however, that implementations are free to limit the number of
> > > remembered responses, so this trick will only work if the implementation allows
> > > it.

> The implementation may limit the number of remembered responses, but not
> the number of databases.

I guess I don't understand what the "number of databases" is or why it should
be limited. It certainly doesn't seem like you're talking about the number of
database entries, which in any case would be the same as the number of
remembered responses.

> What's a reasonable lowest limit that scripts
> should be able to rely on? 100?

> I don't know if it's worth an entry under "security considerations", but
> allowing the implementation to limit of number of databases sounds useful
> to me.  I never thought about a script using a large number of them.

This now sounds like you're talking about limiting how many handles a given
script can specify. But I still don't see the point, since a given script can
only have one handle apply per incoming message, so the overriding limit
is still the number of remembered responses.

				Ned

Subject: Re: Working Group Last Call: draft-ietf-sieve-rfc3598bis-00.txt  (Subaddress)
From: Kjetil Torgrim Homme <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: MTA filtering mailing list <[email-masked]>
Date: Wed, 13 Jul 2005 15:55:19 +0200
Sender: [email-masked]

On Wed, 2005-07-13 at 13:24 +0100, Alexey Melnikov wrote:
> I would like to draw your attention to the following draft:
> 
> <http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt 
> 
> NB If you do review this document, but have no issues or comments, 
> please send both co-chairs (or the list) an email to indicate you have 
> looked at it. This will allow the chairs to gauge the scope of reviews 
> of this document to aid in our determination on whether to send it to 
> the IESG. Thanks.

a niggle, there is a missing "}" in the example:

# If a message is not to me (ignoring +detail), junk it.
      if not allof (address :user ["to", "cc", "bcc"] "ken",
           address :domain ["to", "cc", "bcc"] "example.org") {
        discard;

other than that it looks good to me.
-- 
Kjetil T.

Date: Wed, 13 Jul 2005 08:58:23 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>, MTA filtering mailing list <[email-masked]>
Subject: Re: Proposed agenda for the Sieve WG meeting in Paris
Sender: [email-masked]

Hi Alexey,

-- 
Cyrus Daboo

Date: Wed, 13 Jul 2005 13:29:40 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Re: Working Group Last Call: draft-ietf-sieve-rfc3598bis-00.txt  (Subaddress)
Sender: [email-masked]

Correction:

Alexey Melnikov wrote:

> I would like to draw your attention to the following draft:
>
> <http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt 
> <http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-00.txt>>

This should be just 
"<http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt>".

> A two week working group last call of this document starts on 14th 
> July 2005 (tomorrow) and ends on 28th July 2005 at 6 pm EST.
>
> Please review this document and send issues to the list or directly to 
> the author(s).
>
> NB If you do review this document, but have no issues or comments, 
> please send both co-chairs (or the list) an email to indicate you have 
> looked at it. This will allow the chairs to gauge the scope of reviews 
> of this document to aid in our determination on whether to send it to 
> the IESG. Thanks.
>
> -- 
> Alexey Melnikov
> SIEVE WG Co-chair

Date: Wed, 13 Jul 2005 13:24:44 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Working Group Last Call: draft-ietf-sieve-rfc3598bis-00.txt  (Subaddress)
Sender: [email-masked]

I would like to draw your attention to the following draft:

<http://www.ietf.org/internet-drafts/draft-ietf-sieve-rfc3598bis-00.txt 
<http://www.ietf.org/internet-drafts/draft-ietf-sieve-body-00.txt>>

A two week working group last call of this document starts on 14th July 
2005 (tomorrow) and ends on 28th July 2005 at 6 pm EST.

Please review this document and send issues to the list or directly to 
the author(s).

NB If you do review this document, but have no issues or comments, 
please send both co-chairs (or the list) an email to indicate you have 
looked at it. This will allow the chairs to gauge the scope of reviews 
of this document to aid in our determination on whether to send it to 
the IESG. Thanks.

--
Alexey Melnikov
SIEVE WG Co-chair

Date: Tue, 12 Jul 2005 18:54:58 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Proposed agenda for the Sieve WG meeting in Paris
Sender: [email-masked]

Hi everyone,

Agenda

Total: 120 minutes.

========
Notes:

Alexey,
SIEVE WG co-chair.

To: [email-masked]
Subject: Re: Complexity of :matches ?
From: Michael Haardt <[email-masked]>
Date: Tue, 12 Jul 2005 14:13:45 +0200
Sender: [email-masked]

> In particular, counted repetitions ({n,m}) have to be expanded when
> figuring the 'N' of the size of the regexp, while back-references
> (\digit) are, IIRC, worst-case exponential to match.

That's because back-references leave the class of regular languages,
but I am surprised right now that the manual does not say so.

This egrep expression shows that is an entertaining way:

^(aa+)\1+$

It does not match if the length of the string of a characters is prime,
because the group is a prime factor.  A DFA can not do that, because it
contains only a finite number of states, thus being unable to compute
with natural numbers.  I guess I should think less about regular
expressions. :)

Michael

From: Philip Guenther <[email-masked]>
To: [email-masked]
Subject: Re: Complexity of :matches ? 
Date: Tue, 12 Jul 2005 04:36:51 -0700
Sender: [email-masked]

I, Philip Guenther <[email-masked]> wrote:
...
>For an NFA, the setup is O(N) in time and space and matching is O(N*M).
>
>For a non-lazy DFA, the matching is O(M), but the worst-case setup is
>O(2^N).
...

I should note that the dragon book's statements, as I tried to
summarize above, are only true of the original, truly regular
"regular expressions" and not the extended or 'irregular' regular
expressions seen in most tools.

In particular, counted repetitions ({n,m}) have to be expanded when
figuring the 'N' of the size of the regexp, while back-references
(\digit) are, IIRC, worst-case exponential to match.  If you're
capturing matches then the POSIX matching rules apparently are
worst-case exponential on the number of characters in the regexp,
or something like that.

Philip Guenther

Date: Tue, 12 Jul 2005 13:26:31 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: Complexity of :matches ?
Sender: [email-masked]

On Tue, Jul 12, 2005 at 03:52:01AM -0700, Philip Guenther wrote:
> Given that you can obviously locate a match for a string of 'n' non-star
> characters in O(n*M), isn't the algorithm that Tim described a
> worst-case of O(N*M) where N is the number of characters _other_ than
> star in the pattern?  (Stars are the cheap part, as they can never force
> a backtrack!)

I thought about things by looking at a*b and the string abb.  A non-greedy
match could match a with a, * with nothing, b with b, and then find out it
has to backtrack.  A greedy match with a*b and abc would match a with a,
* with bc, backtrack, match * with b, fail on b and c.  Things got worse
with more stars.  As I said, a naive approach, but I was too focused on
regular expressions and failed to see how I can make use of wildcards
being a subset.

Now that Tim answered it all, it is obvious where exactly the difference
to regular expressions is, why backtracking can be replaced by string
search, that it is not related to greedyness and that in wildcard
expressions, stars are cheap if done right.

Michael

To: Michael Haardt <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: Complexity of :matches ? 
Date: Tue, 12 Jul 2005 03:52:01 -0700
Sender: [email-masked]

Michael Haardt <[email-masked]> writes:
...
>So you suspect that there might be an algorithm that does not perform
>backtracking? I can not imagine O(N*M) as worst case unless we can kick
>backtracking out, although it might well be an average case.  But that's
>just a feeling and it may be wrong.

Does your :contains implementation (substr()?) perform backtracking?
What's its worst-case big-oh?

...
>If we can not come up with a proof for worst-case complexity of non-greedy
>algorithms,  <...>

Given that you can obviously locate a match for a string of 'n' non-star
characters in O(n*M), isn't the algorithm that Tim described a
worst-case of O(N*M) where N is the number of characters _other_ than
star in the pattern?  (Stars are the cheap part, as they can never force
a backtrack!)

...
><Ned wrote>
>> In any case, the computational complexity of glob-style matches is
>> clearly less than that of regexp, which is hardly surprising given
>> that regexps require a NFA. This is why we disable regexp support by
>> default in our implementation.
>
>That's what I wonder about.  The NFA can be converted to a DFA,
>backtracking being an alternative, but that same appears to be true for
>wildcard patterns to me, if multiple stars are allowed.

<pause to pull out the dragon book>

(To reiterate, N is the size of the pattern, M is the size of the text
being matched against.)

For an NFA, the setup is O(N) in time and space and matching is O(N*M).

For a non-lazy DFA, the matching is O(M), but the worst-case setup is
O(2^N).

A lazy DFA has setup O(N), but the dragon book glosses its matching
complexity: they claim it's "almost as fast" as a non-lazy DFA, but I
think it degenerates to a weird NFA if no states are reused, making it
O(N*M).  A precise statement of the complexity of calculating a new
state is needed to fully characterize that.

Philip Guenther

To: [email-masked]
Subject: Re: Complexity of :matches ?
From: Michael Haardt <[email-masked]>
Date: Tue, 12 Jul 2005 11:44:17 +0200
Sender: [email-masked]

> The algorithm I used was just essentially this.
> - ? can be handled trivially.
> - if the pattern doesn't begin with *
>    - handle that leader
> - find the next string between stars
>    - search for string between the stars
>    - search returns a location of that substring
>    - no match -> return false
>    - we have a match; from now on, search only
>      from the location of that match forward
> - if the pattern doesn't end with *
>    - handle that trailer

Just forget my previous mail.  Here we go, an algorithm without
backtracking/NFA.

Now that I see it, the fundamental difference to regexps is obvious:
Regexps can contain looping patterns, and wildcard patterns can not,
thus allowing to replace backtracking by string search.  Pretty cool,
Tim, and thanks for opening my eyes!

> I think that this implements right-to-left greedy matching, that is,
> completely ungreedy matching.  If you wanted greedy matching, you might
> be able to get it by doing the search backwards.  I'm real sure that I
> can't prove this.

It does sound reasonable, though.  A backward search will match the
shortest possible pattern right to a string, starting at the last one,
leaving the longest possible one left to it.  Since that applies to all
strings, indeed you get greedy matching that way. (Unless I write crap
again. ;-)

I am unsure how to proceed on documenting something in the security
considerations.  Tim showed that there is no risk really, if you do
things right.  I showed how tempting a naive approach is.  Ned was sure
that his code was not to be breaken easily, but didn't want to exclude
that there might be a pathological case for sure.

It feels silly to document that done wrong, there is a security risk.
Then again, it is not easy to see that it is possible to avoid exponential
complexity even for the worst case, and how.  Future implementations
might work unsafe, if we don't document something.

Michael

To: [email-masked]
Subject: Re: Complexity of :matches ?
From: Michael Haardt <[email-masked]>
Date: Tue, 12 Jul 2005 10:52:55 +0200
Sender: [email-masked]

> The complexity of a naive implementation of greedy glob-matching is pretty
> clearly O(M*N**K), where N is the length of the target string, M is the length
> of the pattern, and K is the number of *'s present in the pattern, and it is
> pretty easy to construct inputs that will exhibit worse-case behavior - all you
> need is lots of *'s in a pattern that never matches.

Right, and that's what happened.  It wasn't a problem until recently, when
users started to use multiple stars instead of one, like ********@domain.
Don't ask me why.

> However, I do not know if simply using an optimized non-greedy approach
> actually reduces the computational complexity to something less than O(M*N**K)
> for extremely pathological cases. If it doesn't, my guess is that there's some
> sort of dynamic programming trick that can get the complexity down to O(N*M) or
> something similar. I haven't tried to find such a trick, mostly because we've
> never encountered a pathological case when using optimized non-greedy matching
> that would make it relevant.

So you suspect that there might be an algorithm that does not perform
backtracking? I can not imagine O(N*M) as worst case unless we can kick
backtracking out, although it might well be an average case.  But that's
just a feeling and it may be wrong.

> In any case, the computational complexity of glob-style matches is clearly less
> than that of regexp, which is hardly surprising given that regexps require
> a NFA. This is why we disable regexp support by default in our implementation.

That's what I wonder about.  The NFA can be converted to a DFA,
backtracking being an alternative, but that same appears to be true for
wildcard patterns to me, if multiple stars are allowed.

> Perhaps it makes sense to recommend a non-greedy approach in the security
> considerations section of the base specification? Allowing implementations to
> limit the number of *'s they allog in glob-style patterns might also make
> sense, if only to catch cases where users used :matches when they intended to
> use :contains.

If we can not come up with a proof for worst-case complexity of non-greedy
algorithms, then I think we should document that pattern matching may
require time exponential to the number of stars in the pattern and indeed
allow implementations to put a limit on pattern matching, generating an
error if the limit is exceeded.  The implementation should be free to express
the limit in the number of stars, the number of performed backtracking
operations, used CPU time or whatever else.

Interestingly, the PCRE library already contains functions to do that
for regular expressions, which is why the Exim filter language (quite
similar to Sieve) can offer regular expressions at no risk.

Michael

Date: Mon, 11 Jul 2005 12:50:49 -0700
From: Tim Showalter <[email-masked]>
To: Ned Freed <[email-masked]>
CC: Michael Haardt <[email-masked]>, [email-masked]
Subject: Re: Complexity of :matches ?
Sender: [email-masked]

> However, I do not know if simply using an optimized non-greedy approach
> actually reduces the computational complexity to something less than O(M*N**K)
> for extremely pathological cases.

I had a implementation (wasn't Sieve, but had * and ? and worked the 
same way) that was only O(M^N), that is, the number of stars didn't 
matter.  Barring pathological cases that reduce the match to O(N^2), I 
believe this algorithm is Î˜(N).  It's possible to be actually O(M+N) if 
you use Boyer-Moore.

The algorithm I used was just essentially this.
- ? can be handled trivially.
- if the pattern doesn't begin with *
   - handle that leader
- find the next string between stars
   - search for string between the stars
   - search returns a location of that substring
   - no match -> return false
   - we have a match; from now on, search only
     from the location of that match forward
- if the pattern doesn't end with *
   - handle that trailer

That is, for with a pattern like *a*b*c* is three strings, search for 
each of which can be found in best-case O(N) time.  If we search for 'a' 
first and find the first match at position 10, we start searching for 
'b' at position 11.  For a pattern like a*b*c*d*e, we do the same thing 
after making sure the string starts with a and ends with e.

The pathological case for this, like string comparison, is only, uh, 
O((M+N)^2) or so, and then only if you're using naive string comparisons 
(which my implementation did).

Since I'm not sure that I've gotten my point across, let me make a more 
straightforward statement.  If you're doing globbing, and you don't care 
if it's greedy or not, and you're doing recursion, then you're doing 
something wrong.  Finding one match is something that can be done in 
roughly linear time.  Recursion is only necessary for finding the "best" 
match which is not something that's in the base Sieve spec.

I think that this implements right-to-left greedy matching, that is, 
completely ungreedy matching.  If you wanted greedy matching, you might 
be able to get it by doing the search backwards.  I'm real sure that I 
can't prove this.

Tim

Cc: [email-masked]
To: Michael Haardt <[email-masked]>
Date: Mon, 11 Jul 2005 08:15:21 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Complexity of :matches ?
Sender: [email-masked]

> I just had a user using a wildcard pattern that took a large amount of
> CPU time.  Now that this particular problem is solved, I wonder about
> ":matches".

> Does anybody know the complexity order of recognising wildcard patterns,
> which are a subset of regular expressions?

The complexity of a naive implementation of greedy glob-matching is pretty
clearly O(M*N**K), where N is the length of the target string, M is the length
of the pattern, and K is the number of *'s present in the pattern, and it is
pretty easy to construct inputs that will exhibit worse-case behavior - all you
need is lots of *'s in a pattern that never matches.

Using a  non-greedy approach makes it much harder to construct inputs that will
take a long time to process, especially if you also do some simple
optimizations like collapsing strings of repeated *'s. In practice this seems
to avoid many of the more common problems, such as when users specify an
argument to :matches that only makes sense to give to :contains, such as
"******** ATTENTION ********".

However, I do not know if simply using an optimized non-greedy approach
actually reduces the computational complexity to something less than O(M*N**K)
for extremely pathological cases. If it doesn't, my guess is that there's some
sort of dynamic programming trick that can get the complexity down to O(N*M) or
something similar. I haven't tried to find such a trick, mostly because we've
never encountered a pathological case when using optimized non-greedy matching
that would make it relevant.

In any case, the computational complexity of glob-style matches is clearly less
than that of regexp, which is hardly surprising given that regexps require
a NFA. This is why we disable regexp support by default in our implementation.

Perhaps it makes sense to recommend a non-greedy approach in the security
considerations section of the base specification? Allowing implementations to
limit the number of *'s they allog in glob-style patterns might also make
sense, if only to catch cases where users used :matches when they intended to
use :contains.

				Ned

P.S. I'm curious what pattern you encountered that took a long time. Mind
sharing it?

To: [email-masked]
Subject: Complexity of :matches ?
From: Michael Haardt <[email-masked]>
Date: Mon, 11 Jul 2005 12:48:37 +0200
Sender: [email-masked]

Hello,

I just had a user using a wildcard pattern that took a large amount of
CPU time.  Now that this particular problem is solved, I wonder about
":matches".

Does anybody know the complexity order of recognising wildcard patterns,
which are a subset of regular expressions? Is giving users control of
patterns to be matched a security risk?

Michael

To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: more 3028bis comments 
Date: Fri, 08 Jul 2005 00:08:56 -0700
Sender: [email-masked]

"Mark E. Mallett" <[email-masked]> writes:
...
>2.4.2.4. MIME Parts
>
>   > In a few places, [MIME] body parts are represented as strings.  These
>   > parts include MIME headers and the body.  This provides a way of
>   > embedding typed data within a Sieve script so that, among other
>   > things, character sets other than UTF-8 can be used for output
>   > messages.
>
>This still confuses me.  Where is it referenced?

I don't think it is.  Indeed, it don't think it was in RFC 3028 either.
The 'vacation' extension uses something like this with its ':mime'
keyword, but it has enough of a definition there that I don't think the
text in 3028bis is needed.  Everyone say "bye" to section 2.4.2.4...

>2.8.     Blocks
>
>   > A control structure is a command that happens to take a test and a
>   > block as one of its arguments; depending on the result of the test
>   > supplied as another argument, it runs the code in the block some
>   > number of times.
>
>This is inconsistent since "require" and "stop" are defined to be 
>control structures, and neither of them take a block as an argument.
>Either reword the above, or change 3.2 and 3.3 so that they don't
>use the word "structure" (with perhaps an intro in '3' defining
>the use of "control structure" vs just "control").

Yeah, calling just the subset of control commands that have a block
argument "control structures" seems the way to go.  I'll make the
wording tweaks.

...
>3.1.     Control Structure If
>Shouldn't this be "else <block3: block> ?
...
>5.1.     Test address
>recommend adding the  closing "}" here 
...
>5.8.     Test not
>Shouldn't this be something like  "not <test1: test>"  ?

I agree.  Done.

>Other random comments:
>
>The spec provides no way to access the "From_" line (which is the
>"From<sp>" line with no colon that is added by some mail software.
>While it's not part of RFC2822, and admittedly a minor concern at
>best, that 'header' is often there, but inaccessible.  I have no
>solution, other than perhaps making "From_" a special header name.

See my other reply for why I disagree with this.

>There was some discussion about adding character escapes (\u etc);
>this probably falls under the "substantive changes or additions"
>prohibition, but maybe not.

Right.  The discussion on the list following the Minneapolis meeting
concluded that such a change should be left an extension, if anyone
wants to define one, and that the base spec should simply recommend that
scripts avoid superfluous backslashes.

>The 'variables' extension has to specify greedy/non-greedy ":matches" ;
>this really ought to be in 3028 so that extensions can consistently
>follow whatever rule there is; although, again, it might fail the
>"non-substantive" test.

I agree with Kjetil and Ned: this is only observable in the presence of
such extensions, so it should be left to them.  Indeed, two extensions
could define the opposite semantics if they had separate means of
observing the results and while it would be a pain to implement, it
wouldn't be impossible.

(I somehow doubt the latter such extension would pass its last-call if
it didn't have a superb justification for the different choice.  The
base-spec doesn't have to ban annoying extensions to keep them from
happening; alternatively, given the efficacy of the previous ban on
side-effects-in-tests, we should be disinclined from going that path
again.  A ban is neither necessary, nor sufficient.)

Philip Guenther

Cc: [email-masked]
To: Barry Leiba <[email-masked]>
Date: Thu, 07 Jul 2005 13:51:22 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: 3028bis open issue #3: require 2047 decoding?
Sender: [email-masked]

> >>How about the following for the second paragraph of 2.7.2:
> >>      Comparisons are performed in Unicode.  Implementations convert
> >>      text from header fields in all charsets [HEADER-CHARSET] to
> >>      Unicode as input to the comparator (see 2.7.3).  Implementations
> >>      MUST be capable of converting US-ASCII, ISO-8859-1, the US-ASCII
> >>      subset of ISO-8859-* character sets, and UTF-8.  Text that the
> >>      implementation cannot convert to Unicode for any reason, MAY be
> >>      omitted, treated as plain US-ASCII (including any [HEADER-CHARSET]
> >>      syntax), or processed according to local conventions,
> >
> >
> > Thought I sent a note about this, but cannot find it... Anyway, I think this
> > is fine, although I'd be tempted to out "treat as plain US-ASCII" first on the
> > list.

> Definitely first on the list.  I actually think this *is* best practice,
> despite Ned's having said he doesn't think there is a best practice on
> this, and, in particular, I'd like to eliminate "omitted".

I certainly could live with omitting "omitted".

>  Consider:

>     Subject: =?bogus-charset?Q?Buy Viagra now!?=

> Do you really want my rule that says
>     if (header :contains ["subject"] ["viagra"]) {
>        discard;
>        stop;
>     }
> to be ignored because the spammer put in a bogus character set name
> (perhaps purposefully, to screw up Sieve scripts)?

Good point.

				Ned

Cc: Ned Freed <[email-masked]>, Kjetil Torgrim Homme <[email-masked]>, [email-masked]
To: Mark E Mallett <[email-masked]>
Date: Thu, 07 Jul 2005 13:09:06 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: more 3028bis comments
Sender: [email-masked]

> On Wed, Jul 06, 2005 at 12:31:40PM -0700, Ned Freed wrote:
> > > > > > The 'variables' extension has to specify greedy/non-greedy ":matches" ;
> > > > > > this really ought to be in 3028 so that extensions can consistently
> > > > > > follow whatever rule there is; although, again, it might fail the
> > > > > > "non-substantive" test.
> >
> > > > > I don't think this behaviour description is relevant for the main spec.
> > > > > it can not be observed without the variables extension.
> >
> > > > Yes, exactly so. This could easily come back to bite us when we need to move to
> > > > draft.
> >
> > > sorry, I'm dense.  are you agreeing with Mark or with me?
> >
> > With you.
> >
> > My worry here is that requiring non-greedy in the base specification is, as you
> > put it, requiring nonobservable behavior that cannot be externally verified.

> The issue I had in mind was that of setting precedent; now that we know
> that an extension wants to define the greediness behaviour, it seemed to
> me that it would have been best to have that precedent set in the base
> document.

I disagree. It is far more important that specifications specify the minumum
necessary and not specify stuff that cannot be verified externally.

> Otherwise, unrelated extensions could simply state what
> behaviour they require, with one contradicting another.  I imagine that
> in practice this would be unlikely; precedent set by one extension is
> likely to be followed by another.

And to the extent it possible for one specification to contradict another, it
is also possible for an extension to unwittingly contradict the base
specification. Irrespective of the liklihood of such a contradiction occurring,
the increased likihood of it happening because of the location of one of the
contradictory statements is bound to be marginal.

> Nevertheless, since we know that this
> definition is needed by an extension, it feels much better to me to
> specify it in the base document.  So I'm looking for neatness more than
> anything.  Also, having it there, even if the behaviour is not visible,
> makes it easier for an implementation coded to that specification to be
> able to support extensions that do make it visible.

But it may make it harder to advance the base to draft. And advancement to
draft is what we're supposed to be focused on per the charter.

> > When documents go to draft we at a minimum will have to write up a list of
> > conformance criteria and then make sure we have enough conforming
> > implementations. Including non-observable criteria on the list means that
> > the only person who can submit information about a given implementation
> > is someone able to see the code for that implementation. This would have
> > been a problem for several past conformance list exercises.

> I too am probably being dense... but in another message you said, regarding
> the CRLF/CR/LF wording:

> > I see no "MUST generate an error" clause here, so as far as I'm concerned the
> > handling of bare CR and bare LF means you have a nonconformant script, but how
> > implementations handle this particular form of nonconformance is undefined.

> which seems to be somewhat opposite reasoning, justifying the
> specification of something entirely on the basis that nonconformance
> can't be observed.  I'll probably reply to that separately, tho.

First, it is axiomatic that conformance criteria only apply to defined
behavior. Undefined behavior cannot be something you test conformance to.
Second, advancement to draft involves checking for sieve implementation
compliance; the existance or nonexistance of incompliant scripts in the wild
isn't particularly relevant unless, I suppose, it could be shown that all
scripts out there are incompliant.

				Ned

Cc: Ned Freed <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Thu, 07 Jul 2005 12:54:58 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

> Ned Freed <[email-masked]> writes:
> ...
> >IMO it would be better to say something along the lines of
> >
> >    Header lines are unfolded as described in [IMAIL] section 2.2.3.
> >    Interpretation of header data should be done according to [MIME3]
> >    section 6.2.
> >
> >This makes the required sequence of operations clear. Note that I also removed
> >the term "long" - in practice it isn't only long lines that are folded.

> I like it, though I think there should be a forward reference to 2.7.2
> as that's where the precise requirements are stated.  So:

>    Header lines are unfolded as described in [IMAIL] section 2.2.3.
>    Interpretation of header data SHOULD be done according to [MIME3]
>    section 6.2 (see 2.7.2 below for details).

The added reference is a useful addition IMO.

				Ned

Cc: Mark E Mallett <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Thu, 07 Jul 2005 12:25:54 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: more 3028bis comments
Sender: [email-masked]

> "Mark E. Mallett" <[email-masked]> writes:
> >On Tue, Jul 05, 2005 at 05:46:18PM -0700, Ned Freed wrote:
> >> > On Tue, 2005-07-05 at 16:52 -0400, Mark E. Mallett wrote:
> >> > > 2.10.6.  Errors
> ...
> >> > good question.  delivering an error report to INBOX in addition to the
> >> > message is one way of solving this, but does anyone do this?
> >>
> >> We do. Our implementation has the concept of a "responsible
> >> address" that's attached to each sieve script. When an error
> >> occurs this address is notified of the problem. For user sieves
> >> this is the user's own address, system sieves have the main
> >> system postmaster as the responsible address, and so on.

> Sendmail's implementation does that too.

Good to hear.

> >> > > The spec provides no way to access the "From_" line (which is the
> >> > > "From<sp>" line with no colon that is added by some mail software.
> >> > > While it's not part of RFC2822, and admittedly a minor concern at
> >> > > best, that 'header' is often there, but inaccessible.  I have no
> >> > > solution, other than perhaps making "From_" a special header name.
> >>
> >> > we have envelope "from" for the e-mail address.  we don't have anything
> >> > for the delivery date, but I suspect Ned is considering that for his
> >> > "date" extension.
> >
> >I was talking about accessing the header field in same ways that others
> >can be accessed, not other ways of getting at the specific parts of that
> >field that we might predict people might want.

> The "From " line is neither a header field nor permitted by RFC 822
> or RFC 2822.  It's simply part of a mailbox format that dates back
> to V7 UNIX.  Indeed, it is the bane of UNIX mail software to deal
> with that wart and treating it as part of the message header is,
> IMHO, a mistake as it complicates handling of other mailbox formats
> or even of resending message.

Agreed. It's what X.400 would call "delivery info", a remnant of what was in
the envelope at the time of final delivery. So, to the extent it contains
useful information, it should be treated as envelope material, not header
material.

> Confusing framing info with data is a mistake.  Providing access
> to incoming framing info is one thing, as is providing a means to
> set it on output (in those cases where it is used), but that should
> not be done by the backdoor of an invalid field name.

Exactly so.

				Ned

To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: more 3028bis comments 
Date: Wed, 06 Jul 2005 15:53:31 -0700
Sender: [email-masked]

"Mark E. Mallett" <[email-masked]> writes:
>On Tue, Jul 05, 2005 at 05:46:18PM -0700, Ned Freed wrote:
>> > On Tue, 2005-07-05 at 16:52 -0400, Mark E. Mallett wrote:
>> > > 2.10.6.  Errors
...
>> > good question.  delivering an error report to INBOX in addition to the
>> > message is one way of solving this, but does anyone do this?
>>
>> We do. Our implementation has the concept of a "responsible
>> address" that's attached to each sieve script. When an error
>> occurs this address is notified of the problem. For user sieves
>> this is the user's own address, system sieves have the main
>> system postmaster as the responsible address, and so on.

Sendmail's implementation does that too.

>> > > The spec provides no way to access the "From_" line (which is the
>> > > "From<sp>" line with no colon that is added by some mail software.
>> > > While it's not part of RFC2822, and admittedly a minor concern at
>> > > best, that 'header' is often there, but inaccessible.  I have no
>> > > solution, other than perhaps making "From_" a special header name.
>> 
>> > we have envelope "from" for the e-mail address.  we don't have anything
>> > for the delivery date, but I suspect Ned is considering that for his
>> > "date" extension.
>
>I was talking about accessing the header field in same ways that others
>can be accessed, not other ways of getting at the specific parts of that
>field that we might predict people might want.

The "From " line is neither a header field nor permitted by RFC 822
or RFC 2822.  It's simply part of a mailbox format that dates back
to V7 UNIX.  Indeed, it is the bane of UNIX mail software to deal
with that wart and treating it as part of the message header is,
IMHO, a mistake as it complicates handling of other mailbox formats
or even of resending message.

For example, procmail _usually_ (but not always!) inserts or updates
a "From " line before processing any rcfiles.  It also tries to
strip that line before forwarding the message.  However, it fails
to remove it when saving the message in mailboxes whose format does
not call for (or permit!) a "From " line, such as MH or maildir.
It also fails to enforce the presence of a "From " line when saving
to V7-format mailboxes, meaning that a poorly written rcfile can
remove or munge it, resulting in a broken mailbox.

Confusing framing info with data is a mistake.  Providing access
to incoming framing info is one thing, as is providing a means to
set it on output (in those cases where it is used), but that should
not be done by the backdoor of an invalid field name.

>I've seen occasions
>where script writers either want to suppress the line if supplied, or
>add one if not, e.g.:
>
>    if exists "From_" { deleteheader "From_" };
>
>as a very simple case.

All the cases I've seen of people deleting "From " lines fall into
three groups:

1) the @($* software isn't removing the "From " line when saving
   the message in other formats or when feeding the message to
   another program
2) the *#@( software is letting through a bad "From " line from
   'upstream' and has to be forced to generate a new, correct one
   manually by deleting the bogus one
3) the user is confused as ends up breaking their V7 mailboxes

All three of those cases are best handled by fixing the softwware
to strip "From " lines on input, perhaps after taking a stab at
extracting an address and date from them for later use, and then
only generating a new "From " line in output when writing directly
to a V7 mailbox or when otherwise directed to.

Philip Guenther

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-3028bis-03.txt 
Date: Wed, 06 Jul 2005 18:50:01 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve: An Email Filtering Language
	Author(s)	: T. Showalter, P. Guenther
	Filename	: draft-ietf-sieve-3028bis-03.txt
	Pages		: 37
	Date		: 2005-7-6
	
This document describes a language for filtering email messages at
   time of final delivery.  It is designed to be implementable on either
   a mail client or mail server.  It is meant to be extensible, simple,
   and independent of access protocol, mail architecture, and operating
   system.  It is suitable for running on a mail server where users may
   not be allowed to execute arbitrary programs, such as on black box
   Internet Message Access Protocol (IMAP) servers, as it has no
   variables, loops, or ability to shell out to external programs.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-03.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-sieve-3028bis-03.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-3028bis-03.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-3028bis-03.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-3028bis-03.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 6 Jul 2005 17:37:56 -0400
To: Ned Freed <[email-masked]>
Cc: Kjetil Torgrim Homme <[email-masked]>, Mark E Mallett <[email-masked]>, [email-masked]
Subject: Re: more 3028bis comments
Sender: [email-masked]

On Tue, Jul 05, 2005 at 05:46:18PM -0700, Ned Freed wrote:
> > On Tue, 2005-07-05 at 16:52 -0400, Mark E. Mallett wrote:
> > > 2.2.     Whitespace
> > >
> > >    > Whitespace is used to separate tokens.  Whitespace is made up of
> > >    > tabs, newlines (CRLF, never just CR or LF), and the space character.
> > >    > The amount of whitespace used is not significant.
> > >
> > > This is specifically about whitespace in the Sieve language.  So... how
> > > many implementations violate this?  i.e., does everyone generate an
> > > error if a script's whitespace contains a naked CR or LF ?
> 
> > does it matter?  this text is unchanged from RFC 3028, so it will not
> > affect conformance.
> 
> I see no "MUST generate an error" clause here, so as far as I'm concerned the
> handling of bare CR and bare LF means you have a nonconformant script, but how
> implementations handle this particular form of nonconformance is undefined.
> 
> Attempting to nail this down to a "MUST generate an error" would be a huge
> mistake IMO, given the use of bare LF and bare CR as the default line
> terminators on many systems. And I really don't want to repeat the whole
> "canonical format versus local storage format" discussion again, thank you very
> much. So let's let this particular sleeping dog lie, OK?

Oh- I had forgotten that you asked that, when I wrote my other response,
sorry.  I have a hard time letting go that the spec mandates something
that is probably impossible to follow, and likely derives from something
that's not relevant (i.e., wire format).  But other than making this
side comment, I will indeed let it go :-)

> > > 2.10.6.  Errors
> > >
> > >    > When an error happens, implementations MUST notify the user that an
> > >    > error occurred, which actions (if any) were taken, and do an implicit
> > >    > keep.
> > >
> > > Probably extremely nit-picky, but I've always wondered when I read
> > > this "what user, and how do we notify them?"  Some users have no
> > > access to anything other than their mailboxes.  I suspect that many
> > > implementations will do some system-wide logging, which notifies the
> > > admin, but not the user.
> 
> > good question.  delivering an error report to INBOX in addition to the
> > message is one way of solving this, but does anyone do this?

I do something like that, in a way that's not very clean and I'm not
very proud of, which is one reason I asked the question..  I file any
error message text along with the same message that caused it.  (We also
have a personal logfile facility, but that's not guaranteed to be accessible
by the end user, nor is the end user likely to check it very often.)

> We do. Our implementation has the concept of a "responsible address" that's
> attached to each sieve script. When an error occurs this address is notified of
> the problem. For user sieves this is the user's own address, system sieves have
> the main system postmaster as the responsible address, and so on.

That sounds like an interesting idea and is probably worth pursuing as
an alternative.  Thanks..

> > > The spec provides no way to access the "From_" line (which is the
> > > "From<sp>" line with no colon that is added by some mail software.
> > > While it's not part of RFC2822, and admittedly a minor concern at
> > > best, that 'header' is often there, but inaccessible.  I have no
> > > solution, other than perhaps making "From_" a special header name.
> 
> > we have envelope "from" for the e-mail address.  we don't have anything
> > for the delivery date, but I suspect Ned is considering that for his
> > "date" extension.

I was talking about accessing the header field in same ways that others
can be accessed, not other ways of getting at the specific parts of that
field that we might predict people might want.  I've seen occasions
where script writers either want to suppress the line if supplied, or
add one if not, e.g.:

    if exists "From_" { deleteheader "From_" };

as a very simple case.  Knowing that you can't access that (admittedly
fake) is a thing by itself, without having to say all the things that
you might want to do with it.  Anyway it was simply on my collected list
of known issues that I was looking at while making the posting.

-mm-

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 6 Jul 2005 17:22:36 -0400
To: Ned Freed <[email-masked]>
Cc: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Subject: Re: more 3028bis comments
Sender: [email-masked]

On Wed, Jul 06, 2005 at 12:31:40PM -0700, Ned Freed wrote:
> > > > > The 'variables' extension has to specify greedy/non-greedy ":matches" ;
> > > > > this really ought to be in 3028 so that extensions can consistently
> > > > > follow whatever rule there is; although, again, it might fail the
> > > > > "non-substantive" test.
> 
> > > > I don't think this behaviour description is relevant for the main spec.
> > > > it can not be observed without the variables extension.
> 
> > > Yes, exactly so. This could easily come back to bite us when we need to move to
> > > draft.
> 
> > sorry, I'm dense.  are you agreeing with Mark or with me?
> 
> With you.
> 
> My worry here is that requiring non-greedy in the base specification is, as you
> put it, requiring nonobservable behavior that cannot be externally verified.

The issue I had in mind was that of setting precedent; now that we know
that an extension wants to define the greediness behaviour, it seemed to
me that it would have been best to have that precedent set in the base
document.  Otherwise, unrelated extensions could simply state what
behaviour they require, with one contradicting another.  I imagine that
in practice this would be unlikely; precedent set by one extension is
likely to be followed by another.  Nevertheless, since we know that this
definition is needed by an extension, it feels much better to me to
specify it in the base document.  So I'm looking for neatness more than
anything.  Also, having it there, even if the behaviour is not visible,
makes it easier for an implementation coded to that specification to be
able to support extensions that do make it visible.

This may or may not be outside the charter; again it's just something I
wanted to bring up while it's possible to talk about it.

> When documents go to draft we at a minimum will have to write up a list of
> conformance criteria and then make sure we have enough conforming
> implementations. Including non-observable criteria on the list means that
> the only person who can submit information about a given implementation
> is someone able to see the code for that implementation. This would have
> been a problem for several past conformance list exercises.

I too am probably being dense... but in another message you said, regarding
the CRLF/CR/LF wording:

> I see no "MUST generate an error" clause here, so as far as I'm concerned the
> handling of bare CR and bare LF means you have a nonconformant script, but how
> implementations handle this particular form of nonconformance is undefined.

which seems to be somewhat opposite reasoning, justifying the
specification of something entirely on the basis that nonconformance
can't be observed.  I'll probably reply to that separately, tho.

mm

Cc: Ned Freed <[email-masked]>, [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 06 Jul 2005 12:31:40 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: more 3028bis comments
Sender: [email-masked]

> > > > The 'variables' extension has to specify greedy/non-greedy ":matches" ;
> > > > this really ought to be in 3028 so that extensions can consistently
> > > > follow whatever rule there is; although, again, it might fail the
> > > > "non-substantive" test.

> > > I don't think this behaviour description is relevant for the main spec.
> > > it can not be observed without the variables extension.

> > Yes, exactly so. This could easily come back to bite us when we need to move to
> > draft.

> sorry, I'm dense.  are you agreeing with Mark or with me?

With you.

My worry here is that requiring non-greedy in the base specification is, as you
put it, requiring nonobservable behavior that cannot be externally verified.
When documents go to draft we at a minimum will have to write up a list of
conformance criteria and then make sure we have enough conforming
implementations. Including non-observable criteria on the list means that
the only person who can submit information about a given implementation
is someone able to see the code for that implementation. This would have
been a problem for several past conformance list exercises.

				Ned

Cc: Philip Guenther <[email-masked]>, Ned Freed <[email-masked]>, [email-masked]
To: Alexey Melnikov <[email-masked]>
Date: Wed, 06 Jul 2005 12:23:40 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

> Philip Guenther wrote:

> >Hmm, applying 2047 decoding requires knowing whether the header is
> >'structured': if a header is structured, then encoded words can occur
> >after an open-paren or before a close-paren without intervening
> >whitespace, something which isn't true of unstructured headers.  Do we
> >need to give guidence to implementors on this or just punt it as a
> >general 2047 issue not specific to sieve?
> >
> >
> I say we should punt it, as this affects USEFOR (Usenet Format) WG as well.

Punting seems like the best choice.

				Ned

Date: Wed, 06 Jul 2005 13:37:20 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: Ned Freed <[email-masked]>, [email-masked]
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

Philip Guenther wrote:

>Hmm, applying 2047 decoding requires knowing whether the header is
>'structured': if a header is structured, then encoded words can occur
>after an open-paren or before a close-paren without intervening
>whitespace, something which isn't true of unstructured headers.  Do we
>need to give guidence to implementors on this or just punt it as a
>general 2047 issue not specific to sieve?
>  
>
I say we should punt it, as this affects USEFOR (Usenet Format) WG as well.

To: [email-masked]
Subject: Re: more 3028bis comments
From: Michael Haardt <[email-masked]>
Date: Wed, 06 Jul 2005 11:03:00 +0200
Sender: [email-masked]

>    > Whitespace is used to separate tokens.  Whitespace is made up of
>    > tabs, newlines (CRLF, never just CR or LF), and the space character.
>    > The amount of whitespace used is not significant.
>
> This is specifically about whitespace in the Sieve language.  So... how
> many implementations violate this?  i.e., does everyone generate an
> error if a script's whitespace contains a naked CR or LF ?

I do.  Actually, the Exim implementation does not accept CRLF by default,
but uses just LF or whatever \n is.  You can decide to change that by
recompiling Exim with a different #define or by translating the Sieve
script from CRLF to LF at runtime, which isn't too hard with Exim.
No matter if LF or CRLF is used, anything but the specified line
terminator causes a syntax error.  So far, everybody was happy with that.

3028bis gives a detailed formal specification of lexical tokens.
Anything not covered by that causes a lexical error.  There is no room
for interpretation on that.

Ignoring this lexical error violates the specification, because it changes
the recognised language.  So does changing the specification of CRLF,
like I did.

For a change, I see no need to change the specification. :)

> 2.4.2.4. MIME Parts
>
>    > In a few places, [MIME] body parts are represented as strings.  These
>    > parts include MIME headers and the body.  This provides a way of
>    > embedding typed data within a Sieve script so that, among other
>    > things, character sets other than UTF-8 can be used for output
>    > messages.
>
> This still confuses me.  Where is it referenced?

The vacation extension uses it and I could imagine more extensions to
do so.  It confused me, too.  Perhaps something should be said why the
base spec defines it?

> 2.10.6.  Errors
>
>    > When an error happens, implementations MUST notify the user that an
>    > error occurred, which actions (if any) were taken, and do an implicit
>    > keep.
>
> Probably extremely nit-picky, but I've always wondered when I read
> this "what user, and how do we notify them?"  Some users have no
> access to anything other than their mailboxes.  I suspect that many
> implementations will do some system-wide logging, which notifies the
> admin, but not the user.

That's still on my TODO list.  Since messages are filed to inbox, and
I don't want to potentially double the number of messages, I thought of
adding a header that tells why the message went to the inbox.

> The spec provides no way to access the "From_" line (which is the
> "From<sp>" line with no colon that is added by some mail software.
> While it's not part of RFC2822, and admittedly a minor concern at
> best, that 'header' is often there, but inaccessible.  I have no
> solution, other than perhaps making "From_" a special header name.

Storage specific data should go into storage specific extensions.
With Exim, the BSD mailbox postmark is generated when storing a message,
i.e. as result of a Sieve fileinto, so you could not access it in Sieve,
but that's an implementation detail.  Other implementations might work
the same.  Accessing the current date and time of the sieve execution,
which is the closest you could get, clearly belongs to a different
extension, too.

> There was some discussion about adding character escapes (\u etc);
> this probably falls under the "substantive changes or additions"
> prohibition, but maybe not.

I am afraid it does.  The base spec could say that extensions might
define a new meaning to quoted-other in order to explain why it SHOULD
NOT be used.

Michael

To: Ned Freed <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: My open issues with RFC3028bis 
Date: Tue, 05 Jul 2005 23:54:40 -0700
Sender: [email-masked]

Ned Freed <[email-masked]> writes:
...
>IMO it would be better to say something along the lines of
>
>    Header lines are unfolded as described in [IMAIL] section 2.2.3.
>    Interpretation of header data should be done according to [MIME3]
>    section 6.2.
>
>This makes the required sequence of operations clear. Note that I also removed
>the term "long" - in practice it isn't only long lines that are folded.

I like it, though I think there should be a forward reference to 2.7.2
as that's where the precise requirements are stated.  So:

   Header lines are unfolded as described in [IMAIL] section 2.2.3.
   Interpretation of header data SHOULD be done according to [MIME3]
   section 6.2 (see 2.7.2 below for details).

Hmm, applying 2047 decoding requires knowing whether the header is
'structured': if a header is structured, then encoded words can occur
after an open-paren or before a close-paren without intervening
whitespace, something which isn't true of unstructured headers.  Do we
need to give guidence to implementors on this or just punt it as a
general 2047 issue not specific to sieve?

Philip Guenther

Subject: Re: more 3028bis comments
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Wed, 06 Jul 2005 08:14:23 +0200
Sender: [email-masked]

On Tue, 2005-07-05 at 17:46 -0700, Ned Freed wrote:
> > > The spec provides no way to access the "From_" line (which is the
> > > "From<sp>" line with no colon that is added by some mail software.
> > > While it's not part of RFC2822, and admittedly a minor concern at
> > > best, that 'header' is often there, but inaccessible.  I have no
> > > solution, other than perhaps making "From_" a special header name.
> 
> > we have envelope "from" for the e-mail address.  we don't have anything
> > for the delivery date, but I suspect Ned is considering that for his
> > "date" extension.
> 
> Actually I'm not. The current time, yes, but since the time of sieve evalaution
> isn't exactly specified in terms of its relationship to final delivery it
> doesn't make sense to have this as part of a general date extension. It would
> at best be another, separate extension.

true, it would be best to make it a separate extension so that "date"
deployment is not hampered.  the "deliverydate" extension could probably
be specified by adding a couple of paragraphs to the "date" document,
though.  whether that feels natural we'll see later.

> > > The 'variables' extension has to specify greedy/non-greedy ":matches" ;
> > > this really ought to be in 3028 so that extensions can consistently
> > > follow whatever rule there is; although, again, it might fail the
> > > "non-substantive" test.
> 
> > I don't think this behaviour description is relevant for the main spec.
> > it can not be observed without the variables extension.
> 
> Yes, exactly so. This could easily come back to bite us when we need to move to
> draft.

sorry, I'm dense.  are you agreeing with Mark or with me?

I don't worry about this issue since I trust we will be able to avoid
publishing extensions with incompatible behaviours in the future, even
when the extensions don't depend on each other.  there will always pop
up be cases such as this, and we can't get all such behaviours described
in the base spec.
-- 
Kjetil T.

Cc: Mark E Mallett <[email-masked]>, [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 05 Jul 2005 17:46:18 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: more 3028bis comments
Sender: [email-masked]

> On Tue, 2005-07-05 at 16:52 -0400, Mark E. Mallett wrote:
> > 2.2.     Whitespace
> >
> >    > Whitespace is used to separate tokens.  Whitespace is made up of
> >    > tabs, newlines (CRLF, never just CR or LF), and the space character.
> >    > The amount of whitespace used is not significant.
> >
> > This is specifically about whitespace in the Sieve language.  So... how
> > many implementations violate this?  i.e., does everyone generate an
> > error if a script's whitespace contains a naked CR or LF ?

> does it matter?  this text is unchanged from RFC 3028, so it will not
> affect conformance.

I see no "MUST generate an error" clause here, so as far as I'm concerned the
handling of bare CR and bare LF means you have a nonconformant script, but how
implementations handle this particular form of nonconformance is undefined.

Attempting to nail this down to a "MUST generate an error" would be a huge
mistake IMO, given the use of bare LF and bare CR as the default line
terminators on many systems. And I really don't want to repeat the whole
"canonical format versus local storage format" discussion again, thank you very
much. So let's let this particular sleeping dog lie, OK?

> > 2.10.6.  Errors
> >
> >    > When an error happens, implementations MUST notify the user that an
> >    > error occurred, which actions (if any) were taken, and do an implicit
> >    > keep.
> >
> > Probably extremely nit-picky, but I've always wondered when I read
> > this "what user, and how do we notify them?"  Some users have no
> > access to anything other than their mailboxes.  I suspect that many
> > implementations will do some system-wide logging, which notifies the
> > admin, but not the user.

> good question.  delivering an error report to INBOX in addition to the
> message is one way of solving this, but does anyone do this?

We do. Our implementation has the concept of a "responsible address" that's
attached to each sieve script. When an error occurs this address is notified of
the problem. For user sieves this is the user's own address, system sieves have
the main system postmaster as the responsible address, and so on.

Of course the delivery of such notifications has to be done in a way
that doesn't go through sieve processing and generate an exponentially
growing loop.

> > Other random comments:
> >
> > The spec provides no way to access the "From_" line (which is the
> > "From<sp>" line with no colon that is added by some mail software.
> > While it's not part of RFC2822, and admittedly a minor concern at
> > best, that 'header' is often there, but inaccessible.  I have no
> > solution, other than perhaps making "From_" a special header name.

> we have envelope "from" for the e-mail address.  we don't have anything
> for the delivery date, but I suspect Ned is considering that for his
> "date" extension.

Actually I'm not. The current time, yes, but since the time of sieve evalaution
isn't exactly specified in terms of its relationship to final delivery it
doesn't make sense to have this as part of a general date extension. It would
at best be another, separate extension.

> > The 'variables' extension has to specify greedy/non-greedy ":matches" ;
> > this really ought to be in 3028 so that extensions can consistently
> > follow whatever rule there is; although, again, it might fail the
> > "non-substantive" test.

> I don't think this behaviour description is relevant for the main spec.
> it can not be observed without the variables extension.

Yes, exactly so. This could easily come back to bite us when we need to move to
draft.

				Ned

Subject: Re: more 3028bis comments
From: Kjetil Torgrim Homme <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
Date: Wed, 06 Jul 2005 00:48:12 +0200
Sender: [email-masked]

On Tue, 2005-07-05 at 16:52 -0400, Mark E. Mallett wrote:
> 2.2.     Whitespace
> 
>    > Whitespace is used to separate tokens.  Whitespace is made up of
>    > tabs, newlines (CRLF, never just CR or LF), and the space character.
>    > The amount of whitespace used is not significant.
> 
> This is specifically about whitespace in the Sieve language.  So... how
> many implementations violate this?  i.e., does everyone generate an
> error if a script's whitespace contains a naked CR or LF ?

does it matter?  this text is unchanged from RFC 3028, so it will not
affect conformance.

> 2.4.2.4. MIME Parts
> 
>    > In a few places, [MIME] body parts are represented as strings.  These
>    > parts include MIME headers and the body.  This provides a way of
>    > embedding typed data within a Sieve script so that, among other
>    > things, character sets other than UTF-8 can be used for output
>    > messages.
> 
> This still confuses me.  Where is it referenced?

I think it's only "reject" in the original spec.  I agree the text is
confusingly placed even in RFC 3028, and now that reject has been moved
to a separate document, the above text should follow along.

> 2.10.6.  Errors
> 
>    > When an error happens, implementations MUST notify the user that an
>    > error occurred, which actions (if any) were taken, and do an implicit
>    > keep.
> 
> Probably extremely nit-picky, but I've always wondered when I read
> this "what user, and how do we notify them?"  Some users have no
> access to anything other than their mailboxes.  I suspect that many
> implementations will do some system-wide logging, which notifies the
> admin, but not the user.

good question.  delivering an error report to INBOX in addition to the
message is one way of solving this, but does anyone do this?

> Other random comments:
> 
> The spec provides no way to access the "From_" line (which is the
> "From<sp>" line with no colon that is added by some mail software.
> While it's not part of RFC2822, and admittedly a minor concern at
> best, that 'header' is often there, but inaccessible.  I have no
> solution, other than perhaps making "From_" a special header name.

we have envelope "from" for the e-mail address.  we don't have anything
for the delivery date, but I suspect Ned is considering that for his
"date" extension.

> There was some discussion about adding character escapes (\u etc);
> this probably falls under the "substantive changes or additions"
> prohibition, but maybe not.

Philip has tried to pave the way for it:

   A quoted string starts and ends with a single double quote (the <">
   character, ASCII 34).  A backslash ("\", ASCII 92) inside of a quoted
   string is followed by either another backslash or a double quote.
   This two-character sequence represents a single backslash or double-
   quote within the string, respectively.

   Scripts SHOULD NOT escape other characters with a backslash.

   An undefined escape sequence (such as "\a" in a context where "a" has
   no special meaning) is interpreted as if there were no backslash (in
   this case, "\a" is just "a").

this allows future extensions to define a special meaning for "\a".

(you can find the discussion on this around 2005-03-08)

> The 'variables' extension has to specify greedy/non-greedy ":matches" ;
> this really ought to be in 3028 so that extensions can consistently
> follow whatever rule there is; although, again, it might fail the
> "non-substantive" test.

I don't think this behaviour description is relevant for the main spec.
it can not be observed without the variables extension.
-- 
Kjetil T.

Subject: Re: My open issues with RFC3028bis
From: Kjetil Torgrim Homme <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
Cc: [email-masked]
Date: Wed, 06 Jul 2005 00:04:23 +0200
Sender: [email-masked]

On Tue, 2005-07-05 at 15:30 -0400, Mark E. Mallett wrote:
> I do case-insensitive comparisons, because in general I prefer to ignore
> differences in case unless there's a reason not to; however, our script
> generation tools always generate lowercase.  So changing to case-sensitive
> wouldn't hurt here, and I wouldn't really object to it.
> 
> However, there has been other support for case-insensitivity in other
> discussions; e.g.  for variable names and in the Sieve language
> definition itself.  As I say, I don't particularly object to making
> "require" case-sensitive, but consistency is always good (as is the "be
> liberal in what you accept" tenet).

I agree, case-insensitive matching is pervasive in IETF standards.  e.g.
all MIME types and parameters are case-insensitive, and everything else
in Sieve is case-insensitive.  the problem is that either way we will be
making some existing implementations non-conforming.  if the role of
3028bis is to document current practice, case-sensitive seems to be the
appropriate choice, even if it is a wart.
-- 
Kjetil T.

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 5 Jul 2005 16:52:00 -0400
To: [email-masked]
Cc: "Mark E. Mallett" <[email-masked]>
Subject: more 3028bis comments
Sender: [email-masked]

Here's a few more comments about 3028bis (3028bis-02 specifically,
which is the latest one that I have seen).  Sorry if any of this
has already been addressed.

2.2.     Whitespace

   > Whitespace is used to separate tokens.  Whitespace is made up of
   > tabs, newlines (CRLF, never just CR or LF), and the space character.
   > The amount of whitespace used is not significant.

This is specifically about whitespace in the Sieve language.  So... how
many implementations violate this?  i.e., does everyone generate an
error if a script's whitespace contains a naked CR or LF ?

2.4.2.4. MIME Parts

   > In a few places, [MIME] body parts are represented as strings.  These
   > parts include MIME headers and the body.  This provides a way of
   > embedding typed data within a Sieve script so that, among other
   > things, character sets other than UTF-8 can be used for output
   > messages.

This still confuses me.  Where is it referenced?

2.8.     Blocks

   > A control structure is a command that happens to take a test and a
   > block as one of its arguments; depending on the result of the test
   > supplied as another argument, it runs the code in the block some
   > number of times.

This is inconsistent since "require" and "stop" are defined to be 
control structures, and neither of them take a block as an argument.
Either reword the above, or change 3.2 and 3.3 so that they don't
use the word "structure" (with perhaps an intro in '3' defining
the use of "control structure" vs just "control").

2.9.     Commands

   ...

   > A control command is similar, but it takes a test as an argument, and
   > ends with a block instead of a semicolon.

same sort of thing as previous comment.  Maybe more like:

   A control command is a command that affects the flow of execution of
   the Sieve script in some way.  One form of a control command is a
   "control structure" which ends with a block instead of a semicolon.

   > A test command is used as part of a control command.  It is used to
   > specify whether or not the block of code given to the control command
   > is executed.

similarly, maybe say "control structure command"

2.10.5.  Extensions and Optional Features

   > Because of the differing capabilities of many mail systems, several
   > features of this specification are optional.  Before any of these
   > extensions can be executed, they must be declared with the "require"
   > action.

"require" is defined to be a control.  So... '"require" control' rather
than '"require" action'.

2.10.6.  Errors

   > When an error happens, implementations MUST notify the user that an
   > error occurred, which actions (if any) were taken, and do an implicit
   > keep.

Probably extremely nit-picky, but I've always wondered when I read
this "what user, and how do we notify them?"  Some users have no
access to anything other than their mailboxes.  I suspect that many
implementations will do some system-wide logging, which notifies the
admin, but not the user.

3.      Control Commands
3.2.     Control Structure Require
3.3.     Control Structure Stop

The whole "structure" thing mentioned above.

3.1.     Control Structure If

   > Syntax:   if <test1: test> <block1: block>
   >
   > Syntax:   elsif <test2: test> <block2: block>
   >
   > Syntax:   else <block>

Shouldn't this be "else <block3: block> ?

5.1.     Test address

   > Example:  if address :is :all "from" "[email-masked]" {
   >              discard;

recommend adding the  closing "}" here 

5.8.     Test not

   > Syntax:   not <test>

Shouldn't this be something like  "not <test1: test>"  ?

Other random comments:

The spec provides no way to access the "From_" line (which is the
"From<sp>" line with no colon that is added by some mail software.
While it's not part of RFC2822, and admittedly a minor concern at
best, that 'header' is often there, but inaccessible.  I have no
solution, other than perhaps making "From_" a special header name.

There was some discussion about adding character escapes (\u etc);
this probably falls under the "substantive changes or additions"
prohibition, but maybe not.

The 'variables' extension has to specify greedy/non-greedy ":matches" ;
this really ought to be in 3028 so that extensions can consistently
follow whatever rule there is; although, again, it might fail the
"non-substantive" test.

Yours,
-mm-

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 5 Jul 2005 15:30:03 -0400
To: Ned Freed <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

On Fri, Jul 01, 2005 at 02:30:53PM -0700, Ned Freed wrote:
> 
> > The only feedback I got when I changed it from caseless to caseful was that
> > a regression test failed that used FILEINTO.  That was easily fixed.
> 
> I was frankly surprised that I coded it using a case-sensitive comparison and
> that nobody has complained about it. Had I thought about it I probably would
> have used a case-insensitive comparison.
> 
> Sometimes you get lucky, that's all.

I do case-insensitive comparisons, because in general I prefer to ignore
differences in case unless there's a reason not to; however, our script
generation tools always generate lowercase.  So changing to case-sensitive
wouldn't hurt here, and I wouldn't really object to it.

However, there has been other support for case-insensitivity in other
discussions; e.g.  for variable names and in the Sieve language
definition itself.  As I say, I don't particularly object to making
"require" case-sensitive, but consistency is always good (as is the "be
liberal in what you accept" tenet).

mm

From: "Mark E. Mallett" <[email-masked]>
Date: Tue, 5 Jul 2005 15:00:01 -0400
To: Philip Guenther <[email-masked]>
Cc: Ned Freed <[email-masked]>, Michael Haardt <[email-masked]>, [email-masked]
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

On Fri, Jul 01, 2005 at 05:58:04PM -0700, Philip Guenther wrote:
> Works for me.  2.4.2.2 now reads:

   ...

>    Similarly, synactically invalid header names cause the same result as
>    syntactically valid header names that are not present in the message.
>    In particular, an implementation MUST NOT cause an error for
>    synactically invalid header names.

Would it be reasonable to add that says this refers specifically to
header tests (e.g. "in header tests, ...")?  Otherwise something like
the 'editheader' extension will conflict with this prohibition, as
editheader says (or, at least, implies, and probably should say) that
bad header names are to cause an error.

mm

Subject: Re: My open issues with RFC3028bis
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Tue, 05 Jul 2005 17:09:22 +0200
Sender: [email-masked]

On Tue, 2005-07-05 at 07:48 -0700, Ned Freed wrote:
> IMO it would be better to say something along the lines of
> 
>     Header lines are unfolded as described in [IMAIL] section 2.2.3.
>     Interpretation of header data should be done according to [MIME3]
>     section 6.2.
> 
> This makes the required sequence of operations clear. Note that I also removed
> the term "long" - in practice it isn't only long lines that are folded.

you are of course right, and in that light the reference to [MIME3] may
be redundant.  I think it is a useful reminder for implementors, and
think it is worthwhile to keep it, though.  (it would be strange to
argue otherwise since I messed it up :-)
-- 
Kjetil T.

Cc: Philip Guenther <[email-masked]>, [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 05 Jul 2005 07:48:15 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

> On Fri, 2005-07-01 at 17:58 -0700, Philip Guenther wrote:
> > Works for me.  2.4.2.2 now reads: [...]
> >
> >    Folding of long header lines (as described in [IMAIL] 2.2.3) is
> >    removed prior to interpretation of the data.  The folding syntax (the
> >    CRLF that ends a line plus any leading whitespace at the beginning of
> >    the next line that indicates folding) are interpreted as if they were
> >    a single space.
> >
> > <pause>
> >
> > Hmmm, that last paragraph actually differs from RFC 2822, section 2.2.3,
> > which says:
> >    Unfolding is accomplished by simply removing any CRLF
> >    that is immediately followed by WSP.
> >
> > I.e., the leading whitespace should *not* be treated as a single space
> > but rather be left as is.  Unless I hear screams, I'm going to remove
> > the sentence that starts "The folding syntax..." as conflicting.

> that's fine, but I find it a bit misleading to only refer to [IMAIL],
> since RFC 2047 changes the folding rule in the presence of two
> encoded-words:  any folding whitespace between them must be removed (see
> section 6.2 paragraph 3).

Er, no. RFC 2047 makes no changes to how header fields are folded or
unfolded. Indeed, the word "fold" appears nowhere in RFC 2047.

What RFC 2047 does is state that white space between encoded words should
be ignored when displaying. That's a VERY different thing from changing
the unfolding rule to call for space to be removed entirely under some 
circumstances.

> e.g.,
>    Subject: =?utf-8?q?hell?=
>        =?utf-8?q?o?=
> should be presented as:
>    Subject: hello

And

  Subject: =?utf-8?q?hell?=     =?utf-8?q?o?=

should be displayed in the same way; there is no interaction between the
unfolding operation and the decoding of encoded words.

> on the other hand,
>    Subject: =?utf-8?q?hell?=
>        o
> should be folded into
>    Subject: hell    o

> my suggestion is to add the reference, perhaps like so

>         Long header lines are unfolded (as described in [IMAIL] 2.2.3
>         and [MIME] part 3 6.2) prior to interpretation of the data.

IMO it would be better to say something along the lines of

    Header lines are unfolded as described in [IMAIL] section 2.2.3.
    Interpretation of header data should be done according to [MIME3]
    section 6.2.

This makes the required sequence of operations clear. Note that I also removed
the term "long" - in practice it isn't only long lines that are folded.

				Ned

Subject: Re: My open issues with RFC3028bis
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: [email-masked]
Date: Tue, 05 Jul 2005 15:11:30 +0200
Sender: [email-masked]

On Fri, 2005-07-01 at 17:58 -0700, Philip Guenther wrote:
> Works for me.  2.4.2.2 now reads: [...]
> 
>    Folding of long header lines (as described in [IMAIL] 2.2.3) is
>    removed prior to interpretation of the data.  The folding syntax (the
>    CRLF that ends a line plus any leading whitespace at the beginning of
>    the next line that indicates folding) are interpreted as if they were
>    a single space.
> 
> <pause>
> 
> Hmmm, that last paragraph actually differs from RFC 2822, section 2.2.3,
> which says:
>    Unfolding is accomplished by simply removing any CRLF
>    that is immediately followed by WSP.
> 
> I.e., the leading whitespace should *not* be treated as a single space
> but rather be left as is.  Unless I hear screams, I'm going to remove
> the sentence that starts "The folding syntax..." as conflicting.

that's fine, but I find it a bit misleading to only refer to [IMAIL],
since RFC 2047 changes the folding rule in the presence of two
encoded-words:  any folding whitespace between them must be removed (see
section 6.2 paragraph 3).

e.g.,
   Subject: =?utf-8?q?hell?=
       =?utf-8?q?o?=
should be presented as:
   Subject: hello

on the other hand,
   Subject: =?utf-8?q?hell?=
       o
should be folded into
   Subject: hell    o

my suggestion is to add the reference, perhaps like so

        Long header lines are unfolded (as described in [IMAIL] 2.2.3
        and [MIME] part 3 6.2) prior to interpretation of the data.

-- 
Kjetil T.

Date: Mon, 04 Jul 2005 19:44:21 +0100
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: sieve WG charter
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Philip,
>
> --On June 30, 2005 5:59:02 PM -0700 Philip Guenther 
> <[email-masked]> wrote:
>
>> Hmm, perhaps we can last call this thing just before IETF-63.
>
> That would be great.
>
> As a reminder to all, the document cut-off for the meeting is July 
> 18th, 9am EST.
>
> Apologies from WG co-chair as I have been somewhat out-of-touch on 
> SIEVE stuff this last few months. Will catch up in the next few days.
>
> Things we need to do:
>
> - Update milestones.
> - Finish off last call and IESG submit of editheader and body drafts.
> - Last call vacation (I think its ready to go).
> - Last call 3028bis.

I think we should Last Call 3028bis before vacation. After that we 
should be ready to send Sieve "variables" to IESG.

> - Catch up on all other drafts etc.
> - Agenda for IETF63.

Can I ask authors of documents to write a short paragraph describing the 
list of issues and/or state of the document (Philip has already done his 
bit). You can send this to the list or directly to me & Cyrus. Thanks!

Alexey

Subject: Re: 3028bis open issue #2: unknown envelope-part names
From: Alexandros Vellis <[email-masked]>
To: [email-masked]
Organization: National & Kapodistrian University of Athens
Date: Mon, 04 Jul 2005 21:23:02 +0300
Sender: [email-masked]

On Wed, 2005-06-29 at 18:16 -0400, Mark E. Mallett wrote: 
> On Wed, Jun 29, 2005 at 02:01:31PM -0700, Philip Guenther wrote:
> > 
> > B) require implementations to reject parts other than "from" and "to"
> >    unless declared via some new extension
> 
> I'd go with this one.  I've silently ignored others up until this
> point but have had a long-standing private implementation note that
> it needs to be resolved, vis:
> 
> TODO    /* Not really clear if this is an error- but for now we'll
>            just treat it as a non-match or a zero count.
>         */
> 
> 
> >    CONS: makes Cyrus implementation non-conforming; breaks scripts that
> >          haven't been updated to require the new extension
> 
> There are other dimensions to the question, of course.  One is whether a
> prohibition under (B) will be a "SHOULD NOT" or a "MUST NOT" (MUST NOT
> seems better to me, but SHOULD NOT will work better as a retroactive
> recommendation); another is whether the spec should have an opinion on
> whether the error should be detected at parse/compile time or at
> execution time (I'd either go with either the latter or unspecified).

Based on the above comments, with which I wholeheartedly agree, I would
like to propose actually adding the envelope-auth capability in the base
spec. It is only a matter of one more paragraph clarifying the "auth"
value to the envelope argument.

I would like your comments on the following amendments:

5.4.
(......existing paragraphs:)
   If one of the envelope-part strings is (case insensitive) "from",
   then matching occurs against the FROM address used in the SMTP MAIL
   command.

   If one of the envelope-part strings is (case insensitive) "to", then
   matching occurs against the TO address used in the SMTP RCPT command
   that resulted in this message getting delivered to this user.  Note
   that only the most recent TO is available, and only the one relevant
   to this user.

----------------------------
(....add this one:)

   If one of the envelope-part strings is (case insensitive) "auth",
then
   matching occurs against the value of the AUTH argument to the
   SMTP MAIL FROM command, as described in [SMTP-AUTH], section 5.
   Only implementations that advertise the capability "envelope-auth"
   should accept "auth" in the envelope-part string lists.

----------------------------
(....and this one probably?)

  Any other strings other than "from" or "to", as well as "auth" 
  envelope-auth is supported by the implementation, SHOULD
  result in an error.

(Probably someone can elaborate and describe better than me how this
error could be handled).

----------------------------
(...and this capability registration, but I'm not sure if a more formal
procedure for this is needed:)

   Capability name:        envelope-auth
   Capability keyword:     envelope-auth
   Capability arguments:  as in "envelope"
   Standards Track/IESG-approved experimental RFC number:
           This RFC (Sieve base spec, first revised edition)
   Person and email address to contact for further information:
           The Sieve discussion list <[email-masked]>

----------------------------
...and, LTBNL, probably add this informative reference as well:

[SMTP-AUTH]  J. Myers,  SMTP Service Extension for Authentication, March
1999

Alexandros

From: "Mark E. Mallett" <[email-masked]>
Date: Mon, 4 Jul 2005 13:48:28 -0400
To: Ned Freed <[email-masked]>
Cc: Mark E Mallett <[email-masked]>, [email-masked]
Subject: Re: vacation stuff
Sender: [email-masked]

Hi-

Thanks for the clarifications.

> > I infer from your comments (and correct me if I am wrong; I'm only
> > trying to understand) that you would mandate the use of helper tools in
> > maintaining seive scripts.
> 
> Of course such tools aren't mandatory. But it is silly to pretend that the vast
> majority of users can be well werved by having them construct their own scripts
> by hand, instantiating the underlying services, and so on.

Yes that would be silly.  But that's kind of a flip of what I am saying;

> Our customers employ sieve to provide various services to literally millions of
> end users, including but not limited to vacation reminders. And while there
> have been issues, this sort of thing just doesn't come up at all, ever.

I can't speak for millions, but I completely agree with you that most
people, here too, will use helper tools to create and install Sieve
scripts.  But of course I wasn't trying to address them.  My feeling is
that any script and any defined language should, if it can be arranged
easily, work as well as it can per its definition, without such external
tools.

But I don't really want to repeat myself.  Others can weigh in if they
care, as some have; and (as with the 'auth envelope' thing) I can
certainly have a private extension that enables this sort of facility.
But I did want to bring it up.

Yours,
-mm-

Date: Mon, 04 Jul 2005 17:11:33 +0100
From: Alexey Melnikov <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
CC: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

Alexey Melnikov wrote:

>> another is whether the spec should have an opinion on
>> whether the error should be detected at parse/compile time or at
>> execution time (I'd either go with either the latter or unspecified).
>
> IMHO, my general answer to this is "as soon as possible". But if Sieve 
> "variables" are also implemented, it might not be possible to check 
> this until runtime.

And this also suggests that if such errors are detected at runtime, they 
should not cause script execution error.

Date: Mon, 04 Jul 2005 17:08:42 +0100
From: Alexey Melnikov <[email-masked]>
To: "Mark E. Mallett" <[email-masked]>
CC: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

Mark E. Mallett wrote:

>On Wed, Jun 29, 2005 at 02:01:31PM -0700, Philip Guenther wrote:
>  
>
>>B) require implementations to reject parts other than "from" and "to"
>>   unless declared via some new extension
>>    
>>
>
>I'd go with this one.  I've silently ignored others up until this
>point but have had a long-standing private implementation note that
>it needs to be resolved, vis:
>
>TODO    /* Not really clear if this is an error- but for now we'll
>           just treat it as a non-match or a zero count.
>        */
>  
>
>>   CONS: makes Cyrus implementation non-conforming; breaks scripts that
>>         haven't been updated to require the new extension
>>    
>>
>
>There are other dimensions to the question, of course.  One is whether a
>prohibition under (B) will be a "SHOULD NOT" or a "MUST NOT" (MUST NOT
>seems better to me, but SHOULD NOT will work better as a retroactive
>recommendation);
>
As a user of Cyrus Sieve I would obviously prefer SHOULD NOT here.

> another is whether the spec should have an opinion on
>whether the error should be detected at parse/compile time or at
>execution time (I'd either go with either the latter or unspecified).
>  
>
IMHO, my general answer to this is "as soon as possible". But if Sieve 
"variables" are also implemented, it might not be possible to check this 
until runtime.

Date: Mon, 04 Jul 2005 16:53:18 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: sieve WG charter
Sender: [email-masked]

Philip Guenther wrote:

>Mark E. Mallett <[email-masked]> writes:
>...
>  
>
>>OTOH, I'm probably being dumb again (feel free to agree), but what's the
>>meta-stuff?  i.e.  how has 3028bis come about, what's the charter for
>>its creation, is there a call for submissions of items, etc?
>>    
>>
>
>http://www.ietf.org/html.charters/sieve-charter.html
>
>	(1) Revise the base sieve specification, RFC 3028, with the
>	intention of moving it to draft standard. Substantive
>	additions or revisions to the base specification are out
>	of scope of this working group. However, the need to loosen
>	current restrictions on side effects of tests as well as
>	the need for a normative reference to the newly-defined
>	comparators registry may necessitate a recycle at proposed.
>
>[...]
>
>Similarly, changing section 2.7.2 to require support for 2047 should
>be considered out of scope as well.
>  
>
I think this change can be described as being consistent with the 
original Sieve's intent, so it should be in scope.

>Hmm, perhaps we can last call this thing just before IETF-63.
>  
>
That would be great.

Date: Mon, 04 Jul 2005 15:30:02 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: sieve WG charter
Sender: [email-masked]

Philip Guenther wrote:

>On the issue of unknown envelope-part names, it sounded like the
>consensus was to make them an error (...and encourage the Cyrus people
>to whip up an 'envelope-auth' draft...), yes?
>  
>
I guess I can write a short draft on this, unless somebody else wants to 
do this.

Date: Mon, 04 Jul 2005 13:43:48 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

Philip Guenther wrote:

>   Folding of long header lines (as described in [IMAIL] 2.2.3) is
>   removed prior to interpretation of the data.  The folding syntax (the
>   CRLF that ends a line plus any leading whitespace at the beginning of
>   the next line that indicates folding) are interpreted as if they were
>   a single space.
>
><pause>
>
>Hmmm, that last paragraph actually differs from RFC 2822, section 2.2.3,
>which says:
>   Unfolding is accomplished by simply removing any CRLF
>   that is immediately followed by WSP.
>
>I.e., the leading whitespace should *not* be treated as a single space
>but rather be left as is.  Unless I hear screams, I'm going to remove
>the sentence that starts "The folding syntax..." as conflicting.
>  
>
RFC 2822, section 3.2.3 also contains:

   Runs of FWS, comment or CFWS that occur between lexical tokens in a
   structured field header are semantically interpreted as a single
   space character.

To: [email-masked]
Subject: Re: NUL handling and security considerations	[was: Re: My open issues with RFC3028bis]
From: Michael Haardt <[email-masked]>
Date: Mon, 04 Jul 2005 14:32:27 +0200
Sender: [email-masked]

> > >I may be stretching it too far here, but AFAIK, there are implementations
> > >that truncate strings, thus corrupting test results.  Trying to label them
> > >non-conforming probably won't succeed, but we should not silently ignore
> > >this problem.
>
> > I guess there are two choices:
> > A) Require correct handling of NUL
> > B) Strongly prefer correct handling of NUL and warn about the dangers of
> >    not doing so in the security considerations
>
> I have no major problem with A but I think B is a better choice. FWIW, the
> implementation I work on has no problem handling NULs, but I worry that
> this will make many other implementations non-conforming.

Same here.  So how about a "SHOULD handle NUL characters correctly"?

> > The question is: is there anything we can say on this other than "this
> > is a problem"?  Recommending that sieve implementations be more liberal
> > (than MUAs) doesn't work because the attack works both ways.  Rewriting
> > messages to make them strictly compliant has a horrible history, while
> > rejecting everything that's even slightly suspect is politically
> > impossible.  Do we just document it, grunt, and move on?

I guess so.  It should be documented indeed.

Michael

Date: Mon, 04 Jul 2005 11:52:49 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: 3028bis open issue #3: require 2047 decoding?
Sender: [email-masked]

Philip Guenther wrote:

>Tim Showalter <[email-masked]> writes:
>  
>
>>Michael Haardt wrote:
>>    
>>
>>>Shouldn't the implementation be free what to convert them to? It may
>>>chose a different unicode representation.  Do we need to enforce UTF-8?
>>>      
>>>
>>I realize the language in 3028 clearly implies implementations are 
>>required to convert to UTF-8 but if an implementation wants to use UCS-4 
>>or UCS-2 or UTF-7 internally, that must be allowed.  The specification 
>>has no power to specify behavior that can't be externally observed, and 
>>the text you cited is just wrong.
>>    
>>
>...
>
>For 3028bis, 2.7.2, paragraph 2, how about:
>
>	Comparisons are performed in Unicode.  Implementations convert
>	text from header fields in all charsets [HEADER-CHARSET] to
>	Unicode as input to the comparator (see 2.7.3).  Implementations
>	must be capable of decoding US-ASCII, ISO-8859-1, the US-ASCII
>  
>
Shouldn't this be MUST?

>	subset of ISO-8859-* character sets, and UTF-8.
>
>with the new normative reference:
>
>[HEADER-CHARSET]	Moore, K., "MIME (Multipurpose Internet Mail
>			Extensions) Part Three: Message Header
>			Extensions for Non-ASCII Text", RFC 2047,
>			November 1996
>  
>
Sounds good to me.

>Hmm, I think the paragraph needs to also specify that text in unknown
>charsets never matches, no?
>
>(You can't just map them to U+FFFD (replacement character) because you
>don't know how many characters are encoded!)
>  
>
Alexey

Date: Mon, 04 Jul 2005 11:30:53 +0100
From: Alexey Melnikov <[email-masked]>
To: Michael Haardt <[email-masked]>
CC: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

Michael Haardt wrote:

>>D) roll "auth" into the base-spec
>>   CONS: breaks lots of implementations; have to define what it means in
>>         implementations that don't have access to SMTP/LMTP AUTH info;
>>         why have an extension mechanism if we're not going to use it?
>>   PROS: doesn't break Cyrus implementation or scripts using "auth"
>>    
>>
>
>Authenticated SMTP is common these days and being able to use it in the
>envelope test makes sense to me.  The Exim implementation currently
>rejects anything but "to" and "from", but I had no problem to add
>"auth", thus having one broken implementation less.  Although I don't
>like how Cyrus silently introduced it, it _is_ useful and having to use
>"envelope-auth" besides "envelope" looks awkward to me.  Opinions from
>other implementations? Would the change be a problem?
>
>But: What does "auth" actually do? Does it contain the authenticating ID
>or the authenticated sender?
>
It uses value of the "AUTH" MAIL FROM parameter, which is authentication ID.

> Neither is a problem for me, I just have
>to know.  Implementations that can not access this information could
>behave as with comparing header fields not present in the message.
>  
>
That is why the option B is better: implementation that can't access it 
will not support "auth".

>>B) require implementations to reject parts other than "from" and "to"
>>   unless declared via some new extension
>>   CONS: makes Cyrus implementation non-conforming; breaks scripts that
>>         haven't been updated to require the new extension
>>   PROS: <standard "declare before use" argument>, scripts using "auth"
>>         aren't portable anyway
>>    
>>
>
>If we decide that D is not an option, this is my second choice.  Silently
>ignoring mistakes never pays off in the long run, so let's stay safe
>and clean.  Cyrus should not have introduced a new feature that way,
>but indeed RFC 3028 was not clear how keys other than "from" and "to"
>MUST be processed.
>
>I don't like A and C at all.
>  
>

Date: Mon, 04 Jul 2005 11:25:29 +0100
From: Alexey Melnikov <[email-masked]>
To: Philip Guenther <[email-masked]>
CC: [email-masked]
Subject: Re: 3028bis open issue #2: unknown envelope-part names
Sender: [email-masked]

Philip Guenther wrote:

>Alexandros Vellis <[email-masked]> writes:
>  
>
>>Alexey Melnikov wrote:
>>    
>>
>>>I still think that an implementation should treat an unrecognized 
>>>envelope part as an error. Scripts should not be using a feature without 
>>>a proper require statement.
>>>      
>>>
>>I don't feel strongly about my vote, and I'm not a guru of SMTP
>>protocols either, so as long as the 'auth' is added in the
>>clarification, that's fine with me too. 8-)
>>    
>>
>
>I think we have to choose among the following:
>
>A) leave the behavior undefined if a script uses envelope-parts other
>   than "from" and "to"
>   CONS: such parts (ala "auth") effectively become undeclared, unnamed
>         extensions; scripts can't depend on support; no way to test for
>         support (e.g., in managesieve)
>   PROS: breaks neither existing implementations nor existing scripts
>
>B) require implementations to reject parts other than "from" and "to"
>   unless declared via some new extension
>   CONS: makes Cyrus implementation non-conforming; breaks scripts that
>         haven't been updated to require the new extension
>  
>
Cyrus can be extended to have a compatibility option to accept "auth"
without a proper "require". I don't think this is a big deal.

>   PROS: <standard "declare before use" argument>, scripts using "auth"
>         aren't portable anyway
>
>C) require implementations to ignore all unknown parts, or maybe just
>   "auth"
>   CONS: breaks lots of implementations; script behavior could change if
>         an implementation started recognizing new parts
>   PROS: doesn't break existing scripts using "auth"
>
>D) roll "auth" into the base-spec
>   CONS: breaks lots of implementations; have to define what it means in
>         implementations that don't have access to SMTP/LMTP AUTH info;
>         why have an extension mechanism if we're not going to use it?
>   PROS: doesn't break Cyrus implementation or scripts using "auth"
>
>
>It sounds like you're suggesting (C), yes?  Do yo have any comments on
>the pros and cons in the above list or the tradeoffs among them?
>  
>
I advocate B. I personally don't have a problem with defining "auth" as
an extension in the base document, but I wouldn't object to it being in
a separate document either.

I dislike C and A.

Subject: Re: sieve WG charter
From: Alexandros Vellis <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: [email-masked]
Organization: National & Kapodistrian University of Athens
Date: Mon, 04 Jul 2005 12:37:45 +0300
Sender: [email-masked]

On Fri, 2005-07-01 at 18:43 -0700, Philip Guenther wrote:

> On the issue of unknown envelope-part names, it sounded like the
> consensus was to make them an error (...and encourage the Cyrus people
> to whip up an 'envelope-auth' draft...), yes?

I think we didn't reach a consensus yet. :)

However, I was hoping that we could just resolve this in 3028bis as
well, and not bother with a new envelope-auth draft at all.

I'll follow-up on that thread, though.

Alexandros

To: [email-masked]
Subject: Re: vacation stuff
From: Michael Haardt <[email-masked]>
Date: Mon, 04 Jul 2005 10:41:19 +0200
Sender: [email-masked]

> > I fail to see any real added risk here. Consider: Vacation can be executed at
> > most once per incoming message, so the maximum number of database entries the
> > vacation action can produce per incoming message is one. So, if the intent is
> > to fill up the database, the simplest way to do it is set up regular vacation
> > without a handle setting and send in a bunch of messages from different
> > sources. Note. however, that implementations are free to limit the number of
> > remembered responses, so this trick will only work if the implementation allows
> > it.

The implementation may limit the number of remembered responses, but not
the number of databases.  What's a reasonable lowest limit that scripts
should be able to rely on? 100?

I don't know if it's worth an entry under "security considerations", but
allowing the implementation to limit of number of databases sounds useful
to me.  I never thought about a script using a large number of them.

Michael

To: Barry Leiba <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: 3028bis open issue #3: require 2047 decoding? 
Date: Sat, 02 Jul 2005 22:40:41 -0700
Sender: [email-masked]

Barry Leiba <[email-masked]> writes:
>Ned Freed wrote:
>> Thought I sent a note about this, but cannot find it... Anyway, I
>> think this is fine, although I'd be tempted to out "treat as plain
>> US-ASCII" first on the list.
>
>Definitely first on the list.  <...>

Okay.  I have reordered the list so that the sentence now reads:

   Text that the implementation cannot convert to Unicode for any reason
   MAY be treated as plain US-ASCII (including any [HEADER-CHARSET]
   syntax), omitted, or processed according to local conventions.

Philip Guenther

Date: Sat, 02 Jul 2005 21:31:43 -0400
From: Barry Leiba <[email-masked]>
To: [email-masked]
Subject: Re: 3028bis open issue #3: require 2047 decoding?
Sender: [email-masked]

>>How about the following for the second paragraph of 2.7.2:
>>      Comparisons are performed in Unicode.  Implementations convert
>>      text from header fields in all charsets [HEADER-CHARSET] to
>>      Unicode as input to the comparator (see 2.7.3).  Implementations
>>      MUST be capable of converting US-ASCII, ISO-8859-1, the US-ASCII
>>      subset of ISO-8859-* character sets, and UTF-8.  Text that the
>>      implementation cannot convert to Unicode for any reason, MAY be
>>      omitted, treated as plain US-ASCII (including any [HEADER-CHARSET]
>>      syntax), or processed according to local conventions,
> 
> 
> Thought I sent a note about this, but cannot find it... Anyway, I think this
> is fine, although I'd be tempted to out "treat as plain US-ASCII" first on the
> list.

Definitely first on the list.  I actually think this *is* best practice, 
despite Ned's having said he doesn't think there is a best practice on
this, and, in particular, I'd like to eliminate "omitted".  Consider:

    Subject: =?bogus-charset?Q?Buy Viagra now!?=

Do you really want my rule that says
    if (header :contains ["subject"] ["viagra"]) {
       discard;
       stop;
    }
to be ignored because the spammer put in a bogus character set name
(perhaps purposefully, to screw up Sieve scripts)?

Barry

--
Barry Leiba, Pervasive Computing Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba
http://www.research.ibm.com/spam

Cc: Ned Freed <[email-masked]>, Mark E Mallett <[email-masked]>, [email-masked]
To: Mark E Mallett <[email-masked]>
Date: Sat, 02 Jul 2005 14:36:32 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: vacation stuff
Sender: [email-masked]

> On Fri, Jul 01, 2005 at 11:16:30AM -0700, Ned Freed wrote:
> > > One thing that's bugged me about the vacation extension is that it's not
> > > entirely self-contained, in that the database has to be attended to via
> > > some external means.
> >
> > I, OTOH, believe this to be one of the main strengths of this proposal,
> > and indeed of sieve in general.

> Maybe you could explain more?  The vacation database is entirely owned
> by the delivery application, which ought to be able to completely control
> that data as well, including instantiating it and validating it.

> I infer from your comments (and correct me if I am wrong; I'm only
> trying to understand) that you would mandate the use of helper tools in
> maintaining seive scripts.

Of course such tools aren't mandatory. But it is silly to pretend that the vast
majority of users can be well werved by having them construct their own scripts
by hand, instantiating the underlying services, and so on.

> That's a fine administrative perspective,
> but I wouldn't want to see it as part of the language document, nor
> require other administrators to think the same way.  On the contrary, I
> would hope that, as much as possible, the language would be designed to
> simply work when correct syntax is used.  I've described one problem
> that the vacation extension has in this regard, and suggested a way to
> address it.  There may be better ways; it was just an idea.

> Perhaps I'm alone in this; if so, well, that's the way it goes..

Our customers employ sieve to provide various services to literally millions of
end users, including but not limited to vacation reminders. And while there
have been issues, this sort of thing just doesn't come up at all, ever.

If you want an example of a real issue, the obvious one is sieve script
internationalization. More specifically, given vacation text in multiple
languages, what criteria do you use in your script to figure out which variant
to use?

Part of the problem with this particular issue is that there is often useful
out-of-band information that's available to the messaging system but there's no
way to access it through sieve. And even if I make the information available
through some sort of extension, using existing sieve language facilities to
perform language tag comparison and selection isn't easy.

If I were to propose additions to vacation, it would be in this general area,
not in enhancing the vacation database - what's there already appears to be
sufficient for the vast majority of users and uses.

				Ned

From: "Mark E. Mallett" <[email-masked]>
Date: Sat, 2 Jul 2005 13:53:36 -0400
To: Ned Freed <[email-masked]>
Cc: Mark E Mallett <[email-masked]>, [email-masked]
Subject: Re: vacation stuff
Sender: [email-masked]

On Fri, Jul 01, 2005 at 11:16:30AM -0700, Ned Freed wrote:
> > One thing that's bugged me about the vacation extension is that it's not
> > entirely self-contained, in that the database has to be attended to via
> > some external means.
> 
> I, OTOH, believe this to be one of the main strengths of this proposal,
> and indeed of sieve in general.

Maybe you could explain more?  The vacation database is entirely owned
by the delivery application, which ought to be able to completely control
that data as well, including instantiating it and validating it.

I infer from your comments (and correct me if I am wrong; I'm only
trying to understand) that you would mandate the use of helper tools in
maintaining seive scripts.  That's a fine administrative perspective,
but I wouldn't want to see it as part of the language document, nor
require other administrators to think the same way.  On the contrary, I
would hope that, as much as possible, the language would be designed to
simply work when correct syntax is used.  I've described one problem
that the vacation extension has in this regard, and suggested a way to
address it.  There may be better ways; it was just an idea.

Perhaps I'm alone in this; if so, well, that's the way it goes..

> > Deletion.  Script author removes the vacation statement (or disables
> > it).  Deletion of data has to be done separately.  An environment can
> > certainly provide for this in some site-specific way.  OTOH the data
> > doesn't have to be deleted; in practice it does no harm, and users of
> > other vacation mechanisms probably don't always delete their vacation
> > database either (until the next initialization wipes it).  OTOOH,
> > a script writer who always uses different handles would cause old
> > data to pile up.  Might be worth some words of warning.
> 
> I fail to see any real added risk here. Consider: Vacation can be executed at
> most once per incoming message, so the maximum number of database entries the
> vacation action can produce per incoming message is one. So, if the intent is
> to fill up the database, the simplest way to do it is set up regular vacation
> without a handle setting and send in a bunch of messages from different
> sources. Note. however, that implementations are free to limit the number of
> remembered responses, so this trick will only work if the implementation allows
> it.
> 
> Now suppose we add handle to the mix. It cannot be used to get past the "one
> entry per message" limit - there still has to be one message sent per database
> entry. Can it be used to, say, cause duplicate messages to create separate
> database entries? Sure, but only if you're willing to create a vast long script
> that tests some highly variable piece of message data and switches to a
> different handle on that basis. (Remember that use of vacation parameters MUST
> NOT have had variables expanded prior to their use in response tracking.)

As I said, "in practice it does no harm."  I was simply thinking of the
potential for little bits of data to accumulate over time, especially
with somebody who writes scripts by hand and might choose a different
handle each time.  At any rate it's hardly the point of the message,
it was only a casual side remark. 

Yours,
-mm-

Cc: Ned Freed <[email-masked]>, Michael Haardt <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Sat, 02 Jul 2005 07:45:00 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: 3028bis open issue #3: require 2047 decoding?
Sender: [email-masked]

> Ned Freed <[email-masked]> writes:
> ...
> >I think leaving the encoded word intact in this case is slightly
> >better. But only slightly. The bottom line is that if there's a problem
> >decoding (either because you don't know the encoding or the encoded
> >material is gronked somehow) or converting (either because you don't
> >recognize the charset or the material doesn't actually match the
> >charset), getting totally consistent and reasonable match behavior is
> >next to impossible. As such, any preference we express should be just
> >that: A preference. There's no best practice here as far as I can
> >tell.

> How about the following for the second paragraph of 2.7.2:
>       Comparisons are performed in Unicode.  Implementations convert
>       text from header fields in all charsets [HEADER-CHARSET] to
>       Unicode as input to the comparator (see 2.7.3).  Implementations
>       MUST be capable of converting US-ASCII, ISO-8859-1, the US-ASCII
>       subset of ISO-8859-* character sets, and UTF-8.  Text that the
>       implementation cannot convert to Unicode for any reason, MAY be
>       omitted, treated as plain US-ASCII (including any [HEADER-CHARSET]
>       syntax), or processed according to local conventions,

Thought I sent a note about this, but cannot find it... Anyway, I think this
is fine, although I'd be tempted to out "treat as plain US-ASCII" first on the
list.

				Ned

Cc: Michael Haardt <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Sat, 02 Jul 2005 07:35:13 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: NUL handling and security considerations	[was: Re: My open issues with RFC3028bis]
Sender: [email-masked]

> Michael Haardt <[email-masked]> writes:
> >On Thu, Jun 30, 2005 at 10:37:29PM -0700, Philip Guenther wrote:
> ...
> >[RFC 2047] Section 5, almost at the end:
> >
> >   Only printable and white space character data should be encoded using
> >   this scheme.  However, since these encoding schemes allow the
> >   encoding of arbitrary octet values, mail readers that implement this
> >   decoding should also ensure that display of the decoded data on the
> >   recipient's terminal will not cause unwanted side-effects.
> >
> >NUL is not white space.  If a word contains either, its author did not
> >care about this recommendation is and it may be a good idea to drop it
> >or display its decimal representation or whatever else.  Most likely
> >the above addresses ESC sequences.

> It's valid syntax, and Unicode has a NUL codepoint, so it's not like we
> can claim that something like:

> From: Philip =?US-ASCII?Q?=00_Guenther?= <[email-masked]>

> can't be converted to Unicode, so the new text in 2.7.2 doesn't cover it.

> ...
> >I may be stretching it too far here, but AFAIK, there are implementations
> >that truncate strings, thus corrupting test results.  Trying to label them
> >non-conforming probably won't succeed, but we should not silently ignore
> >this problem.

> I guess there are two choices:
> A) Require correct handling of NUL
> B) Strongly prefer correct handling of NUL and warn about the dangers of
>    not doing so in the security considerations

I have no major problem with A but I think B is a better choice. FWIW, the
implementation I work on has no problem handling NULs, but I worry that
this will make many other implementations non-conforming.

> On that thought, there's a security consideration missing.  As a
> first draft:

>    As with any filter on a message stream, if the sieve implementation
>    and the mail agents 'behind' sieve in the message stream differ in
>    their interpretation of the messages, it may be possible for an
>    attacker to subvert the filter.  Of particular note are differences
>    in the interpretation of malformed messages (e.g., missing or extra
>    syntax characters) or those that exhibit corner cases (e.g., NUL
>    octects encoded via [HEADER-CHARSET]).

> Of course, this problem is not specific to email; I recall reading
> in the last 90s about using differences in IP fragment handling to evade
> networking monitoring devices.  The first case of this problem in email
> that I recall involved virus filters that were stricter about their MIME
> handling than MS Outlook was, such that the filter would fail to scan
> MIME parts which had a syntax error in their MIME headers while Outlook
> would accept them sufficient to be infected.

> The question is: is there anything we can say on this other than "this
> is a problem"?  Recommending that sieve implementations be more liberal
> (than MUAs) doesn't work because the attack works both ways.  Rewriting
> messages to make them strictly compliant has a horrible history, while
> rejecting everything that's even slightly suspect is politically
> impossible.  Do we just document it, grunt, and move on?

Seems like the best we can do, doesn't it?

				Ned

To: Michael Haardt <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: NUL handling and security considerations [was: Re: My open issues with RFC3028bis]
Date: Fri, 01 Jul 2005 22:16:01 -0700
Sender: [email-masked]

Michael Haardt <[email-masked]> writes:
>On Thu, Jun 30, 2005 at 10:37:29PM -0700, Philip Guenther wrote:
...
>[RFC 2047] Section 5, almost at the end:
>
>   Only printable and white space character data should be encoded using
>   this scheme.  However, since these encoding schemes allow the
>   encoding of arbitrary octet values, mail readers that implement this
>   decoding should also ensure that display of the decoded data on the
>   recipient's terminal will not cause unwanted side-effects.
>
>NUL is not white space.  If a word contains either, its author did not
>care about this recommendation is and it may be a good idea to drop it
>or display its decimal representation or whatever else.  Most likely
>the above addresses ESC sequences.

It's valid syntax, and Unicode has a NUL codepoint, so it's not like we
can claim that something like:

From: Philip =?US-ASCII?Q?=00_Guenther?= <[email-masked]>

can't be converted to Unicode, so the new text in 2.7.2 doesn't cover it.

...
>I may be stretching it too far here, but AFAIK, there are implementations
>that truncate strings, thus corrupting test results.  Trying to label them
>non-conforming probably won't succeed, but we should not silently ignore
>this problem.

I guess there are two choices:
A) Require correct handling of NUL
B) Strongly prefer correct handling of NUL and warn about the dangers of
   not doing so in the security considerations

On that thought, there's a security consideration missing.  As a
first draft:

   As with any filter on a message stream, if the sieve implementation
   and the mail agents 'behind' sieve in the message stream differ in
   their interpretation of the messages, it may be possible for an
   attacker to subvert the filter.  Of particular note are differences
   in the interpretation of malformed messages (e.g., missing or extra
   syntax characters) or those that exhibit corner cases (e.g., NUL
   octects encoded via [HEADER-CHARSET]).

Of course, this problem is not specific to email; I recall reading
in the last 90s about using differences in IP fragment handling to evade
networking monitoring devices.  The first case of this problem in email
that I recall involved virus filters that were stricter about their MIME
handling than MS Outlook was, such that the filter would fail to scan
MIME parts which had a syntax error in their MIME headers while Outlook
would accept them sufficient to be infected.

The question is: is there anything we can say on this other than "this
is a problem"?  Recommending that sieve implementations be more liberal
(than MUAs) doesn't work because the attack works both ways.  Rewriting
messages to make them strictly compliant has a horrible history, while
rejecting everything that's even slightly suspect is politically
impossible.  Do we just document it, grunt, and move on?

Philip Guenther

To: Cyrus Daboo <[email-masked]>
cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: sieve WG charter 
Date: Fri, 01 Jul 2005 18:43:15 -0700
Sender: [email-masked]

Cyrus Daboo <[email-masked]> writes:
>Things we need to do:
...
>- Finish off last call and IESG submit of editheader and body drafts.

I believe draft-ietf-sieve-editheader-01.txt is ready to be submitted.
I-D nits catches a few whitespace and line length ticks but nothing
substantive.

'body' will need one more rev to line up with the base-spec revision on
charset conversion requirements.  Once it appears that that issue is
settled in 3028bis, I'll submit body-02.  That should then be ready to
submit to the IESG.

...
>- Last call 3028bis.

Expect a new rev early next week.

Changes from draft-ietf-sieve-3028bis-02.txt
 1. Change "ASCII" to "US-ASCII" throughout
 2. Tweak section 2.7.2 to not require use of UTF-8 internally and
    to explicitly leave implementation-defined the handling of text
    that can't be converted to Unicode
 3. Add reference to RFC 2047
 4. Clarify that capability strings are case-sensitive
 5. Clarify that address, envelope, and header return false if no
    combination of arguments match
 6. Directly state that code that isn't reached may still be checked
    for errors.
 7. Invalid header name syntax is not an error
 8. Remove description of header unfolding that conflicts with
    [IMAIL]

On the issue of unknown envelope-part names, it sounded like the
consensus was to make them an error (...and encourage the Cyrus people
to whip up an 'envelope-auth' draft...), yes?

Philip Guenther

Cc: Ned Freed <[email-masked]>, Michael Haardt <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Fri, 01 Jul 2005 18:29:45 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

> Ned Freed <[email-masked]> writes:
> >Michael Haardt wrote:
> >> Ok, if the specification is to be understood that way, then how about:
> >
> >>   Synactically invalid header names cause the same result as syntactically
> >>   valid header names that are not present in the message.  In particular,
> >>   an implementation MUST not cause an error for synactically invalid
> >>   header names.
> >
> >Very nice. I support adding this text to the document. One change:
> >"MUST not" -> "MUST NOT".

> Works for me.  2.4.2.2 now reads:
>    Headers are a subset of strings.  In the Internet Message
>    Specification [IMAIL], each header line is allowed to have whitespace
>    nearly anywhere in the line, including after the field name and
>    before the subsequent colon.  Extra spaces between the header name
>    and the ":" in a header field are ignored.

>    A header name never contains a colon.  The "From" header refers to a
>    line beginning "From:" (or "From   :", etc.).  No header will match
>    the string "From:" due to the trailing colon.

>    Similarly, synactically invalid header names cause the same result as
>    syntactically valid header names that are not present in the message.
>    In particular, an implementation MUST NOT cause an error for
>    synactically invalid header names.

>    Folding of long header lines (as described in [IMAIL] 2.2.3) is
>    removed prior to interpretation of the data.  The folding syntax (the
>    CRLF that ends a line plus any leading whitespace at the beginning of
>    the next line that indicates folding) are interpreted as if they were
>    a single space.

> <pause>

> Hmmm, that last paragraph actually differs from RFC 2822, section 2.2.3,
> which says:
>    Unfolding is accomplished by simply removing any CRLF
>    that is immediately followed by WSP.

> I.e., the leading whitespace should *not* be treated as a single space
> but rather be left as is.  Unless I hear screams, I'm going to remove
> the sentence that starts "The folding syntax..." as conflicting.

Nice catch. No screams from this quarter - this should align with RFC 2822 IMO.

				Ned

To: Ned Freed <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: My open issues with RFC3028bis 
Date: Fri, 01 Jul 2005 17:58:04 -0700
Sender: [email-masked]

Ned Freed <[email-masked]> writes:
>Michael Haardt wrote:
>> Ok, if the specification is to be understood that way, then how about:
>
>>   Synactically invalid header names cause the same result as syntactically
>>   valid header names that are not present in the message.  In particular,
>>   an implementation MUST not cause an error for synactically invalid
>>   header names.
>
>Very nice. I support adding this text to the document. One change:
>"MUST not" -> "MUST NOT".

Works for me.  2.4.2.2 now reads:
   Headers are a subset of strings.  In the Internet Message
   Specification [IMAIL], each header line is allowed to have whitespace
   nearly anywhere in the line, including after the field name and
   before the subsequent colon.  Extra spaces between the header name
   and the ":" in a header field are ignored.

   A header name never contains a colon.  The "From" header refers to a
   line beginning "From:" (or "From   :", etc.).  No header will match
   the string "From:" due to the trailing colon.

   Similarly, synactically invalid header names cause the same result as
   syntactically valid header names that are not present in the message.
   In particular, an implementation MUST NOT cause an error for
   synactically invalid header names.

   Folding of long header lines (as described in [IMAIL] 2.2.3) is
   removed prior to interpretation of the data.  The folding syntax (the
   CRLF that ends a line plus any leading whitespace at the beginning of
   the next line that indicates folding) are interpreted as if they were
   a single space.

<pause>

Hmmm, that last paragraph actually differs from RFC 2822, section 2.2.3,
which says:
   Unfolding is accomplished by simply removing any CRLF
   that is immediately followed by WSP.

I.e., the leading whitespace should *not* be treated as a single space
but rather be left as is.  Unless I hear screams, I'm going to remove
the sentence that starts "The folding syntax..." as conflicting.

Philip Guenther

To: Ned Freed <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: 3028bis open issue #3: require 2047 decoding? 
Date: Fri, 01 Jul 2005 17:45:56 -0700
Sender: [email-masked]

Ned Freed <[email-masked]> writes:
...
>I think leaving the encoded word intact in this case is slightly
>better. But only slightly. The bottom line is that if there's a problem
>decoding (either because you don't know the encoding or the encoded
>material is gronked somehow) or converting (either because you don't
>recognize the charset or the material doesn't actually match the
>charset), getting totally consistent and reasonable match behavior is
>next to impossible. As such, any preference we express should be just
>that: A preference. There's no best practice here as far as I can
>tell.

How about the following for the second paragraph of 2.7.2:
      Comparisons are performed in Unicode.  Implementations convert
      text from header fields in all charsets [HEADER-CHARSET] to
      Unicode as input to the comparator (see 2.7.3).  Implementations
      MUST be capable of converting US-ASCII, ISO-8859-1, the US-ASCII
      subset of ISO-8859-* character sets, and UTF-8.  Text that the
      implementation cannot convert to Unicode for any reason, MAY be
      omitted, treated as plain US-ASCII (including any [HEADER-CHARSET]
      syntax), or processed according to local conventions,

Philip Guenther

Cc: Michael Haardt <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Fri, 01 Jul 2005 14:37:20 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: 3028bis open issue #3: require 2047 decoding?
Sender: [email-masked]

> Michael Haardt <[email-masked]> writes:
> >On Thu, Jun 30, 2005 at 05:43:53PM -0700, Philip Guenther wrote:
> >> 	Comparisons are performed in Unicode.  Implementations convert
> >> 	text from header fields in all charsets [HEADER-CHARSET] to
> >> 	Unicode as input to the comparator (see 2.7.3).  Implementations
> >> 	must be capable of decoding US-ASCII, ISO-8859-1, the US-ASCII
> >> 	subset of ISO-8859-* character sets, and UTF-8.
> >
> >That sounds good.
> >
> >> Hmm, I think the paragraph needs to also specify that text in unknown
> >> charsets never matches, no?
> >
> >Either that or not decoding it, if it can not be converted.

> So, if the implementation didn't understand the charset, it would
> instead convert the raw ASCII (e.g., "=?charset?Q?blah?=") to Unicode
> (an identity function, yes) and feed that to the comparator?  I guess I
> can see _a_ use to that (scripts could match "=?charset?" to check for
> use of particular charsets that the implementation doesn't support).

> Hmm, I don't see any guidance in RFC 2047 or 2048 that could apply to
> this.  Oh well...

I think leaving the encoded word intact in this case is slightly better. But
only slightly. The bottom line is that if there's a problem decoding (either
because you don't know the encoding or the encoded material is gronked somehow)
or converting (either because you don't recognize the charset or the material
doesn't actually match the charset), getting totally consistent and reasonable
match behavior is next to impossible. As such, any preference we express should
be just that: A preference. There's no best practice here as far as I can tell.

> > But now that you cited the scope of 3028bis, can we do that?

> Good question.  RFC 3028 did not specify how an implementation should
> handle charsets that it doesn't understand, effectively leaving it
> implementation defined.  Is that causing interoperability problems?

I haven't seen actual problems in this space, but others may have
different experiences to report.

> If
> so, then my understanding it that fixing it is in scope.

It's in scope if a fix is possible. Aside from having a minimal set of charsets
you have to support (which I believe we already have) I don't see how to
fix this.

				Ned

Cc: [email-masked]
To: Michael Haardt <[email-masked]>
Date: Fri, 01 Jul 2005 14:30:53 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

> On Thu, Jun 30, 2005 at 03:44:04PM -0700, Ned Freed wrote:
> > >    A header name never contains a colon.  The "From" header refers to a
> > >    line beginning "From:" (or "From   :", etc.).  No header will match
> > >    the string "From:" due to the trailing colon.

> > > If users
> > > write "From:", most likely they made a mistake.  Should that really not
> > > cause an error?
> >
> > If we were specifying this for the first time, yes, absolutely. But we're not
> > doing that.

> Ok, if the specification is to be understood that way, then how about:

>   Synactically invalid header names cause the same result as syntactically
>   valid header names that are not present in the message.  In particular,
>   an implementation MUST not cause an error for synactically invalid
>   header names.

Very nice. I support adding this text to the document. One change: "MUST not"
-> "MUST NOT".

> > (1) What to do about "incorrect" encoded words, where the "incorrectness"
> >     manifests as them being syntactically invalid according to RFC 2047.
> >     In this case I believe RFC 2047 rules should apply: This is simply
> >     literal text, not encoded words.

> Agreed, that is specified clearly.

> > (2) What to do about "incorrect" encoded words where the encoded payload
> >     doesn't match up with the rules for the specified encoding. From what
> >     I can tell RFC 2047 is silent on this issue. However, RFC 2047 does
> >     suggest that encoded-words that specify an unrecognized encoding might
> >     best be treated as ordinary text. I think it appropriate to suggest, but
> >     not require, this handling for encoded words with busted encoded
> >     content.

> What do you mean by ordinary text? Treating them literally or decoding, but
> not converting them?

Treating them literally. Hard to decode something you don't understand the
encoding for.

> > We do case-sensitive matches on require strings, so this is OK as far as we're
> > concerned. Hopefully there isn't a popular implementation that has spawned a
> > corpus of require "ENVELOPE"; scripts out there.

> The only feedback I got when I changed it from caseless to caseful was that
> a regression test failed that used FILEINTO.  That was easily fixed.

I was frankly surprised that I coded it using a case-sensitive comparison and
that nobody has complained about it. Had I thought about it I probably would
have used a case-insensitive comparison.

Sometimes you get lucky, that's all.

				Ned

To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Subject: Re: 3028bis open issue #3: require 2047 decoding? 
Date: Fri, 01 Jul 2005 13:56:21 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Michael Haardt <[email-masked]> writes:
>On Thu, Jun 30, 2005 at 05:43:53PM -0700, Philip Guenther wrote:
>> 	Comparisons are performed in Unicode.  Implementations convert
>> 	text from header fields in all charsets [HEADER-CHARSET] to
>> 	Unicode as input to the comparator (see 2.7.3).  Implementations
>> 	must be capable of decoding US-ASCII, ISO-8859-1, the US-ASCII
>> 	subset of ISO-8859-* character sets, and UTF-8.
>
>That sounds good.
>
>> Hmm, I think the paragraph needs to also specify that text in unknown
>> charsets never matches, no?
>
>Either that or not decoding it, if it can not be converted.

So, if the implementation didn't understand the charset, it would
instead convert the raw ASCII (e.g., "=?charset?Q?blah?=") to Unicode
(an identity function, yes) and feed that to the comparator?  I guess I
can see _a_ use to that (scripts could match "=?charset?" to check for
use of particular charsets that the implementation doesn't support).

Hmm, I don't see any guidance in RFC 2047 or 2048 that could apply to
this.  Oh well...

>But now that you cited the scope of 3028bis, can we do that?

Good question.  RFC 3028 did not specify how an implementation should
handle charsets that it doesn't understand, effectively leaving it
implementation defined.  Is that causing interoperability problems?  If
so, then my understanding it that fixing it is in scope.  If not, then
we shouldn't add new constraints on implementations.  Is there consensus
that it's an interoperability problem with _and_ that we can obtain
consensus on how to resolve it?

Meanwhile, perhaps one of our chairs could speak on the procedural
point?

Philip Guenther

Cc: [email-masked]
To: Mark E Mallett <[email-masked]>
Date: Fri, 01 Jul 2005 11:16:30 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: vacation stuff
Sender: [email-masked]

> One thing that's bugged me about the vacation extension is that it's not
> entirely self-contained, in that the database has to be attended to via
> some external means.

I, OTOH, believe this to be one of the main strengths of this proposal,
and indeed of sieve in general.

> For any given vacation handle there are at least
> the following data duties:

> Instantiation.  This is easy enough: the very first time the vacation
> statement is executed, the implementation can simply create the
> database.

> Deletion.  Script author removes the vacation statement (or disables
> it).  Deletion of data has to be done separately.  An environment can
> certainly provide for this in some site-specific way.  OTOH the data
> doesn't have to be deleted; in practice it does no harm, and users of
> other vacation mechanisms probably don't always delete their vacation
> database either (until the next initialization wipes it).  OTOOH,
> a script writer who always uses different handles would cause old
> data to pile up.  Might be worth some words of warning.

I fail to see any real added risk here. Consider: Vacation can be executed at
most once per incoming message, so the maximum number of database entries the
vacation action can produce per incoming message is one. So, if the intent is
to fill up the database, the simplest way to do it is set up regular vacation
without a handle setting and send in a bunch of messages from different
sources. Note. however, that implementations are free to limit the number of
remembered responses, so this trick will only work if the implementation allows
it.

Now suppose we add handle to the mix. It cannot be used to get past the "one
entry per message" limit - there still has to be one message sent per database
entry. Can it be used to, say, cause duplicate messages to create separate
database entries? Sure, but only if you're willing to create a vast long script
that tests some highly variable piece of message data and switches to a
different handle on that basis. (Remember that use of vacation parameters MUST
NOT have had variables expanded prior to their use in response tracking.)

Constructing a vast script in order to do something with duplicate message that
could much more easily be done with distinct message is a truly negligable risk
IMO.

> Reinstantiaton: Script author re-activates a vacation statement with the
> same handle as before.  Stale data can be a problem here if the re-use
> happens soon after the previous use.

Only if the reactivation mechanism is totally incompetent and doesn't update
the handle.

> One possible solution would be to introduce yet another argument or
> option to the vacation statement, one that gives an instance key.  The
> key could be any string, but there could be recommendations of common
> choices, e.g. a date string 'yyyymmdd' of when the vacation is over
> would be a convenient unique value.  This instance key would be stored
> in the database that's associated with the vacation handle's data;
> whenever a vacation statement is executed, if the stored key doesn't
> match the specified key, the data would be re-initialized.

I think the costs of adding this far exsceed the benefits.

> Alternatively, one could make it explicitly an ":end" value, so that
> vacation action stops working on that date even if the author doesn't
> update the script right away.

This is really a different issue, and one that is best addressed with
a date sieve test (which is planned).

> A downside would be that if the script has related vacation statements
> in multiple places (which is sort of the point of :handle), the author
> would have to be careful to keep the instance keys the same.  But that's
> only a slightly greater burden than keeping the handles the same.

> Thoughts?

I'm afraid I see nothing here that warrants an addition to the vacation
specification.

				Ned

From: "Mark E. Mallett" <[email-masked]>
Date: Fri, 1 Jul 2005 13:25:00 -0400
To: [email-masked]
Subject: vacation stuff
Sender: [email-masked]

Hi-

One thing that's bugged me about the vacation extension is that it's not
entirely self-contained, in that the database has to be attended to via
some external means.  For any given vacation handle there are at least
the following data duties:

Instantiation.  This is easy enough: the very first time the vacation
statement is executed, the implementation can simply create the
database.

Deletion.  Script author removes the vacation statement (or disables
it).  Deletion of data has to be done separately.  An environment can
certainly provide for this in some site-specific way.  OTOH the data
doesn't have to be deleted; in practice it does no harm, and users of
other vacation mechanisms probably don't always delete their vacation
database either (until the next initialization wipes it).  OTOOH,
a script writer who always uses different handles would cause old
data to pile up.  Might be worth some words of warning.

Reinstantiaton: Script author re-activates a vacation statement with the
same handle as before.  Stale data can be a problem here if the re-use
happens soon after the previous use.

One possible solution would be to introduce yet another argument or
option to the vacation statement, one that gives an instance key.  The
key could be any string, but there could be recommendations of common
choices, e.g. a date string 'yyyymmdd' of when the vacation is over
would be a convenient unique value.  This instance key would be stored
in the database that's associated with the vacation handle's data;
whenever a vacation statement is executed, if the stored key doesn't
match the specified key, the data would be re-initialized.

Alternatively, one could make it explicitly an ":end" value, so that
vacation action stops working on that date even if the author doesn't
update the script right away.

A downside would be that if the script has related vacation statements
in multiple places (which is sort of the point of :handle), the author
would have to be careful to keep the instance keys the same.  But that's
only a slightly greater burden than keeping the handles the same.

Thoughts?

mm

From: "Mark E. Mallett" <[email-masked]>
Date: Fri, 1 Jul 2005 13:21:34 -0400
To: Philip Guenther <[email-masked]>
Cc: "Mark E. Mallett" <[email-masked]>, [email-masked]
Subject: Re: sieve WG charter
Sender: [email-masked]

On Thu, Jun 30, 2005 at 05:59:02PM -0700, Philip Guenther wrote:
> Mark E. Mallett <[email-masked]> writes:
> ...
> >OTOH, I'm probably being dumb again (feel free to agree), but what's the
> >meta-stuff?  i.e.  how has 3028bis come about, what's the charter for
> >its creation, is there a call for submissions of items, etc?
> 
> http://www.ietf.org/html.charters/sieve-charter.html

Right.  Check.  I knew "dumb" would be the answer :-)

Thanks,
mm

Date: Fri, 01 Jul 2005 09:58:51 -0400
From: Cyrus Daboo <[email-masked]>
To: Philip Guenther <[email-masked]>, "Mark E. Mallett" <[email-masked]>
cc: [email-masked]
Subject: Re: sieve WG charter
Sender: [email-masked]

Hi Philip,

--On June 30, 2005 5:59:02 PM -0700 Philip Guenther 
<[email-masked]> wrote:

> Hmm, perhaps we can last call this thing just before IETF-63.
>

That would be great.

As a reminder to all, the document cut-off for the meeting is July 18th, 
9am EST.

Apologies from WG co-chair as I have been somewhat out-of-touch on SIEVE 
stuff this last few months. Will catch up in the next few days.

Things we need to do:

- Update milestones.
- Finish off last call and IESG submit of editheader and body drafts.
- Last call vacation (I think its ready to go).
- Last call 3028bis.
- Catch up on all other drafts etc.
- Agenda for IETF63.

-- 
Cyrus Daboo

Date: Fri, 1 Jul 2005 11:12:31 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

On Thu, Jun 30, 2005 at 03:44:04PM -0700, Ned Freed wrote:
> >    A header name never contains a colon.  The "From" header refers to a
> >    line beginning "From:" (or "From   :", etc.).  No header will match
> >    the string "From:" due to the trailing colon.

> > If users
> > write "From:", most likely they made a mistake.  Should that really not
> > cause an error?
> 
> If we were specifying this for the first time, yes, absolutely. But we're not
> doing that.

Ok, if the specification is to be understood that way, then how about:

  Synactically invalid header names cause the same result as syntactically
  valid header names that are not present in the message.  In particular,
  an implementation MUST not cause an error for synactically invalid
  header names.

> (1) What to do about "incorrect" encoded words, where the "incorrectness"
>     manifests as them being syntactically invalid according to RFC 2047.
>     In this case I believe RFC 2047 rules should apply: This is simply
>     literal text, not encoded words.

Agreed, that is specified clearly.

> (2) What to do about "incorrect" encoded words where the encoded payload
>     doesn't match up with the rules for the specified encoding. From what
>     I can tell RFC 2047 is silent on this issue. However, RFC 2047 does
>     suggest that encoded-words that specify an unrecognized encoding might
>     best be treated as ordinary text. I think it appropriate to suggest, but
>     not require, this handling for encoded words with busted encoded
>     content.

What do you mean by ordinary text? Treating them literally or decoding, but
not converting them?

> We do case-sensitive matches on require strings, so this is OK as far as we're
> concerned. Hopefully there isn't a popular implementation that has spawned a
> corpus of require "ENVELOPE"; scripts out there.

The only feedback I got when I changed it from caseless to caseful was that
a regression test failed that used FILEINTO.  That was easily fixed.

Michael

Date: Fri, 1 Jul 2005 11:02:27 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: My open issues with RFC3028bis
Sender: [email-masked]

On Thu, Jun 30, 2005 at 10:37:29PM -0700, Philip Guenther wrote:
> Is it unclear that an 'address' or 'header' test of header fields
> that are not present in the actual message evaluates to false?

No, that's fine.  It is unclear what a test of header fields does
if the specified field name is syntactically invalid.

Compare this with redirect: What happens if a redirect address string
does not contain a syntactically valid address?

All it says is:

   Implementations MUST ensure that the addresses are syntactically
   valid, but need not ensure that they actually identify an email
   recipient.

So whatever happens, implementations MUST not execute redirect with
an invalid address.  But what else will be the result? An error?

> By 'base spec' you mean the MIME base spec(s) and not the sieve
> base spec?

Yes.

> Regarding RFC 2047, section 6.3, I would read that as permitting
> implementations to skip malformed encoded words, even as it bans
> them from refusing to process the message.

That makes sense, now that I read it again, as does section 6.2,
which (among other) allows treating it literally.

> Regarding the NUL character issue, you wrote:
> >>Note that by considering Sieve to be a MUA, RFC 2047 can be
> >>interpreted in a way that NUL characters truncating strings is
> >>allowed for Sieve implementations, although not recommended.  It
> >>is further allowed to use encoded NUL characters in headers, but
> >>that's not recommended either.

Section 5, almost at the end:

   Only printable and white space character data should be encoded using
   this scheme.  However, since these encoding schemes allow the
   encoding of arbitrary octet values, mail readers that implement this
   decoding should also ensure that display of the decoded data on the
   recipient's terminal will not cause unwanted side-effects.

NUL is not white space.  If a word contains either, its author did not
care about this recommendation is and it may be a good idea to drop it
or display its decimal representation or whatever else.  Most likely
the above addresses ESC sequences.

NUL characters do truncate strings in some Sieve implementations, clearly
an unwanted side effect of something not being printable or white space.
The Sieve implementation should ensure that does not happen, but it's just
"should", not MUST, as MUA implementations may display ESC sequences,
even if that's a bad idea.

I may be stretching it too far here, but AFAIK, there are implementations
that truncate strings, thus corrupting test results.  Trying to label them
non-conforming probably won't succeed, but we should not silently ignore
this problem.

> 	Capability string are case-sensitive; for example, "foo"
> 	and "FOO" are different capabilities.

Good.

>    Implementations MUST perform syntactic, semantic, and run-time checks
>    on code that is actually executed.  Implementations MAY perform those
>    checks or any part of them on code that is not reached during
>    execution.

Much better than my suggestion!

Michael

Date: Fri, 1 Jul 2005 10:22:31 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: 3028bis open issue #3: require 2047 decoding?
Sender: [email-masked]

On Thu, Jun 30, 2005 at 05:43:53PM -0700, Philip Guenther wrote:
> 	Comparisons are performed in Unicode.  Implementations convert
> 	text from header fields in all charsets [HEADER-CHARSET] to
> 	Unicode as input to the comparator (see 2.7.3).  Implementations
> 	must be capable of decoding US-ASCII, ISO-8859-1, the US-ASCII
> 	subset of ISO-8859-* character sets, and UTF-8.

That sounds good.

> Hmm, I think the paragraph needs to also specify that text in unknown
> charsets never matches, no?

Either that or not decoding it, if it can not be converted.

But now that you cited the scope of 3028bis, can we do that?

   If implementations fail to support the above behavior, they MUST
   conform to the following:

      No two strings can be considered equal if one contains octets
      greater than 127.

If we agree that this only concers implementations that do not support
Unicode at all, of course we are free to specify what the result of
unicode supporting implementations is, if the conversion to it fails.
But I am not sure how exactly this is meant.

Michael