Subject: Re: Conflict of draft-martin-managesieve-07.txt vs draft-ietf-sieve-3028bis-12.txt

From: Alexey Melnikov <[email-masked]>
Date: Fri, 13 Apr 2007 08:47:14 +0100

Stephan Bosch wrote:

> Hello Alexey,

Hi Stephan,

> I am implementing a managesieve daemon and it seems that recent 
> (draft) changes to the sieve specification will conflict with the 
> current managesieve specification (at least the examples given therein).
>
> The new sieve specification states in chapter 6 'Extensibility':
>
> Any extensions to this language MUST define a capability string that 
> uniquely identifies that extension.  Capability string are case- 
> sensitive; for example, "foo" and "FOO" are different capabilities. If 
> a new version of an extension changes the functionality of a 
> previously defined extension, it MUST use a different name. Extensions 
> may register a set of related capabilities by registering just a 
> unique prefix for them.  The "comparator-" prefix is an example of 
> this.  The prefix MUST end with a "-" and MUST NOT overlap any 
> existing registrations.
>
> The latest managesieve draft gives the following example in section 
> 1.8 'Capabilities':
>
> S: "IMPlemENTATION" "Example1 ManageSieved v001"
> S: "SASl" "DIGEST-MD5 GSSAPI"
> S: "SIeVE" "FILEINTO VACATION"
> S: "StaRTTLS"
> S: "NOTIFY" "xmpp mailto"
> S: OK
>
> The capability fileinto for example is replied in upper case and the 
> sieve specifications currently define all extensions in lower case.

Good catch. I've fixed examples in my copy, so this will be fixed in -08.

> I based my original implementation on this example and I encountered 
> conflicts with some clients. I initially assumed these clients were 
> the cause. But, actually this is not quite clear from the original 
> Sieve specification. The new draft Sieve specification explains this 
> much more explicitly, so you might want to update the managesieve 
> specification in this respect as well. 

Regards,
Alexey

---

From: "Aaron Stone" <[email-masked]>
Date: Fri, 13 Apr 2007 18:36:38 -0000

> Good catch. I've fixed examples in my copy, so this will be fixed in -08.

I still don't understand why we've created the capability prefix registry
and then decided not to use it for notify. This seems silly:

  S: "Implementation" "ManageSieved"
  S: "SASL" "DIGEST-MD5"
> S: "SIEVE" "fileinto vacation comparator-i;ascii-numeric"
  S: "STARTTLS"
  S: OK

I suggested this, but Alexey and others pointed out that it breaks extant
servers and clients quite badly:

  S: "Implementation" "ManageSieved"
  S: "SASL" "DIGEST-MD5"
  S: "SIEVE" "fileinto vacation"
  S: "STARTTLS"
> S: "COMPARATOR" "i;ascii-numeric"
  S: OK

Since we have existing implementations that expect comparator-foo, and we
created an IANA registry for extension prefixes, notify should use that:

  S: "Implementation" "ManageSieved"
  S: "SASL" "DIGEST-MD5"
> S: "SIEVE" "fileinto vacation comparator-i;ascii-numeric notify-xmpp
> S:  notify-mailto"
  S: "STARTTLS"
  S: OK

In draft-ietf-notify-07, I suggest removing this from IANA:

8.  IANA Considerations

   The following template specifies the IANA registration of the notify
   Sieve extension specified in this document:

-   To: [email-masked]
-   Subject: Registration of new Sieve extension
-   Capability name: enotify
-   Description: adds the 'notify' action for notifying user about the
-   received message.  It also provides two new test: valid_notify_method
-   checks notification URIs for validity; notify_method_capability can
-   check recipients capabilities.
-   RFC number: this RFC
-   Contact address:
-       The Sieve discussion list <[email-masked]>

   This information should be added to the list of sieve extensions
   given on http://www.iana.org/assignments/sieve-extensions.

And replace with this (follows the format used in draft-3028bis-12):

   Capability name: enotify
   Description:     adds the 'notify' action for notifying a user via
                    some external method that a message has arrived,
                    adds the 'valid_notification_method' test to check
                    a notification URI for validity,
                    adds the 'notify_method_capability' test to check
                    if a notification method supports additional
                    capabilities.                    
                    transport sender and recipient address
   RFC number:      this RFC (Sieve notify base spec)
   Contact address: The Sieve discussion list <[email-masked]>

   Capability name: notify-* (anything starting with "notify-")
   Description:     adds the indicated notification method for use
                    with the notify action
   RFC number:      this RFC (Sieve notify base spec)
   Contact address: The Sieve discussion list <[email-masked]>

I think we also need a notification capability registry for the
notification method documents to register their method-specific
capabilities with. The test in draft-ietf-sieve-notify-07.txt is this:

   This document defines a single notification-capability value
   "online", which is described below.  Additional notification-
   capability values may be defined by a Standard Track or Experimental
   RFC.

Is that sufficient without an IANA registry?

Aaron

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 26 Apr 2007 10:15:31 +0100

Aaron Stone wrote:

 [...]

>Is that sufficient without an IANA registry?
There is no need to create an IANA registry unless the WG believes that 
there would be many different capabilities in the future.

I don't think there would be that many <notification-capability>s.

Opinions?

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 26 Apr 2007 10:25:02 +0100

Aaron Stone wrote:

[...]

>  S: "SIEVE" "fileinto vacation comparator-i;ascii-numeric notify-xmpp

Currently notification methods are not Sieve extensions. Strictly
speaking they are in separate namespaces.
One implication of your second suggestion is that 'require
"notify-xmpp";' is now legal. (This is not necessarily bad, but I don't
think this is necessary.)

Frankly, I prefer your '"COMPARATOR" "i;ascii-numeric"' idea, except
that it breaks backward compatibility.

>                    if a notification method supports additional

I can use your text. Thanks.

>   Contact address: The Sieve discussion list <[email-masked]>

I need to poll the WG regarding the second change.

Date: Thu, 26 Apr 2007 10:25:02 +0100
From: Alexey Melnikov <[email-masked]>
To: Aaron Stone <[email-masked]>
CC: Stephan Bosch <[email-masked]>, [email-masked]
Subject: Sieve NOTIFY and ManageSieve
Sender: [email-masked]

Aaron Stone wrote:

[...]

>  S: "SIEVE" "fileinto vacation comparator-i;ascii-numeric notify-xmpp

Currently notification methods are not Sieve extensions. Strictly
speaking they are in separate namespaces.
One implication of your second suggestion is that 'require
"notify-xmpp";' is now legal. (This is not necessarily bad, but I don't
think this is necessary.)

Frankly, I prefer your '"COMPARATOR" "i;ascii-numeric"' idea, except
that it breaks backward compatibility.

>                    if a notification method supports additional

I can use your text. Thanks.

>   Contact address: The Sieve discussion list <[email-masked]>

I need to poll the WG regarding the second change.

Date: Thu, 26 Apr 2007 10:15:31 +0100
From: Alexey Melnikov <[email-masked]>
To: Aaron Stone <[email-masked]>
CC: [email-masked]
Subject: IANA registry for Sieve NOTIFY <notification-capability>
Sender: [email-masked]

Aaron Stone wrote:

 [...]

Opinions?

Cc: Philip Guenther <[email-masked]>, Kjetil Torgrim Homme <[email-masked]>, Ned Freed <[email-masked]>, [email-masked]
Date: Wed, 25 Apr 2007 11:48:48 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Editheader issues
To: Mark E Mallett <[email-masked]>
Sender: [email-masked]

> On Tue, Apr 24, 2007 at 05:41:31PM -0600, Philip Guenther wrote:
> >
> > On Wed, 25 Apr 2007, Kjetil Torgrim Homme wrote:
> > >On Tue, 2007-04-24 at 14:21 -0700, Ned Freed wrote:
> > ...
> > >>We also need to specify what happens if :last is specified in
> > >>deleteheader without :index. I suggest saying that it will be ignored.
> > >>The alternative is for it to assume an index of 1, but clarity should
> > >>trump having a more concise way of specifying "delete the last field"
> > >
> > >I think that's a syntax error according to the draft.  if :index <n>
> > >and :last were independently optional, I would agree with your analysis.
> >
> > Right.  SMI's implementation has always considered
> > 	deleteheader :last "foo";
> >
> > to be a syntax error, as reflected in the grammar.

> Hmm, so it is.  I have explicitly detected the missing :index and
> defaulted to 1, but I'm happy to correct that... guess I will do it
> now while I'm thinking about it.

I have no problem with this being an error, but I'm a little leery about
depending on the grammar to call it out as such, especially since the grammar
literally implies that :last has to follow :index, which of course it doesn't
since tagged arguments can be given in any order. And yes, I'm well aware that
the practice of using ABNF-like rules but saying "the order imposed by the
grammar is not actually required", but I worry about people inferring that
other grammar-imposed restrictions are simply artifacts.

> -mm-  (then again, I still have "replaceheader")

Yeah, me too... And what a pain it was to implement!

				Ned

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 25 Apr 2007 12:17:51 -0400
To: Philip Guenther <[email-masked]>
Cc: Kjetil Torgrim Homme <[email-masked]>, Ned Freed <[email-masked]>, [email-masked]
Subject: Re: Editheader issues
Sender: [email-masked]

On Tue, Apr 24, 2007 at 05:41:31PM -0600, Philip Guenther wrote:
> 
> On Wed, 25 Apr 2007, Kjetil Torgrim Homme wrote:
> >On Tue, 2007-04-24 at 14:21 -0700, Ned Freed wrote:
> ...
> >>We also need to specify what happens if :last is specified in 
> >>deleteheader without :index. I suggest saying that it will be ignored. 
> >>The alternative is for it to assume an index of 1, but clarity should 
> >>trump having a more concise way of specifying "delete the last field"
> >
> >I think that's a syntax error according to the draft.  if :index <n>
> >and :last were independently optional, I would agree with your analysis.
> 
> Right.  SMI's implementation has always considered
> 	deleteheader :last "foo";
> 
> to be a syntax error, as reflected in the grammar.

Hmm, so it is.  I have explicitly detected the missing :index and
defaulted to 1, but I'm happy to correct that... guess I will do it
now while I'm thinking about it.

-mm-  (then again, I still have "replaceheader")

Date: Wed, 25 Apr 2007 15:29:45 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked]
Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>>>   "$${hex:40}" -> "$@"
>>>   "${hex: 40 }" -> "@"
>>>   "${HEX: 40}" -> "@"
>>>      
>>>
>the second is superfluous.  
>
It shows a trailing space, so I let it be.

>>>   "${unicode:40}" -> "@"
>>>   "${ unicode:40}" -> "${ unicode:40}"
>>>   "${UNICODE:40}" -> "@"      
>>>
>
>if we include the last example, the first in this set is superfluous
>IMHO.  I'd like one of the examples to include backslash quoting, we
>could extend that example, e.g., "\${UNICODE:40}" -> "@"
>
I didn't delete the last one, but more examples is better than less.

>>>   "${Unicode:40}" -> "@"      
>unnecessary, IMHO.
This one wasn't removed either.

Date: Wed, 25 Apr 2007 14:43:16 +0200
From: Dilyan Palauzov <[email-masked]>
To: [email-masked]
Subject: notify from: / Re: Minutes from the Prague IETF meeting
Sender: [email-masked]

   Hello,

   In the minutes is mentioned "Sendmail implementation". Is there any?

   Below is the extract from the minutes that talks about the notify  
from: issue.
   I do not see a reason to have different logic when sending a  
notification to the recipient, and when sending an out-of-office  
message to the sender. Namely:
   - if from: is applied, use its value
   - if a mail is send (envelopely) to the address [email-masked], use  
this as notify from:
   - mailbox name and message headers do not count
   - if a mail is sent (in the same envelope) to the same maibox (and  
therefore user), mentioning more than one of her/his addresses in the  
envelope, then which of them to use in the notify? e.g.
   Mail from: peter
   Rcpt to: [email-masked]
   Rcpt to: [email-masked]

and postmaster and webmaster are the same mailbox.

   Well, I don't know if in this case one or two notifications shall be send.

   And I don't want if in similar case an autoresponder shall answer,  
if he has to send one or two vacation messages.

   Със здраве,
     Дилян

> 3. Message origin: does the notification message come from the original
> "RCPT TO", or from the notification system?
>
> Alexey pointed out a suggestion to make it an option.
> Pete asked what the From is for and whether we were going to reply to
> these messages.
>
> Philip: if the notification system can autoprocess bounces, then it
> belongs in the envelope MAIL FROM
> Maybe always is better, to block the loop possibility.
>
> Dave Cridland: not clear why worry about "from" for notifications: are
> they to send msgs *to* a user, not necessarily be *from* anyone in  
> particular.
>
> Philip/Pete: The important 'from' here is the envelope from. IMHO, the
> env-from should *never* be the original.
>
> No clear consensus on this issue in the room.

Date: Wed, 25 Apr 2007 11:38:17 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Sieve session during the upcoming Chicago IETF meeting
Sender: [email-masked]

I've requested a 2 hour slot in Chicago. The first level conflicts 
listed are:

 imapext lemonade sasl usefor eai calsify dkim kitten

Are there any other conflicts people would like to be included?

Date: Wed, 25 Apr 2007 11:15:48 +0100
From: Alexey Melnikov <[email-masked]>
To: Michael Haardt <[email-masked]>
CC: [email-masked]
Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
Sender: [email-masked]

Michael Haardt wrote:

>>>>   "${unicode:1000000}" -> error
>>>>   "${unicode:200000}"  -> error
>>>>        
>>>>
>Now that we dropped the 6-digit-restriction, we no longer need two examples
>of range violations.
>  
>
Ok.
I've removed the first one.

Date: Wed, 25 Apr 2007 11:15:27 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked]
Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>>>   "${hex:40${hex:40}}" -> "${hex:40@}"
>>>      
>>>
>"${hex:40${hex:30}}" -> "${hex:400}"
>
>would demonstrate that only one pass is done, too.
>  
>
I've dropped one 0 (i.e. "${hex:4${hex:30}}"), as 400 is invalid hex anyway.

>>>   "${unicode:40}" -> "@"
>>>   "${ unicode:40}" -> "${ unicode:40}"
>>>   "${UNICODE:40}" -> "@"
>>>      
>>>
>if we include the last example, the first in this set is superfluous
>IMHO.  I'd like one of the examples to include backslash quoting, we
>could extend that example, e.g., "\${UNICODE:40}" -> "@"
>  
>
3028bis-12, section 2.4.2 says:

A quoted string starts and ends with a single double quote (the <"> 
character, US-ASCII 34). A backslash ("\", US-ASCII 92) inside of a 
quoted string is followed by either another backslash or a double quote. 
These two-character sequences represent a single backslash or double 
quote within the value, respectively.

Scripts SHOULD NOT escape other characters with a backslash.

So I would rather not demonstrate something which violates the SHOULD NOT.

Date: Tue, 24 Apr 2007 17:41:31 -0600
From: Philip Guenther <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
cc: Ned Freed <[email-masked]>, [email-masked]
Subject: Re: Editheader issues
Sender: [email-masked]

On Wed, 25 Apr 2007, Kjetil Torgrim Homme wrote:
> On Tue, 2007-04-24 at 14:21 -0700, Ned Freed wrote:
...
>> We also need to specify what happens if :last is specified in 
>> deleteheader without :index. I suggest saying that it will be ignored. 
>> The alternative is for it to assume an index of 1, but clarity should 
>> trump having a more concise way of specifying "delete the last field"
>
> I think that's a syntax error according to the draft.  if :index <n>
> and :last were independently optional, I would agree with your analysis.

Right.  SMI's implementation has always considered
 	deleteheader :last "foo";

to be a syntax error, as reflected in the grammar.

Philip Guenther

Subject: Re: Editheader issues
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Wed, 25 Apr 2007 01:09:42 +0200
Sender: [email-masked]

On Tue, 2007-04-24 at 14:21 -0700, Ned Freed wrote:
> I think the only sensible thing is for deleteheader to act immediately and
> renumber.

agreed.

> We also need to specify how addheader interacts with deleteheader. The
> following should be essentially a no-op IMO:
> 
>    require "editheader";
>    addheader "X-test" "whatever";
>    deleteheader :index 1 "X-Test"

yes.

> A couple of other issues occured to me while looking at this general area. First, we need to
> specify what happens if the comparator or match-type arguments are present in
> deleteheader but no value-pattern(s) is(are) specified. I suggest saying that
> the comparator and match-type are ignored in this case.

I don't think it needs spelling out, but I won't object either.

> We also need to specify what happens if :last is specified in deleteheader
> without :index. I suggest saying that it will be ignored. The alternative is
> for it to assume an index of 1, but clarity should trump having a more concise
> way of specifying "delete the last field"

I think that's a syntax error according to the draft.  if :index <n>
and :last were independently optional, I would agree with your analysis.

-- 
Kjetil T.

Date: Tue, 24 Apr 2007 14:21:39 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Editheader issues
To: [email-masked]
Sender: [email-masked]

We jyst had an interesting bug filed on our editheader implementation. The
header in question contained:

  X-Test: instance 1
  X-Test: instance 2
  X-Test: instance 3
  X-Test: instance 4
  X-Test: instance 5
  X-Test: instance 6

And here's the sieve script:

  require "editheader";
  deleteheader :index 1 "X-Test";
  deleteheader :index 2 "X-Test";
  deleteheader :index 3 "X-Test";
  deleteheader :index 4 "X-Test";
  deleteheader :index 5 "X-Test";
  deleteheader :index 6 "X-Test";

They expected the resulting header not to contain any X-Test fields. What they
ended up with was:

  X-Test: instance 2
  X-Test: instance 4
  X-Test: instance 6

The issue, of course, is whether deleteheader acts immediately and removes the
headser in question, renumbering the remaining fields with the same label, or
whether it removes the field but keeps the numbering. The draft isn't specific
about the interaction of deleteheader with itself, however, it does say that
any edits you make are immediately visible to subsequent header and address
tests.

I think the only sensible thing is for deleteheader to act immediately and
renumber. Having some things consider the original header and others consider
the modified one would be terribly confusing. (The example shown above might
actually be a good one to include in the draft to show how this is supposed to
work.)
    
We also need to specify how addheader interacts with deleteheader. The
following should be essentially a no-op IMO:

   require "editheader";
   addheader "X-test" "whatever";
   deleteheader :index 1 "X-Test"

A couple of other issues occured to me while looking at this general area. First, we need to
specify what happens if the comparator or match-type arguments are present in
deleteheader but no value-pattern(s) is(are) specified. I suggest saying that
the comparator and match-type are ignored in this case.

We also need to specify what happens if :last is specified in deleteheader
without :index. I suggest saying that it will be ignored. The alternative is
for it to assume an index of 1, but clarity should trump having a more concise
way of specifying "delete the last field"

				Ned

Subject: Re: list lookups
From: Kjetil Torgrim Homme <[email-masked]>
To: Aaron Stone <[email-masked]>
Cc: [email-masked]
Date: Tue, 24 Apr 2007 23:13:02 +0200
Sender: [email-masked]

On Mon, 2007-04-23 at 18:46 +0000, Aaron Stone wrote:
> On Mon, Apr 23, 2007, Nigel Swinson <[email-masked]> said:
> > FYI we support it this kind of way:
> > 
> > if header :is "From" "${Whitelist}" {...}
> > 
> > Where "Whitelist" is an externally configured list of email addresses.  Were 
> > I to re-implement, then I'd put "Whitelist" in a namespace.  I consider it 
> > along the lines of accessing server configuration, for which we've already 
> > discussed adding a namespace.

yes, definitely put it in a namespace, IMO it's needed to conform to
"variables".

intuitively, I don't like the use of ":is" here, but it may be down to
wording.  if you said "externally configured search" instead of "list",
I don't think I would get the wrong association to Sieve lists.

there are some issues, what does "${whitelist} foo" mean?  stuff like
that needs to be specified.

> I would like to see a namespace or function space of some kind. A few
> people pointed out that it might be good to specify an LDAP URI inside the
> list description, so we're really talking about functions, imho.

I would like to stay away from functions if possible.  how complex do we
need to be to handle the use cases people are thinking of?  consider:

        if address "From" "${addressbook.mailaddress}" ...
        if header "From" "${addressbook.fullname}" ...
        if header "From" "${addressbook.lastname}" ...

could be translated into an LDAP search for attributes "mail", "cn" or
"sn" respectively.  OR and AND can be done at the Sieve level.

the hypothetical addressbook extension could also support

    set "addressbook.baseuri" "ldap://ldap.uio.no/cn=mail,dc=uio,dc=no";

but this quickly gets difficult to get interoperable since there is
little standardisation wrt which LDAP schema to use, and of course LDAP
isn't the only game in town.

HOWEVER: why use variables syntax -- this is similarily expressive:

        if addressbook :fullname "From" ...

and we could even do

        require "variables";
        if address :domain :matches "From" "*" {
                if addressbook :mailaddress :value :matches "*@${1}" ...

although it would probably better to make two tests, one for explicit
lookup value and one for lookup value fetched from headers.

yet another option is to use tagged arguments:

        if envelope :addressbook :matches "From" "*"
        if address :addressbook "From" :matches "[email-masked]" ...

it may be more natural to have only one mandatory argument and leave out
the "*", but I don't think we can change the syntax that way.  instead
the above test could FIRST do a normal match, and THEN a list lookup.

(there is no way to do a search on surname or full name without
variables, however, since ADDRESS-PART doesn't include the tagged
arguments to extract those elements.)

I think I favour this approach as it is more general -- the tag can be
reused for "address", "header", "envelope" and "string".  it's all a
hypothetical example, but I hope it illustrates some options.  sorry for
rambling on :-)

> > The idea of IO bound operations from a sieve script such as LDAP/RBL lookups 
> > makes me twitchy about performance, but these concerns are of course 
> > solvable using multiple threads/processes.  I think it's the first time 
> > we've suggested doing this kind of thing, so may be a knock to the 
> > architecture of a few implementations and therefore make it difficult for 
> > them to implement.

indeed.  defered delivery is not uncommon in e-mail systems, but the
Sieve base document does not offer that as an error mode.  is that
implicitly allowed?  I think not, and wonder if we could/should add
wording which says something about temporary errors, e.g. that there
MUST be an upper time limit to defered delivery caused by problems
during execution before the default fallback action (ie., keep) is done
instead.

> By using strict evaluation of variables, we set ourselves up for a
> performance problem because we may be asking the interpreter to first
> expand the full external list and then pass that into the test. Variables
> says that implementations should be able to pass unexpanded material into
> a test, but we still have keep the syntax of list expansions in mind so
> that we don't force everyone into nasty interpreter hacks.

agreed.  the idea is that variables without a namespace can always be
expanded.  variables with a namespace may have magic associated with
them so whether to expand them before calling the action or test should
be determined per namespace.

-- 
Kjetil T.

Date: Mon, 23 Apr 2007 18:46:15 -0000
To: "Nigel Swinson" <[email-masked]>, "Alexey Melnikov" <[email-masked]>
Subject: Re: Minutes from the Prague IETF meeting
From: "Aaron Stone" <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Mon, Apr 23, 2007, Nigel Swinson <[email-masked]> said:

> 
>> Alexey presented a new (not yet written) draft on "Externally stored 
>> lists".
>> The basic idea is to be able to store lists of users or email addresses in 
>> SQL/LDAP/ACAP/...
>> Such lists can be updated externally, without the need to update Sieve 
>> script.
>> The proposal is to extend redirect action and header/envelope tests to 
>> reference externally
>> stored lists.
> 
> FYI we support it this kind of way:
> 
> if header :is "From" "${Whitelist}" {...}
> 
> Where "Whitelist" is an externally configured list of email addresses.  Were 
> I to re-implement, then I'd put "Whitelist" in a namespace.  I consider it 
> along the lines of accessing server configuration, for which we've already 
> discussed adding a namespace.

I would like to see a namespace or function space of some kind. A few
people pointed out that it might be good to specify an LDAP URI inside the
list description, so we're really talking about functions, imho.

> The idea of IO bound operations from a sieve script such as LDAP/RBL lookups 
> makes me twitchy about performance, but these concerns are of course 
> solvable using multiple threads/processes.  I think it's the first time 
> we've suggested doing this kind of thing, so may be a knock to the 
> architecture of a few implementations and therefore make it difficult for 
> them to implement.

By using strict evaluation of variables, we set ourselves up for a
performance problem because we may be asking the interpreter to first
expand the full external list and then pass that into the test. Variables
says that implementations should be able to pass unexpanded material into
a test, but we still have keep the syntax of list expansions in mind so
that we don't force everyone into nasty interpreter hacks.

Aaron

X-Spam-Score-scoring_explanation: 
X-Spam-Score-spf_status: 
X-Spam-Score-Spamcatcher1: 7daf846cc339090d91c9fe4301c74899
X-Spam-Score-Summary:  2,0,0,da7bacca5254c4fa,b219b43be74849f1,[email-masked],,RULES_HIT:355:379:481:539:540:541:542:543:567:599:601:945:946:980:982:988:989:1155:1156:1261:1277:1311:1313:1314:1345:1437:1515:1516:1518:1534:1541:1587:1593:1594:1711:1730:1747:1766:1792:2073:2075:2078:2393:2553:2559:2562:2828:2895:3027:3352:3865:3866:3867:3868:3869:3870:3871:3872:4250:5007,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:,MSBL:none,DNSBL:none
X-Spam-Score-rbl_summary: none
X-Spam-Score-Phishing_status: no
From: "Nigel Swinson" <[email-masked]>
To: "Alexey Melnikov" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Minutes from the Prague IETF meeting
Date: Mon, 23 Apr 2007 18:13:09 +0100
Sender: [email-masked]

> Alexey presented a new (not yet written) draft on "Externally stored 
> lists".
> The basic idea is to be able to store lists of users or email addresses in 
> SQL/LDAP/ACAP/...
> Such lists can be updated externally, without the need to update Sieve 
> script.
> The proposal is to extend redirect action and header/envelope tests to 
> reference externally
> stored lists.

FYI we support it this kind of way:

if header :is "From" "${Whitelist}" {...}

Where "Whitelist" is an externally configured list of email addresses.  Were 
I to re-implement, then I'd put "Whitelist" in a namespace.  I consider it 
along the lines of accessing server configuration, for which we've already 
discussed adding a namespace.

The idea of IO bound operations from a sieve script such as LDAP/RBL lookups 
makes me twitchy about performance, but these concerns are of course 
solvable using multiple threads/processes.  I think it's the first time 
we've suggested doing this kind of thing, so may be a knock to the 
architecture of a few implementations and therefore make it difficult for 
them to implement.

Nigel 

Date: Thu, 19 Apr 2007 12:54:28 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Minutes from the Prague IETF meeting
Sender: [email-masked]

And at the very last moment Alexey submits minutes...
Enjoy!
===========
Thank you to Dave Cridland for being the Jabber scribe this time.

Alexey was the only co-chair present this time. He briefly talked about
status of different documents (see slides). About half WG docs are in 
RFC ed queue.
Collation document got published as RFC 4790, thanks to Arnt.
Base spec was submitted for IESG review.

Philip promised to update Sieve body one more time before sending it to 
IESG for publication.
Editheader - need to resolve handling of mail loops.
Alexey/Cyrus think that notify extension is basically ready, but needs 
reviews and feedback on changes.
Mailto notification is in WGLC, has some open issues, but not much feedback.
Refuse/Reject: Alexey has been too busy recently for an update.

The WG briefly talked about Editheader interaction with redirect/reject. 
Philip replied
in jabber that the new text doesn't affect reject, which still requires 
to return the original
headers, as per another paragraph in the document. And he basically 
agrees with other comments
and will do another revision of the draft before sending it to IESG.

Alexey talked about recent changes to Notify. No comments or objections 
were received,
so Alexey & Barry think that the document is done.

TODO: Dave Cridland & Randy Gellens to do a full review of notify.

Barry has presented the Mailto notification draft. He talked about open 
issues:

1. URI Parameter conflicts with other notify parameters: Do notify 
parameter Override the URI parameters,
   is it an Error or something else?

 Barry likes "Error".
 Alexey & Michael prefer Override. Alexey is thinking of external URIs 
being pulled in (perhaps from IMAP METADATA).
 Barry thinks that Ned might have useful opinion.

 Philip: I suppose a script can code itself defensively to test the URI 
before it uses it. Possible, but ugly.
 Later he added that there were no practical way to test the contents of 
a URI using just ":matches",
 one needed regexps to break them down.

 TODO: Ask Ned for opinion. Unless more people say there opinion, the 
Override option will prevail.

2. Undesirable URI params: what do we do if a URI specifies unsafe headers?

 Barry slightly prefer ignoring bad headers.
 Ted saying the mailto scheme RFC is pretty limited on these points, so 
Sieve advice and mailto RFC advice
 would probably be different.

3. Message origin: does the notification message come from the original 
"RCPT TO", or from the notification system?

 Alexey pointed out a suggestion to make it an option.
 Pete asked what the From is for and whether we were going to reply to 
these messages.

 Philip: if the notification system can autoprocess bounces, then it 
belongs in the envelope MAIL FROM
 Maybe always is better, to block the loop possibility.

 Dave Cridland: not clear why worry about "from" for notifications: are 
they to send msgs *to* a user, not necessarily
 be *from* anyone in particular.

 Philip/Pete: The important 'from' here is the envelope from. IMHO, the 
env-from should *never* be the original.

 No clear consensus on this issue in the room.

4. Body excerpts: "is Barry being silly" for wanting this?

 Alexey commented that the Sieve Notify used to have body extraction 
action, which was removed.
 But it will move to MIME-loops, so he thinks that this is still useful.
 Dave Crocker said that there was a long history of an excerpt being 
useful. But it's tricky to get it perfect.
 Randy was discussing whether the spec ought to say more than "this is a 
good idea", or should we stipulate
 down to the last octet.
 Barry likes the idea of vague specification, with a view that 
mime-loops and body extraction to a variable
 would provide a strictly specified method for doing this.
 Randy: I'm concerned that if we try and standardize it we end up with a 
sprintf format for how the script
 specifies how much of the body to use and how it appears. Since that 
prospect is so scary I urge us to
 instead say "engine may do this if it wants, it is helpful"
 Barry commented that this automatic vague excerpt could be the default.

 TODO: verify consensus for "vague description of body excerpts" in the 
document.

Tony has presented the updated MIME loops document:

Open issues:

1. Should enclose throw away modifications?

 Philip: I think enclose should honor changes. Anyone using enclose to 
create bounces should be shot.
 Aaron: why would we *not* want to honor changes?
 Ted says we can always get unmodified behavior by structuring the 
script carefully (As in not doing
 any modifications).

 Consensus to use the modified version.

2. Should we synthesize Date and From headers to reflect the current 
user and time?

 Philip: Is there text that can be stolen from the submission spec? It's 
kinda like a submission,
 the creation of a full message from a partial one...

 Alexey prefer synthesized: the original are still in the enclosed part. 
Sometimes it is useful
 to see when the message was created and when it was enclosed.

 Note from the room about synthesizing new message id.

 Kjetil and Tony were discussing syntax for extracting key/value pairs 
from variables). Everybody
 else except Philip don't have a clue about the discussion, because 
people haven't read the most recent
 message from Kjetil on the mailing list.

 Philip: Kjetil's idea is cute, but they're not much better than simply 
using a comma separated list,
 partly because there's no way to quote the value properly when adding it.
 You can put quotes around it, but that won't handle embedded quotes or 
backslashes
 Regexps would be as good (or better), but you can't do the extraction 
with just :matches.
 Time for that SQL access extension.

General comment from Alexey (as a co-chair): the list of open issues 
that Tony presented
differs from the one from the latest San Diego IETF, and the issues were 
not addressed.
Tony agrees to review the San Diego list.

Alexey: Assuming all issues are resolved, MIME loops will be last called 
in a couple of weeks.

Alexey briefly talked about Ned's individual Date-Index Draft.
The document has no major issues now, Alexey to shepherd to IESG.

Alexey has briefly talked about Ned's Environment & Notary (DSN) extension.
Barry/Alexey: the two extensions don't belong to the same document.
TODO: Ask Ned to split the draft into 2.

Next Alexey presented his draft on "Accessing IMAP per-mailbox 
annotations from Sieve".
Alexey described features of the extension:
1). Ability to test for mailbox existence
2). Ability to create mailbox on fileinto. Without the new :create tag 
fileinto creates mailboxes
    in some implementations, but doesn't in others.

    Some short discussion about Sendmail implementation: it still treats 
:create as an optional request,
    which is arguably a bug. Sendmail implements Sieve in SMTP server, 
so fileinto turns into
    +detail email address, thus the mailbox creation request might not 
be honored.

Alexey talked about an example: an annotation can be used to turn 
vacation autoreply on or off.

Open issues:

1). Do we want access to server annotations? - Alexey thinks "yes"

 Chris pointed out that testing server annotations and testing 
environment are basically the same thing.
 Barry agreed. Barry commented that there is no namespace-like thing in 
Environment that might support this.

 TODO: Talk to Ned about namespaces in Environment.

2). Do we want access to LIST-EXTENDED data? ACLs? - Alexey thinks 
"maybe" on LIST-EXTENDED data.

 Philip threatened to lose the breakfast he just ate.

Alexey presented a new (not yet written) draft on "Externally stored lists".
The basic idea is to be able to store lists of users or email addresses 
in SQL/LDAP/ACAP/...
Such lists can be updated externally, without the need to update Sieve 
script.
The proposal is to extend redirect action and header/envelope tests to 
reference externally
stored lists.

 Barry showed his Sieve script. It has many email addresses hardcoded, 
so he would like
 to have such an extension.

 Randy said that maybe an external list reference should be via URLs. 
(The proposal as described
 had list labels as opaque UTF-8 strings, e.g. "blacklist")
 Philip agrees.
 Philip: hmm, how about using imap: URLs to access annotations/metadata 
in IMAP?

 Aaron: so, :list "string" is probably a bad name because its confusable 
with :param ["string", "list"].
 And, this would be cool together with variable namespaces: ${external:name}

 Kjetil suggests that this probably doesn't scale too well, for example 
if the list contains
 ten thousands of addresses (like an enterprise address book). Kjetil 
suggests a comparator which
 takes a URI on one side.
 Alexey: Sure, testing membership in big lists can be optimized (e.g. 
use LDAP search).
 This is an implementation detail.

 Randy pointed out that we needed to handle updates, and the LDAP server 
going down.

 Pete says there are email clients which already have addressbook 
interaction in filtering.

 Aaron: I'm in favor of some kind of managesieve extension to maintain 
the external lists.

 Randy said that the extension should also cover management of mailing 
lists.
 (Alexey didn't like the idea)
 Randy: It's a generalization of Eudora's 'intersects address book' 
test, and
 'add to address book entry' action.
 Aaron: Automatically adding inbound/outbound addresses to the address book?

 Kjetil: with "include" in Sieve, it wouldn't be that much of a 
difference to
 upload a new list using ManageSieve. But I want it to use your *real* 
address
 book which is already stored in LDAP or whatever. Also, duplicating 
lists is not good.

 A huge discussion started on whether the identifier for a list has to 
be an opaque string or
 an URI:
  Alexey: I don't like URIs in this case. I don't want to force users to 
write LDAP URIs,
  they are just too complex. The whole point of the extension is to make 
this easier for users.
  In LDAP case users don't need to know the base DN for search, search 
criteria (e.g. which object
  classes), etc.

  Philip: who says the user will be writing the URI? won't the MUA be 
generating the URI to match
  the URI it uses for the address book?

  Pete suggested that the "mylist" can be a relative reference, so the 
script engine uses
  a base URI to resolve it to an absolute URI.

  Alexey thinks that this would not work with LDAP, for example.

  Barry likes the notion of a magical URI for "Your data".

  Kurt thinks it's all too complex.

  Pete says "Thou shalt support URIs", but "Thou may support arbitrary 
tokens".

  Philip: one catch with URIs is that they all have distinct quoting 
requirements for various fields.
  Given an arbitrary string, generating an LDAP URI that searches for 
entries that have a particular
  attribute with that value is non-trivial.
  Randy agrees.
  Philip: have to apply both the LDAP filter escaping *and* the URL 
escaping, in that order.

And finally Alexey talked about ManageSIEVE draft.
One update since last IETF, mostly bugfixes. NOTIFY capability text 
moved here.

Open Issues:

1. Reference implementation doesn't implement non synchronizing literals.

So the spec will be updated to match the real world.

2.  Suggestion from Arnt: Remove anonymous script verification mode.

 Any objections?

 Kjetil: Means the UA doesn't have to store the password for syntax 
checking.
 Alexey: But clients already cache passwords, at least while they are 
running.

 Aaron notes that different users can have different capabilities granted to
 them, so it is important to authenticate properly.

 TODO: Consensus to remove the text.

3. IANA registry for response codes

 Dave Cridland: Could reuse 
http://www.iana.org/assignments/acap-registrations for response codes.

Alexey talked about WG milestones. There was a proposal to update them, 
but the WG is already
1 month behind on the updated ones.

Other business: Alexey wants to talk about Sieve interop on Tuesday or 
Wednesday.
Alexey instructs WG members to drink Czech beer.
The meeting has ended.

Date: Thu, 19 Apr 2007 10:04:31 +0200
To: [email-masked]
Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> "${hex:40${hex:30}}" -> "${hex:400}"
>
> would demonstrate that only one pass is done, too.

Great!

> > >    "${unicode:1000000}" -> error
> > >    "${unicode:200000}"  -> error

Now that we dropped the 6-digit-restriction, we no longer need two examples
of range violations.

> > And I've missed Michael's favorite:
> > 
> >    "${hex:" -> "${hex:"
>
> I think the fourth example from the top covers this already, that is
> "${hex:40".

I agree.  Everything else is fine with me.  I'll adapt my code.

Michael

Date: Wed, 18 Apr 2007 17:49:44 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Informal Working Group Last Call on draft-freed-sieve-date-index-04.txt
Cc: Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>, Aaron Stone <[email-masked]>
Sender: [email-masked]

Ned Freed writes:
> We do and I actually use this sort of construct quite a bit, modulo 
> proper use of double quotes ;-) We also do "if it is after business 
> hourse don't send message to my pager" sorts of tests fairly 
> regularly.

If this extension can do that, I want to support it.

Will review when I can find a spare moment.

Arnt

Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Date: Wed, 18 Apr 2007 08:05:21 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Informal Working Group Last Call on draft-freed-sieve-date-index-04.txt
To: Aaron Stone <[email-masked]>
Sender: [email-masked]

> On Mon, 2007-04-16 at 20:22 +0100, Alexey Melnikov wrote:

> > http://www.ietf.org/internet-drafts/draft-freed-sieve-date-index-05.txt

> > At this point I would like to see a show of hands from people who are
> > interested in implementing this extension.

> Side question, does anybody else support this yet?

We do and I actually use this sort of construct quite a bit, modulo proper use
of double quotes ;-) We also do "if it is after business hourse don't send
message to my pager" sorts of tests fairly regularly.

>   if currentdate :matches �month� �*� { set �month� �${1}�; }
>   if currentdate :matches �year� �*� { set �year� �${1}�; }
>   fileinto �${month}-${year}�;

> (from http://www3.ietf.org/proceedings/05aug/slides/sieve-1.pdf)

				Ned

Cc: Alexey Melnikov <[email-masked]>
Date: Wed, 18 Apr 2007 07:37:49 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Informal Working Group Last Call on draft-freed-sieve-date-index-04.txt
To: [email-masked]
Sender: [email-masked]

> Alexey Melnikov wrote:

> > We would like to draw your attention to the following draft:

> > <http://www.ietf.org/internet-drafts/draft-freed-sieve-date-index-04.txt>

> > A three week informal last call of this document starts today on
> > March 6th 2007 and ends on March 27th 2007 at 6 pm EST.

> The informal last call has ended.

> I believe that Ned has addressed all outstanding issues in
> draft-freed-sieve-date-index-05.txt, so I will recommend its
> publication as an RFC. Thanks to all who reviewed the document.

The one issue several people have raised is that the document needs more
examples. I'm going to push one more revision out for sure to address this. The
other (very minor) change I want to make is to allow the date test to
optionally work on non RFC2822 format date-time information.

There is one other issue that occurred to me recently which I haven't addressed
and may be worthin considering. The currentdate test either operaters in a
specified time zone or in the "local" zone, which makes sense given those are
the only possibly relevant zones. However, the date test either operates in the
"local" zone or in a specified zone as  well. This provides no way to extract
the original zone from the header.

The first question is do we care? I'm having trouble coming up with a case
where we do, but of course there's always the generic "someone might want to
know the original zone".

The second question is if we do care, how to implement it? An "original zone"
part could be added. Or it could be done with an :originalzone argument.
Or it could be done as a distinguished argument value to :sone, e.g.

    :zone "original"

I prefer the third approach personally, but could be persuaded that either
of the other two is better.

> At this point I would like to see a show of hands from people who are
> interested in implementing this extension.

FWIW, I have implemented both date and currentdate and we're in the process of
transitioning a lot of use of the old setdate test from an early version of the
variables draft to currentdate. (Currentdate is much more straightforward and
this makes our generated Sieves a bit tidier.)

:index is much more difficult for our implementation. I have it mostly working
but there are some intricacies involving the editheader extension I still don't
have nailed down.
 
> Please reply to this message or directly to Ned and myself.

Yes please. And any examples you think make sense would also be welcome.

				Ned

Subject: Re: Informal Working Group Last Call on         draft-freed-sieve-date-index-04.txt
From: Aaron Stone <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked]
Date: Tue, 17 Apr 2007 01:09:32 -0700
Sender: [email-masked]

On Mon, 2007-04-16 at 20:22 +0100, Alexey Melnikov wrote:

> http://www.ietf.org/internet-drafts/draft-freed-sieve-date-index-05.txt

> At this point I would like to see a show of hands from people who are 
> interested in implementing this extension.

Side question, does anybody else support this yet?

  if currentdate :matches �month� �*� { set �month� �${1}�; }
  if currentdate :matches �year� �*� { set �year� �${1}�; }
  fileinto �${month}-${year}�;

(from http://www3.ietf.org/proceedings/05aug/slides/sieve-1.pdf)

Aaron

Subject: Re: Informal Working Group Last Call on         draft-freed-sieve-date-index-04.txt
From: Aaron Stone <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked]
Date: Tue, 17 Apr 2007 01:09:28 -0700
Sender: [email-masked]

On Mon, 2007-04-16 at 20:22 +0100, Alexey Melnikov wrote:

> > We would like to draw your attention to the following draft:
> >
> http://www.ietf.org/internet-drafts/draft-freed-sieve-date-index-05.txt

> At this point I would like to see a show of hands from people who are 
> interested in implementing this extension.

Yes, definitely!

Aaron

Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
From: Aaron Stone <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Date: Tue, 17 Apr 2007 00:54:13 -0700
Sender: [email-masked]

On Tue, 2007-04-17 at 01:32 +0200, Kjetil Torgrim Homme wrote:

> > >    "${Unicode:Cool}" -> "${Unicode:Cool}"

I continue to find this very unnatural, but I re-read variables, and it
does the same thing. There's certainly been lots of thought on the
issue, so I won't rehash.

Now my thinking is this: if we do end up going for a general
${function:argument} extension down the road, as was previously
suggested on the list, we'll have a precedent that the syntactic
validity of 'argument' depends upon the semantic value of 'function'.
That should be interesting ;-)

As for the other items, I thought Kjetil's suggestions about which were
superfluous and other edits were spot on.

Aaron

Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
From: Kjetil Torgrim Homme <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked]
Date: Tue, 17 Apr 2007 01:32:27 +0200
Sender: [email-masked]

On Mon, 2007-04-16 at 20:15 +0100, Alexey Melnikov wrote:
> Alexey Melnikov wrote:
> 
> > And add the following paragraph to the end of the section:
> >
> >  The following examples demonstrate valid and invalid encodings
> >  and how they are handled:

the list is quite long, I feel we can demonstrate more than one issue in
each example.

> >    "$${hex:40}" -> "$@"
> >    "${hex: 40 }" -> "@"
> >    "${HEX: 40}" -> "@"

the second is superfluous.

> >    "${hex:40" -> "${hex:40"
> >    "${hex:400}" -> "${hex:400}"

these are good.

> >    "${hex:40${hex:40}}" -> "${hex:40@}"

"${hex:40${hex:30}}" -> "${hex:400}"

would demonstrate that only one pass is done, too.

> >    "${unicode:40}" -> "@"
> >    "${ unicode:40}" -> "${ unicode:40}"
> >    "${UNICODE:40}" -> "@"

if we include the last example, the first in this set is superfluous
IMHO.  I'd like one of the examples to include backslash quoting, we
could extend that example, e.g., "\${UNICODE:40}" -> "@"

> >    "${UnICoDE:0000040}" -> "@"

fine.

> >    "${Unicode:40}" -> "@"

unnecessary, IMHO.

> >    "${Unicode:Cool}" -> "${Unicode:Cool}"
> >    "${unicode:1000000}" -> error
> >    "${unicode:200000}"  -> error
> >    "${Unicode:DF01}     -> error

ok.

> And I've missed Michael's favorite:
> 
>    "${hex:" -> "${hex:"

I think the fourth example from the top covers this already, that is
"${hex:40".
-- 
Kjetil T.

Date: Mon, 16 Apr 2007 20:22:24 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: Informal Working Group Last Call on draft-freed-sieve-date-index-04.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> We would like to draw your attention to the following draft:
>
> <http://www.ietf.org/internet-drafts/draft-freed-sieve-date-index-04.txt>
>
> A three week informal last call of this document starts today on March 
> 6th 2007 and ends on March 27th 2007 at 6 pm EST.

The informal last call has ended.
I believe that Ned has addressed all outstanding issues in 
draft-freed-sieve-date-index-05.txt, so I will recommend its publication 
as an RFC. Thanks to all who reviewed the document.

At this point I would like to see a show of hands from people who are 
interested in implementing this extension.
Please reply to this message or directly to Ned and myself.

Date: Mon, 16 Apr 2007 20:15:31 +0100
From: Alexey Melnikov <[email-masked]>
To: Alexey Melnikov <[email-masked]>
CC: [email-masked]
Subject: Re: Collected changes to 3028bis-12, section 2.4.2.4
Sender: [email-masked]

Alexey Melnikov wrote:

> And add the following paragraph to the end of the section:
>
>  The following examples demonstrate valid and invalid encodings
>  and how they are handled:
>    "$${hex:40}" -> "$@"
>    "${hex: 40 }" -> "@"
>    "${HEX: 40}" -> "@"
>    "${hex:40" -> "${hex:40"
>    "${hex:400}" -> "${hex:400}"
>    "${hex:40${hex:40}}" -> "${hex:40@}"
>    "${unicode:40}" -> "@"
>    "${ unicode:40}" -> "${ unicode:40}"
>    "${UNICODE:40}" -> "@"
>    "${UnICoDE:0000040}" -> "@"
>    "${Unicode:40}" -> "@"
>    "${Unicode:Cool}" -> "${Unicode:Cool}"
>    "${unicode:1000000}" -> error
>    "${unicode:200000}"  -> error
>    "${Unicode:DF01}     -> error

And I've missed Michael's favorite:

   "${hex:" -> "${hex:"

Date: Mon, 16 Apr 2007 20:12:33 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Collected changes to 3028bis-12, section 2.4.2.4
Sender: [email-masked]

Here is the collected list of changes to section 2.4.2.4:

3rd paragraph:

OLD:
 encoded-arb-octets   = "${hex:" hex-pair-seq "}"
 hex-pair-seq         = hex-pair *(WSP hex-pair)
                                   ^^^
 hex-pair             = 1*2HEXDIG

NEW:
 blank                = WSP / CRLF
 encoded-arb-octets   = "${hex:" hex-pair-seq "}"
 hex-pair-seq         = *blank hex-pair *(1*blank hex-pair) *blank
                        ^^^^^^            ^^^^^^^           ^^^^^^
 hex-pair             = 1*2HEXDIG

5th paragraph:

OLD:
 encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
 unicode-hex-seq      = unicode-hex *(WSP unicode-hex)
                                      ^^^
 unicode-hex          = 1*6HEXDIG
                        ^^^
NEW:
 encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
 unicode-hex-seq      = *blank unicode-hex *(1*blank unicode-hex) *blank
                        ^^^^^^               ^^^^^^^              ^^^^^^
 unicode-hex          = 1*HEXDIG
                        ^^

Insert after the following paragraph:
   It is an error for a script to use a hexadecimal value that isn't in
   either the range 0 to D7FF or the range E000 to 10FFFF.  (The range
   D800 to DFFF is excluded as those character numbers are only used as
   part of the UTF-16 encoding form and are not applicable to the UTF-8
   encoding that the syntax here represents.)

a new paragraph:

   Note: Implementations MUST NOT raise an error for an out of
   range Unicode value unless the sequence containing it is
   well-formed according to the grammar.

8th paragraph:
OLD:
  In the following script, message A is discarded, since the specified
                                   ^
  test string is equivalent to "$$$".
NEW:
  In the following script, message B is discarded, since the specified
                                   ^
  test string is equivalent to "$$$".

And add the following paragraph to the end of the section:

  The following examples demonstrate valid and invalid encodings
  and how they are handled:
    "$${hex:40}" -> "$@"
    "${hex: 40 }" -> "@"
    "${HEX: 40}" -> "@"
    "${hex:40" -> "${hex:40"
    "${hex:400}" -> "${hex:400}"
    "${hex:40${hex:40}}" -> "${hex:40@}"
    "${unicode:40}" -> "@"
    "${ unicode:40}" -> "${ unicode:40}"
    "${UNICODE:40}" -> "@"
    "${UnICoDE:0000040}" -> "@"
    "${Unicode:40}" -> "@"
    "${Unicode:Cool}" -> "${Unicode:Cool}"
    "${unicode:1000000}" -> error
    "${unicode:200000}"  -> error
    "${Unicode:DF01}     -> error

Date: Mon, 16 Apr 2007 14:45:20 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: A nearly completed draft: draft-crispin-collation-unicasemap-02.txt
Sender: [email-masked]

Hi,
This is somewhat related to the Sieve WG, so I would like to solicit 
reviews of draft-crispin-collation-unicasemap-02.txt. The document 
defines a case-insensitive comparator that operates on Unicode strings 
and is better than i;ascii-casemap for human readable text.

Please let me know if:
1). you review the document and find it ready/non-ready for publication;
2). you are interested in implementing this in your Sieve engine.

Thanks,
Alexey

Cc: Ned Freed <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked]
Date: Sun, 15 Apr 2007 19:44:54 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Poll: consensus to change the encoded-character extension
To: Kjetil Torgrim Homme <[email-masked]>
Sender: [email-masked]

> On Sun, 2007-04-15 at 13:17 -0700, Ned Freed wrote:
> > I'm fine with this. I do note, however, that error handling here is kinda
> > tricky - since you cannot be sure until you reach the } that the syntax
> > rules are met, you have to flag any errors you see in unicode values and
> > only report them at the end. Perhaps a note about this would be in order...

> yes, I guess it's easy to trip over this, causing a potential
> interoperability issue.  suggested text:

>         It is an error for a script to use a hexadecimal value that
>         isn't in either the range 0 to D7FF or the range E000 to
>         10FFFF.[...]

>         Note: Implementations MUST NOT raise an error for an out of
>         range Unicode value unless the sequence containing it is
>         well-formed according to the grammar.

Nice wording! I think thie definitely belongs in the specification.

				Ned

Subject: Re: Poll: consensus to change the encoded-character extension
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Date: Mon, 16 Apr 2007 03:11:11 +0200
Sender: [email-masked]

On Sun, 2007-04-15 at 13:17 -0700, Ned Freed wrote:
> I'm fine with this. I do note, however, that error handling here is kinda
> tricky - since you cannot be sure until you reach the } that the syntax
> rules are met, you have to flag any errors you see in unicode values and
> only report them at the end. Perhaps a note about this would be in order...

yes, I guess it's easy to trip over this, causing a potential
interoperability issue.  suggested text:

        It is an error for a script to use a hexadecimal value that
        isn't in either the range 0 to D7FF or the range E000 to
        10FFFF.[...]

        Note: Implementations MUST NOT raise an error for an out of
        range Unicode value unless the sequence containing it is
        well-formed according to the grammar.

-- 
Kjetil T.

Cc: [email-masked]
Date: Sun, 15 Apr 2007 13:17:48 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Poll: consensus to change the encoded-character extension
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> The update proposal is like this:

> OLD:
>   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
>   hex-pair-seq         = hex-pair *(WSP hex-pair)
>                                     ^^^
> NEW:
>   blank                = WSP / CRLF
>   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
>   hex-pair-seq         = *blank hex-pair *(1*blank hex-pair) *blank
>                          ^^^^^^            ^^^^^^^           ^^^^^^

> OLD:
>   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
>   unicode-hex-seq      = unicode-hex *(WSP unicode-hex)
>                                        ^^^
> NEW:
>   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
>   unicode-hex-seq      = *blank unicode-hex *(1*blank unicode-hex) *blank
>                          ^^^^^^               ^^^^^^^              ^^^^^^
>   unicode-hex          = 1*6HEXDIG

> Does this look right?

Modulo the change to make unicode-hex to 1*HEXDIG, yes.

> Note that people have suggested some other editorial changes in the same
> section, e.g. extra examples. I will post a message on them separately.
> For now I want to get closure on the proposed syntax.

I'm find with this. I do note, however, that error handling here is kinda
tricky - since you cannot be sure until you reach the } that the syntax
rules are met, you have to flag any errors you see in unicode values and
only report them at the end. Perhaps a note about this would be in order...

				Ned

Date: Fri, 13 Apr 2007 18:36:38 -0000
To: "Alexey Melnikov" <[email-masked]>, "Stephan Bosch" <[email-masked]>
Subject: Re: Conflict of draft-martin-managesieve-07.txt vs         draft-ietf-sieve-3028bis-12.txt
From: "Aaron Stone" <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

  S: "STARTTLS"
  S: OK

  S: OK

  S: "STARTTLS"
  S: OK

8.  IANA Considerations

-   To: [email-masked]

   Capability name: enotify

Aaron

Date: Fri, 13 Apr 2007 10:54:11 -0600
From: Philip Guenther <[email-masked]>
To: Michael Haardt <[email-masked]>
cc: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

On Fri, 13 Apr 2007, Michael Haardt wrote:
> On Thu, Apr 12, 2007 at 09:36:06PM +0100, Alexey Melnikov wrote:
...
>> BTW, are "${hex:"/"�{unicode:" prefixes case insensitive? I say yes.
>
> According to the grammar, they are.  Given that most everything in
> sieve is case insensitive, it makes sense, too.

Yes, definitely.

Philip Guenther

Date: Fri, 13 Apr 2007 08:47:14 +0100
From: Alexey Melnikov <[email-masked]>
To: Stephan Bosch <[email-masked]>
CC: [email-masked]
Subject: Re: Conflict of draft-martin-managesieve-07.txt vs draft-ietf-sieve-3028bis-12.txt
Sender: [email-masked]

Hi Stephan,

Regards,
Alexey

Date: Fri, 13 Apr 2007 09:20:10 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

On Thu, Apr 12, 2007 at 09:36:06PM +0100, Alexey Melnikov wrote:
> >IMHO, the leading and trailing white space is not part of the sequence.
> >If you think different, then put it back into hex-seq.
> >
> I think they are part of the hex-seq, but this is not something to argue 
> about.

Indeed, it's not the same grammar, but still the same language. :)

> I think we've only agreed to remove length restriction on "${unicode:".

Oops.  Well, with hex I don't mind pairs, in case anybody here
likes them more.

> BTW, are "${hex:"/"�{unicode:" prefixes case insensitive? I say yes.

According to the grammar, they are.  Given that most everything in
sieve is case insensitive, it makes sense, too.

Michael

Date: Thu, 12 Apr 2007 21:36:06 +0100
From: Alexey Melnikov <[email-masked]>
To: Michael Haardt <[email-masked]>
CC: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

Michael Haardt wrote:

>>The update proposal is like this:
>>
>>NEW:
>>  blank                = WSP / CRLF
>>  encoded-arb-octets   = "${hex:" hex-pair-seq "}"
>>  hex-pair-seq         = *blank hex-pair *(1*blank hex-pair) *blank
>>                         ^^^^^^            ^^^^^^^           ^^^^^^
>>    
>>
>The white space looks fine, but you missed defining hex-pair.
>  
>
No, it just stays as it is defined now. That is why I haven't listed it.

>>NEW:
>>  encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
>>  unicode-hex-seq      = *blank unicode-hex *(1*blank unicode-hex) *blank
>>                         ^^^^^^               ^^^^^^^              ^^^^^^
>>  unicode-hex          = 1*6HEXDIG
>>    
>>
>
>You did define unicode-hex here, but unchanged, not as 1*HEXDIG.
>
Ah, sorry about that. Yes, I meant "1*HEXDIG".

>How
>about that?
>
>  blank                 = WSP / CRLF
>  encoded-arb-octets    = "${hex:" *blank hex-seq *blank "}"
>  encoded-unicode-chars = "${unicode:" *blank hex-seq *blank "}"
>  hex-seq               = hex *(1*blank hex)
>  hex                   = 1*HEXDIG
>
>IMHO, the leading and trailing white space is not part of the sequence.
>If you think different, then put it back into hex-seq.
>
I think they are part of the hex-seq, but this is not something to argue 
about.

>If we omit the length restriction, we may as well merge the sequences into one
>production.
>  
>
I think we've only agreed to remove length restriction on "${unicode:".

BTW, are "${hex:"/"�{unicode:" prefixes case insensitive? I say yes.

Date: Thu, 12 Apr 2007 22:08:58 +0200
To: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> The update proposal is like this:
>
> NEW:
>   blank                = WSP / CRLF
>   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
>   hex-pair-seq         = *blank hex-pair *(1*blank hex-pair) *blank
>                          ^^^^^^            ^^^^^^^           ^^^^^^

The white space looks fine, but you missed defining hex-pair.

> NEW:
>   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
>   unicode-hex-seq      = *blank unicode-hex *(1*blank unicode-hex) *blank
>                          ^^^^^^               ^^^^^^^              ^^^^^^
>   unicode-hex          = 1*6HEXDIG

You did define unicode-hex here, but unchanged, not as 1*HEXDIG.  How
about that?

  blank                 = WSP / CRLF
  encoded-arb-octets    = "${hex:" *blank hex-seq *blank "}"
  encoded-unicode-chars = "${unicode:" *blank hex-seq *blank "}"
  hex-seq               = hex *(1*blank hex)
  hex                   = 1*HEXDIG

IMHO, the leading and trailing white space is not part of the sequence.
If you think different, then put it back into hex-seq.  If we omit
the length restriction, we may as well merge the sequences into one
production.

Michael

Date: Thu, 12 Apr 2007 20:39:52 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: Aaron Stone <[email-masked]>, [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>On Tue, 2007-04-10 at 19:12 +0000, Aaron Stone wrote:
>  
>
>>Something like this:
>>
>>   encoded-character    = "${" encoded-char-scheme ":" encoded-char-seq
>>"}"
>>   encoded-char-scheme  = hex / unicode
>>   encoded-char-seq     = *(LWSP WSP 1*HEXDIG) LWSP
>>    
>>
>if we allow ${hex:100} in the grammar, we need to say something in the
>text about the valid range.  I would prefer to stick to separate
>productions for encoded-arb-octets and encoded-unicode-char to keep the
>text simple and to minimise the change to the text.
>  
>
Speaking as a contributor: I agree.

>>Note that LWSP is optional by definition,
>>    
>>
>ouch, good catch!
>  
>
Indeed.

Date: Thu, 12 Apr 2007 20:30:28 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

Alexey Melnikov wrote:

> From my reading of the mailing list it sounds like there is consensus 
> to do the following change:
>
> OLD:
>   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
>   hex-pair-seq         = hex-pair *(WSP hex-pair)
>                                     ^^^
> NEW:
>   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
>   hex-pair-seq         = hex-pair *(LWSP hex-pair)
>                                     ^^^^
>
> OLD:
>   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
>   unicode-hex-seq      = unicode-hex *(WSP unicode-hex)
>                                        ^^^
> NEW:
>   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
>   unicode-hex-seq      = unicode-hex *(LWSP unicode-hex)
>                                        ^^^
>   unicode-hex          = 1*6HEXDIG
>
>
> And this needs to be updated if people want to allow for trailing LSWP 
> (before the closing "}")

The update proposal is like this:

OLD:
  encoded-arb-octets   = "${hex:" hex-pair-seq "}"
  hex-pair-seq         = hex-pair *(WSP hex-pair)
                                    ^^^
NEW:
  blank                = WSP / CRLF
  encoded-arb-octets   = "${hex:" hex-pair-seq "}"
  hex-pair-seq         = *blank hex-pair *(1*blank hex-pair) *blank
                         ^^^^^^            ^^^^^^^           ^^^^^^

OLD:
  encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
  unicode-hex-seq      = unicode-hex *(WSP unicode-hex)
                                       ^^^
NEW:
  encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
  unicode-hex-seq      = *blank unicode-hex *(1*blank unicode-hex) *blank
                         ^^^^^^               ^^^^^^^              ^^^^^^
  unicode-hex          = 1*6HEXDIG

Does this look right?

Note that people have suggested some other editorial changes in the same 
section, e.g. extra examples. I will post a message on them separately. 
For now I want to get closure on the proposed syntax.

Date: Thu, 12 Apr 2007 20:20:28 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

Alexey Melnikov wrote:

> Is there any consensus for the following change
>
> OLD:
>   unicode-hex          = 1*6HEXDIG
>                          ^^^
>
> NEW:
>   unicode-hex          = 1*HEXDIG
>                          ^^
> ?
>
> (And any sequence of more than 6 HEXDIG will cause error)

It looks to me that we've reached rough consensus in favor of this change.

From: "Mark E. Mallett" <[email-masked]>
Date: Wed, 11 Apr 2007 15:44:26 -0400
To: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

On Tue, Apr 10, 2007 at 09:51:41PM +0200, Kjetil Torgrim Homme wrote:
> On Tue, 2007-04-10 at 19:12 +0000, Aaron Stone wrote:
> > Note that LWSP is optional by definition,
> 
> ouch, good catch!

very.

> LWSP requires WSP after CRLF, too, so it's simply not what we want, we
> need to add another basic terminal, perhaps
> 
>    blank = WSP / CRLF
> 
> I suggest we stick to the poll question from Alexey, but with "1*blank"
> replacing LWSP in his suggested new text.

That looks right (name doesn't have to be "blank" but whatever).

> > I concur that comments should not be allowed.

ditto

mm

Date: Wed, 11 Apr 2007 19:10:29 -0000
To: "Alexey Melnikov" <[email-masked]>
Subject: Re: encoded-character and unicode range violations
From: "Aaron Stone" <[email-masked]>
Cc: "Michael Haardt" <[email-masked]>, <[email-masked]>
Sender: [email-masked]

On Wed, Apr 11, 2007, Alexey Melnikov <[email-masked]> said:

> Aaron Stone wrote:
> 
>>On Wed, Apr 11, 2007, Alexey Melnikov <[email-masked]> said:
>>  
>>
>>>Michael Haardt wrote:
>>>    
>>>
>>>>Hello,
>>>>
>>>>just sitting at the code, what shoulb be the result of:
>>>>
>>>> "${unicode:0000000}"
>>>>
>>>>Seven digits, yet in range.  To me, it should be a NUL character, not
>>>>an error.  Let's not specify the number of digits at all, neither in
>>>>the grammar nor in the text, and just specify the range.
>>>>      
>>>>
>>>Speaking as a contributor: this would be fine with me.
>>>    
>>>
>>Ok, and is this an error or ignored?
>>
>>    "${unicode:ffffff}"
>>  
>>
> For "unicode:" it is an error, as FFFFFF is bigger than 10FFFF

Ok, that resolves all of my outstanding questions.
I'm all set to see updated text whenever you're ready.

Aaron

Date: Wed, 11 Apr 2007 19:55:05 +0100
From: Alexey Melnikov <[email-masked]>
To: Aaron Stone <[email-masked]>
CC: Michael Haardt <[email-masked]>, [email-masked]
Subject: Re: encoded-character and unicode range violations
Sender: [email-masked]

Aaron Stone wrote:

>On Wed, Apr 11, 2007, Alexey Melnikov <[email-masked]> said:
>  
>
>>Michael Haardt wrote:
>>    
>>
>>>Hello,
>>>
>>>just sitting at the code, what shoulb be the result of:
>>>
>>> "${unicode:0000000}"
>>>
>>>Seven digits, yet in range.  To me, it should be a NUL character, not
>>>an error.  Let's not specify the number of digits at all, neither in
>>>the grammar nor in the text, and just specify the range.
>>>      
>>>
>>Speaking as a contributor: this would be fine with me.
>>    
>>
>Ok, and is this an error or ignored?
>
>    "${unicode:ffffff}"
>  
>
For "unicode:" it is an error, as FFFFFF is bigger than 10FFFF

Date: Wed, 11 Apr 2007 18:41:08 -0000
To: "Alexey Melnikov" <[email-masked]>, "Michael Haardt" <[email-masked]>
Subject: Re: encoded-character and unicode range violations
From: "Aaron Stone" <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Wed, Apr 11, 2007, Alexey Melnikov <[email-masked]> said:

> Michael Haardt wrote:
> 
>>Hello,
>>
>>just sitting at the code, what shoulb be the result of:
>>
>>  "${unicode:0000000}"
>>
>>Seven digits, yet in range.  To me, it should be a NUL character, not
>>an error.  Let's not specify the number of digits at all, neither in
>>the grammar nor in the text, and just specify the range.
>>  
>>
> Speaking as a contributor: this would be fine with me.

Ok, and is this an error or ignored?

    "${unicode:ffffff}"

Aaron

Date: Wed, 11 Apr 2007 17:54:52 +0100
From: Alexey Melnikov <[email-masked]>
To: Michael Haardt <[email-masked]>
CC: [email-masked]
Subject: Re: encoded-character and unicode range violations
Sender: [email-masked]

Michael Haardt wrote:

>Hello,
>
>just sitting at the code, what shoulb be the result of:
>
>  "${unicode:0000000}"
>
>Seven digits, yet in range.  To me, it should be a NUL character, not
>an error.  Let's not specify the number of digits at all, neither in
>the grammar nor in the text, and just specify the range.
>  
>
Speaking as a contributor: this would be fine with me.

Date: Wed, 11 Apr 2007 09:36:26 +0200
To: [email-masked]
Subject: encoded-character and unicode range violations
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

Hello,

just sitting at the code, what shoulb be the result of:

  "${unicode:0000000}"

Seven digits, yet in range.  To me, it should be a NUL character, not
an error.  Let's not specify the number of digits at all, neither in
the grammar nor in the text, and just specify the range.

Michael

Subject: Re: Poll: consensus to change the encoded-character extension
From: Kjetil Torgrim Homme <[email-masked]>
To: Aaron Stone <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, [email-masked]
Date: Tue, 10 Apr 2007 21:51:41 +0200
Sender: [email-masked]

On Tue, 2007-04-10 at 19:12 +0000, Aaron Stone wrote:
> Something like this:
> 
>    encoded-character    = "${" encoded-char-scheme ":" encoded-char-seq
> "}"
>    encoded-char-scheme  = hex / unicode
>    encoded-char-seq     = *(LWSP WSP 1*HEXDIG) LWSP

if we allow ${hex:100} in the grammar, we need to say something in the
text about the valid range.  I would prefer to stick to separate
productions for encoded-arb-octets and encoded-unicode-char to keep the
text simple and to minimise the change to the text.

> Note that LWSP is optional by definition,

ouch, good catch!

> so we have to include SP or WSP
> to force some kind of separator between 1*HEXDIG's. Note that this is not
> valid according to the syntax above,
> 
> ${unicode:
> 123
> ABC
> }
> 
> ..because 123 and ABC do not have WSP between them. Use WSP / CR / LF? Is
> there some variant of LWSP that mandates at least one character of
> something be present?

LWSP requires WSP after CRLF, too, so it's simply not what we want, we
need to add another basic terminal, perhaps

   blank = WSP / CRLF

I suggest we stick to the poll question from Alexey, but with "1*blank"
replacing LWSP in his suggested new text.

> I think there are three options for values that are out of range:
> 
>  1. Throw an error and reject the script.
>  2. Ignore the offending value.
>  3. Insert some placeholder like ' ' or '?'.

I don't think we need to revisit this question.

> I concur that comments should not be allowed.

-- 
Kjetil T.

Subject: Re: Poll: consensus to change the encoded-character extension
From: Kjetil Torgrim Homme <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Date: Tue, 10 Apr 2007 21:24:21 +0200
Sender: [email-masked]

On Tue, 2007-04-10 at 20:55 +0200, Michael Haardt wrote:
> Btw: If you want to encode "${hex:00}" literally, you need to write
> "${hex:24 7b 68 65 78 3a 30 30 7d}".

"${hex:24}{hex:00}" is shorter and IMHO clearer.

> Hmm. 2.4.2 says further:
> 
>   NUL (US-ASCII 0) is not allowed in strings.
> 
> How about:
> 
>   An unencoded NUL (US-ASCII 0) is not allowed in strings, see section
>   2.4.2.4 for encoded characters.

I think that clarification is useful.  "Literal NUL ..." works, too.
-- 
Kjetil T.

Date: Tue, 10 Apr 2007 19:12:24 -0000
To: "Kjetil Torgrim Homme" <[email-masked]>, "Alexey Melnikov" <[email-masked]>
Subject: Re: Poll: consensus to change the encoded-character extension
From: "Aaron Stone" <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue, Apr 10, 2007, Kjetil Torgrim Homme <[email-masked]> said:

> 
> On Tue, 2007-04-10 at 18:08 +0100, Alexey Melnikov wrote:
>>  From my reading of the mailing list it sounds like there is consensus 
>> to do the following change: [...]
> 
>> And this needs to be updated if people want to allow for trailing LSWP 
>> (before the closing "}")
> 
> if trailing LWSP is allowed, leading LWSP should be allowed, too.  I
> don't mind making such a change.  I don't mind making the other changes,
> either (including 1*HEXDIG).

Something like this:

   encoded-character    = "${" encoded-char-scheme ":" encoded-char-seq
"}"
   encoded-char-scheme  = hex / unicode
   encoded-char-seq     = *(LWSP WSP 1*HEXDIG) LWSP

Note that LWSP is optional by definition, so we have to include SP or WSP
to force some kind of separator between 1*HEXDIG's. Note that this is not
valid according to the syntax above,

${unicode:
123
ABC
}

..because 123 and ABC do not have WSP between them. Use WSP / CR / LF? Is
there some variant of LWSP that mandates at least one character of
something be present?

I think there are three options for values that are out of range:

 1. Throw an error and reject the script.
 2. Ignore the offending value.
 3. Insert some placeholder like ' ' or '?'.

I concur that comments should not be allowed.

Aaron

Date: Tue, 10 Apr 2007 20:55:16 +0200
To: [email-masked]
Subject: Re: Poll: consensus to change the encoded-character extension
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> if trailing LWSP is allowed, leading LWSP should be allowed, too.  I
> don't mind making such a change.  I don't mind making the other changes,
> either (including 1*HEXDIG).

Trailing and leading white space would be great indeed, because together
with LWSP you could encode a regular hexdump.

> > I don't think we have consensus to allow for comments.
>
> I think we have consensus for not allowing them.

Just looking at ${hex:...} and ${unicode:...}, I really don't need them,
but should that syntax (enriched by something to split arguments) be
used for functions, I disagree with Ned on not needing comments for that.
I did use comments to comment arguments given to functions in the past,
but I admit it's like once every few years.

But... I don't insist on comments.  Go ahead with LWSP and without comments.

Alexey: Could you please include an example that shows "${hex:" means
literally that and it is not an error? We can discuss that being right
or wrong, but at this time, everybody but me considers is being right
and that needs to be documented more clearly.

Btw: If you want to encode "${hex:00}" literally, you need to write
"${hex:24 7b 68 65 78 3a 30 30 7d}".  "\$\{\h\e\x\:\0\0\}" is a string
with one NUL byte, plus lacking the good reason to abuse \ asked for
by the SHOULD NOT in section 2.4.2.  I suppose everybody already knew
that, because the spec says, expansion happens after scanning a string.
If it surprises anybody, please speak up.

Hmm. 2.4.2 says further:

  NUL (US-ASCII 0) is not allowed in strings.

How about:

  An unencoded NUL (US-ASCII 0) is not allowed in strings, see section
  2.4.2.4 for encoded characters.

Michael

Subject: Re: Poll: consensus to change the encoded-character extension
From: Kjetil Torgrim Homme <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked]
Date: Tue, 10 Apr 2007 20:13:24 +0200
Sender: [email-masked]

On Tue, 2007-04-10 at 18:08 +0100, Alexey Melnikov wrote:
>  From my reading of the mailing list it sounds like there is consensus 
> to do the following change: [...]

> And this needs to be updated if people want to allow for trailing LSWP 
> (before the closing "}")

if trailing LWSP is allowed, leading LWSP should be allowed, too.  I
don't mind making such a change.  I don't mind making the other changes,
either (including 1*HEXDIG).

> I don't think we have consensus to allow for comments.

I think we have consensus for not allowing them.
-- 
Kjetil T.

Date: Tue, 10 Apr 2007 18:12:06 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

Ned Freed wrote:

>>>>"${unicode:200000}" -> error
>>>>"${unicode:2000000}" -> "${unicode:2000000}"
>>>>
>>>>I don't particularly like that, because most likely the second was
>>>>never meant that way.  Is there any way to change that at this point?
>>>>        
>>>>
>>>you want to change unicode-hex to 1*HEXDIG instead?  the wording should
>>>already handle it, so it's just the ABNF which needs a tweak.  that's
>>>fine with me.  I think it's Philip's call.
>>>      
>>>
>>Yes, that would be more logical.  I consider ${hex: and ${unicode: as
>>functions of constant arguments.  No matter which argument is passed
>>to them: Syntax errors (like a missing brace) should cause an error,
>>and semantic errors like range overflows should cause an error, too.
>>It's bizarre to see 0x200000 being an overflow, but 0x2000000 causing
>>everything to be taken literally.
>>    
>>
>I'm not wild about changing from 1*6 to 1* but I can live with it if need be.  
>
Is there any consensus for the following change

OLD:
   unicode-hex          = 1*6HEXDIG
                          ^^^

NEW:
   unicode-hex          = 1*HEXDIG
                          ^^
?

(And any sequence of more than 6 HEXDIG will cause error)

Date: Tue, 10 Apr 2007 18:08:45 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Poll: consensus to change the encoded-character extension
Sender: [email-masked]

Ned Freed wrote:

>>>>>yes.  whitespace is only allowed between hex-pairs.  btw, how do you
>>>>>feel about allowing CRLF as well as SPC and TAB between hex-pairs?
>>>>>          
>>>>>
>>>>Is CRLF allowed inside other ${} expressions (variables)?
>>>>        
>>>>
>>>variables doesn't allow any whitespace at all.
>>>      
>>>
>>Hmm, right, variables contain no arguments and we don't have functions
>>yet.  Thinking about string expressions, I certainly would like to
>>have CRLF as white space, but I also would like embedded comments in
>>that case.    
>>
>I would like to allow CRLF but comments IMO go way too far.
>  
>
 From my reading of the mailing list it sounds like there is consensus 
to do the following change:

OLD:
   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
   hex-pair-seq         = hex-pair *(WSP hex-pair)
                                     ^^^
NEW:
   encoded-arb-octets   = "${hex:" hex-pair-seq "}"
   hex-pair-seq         = hex-pair *(LWSP hex-pair)
                                     ^^^^

OLD:
   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
   unicode-hex-seq      = unicode-hex *(WSP unicode-hex)
                                        ^^^
NEW:
   encoded-unicode-char = "${unicode:" unicode-hex-seq "}"
   unicode-hex-seq      = unicode-hex *(LWSP unicode-hex)
                                        ^^^
   unicode-hex          = 1*6HEXDIG

And this needs to be updated if people want to allow for trailing LSWP 
(before the closing "}")

>The argument for allowing CRLF is that it is really needed to allow reasonable
>formatting of long runs of hex-encoded stuff. If, say, you want to write a few
>hundred bytes worth of material, with the current proposal you either have to
>put it all on one line, hope you can find a CRLF in there that you can leave
>unencoded and thus create a line break (unlikely), or use a series of set
>actions to build up the string piecemeal (ugly and requires variables support).
>
>The same cannot be said of comments - you are free to put one in front of the
>string or at the end and end up with something that's readable. Of course I
>suppose you could argue that there are cases where it is clearer to have the
>comment in the middle, but I have to say I find that to be a fairly contrived
>case.
>  
>
I don't think we have consensus to allow for comments.

>  Just looking at encoded-character, I see no need for CRLF
>

Subject: Re: Sieve notify options and escaping
From: Kjetil Torgrim Homme <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: Aaron Stone <[email-masked]>, [email-masked]
Date: Tue, 10 Apr 2007 18:35:53 +0200
Sender: [email-masked]

On Mon, 2007-04-09 at 20:20 +0100, Alexey Melnikov wrote:
> Aaron Stone wrote:
> >So a user can supply a variable that expands into valid options or url
> >syntax. I do think we have to prevent this.
>
> New ":urlencode" modifier to the set action?

what document should add such a modifier?

I'd like to note that it is possible to do this securely, although it's
not convenient.  e.g.

   if string :matches "${var}" "*&*" {
       set "var" "${1}%26${2}";

no, scratch that, we don't have recursion or other looping, so it won't
work for values containing two ampersands.  it would be tempting to add
a replace action:

  replace "var" "&" "%26";

we could allow MATCH-TYPE for more advanced replacements, e.g.

  replace :matches "var" "\\?" "${1}";

would replace a backslash followed by an arbitrary character by that
arbitrary character.

I don't have a real use case for this, so please feel free to disregard
the suggestion.

getting back to the issue at hand, I think it would be better to extend
the size of the notify namespace.  we could have

${notify.quote.subject}  (turn �hey "you"� into �"hey \"you\""�)
${notify.urlencode.subject}  (turn above string into �hey%20"you"�)
${notify.plain.subject}  (verbatim value)

we could also turn the order around, e.g.

${notify.subject.plain}
${notify.subject.urlencode}

which allows us to choose one of them as a "default" when the user
writes just ${notify.subject}.

-- 
Kjetil T.

Date: Mon, 09 Apr 2007 20:16:18 +0100
From: Alexey Melnikov <[email-masked]>
To: Aaron Stone <[email-masked]>
CC: [email-masked]
Subject: Re: Sieve notify options and escaping
Sender: [email-masked]

Aaron Stone wrote:

>On Mon, Mar 26, 2007, Alexey Melnikov <[email-masked]> said:  
>
>>Aaron, Dave has forwarded me your message:
>>    
>>
>Ok, moving back on list since we're more than one side-comment out.
>  
>
>>Dave Cridland wrote:
>>    
>>
>>>Dave,
>>>
>>>Barry and I discussed the need for some text to say what this means:
>>>
>>>   notify :options "body=${summary}" "mailto:foo@bar" 
>>>
>>>Because when you expand ${summary} is pretty important wrt escaping.
>>>Barry said that he'd add some text to handle this. I forgot to mention
>>>it in the jabber room for the official notes.
>>>      
>>>
>>Is this specific to Mailto notification method? Note that there is no 
>>text in notify base saying that options are to be converted to URI 
>>parameters.    
>>
>Same issues arise from this:  mailto "mailto:foo@bar?body=${summary}"
>
>What if ${summary} expands to "safebody&evil=evilbody"? We'll need some
>text to handle this situation I think.
>
If you suggest some, I will include it ;-).

>The issue applies to all
>mechanisms, I'm sure we could just as easily have additional xmpp url
>arguments or options.
>
>Do we have the option for "lazy evaluation" of variable expansion? If the
>expansion takes place inside the action, we have no trouble. If it takes
>place prior to calling the action, we need escaping.
>  
>
>>Also, do we actually want to register the "body" option?    
>>
>Sure, I don't see why not...
>  
>
Personally I dislike this, as this will most likely require a new IANA 
registry.
So far we were able to punt on this, as we didn't have any options.

Date: Mon, 09 Apr 2007 20:20:19 +0100
From: Alexey Melnikov <[email-masked]>
To: Aaron Stone <[email-masked]>
CC: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Subject: Re: Sieve notify options and escaping
Sender: [email-masked]

Aaron Stone wrote:

>On Tue, 2007-03-27 at 14:03 +0200, Kjetil Torgrim Homme wrote:
>  
>
>>On Mon, 2007-03-26 at 17:53 +0000, Aaron Stone wrote:
>>    
>>
>>>Do we have the option for "lazy evaluation" of variable expansion? If the
>>>expansion takes place inside the action, we have no trouble. If it takes
>>>place prior to calling the action, we need escaping.
>>>      
>>>
>>[variables] says:
>>
>>3.  Interpretation of strings
>>   [...]
>>   Tests or actions in future extensions may need to access the
>>   unexpanded version of the string argument and, e.g., do the expansion
>>   after setting variables in its namespace.  The design of the
>>   implementation should allow this.
>>
>>so the door is kept open, but the extension needs to be explicit about
>>it.  notice that a separate namespace should be used for such "dynamic"
>>or "internal" variables, in other words it should be ${notify.summary},
>>not just ${summary}.  for normal variables without a namespace, the
>>behaviour is:
>>
>>   The expanded string MUST use the variable values which are current
>>   when control reaches the statement the string is part of.
>>    
>>
>So a user can supply a variable that expands into valid options or url
>syntax. I do think we have to prevent this.
>  
>
New ":urlencode" modifier to the set action?

Subject: Re: Implementing encoded-character
From: Aaron Stone <[email-masked]>
To: [email-masked]
Date: Mon, 09 Apr 2007 17:27:41 -0700
Sender: [email-masked]

On Thu, 2007-04-05 at 21:10 +0000, Aaron Stone wrote:
> On Thu, Apr 5, 2007, Ned Freed <[email-masked]> said:
> 
> >> > > > "${unicode:200000}" -> error
> >> > > > "${unicode:2000000}" -> "${unicode:2000000}"
> > 
> >> Ugh, if it looks like encoded-char and walks like encoded-char...
> > 
> >> My test implementation left-shifts the current value of the encoded
> >> character, then adds the next hex digit. When it hits whitespace, it
> >> checks if the value is within appropriate bounds; if so, stores the
> >> character then loops, if not, stores '?' then loops. Would we really
> >> rather be very strict about this? I'm in favor of some flexibility.
> > 
> > You need to strictly implement the grammar in the specificaiton, whatever
> > that ends up being. Any flexibility will allow someone to write one of
> > these things that works in your implementation but silently fails and causes
> > wierd results elsewhere.
> > 
> > Past experience with RFC 2047 encoded-words has shown that allowing leeway in
> > this situations is a curse, not a blessing.
> 
> Indeed, point taken!

It's not strict yet (I'll cross that bridge when we agree on where it is ;-),
it just translates the hex values to utf-8. And now, counting from 0-9 in
Western Arabic, Eastern Arabic and Amharic (thanks unicode.org/charts!)...

Converting [${unicode:30 31 32 33 34 35 36 37 38 39}]
        to [0123456789] length 11

Converting [${unicode:06f0 06f1 06f2 06f3 06f4 06f5 06f6 06f7 06f8 06f9}]
        to [۰۱۲۳۴۵۶۷۸۹] length 21

Converting [${unicode:1369 136a 136b 136c 136d 136e 136f 1370 1371 1372}]
        to [፩፪፫፬፭፮፯፰፱፲] length 31

(Are there any number systems up in the four bytes per symbol ranges?)

If anybody would like to use my code, I'd be happy to make it available
without restriction. It's all of 100 lines, and most of the fun was
generating utf-8 by hand.

Aaron

From: "Mark E. Mallett" <[email-masked]>
Date: Fri, 6 Apr 2007 11:11:55 -0400
To: [email-masked]
Subject: Re: Support for encoded-character
Sender: [email-masked]

On Wed, Apr 04, 2007 at 03:46:54AM +0200, Kjetil Torgrim Homme wrote:
> 
> On Mon, 2007-04-02 at 23:36 -0600, Philip Guenther wrote:
> > On Mon, 2 Apr 2007, Ned Freed wrote:
> > > Sorry, missed that was one of the core rules. But that makes me wonder 
> > > if LWSP wouldn't be more appropriate...
> > 
> > That's an interesting idea, as it would let you 'wrap' a string in the 
> > script without including the CRLF in the string's value.  Hmm.
> 
> that could be useful, I guess, although I personally would have
> preferred adopting the common convention of ending lines with backslash
> if we need this capability.  sneaking it in through encoded-character is
> probably easier, though.

FWIW, I agree with the LWSP - I don't see any reason to have to escape
any line endings inside of an encoded-character block when it should be
fairly evident that nobody would want to preserve the CRLF there anyway.
Make it easy to compose the strings, and less easy to make errors.

mm  (IMHO)

Date: Thu, 5 Apr 2007 21:10:09 -0000
To: "Ned Freed" <[email-masked]>
Subject: Re: Implementing encoded-character
From: "Aaron Stone" <[email-masked]>
Cc: "Michael Haardt" <[email-masked]>, <[email-masked]>
Sender: [email-masked]

On Thu, Apr 5, 2007, Ned Freed <[email-masked]> said:

>> > > > "${unicode:200000}" -> error
>> > > > "${unicode:2000000}" -> "${unicode:2000000}"
> 
>> Ugh, if it looks like encoded-char and walks like encoded-char...
> 
>> My test implementation left-shifts the current value of the encoded
>> character, then adds the next hex digit. When it hits whitespace, it
>> checks if the value is within appropriate bounds; if so, stores the
>> character then loops, if not, stores '?' then loops. Would we really
>> rather be very strict about this? I'm in favor of some flexibility.
> 
> You need to strictly implement the grammar in the specificaiton, whatever
> that ends up being. Any flexibility will allow someone to write one of
> these things that works in your implementation but silently fails and causes
> wierd results elsewhere.
> 
> Past experience with RFC 2047 encoded-words has shown that allowing leeway in
> this situations is a curse, not a blessing.

Indeed, point taken!

Aaron

Cc: Michael Haardt <[email-masked]>, [email-masked]
Date: Thu, 05 Apr 2007 10:40:11 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Implementing encoded-character
To: Aaron Stone <[email-masked]>
Sender: [email-masked]

> On Wed, 2007-04-04 at 14:25 +0200, Michael Haardt wrote:
> > > > > yes.  whitespace is only allowed between hex-pairs.  btw, how do you
> > > > > feel about allowing CRLF as well as SPC and TAB between hex-pairs?
> >
> > > > Is CRLF allowed inside other ${} expressions (variables)?
> >
> > > variables doesn't allow any whitespace at all.
> >
> > Hmm, right, variables contain no arguments and we don't have functions
> > yet.  Thinking about string expressions, I certainly would like to
> > have CRLF as white space, but I also would like embedded comments in
> > that case.  Just looking at encoded-character, I see no need for CRLF
> > and even have an odd feeling with, but considering it as syntactic
> > prototype for string expressions, both CRLF and comments sound useful.

> I kind of like the idea of things that look like variables but are
> functions operating on the right side of the colon.

> We had a bit of discussion in Prague about list expansions that access
> external data sources. This would certainly be one way to handle it,
> though we'd have to be careful about strict vs. lazy evaluation. Anyhow,
> that should probably be the subject of a separate thread.

> > > > Before looking at it, I expected that if ${hex: is found, it would be
> > > > an error if it were not followed by arguments and a closing brace.
> > >
> > > well, you don't need to backtrack much:
> >
> > It's no problem really, just confusing.  If someone starts to write
> > ${hex:, most likely he meant to encode data.  Only CS people think stuff
> > like "it's not a word of the grammar, thus of course being an literal
> > as specified". ;-)

> I agree, it'd be confusing for that to happen.

> [snip]

> > > > "${unicode:200000}" -> error
> > > > "${unicode:2000000}" -> "${unicode:2000000}"

> Ugh, if it looks like encoded-char and walks like encoded-char...

> My test implementation left-shifts the current value of the encoded
> character, then adds the next hex digit. When it hits whitespace, it
> checks if the value is within appropriate bounds; if so, stores the
> character then loops, if not, stores '?' then loops. Would we really
> rather be very strict about this? I'm in favor of some flexibility.

You need to strictly implement the grammar in the specificaiton, whatever
that ends up being. Any flexibility will allow someone to write one of
these things that works in your implementation but silently fails and causes
wierd results elsewhere.

Past experience with RFC 2047 encoded-words has shown that allowing leeway in
this situations is a curse, not a blessing.

					ned

Cc: [email-masked]
Date: Thu, 05 Apr 2007 10:25:13 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Implementing encoded-character
To: Michael Haardt <[email-masked]>
Sender: [email-masked]

> > > > yes.  whitespace is only allowed between hex-pairs.  btw, how do you
> > > > feel about allowing CRLF as well as SPC and TAB between hex-pairs?

> > > Is CRLF allowed inside other ${} expressions (variables)?

> > variables doesn't allow any whitespace at all.

> Hmm, right, variables contain no arguments and we don't have functions
> yet.  Thinking about string expressions, I certainly would like to
> have CRLF as white space, but I also would like embedded comments in
> that case.

I would like to allow CRLF but comments IMO go way too far.

The argument for allowing CRLF is that it is really needed to allow reasonable
formatting of long runs of hex-encoded stuff. If, say, you want to write a few
hundred bytes worth of material, with the current proposal you either have to
put it all on one line, hope you can find a CRLF in there that you can leave
unencoded and thus create a line break (unlikely), or use a series of set
actions to build up the string piecemeal (ugly and requires variables support).

The same cannot be said of comments - you are free to put one in front of the
string or at the end and end up with something that's readable. Of course I
suppose you could argue that there are cases where it is clearer to have the
comment in the middle, but I have to say I find that to be a fairly contrived
case.

  Just looking at encoded-character, I see no need for CRLF
> and even have an odd feeling with, but considering it as syntactic
> prototype for string expressions, both CRLF and comments sound useful.

> > > Before looking at it, I expected that if ${hex: is found, it would be
> > > an error if it were not followed by arguments and a closing brace.
> >
> > well, you don't need to backtrack much:

> It's no problem really, just confusing.  If someone starts to write
> ${hex:, most likely he meant to encode data.  Only CS people think stuff
> like "it's not a word of the grammar, thus of course being an literal
> as specified". ;-)

> > > "${unicode:200000}" -> error
> > > "${unicode:2000000}" -> "${unicode:2000000}"
> > >
> > > I don't particularly like that, because most likely the second was
> > > never meant that way.  Is there any way to change that at this point?

> > you want to change unicode-hex to 1*HEXDIG instead?  the wording should
> > already handle it, so it's just the ABNF which needs a tweak.  that's
> > fine with me.  I think it's Philip's call.

> Yes, that would be more logical.  I consider ${hex: and ${unicode: as
> functions of constant arguments.  No matter which argument is passed
> to them: Syntax errors (like a missing brace) should cause an error,
> and semantic errors like range overflows should cause an error, too.
> It's bizarre to see 0x200000 being an overflow, but 0x2000000 causing
> everything to be taken literally.

I'm not wild about changing from 1*6 to 1* but I can live with it if need be.
That said, I completely disagree with your assessment of when it is appropriate
to generate an error. These "overlay" syntaxes always have a tension between
syntactic generality and potential collision with regular strings people might
want to use - the more general you make your syntax the more likely you are to
collide with some legitimate regular string. So, while I have no major issue
with allowing 1*HEXDIG, only to call overly long strings an error, I have a
major problem with allowing something like "${hex:" or even "${hex" or "${" to
match and then generate an error.

				Ned

Cc: Ned Freed <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked]
Date: Thu, 05 Apr 2007 10:20:09 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Support for encoded-character
To: Philip Guenther <[email-masked]>
Sender: [email-masked]

> On Mon, 2 Apr 2007, Ned Freed wrote:
> ...
> > Sorry, missed that was one of the core rules. But that makes me wonder
> > if LWSP wouldn't be more appropriate...

> That's an interesting idea, as it would let you 'wrap' a string in the
> script without including the CRLF in the string's value.  Hmm.

> > We also have a card-before-the-horse problem, in that we don't refer to
> > the use of ABNF for defining our grammars until section 8.1 but the ABNF
> > for encoded-character is in 2.4.2.4. (Well, to be fair, we do mention
> > ABNF in the conventions section, but only to compare it to how Usage:
> > statements are constructed.) The customary way to do this is to put the
> > reference to ABNF in the conventions section. I can understand not
> > bothering when ABNF was only used in one place, but that's no longer
> > true.

> True.  There's also no real explanation of the "Syntax:" lines that
> 'define' MATCH-TYPE, COMPARATOR, and ADDRESS-PART, although real ABNF for
> those is included in section 8.2.  <sigh> Perhaps this should be dealt
> with by
> (1) including a specific statement in section 1.1 that ABNF is used for
>      other syntax specifications in the body of the text, and
> (2) replacing those "Syntax:" lines with their actual ABNF versions.

> (This is turning into a long RFC editors note.  It's feeling like I should
> pull together the changes and submit a new rev...)

> > Finally, I wonder if we should be explicit about how encoded-character
> > constructs interact with text: blocks. My reading is that they should work.
> > Does everyone else agree?

>        If it's a string, then encoded-character affects its interpretation.
>        text: blocks are strings.
> MPP   The interpretation of text: blocks is affected by encoded-character

> In addition, 2.4.2.4 specifically notes that encoded-character processing
> happens after dot-unstuffing, which only applies to text: blocks.

I think that's good enough then - the inference is clear.

				Ned

Date: Thu, 05 Apr 2007 09:49:29 +0200
To: [email-masked]
Subject: Re: Implementing encoded-character
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> > Hmm, right, variables contain no arguments and we don't have functions
> > yet.  Thinking about string expressions, I certainly would like to
> > have CRLF as white space, but I also would like embedded comments in
> > that case.  Just looking at encoded-character, I see no need for CRLF
> > and even have an odd feeling with, but considering it as syntactic
> > prototype for string expressions, both CRLF and comments sound useful.
>
> I kind of like the idea of things that look like variables but are
> functions operating on the right side of the colon.
>
> We had a bit of discussion in Prague about list expansions that access
> external data sources. This would certainly be one way to handle it,
> though we'd have to be careful about strict vs. lazy evaluation. Anyhow,
> that should probably be the subject of a separate thread.

I suggest to look at Exim and the Exim filter, and their string
expressions, as a live and working example that's very similar.

> > > > "${unicode:200000}" -> error
> > > > "${unicode:2000000}" -> "${unicode:2000000}"
>
> Ugh, if it looks like encoded-char and walks like encoded-char...

That's the point.

> My test implementation left-shifts the current value of the encoded
> character, then adds the next hex digit. When it hits whitespace, it
> checks if the value is within appropriate bounds; if so, stores the
> character then loops, if not, stores '?' then loops. Would we really
> rather be very strict about this? I'm in favor of some flexibility.

Given you use C and unsigned integers, or signed ones on a common
architecture where overflows are ignored, you already have a problem.
You could stop aggregating after the 6th nibble, simply parsing more,
and then generate an overflow error if there are really more.

I suggest to make the specification a bit more flexible and have
implementations obey it strictly.

Michael

Subject: Re: Implementing encoded-character
From: Aaron Stone <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Date: Thu, 05 Apr 2007 00:20:01 -0700
Sender: [email-masked]

On Wed, 2007-04-04 at 14:25 +0200, Michael Haardt wrote:
> > > > yes.  whitespace is only allowed between hex-pairs.  btw, how do you
> > > > feel about allowing CRLF as well as SPC and TAB between hex-pairs?
> 
> > > Is CRLF allowed inside other ${} expressions (variables)?
> 
> > variables doesn't allow any whitespace at all.
> 
> Hmm, right, variables contain no arguments and we don't have functions
> yet.  Thinking about string expressions, I certainly would like to
> have CRLF as white space, but I also would like embedded comments in
> that case.  Just looking at encoded-character, I see no need for CRLF
> and even have an odd feeling with, but considering it as syntactic
> prototype for string expressions, both CRLF and comments sound useful.

I kind of like the idea of things that look like variables but are
functions operating on the right side of the colon.

We had a bit of discussion in Prague about list expansions that access
external data sources. This would certainly be one way to handle it,
though we'd have to be careful about strict vs. lazy evaluation. Anyhow,
that should probably be the subject of a separate thread.

> > > Before looking at it, I expected that if ${hex: is found, it would be
> > > an error if it were not followed by arguments and a closing brace.
> >
> > well, you don't need to backtrack much:
> 
> It's no problem really, just confusing.  If someone starts to write
> ${hex:, most likely he meant to encode data.  Only CS people think stuff
> like "it's not a word of the grammar, thus of course being an literal
> as specified". ;-)

I agree, it'd be confusing for that to happen.

[snip]

> > > "${unicode:200000}" -> error
> > > "${unicode:2000000}" -> "${unicode:2000000}"

Ugh, if it looks like encoded-char and walks like encoded-char...

My test implementation left-shifts the current value of the encoded
character, then adds the next hex digit. When it hits whitespace, it
checks if the value is within appropriate bounds; if so, stores the
character then loops, if not, stores '?' then loops. Would we really
rather be very strict about this? I'm in favor of some flexibility.

Aaron

Date: Thu, 5 Apr 2007 12:34:10 +0530
From: "Vihan Pandey" <[email-masked]>
To: [email-masked]
Subject: Re: sieve frequency based filtering
Cc: "Arnt Gulbrandsen" <[email-masked]>
Sender: [email-masked]

------=_Part_49084_4971106.1175756650068
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

> That'll be fun when your server comes up after some downtime and lots of
> queued mail is delivered in the first few minutes.

Good Point. However that aside if i have a situation of say 500,000 emails a
day from a single address that's something wherein that kind of a
functionality might prove useful. It's clearly suspicious behavior and ought
to be filtered.

> Is it possible to do this in the current version of sieve? If yes how?
>
> No.

Thanks for your prompt reply Arnt :-)

Regards,

- vihan

------=_Part_49084_4971106.1175756650068
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

<br>
<div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">That&#39;ll be fun when your server comes up after some downtime and lots of<br>queued mail is delivered in the first few minutes.
</blockquote><div><br>
Good Point. However that aside if i have a situation of say 500,000
emails a day from a single address that&#39;s something wherein that kind
of a functionality might prove useful. It&#39;s clearly suspicious behavior
and ought to be filtered.<br>
</div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">&gt; Is it possible to do this in the current version of sieve? If yes how?<br><br>
No.</blockquote><div><br>
Thanks for your prompt reply Arnt :-)<br>
<br>
Regards,<br>
<br>
- vihan<br>
<br>
<br>
</div></div><br>

------=_Part_49084_4971106.1175756650068--

Date: Thu, 5 Apr 2007 08:52:15 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: sieve frequency based filtering
Cc: Vihan Pandey <[email-masked]>
Sender: [email-masked]

Vihan Pandey writes:
> Hello,
>         i'm new on this list, and i wanted to ask if frequency(with respect
> to time slots) based filtering is possible in sieve.
>
> e.g i'm getting 10 emails from [email-masked] in 1 minute - thus i want 
> to block any emails that are arriving greater than or equal to 10 in 
> number in 1 minute.

That'll be fun when your server comes up after some downtime and lots of 
queued mail is delivered in the first few minutes.

> Is it possible to do this in the current version of sieve? If yes how?

No.

Arnt

Date: Thu, 5 Apr 2007 11:33:22 +0530
From: "Vihan Pandey" <[email-masked]>
To: [email-masked]
Subject: sieve frequency based filtering
Sender: [email-masked]

------=_Part_48350_25164756.1175753002386
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hello,
        i'm new on this list, and i wanted to ask if frequency(with respect
to time slots) based filtering is possible in sieve.

e.g i'm getting 10 emails from [email-masked] in 1 minute - thus i want to
block any emails that are arriving greater than or equal to 10 in number in
1 minute.

Is it possible to do this in the current version of sieve? If yes how?

i've checked the draft on : http://www.ietf.org/rfc/rfc3028.txt?number=3028

and the new draft on :
http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-12.txt

But i couldn't figure out how i would do such a filtering.

Regards,

- vihan

------=_Part_48350_25164756.1175753002386
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

<span><br>
Hello,<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; i&#39;m new on this list, and i
wanted to ask if frequency(with respect to time slots) based filtering
is possible in sieve.<br>
<br>
e.g i&#39;m getting 10 emails from <a href="mailto:[email-masked]" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">[email-masked]</a> in 1 minute - thus i want to
block any emails that are arriving greater than or equal to 10 in
number in 1 minute.<br>
<br>
Is it possible to do this in the current version of sieve? If yes how?<br>
<br>
i&#39;ve checked the draft on : <a href="http://www.ietf.org/rfc/rfc3028.txt?number=3028">http://www.ietf.org/rfc/rfc3028.txt?number=3028</a><br>
<br>
and the new draft on : <a href="http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-12.txt">http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-12.txt</a><br>
<br>
But i couldn&#39;t figure out how i would do such a filtering.<br>
<br>
Regards,<br>
<br>
- vihan<br>
<br>
</span>

------=_Part_48350_25164756.1175753002386--

Date: Wed, 04 Apr 2007 14:25:45 +0200
To: [email-masked]
Subject: Re: Implementing encoded-character
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> > > yes.  whitespace is only allowed between hex-pairs.  btw, how do you
> > > feel about allowing CRLF as well as SPC and TAB between hex-pairs?

> > Is CRLF allowed inside other ${} expressions (variables)?

> variables doesn't allow any whitespace at all.

Hmm, right, variables contain no arguments and we don't have functions
yet.  Thinking about string expressions, I certainly would like to
have CRLF as white space, but I also would like embedded comments in
that case.  Just looking at encoded-character, I see no need for CRLF
and even have an odd feeling with, but considering it as syntactic
prototype for string expressions, both CRLF and comments sound useful.

> > Before looking at it, I expected that if ${hex: is found, it would be
> > an error if it were not followed by arguments and a closing brace.
>
> well, you don't need to backtrack much:

It's no problem really, just confusing.  If someone starts to write
${hex:, most likely he meant to encode data.  Only CS people think stuff
like "it's not a word of the grammar, thus of course being an literal
as specified". ;-)

> > "${unicode:200000}" -> error
> > "${unicode:2000000}" -> "${unicode:2000000}"
> > 
> > I don't particularly like that, because most likely the second was
> > never meant that way.  Is there any way to change that at this point?

> you want to change unicode-hex to 1*HEXDIG instead?  the wording should
> already handle it, so it's just the ABNF which needs a tweak.  that's
> fine with me.  I think it's Philip's call.

Yes, that would be more logical.  I consider ${hex: and ${unicode: as
functions of constant arguments.  No matter which argument is passed
to them: Syntax errors (like a missing brace) should cause an error,
and semantic errors like range overflows should cause an error, too.
It's bizarre to see 0x200000 being an overflow, but 0x2000000 causing
everything to be taken literally.

Michael

Subject: Re: Implementing encoded-character
From: Kjetil Torgrim Homme <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Date: Wed, 04 Apr 2007 13:38:18 +0200
Sender: [email-masked]

On Wed, 2007-04-04 at 11:23 +0200, Michael Haardt wrote:
> > yes.  whitespace is only allowed between hex-pairs.  btw, how do you
> > feel about allowing CRLF as well as SPC and TAB between hex-pairs?
> 
> Is CRLF allowed inside other ${} expressions (variables)?

variables doesn't allow any whitespace at all.

> > I don't understand this statement.
> 
> The grammar matches words inside the character sequence that makes up a
> string.  No matter how much of a word is matched, if it is not complete,
> it will be taken as the literal character sequence.  That means you
> need infinite look-ahead.
> 
> Before looking at it, I expected that if ${hex: is found, it would be
> an error if it were not followed by arguments and a closing brace.

well, you don't need to backtrack much:

  ${unicode:cafe ab ab ab ab ab ab
     ab ab add ${hex:40 41}}

you just go along, and as soon as you find a syntax "error", you bail
and copy what you've buffered so far verbatim (in this case,
"${unicode: ... add "), then restart the state machine.  worst case, the
buffering is the size of the script plus storage for the decoded Unicode
characters while parsing the script.

> So we have:
> 
> "${unicode:200000}" -> error
> "${unicode:2000000}" -> "${unicode:2000000}"
> 
> I don't particularly like that, because most likely the second was
> never meant that way.  Is there any way to change that at this point?

you want to change unicode-hex to 1*HEXDIG instead?  the wording should
already handle it, so it's just the ABNF which needs a tweak.  that's
fine with me.  I think it's Philip's call.

-- 
Kjetil T.

Date: Wed, 04 Apr 2007 11:23:22 +0200
To: [email-masked]
Subject: Re: Implementing encoded-character
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> > "${hex:40" -> "${hex:40"
> > "${hex: 40 }" -> "${hex: 40 }"
>
> yes.  whitespace is only allowed between hex-pairs.  btw, how do you
> feel about allowing CRLF as well as SPC and TAB between hex-pairs?

Is CRLF allowed inside other ${} expressions (variables)?

> > "${unicode:40}" -> "${unicode:40}"
>
> no, this is "@".

Good thing I asked.  I just reread RFC 2234 and found out that I
have to read 1*6HEXDIG as 1*6(HEXDIG), not as 1*(6HEXDIG).

> > There is no word of the encoded-character grammar inside the string,
> > taking everything literal.

> I don't understand this statement.

The grammar matches words inside the character sequence that makes up a
string.  No matter how much of a word is matched, if it is not complete,
it will be taken as the literal character sequence.  That means you
need infinite look-ahead.

Before looking at it, I expected that if ${hex: is found, it would be
an error if it were not followed by arguments and a closing brace.

> > "${hex:40${hex:40}}" -> "${hex:40$}
>
> no, "${hex:40@}"

Oops, I meant to write @.  But you agree on my interpretation how things
are processed.

> > "${unicode:020000}" -> error
> > 
> > Unicode range violation.
>
> no, U+20000 is inside the Unicode range.  ${unicode:0020000} fails due
> to not matching unicode-hex (too many digits), ${unicode:200000} fails
> due to being outside the Unicode range.

My mistake, again.  So we have:

"${unicode:200000}" -> error
"${unicode:2000000}" -> "${unicode:2000000}"

I don't particularly like that, because most likely the second was
never meant that way.  Is there any way to change that at this point?

Michael

Subject: Re: Implementing encoded-character
From: Kjetil Torgrim Homme <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Date: Wed, 04 Apr 2007 10:51:52 +0200
Sender: [email-masked]

On Wed, 2007-04-04 at 10:11 +0200, Michael Haardt wrote:
> Exim will implement encoded-character RSN,

excellent!

> but I wonder if I understood things correctly:
> 
> "$${hex:40}" -> "$@"

yes.

> A dollar does not have to be quoted, does it?

right, only for very unlikely sequences.

> "${hex:40" -> "${hex:40"
> "${hex: 40 }" -> "${hex: 40 }"

yes.  whitespace is only allowed between hex-pairs.  btw, how do you
feel about allowing CRLF as well as SPC and TAB between hex-pairs?

> "${hex:400}" -> "${hex:400}"

yes, hex-pair can't be three digits.

> "${unicode:40}" -> "${unicode:40}"

no, this is "@".

> "${unicode:1000000}" -> "${unicode:1000000}"

yes.

> There is no word of the encoded-character grammar inside the string,
> taking everything literal.

I don't understand this statement.

> "${hex:40${hex:40}}" -> "${hex:40$}

no, "${hex:40@}"

> It looks nested, but actually it's just some junk around a word of the
> grammar.
> 
> "${unicode:020000}" -> error
> 
> Unicode range violation.

no, U+20000 is inside the Unicode range.  ${unicode:0020000} fails due
to not matching unicode-hex (too many digits), ${unicode:200000} fails
due to being outside the Unicode range.

-- 
Kjetil T.

Date: Wed, 04 Apr 2007 10:11:44 +0200
To: [email-masked]
Subject: Implementing encoded-character
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

Hello,

Exim will implement encoded-character RSN, but I wonder if I understood
things correctly:

"$${hex:40}" -> "$@"

A dollar does not have to be quoted, does it?

"${hex:40" -> "${hex:40"
"${hex: 40 }" -> "${hex: 40 }"
"${hex:400}" -> "${hex:400}"
"${unicode:40}" -> "${unicode:40}"
"${unicode:1000000}" -> "${unicode:1000000}"

There is no word of the encoded-character grammar inside the string,
taking everything literal.

"${hex:40${hex:40}}" -> "${hex:40$}

It looks nested, but actually it's just some junk around a word of the
grammar.

"${unicode:020000}" -> error

Unicode range violation.

Michael

Subject: Re: comments on draft-freed-sieve-notary
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Date: Wed, 04 Apr 2007 09:45:51 +0200
Sender: [email-masked]

On Tue, 2007-04-03 at 22:07 -0700, Philip Guenther wrote:
> On Wed, 4 Apr 2007, Kjetil Torgrim Homme wrote:
> > On Mon, 2007-04-02 at 14:58 -0600, Philip Guenther wrote:
> ...
> >> (Hurm.  RFC 3885's seems at least mildly broken, as it apparently implies
> >> that an "@" is not the same as its xtext encoded version.)
> >
> > why do you think that is (mildly) broken?  okay, it says "the syntax of
> > ENVID from RFC 3461 is extended [...]" when it's really *restricting*
> > it.
> 
> It implies that
>   	MAIL FROM:<[email-masked]> ENVID=[email-masked]
> is not the same as
>   	MAIL FROM:<[email-masked]> ENVID=foo+40sendmail.com

I don't think that's true for the specific example -- 3885 says it only
modifies the ENVID parameter syntax when a MTRK parameter is present.

if I understand correctly, a message can't be "upgraded" to use MTRK,
only the original sender can set a MTRK tracking identity.  when a
message passes from MTRK+DSN to DSN only, MTRK is removed and ENVID is
kept intact, and DSN has loose interpretation of ENVID.

while the message is in the MTRK domain, the ENVID parameter must
contain at least one verbatim "@".  fqhn can contain neither verbatim
nor encoded "@", so there is no ambiguity.

> changing xtext from purely an encoding to at least partially an escaping 
> mechanism.  You have to save the raw envid and not the decoded form. 
> Does message tracking still work if a relaying MTA changes which 
> characters in the envid are xtext encoded?  Certainly not if there's more 
> than one '@' in the fully decoded form...

I agree the requirement for a verbatim "@" is not very pretty, and
probably not intended.

Tony, what do you say?

> >> This extension provides a means to test the notary values on incoming
> >> messages, but not set them on messages sent via 'redirect'.
> >
> > I guess you mean the NOTIFY parameter here, I don't think we want to
> > allow users to play with any of the others.  that could probably be
> > useful (in particular NOTIFY=NEVER).  I don't think it needs to be in a
> > separate capability.
> 
> RET, and maybe ENVID, could also be useful.  The redirected message may 
> have an ORCPT pointing at the redirecting account automatically added if 
> it didn't have one before (per RFC 3461, 5.2.1(d)) *and* the envelope 
> sender is not changed by the implementation's 'redirect' command.

I'm skeptical to allowing access to ENVID, since Sieve has no facility
for generating guaranteed unique tokens.  at most we should allow the
user to specify a cookie to be embedded in the system's generated ENVID.

good point about ORCPT, but you're not advocating that the user should
be allowed to influence it, are you?  in my opinion, the interaction
between "redirect" and ORCPT is not relevant for this document since it
is relevant for all mail systems which implement DSN.  in other words,
it should be in the base spec -- something for 3028ter?

> > (also a tiny typo, "case-insensitivie", and I think it should say "SMTP
> > RCPT TO command", not "SMTP RCPT command")
> 
> Disagree.  It should say "ESMTP RCPT command" to match RFC 3461.

okay, in that case "SMTP MAIL FROM" should be changed to "ESMTP MAIL",
too.

-- 
Kjetil T.

Date: Tue, 3 Apr 2007 22:07:03 -0700
From: Philip Guenther <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: comments on draft-freed-sieve-notary
Sender: [email-masked]

On Wed, 4 Apr 2007, Kjetil Torgrim Homme wrote:
> On Mon, 2007-04-02 at 14:58 -0600, Philip Guenther wrote:
...
>> (Hurm.  RFC 3885's seems at least mildly broken, as it apparently implies
>> that an "@" is not the same as its xtext encoded version.)
>
> why do you think that is (mildly) broken?  okay, it says "the syntax of
> ENVID from RFC 3461 is extended [...]" when it's really *restricting*
> it.

It implies that
  	MAIL FROM:<[email-masked]> ENVID=[email-masked]
is not the same as
  	MAIL FROM:<[email-masked]> ENVID=foo+40sendmail.com

changing xtext from purely an encoding to at least partially an escaping 
mechanism.  You have to save the raw envid and not the decoded form. 
Does message tracking still work if a relaying MTA changes which 
characters in the envid are xtext encoded?  Certainly not if there's more 
than one '@' in the fully decoded form...

>> This extension provides a means to test the notary values on incoming
>> messages, but not set them on messages sent via 'redirect'.
>
> I guess you mean the NOTIFY parameter here, I don't think we want to
> allow users to play with any of the others.  that could probably be
> useful (in particular NOTIFY=NEVER).  I don't think it needs to be in a
> separate capability.

RET, and maybe ENVID, could also be useful.  The redirected message may 
have an ORCPT pointing at the redirecting account automatically added if 
it didn't have one before (per RFC 3461, 5.2.1(d)) *and* the envelope 
sender is not changed by the implementation's 'redirect' command.

> another comment on the draft:
>
>   orcpt  Match the original recipient, or ORCPT, value in decoded form
>      associated with the TO address used in the SMTP RCPT command that
>      resulted in this message getting delivered to this user.  The
>      syntax and semantics of the ORCPT parameter are defined in RFC
>      3461 [RFC3461] section 4.2.
>
> is the addr-type ("rfc822;") part of the value supposed to be stripped
> off?  perhaps we should extend ADDRESS-PART to make this explicit?  or
> we could say that :localpart and :domain strips off addr-type, but when
> neither is specified, the complete value is used.

Ooog, this get ugly when the type _isn't_ rfc822.  Hmmm...

> (also a tiny typo, "case-insensitivie", and I think it should say "SMTP
> RCPT TO command", not "SMTP RCPT command")

Disagree.  It should say "ESMTP RCPT command" to match RFC 3461.

Philip Guenther

Subject: Re: Support for encoded-character
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: Ned Freed <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked]
Date: Wed, 04 Apr 2007 03:46:54 +0200
Sender: [email-masked]

On Mon, 2007-04-02 at 23:36 -0600, Philip Guenther wrote:
> On Mon, 2 Apr 2007, Ned Freed wrote:
> > Sorry, missed that was one of the core rules. But that makes me wonder 
> > if LWSP wouldn't be more appropriate...
> 
> That's an interesting idea, as it would let you 'wrap' a string in the 
> script without including the CRLF in the string's value.  Hmm.

that could be useful, I guess, although I personally would have
preferred adopting the common convention of ending lines with backslash
if we need this capability.  sneaking it in through encoded-character is
probably easier, though.
-- 
Kjetil T.

Subject: Re: comments on draft-freed-sieve-notary
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: [email-masked], [email-masked]
Date: Wed, 04 Apr 2007 03:29:42 +0200
Sender: [email-masked]

On Mon, 2007-04-02 at 14:58 -0600, Philip Guenther wrote:
> I found the extension name to be relatively obscure, having completely 
> forgotten that the DSN work came from the notary WG.  "smtp-dsn" would 
> have been more obvious to me,

+1

> It should be stated that any ADDRESS-PART argument is ignored when 
> matching against the 'notify' or 'ret' envelope parts.  I suggest that it 
> _not_ be ignored for 'envid' due to RFC 3885's tightening of the envid 
> syntax when message tracking is used.
> 
> (Hurm.  RFC 3885's seems at least mildly broken, as it apparently implies 
> that an "@" is not the same as its xtext encoded version.)

why do you think that is (mildly) broken?  okay, it says "the syntax of
ENVID from RFC 3461 is extended [...]" when it's really *restricting*
it.

> Speaking of RFC 3885, would it be reasonable to have this "notary" 
> extension also define a "mtrk" envelope-part given how MTRK is tied to the 
> notary parameters?  Or should that be a different capability?

it should be a different capability, MTRK is non-trivial to deploy in
clustered mail systems, DSN is much simpler.  could be in the same
document, though.

>      mtrk  Match the message tracking, or MTRK, value given in the
>  	SMTP MAIL FROM command.  The syntax and semantics of the MTRK
>  	parameter are defined in RFC 3885 [RFC3885] section 3.1.
> 
> (It may make sense to do MTRK in a separate extension simply because of 
> its lower maturity level and deployment compared to DSN.)

yes.

> This extension provides a means to test the notary values on incoming 
> messages, but not set them on messages sent via 'redirect'.

I guess you mean the NOTIFY parameter here, I don't think we want to
allow users to play with any of the others.  that could probably be
useful (in particular NOTIFY=NEVER).  I don't think it needs to be in a
separate capability.

another comment on the draft:

   orcpt  Match the original recipient, or ORCPT, value in decoded form
      associated with the TO address used in the SMTP RCPT command that
      resulted in this message getting delivered to this user.  The
      syntax and semantics of the ORCPT parameter are defined in RFC
      3461 [RFC3461] section 4.2.

is the addr-type ("rfc822;") part of the value supposed to be stripped
off?  perhaps we should extend ADDRESS-PART to make this explicit?  or
we could say that :localpart and :domain strips off addr-type, but when
neither is specified, the complete value is used.

(also a tiny typo, "case-insensitivie", and I think it should say "SMTP
RCPT TO command", not "SMTP RCPT command")

-- 
Kjetil T.

X-Spam-Score-scoring_explanation: 
X-Spam-Score-spf_status: 
X-Spam-Score-Spamcatcher1: 6c3a050b8d523931eff24712910dd145
X-Spam-Score-Summary:  2,0,0,02ef14c6bc4b15f8,ef3b67f8c0e6e6e1,[email-masked],,RULES_HIT:355:379:539:540:541:542:543:567:599:601:988:989:1155:1156:1261:1277:1311:1313:1314:1345:1437:1513:1515:1516:1518:1521:1534:1539:1587:1593:1594:1711:1714:1730:1747:1766:1792:1801:2073:2075:2078:2393:2559:2562:2828:3027:3351:3742:3865:3867:3869:3870:3871:3873:3874:4605:5007,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:,MSBL:none,DNSBL:none
X-Spam-Score-rbl_summary: none
X-Spam-Score-Phishing_status: no
From: "Nigel Swinson" <[email-masked]>
To: "Alexey Melnikov" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: AD review of draft-ietf-sieve-3028bis-12
Date: Tue, 3 Apr 2007 14:49:09 +0100
Sender: [email-masked]

> OLD:
>    Note that for a message that is exactly 4,000 octets, the message is
>    neither ":over" 4000 octets or ":under" 4000 octets.
>                                ^^
> 
> NEW:
>    Note that for a message that is exactly 4,000 octets, the message is
>    neither ":over" 4000 octets nor ":under" 4000 octets.
>                                ^^^

If you are going to be in here, you might as well say:

    Note that for a message that is exactly 4,000 octets, the message is
-    neither ":over" 4000 octets or ":under" 4000 octets.
+   neither ":over" nor ":under" 4000 octets.

Nigel

Date: Mon, 2 Apr 2007 23:36:26 -0600
From: Philip Guenther <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Re: Support for encoded-character
Sender: [email-masked]

On Mon, 2 Apr 2007, Ned Freed wrote:
...
> Sorry, missed that was one of the core rules. But that makes me wonder 
> if LWSP wouldn't be more appropriate...

That's an interesting idea, as it would let you 'wrap' a string in the 
script without including the CRLF in the string's value.  Hmm.

> We also have a card-before-the-horse problem, in that we don't refer to 
> the use of ABNF for defining our grammars until section 8.1 but the ABNF 
> for encoded-character is in 2.4.2.4. (Well, to be fair, we do mention 
> ABNF in the conventions section, but only to compare it to how Usage: 
> statements are constructed.) The customary way to do this is to put the 
> reference to ABNF in the conventions section. I can understand not 
> bothering when ABNF was only used in one place, but that's no longer 
> true.

True.  There's also no real explanation of the "Syntax:" lines that 
'define' MATCH-TYPE, COMPARATOR, and ADDRESS-PART, although real ABNF for 
those is included in section 8.2.  <sigh> Perhaps this should be dealt 
with by
(1) including a specific statement in section 1.1 that ABNF is used for
     other syntax specifications in the body of the text, and
(2) replacing those "Syntax:" lines with their actual ABNF versions.

(This is turning into a long RFC editors note.  It's feeling like I should 
pull together the changes and submit a new rev...)

> Finally, I wonder if we should be explicit about how encoded-character
> constructs interact with text: blocks. My reading is that they should work.
> Does everyone else agree?

       If it's a string, then encoded-character affects its interpretation.
       text: blocks are strings.
MPP   The interpretation of text: blocks is affected by encoded-character

In addition, 2.4.2.4 specifically notes that encoded-character processing 
happens after dot-unstuffing, which only applies to text: blocks.

Philip Guenther

Cc: Ned Freed <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked]
Date: Mon, 02 Apr 2007 13:52:54 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Support for encoded-character
To: Philip Guenther <[email-masked]>
Sender: [email-masked]

> On Sun, 1 Apr 2007, Ned Freed wrote:
> ...
> > Another issue I recently noticed about encoded-character refers to the WSP
> > production but does not define it. I don't see it defined or referenced
> > anywhere else in the specification either. Perhaps this should be changed to
> > white-space, which is defined?

> No, not unless you really think we want this:

> "${unicode:110 /* a comment with a close brace } */ 69}"

> to be an encoding of "Hi".

> WSP is defined in the ABNF "Core rules" which are assumed as the base of
> all the ABNF snippets in 3028bis (c.f., DIGIT, CRLF, etc).

Sorry, missed that was one of the core rules. But that makes me wonder if LWSP
wouldn't be more appropriate...

We also have a card-before-the-horse problem, in that we don't refer to the use
of ABNF for defining our grammars until section 8.1 but the ABNF for
encoded-character is in 2.4.2.4. (Well, to be fair, we do mention ABNF in the
conventions section, but only to compare it to how Usage: statements are
constructed.) The customary way to do this is to put the reference to ABNF in
the conventions section. I can understand not bothering when ABNF was only used
in one place, but that's no longer true.

Finally, I wonder if we should be explicit about how encoded-character
constructs interact with text: blocks. My reading is that they should work.
Does everyone else agree?

				Ned

Date: Mon, 2 Apr 2007 14:58:35 -0600
From: Philip Guenther <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: comments on draft-freed-sieve-notary
Sender: [email-masked]

I found the extension name to be relatively obscure, having completely 
forgotten that the DSN work came from the notary WG.  "smtp-dsn" would 
have been more obvious to me, but perhaps that implies a restriction to 
SMTP that doesn't exist.  (Is there a reasonable X.400 mapping for these 
envelope values?)

It should be stated that any ADDRESS-PART argument is ignored when 
matching against the 'notify' or 'ret' envelope parts.  I suggest that it 
_not_ be ignored for 'envid' due to RFC 3885's tightening of the envid 
syntax when message tracking is used.

(Hurm.  RFC 3885's seems at least mildly broken, as it apparently implies 
that an "@" is not the same as its xtext encoded version.)

Speaking of RFC 3885, would it be reasonable to have this "notary" 
extension also define a "mtrk" envelope-part given how MTRK is tied to the 
notary parameters?  Or should that be a different capability?

     mtrk  Match the message tracking, or MTRK, value given in the
 	SMTP MAIL FROM command.  The syntax and semantics of the MTRK
 	parameter are defined in RFC 3885 [RFC3885] section 3.1.

(It may make sense to do MTRK in a separate extension simply because of 
its lower maturity level and deployment compared to DSN.)

An example of how to test whether 'notify' contained a specific set of 
values would help clarify the behavior.  Perhaps something like:

 	# Check whether only FAILURE notifications were requested
 	if allof ( envelope "notify" "FAILURE",
 		   envelope :comparator "i;ascii-numeric" :count "eq"
 			    "notify" "1"
 		 )
 	{
 		# do whatever
 	}

This extension provides a means to test the notary values on incoming 
messages, but not set them on messages sent via 'redirect'.  Has anyone 
played with extensions for setting SMTP parameters on outgoing messages in 
sieve?

Philip Guenther

Date: Mon, 2 Apr 2007 12:56:35 -0600
From: Philip Guenther <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Re: Support for encoded-character
Sender: [email-masked]

On Sun, 1 Apr 2007, Ned Freed wrote:
...
> Another issue I recently noticed about encoded-character refers to the WSP
> production but does not define it. I don't see it defined or referenced
> anywhere else in the specification either. Perhaps this should be changed to
> white-space, which is defined?

No, not unless you really think we want this:

"${unicode:110 /* a comment with a close brace } */ 69}"

to be an encoding of "Hi".

WSP is defined in the ABNF "Core rules" which are assumed as the base of 
all the ABNF snippets in 3028bis (c.f., DIGIT, CRLF, etc).

Philip Guenther

Cc: Aaron Stone <[email-masked]>, [email-masked]
Date: Sun, 01 Apr 2007 18:05:31 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Support for encoded-character
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> >After the Sieve interop yesterday evening sort of crumbled over
> >encoded-char, I implemented the hex side of things for libSieve.
> >
> >I also noticed that in 2038bis-12, 2.4.2.4, the sentence:
> >
> >   In the following script, message A is discarded, since the specified
> >   test string is equivalent to "$$$".
> >
> >...should refer to 'message B'.
> >
> >
> Indeed. I've sent Lisa/Chris a note saying that this needs to be fixed
> before publication.

Another issue I recently noticed about encoded-character refers to the WSP
production but does not define it. I don't see it defined or referenced
anywhere else in the specification either. Perhaps this should be changed to
white-space, which is defined?

				Ned

Date: Sun, 01 Apr 2007 15:12:27 +0100
From: Alexey Melnikov <[email-masked]>
To: Aaron Stone <[email-masked]>
CC: Chris Newman <[email-masked]>, [email-masked]
Subject: Re: AD review of draft-ietf-sieve-3028bis-12
Sender: [email-masked]

Aaron Stone wrote:

>Everything looks good. A few more wordsmithing comments below.
>
>On Fri, Mar 30, 2007, Alexey Melnikov <[email-masked]> said:
>  
>
>>  Chris Newman wrote:
>>    
>>
>>>Alexey Melnikov wrote on 3/29/07 18:15 +0100:
>>>      
>>>
>>>>After discussing this with Cyrus: chairs would like to publish -12 +
>>>>RFC-editor's notes.
>>>>        
>>>>
>>>That's fine.  Send me RFC-editor notes I can paste into the ballot and 
>>>I'll push the draft forward.
>>>      
>>>
>>Ok, here is the list of changes I would like to propose (Can please 
>>someone else sanity check this!).
>>    
>>
>[snip]
>  
>
>>In section 2.7.1, 1st paragraph, replace the 2nd sentence:
>>
>>OLD:
>>   Match type arguments are supplied to those commands which allow them 
>>to specify
>>   what kind of match is to be performed.
>>NEW:
>>  Match type arguments control what kind of match is to be performed by 
>>commands.
>>    
>>
>The whole thing is pretty awkward. Here's the whole paragraph in -12:
>
>   There are three match types describing the matching used in this
>   specification: ":is", ":contains", and ":matches".  Match type
>   arguments are supplied to those commands which allow them to specify
>   what kind of match is to be performed.
>
>   These are used as optional arguments to tests that perform string
>   comparison.
>
>Suggest:
>
>   Commands which perform string comparisons may have an optional match
>   type argument. The three match types in this specification are ":is",
>   ":contains", and ":matches". 
>  
>
I like that.

>I don't think we need to define the word argument "argument". We only need
>to explain what "match type arguments" are.
>
>
>A bit later in the section, I suggest dropping the highlighted clause:
>
>2.7.3. Comparators
>
>
>   In order to allow for language-independent, case-independent matches,
>   the match type may be coupled with a comparator name.  The Internet
>   Application Protocol Collation Registry [COLLATION] provides the
>   framework for describing and naming comparators as used by this
>   specification.                                  ^^^^^^^^^^^^^^^
>   ^^^^^^^^^^^^^^
>
>The purpose of the registry is not to document what Sieve does. Rather,
>Sieve is just using the registry.
>  
>
Good point.

>[snip]
>  
>
>>Add to the end:
>>
>>10. Added encoded-character capability and deprecated (but did not remove)
>>   use of arbitrary binary octets in Sieve scripts.
>>11. Updated IANA registration template, and permit capability prefix
>>   registrations.  Prefix registrations outside "vnd." require IESG 
>>approval.
>>12. Added .sieve as a valid extension for sieve scripts.
>>    
>>
>Suggest:
>
>    11. Updated IANA registration template, and added IANA considerations
>        to permit capability prefix registrations.
>
>We don't need to repeat the "vnd." requirement, we just have to point the
>reader to that section of the document.
>  
>
Yes, this is better.