Subject: Re: List of open issues with Sieve reject draft

From: Matthew Elvey <[email-masked]>
Date: Sat, 28 Oct 2006 22:53:30 -0700

On 7/18/06 1:36 PM, and at other times, Kristin Hubner wrote extensively 
about 1l8n, and pushed for :exacttext or something like it.

Kristin, will you be at IETF 67?  Alexey and Randall Gellens (and I, if 
I attend), will be getting together to discuss :exacttext.
(See attached email (which I trust they don't mind me sharing) for 
discussion points.)  I'm thinking it would be good if you could attend.

My own view was last expressed: "The /i18n/ solution is complex, but it 
does seem to be the simplest
solution given the constraints."  As much as I hate permitting MDNs and 
DSNs,  I have a hard time imagining telling a Chinese, say, that he 
can't communicate in his own language, and not feeling ridiculous.

I do wonder if punycode couldn't be an alternative.  I am really loathe 
to allow any unnecessary blowback.  (Incidentally, I personally have 
been receiving huge amounts of blowback over the past month or so.) 

Subject: Re: Review of draft-ietf-sieve-refuse-reject-03
From: Randall Gellens <[email-masked]>
To: Alexey Melnikov <[email-masked]>, Randall Gellens <[email-masked]>
Cc: Matthew Elvey <[email-masked]>
Date: Wed, 2 Aug 2006 11:56:01 -0700

At 11:04 AM +0100 8/2/06, Alexey Melnikov wrote:

>  Randall Gellens wrote:
>
>>  My comments on -02 apply to -03, with one extra:
>>
>>  Is the exacttext option worth it?  It seems to be adding a lot of 
>> complexity for fairly small gain.  I understand users wanting to 
>> put reject reasons in their native language.  However, the intent 
>> of this draft is that reject will operate at the SMTP level 
>> wherever possible.  Hence, the reject reason should be very short 
>> and must be usable in an SMTP response.  Use of ASCII is best for 
>> interoperability.  Use of UTF-8 will only work when EAI extensions 
>> are also being used, or when MDNs are being generated instead of 
>> SMTP protocol responses.  Since EAI is new, I don't think we want 
>> to rely on it.
>
>  This was discussed a lot on the mailing list. I thought we almost 
> came to an agreement of not having this option. But then Kristine 
> Hubner sent the following message:
>
>  &lt;http://www.imc.org/ietf-mta-filters/mail-archive/msg03079.html&gt;
>
>  and I think the message has a point.

To me, the message argues that some mechanism is needed to reject 
non-spam messages.  Such a mechanism needs UTF-8 and therefore 
shouldn't be at the SMTP level.  This argues for a command that 
generates an MDN.  I'd be fine with that.

But this draft is targeted at rejections where doing it at the SMTP 
level is far more important than delivering a nice reject reason. 
That argues that only short ASCII strings be permitted.

>   So I added the option.
>  But note, it is controlled by another capability. If EAI (or 
> whatever) enables UTF-8 response text in SMTP, this extension would 
> just go away.

That is asking for interoperability problems at least until EAI is 
widely deployed.

>
>>    And the intent is to get away from MDNs.  Hence, we should 
>> encourage brevity and mandate ASCII.
>
>>  Once EAI extensions are deployed we can extend reject to permit UTF-8.
>
>  EAI is not required by the draft, the text is inentionally vague to 
> allow anything :-).
>
>  Unfortunately we can't just require reject reason string to be in 
> ASCII, people want to use UTF-8.

Of course people want UTF-8, but the question is why and when.  In 
other words, which is more important: delivering nice rejection 
reasons, or doing SMTP-level rejections?  If the nice reasons are 
more important, drop this draft and stick to MDNs.  If SMTP-level is 
more important, require short ASCII strings.  If both are needed then 
introduce a new command that always does MDNs, or change this draft 
to use a new command that always does SMTP with a fall-back to DSN.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
As soon as we started programming, we found to our surprise that it
wasn't as easy to get programs right as we had thought.  Debugging had
to be discovered.  I can remember the exact instant when I realized
that a large part of my life from then on was going to be spent in
finding mistakes in my own programs.
 --Maurice Wilkes discovers debugging, 1949 (courtesy of Paul Robinson)

---

From: Matthew Elvey <[email-masked]>
Date: Sun, 29 Oct 2006 00:28:09 -0700

Oops; didn't mean to cc the whole list with that last post. Sorry, folks.

Date: Sun, 29 Oct 2006 00:28:09 -0700
From: Matthew Elvey <[email-masked]>
To: [email-masked]
Subject: Re: List of open issues with Sieve reject draft
Sender: [email-masked]

Oops; didn't mean to cc the whole list with that last post. Sorry, folks.

Date: Sat, 28 Oct 2006 22:53:30 -0700
From: Matthew Elvey <[email-masked]>
To: Kristin Hubner <[email-masked]>
CC: Randall Gellens <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Re: List of open issues with Sieve reject draft
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------090206060205070600020508
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

--------------090206060205070600020508
Content-Type: message/rfc822;
 name="Attached Message"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="Attached Message"

Date: Wed, 2 Aug 2006 11:56:01 -0700
To: Alexey Melnikov <[email-masked]>,
        Randall Gellens <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: Review of draft-ietf-sieve-refuse-reject-03
Cc: Matthew Elvey <[email-masked]>

-------------- Randomly-selected tag: ---------------

--------------090206060205070600020508--

Date: Thu, 26 Oct 2006 18:10:01 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: Tentative WG meeting agenda for San Diego
Sender: [email-masked]

Alexey Melnikov wrote:

> Hi folks,
> Here is a tentative proposal from WG chairs:
>
> - Introduction (blue sheets, scribe etc) (1 min)
> - Agenda Bashing (2 mins)
> - Report on completed work (7 mins)
> - Outstanding issues with Base spec (20 mins)
>  o escaping non-UTF-8 sequences
> - Reject/refuse draft - draft-ietf-sieve-refuse-reject-04.txt (25 mins)
>  o :exacttext parameter to reject
>  o extension for quering runtime environment
> - MIME loops draft - draft-ietf-sieve-mime-loop-01.txt (25 mins)
> - Notification draft - draft-ietf-sieve-notify-04.txt (20 mins)
> - Mailto Notification draft - draft-ietf-sieve-notify-mailto-01.txt 
> (10 mins)
> - Other topics (10 mins)

Speaking as a WG participant: I would like to quickly talk about 
draft-melnikov-sieve-imapext-metadata-00.txt, if people are interested.

> Total: 120 minutes
>
> Any comments/changes/additions/etc.?

Date: Thu, 26 Oct 2006 15:35:42 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Changes to Sieve refuse in -04
Sender: [email-masked]

Revision -04 has mostly editorial changes.

There is one change that requires special attention:
 Implementations that are running in MTA/MDAs are now prohibited from 
generating MDNs. If anybody think that this is a problem, please speak 
up now.

Date: Thu, 26 Oct 2006 15:28:39 +0100
From: Alexey Melnikov <[email-masked]>
To: undisclosed-recipients:;
Subject: Tentative WG meeting agenda for San Diego
Sender: [email-masked]

Hi folks,
Here is a tentative proposal from WG chairs:

- Introduction (blue sheets, scribe etc) (1 min)
- Agenda Bashing (2 mins)
- Report on completed work (7 mins)
- Outstanding issues with Base spec (20 mins)
  o escaping non-UTF-8 sequences
- Reject/refuse draft - draft-ietf-sieve-refuse-reject-04.txt (25 mins)
  o :exacttext parameter to reject
  o extension for quering runtime environment
- MIME loops draft - draft-ietf-sieve-mime-loop-01.txt (25 mins)
- Notification draft - draft-ietf-sieve-notify-04.txt (20 mins)
- Mailto Notification draft - draft-ietf-sieve-notify-mailto-01.txt (10 mins)
- Other topics (10 mins)

*
*

Total: 120 minutes

Any comments/changes/additions/etc.?

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-mime-loop-01.txt 
Date: Tue, 24 Oct 2006 15:50:02 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: SIEVE Email Filtering:  MIME part Tests, Iteration, Replacement and Enclosure
	Author(s)	: T. Hansen, C. Daboo
	Filename	: draft-ietf-sieve-mime-loop-01.txt
	Pages		: 12
	Date		: 2006-10-24
	
The SIEVE email filtering language has no way to examine individual
   MIME parts or any way to manipulate those individual parts.  However,
   being able to filter based on MIME content is important.  This
   document defines extensions for these needs.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-mime-loop-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-ietf-sieve-mime-loop-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-mime-loop-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-mime-loop-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-mime-loop-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Sun, 22 Oct 2006 23:30:38 -0600
From: Philip Guenther <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
cc: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Sat, 21 Oct 2006, Kjetil Torgrim Homme wrote:
> here's my amended 2.4.2.4, which attempts to fix it.  the wording is a
> bit weasely, but I think it will work in practice.  if anyone can fix it
> more formally in ABNF, please help out.
...
+   encoded-seq         = "${" enc-method ":" enc-argument "}"
+   enc-method          = "hex" / "unicode"
...
+   Values for enc-method or enc-argument which don't match the above
+   syntax SHOULD cause a syntax error.

Hmm, that won't work, as there isn't a defined meaning for something to 
not match _part_ of a syntax.  The only sensible interpretation I can see 
would be to match *anything* for enc-method and enc-argument, and then 
compare them to their expected forms.  The problem, with that is that it 
would render this string a syntax error:
 	"${name}: ${value}"

because it's an attempt to use encoded-seq with an enc-method of "name}" 
and a enc-argument of " ${value".  That's obviously not the desired 
result.

To obtain the desired result, we have to give the syntax for all the 
sequences that should be covered by the encoded-character extension, both 
those that have a defined expansion and those that should be treated as a 
syntax error.  How broad do we want to make that?  The broadest would 
cover any sequence matching this:

encoded-seq     = "${" enc-method ":" enc-argument "}"
enc-method      = *(%x01-39 / %x3b-7c / %x7e-ff)
 		  ; zero or more characters other than ':' or '}'
enc-argument    = *(%x01-7c / %x7e-ff)
 		  ; zero or more characters other than '}'

I.e., any sequence that starts with '${', followed by zero or more 
octets other than ':' and '}', followed by a ':', then zero or more 
octets other than '}', then finally a '}', would be considered a use of 
the encoded-seq syntax.

To put it another way, if you find a '${', and there's at least one colon 
between that and the next '}', it's an encoded-seq.

That would leave
 	"${name}: ${value}"
with its expected value, but make this:
 	 "${name: sdlkfjs}"
or this:
 	"${a.44:}"
a syntax error.  On the other hand, this:
 	"${hex.ff}"
would _not_ be a syntax error, because it doesn't contain a colon between 
the braces.  That's correct, of course, because we _want_ it to be a 
variable reference instead.

The above is the broadest syntax as makes sense.  We could tighten it up 
and only cover sequences where the enc-method is, say, alphanumeric.  If 
we did that, then this:
 	"${a.44:}"
would not be a syntax error.  Opinions?

Philip Guenther

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: [email-masked]
Date: Sat, 21 Oct 2006 22:05:28 +0200
Sender: [email-masked]

--=-bshwpV56rNS3vYZbVhLk
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

> oh.  I just realised my suggestion has the same shortcoming as the
> variables syntax:  it won't raise syntax errors, since it only matches
> well-formed character sequences.

here's my amended 2.4.2.4, which attempts to fix it.  the wording is a
bit weasely, but I think it will work in practice.  if anyone can fix it
more formally in ABNF, please help out.

I hope it addresses the issues brought up (6 digit Unicode, single pass,
NUL, extension name).  thanks for the feedback, everyone!

-- 
Kjetil T.

--=-bshwpV56rNS3vYZbVhLk
Content-Description: 
Content-Disposition: inline; filename=encoded-character.patch
Content-Type: text/x-patch; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

--- draft-ietf-sieve-3028bis-09.txt	2006-10-06 01:47:33.989869000 +0200
+++ draft-ietf-sieve-3028bis-kjetilho.txt	2006-10-21 21:52:27.461172000 +0200
@@ -393,6 +393,10 @@
    invalid data and in arguments containing raw MIME parts for extension
    actions that generate outgoing messages.
 
+   The extension "encoded-character" may be used to encode arbitrary
+   characters as a sequence of US-ASCII characters (see 2.4.2.4 for
+   details).
+
    For entering larger amounts of text, such as an email message, a
    multi-line form is allowed.  It starts with the keyword "text:",
    followed by a CRLF, and ends with the sequence of a CRLF, a single
@@ -470,6 +474,46 @@
    valid, but need not ensure that they actually identify an email
    recipient.
 
+2.4.2.4. Encoding characters using "encoded-character"
+
+   When the "encoded-character" extension is in effect, character
+   sequences in strings which match the encoded-seq syntax are
+   replaced by the decoded value.  This matching happens after escape
+   sequences are interpreted and dot-unstuffing has been done.  A
+   single pass is done.
+
+   encoded-seq         = "${" enc-method ":" enc-argument "}"
+   enc-method          = "hex" / "unicode"
+   enc-argument        = hex-list
+   hex-list            = hex-group *(WSP hex-group)
+   hex-group           = 1*6HEXDIG
+
+   Arbitrary octets can be embedded in strings by using the encoding
+   method "hex".  The sequence is replaced by the octets with the
+   hexadecimal values given by each hex-group.  Values greater than
+   255 ("ff") are a syntax error.
+
+   It may be inconvenient or undesirable to enter Unicode characters
+   verbatim, and in these cases the method "unicode" can be used. The
+   sequence is replaced by the UTF-8 encoding of the specified Unicode
+   characters, whose code points are identified by the hexadecimal
+   value of each hex-group.
+
+   Values for enc-method or enc-argument which don't match the above
+   syntax SHOULD cause a syntax error.  Implementations SHOULD support
+   encoded NUL octets.
+
+   The capability string for use with the require command is
+   "encoded-character".
+
+   In the following script, message A is discarded, since the
+   specified test string is equivalent to "$$$".
+
+   Example:   require "encoded-character";
+              if header :contains "Subject" "$${hex:24 24}" {
+                    discard;
+              }
+
 2.5.     Tests
 
    Tests are given as arguments to commands in order to control their
@@ -1075,7 +1119,7 @@
    Implementations MUST support the "keep", "discard", and "redirect"
    actions.
 
-   Implementations SHOULD support "fileinto".
+   Implementations SHOULD support "fileinto" and "encoded-character".
 
    Implementations MAY limit the number of certain actions taken (see
    section 2.10.4).
@@ -1561,6 +1605,12 @@
    RFC number:      this RFC (Sieve base spec)
    Contact address: The Sieve discussion list <[email-masked]>
 
+   Capability name: encoded-character
+   Description:     changes the parsing of strings to allow arbitrary
+                    characters to be embedded
+   RFC number:      this RFC (Sieve base spec)
+   Contact address: The Sieve discussion list <[email-masked]>
+
    Capability name: comparator-* (anything starting with "comparator-")
    Description:     adds the indicated comparator for use with the
                     :comparator argument

--=-bshwpV56rNS3vYZbVhLk--

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-notify-04.txt 
Date: Fri, 20 Oct 2006 15:50:01 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Extension: Notifications
	Author(s)	: A. Melnikov, et al.
	Filename	: draft-ietf-sieve-notify-04.txt
	Pages		: 16
	Date		: 2006-10-20
	
Users go to great lengths to be notified as quickly as possible that
   they have received new mail.  Most of these methods involve polling
   to check for new messages periodically.  A push method handled by the
   final delivery agent gives users quicker notifications and saves
   server resources.  This document does not specify the notification
   method but is expected that using existing instant messaging
   infrastructure such as Zephyr, Jabber, or SMS messages will be
   popular.  This draft describes an extension to the Sieve mail
   filtering language that allows users to give specific rules for how
   and when notifications should be sent.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-notify-04.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-ietf-sieve-notify-04.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-notify-04.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-notify-04.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-notify-04.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: [email-masked]
Date: Fri, 20 Oct 2006 15:21:09 +0200
Sender: [email-masked]

On Thu, 2006-10-19 at 18:28 -0600, Philip Guenther wrote:
> As for the name of the extension, I think it should be "encoded-character" 
> instead of "quoted-character": this is not a quoting scheme but rather an 
> encoding scheme.

agreed.

> Regarding the description of the language in section 2.1, how does the 
> following addition (based on text from Alexey) look:
> 
>        While this specification permits arbitrary octets to appear in
>        sieve scripts inside strings and comments, this has made it
>        difficult to robustly handle sieve scripts in user interfaces.
>        The "encoded-character" capability (section 2.4.2.4) provides an
>        alternative means of representing such octets in strings using
>        just US-ASCII characters.  As such, the use of non-UTF-8 text in
>        scripts should be considered a deprecated feature that may be
>        abandoned.
> 
> That would be inserted as the third paragraph in that section and given 
> additional indentation (ala the behaviors described in section 2.7.2).

this is good enough for me.  I think "deprecated" is stronger than
"SHOULD NOT" in practice :-)

-- 
Kjetil T.

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Nigel Swinson <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
Date: Fri, 20 Oct 2006 15:17:48 +0200
Sender: [email-masked]

On Fri, 2006-10-20 at 13:41 +0100, Nigel Swinson wrote:
> > I disagree. I think encoded-octet and variables should be independent 
> > extensions.
> 
> I agree

ditto.

> > That's 2:1 now saying we need a new extension to evaluate "${hex:${variable}}".
> > Anybody else?
> 
> I would want ${hex:${variable}} to be a syntax error if "hex" was
> requir()ed, and valid if variables was require()ed and "hex" was not.
> I don't see a need for recursive evaluation.  I'm used to one pass,
> just like dequoting.  At the moment I can't think of any recursive
> string expansion out there...  Instead put your hex into ${variable}
> when constructing it.

this implies variables should evaluated first.

I must admit I envisaged a single pass (concurrent evaluation) with no
backtracking, which means

  "${hex:${var}}" -> "${hex:VALUE}"
  "${v${hex:61}}" -> "${var}"

but I now think it makes more sense to order them explicitly.  it makes
most sense to have encoded-character evaluated before variables (since
it's defined in the base spec), but it's probably more useful to do
variables before encoded-characters, as Nigel demonstrates.

oh.  I just realised my suggestion has the same shortcoming as the
variables syntax:  it won't raise syntax errors, since it only matches
well-formed character sequences.  I may have a look at fixing that
during the weekend.
-- 
Kjetil T.

Date: Fri, 20 Oct 2006 14:50:00 +0200
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> > I disagree. I think encoded-octet and variables should be independent 
> > extensions.
>
> I agree

Ok, so let both be top-level things, working independent of each other.

> > That's 2:1 now saying we need a new extension to evaluate "${hex:${variable}}".
> > Anybody else?
>
> I would want ${hex:${variable}} to be a syntax error if "hex" was requir()ed, and valid if variables was require()ed and "hex" was not.  I don't see a need for recursive evaluation.  I'm used to one pass, just like dequoting.  At the moment I can't think of any recursive string expansion out there...  Instead put your hex into ${variable} when constructing it.

Hmm.  So the result of the second would be "${hex:variable value}"?

I tend to say: If ${ is seen and there is no }, or stuff inside that
can't be processed for whatever reason, then cause an error.  But I am
not sure how variables defined that so far.

Michael

X-Spam-Score-scoring_explanation: 
X-Spam-Score-spf_status: 
X-Spam-Score-Spamcatcher1: 2ccc5e07bb0018aea219e173bd0975b2
X-Spam-Score-Summary:  2,0,0,1042f17134ee755d,43cc87f922f1c253,[email-masked],,RULES_HIT:355:379:539:540:541:542:543:567:599:601:945:988:989:1155:1156:1261:1277:1311:1313:1314:1345:1437:1513:1515:1516:1518:1521:1534:1538:1587:1593:1594:1711:1714:1730:1747:1766:1792:2073:2075:2078:2393:2559:2562:2828:3027:3351:3865:3866:3867:3868:3869:3872:3874:4470,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:,MSBL:none,DNSBL:none
X-Spam-Score-rbl_summary: none
X-Spam-Score-Phishing_status: no
From: "Nigel Swinson" <[email-masked]>
To: "Michael Haardt" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Date: Fri, 20 Oct 2006 13:41:51 +0100
Sender: [email-masked]

> I disagree. I think encoded-octet and variables should be independent 
> extensions.

I agree

> That's 2:1 now saying we need a new extension to evaluate "${hex:${variable}}".
> Anybody else?

I would want ${hex:${variable}} to be a syntax error if "hex" was requir()ed, and valid if variables was require()ed and "hex" was not.  I don't see a need for recursive evaluation.  I'm used to one pass, just like dequoting.  At the moment I can't think of any recursive string expansion out there...  Instead put your hex into ${variable} when constructing it.

Nigel

Date: Fri, 20 Oct 2006 14:06:43 +0200
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> >>Which comes first, encoding replacement or variable expansion?  Or are 
> >>they concurrent?  Whatever the answer, the variables I-D will need to make 
> >>that clear.
> >>(I think encoding replacement should come before variable expansion.)
> >>    
> >>
> >I think they should be processed inside-out, but not in separate
> >passes.  Variables introduce of the concept of strings (aka test
> >and action arguments) not being literals, but string expressions that
> >are evaluated.  It makes sense that arguments of string functions
> >turn from literals to string expressions, too.
> >
> >As a result, without variables, "${hex:${hex:4646}}" is a syntax error.
> >
> >With variables, it is "FF".
> >  
> >
> I disagree. I think encoded-octet and variables should be independent 
> extensions.

That's 2:1 now saying we need a new extension to evaluate "${hex:${variable}}".
Anybody else?

> Speaking as implementor I am more likely to implement encoded-octets first.

So am I.

Michael

Date: Fri, 20 Oct 2006 14:01:02 +0200
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> If you're referring to RFC 2047's Q encoding, 3028bis currently says:
>
>        <...>  An encoded NUL octet
>        (character zero) SHOULD NOT cause early termination of the header
>        content being compared against.
>
> So an implementation that refused to match beyond the =00 would still be 
> "conditionally compliant".

So how about handling an encoded NUL in strings just the same? SHOULD NOT
is reasonable.

> The original MIME documents didn't make an exception for NUL...and that 
> had to be changed when they were revised as part of moving to Draft 
> Standard.  RFC 2049 says:
>      (17)  The definitions of "7bit" and "8bit" have been
>            tightened so that use of bare CR, LF can only be used
>            as end-of-line sequences.  The document also no longer
>            requires that NUL characters be preserved, which brings
>            MIME into alignment with real-world implementations.

Without looking it up, it sounds as if that applies to unencoded NUL
characters.  I agree there is a real-world problem with them and the
above makes perfect sense.  I would not like if quoted-printable may
drop =00.

> 1) Given that variables are *explicitly* not handled that way, why should
>     that be true of these?
>
> 2) Why would pulling in the variable capabilty change the behavior of the
>     above?
>
> 2) The behavior you describe would be useful how?

The variable capability changes how argument string are interpreted.
Arguments to string functions are strings, too.  That's why it makes
sense to me that variables cause recursive evaluation, but that may
just be me.

Implementations that do not implement variables can scan strings
with a linear effort and without the need for recursion or a parser.
Side-effect free functions of constant arguments can be evaluated until
you reach the top level, which can not be represented in UTF-8, hence
we need the top level to encode the result, but no more.

I don't insist on this behaviour, it was just a suggestion.  Using
${hex:...} opens the door to other string functions, and we are really
talking about introducing string functions here and the question is:
Which extension introduces recursive evaluation? I suggested variables,
but it may as well be some not-yet-specified extension.

> My interactions with Sendmail's Japanese customers tell me that 
> ISO-2022-JP is still heavily preferred over UTF-8 in email in Japan.  Are 
> you claiming otherwise?  Or do you think that it doesn't matter?

Actually, my feedback from customers is that national character sets
are still preferred over UTF-8 - independent of their location.  I am
REAL GLAD Sieve went the UTF-8 route to begin with, because national
character sets are a royal pain and I hope that historic crap disappears
some day.  Each day Unicode is used a little more, we are a little closer
to that point.

Personally, I have no stuff capable of displaying UTF-8, and looking at
other (Linux) installations, the claim of being able to switch completely
to Unicode is not YET true.  iconv(1) and similar tools help a lot
with existing installations that are tied to national character sets,
e.g. when viewing a Sieve script stored in UTF-8 on my ISO-8859-15
display.  Japanese may be worse, but German has a non-US-ASCII letter
that only exists in lower case.  Translating it to upper case results
in two letters and you can not convert it back without a dictionary.
It's not like I would be ignorant of non-US-ASCII text.

I had hordes of customers yell and scream at me that they will cease to
exist without their open SMTP relay.  Some probably still hate me for
having to close it.  I don't always listen to their wishes.

Michael

Date: Fri, 20 Oct 2006 05:01:19 -0600
From: Philip Guenther <[email-masked]>
To: Michael Haardt <[email-masked]>
cc: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Fri, 20 Oct 2006, Michael Haardt wrote:
>> Do implementations have to support encodings of NUL ala ${hex:0}?  If so,
>> that will be a barrier to this extension being broadly supported.
>> (I think it should be a "MAY support".)
>
nn> Note that =00 in quoted printable has to be dealt with already.

If you're referring to RFC 2047's Q encoding, 3028bis currently says:

       <...>  An encoded NUL octet
       (character zero) SHOULD NOT cause early termination of the header
       content being compared against.

So an implementation that refused to match beyond the =00 would still be 
"conditionally compliant".

> An implementation that uses nul-terminated strings is already broken
> and things will not get worse using an encoded NUL character in a string.

So, in your opinion, being able to insert NUL into strings is as important 
as the ability to encode other, non-UTF-8, non-NUL octets, and that 
support for those capabilities should therefore be tied?  I think the 
result will be fewer implementations of this extension as well as 
implementation that don't actually comply.

The original MIME documents didn't make an exception for NUL...and that 
had to be changed when they were revised as part of moving to Draft 
Standard.  RFC 2049 says:
     (17)  The definitions of "7bit" and "8bit" have been
           tightened so that use of bare CR, LF can only be used
           as end-of-line sequences.  The document also no longer
           requires that NUL characters be preserved, which brings
           MIME into alignment with real-world implementations.

>> Which comes first, encoding replacement or variable expansion?  Or are
>> they concurrent?  Whatever the answer, the variables I-D will need to make
>> that clear.
>> (I think encoding replacement should come before variable expansion.)
>
> I think they should be processed inside-out, but not in separate
> passes.  Variables introduce of the concept of strings (aka test
> and action arguments) not being literals, but string expressions that
> are evaluated.  It makes sense that arguments of string functions
> turn from literals to string expressions, too.
>
> As a result, without variables, "${hex:${hex:4646}}" is a syntax error.

No, it would be replaced with "${hex:FF}"

> With variables, it is "FF".

1) Given that variables are *explicitly* not handled that way, why should
    that be true of these?

2) Why would pulling in the variable capabilty change the behavior of the
    above?

2) The behavior you describe would be useful how?

>> I'll note that while a script that uses this extension for all non-UTF8
>> octets may be displayable without munging, the result may still be
>> incomprehensible if, for example, an 8bit ISO-2022-JP MIME part is
>> included in a string.  Indeed, such encoding will probably make it more
>> difficult to display that MIME part readably: currently, if a user is
>> viewing the script with raw ISO-2022-JP in it they can probably display
>> that MIME part by overriding their browser's charset encoding for the
>> page.
>
> Indeed, a hack like that is not as useful any more.  But it was a hack
> to begin with, because overriding the charset encoding renders contained
> UTF-8 text useless.

No, US-ASCII is a subset of ISO-2022-JP, so at least the syntax portions 
of the script would still be readable.  Is it a hack?  Yes.  Does it work? 
Yes.  Will there be any equivalent when the raw ISO-2022-JP is encoded 
using ${hex}?  I strongly doubt it.

> Convert the ISO-2022-JP part to UTF-8.

My interactions with Sendmail's Japanese customers tell me that 
ISO-2022-JP is still heavily preferred over UTF-8 in email in Japan.  Are 
you claiming otherwise?  Or do you think that it doesn't matter?

Philip Guenther

Date: Fri, 20 Oct 2006 11:06:41 +0100
From: Alexey Melnikov <[email-masked]>
To: Michael Haardt <[email-masked]>
CC: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

Michael Haardt wrote:

>>Do implementations have to support encodings of NUL ala ${hex:0}?  If so, 
>>that will be a barrier to this extension being broadly supported.
>>(I think it should be a "MAY support".)
>>    
>>
>
>Note that =00 in quoted printable has to be dealt with already.
>An implementation that uses nul-terminated strings is already broken
>and things will not get worse using an encoded NUL character in a string.
>  
>
Right.

>>Which comes first, encoding replacement or variable expansion?  Or are 
>>they concurrent?  Whatever the answer, the variables I-D will need to make 
>>that clear.
>>(I think encoding replacement should come before variable expansion.)
>>    
>>
>I think they should be processed inside-out, but not in separate
>passes.  Variables introduce of the concept of strings (aka test
>and action arguments) not being literals, but string expressions that
>are evaluated.  It makes sense that arguments of string functions
>turn from literals to string expressions, too.
>
>As a result, without variables, "${hex:${hex:4646}}" is a syntax error.
>
>With variables, it is "FF".
>  
>
I disagree. I think encoded-octet and variables should be independent 
extensions.
Speaking as implementor I am more likely to implement encoded-octets first.

>>The next rev of the base-spec includes this:
>> 	Extensions MUST NOT change the behavior of the "require"
>> 	control command.
>>I believe that means that this extension can't be used in the capability 
>>argument to 'require', so there's at least one place where Unicode 
>>characters can't be encoded using ${unicode:...}.
>>    
>>
>That's right.  The require argument can not contain variables either,
>and I consider that a good thing [tm].
>  
>
Agreed.

Date: Fri, 20 Oct 2006 11:43:20 +0200
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> Do implementations have to support encodings of NUL ala ${hex:0}?  If so, 
> that will be a barrier to this extension being broadly supported.
> (I think it should be a "MAY support".)

Note that =00 in quoted printable has to be dealt with already.
An implementation that uses nul-terminated strings is already broken
and things will not get worse using an encoded NUL character in a string.

> Which comes first, encoding replacement or variable expansion?  Or are 
> they concurrent?  Whatever the answer, the variables I-D will need to make 
> that clear.
> (I think encoding replacement should come before variable expansion.)

I think they should be processed inside-out, but not in separate
passes.  Variables introduce of the concept of strings (aka test
and action arguments) not being literals, but string expressions that
are evaluated.  It makes sense that arguments of string functions
turn from literals to string expressions, too.

As a result, without variables, "${hex:${hex:4646}}" is a syntax error.

With variables, it is "FF".

> The next rev of the base-spec includes this:
>  	Extensions MUST NOT change the behavior of the "require"
>  	control command.
> I believe that means that this extension can't be used in the capability 
> argument to 'require', so there's at least one place where Unicode 
> characters can't be encoded using ${unicode:...}.

That's right.  The require argument can not contain variables either,
and I consider that a good thing [tm].

> I'll note that while a script that uses this extension for all non-UTF8 
> octets may be displayable without munging, the result may still be 
> incomprehensible if, for example, an 8bit ISO-2022-JP MIME part is 
> included in a string.  Indeed, such encoding will probably make it more 
> difficult to display that MIME part readably: currently, if a user is 
> viewing the script with raw ISO-2022-JP in it they can probably display 
> that MIME part by overriding their browser's charset encoding for the 
> page.

Indeed, a hack like that is not as useful any more.  But it was a hack
to begin with, because overriding the charset encoding renders contained
UTF-8 text useless.  Convert the ISO-2022-JP part to UTF-8.

Michael

Date: Fri, 20 Oct 2006 02:32:37 -0600
From: Philip Guenther <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
cc: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

There are a number of issues that need to be clarified before the proposed 
extension will be ready for inclusion:

Do implementations have to support encodings of NUL ala ${hex:0}?  If so, 
that will be a barrier to this extension being broadly supported.
(I think it should be a "MAY support".)

Which comes first, encoding replacement or variable expansion?  Or are 
they concurrent?  Whatever the answer, the variables I-D will need to make 
that clear.
(I think encoding replacement should come before variable expansion.)

unicode-hex should be 1*6HEXDIG: the Unicode range goes up to 10FFFF

The next rev of the base-spec includes this:
 	Extensions MUST NOT change the behavior of the "require"
 	control command.
I believe that means that this extension can't be used in the capability 
argument to 'require', so there's at least one place where Unicode 
characters can't be encoded using ${unicode:...}.

I'll note that while a script that uses this extension for all non-UTF8 
octets may be displayable without munging, the result may still be 
incomprehensible if, for example, an 8bit ISO-2022-JP MIME part is 
included in a string.  Indeed, such encoding will probably make it more 
difficult to display that MIME part readably: currently, if a user is 
viewing the script with raw ISO-2022-JP in it they can probably display 
that MIME part by overriding their browser's charset encoding for the 
page.

Philip Guenther

Date: Fri, 20 Oct 2006 09:05:50 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Thu, Oct 19, 2006 at 06:28:42PM -0600, Philip Guenther wrote:
> As for the name of the extension, I think it should be "encoded-character" 
> instead of "quoted-character": this is not a quoting scheme but rather an 
> encoding scheme.

Or possibly "encoded-octet"?

> Regarding the description of the language in section 2.1, how does the 
> following addition (based on text from Alexey) look:
> 
>       While this specification permits arbitrary octets to appear in
>       sieve scripts inside strings and comments, this has made it
>       difficult to robustly handle sieve scripts in user interfaces.
>       The "encoded-character" capability (section 2.4.2.4) provides an
>       alternative means of representing such octets in strings using
>       just US-ASCII characters.  As such, the use of non-UTF-8 text in
>       scripts should be considered a deprecated feature that may be
>       abandoned.
> 
> That would be inserted as the third paragraph in that section and given 
> additional indentation (ala the behaviors described in section 2.7.2).

To me, it does not point out strong enough that new implementations
shold not allow non-UTF-8 octet sequences AT ALL.

How about:

       Sieve scripts SHOULD NOT use non-UTF-8 octet sequences.
       If non-UTF-8 octet sequences are required in strings, the
       "encoded-octet" capability (section 2.4.2.4) provides an
       alternative means of representing such octets in strings using
       just US-ASCII characters.

It's not a MUST and backwards compatibility of specific implementations
is certainly a very good reason to act against the SHOULD.

Michael

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-refuse-reject-04.txt 
Date: Fri, 20 Oct 2006 02:50:01 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: The SIEVE mail filtering language - reject extension
	Author(s)	: M. Elvey, A. Melnikov
	Filename	: draft-ietf-sieve-refuse-reject-04.txt
	Pages		: 0
	Date		: 2006-10-19
	
This memo updates the definition the SIEVE mail filtering language 
   (RFC <<3028bis>>) "reject" extension, originally defined in RFC 3028.
   
   A Joe-job is a spam run forged to appear as though it came from an
   innocent party, who is then generally flooded by the bounces,
   Message Disposition Notifications (MDNs) and messages with
   complaints.  The original Sieve "reject" action defined in RFC 3028
   required use of MDNs for rejecting messages, thus contributing to
   the flood of Joe-job spam to victims of Joe-jobs.  This document
   updates the definition of "reject" to require rejecting messages
   during the SMTP transaction (instead of accepting them and then
   sending MDNs back to the alleged sender) wherever possible, thereby
   reducing the problem.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-refuse-reject-04.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-ietf-sieve-refuse-reject-04.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-refuse-reject-04.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-refuse-reject-04.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-refuse-reject-04.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Thu, 19 Oct 2006 18:28:42 -0600
From: Philip Guenther <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
cc: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Wed, 18 Oct 2006, Kjetil Torgrim Homme wrote:
...
> we have three proposals here:
>
>  $%<hex>

I dislike this as too specialized.

>  ${hex:<hex>}
>
> and a slightly more nebulous "a more unique prefix", perhaps this
> qualifies:
>
>  $[hex:<hex>]
>
> that would be fine with me, in any case.

I think these are both acceptable; I have a slight preference for ${}

As for the name of the extension, I think it should be "encoded-character" 
instead of "quoted-character": this is not a quoting scheme but rather an 
encoding scheme.

Regarding the description of the language in section 2.1, how does the 
following addition (based on text from Alexey) look:

       While this specification permits arbitrary octets to appear in
       sieve scripts inside strings and comments, this has made it
       difficult to robustly handle sieve scripts in user interfaces.
       The "encoded-character" capability (section 2.4.2.4) provides an
       alternative means of representing such octets in strings using
       just US-ASCII characters.  As such, the use of non-UTF-8 text in
       scripts should be considered a deprecated feature that may be
       abandoned.

That would be inserted as the third paragraph in that section and given 
additional indentation (ala the behaviors described in section 2.7.2).

Philip Guenther

Date: Thu, 19 Oct 2006 23:43:21 +0200
To: [email-masked]
Subject: Re: non-UTF-8 sequences in Sieve scripts
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> >[3028bis-09: 2.1. Form of the Language (Â§2)]:
> >|   With the exceptions of strings and comments, the language is limited
> >|   to US-ASCII characters.  Strings and comments may contain octets
> >|   outside the US-ASCII range.  Specifically, they will normally be in
> >|   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
> >|   in scripts, while CR and LF can only appear as the CRLF line ending.
> >
> >[my suggestion]:
> >|   With the exceptions of strings and comments, the language is limited
> >|   to US-ASCII characters.  Strings and comments are encoded in
> >|   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
> >|   in scripts, while CR and LF can only appear as the CRLF line ending.

That asks for no more than RFC 3028, it is just more detailed.  IMHO,
whoever thinks RFC 3028 allows non-UTF-8 should still think the same.
Sounds good to me.

> >[3028bis-09: 2.4.2. Strings (Â§6)]:
> >|   As message header data is converted to [UTF-8] for comparison (see
> >|   section 2.7.2), most strings will use the UTF-8 encoding.  However,
> >|   implementations MUST accept all strings that match the grammar in
> >|   section 8.  The ability to use non-UTF-8 encoded strings matches
> >|   existing practice and has proven to be useful both in tests for
> >|   invalid data and in arguments containing raw MIME parts for extension
> >|   actions that generate outgoing messages.
> >
> >[my suggestion]:
> >|   [strike paragraph 6 in its entirety]

I think it is important to keep "Message header data is converted to
[UTF-8] for comparison".  Of course implementations must obey the grammar,
no matter how it looks like, so that part seems unneeded.

I thought -09 said that on headers, but I was wrong.  But there's something
else, while I am reading 2.4.2.2 (just nitpicking):

s/synactically/syntactically/

   Headers are a subset of strings.  In the Internet Message
   Specification [IMAIL], each header line is allowed to have whitespace
   nearly anywhere in the line, including after the field name and
   before the subsequent colon.  Extra spaces between the header name
   and the ":" in a header field are ignored.

If you think it's really just nitpicking, just forget about my
suggestion:

   Header field names are a subset of strings.  The obsolete header
   field syntax from [IMAIL], section 4.5, must be implemented,
   matching a header with white space between the field name and the
   subsequent colon.

But: Can a Sieve header contain trailing white space that is being
ignored, too? Like "from   "? 

Sorry I bring this up so late, never thought about it before.

> I think the rough consensus on this issue around Montreal IETF was not 
> to break existing implementations (which accept arbitrary octets).
> I believe the existing -09 text represents this rough consensus.

That is correct, there was agreement not to word things that those
implementations DO violate the standard, but also not to word things in
a way that encourages to repeat their behaviour.

> Personally, I think I would agree with your proposal 6 months after 
> 3028bis is published, when we try to move 3028bis RFC to Draft status 
> and if your octet encoding extension is deployed.

Sounds good to me.  Once there is a draft on that extension, I will
certainly implement it in Exim.

Michael

Date: Thu, 19 Oct 2006 19:56:09 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked]
Subject: Re: non-UTF-8 sequences in Sieve scripts
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>okay, fine, we don't have to make 3028bis more explicit than 3028 about
>disallowing arbitrary octets.  but we should definitely not add explicit
>text which allows it either!  let's stick to the original wording, but
>keep some of the clarifications from the draft:
>
>[3028bis-09: 2.1. Form of the Language (§2)]:
>|   With the exceptions of strings and comments, the language is limited
>|   to US-ASCII characters.  Strings and comments may contain octets
>|   outside the US-ASCII range.  Specifically, they will normally be in
>|   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
>|   in scripts, while CR and LF can only appear as the CRLF line ending.
>
>[my suggestion]:
>|   With the exceptions of strings and comments, the language is limited
>|   to US-ASCII characters.  Strings and comments are encoded in
>|   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
>|   in scripts, while CR and LF can only appear as the CRLF line ending.
>
>[3028bis-09: 2.4.2. Strings (§6)]:
>|   As message header data is converted to [UTF-8] for comparison (see
>|   section 2.7.2), most strings will use the UTF-8 encoding.  However,
>|   implementations MUST accept all strings that match the grammar in
>|   section 8.  The ability to use non-UTF-8 encoded strings matches
>|   existing practice and has proven to be useful both in tests for
>|   invalid data and in arguments containing raw MIME parts for extension
>|   actions that generate outgoing messages.
>
>[my suggestion]:
>|   [strike paragraph 6 in its entirety]
>
>the text from 8.1 is unchanged in 3028bis-09, so this is all we need to
>maintain status quo.  we'll also keep the accurate UTF-8 definitions of
>characters out of the ABNF, but may decide to change that later, in the
>Standard revision of the document.
>
>I hope this proposal is agreeable to all concerned.
>  
>
I think the rough consensus on this issue around Montreal IETF was not 
to break existing implementations (which accept arbitrary octets).
I believe the existing -09 text represents this rough consensus.
If you disagree that this is the case, I would let you (as a chair) to 
have one more round of discussion of this issue on the mailing list. The 
discussion must end by the Sieve meeting at San Diego IETF (November 6th).

Personally, I think I would agree with your proposal 6 months after 
3028bis is published, when we try to move 3028bis RFC to Draft status 
and if your octet encoding extension is deployed.

X-Spam-Score-scoring_explanation: 
X-Spam-Score-spf_status: 
X-Spam-Score-Spamcatcher1: 30f3609119268c73fe48372614b125f5
X-Spam-Score-Summary:  2,0,0,8e736b7c9d990b95,43cc87f922f1c253,[email-masked],,RULES_HIT:355:379:539:540:541:542:543:567:599:601:973:988:989:1155:1156:1261:1277:1311:1313:1314:1345:1437:1513:1515:1516:1518:1521:1534:1537:1566:1587:1593:1594:1711:1714:1730:1747:1766:1792:2073:2075:2078:2198:2199:2393:2559:2562:2828:3027:3865:3866:3867:3872:3874:4470,0,RBL:none,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:,MSBL:none,DNSBL:none
X-Spam-Score-rbl_summary: none
X-Spam-Score-Phishing_status: no
From: "Nigel Swinson" <[email-masked]>
To: <[email-masked]>
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Date: Thu, 19 Oct 2006 15:22:52 +0100
Sender: [email-masked]

> I like ${...}, because it specifies the borders of "magic inside".
> All the magic we currently have are variables, but that does not mean
> things have to stay that way.

Me too.  :o)  I think $[ or $( just throw things more open to error, forgetting which braket to use :o/

Nigel

Date: Thu, 19 Oct 2006 15:16:07 +0200
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Michael Haardt <[email-masked]>
Sender: [email-masked]

> > I like ${hex:<hex>}, because it looks like a function and we can use the
> > same syntax for other string expansions, too.
>
> I think it only looks like a function to you because you're used to
> exim.conf :-)  if you know of other languages where functions looks like
> this, I'm interested in hearing about it.

Yes and no. :) Yes: It IS familar, because I use Exim, and I know of no
other language using that notation.  No: Seen from an abstract point
of view, a variable and a function call differ only in some syntactic
element that specifies arguments, because both deliver a rhs value
(assuming strict evaluation).

That's why I like to keep function calls very close to variables.

If you dislike the colon, I don't mind exchanging that against some other
token, as long as it is not in the follow-set of the variable identifier
in the variables extension.

I like ${...}, because it specifies the borders of "magic inside".
All the magic we currently have are variables, but that does not mean
things have to stay that way.

Michael

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Date: Thu, 19 Oct 2006 14:49:38 +0200
Sender: [email-masked]

On Thu, 2006-10-19 at 08:36 +0200, Michael Haardt wrote:
> On Wed, Oct 18, 2006 at 07:49:55PM +0200, Kjetil Torgrim Homme wrote:
> > we have three proposals here:
> > 
> >   $%<hex>
> >   ${hex:<hex>}
> > 
> > and a slightly more nebulous "a more unique prefix", perhaps this
> > qualifies:
> > 
> >   $[hex:<hex>]
> > 
> > that would be fine with me, in any case.
> 
> I like ${hex:<hex>}, because it looks like a function and we can use the
> same syntax for other string expansions, too.

I think it only looks like a function to you because you're used to
exim.conf :-)  if you know of other languages where functions looks like
this, I'm interested in hearing about it.

if we want a syntax well known from other languages, we could do

  $(hex <hex> <hex>)

this has a strong similarity to POSIX shell.

which of ${, $[ or $( we use is not important to me.
-- 
Kjetil T.

Date: Thu, 19 Oct 2006 08:36:09 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Wed, Oct 18, 2006 at 07:49:55PM +0200, Kjetil Torgrim Homme wrote:
> we have three proposals here:
> 
>   $%<hex>
>   ${hex:<hex>}
> 
> and a slightly more nebulous "a more unique prefix", perhaps this
> qualifies:
> 
>   $[hex:<hex>]
> 
> that would be fine with me, in any case.

I like ${hex:<hex>}, because it looks like a function and we can use the
same syntax for other string expansions, too.

> > >I want to note that a syntax mix-up can be flagged at upload time, so I
> > >don't think it's a big problem in practice.
> 
> this is not entirely correct, a typo like ${hex.ff} can only be flagged
> if "variables" is in use, otherwise it will be used verbatim.

This particular typo will indeed be used verbatim, whereas ${hex:fg} can
be recognised.

Michael

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-sieve-notify-mailto-01.txt 
Date: Wed, 18 Oct 2006 15:50:03 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Sieve Mail Filtering Language Working Group of the IETF.

	Title		: Sieve Notification Mechanism: mailto
	Author(s)	: B. Leiba, M. Haardt
	Filename	: draft-ietf-sieve-notify-mailto-01.txt
	Pages		: 14
	Date		: 2006-10-18
	
This document describes a profile of the Sieve extension for
   notifications, to allow notifications to be sent by electronic mail.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-sieve-notify-mailto-01.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of 
the message. 
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the 
username "anonymous" and a password of your e-mail address. After 
logging in, type "cd internet-drafts" and then 
"get draft-ietf-sieve-notify-mailto-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-sieve-notify-mailto-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-sieve-notify-mailto-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-sieve-notify-mailto-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Wed, 18 Oct 2006 18:54:25 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>On Fri, 2006-10-06 at 18:01 +0100, Alexey Melnikov wrote:
>  
>
>>Kjetil Torgrim Homme wrote:
>>    
>>
>>>On Fri, 2006-10-06 at 15:27 +0100, Alexey Melnikov wrote:
>>>      
>>>
>>>>I would prefer if we pick a more unique prefix. Something starting with 
>>>>'$' but not followed by '{' would be great. However if others feel 
>>>>strongly in favor of your variant, that would be fine too. Apart from 
>>>>that your proposal is fine with me.
>>>>        
>>>>
>>>glad to hear that.  yes, the resemblance to variables syntax is a mixed
>>>blessing.
>>>
>>>pro: it's easier to recognise magic sequences in the string.
>>>con: it's easier to mix up the two syntaxes.
>>>      
>>>
>>I think cons. outweigh pros. in this case. Somebody can forget to use 
>>':' after hex, etc.
>>    
>>
>we have three proposals here:
>
>  $%<hex>
>  ${hex:<hex>}
>
>and a slightly more nebulous "a more unique prefix", perhaps this
>qualifies:
>
>  $[hex:<hex>]
>
>that would be fine with me, in any case.
>  
>
I like the last one, where <hex> can be a sequence of space separated 
hex pairs.

>>>I want to note that a syntax mix-up can be flagged at upload time, so I
>>>don't think it's a big problem in practice.
>>>      
>>>
>
>this is not entirely correct, a typo like ${hex.ff} can only be flagged
>if "variables" is in use, otherwise it will be used verbatim.
>  
>
>>>this was a bit lazy editting on my part.  I made the argument for the
>>>reinstating of status quo in a different thread, so please ignore the
>>>removal here.
>>>      
>>>
>>So I will argue in another thread ;-).
>>    
>>
>
>you haven't done so yet :-)
>
No, I haven't. Sorry. I will post a message on this shortly.

>it may have been lost in the long thread,
>I'll repost my suggestion in a new thread.
>  
>
No need.

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: [email-masked]
Date: Wed, 18 Oct 2006 19:49:55 +0200
Sender: [email-masked]

On Fri, 2006-10-06 at 18:01 +0100, Alexey Melnikov wrote:
> Kjetil Torgrim Homme wrote:
> >On Fri, 2006-10-06 at 15:27 +0100, Alexey Melnikov wrote:
> >>I would prefer if we pick a more unique prefix. Something starting with 
> >>'$' but not followed by '{' would be great. However if others feel 
> >>strongly in favor of your variant, that would be fine too. Apart from 
> >>that your proposal is fine with me.
> >
> >glad to hear that.  yes, the resemblance to variables syntax is a mixed
> >blessing.
> >
> >pro: it's easier to recognise magic sequences in the string.
> >con: it's easier to mix up the two syntaxes.
>
> I think cons. outweigh pros. in this case. Somebody can forget to use 
> ':' after hex, etc.

we have three proposals here:

  $%<hex>
  ${hex:<hex>}

and a slightly more nebulous "a more unique prefix", perhaps this
qualifies:

  $[hex:<hex>]

that would be fine with me, in any case.

> >I want to note that a syntax mix-up can be flagged at upload time, so I
> >don't think it's a big problem in practice.

this is not entirely correct, a typo like ${hex.ff} can only be flagged
if "variables" is in use, otherwise it will be used verbatim.

> >this was a bit lazy editting on my part.  I made the argument for the
> >reinstating of status quo in a different thread, so please ignore the
> >removal here.
> 
> So I will argue in another thread ;-).

you haven't done so yet :-)  it may have been lost in the long thread,
I'll repost my suggestion in a new thread.

-- 
Kjetil T.

Date: Wed, 18 Oct 2006 17:58:55 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Last Call: 'Sieve: An Email Filtering Language' to Proposed Standard (draft-ietf-sieve-3028bis)
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>On Wed, 2006-10-18 at 12:28 -0400, The IESG wrote:
>  
>
>>The IESG has received a request from the Sieve Mail Filtering Language 
>>WG to consider the following document:
>>
>>- 'Sieve: An Email Filtering Language '
>>   <draft-ietf-sieve-3028bis-09.txt> as a Proposed Standard
>>
>>The IESG plans to make a decision in the next few weeks, and solicits
>>final comments on this action.  Please send any comments to the
>>[email-masked] or [email-masked] mailing lists by 2006-11-01.
>>    
>>
>ugh.
>
>I think the Sieve WG should have been notified prior to sending it to
>the IESG for IETF wide last call.
>
>we still don't have closure on non-ASCII characters.  there have been
>several suggestions for edits posted since -09 was submitted, and it
>should NOT be accepted in its current form without further discussion.
>  
>
Yes, I told Lisa that this is the case. -09 is not going to IESG, -10 or 
later will.

Subject: Re: Last Call: 'Sieve: An Email Filtering Language' to Proposed    Standard (draft-ietf-sieve-3028bis)
From: Kjetil Torgrim Homme <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: Wed, 18 Oct 2006 18:46:44 +0200
Sender: [email-masked]

On Wed, 2006-10-18 at 12:28 -0400, The IESG wrote:
> The IESG has received a request from the Sieve Mail Filtering Language 
> WG to consider the following document:
> 
> - 'Sieve: An Email Filtering Language '
>    <draft-ietf-sieve-3028bis-09.txt> as a Proposed Standard
> 
> The IESG plans to make a decision in the next few weeks, and solicits
> final comments on this action.  Please send any comments to the
> [email-masked] or [email-masked] mailing lists by 2006-11-01.

ugh.

I think the Sieve WG should have been notified prior to sending it to
the IESG for IETF wide last call.

we still don't have closure on non-ASCII characters.  there have been
several suggestions for edits posted since -09 was submitted, and it
should NOT be accepted in its current form without further discussion.

-- 
Kjetil T.

To: IETF-Announce <[email-masked]>
From: The IESG <[email-masked]>
Subject: Last Call: 'Sieve: An Email Filtering Language' to Proposed  Standard (draft-ietf-sieve-3028bis) 
Reply-To: [email-masked]
Cc: <[email-masked]>
Date: Wed, 18 Oct 2006 12:28:12 -0400
Sender: [email-masked]

The IESG has received a request from the Sieve Mail Filtering Language 
WG to consider the following document:

- 'Sieve: An Email Filtering Language '
   <draft-ietf-sieve-3028bis-09.txt> as a Proposed Standard

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action.  Please send any comments to the
[email-masked] or [email-masked] mailing lists by 2006-11-01.

The file can be obtained via
http://www.ietf.org/internet-drafts/draft-ietf-sieve-3028bis-09.txt

Date: Mon, 16 Oct 2006 12:14:59 +0100
From: Alexey Melnikov <[email-masked]>
To: MTA filtering mailing list <[email-masked]>
Subject: Sieve meeting at San Diego IETF
Sender: [email-masked]

Hi folks,
We have a Sieve session allocated during San Diego IETF:

SIEVE Session: 2 hours
Monday, November 6, Afternoon Session II 1520-1720

I would like to ask authors of remaining WG documents to report on progress directly to me and Cyrus. Remember that October 23rd (next Monday) is the submission deadline.

Now is also a good time to talk about the meeting agenda. Suggestions?

Alexey,
Sieve co-chair.

Date: Fri, 06 Oct 2006 19:52:38 +0100
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Subject: Re: the part argument to "date"
Sender: [email-masked]

Ned Freed wrote:

>>On Wed, 2006-10-04 at 16:08 -0700, Ned Freed wrote:
>>    
>>
>>>The main reason I chose to make this a positional argument is that tagged
>>>arguments are properly optional and I don't think the type of date test should
>>>be allowed to be optional. This is quite unlike the sitations with other tagged
>>>arguments - they have sensible defaults.
>>>      
>>>
>>my suggested default is "iso8601" which seems useful to me, at least for
>>a relational comparator.  but if you don't like it, by all means make
>>PART required.  (I should've called it DATE-PART, btw.)
>>    
>>
I prefer to be explicit about the part type.

>I like having a required tagged argument (AFAIK that would be a first in sieve)
>even less than I like giving part a default value.
>
>In any case, unless someone else chimes in with an opinion on this I'm sticking
>with a positional argument.
>  
>
I don't feel strongly one way or another.

Cc: Ned Freed <[email-masked]>, Kjetil Torgrim Homme <[email-masked]>, [email-masked]
To: Philip Guenther <[email-masked]>
Date: Fri, 06 Oct 2006 11:34:17 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: the part argument to "date"
Sender: [email-masked]

> On Wed, 4 Oct 2006, Ned Freed wrote:
> ...
> >> I don't quite like the part argument to "date", since the possible
> >> values are listed explicitly.  making each of them a tagged argument is
> >> easier to express in the grammar, and it avoids the problem of
> >> "${format}" which can fail during run-time.  outline of suggested
> >> change:
> >
> > The main reason I chose to make this a positional argument is that
> > tagged arguments are properly optional and I don't think the type of
> > date test should be allowed to be optional.

> Tagged arguments != Optional arguments, despite looking the same.  C.f.
> section 2.6 of RFC 3028.

> > This is quite unlike the sitations with other tagged arguments - they
> > have sensible defaults.

> It's like the :over/:user arguments to the 'size' test...which are tagged
> and not optional.

Forgot about that one. It's one of things I like least in the base sieve
specification, so this strengthens my resolve to keep this as a positional
argument.

				Ned

Cc: Ned Freed <[email-masked]>, [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 06 Oct 2006 11:26:09 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: the part argument to "date"
Sender: [email-masked]

> On Wed, 2006-10-04 at 16:08 -0700, Ned Freed wrote:
> > The main reason I chose to make this a positional argument is that tagged
> > arguments are properly optional and I don't think the type of date test should
> > be allowed to be optional. This is quite unlike the sitations with other tagged
> > arguments - they have sensible defaults.

> my suggested default is "iso8601" which seems useful to me, at least for
> a relational comparator.  but if you don't like it, by all means make
> PART required.  (I should've called it DATE-PART, btw.)

I like having a required tagged argument (AFAIK that would be a first in sieve)
even less than I like giving part a default value.

In any case, unless someone else chimes in with an opinion on this I'm sticking
with a positional argument.

				Ned

Date: Fri, 06 Oct 2006 18:01:17 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>On Fri, 2006-10-06 at 15:27 +0100, Alexey Melnikov wrote:
>  
>
>>Kjetil Torgrim Homme wrote:
>>    
>>
>>>in other words, we can do whatever we like with ${keyword:data}.  I
>>>prefer an extensible syntax over a compact one (Alexey's $%xx
>>>suggestion), so my vote is for ${hex:7e}.  please see suggested patch
>>>below.
>>>      
>>>
>>I would prefer if we pick a more unique prefix. Something starting with 
>>'$' but not followed by '{' would be great. However if others feel 
>>strongly in favor of your variant, that would be fine too. Apart from 
>>that your proposal is fine with me.
>>    
>>
>glad to hear that.  yes, the resemblance to variables syntax is a mixed
>blessing.
>
>pro: it's easier to recognise magic sequences in the string.
>con: it's easier to mix up the two syntaxes.
>  
>
I think cons. outweigh pros. in this case. Somebody can forget to use 
':' after hex, etc.

>I want to note that a syntax mix-up can be flagged at upload time, so I
>don't think it's a big problem in practice.
>  
>
>>>-   As message header data is converted to [UTF-8] for comparison (see
>>>-   section 2.7.2), most strings will use the UTF-8 encoding.  However,
>>>-   implementations MUST accept all strings that match the grammar in
>>>-   section 8.  The ability to use non-UTF-8 encoded strings matches
>>>-   existing practice and has proven to be useful both in tests for
>>>-   invalid data and in arguments containing raw MIME parts for extension
>>>-   actions that generate outgoing messages.
>>>+   The extension "quoted-character" may be used to encode arbitrary
>>>+   characters as a sequence of US-ASCII characters (see 2.4.2.4 for
>>>+   details).
>>>
>>>   For entering larger amounts of text, such as an email message, a
>>>   multi-line form is allowed.  It starts with the keyword "text:",
>>>      
>>>
>>I am against this change, as it doesn't agree with the rough consensus 
>>in the group, which is to try keep existing implementations compliant.
>>    
>>
>
>this was a bit lazy editting on my part.  I made the argument for the
>reinstating of status quo in a different thread, so please ignore the
>removal here.
>  
>
So I will argue in another thread ;-).

>>>+   quoted-arb-octets   = "${hex:" hex-pair-seq "}"
>>>+   hex-pair-seq        = hex-pair *(WSP hex-pair)
>>>+   hex-pair            = 1*2HEXDIG
>>>      
>>>
>>Did you really want to allow for
>>${hex: 7 8 9}
>>?
>>    
>>
>
>not sure what you're pointing at here ...
>
>a) yes, it may be a good idea to add leading and trailing WSP* in
>quoted-arb-octets to allow arbitrary extra whitespace.
>  
>
This would be fine with me, but I was pointing out at single character 
hex-pair.

>b) yes, it may be more readable to specify hex-pair as 2HEXDIG.  I made
>it 1*2HEXDIG since the Unicode is 1*5HEXDIG, so I suggest we change the
>latter to 2*5 if the former is 2HEXDIG.
>  
>
I would rather use 2HEXDIG for octets and 1*5HEXDIG for Unicode, but 
2*5HEXDIG is fine as well.

>so I'm game whatever you say :-)
>  
>

Date: Fri, 6 Oct 2006 17:05:06 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Fri, Oct 06, 2006 at 03:27:25PM +0100, Alexey Melnikov wrote:
> I would prefer if we pick a more unique prefix. Something starting with 
> '$' but not followed by '{' would be great. However if others feel 
> strongly in favor of your variant, that would be fine too. Apart from 
> that your proposal is fine with me.

Variables can be seen as functions without arguments.  I don't think
of the suggested extension it as quoting, but as function evaluation of
constant arguments.  Add variables, and you have functions of variable
arguments.

That's why I would like to use the syntax for argument-less functions,
aka variables, for functions with arguments as well. :)

Michael

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: Philip Guenther <[email-masked]>, Michael Haardt <[email-masked]>, [email-masked]
Date: Fri, 06 Oct 2006 16:57:09 +0200
Sender: [email-masked]

On Fri, 2006-10-06 at 15:27 +0100, Alexey Melnikov wrote:
> Kjetil Torgrim Homme wrote:
> >in other words, we can do whatever we like with ${keyword:data}.  I
> >prefer an extensible syntax over a compact one (Alexey's $%xx
> >suggestion), so my vote is for ${hex:7e}.  please see suggested patch
> >below.
>
> I would prefer if we pick a more unique prefix. Something starting with 
> '$' but not followed by '{' would be great. However if others feel 
> strongly in favor of your variant, that would be fine too. Apart from 
> that your proposal is fine with me.

glad to hear that.  yes, the resemblance to variables syntax is a mixed
blessing.

pro: it's easier to recognise magic sequences in the string.
con: it's easier to mix up the two syntaxes.

I want to note that a syntax mix-up can be flagged at upload time, so I
don't think it's a big problem in practice.

> >-   As message header data is converted to [UTF-8] for comparison (see
> >-   section 2.7.2), most strings will use the UTF-8 encoding.  However,
> >-   implementations MUST accept all strings that match the grammar in
> >-   section 8.  The ability to use non-UTF-8 encoded strings matches
> >-   existing practice and has proven to be useful both in tests for
> >-   invalid data and in arguments containing raw MIME parts for extension
> >-   actions that generate outgoing messages.
> >+   The extension "quoted-character" may be used to encode arbitrary
> >+   characters as a sequence of US-ASCII characters (see 2.4.2.4 for
> >+   details).
> > 
> >    For entering larger amounts of text, such as an email message, a
> >    multi-line form is allowed.  It starts with the keyword "text:",
>
> I am against this change, as it doesn't agree with the rough consensus 
> in the group, which is to try keep existing implementations compliant.

this was a bit lazy editting on my part.  I made the argument for the
reinstating of status quo in a different thread, so please ignore the
removal here.

> >+   quoted-arb-octets   = "${hex:" hex-pair-seq "}"
> >+   hex-pair-seq        = hex-pair *(WSP hex-pair)
> >+   hex-pair            = 1*2HEXDIG
> >  
> >
> Did you really want to allow for
> ${hex: 7 8 9}
> ?

not sure what you're pointing at here ...

a) yes, it may be a good idea to add leading and trailing WSP* in
quoted-arb-octets to allow arbitrary extra whitespace.

b) yes, it may be more readable to specify hex-pair as 2HEXDIG.  I made
it 1*2HEXDIG since the Unicode is 1*5HEXDIG, so I suggest we change the
latter to 2*5 if the former is 2HEXDIG.

so I'm game whatever you say :-)
-- 
Kjetil T.

Date: Fri, 06 Oct 2006 15:27:25 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: Philip Guenther <[email-masked]>, Michael Haardt <[email-masked]>, [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>On Sun, 2006-10-01 at 16:04 +0200, Kjetil Torgrim Homme wrote:
>  
>
>>variables don't allow a reference which acts as a function with
>>arbitrary input (e.g., "${hex:e6 f8 e5}"), the tail end has to be an
>>identifier or numbered variable.  unfortunately, this means ${hex:7e} is
>>disallowed, since "7e" is neither.
>>    
>>
>
>I'm sorry, I was very confused.  the "variables" syntax uses period, not
>colon, to separate namespaces, so the suggested syntax would _resemble_
>it, but be completely independent of it ("variables" only affects
>substrings which match the specified syntax exactly, so there's
>definitely no conflict).
>
>in other words, we can do whatever we like with ${keyword:data}.  I
>prefer an extensible syntax over a compact one (Alexey's $%xx
>suggestion), so my vote is for ${hex:7e}.  please see suggested patch
>below.
>  
>
I would prefer if we pick a more unique prefix. Something starting with 
'$' but not followed by '{' would be great. However if others feel 
strongly in favor of your variant, that would be fine too. Apart from 
that your proposal is fine with me.

>a couple of notes:
>
>a) the extension name, "quoted-character", is off the top of my head.
>feel free to use a different one if you prefer ("encode-char",
>perhaps?).
>  
>
"encode-char" is slightly better, IMHO. quoted-* has strong association 
with quoted strings.

>b) the syntax requires spaces between the items.  it's possible to allow
>${hex:ABCD} if we use this syntax instead:
>
>   quoted-arb-octets   = "${hex:" hex-pair-seq "}"
>   hex-pair-seq        = hex-pair *(*WSP hex-pair)
>   hex-pair            = 2HEXDIG
>
>on the other hand, some Unicode code points may need five hex digits
>(such as U+1D10A, which is ð„Š, the musical symbol "da capo"), but these
>are quite rare, so most people will probably want to write just four
>digits (e.g. U+4e2d U+56fd, which is ä¸­å›½, the name of China).  this
>means a sequence of Unicode characters can't be written unambiguosly and
>conveniently as one long string of hex digits.  therefore, to make them
>consistent, both encodings require the values to be split by whitespace.
>I think this improves readability, anyway.
>
>c) it may be presumptious of me to add this extension to the "SHOULD be
>implemented" list, I won't be offended if it's listed as "MAY" ;-)
>  
>
Speaking personally, SHOULD is fine with me.

>d) I'm no expert on ABNF, so please review.
>  
>
====

>--- draft-ietf-sieve-3028bis-09.txt	2006-10-06 01:47:33.989869000 +0200
>+++ draft-ietf-sieve-3028bis-kjetilho.txt	2006-10-06 03:22:35.025194000 +0200
>@@ -385,13 +385,9 @@
>    are permitted in quoted strings.  Quoted strings MAY span multiple
>    lines.  NUL (US-ASCII 0) is not allowed in strings.
> 
>-   As message header data is converted to [UTF-8] for comparison (see
>-   section 2.7.2), most strings will use the UTF-8 encoding.  However,
>-   implementations MUST accept all strings that match the grammar in
>-   section 8.  The ability to use non-UTF-8 encoded strings matches
>-   existing practice and has proven to be useful both in tests for
>-   invalid data and in arguments containing raw MIME parts for extension
>-   actions that generate outgoing messages.
>+   The extension "quoted-character" may be used to encode arbitrary
>+   characters as a sequence of US-ASCII characters (see 2.4.2.4 for
>+   details).
> 
>    For entering larger amounts of text, such as an email message, a
>    multi-line form is allowed.  It starts with the keyword "text:",
>  
>
I am against this change, as it doesn't agree with the rough consensus 
in the group, which is to try keep existing implementations compliant.

However your change is fine, as long as the original text is not deleted.

>@@ -470,6 +466,42 @@
>    valid, but need not ensure that they actually identify an email
>    recipient.
> 
>+2.4.2.4. Encoding characters using "quoted-character"
>+
>+   When the "quoted-character" extension is in effect, certain
>+   character sequences in strings are replaced by the unencoded value.
>+   This happens after escape sequences are interpreted and
>+   dot-unstuffing has been done.
>+
>+   Arbitrary octets can be embedded in strings by using the syntax
>+   quoted-arb-octets.  The sequence is replaced by the octets with the
>+   hexadecimal values given by each hex-pair.
>+
>+   quoted-arb-octets   = "${hex:" hex-pair-seq "}"
>+   hex-pair-seq        = hex-pair *(WSP hex-pair)
>+   hex-pair            = 1*2HEXDIG
>  
>
Did you really want to allow for
${hex: 7 8 9}
?

The rest looks fine to me.

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
Date: Fri, 06 Oct 2006 03:24:37 +0200
Sender: [email-masked]

--=-O5vkLiE+eGNxPWUCCZ8B
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Sun, 2006-10-01 at 16:04 +0200, Kjetil Torgrim Homme wrote:
> variables don't allow a reference which acts as a function with
> arbitrary input (e.g., "${hex:e6 f8 e5}"), the tail end has to be an
> identifier or numbered variable.  unfortunately, this means ${hex:7e} is
> disallowed, since "7e" is neither.

I'm sorry, I was very confused.  the "variables" syntax uses period, not
colon, to separate namespaces, so the suggested syntax would _resemble_
it, but be completely independent of it ("variables" only affects
substrings which match the specified syntax exactly, so there's
definitely no conflict).

in other words, we can do whatever we like with ${keyword:data}.  I
prefer an extensible syntax over a compact one (Alexey's $%xx
suggestion), so my vote is for ${hex:7e}.  please see suggested patch
below.

a couple of notes:

a) the extension name, "quoted-character", is off the top of my head.
feel free to use a different one if you prefer ("encode-char",
perhaps?).

b) the syntax requires spaces between the items.  it's possible to allow
${hex:ABCD} if we use this syntax instead:

   quoted-arb-octets   =3D "${hex:" hex-pair-seq "}"
   hex-pair-seq        =3D hex-pair *(*WSP hex-pair)
   hex-pair            =3D 2HEXDIG

on the other hand, some Unicode code points may need five hex digits
(such as U+1D10A, which is =F0=9D=84=8A, the musical symbol "da capo"), but=
 these
are quite rare, so most people will probably want to write just four
digits (e.g. U+4e2d U+56fd, which is =E4=B8=AD=E5=9B=BD, the name of China)=
.  this
means a sequence of Unicode characters can't be written unambiguosly and
conveniently as one long string of hex digits.  therefore, to make them
consistent, both encodings require the values to be split by whitespace.
I think this improves readability, anyway.

c) it may be presumptious of me to add this extension to the "SHOULD be
implemented" list, I won't be offended if it's listed as "MAY" ;-)

d) I'm no expert on ABNF, so please review.

--=20
Kjetil T.

--=-O5vkLiE+eGNxPWUCCZ8B
Content-Description: 
Content-Disposition: inline; filename=quoted-character.diff
Content-Type: text/x-patch; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

--- draft-ietf-sieve-3028bis-09.txt	2006-10-06 01:47:33.989869000 +0200
+++ draft-ietf-sieve-3028bis-kjetilho.txt	2006-10-06 03:22:35.025194000 +0200
@@ -385,13 +385,9 @@
    are permitted in quoted strings.  Quoted strings MAY span multiple
    lines.  NUL (US-ASCII 0) is not allowed in strings.
 
-   As message header data is converted to [UTF-8] for comparison (see
-   section 2.7.2), most strings will use the UTF-8 encoding.  However,
-   implementations MUST accept all strings that match the grammar in
-   section 8.  The ability to use non-UTF-8 encoded strings matches
-   existing practice and has proven to be useful both in tests for
-   invalid data and in arguments containing raw MIME parts for extension
-   actions that generate outgoing messages.
+   The extension "quoted-character" may be used to encode arbitrary
+   characters as a sequence of US-ASCII characters (see 2.4.2.4 for
+   details).
 
    For entering larger amounts of text, such as an email message, a
    multi-line form is allowed.  It starts with the keyword "text:",
@@ -470,6 +466,42 @@
    valid, but need not ensure that they actually identify an email
    recipient.
 
+2.4.2.4. Encoding characters using "quoted-character"
+
+   When the "quoted-character" extension is in effect, certain
+   character sequences in strings are replaced by the unencoded value.
+   This happens after escape sequences are interpreted and
+   dot-unstuffing has been done.
+
+   Arbitrary octets can be embedded in strings by using the syntax
+   quoted-arb-octets.  The sequence is replaced by the octets with the
+   hexadecimal values given by each hex-pair.
+
+   quoted-arb-octets   = "${hex:" hex-pair-seq "}"
+   hex-pair-seq        = hex-pair *(WSP hex-pair)
+   hex-pair            = 1*2HEXDIG
+
+   It may be inconvenient or undesirable to enter Unicode characters
+   verbatim, and in these cases the syntax quoted-unicode-char can be
+   used. The sequence is replaced by the UTF-8 encoding of the
+   specified Unicode characters, which are identified by the
+   hexadecimal value of unicode-hex.
+
+   quoted-unicode-char = "${unicode:" unicode-hex-seq "}"
+   unicode-hex-seq     = unicode-hex *(WSP unicode-hex)
+   unicode-hex         = 1*5HEXDIG
+
+   The capability string for use with the require command is
+   "quoted-character".
+
+   In the following script, message A is discarded, since the
+   specified test string is equivalent to "$$$".
+
+   Example:   require "quoted-character";
+              if header :contains "Subject" "$${hex:24 24}" {
+                    discard;
+              }
+
 2.5.     Tests
 
    Tests are given as arguments to commands in order to control their
@@ -1075,7 +1107,7 @@
    Implementations MUST support the "keep", "discard", and "redirect"
    actions.
 
-   Implementations SHOULD support "fileinto".
+   Implementations SHOULD support "fileinto" and "quoted-character".
 
    Implementations MAY limit the number of certain actions taken (see
    section 2.10.4).
@@ -1561,6 +1593,12 @@
    RFC number:      this RFC (Sieve base spec)
    Contact address: The Sieve discussion list <[email-masked]>
 
+   Capability name: quoted-character
+   Description:     changes the parsing of strings to allow arbitrary
+                    characters to be embedded
+   RFC number:      this RFC (Sieve base spec)
+   Contact address: The Sieve discussion list <[email-masked]>
+
    Capability name: comparator-* (anything starting with "comparator-")
    Description:     adds the indicated comparator for use with the
                     :comparator argument

--=-O5vkLiE+eGNxPWUCCZ8B--

Subject: Re: non-UTF-8 sequences in Sieve scripts
From: Kjetil Torgrim Homme <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
Date: Fri, 06 Oct 2006 02:36:16 +0200
Sender: [email-masked]

On Mon, 2006-09-18 at 11:07 -0400, Cyrus Daboo wrote:
> I cannot see any restriction that would prevent use of 
> Content-Transfer-Encoding: 8-bit, in which case the MIME parts are not 
> necessarily US-ASCII and not necessarily UTF-8, e.g. a text/plain part with 
> charset=iso8859-1.

in my opinion, it is quite clear that the intent of RFC 3028 is that a
Sieve script is pure Unicode, it is actually quite explicit about it:

[RFC 3028: 2.1. Form of the Language]:
|   [...]
|   The language is represented in UTF-8, as specified in [UTF-8].

[RFC 3028: 8.1. Lexical Tokens]:
|
|   Sieve scripts are encoded in UTF-8.  The following assumes a valid
|   UTF-8 encoding; special characters in Sieve scripts are all ASCII.

it then goes on to define the grammar, but it takes a shortcut and says:

|   CHAR-NOT-STAR = (%x00-51 / %x53-ff)
|   quoted-string = DQUOTE *CHAR DQUOTE

etc., which may seem to allow arbitrary octets, but it is constrained by
the introductory paragraph.

the embedded MIME part is no different from the rest of the script, and
needs to conform to this.  it shouldn't be necessary to state the
restriction explicitly.  these kinds of restrictions are very familiar
in the context of MIME, and several options are available to make a MIME
part which conforms to a UTF-8 transport.

okay, fine, we don't have to make 3028bis more explicit than 3028 about
disallowing arbitrary octets.  but we should definitely not add explicit
text which allows it either!  let's stick to the original wording, but
keep some of the clarifications from the draft:

[3028bis-09: 2.1. Form of the Language (§2)]:
|   With the exceptions of strings and comments, the language is limited
|   to US-ASCII characters.  Strings and comments may contain octets
|   outside the US-ASCII range.  Specifically, they will normally be in
|   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
|   in scripts, while CR and LF can only appear as the CRLF line ending.

[my suggestion]:
|   With the exceptions of strings and comments, the language is limited
|   to US-ASCII characters.  Strings and comments are encoded in
|   UTF-8, as specified in [UTF-8].  NUL (US-ASCII 0) is never permitted
|   in scripts, while CR and LF can only appear as the CRLF line ending.

[3028bis-09: 2.4.2. Strings (§6)]:
|   As message header data is converted to [UTF-8] for comparison (see
|   section 2.7.2), most strings will use the UTF-8 encoding.  However,
|   implementations MUST accept all strings that match the grammar in
|   section 8.  The ability to use non-UTF-8 encoded strings matches
|   existing practice and has proven to be useful both in tests for
|   invalid data and in arguments containing raw MIME parts for extension
|   actions that generate outgoing messages.

[my suggestion]:
|   [strike paragraph 6 in its entirety]

the text from 8.1 is unchanged in 3028bis-09, so this is all we need to
maintain status quo.  we'll also keep the accurate UTF-8 definitions of
characters out of the ABNF, but may decide to change that later, in the
Standard revision of the document.

I hope this proposal is agreeable to all concerned.
-- 
Kjetil T.

Date: Thu, 05 Oct 2006 16:58:47 +0100
From: Alexey Melnikov <[email-masked]>
To: Kjetil Torgrim Homme <[email-masked]>
CC: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

Kjetil Torgrim Homme wrote:

>I would still like to see a escape mechanism which also allows the entry
>of Unicode (in UTF-8) in a pure US-ASCII script, I only mention this
>since $%xx isn't really flexible enough for this -- the user shouldn't
>have to do the UTF-8 transformation themselves.
>  
>
The current proposal ($%xx) is sufficient to address the issue in the 
base spec in order to be able to send it to IESG.
If you want to add Unicode, you need to suggest some text now.

Alexey,
Sieve co-chair

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Michael Haardt <[email-masked]>
Cc: [email-masked]
Date: Thu, 05 Oct 2006 11:16:31 +0200
Sender: [email-masked]

On Mon, 2006-10-02 at 13:39 +0200, Michael Haardt wrote:
> On Sun, Sep 24, 2006 at 10:06:44PM +0100, Alexey Melnikov wrote:
> > After thinking more about this (and reviewing list of options provided 
> > by Philip), I would like to propose a variables-like syntax close to 
> > what you suggested above:
> > $%<hex>
> > controlled by a new Sieve capability "string-escape".
> > This would give existing implementations an ability to provide the 
> > escaping functionality in both quoted and multiline strings, without the 
> > need to implement variables.
> > Any objections?
> 
> Just curious: What speaks against using the syntax of variables?  It is
> certainly a MUST that the new extension does not require variables,
> but that does not forbid to use their syntax.
> 
> $%xx$%xx is OK for me, but ${hex:xxxx} is easier to read and we don't
> get in trouble, should someone really miss ${frombase64:xxxx} and add it.

as noted in my message on Sunday, you can't make ${frombase64:xxxx}.
even quoted-printable is a stretch, as I demonstrated (_jokingly_ I
hasten to add, although perhaps that wasn't clear).

I would still like to see a escape mechanism which also allows the entry
of Unicode (in UTF-8) in a pure US-ASCII script, I only mention this
since $%xx isn't really flexible enough for this -- the user shouldn't
have to do the UTF-8 transformation themselves.
-- 
Kjetil T.

Subject: Re: the part argument to "date"
From: Kjetil Torgrim Homme <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: [email-masked]
Date: Thu, 05 Oct 2006 11:09:50 +0200
Sender: [email-masked]

On Wed, 2006-10-04 at 16:08 -0700, Ned Freed wrote:
> The main reason I chose to make this a positional argument is that tagged
> arguments are properly optional and I don't think the type of date test should
> be allowed to be optional. This is quite unlike the sitations with other tagged
> arguments - they have sensible defaults.

my suggested default is "iso8601" which seems useful to me, at least for
a relational comparator.  but if you don't like it, by all means make
PART required.  (I should've called it DATE-PART, btw.)

-- 
Kjetil T.

Date: Wed, 4 Oct 2006 18:22:58 -0600
From: Philip Guenther <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Subject: Re: the part argument to "date"
Sender: [email-masked]

On Wed, 4 Oct 2006, Ned Freed wrote:
...
>> I don't quite like the part argument to "date", since the possible
>> values are listed explicitly.  making each of them a tagged argument is
>> easier to express in the grammar, and it avoids the problem of
>> "${format}" which can fail during run-time.  outline of suggested
>> change:
>
> The main reason I chose to make this a positional argument is that 
> tagged arguments are properly optional and I don't think the type of 
> date test should be allowed to be optional.

Tagged arguments != Optional arguments, despite looking the same.  C.f. 
section 2.6 of RFC 3028.

> This is quite unlike the sitations with other tagged arguments - they 
> have sensible defaults.

It's like the :over/:user arguments to the 'size' test...which are tagged 
and not optional.

Philip Guenther

Cc: [email-masked]
To: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 04 Oct 2006 16:08:50 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: the part argument to "date"
Sender: [email-masked]

(sorry for the delay getting back to this)

> I don't quite like the part argument to "date", since the possible
> values are listed explicitly.  making each of them a tagged argument is
> easier to express in the grammar, and it avoids the problem of
> "${format}" which can fail during run-time.  outline of suggested
> change:

The main reason I chose to make this a positional argument is that tagged
arguments are properly optional and I don't think the type of date test should
be allowed to be optional. This is quite unlike the sitations with other tagged
arguments - they have sensible defaults.

> 4.  Date Test

>    Usage:   date [":zone" <time-zone: string>] [COMPARATOR]
>                  [MATCH-TYPE] [PART] <header-name: string>
>                  <key-list: string-list>
> [...]

> 4.2.  Part argument

>    The "PART" syntax element is defined as follows:

>    Syntax:   ":year" / ":month" / ":day" [...]

>    The optional part argument specifies a particular part of the
>    resulting date/time value to match against the key-list.  If no part
>    argument is given, ":iso8601" is used.  The available parts are:

>    [description of each part suitably edited]

> Ned, I can write a more specific patch if you like, if you send me your
> source.

I'd like to hear from anyone else with an opinion on this. If there's
a consensus to make the change I have no problem, but if its just you any
me who care I'd prefer to stick with my approach.

> (BTW, I changed "parameter" to "argument" since that's what the base
> spec calls them.)

Noted. I'll change things accordingly.

					Ned

Date: Mon, 2 Oct 2006 13:39:47 +0200
From: Michael Haardt <[email-masked]>
To: [email-masked]
Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
Sender: [email-masked]

On Sun, Sep 24, 2006 at 10:06:44PM +0100, Alexey Melnikov wrote:
> After thinking more about this (and reviewing list of options provided 
> by Philip), I would like to propose a variables-like syntax close to 
> what you suggested above:
> $%<hex>
> controlled by a new Sieve capability "string-escape".
> This would give existing implementations an ability to provide the 
> escaping functionality in both quoted and multiline strings, without the 
> need to implement variables.
> Any objections?

Just curious: What speaks against using the syntax of variables?  It is
certainly a MUST that the new extension does not require variables,
but that does not forbid to use their syntax.

$%xx$%xx is OK for me, but ${hex:xxxx} is easier to read and we don't
get in trouble, should someone really miss ${frombase64:xxxx} and add it.

Michael

Subject: Re: Proposal for escaping on non-UTF-8 sequences in Sieve
From: Kjetil Torgrim Homme <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: Michael Haardt <[email-masked]>, [email-masked]
Date: Sun, 01 Oct 2006 16:04:09 +0200
Sender: [email-masked]

On Mon, 2006-09-18 at 15:12 -0600, Philip Guenther wrote:
> On Mon, 18 Sep 2006, Michael Haardt wrote:
> > On Mon, Sep 18, 2006 at 09:44:20AM -0700, Ned Freed wrote:
> >> There are all sorts of ways to do it. Here's an obvious one: If the
> >> octet-value extension is enabled, any occurances of ${x} where x takes
> >> the form of a space-separated list of decimal values is replaced with
> >> a sequence of octets corresponding to each value.
> >
> > What a brilliant idea! How about using the same syntax variables do,
> > and embedding arbitrary data written as a function?
> 
> You mean something like option (3) from my message of March 2005?
>     http://www.imc.org/ietf-mta-filters/mail-archive/msg02018.html

(sorry about responding so late to the discussion!)

the difference is that your suggestion added a dependency on variables.
the new idea is that we're using a syntax which is compatible with
variables, but can be used and implemented on its own.

some issues which would need ironing out:

variables don't allow a reference which acts as a function with
arbitrary input (e.g., "${hex:e6 f8 e5}"), the tail end has to be an
identifier or numbered variable.  unfortunately, this means ${hex:7e} is
disallowed, since "7e" is neither.  to solve this, you'd either need to
abandon hex and go for decimal values, or to introduce a prefix
character in addition to the namespace: ${hex:x7e}.  if we choose the
latter, we could consider using one namespace for several encodings,
e.g., ${data:x7e}, ${data:u007e} and, if we want to stretch it
maximally, ${data:q_53ort__of__q_2dp}

-- 
Kjetil T.