Subject: The last remaining issue in the Sieve variables draft

From: Alexey Melnikov <[email-masked]>
Date: Sun, 24 Apr 2005 15:07:40 +0100

Hi group,

I would like to solicit some comments on the last remaining issue in the 
Sieve variables draft.
The latest version of the document prohibits all numeric second (and 
lower) level namespaces, e.g. a variable name "foo.1.3" is not legal.
My personal opinion (chair hat off) is that we should allow for such 
variable names, just in case we later on decide to allow variable names 
based on MIME part numbering, or something similar. Does anybody have 
any opinion in support or against this change?

Alexey

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 24 Apr 2005 19:34:16 +0200

Alexey Melnikov writes:
> Does anybody have any opinion in support or against this change?

It seems reasonable to forbid them now, but add a note to implementers 
that a later extension may change this to allow them, and mention part 
numbers as possible motivation.

Arnt

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 24 Apr 2005 19:48:30 +0200

> any opinion in support or against this change?

sorry to go off on a tangent, but going over the text again, I realised
that the draft has no actual connection between the term "numbered
variable" and the "num-variable" syntax element.  I therefore added the
following to clarify:

3.2.  Numbered variables

   A "numbered variable" has a name consisting only of decimal digits
   and has no namespace component.

another issue is what to do about unknown namespaces.  the current draft
is actually silent, so I'd say this applies:  "Unknown variables are
replaced
by the empty string."  that is bad, IMHO.  I suggest an addition to
section 3:

   Namespaces are meant for future extensions which make internal state
   available through variables.  These variables SHOULD be put in a
   namespace with the same name as its capability string.  Notice that
   the user can not specify a namespace when setting variables with SET.

+  References to namespaces without a prior require statement for the
+  relevant extension MUST cause a syntax error.

the other option is to leave the variable reference unexpanded, the same
way "${foo,bar}" is.  that's fine, too, but I think users will prefer
the error (typo) to be flagged.
-- 
Kjetil T.

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 24 Apr 2005 19:02:28 +0100

> numbers as possible motivation.

Allowing them later will force people to change their parsers, I would 
rather avoid this. Is there any particular reason why you think this 
should be forbidden?

Alexey

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 24 Apr 2005 20:06:45 +0200

> numbers as possible motivation.

well, there are currently no extensions using namespaces, so there's
nothing much to forbid.  this mostly affects the BNF for the variable
reference syntax, and I don't think it's good to restrict the syntax,
and then turn around and say the syntax may be broken by extensions in
the future.  in that case we could just as well left namespaces out of
the specification altogether, IMHO.

here's the BNF I think we need to allow this syntax:

        variable-ref    =       "${" [namespace] variable-name "}"
        namespace       =       identifier "." *sub-namespace
        sub-namespace   =       variable-name "."
        variable-name   =       num-variable / identifier
        num-variable    =       1*DIGIT

this is the current BNF:

        variable-ref    =       "${" *namespace variable-name "}"
        namespace       =       identifier "."
        variable-name   =       num-variable / identifier
        num-variable    =       1*DIGIT
-- 
Kjetil T.

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 24 Apr 2005 19:07:11 +0100

>   and has no namespace component.
Good.

>   the user can not specify a namespace when setting variables with SET.
Which reminded me: can we change the last sentence to:

   Unless explicitly allowed by a Sieve extension, the user can not 
specify a namespace when setting variables with SET.

Once again, I don't like to be too restrictive for no good reason.

>the error (typo) to be flagged.
I would prefer an error.
(Don't you hate those spam messages containing "Dear %s" ;-)?)

Alexey

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 24 Apr 2005 19:09:48 +0100

Kjetil Torgrim Homme wrote:

>nothing much to forbid.
I was about to mention that.

>  this mostly affects the BNF for the variable
>the future.
Exactly my point.

>  in that case we could just as well left namespaces out of
Looks good to me.

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 24 Apr 2005 20:15:41 +0200

> Once again, I don't like to be too restrictive for no good reason.

I'd prefer other extension to add their own action if they need it.  the
way I figure is that most uses of namespaces will be read-only, or with
specialised needs for setting values.  you don't really want

   SET "mimestructure.body.2.1" "Hi there";

do you?
-- 
Kjetil T.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 25 Apr 2005 10:39:15 +0200

Alexey Melnikov writes:
>> It seems reasonable to forbid them now, but add a note to 
>> implementers that a later extension may change this to allow them, 
>> and mention part numbers as possible motivation.
>

Yes - I think that variables linked to bodypart numbers are likely to be 
magic in one way or another. Going from "variable name is illegal" to 
"variable name is magic" is easier than from "variable name is a 
regular variable" to "is magic", since the former cannot change the 
meaning of an existing valid sieve script.

Kjetil posted an example of such magic yesterday.

The reason to add the note is so parser authors will be aware that such 
a change might come, and won't paint themselves into a corner.

Arnt

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 25 Apr 2005 10:38:29 +0100

>    SET "mimestructure.body.2.1" "Hi there";

I would also prefer extensions to explicitly allow specifying a namespace.  And as Kjetil points out I think it very unlikely that we'd agree to any extension that permitted setting of "silly" namespaces, but I think it likely that we'll use namespaces a lot in the future of the language, and prohibiting the possibility of setting variables in namespaces sounds to me like something we'd regret.  You said yourself "most" uses will be read-only.  You didn't say "all" uses will be read only.

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 25 Apr 2005 12:32:55 +0200

> Kjetil posted an example of such magic yesterday.

I added wording which makes references to unknown namespaces a syntax
error yesterday.  is that sufficient for you?
-- 
Kjetil T.

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 25 Apr 2005 11:27:57 +0100

>   SET "mimestructure.body.2.1" "Hi there";
I was thinking more about something like

   SET "annotate.body.2.1" "\\Seen";

(where "annotate" is an extension to set IMAP annotations).

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 25 Apr 2005 12:31:09 +0200

> > > Once again, I don't like to be too restrictive for no good reason.

the Syntax line would have to change, too, to refer to variable-name
rather than identifier.  the above change would make it legal to set
numbered variables.

> I would also prefer extensions to explicitly allow specifying a
> namespace.  And as Kjetil points out I think it very unlikely that
> we'd agree to any extension that permitted setting of "silly"
> namespaces, but I think it likely that we'll use namespaces a lot in
> the future of the language, and prohibiting the possibility of setting
> variables in namespaces sounds to me like something we'd regret.  You
> said yourself "most" uses will be read-only.  You didn't say "all"
> uses will be read only.

okay, but it seems like a lot of hair to add, since we now have to add
in wording about the validity of a namespace and the use of numbered
variables.  do you have a use case?
-- 
Kjetil T.

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 25 Apr 2005 12:01:53 +0100

> variables.  do you have a use case?

What if you have a fairly complex script, and you choose to define your own namespaces to avoid namespace clashes.  Combined with the include capability where you have multiple .siv which each do something different and you don't want all your SET calls to affect other scripts.

Or maybe we'd want to expose spamscore in a namespace, but then provide the ability to override the score from the spam engine with our own score based on some custom whitelist/blacklist/content filtering rule?

Nigel

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 25 Apr 2005 13:08:50 +0200

> error yesterday.  is that sufficient for you?

Sure.

Arnt

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Mon, 25 Apr 2005 18:05:11 +0200

here's an attempt at incorporating this into the draft:

    Namespaces are meant for future extensions which make internal state
    available through variables.  These variables SHOULD be put in a
-   namespace with the same name as its capability string.  Notice that
-   the user can not specify a namespace when setting variables with
SET.
+   namespace whose first component is the same as its capability
string.
+   Such extensions should state whether its namespace is modifiable
with
+   the "set" action.

[...]

 4.  Action set
 
    Syntax:   set [MODIFIER] [COMPARATOR] <name: string> <value: string>
 
    The "set" action stores the specified value in the variable
    identified by name.  The name MUST be a constant string and conform
-   to the syntax of identifier.  An illegal name MUST be detected as a
-   syntax error.
+   to the syntax of variable-name.  Numbered variables can not be set.
+   A namespace can not be used unless an extension explicitly allows
its
+   use in "set".  An invalid name MUST be detected as a syntax error.

-- 
Kjetil T.

---

From: "Nigel Swinson" <[email-masked]>
Date: Mon, 25 Apr 2005 18:23:33 +0100

> +   Such extensions should state whether its namespace is modifiable

Now I'm nervous about an extension which exposes some const and some non 
const variables in a namespace.... :o/  I guess we could expose two 
namespaces which goes against the SHOULD but then you didn't say MUST so 
maybe that's ok...

Should we say:
-   Such extensions should state whether its namespace is modifiable
+   Such extensions SHOULD state whether its namespace is modifiable

> +   use in "set".  An invalid name MUST be detected as a syntax error.

I'm wondering if saying that "Numbered variables can not be set" is made 
redundant by your comments that an extension should state whether its 
namespace is modifiable with the set action.  Certainly it feels quite 
restrictive to have it said in both places.

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 26 Apr 2005 02:36:26 +0200

> +   Such extensions SHOULD state whether its namespace is modifiable

sorry, just a glitch that I didn't write SHOULD, really.

> > +   A namespace can not be used unless an extension explicitly allows its

there's no namespace for numbered variables, so I don't think it is
redundant.
-- 
Kjetil T.

---

From: "Nigel Swinson" <[email-masked]>
Date: Tue, 26 Apr 2005 11:39:38 +0100

> > namespaces which goes against the SHOULD but then you didn't say MUST so 

You didn't comment on the above, so you prefer the idea that if extensions need to expose readonly/writable varaibles, that they expose two namespaces?  I'd really rather they just expose the one.  The extension can then detail which if any varaibles in the namespace are writable.

> there's no namespace for numbered variables, so I don't think it is

You are saying that an extension may expose a namespace, and that namespace may itself contain numbered variables can't it?  So that extension will also detail if it's variables are writable.  Hence why can't a new extension expose a namespace containing writable numbered variables?

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 26 Apr 2005 13:34:39 +0200

On Tue, 2005-04-26 at 11:39 +0100, Nigel Swinson wrote:
> You didn't comment on the above, so you prefer the idea that if
> extensions need to expose readonly/writable varaibles, that they
> expose two namespaces? 

no.

> I'd really rather they just expose the one.  The extension can then
> detail which if any varaibles in the namespace are writable.

exactly.  I don't think the wording restricts this?

> You are saying that an extension may expose a namespace, and that
> namespace may itself contain numbered variables can't it?

no, my current copy includes this definition:

the term "numbered variable" is restricted to the magic match variables.
perhaps I should replace the term with "match variable" -- might be less
confusing?

> So that extension will also detail if it's variables are writable.
> Hence why can't a new extension expose a namespace containing writable
> numbered variables?

they can expose writable variables with all-digit name components.
-- 
Kjetil T.

---

From: "Nigel Swinson" <[email-masked]>
Date: Tue, 26 Apr 2005 13:21:25 +0100

> exactly.  I don't think the wording restricts this?

I guess it doesn't restrict it, but it does discourage it, and I don't think 
it should be discouraged in the way it currently does.  I'd be happier we we 
said:

-   Such extensions SHOULD state whether its namespace is modifiable with
-   the "set" action.
+   Such extensions SHOULD state which if any of the variables in its 
namespace
+  are modifiable with the "set" action.

>   and has no namespace component.

Ok I checked draft-ietf-sieve-variables-02.txt before I posted and didn't 
see a mention about this.  I don't think you should say that.  I don't see 
why numbered varaibles shouldn't have a namespace component.  What if we 
decide to expose the mime structure in variables?  Surely we'll want 
numbered variables?

> perhaps I should replace the term with "match variable" -- might be less

Well I've been thinking of Alexey's example:

I thought he'd used numbered variables with the SET action, so whatever we 
do, we really ought to permit that kind of thing if the annoate extension or 
some other such extension wishes it.

Why can't we just detail that the numbered varaibles made available as a 
side effect of using matches or regex are read only rather than saying that 
all numbered variables can not be set?  I guess if you want to 
introduce/replace with the term "match variables" then that should give the 
clarity and freedom I'm hoping for :o)

> they can expose writable variables with all-digit name components.

Ok good :o)  It's definately seems to be a terminology issue then, cos we 
have num-variable defined in draft-ietf-sieve-variables-02.txt by:

Yet when I hear "Numeric Variables", I think it's a variable-ref with a 
variable-name which is a num-variable.

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 26 Apr 2005 16:42:30 +0200

On Tue, 2005-04-26 at 13:21 +0100, Nigel Swinson wrote:
> +   Such extensions SHOULD state which if any of the variables in its namespace

sounds good, incorporated.

> clarity and freedom I'm hoping for :o)

it seems clear to me that the term "numbered variable" really muddles
the understanding, so I have changed it into "match variable" in my copy
now.  I'll publish this version presently, there has been quite a few
changes lately.
-- 
Kjetil T.