Subject: Variables : list : scope : system : const

From: "Nigel Swinson" <[email-masked]>
Date: Thu, 8 May 2003 18:49:04 +0100

(Ug, I go away for a long weekend and come back to 50 messages from this
list along... I've only just caught up.... I've probably missed some stuff
that I wanted to say, but I'll try to stick it all in one post.  I'm
starting a new thread, cos it's strayed somewhat from just discussing "date
variables")

> this could be solved by making making a new list variable syntax,
> e.g., @{folder} becomes a list, while ${folder} is becomes a single
> string with a single space as delimiter or something.

With all the stuff and points that are getting raised I'm becoming
increasingly convinced that:

1) We should implement list variables (maybe even associative variables),
and implement the imapflags extension using the list variables.  Perhaps one
extension that deals with the syntax, the operators (including operators to
turn it into a string), or perhaps it could be part of the imapflags one...
I'd prefer them separate I think...  If we implement imapflags without list
variables, then get further down the line and find we can't live without
them, then we'll just end up wishing we had them for the imapflags
extension.

2) We should provide a proper scoping mechanism, if that be
scope.variablename then so be it :o)  I'd prefer :: for scope, as that's
certainly much more familiar.  Would that syntactically be allowed?

3) All variables that are ever defined in any RFC extension must be in some
kind of scope.

4) The only "unscoped" variables would be defined by the user and would be
in the "default scope".

5) We don't have any "system variables" or "magic variables", JUST
"variables".  So if we want setdate to produce variables for us to use
elsewhere then they either go out to some associative array, or a set of
variables in an appropriate namespace that can subsequently be written to if
the script author so desires.  I don't want to have to deal with two+
classes of variables :o(

6) If we really want a class of variable that can't be written to then lets
implement const variables.

7) If we have variables that can only get modified by fancy operators like
addflag, removeflag, I'd rather that I couldn't even access them through
${#imapflags}" etc, but instead have an expicit set action somewhere that
put the value into a general purpose, bog standard variable, just like all
my other variables that I could them manipulate in any way I wanted.

BTW another thing I'd like to be able to do is gain access to the spam score
or virus name to use for prepending the subject field.  Perhaps we'd want a
special automatic variable for this too?  Access to mailbox/domain
configuration for server/domain scripts to decide how best to filter
messages in those scripts?

It seems that the initial goal of Sieve was to make a fool proof scripting
language, but it seems we want to do "more than a fool" would want to do
with it.  So I recon either we don't allow all these "complex" extensions
like variables, or we do it properly, and scalably, and we end up heading
towards an email tailored, fully fledged programming language...  Scope,
const, array, list, this seems to be a whole stack of issues that have been
solved properly by other programming languages, and we're just shying away
from implementing them properly cos we're trying to keep it simple :o(
Maybe for those servers that have nothing but "fools" as users, they just
won't let them use all these extensions like the variables extension, which
will leave Sieve pretty simple like it is in the base RFC.

Hmmm, why do I feel like I'm about to be flamed? :o)

Nigel

---

From: [email-masked]
Date: Thu, 08 May 2003 11:13:34 -0700 (PDT)

> them, then we'll just end up wishing we had them for the imapflags

I disagree 100%. I think this is far too complex. There are also very
substantive syntactic difficulties with making such usage compatible with
the base sieve specification.

> certainly much more familiar.  Would that syntactically be allowed?

I can live with this as a naming convention. Scoping, however, is more than a
naming convention. I object to scoping on this basis as it is far too
complex.

> 3) All variables that are ever defined in any RFC extension must be in some

A naming convention is fine, scoping is not.

> 4) The only "unscoped" variables would be defined by the user and would be

A separate naming convention for user variables is fine.

> the script author so desires.  I don't want to have to deal with two+

You may not want to deal with them, but it may make sense to implement things
using a multi-tiered storage approach.

> implement const variables.

Vastly too complex IMNSHO.

> my other variables that I could them manipulate in any way I wanted.

This doesn't bother me either way.

> messages in those scripts?

I see no need for it. Existing extensions can be used to do this.

> from implementing them properly cos we're trying to keep it simple :o(

Simplicity per se is not the issue, nor is the ability for fools to write
scripts. The central concerns here all revolve around implementability,
performance, and security. When complexity comes up as a concern, it is because
something is seen as being too difficult to implement in the context of a a
high performance messaging server. Things like lists, associative arrays, and
so on raise the bar hugely, creating implementation complexity and thereby
increasing the risk of security problems.

Like it or not, few programming languages are designed with these sorts of
constraints in mind. This means the willy-nilly importation of features from
other languages tends not to work very well.

> will leave Sieve pretty simple like it is in the base RFC.

Even servers that serve fools have have script construction facilities. And
even if this weren't true, this is a strawman since usage simplicity isn't
the central concern here.

				Ned

---

From: Jutta Degener <[email-masked]>
Date: Thu, 8 May 2003 14:33:16 -0700

Just so my silence isn't misinterpreted as consent,
I strongly disagree with Nigel, and agree with Ned,
on this one.

You're introducing mechanisms commonly found in
object-oriented languages into something that hardly
has variables.  Don't do that.

Keep it simple.  Not everything in the world needs to be
scoped, access-controlled, typed, prefixed, and placed
in a name space.  Yes, it would make sense, if this
were a really big language, where variables somehow
are the main system interface.  But it isn't, and
it (in my opinion) shouldn't be.

By the way, I still haven't seen a reason why the
message flags extension needs to be so complicated, and
why we otherwise need to manipulate lists of strings.

(Alexey described that he doesn't want to have to
keep track of what flags exist, but I still haven't
seen what his users actually _do_ with all those
flags that they don't want to keep track of.)

Jutta

---

From: Simon Josefsson <[email-masked]>
Date: Fri, 09 May 2003 00:40:27 +0200

> flags that they don't want to keep track of.)

Here is what some users do:

For each employee, there is a set of flags jim-important, jon-todo,
ann-stalled, etc, and another set of flags qa-important, qa-stalled,
hr-important etc for each department (and another set for projects,
etc).  The mail clients are configured to make these articles visible
easier, depending on the customizations by the user.  A message can
have any number or combination of flags, e.g., jon-todo, ann-stalled,
and qa-important, which all can be set individually.

In this scenario, it isn't (in general) unreasonable to want to be
able to extract a string from the message and set a flag corresponding
to the flag.  (I haven't followed the imapflags/variables thread
closely, so I don't know if this is a reasonable goal there.)

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: 09 May 2003 13:55:59 +0200

[Ned Freed]:
>   > > e.g., @{folder} becomes a list, while ${folder} becomes a single
>   > With all the stuff and points that are getting raised I'm
>   > becoming increasingly convinced that:
>   >
>   > 1) We should implement list variables (maybe even associative
>   > variables), and implement the imapflags extension using the list
>   > variables.  Perhaps one extension that deals with the syntax,
>   > the operators (including operators to turn it into a string), or
>   > perhaps it could be part of the imapflags one...  I'd prefer
>   > them separate I think...  If we implement imapflags without list
>   > variables, then get further down the line and find we can't live
>   > without them, then we'll just end up wishing we had them for the
>   > imapflags extension.
>   
>   I disagree 100%. I think this is far too complex.

what's your preferred alternative in the imapflags case?

>   There are also very substantive syntactic difficulties with making
>   such usage compatible with the base sieve specification.

please state them so that they can be addressed.

>   > scope.variablename then so be it :o) I'd prefer :: for scope, as
>   > that's certainly much more familiar.  Would that syntactically
>   > be allowed?
>   
>   I can live with this as a naming convention. Scoping, however, is
>   more than a naming convention. I object to scoping on this basis
>   as it is far too complex.

I agree.

>   > 5) We don't have any "system variables" or "magic variables",
>   > JUST "variables".  So if we want setdate to produce variables
>   > for us to use elsewhere then they either go out to some
>   > associative array, or a set of variables in an appropriate
>   > namespace that can subsequently be written to if the script
>   > author so desires.  I don't want to have to deal with two+
>   > classes of variables :o(
>   
>   You may not want to deal with them, but it may make sense to
>   implement things using a multi-tiered storage approach.

what do you mean?

>   > 6) If we really want a class of variable that can't be written
>   > to then lets implement const variables.
>   

well, it's the alternative to system variables, which I find vastly
too complex ;-)

>   > 7) If we have variables that can only get modified by fancy
>   > operators like addflag, removeflag, I'd rather that I couldn't
>   > even access them through ${#imapflags}" etc, but instead have an
>   > expicit set action somewhere that put the value into a general
>   > purpose, bog standard variable, just like all my other variables
>   > that I could them manipulate in any way I wanted.
>   

obviously I'm with Nigel on this one.

>   > BTW another thing I'd like to be able to do is gain access to
>   > the spam score or virus name to use for prepending the subject
>   > field.  Perhaps we'd want a special automatic variable for this
>   > too?

I'm wary of tests setting variables.  (sometimes I wonder if the
convenience of ${1} is worth it.  SETMATCH ["user", "domain"] (meaning
SET "user" "${1}"; SET "domain" "${2}") is a reasonable alternative,
but quite a bit more verbose.)

>   > Access to mailbox/domain configuration for server/domain scripts
>   > to decide how best to filter messages in those scripts?

I don't understand this ...

>   I see no need for it. Existing extensions can be used to do this.

... so this intrigues me :-)

>   Simplicity per se is not the issue, nor is the ability for fools
>   to write scripts. The central concerns here all revolve around
>   implementability, performance, and security. When complexity comes
>   up as a concern, it is because something is seen as being too
>   difficult to implement in the context of a a high performance
>   messaging server. Things like lists, associative arrays, and so on
>   raise the bar hugely, creating implementation complexity and
>   thereby increasing the risk of security problems.

I agree, but a powerful variables concept may pay off when
implementing later extensions.  I think it is important that we try to
keep the language and mechanisms coherent, and add features that
complement each other and are more powerful in unison.

-- 
Kjetil T.

---

From: [email-masked]
Date: Fri, 09 May 2003 08:21:10 -0700 (PDT)

> what's your preferred alternative in the imapflags case?

A string containing a list of flags separated by spaces. This is hardly
rocket science.

> please state them so that they can be addressed.

They have all been stated before. For exmaple, what syntax do you
use to plug a list variable into an argument to a test? "&list" is
poor. What about "&list &list"? What about ["&list", "&list"].

Sure, you can work around all of this. But it keeps getting uglier and
uglier. And for what? A feature that is nice to have at best?

> what do you mean?

I mean I may not want to create variables like year, month, day, and so on on
demand rather than having setdate poke them into the same structure used for
user-defined variables.

> well, it's the alternative to system variables, which I find vastly

The question is whether we actually need them. I've yet to be convinced we do.
But if we do, this can be done as part of a naming convention, rather than by
defining more operators and such.

> complement each other and are more powerful in unison.

Exactly. The primary problem with list variables is that they clash very very
badly with earlier design choices, so much so that we lose considerable
language coherency by adding them. Now, you can argue that earlier design
choices were poor ones, but that doesn't make them go away.

				Ned

---

From: [email-masked]
Date: Fri, 09 May 2003 08:47:21 -0700 (PDT)

...

> poor. What about "&list &list"? What about ["&list", "&list"].

Used the wrong syntax for variable references here, sorry about that.
I trust my meaning was clear, however.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 04:36:32 +0200

[Ned Freed]:
>   A string containing a list of flags separated by spaces. This is
>   hardly rocket science.

a string with a predefined name, then?  (to cater for MARK/UNMARK.)

>   poor. What about "&list &list"? What about ["&list", "&list"].

my current suggestion:

"@{var}" is an error except in a list context.  a list variable in
string context interpolates as the concatenation of each value in the
list delimited by a single space.  a string variable in list context
is handled as a list with one element.  these are equivalent:

  set "var" "meep";
  set "var" ["meep"];

a string containing multiple list references will be split on the
references, and the list will contain the pre-, in- and post-fixes as
elements in the list.

  ["@{var} @{var}!"] => ["meep", " ", "meep", "!"]

>   Sure, you can work around all of this. But it keeps getting uglier
>   and uglier.

ugliness is in the eyes of the beholder. :-)

>   And for what? A feature that is nice to have at best?

possibly, but lists are a widespread occurence in Sieve, and I think
it does make sense to be able to represent them in variables.

>   I mean I may not want to create variables like year, month, day,
>   and so on on demand rather than having setdate poke them into the
>   same structure used for user-defined variables.

alright, so if I understand you correctly, the "multi-tier" is that
you store the time-of-day internally, and populate the user-defined
variables from that when asked to.

>   > well, it's the alternative to system variables, which I find
>   > vastly too complex ;-)
>
>   The question is whether we actually need them. I've yet to be
>   convinced we do.  But if we do, this can be done as part of a
>   naming convention, rather than by defining more operators and
>   such.

I agree.

>   Exactly. The primary problem with list variables is that they
>   clash very very badly with earlier design choices, so much so that
>   we lose considerable language coherency by adding them. Now, you
>   can argue that earlier design choices were poor ones, but that
>   doesn't make them go away.

I honestly can't see why list variables clash more badly than plain
variables.

-- 
Kjetil T.

---

From: [email-masked]
Date: Sat, 10 May 2003 02:01:19 -0700 (PDT)

>   set "var" ["meep"];

Just because there are no cases now where a string and a list containing
a single element aren't equivalent doesn't mean such cases won't arise
in the future. Additionally, this fails to take into account the very
real possibility of implementation limits being different for strings
and elements of lists.

>   ["@{var} @{var}!"] => ["meep", " ", "meep", "!"]

The effect of these expressions are now entirely nonobvious. Now a simple
count of the elements no longer suffices to determine how many elements there
are in a list. 

There are probably a bunch of additional problems here I can't think of right
now.

> ugliness is in the eyes of the beholder. :-)

I disagree. This is ugly no matter how you slice it. So ugly, in fact, that I
fear it will get very substantial pushback from the IESG.

> it does make sense to be able to represent them in variables.

I might agree had the syntax for adding them been cleaner. But it isn't, so
I don't.

> variables from that when asked to.

Correct.

> I honestly can't see why list variables clash more badly than plain

Your own examples demonstate the truth of this assessment more adequately
than anything I could write. This is really pretty awful.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 15:07:35 +0200

>  > a string with a predefined name, then?  (to cater for MARK/UNMARK.)

you didn't reply to this.

>   Just because there are no cases now where a string and a list
>   containing a single element aren't equivalent doesn't mean such
>   cases won't arise in the future.

the mentioned equivalence only concerns SET.
   set "list" "@{otherlist}";
is an error.  it MUST be written in a list context:
   set "list" [ "@{otherlist}" ];

>   Additionally, this fails to take into account the very real
>   possibility of implementation limits being different for strings
>   and elements of lists.

obviously my text needs fleshing out with details such as those.

>  > references, and the list will contain the pre-, in- and
>  > post-fixes as elements in the list.
>  >
>   The effect of these expressions are now entirely nonobvious. Now a
>   simple count of the elements no longer suffices to determine how
>   many elements there are in a list.

correct.  that will always be the case since a list variable by nature
will have a non-constant number of elements.  but if you prefer, a
string containing anything in addition to the list variable reference
could be disallowed with no loss of expressibility.

>   I disagree. This is ugly no matter how you slice it.  So ugly, in
>   fact, that I fear it will get very substantial pushback from the
>   IESG.

sorry, but I don't see your objection.

>  > possibly, but lists are a widespread occurence in Sieve, and I
>  > think it does make sense to be able to represent them in
>  > variables.
>
>   I might agree had the syntax for adding them been cleaner. But it
>   isn't, so I don't.

I believe a cleaner syntax requires changes to the basic grammar of
Sieve.

>  > I honestly can't see why list variables clash more badly than
>  > plain variables.
>
>   Your own examples demonstate the truth of this assessment more
>   adequately than anything I could write. This is really pretty
>   awful.

obviously not an argument which is going to convince me.

-- 
Kjetil T.

---

From: Marc Mutz <[email-masked]>
Date: Sat, 10 May 2003 19:36:24 +0200

On Saturday 10 May 2003 15:07, Kjetil Torgrim Homme wrote:
<snip>
> >  > a string containing multiple list references will be split on
> >  > the references, and the list will contain the pre-, in- and

On a gut level, I'd thought that
  [ "@{var} @{var}!" ] => [ "meep meep!" ].
And if var contains "foo" and "bar":
 [ "@{var} @{var}!" ]
    => [ "foo foo!", "foo bar!", "bar foo!", "bar bar!" ]
This is probably since brace expansion works like this in UNIX shells:
 $ echo {foo,bar}_{foo,bar}
 foo_foo foo_bar bar_foo bar_bar
and Sieve's header test, too:
 if address :all :is [ "To", "Cc" ]
                     [ "[email-masked]", "[email-masked]" ] {
  # To <-> kde.org, To <-> mail.kde.org
  # Cc <-> kde.org, Cc <-> mail.kde.org
 }

> >   The effect of these expressions are now entirely nonobvious.
<snip>

And not straightforward to define either, it seems.

> I believe a cleaner syntax requires changes to the basic grammar of

That would introduce a catch22:
- You need the grammar to parse "require" commands
- You need to parse the "require" commands to find out the currently
  valid grammar.

At least in my implementation the grammar is hard-coded and any 
extension-dependent code kicks in after parsing into 
command/test/command-list/etc has taken place.

I expect others to do it likewise.

Marc

-- 
The [Sonny Bono Copyright Term Extension Act] expands copyright not
only for future, but also for existing works, even though their
authors obviously don't need any additional incentive to create them.
                -- "The Progress Of Science And Useful Arts":
                    Why Copyright Today Threatens Intellectual Freedom,
                   Free Expression Policy Project

---

From: [email-masked]
Date: Sat, 10 May 2003 11:28:56 -0700 (PDT)

> you didn't reply to this.

That's because I don't especially care how this works.

>    set "list" [ "@{otherlist}" ];

OK, but this doesn't change the fact that you've said that

set "var" "meep";

creates a list variable containing one element. I don't think you want
this...

> could be disallowed with no loss of expressibility.

Making it even more complex and nonobvious.

> sorry, but I don't see your objection.

It isn't _my_ objection.  Based on past experience with the IESG I think
there's a good chance of this proposal going down in flames with the IESG and
never getting published as an RFC.

This group needs to decide whether or not the value of list variables
is worth the possibility of not getting a variables specification
approved, period.

> I believe a cleaner syntax requires changes to the basic grammar of

Exactly.

				Ned

---

From: Marc Mutz <[email-masked]>
Date: Sat, 10 May 2003 21:02:48 +0200

On Saturday 10 May 2003 20:28, [email-masked] wrote:
<snip>
<snip>

Let's defer list variables for the time being. String variables are 
useful enough in itself to get this out as RFC rather sooner; the more 
so as other drafts in the pipeline probably want to make use of it.

We can always come back to try hammering out a listvariables extension 
later, if anyone cares enough.

Marc

-- 
"You're hackers, aren't you," the barman said, eyeing us. No one said
a thing. The darkness of the Eurotunnel rolled by. Apparently we'd
given ourselves away by talking too enthusiastically about IPv6. He
looked around conspiratorially, lowered his voice. "Can you get me
some credit card numbers?"
      -- James J. King "What's the shortest way to hack a Linux box?"
         Telepolis 2001/08/11 (#9293)

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 22:18:05 +0200

>       => [ "foo foo!", "foo bar!", "bar foo!", "bar bar!" ]

hmm, cross product...  is that useful?  seems like an easy way to soak
up memory for script writers ;-)

>   - You need to parse the "require" commands to find out the
>     currently valid grammar.

I think we all agree that such changes shouldn't be done.

-- 
Kjetil T.

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 23:50:30 +0200

> That's because I don't especially care how this works.

but it's essential for making a decision.

according to Marc, the variables extension was wanted to make
imapflags blend in better.  isn't it a mistake to press on before we
have decided how imapflags should operate?

actually, as long as MARK/UNMARK are kept, the variable name must be
implicit and fixed.  and so we're back at the system/read-only
variable discussion ;-)

>   creates a list variable containing one element. I don't think you
>   want this...

not exactly.

every variable can be used as a list variable.  every list variable
can be used as a string variable.  I just think that's easier to
grasp.  I _definitely_ don't want separate namespaces, like in Perl
where @var and $var are different.

> > correct.  that will always be the case since a list variable by
> > nature will have a non-constant number of elements.  but if you
> > prefer, a string containing anything in addition to the list
> > variable reference could be disallowed with no loss of
> > expressibility.
>

yeah.  Marc's cross product idea may be better after all.  but the
truth is, I seriously doubt anyone would use anything but singleton
list variable references.  with my suggestion, explicit list members
are clearer than putting it in the same string.  with Marc's
suggestion, a very powerful construct is gained, but I don't see what
it can be used for in Sieve.  (well, his factorial function might ;-)

>   This group needs to decide whether or not the value of list
>   variables is worth the possibility of not getting a variables
>   specification approved, period.

I'd like to focus on giving imapflags what it needs.  it's not at all
clear list variables are needed.

-- 
Kjetil T.

---

From: [email-masked]
Date: Sat, 10 May 2003 15:13:21 -0700 (PDT)

> have decided how imapflags should operate?

It isn't what I wanted it for. While the imapflags extension can be
useful, the uses I see it being put to don't require variables.

> implicit and fixed.  and so we're back at the system/read-only

And I have stated repeatedly that I view this as "mostly harmless".

> where @var and $var are different.

You're missing the point, which is that variables are typed: Each one is either
a string or a list. The fact that string variables can be coerced into
something else in a list context and vice versa doesn't mean that the
underlying variable isn't typed.

Now, perhaps you're trying to get rid of variable typing by making everything
completely mutable. But this just doesn't work: The rules for how a given
variable are handled in various contexts depend on knowing whether the
underlying variable is a string or a list. And this means

set var "foo"

had darned well better set the variable "var" to the _string_ "foo".

> it can be used for in Sieve.  (well, his factorial function might ;-)

Which means we should not be considering it.

> clear list variables are needed.

Exactly what I have been saying over and over and over again.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 11 May 2003 03:44:29 +0200

[Ned Freed]:
>   > imapflags blend in better.  isn't it a mistake to press on
>   > before we have decided how imapflags should operate?
>
>   It isn't what I wanted it for. While the imapflags extension can
>   be useful, the uses I see it being put to don't require variables.

good.  that's my view as well, so then I'm not completely out of line :)

>   > actually, as long as MARK/UNMARK are kept, the variable name
>   > must be implicit and fixed.  and so we're back at the
>   > system/read-only variable discussion ;-)
>

yeah.  I think I'll start a new thread on this.

>   > every variable can be used as a list variable.  every list
>   > variable can be used as a string variable.  I just think that's
>   > easier to grasp.  I _definitely_ don't want separate namespaces,
>   > like in Perl where @var and $var are different.
>
>   You're missing the point, which is that variables are typed: Each
>   one is either a string or a list.

says who?  make all variables lists internally, or promote strings
when needed.  either would work.

>   The fact that string variables can be coerced into something else
>   in a list context and vice versa doesn't mean that the underlying
>   variable isn't typed.

the user can not tell what the implementation does.

>   Now, perhaps you're trying to get rid of variable typing by making
>   everything completely mutable. But this just doesn't work: The
>   rules for how a given variable are handled in various contexts
>   depend on knowing whether the underlying variable is a string or a
>   list. And this means
>
>   set var "foo"
>

with my proposed semantics there is no observable difference.

but let's leave this discussion, it's not useful.

>   > I'd like to focus on giving imapflags what it needs.  it's not
>   > at all clear list variables are needed.
>

so have I, I just presented three alternatives.  why didn't you help
me flesh out your favourite alternative rather than pull authority on
your least favourite?

-- 
Kjetil T.

---

From: [email-masked]
Date: Sat, 10 May 2003 20:43:09 -0700 (PDT)

> me flesh out your favourite alternative rather than pull authority on

All I see here is a proposal to add list variables. As for authority, I
normally engage in this discussion simply as an implementor and user, nothing
more. I have no more of a "vote" than anyone else.

When I talk about possible IESG reactions, I'm not talking about _my_ reaction
as an IESG member, but what I think the likely reaction of other members might
be.

				Ned

---

From: Tony Hansen <[email-masked]>
Date: Fri, 16 May 2003 16:25:00 -0400

[email-masked] wrote:
 >> 3) All variables that are ever defined in any RFC extension must be
 >> in kind of scope.
 >

I think this statement by Ned is key to his feelings on the topic. I 
think the problem has arisen because of potential misuse or 
misunderstandings of the terms "namespace" and "scope" by various posters.

Note: The term namespace is a naming convention -- it has nothing to do 
with scope. (Whenever I have written about this, I think I have 
consistently used the term "namespace" and not "scope".)

In short, I agree completely with Ned.

I would like to see the ability to have a naming convention such that I 
can write an extension "foo" and name all the variables created by that 
extension with a naming convention such as "foo." or "foo:" or "foo::" 
or something of that nature.

I really dislike using "_" for this purpose, because "_" in most 
languages is expected to be simply a word separator and nothing special. 
And I expect our users to treat it similarly, and not as anything that's 
part of any naming convention.

What would be required for this to be possible? At the minimum, simply 
add "." or ":" to the list of valid characters allowed within a name. 
Treating "." or ":" as part of a naming convention is just that, a 
convention.

(I've consistently used "." in my postings on this topic. Other than not 
using "_" for this purpose, I DON'T CARE which convention is chosen, be 
it ".", ":", "::", or whatever.)

Now, I would also like it agreed upon that all sieve extensions that 
create variables SHOULD use a namespace using this naming convention. I 
would even recommend that a sieve extension named "example" SHOULD use 
"example." as the prefix on all of its variables. (I consider this a 
Best Practice issue.)

	Tony Hansen
	[email-masked]

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 16 May 2003 22:53:00 +0200

[Tony Hansen]:
>   languages is expected to be simply a word separator and nothing
>   special. And I expect our users to treat it similarly, and not as
>   anything that's part of any naming convention.

I believe you are right, although I'm not sure if it is a problem in
practice.

>   What would be required for this to be possible? At the minimum,
>   simply add "." or ":" to the list of valid characters allowed
>   within a name. Treating "." or ":" as part of a naming convention
>   is just that, a convention.

consider this minimal change to the draft:

	variable-ref	=	"${" variable-name "}"
-	variable-name	=	num-variable / identifier
+	variable-name	=	num-variable / *namespace identifier
+       namespace	=	identifier "."
	num-variable	=	1*DIGIT

since SET takes a pure identifier as its first argument, any variable
name with a namespace component effectively becomes read-only.

(I'm not sure we need nested namespaces, but I don't think they do any
harm.)

>   (I've consistently used "." in my postings on this topic. Other
>   than not using "_" for this purpose, I DON'T CARE which convention
>   is chosen, be it ".", ":", "::", or whatever.)

since I grew up with Simula, I prefer "." as well :-)

>   Now, I would also like it agreed upon that all sieve extensions
>   that create variables SHOULD use a namespace using this naming
>   convention. I would even recommend that a sieve extension named
>   "example" SHOULD use "example." as the prefix on all of its
>   variables. (I consider this a Best Practice issue.)

if we do this, I propose we change ${year} into ${date.year} etc.

-- 
Kjetil T.

---

From: "Nigel Swinson" <[email-masked]>
Date: Sat, 17 May 2003 02:24:13 +0100

> In short, I agree completely with Ned.

I agree that there should be a naming convention to create "namespaces" for
variables that relate to an extension.  I would personally prefer "::", as I
think it will look nicest with the existing syntax, and you rarely want to
use "::" in a string, but commonly want to use "." for something else, so  I
think "::" is more specific.

> "example." as the prefix on all of its variables. (I consider this a

The sieve extension name does sound like a good idea, but I would want us to
have date::year rather than variables::year, so there may be cases when one
extension defines several namespaces, and I think it might be too
restrictive and end up confusing to demand that all variables from one
extension have to go by the extension name.

With regards to "scope" are we definately sure that we want:

   All variables have global scope: they are visible until processing
   stops.  Variable names are case insensitive.

Do we want to support any kind of local/file/global scope at all?  It seems
that it's fairly universally agreed that global variables are pretty evil
(Lawrence has already made this point), so are we sure that we want to have
all variables global by default?  (At the moment) I think would be in favour
of a :local :file or :global argument to the set command to indicate whether
or not this variable can/should "bleed" out of this script and into the
parent.  I'd recommend that :file be the default rather than :global.

I note that Ned said:
> I strongly object to this proposal
[http://www.imc.org/ietf-mta-filters/mail-archive/msg01211.html].
> It adds considerable implementation complexity yet achieves negligable
benefits.
>
> Variables give scripts capabilities they didn't have before and which
cannot be
> achieved with any existing mechanisms. The same cannot be said for local
> variables. I fail to see the capability they provide for real world
scripts
> that cannot be achieved any other way.

Could I suggest that we say that "All variables have file scope: they are
visible only within the same .siv file, not spanning include boundaries".
Then later we support a "scope" extension that permitted :local and :global?

I think the argument for scope is to prevent users from making accidental
mistakes (rather than adding additional capabilities).  I'm wondering if a
user would "expect" a variable in one script to be available to him in the
calling script, or in a called script.  I would imagine the author would
expect variables to be local to the file by default, but would appreciate a
mechanism to explicitly ask that a file "bleed" into or out of an included
file etc.

Cheers

Nigel

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 17 May 2003 04:56:08 +0200

[Nigel Swinson]:
>   Do we want to support any kind of local/file/global scope at all?
>   It seems that it's fairly universally agreed that global variables
>   are pretty evil (Lawrence has already made this point), so are we
>   sure that we want to have all variables global by default?

we need to specify global as default, since an extension changing a
previously file local variable to be global has all kinds of hairy
problems.  I have previously demonstrated that going the other way is
quite easy, since the scope extension itself will have file local
scope.

>   Could I suggest that we say that "All variables have file scope:
>   they are visible only within the same .siv file, not spanning
>   include boundaries".
>

I'm very much opposed to this, since it limits the capabilities of an
INCLUDE quite drastically, with no other workaround than a future
"scope" extension which I think will be useless for anything but
getting a global scope.

>   I think the argument for scope is to prevent users from making
>   accidental mistakes (rather than adding additional capabilities).
>   I'm wondering if a user would "expect" a variable in one script to
>   be available to him in the calling script, or in a called script.
>   I would imagine the author would expect variables to be local to
>   the file by default, but would appreciate a mechanism to
>   explicitly ask that a file "bleed" into or out of an included file
>   etc.

the expectation of the author will probably vary with what languages
he's used previously.  unless he was brought up on LISP, I believe he
would expect global scope.  (all scripting languages use global scope
by default, so does C and even Pascal.)

-- 
Kjetil T.

To: <[email-masked]>
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 17 May 2003 04:56:08 +0200
Sender: [email-masked]

[Nigel Swinson]:
>   Do we want to support any kind of local/file/global scope at all?
>   It seems that it's fairly universally agreed that global variables
>   are pretty evil (Lawrence has already made this point), so are we
>   sure that we want to have all variables global by default?

we need to specify global as default, since an extension changing a
previously file local variable to be global has all kinds of hairy
problems.  I have previously demonstrated that going the other way is
quite easy, since the scope extension itself will have file local
scope.

>   Could I suggest that we say that "All variables have file scope:
>   they are visible only within the same .siv file, not spanning
>   include boundaries".
>

I'm very much opposed to this, since it limits the capabilities of an
INCLUDE quite drastically, with no other workaround than a future
"scope" extension which I think will be useless for anything but
getting a global scope.

>   I think the argument for scope is to prevent users from making
>   accidental mistakes (rather than adding additional capabilities).
>   I'm wondering if a user would "expect" a variable in one script to
>   be available to him in the calling script, or in a called script.
>   I would imagine the author would expect variables to be local to
>   the file by default, but would appreciate a mechanism to
>   explicitly ask that a file "bleed" into or out of an included file
>   etc.

the expectation of the author will probably vary with what languages
he's used previously.  unless he was brought up on LISP, I believe he
would expect global scope.  (all scripting languages use global scope
by default, so does C and even Pascal.)

-- 
Kjetil T.

From: "Nigel Swinson" <[email-masked]>
To: <[email-masked]>
Subject: Re: Variables : list : scope : system : const
Date: Sat, 17 May 2003 02:24:13 +0100
Sender: [email-masked]

benefits.
cannot be
scripts

Cheers

Nigel

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Fri, 16 May 2003 22:53:00 +0200
Sender: [email-masked]

-- 
Kjetil T.

Date: Fri, 16 May 2003 16:25:00 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked]
Subject: Re: Variables : list : scope : system : const
Sender: [email-masked]

	Tony Hansen
	[email-masked]

To: Marc Mutz <[email-masked]>
Cc: [email-masked]
Subject: Re: imapflags and variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: Tue, 13 May 2003 00:15:32 +0200
Sender: [email-masked]

[Marc Mutz]:
>
>   On Sunday 11 May 2003 05:16, Kjetil Torgrim Homme wrote:
>   > Marc suggests a system variable for accessing the current value,
>   > ${#imapflags}.
>   
>   Actually, I suggested a test for it, but ok.

sorry, I have missed that.  would you mind repeating it?  I think it
makes sense.

>   > you can get rid of ':globalflags_plus "Flag"' like 
>   > this:
>   >
>   >         keep :flags "${#imapflags} Flag";
>   
>   more like
>   
>     set "safe_flags" "${#imapflags}";
>     addflag "Flag";
>     keep :globalflags
>     setflags "${safe_flags}";

this was the added verbosity I hinted at if :flags was removed.

>   > my suggestion is to add a another action instead, only available when
>   > the variables extension is in effect:
>   >
>   >         STOREFLAGS <variable: string>
>   
>   The downside of that is that you need to come up with such an
>   action for each and every extension that possibly wants to store
>   away state temporarily.

true.  it remains to be seen how common that is, though.

>   So how could imapflags progress in this light? I see two routes:
>   
>   I non-system variables:
>   
>   1. Dump :globalflags, keep only :flags. This means that implicit
>   keep never sets any flags, but you can always end your script with
>   an explicit keep (multiscript might make problems, though).

the implicit keep can't be emulated _that_ easily.  consider

  if ... {
     fileinto "somewhere";
  }
  keep :flags "${myflags}";

(message uniqueness only concerns a single folder.)

so you'd need to keep track of it yourself.

  if ... {
    set "kept" "yes";
    fileinto "somewhere";
  }
  if not string "${kept}" "yes" {
    keep :flags "${myflags}";
  }

thank you for your very useful summary!

-- 
Kjetil T.

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: Re: imapflags and variables
Date: Mon, 12 May 2003 22:21:26 +0200
Cc: 
Reply-To: 
Content-Description: 
Sender: [email-masked]

On Sunday 11 May 2003 05:16, Kjetil Torgrim Homme wrote:
<snip>
> Marc suggests a system variable for accessing the current value,
> ${#imapflags}.

Actually, I suggested a test for it, but ok.

> you can get rid of ':globalflags_plus "Flag"' like 
> this:
>
>         keep :flags "${#imapflags} Flag";
<snip>

more like

  set "safe_flags" "${#imapflags}";
  addflag "Flag";
  keep :globalflags
  setflags "${safe_flags}";

> I'll also note that the first snippet will misbehave with no
> diagnostics if the user has forgotten 'require "variables"'.

"imapflags" would require "variables". I know that there's currently no 
extension that depends on another, but it's not uncommon. E.g. ESMTP's 
BINARYMIME depends on CHUNKING. Variables is a candidate that 
extensions will want to depend on.

> my suggestion is to add a another action instead, only available when
> the variables extension is in effect:
>
>         STOREFLAGS <variable: string>
<snip>

The downside of that is that you need to come up with such an action for 
each and every extension that possibly wants to store away state 
temporarily.

> for both proposals, :globalflags is sufficient, but verbosity is
> slightly increased without :flags <flag-list>.  more radically, we
> could remove both and make :globalflags the default and mandatory
> behaviour for KEEP/FILEINTO.  this way the implicit keep can set
> flags on the message.  I think that would be useful.

Rereading the "progressing various sieve drafts" thread, I get the 
impression that people were ok with :flags, but not with :globalflags. 
The main reason seemed to be that setflags/addflags/etc alter global 
state and that was a new thing for an extension to do.

Thus, there was thought about deferring this until variables are 
available. Says Tim:

On Wednesday 17 July 2002 19:49, Tim Showalter wrote:
> On Wed, 2002-07-17 at 06:57, Ken Murchison wrote:
<snip>
> > I have -03 implemented in Cyrus and I find the global flags nature
> > very useful,
<snip>
> > That being said, I find the :globalflags stuff kind of crufty ;)
>
> There's no argument from me that it's *useful*, but that it's cruft
> that we'll need to replicate again and again in the future that looks
> exactly like variables with an undesirable syntax.

So it seems to me that the feeling "back then" was to let extensions 
with actions that alter global state depend on a (then non-existant) 
variables extension. I think the variables extension needs to reflect 
this. I've proposed the "system variable" concept to this end, I'm open 
to other solutions.

Reviving the old action-affects-global-state method seems undesirable to 
me.

So how could imapflags progress in this light? I see two routes:

I non-system variables:

1. Dump :globalflags, keep only :flags. This means that implicit keep
   never sets any flags, but you can always end your script with an
   explicit keep (multiscript might make problems, though).
2. Iff - additionally - variables is required, then you can either
   2a. fill a user variable with flags and leave the whitespace,
      duplicates and ordering issues to the user. Dump
      *flags/mark/unmark.
   2b. introduce a required string argument to the *flags/*mark actions
      that specifies a variable name for the action to act on. The
      actions ensure validity of their arguments and maintain
      uniqueness, order and whitespace conventions inside the (user)
      variable.
   this way, you can emulate :globalflags with a explicit variable.

II system-variables / no tags

1. Dump :globalflags and :flags altogether.
2. Make imapflags depend on variables and introduce the system variable
   ${#imapflags}.
3. Make keep and fileinto use the current value of ${#imapflags}.
4. keep the set actions with alter #imapflags.

III system-variables / :useflags

like II, but fileinto and keep only use ${#imapflags} if the :useflags 
tag is present on them.

(I) has the advantage of being explicit, but requires the user to 
maintain the variables constraints herself, which is error-prone.

(II/III) have the advantage of being less error-prone, but are not 
_that_ explicit.

Either way, I don't see an approach that combines the advantages of 
both. I'd be glad to be shown one.

Marc

-- 
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.      -- Benjamin Franklin

To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Simon Josefsson <[email-masked]>
Date: Sun, 11 May 2003 11:45:20 +0200
Sender: [email-masked]

Kjetil Torgrim Homme <[email-masked]> writes:

> I'll try to summarise in a new thread, so just a simple answer here.
>
> [Marc Mutz]:
>>
>>   > an ISO 8601 date comparator is not needed, since a normal string
>>   > compare is enough.
>>
>>   Please explain why "a normal string compare" is enough to determine 
>>   which of the two dates
>>     Sat, 10 May 2003 21:18:44 +0200
>>     Sat, 10 May 2003 20:18:44 +0000
>>   is earlier.
>
> those aren't on ISO 8601 format.  still, I forgot about time zones,
> you'd need to normalise those before doing the string compare.

So a ISO 8601 date comparator is needed?

If I recall correctly, ISO 8601 has some quirks that may argue for a
ISO 8601 date comparator in Sieve.  E.g., 00:00 == 24:00; negative
time fields -10:30, time zones 00:00 == 24:00 (but different days?),
double negative time fields --30, fractional fields.  RFC 3339
disallow some of these though.

Date: Sat, 10 May 2003 20:48:59 -0700 (PDT)
From: [email-masked]
Subject: Re: variables and side-effects of tests
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> [Ned Freed]:
> >
> >   [Marc Mutz]:
> >   > So the variables extension, as it stands, violates this MUST,
> >   > since the numerical variables are set as a side-effect of a test
> >   > with :matches match type. :-(
> >   >
> >   > What now?
> >
> >   We have to reach consensus to revise the specification and remove
> >   the MUST.

> I don't object to a revision of RFC 3028, but an alternative may be to
> replace the numerical variables with

>   SETMATCH <variables: string-list>

> the :matches will change the internal state, and this is still
> observable by a user, so it may not be good enough to satisfy language
> lawyers.

Well, previously there was no internal state that wasn't implicit in the
problem structure. Now there is, and the test changed it. I can live with
it, but others may feel differently...

> Example:
>   if address :all :matches "To" "*+*@*.ifi.uio.no" {
>      setmatch [ "user", "", "host" ];
>   }

I actually sort of like this, but it isn't clear to me it really solves the
problem.

				Ned

Date: Sat, 10 May 2003 20:43:09 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

				Ned

To: [email-masked]
Subject: imapflags and variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 11 May 2003 05:16:04 +0200
Sender: [email-masked]

short recap: imapflags consists of five actions

  MARK
  UNMARK
  ADDFLAG <flags: string-list>
  REMOVEFLAG <flags: string-list>
  SETFLAG <flags: string-list>

and a number of tagged arguments for KEEP and FILEINTO

  :flags <flags: string-list>
  :globalflags
  :globalflags_plus <flags: string-list>
  :globalflags_minus <flags: string-list>

without one of the tagged arguments, no flags will be set.

there are two things some people desire:

  a) being able to restore an old set of flags
  b) get rid of some of the tagged argument variants

Marc suggests a system variable for accessing the current value,
${#imapflags}.  you can get rid of ':globalflags_plus "Flag"' like
this:

        keep :flags "${#imapflags} Flag";

and ':globalflags_minus "Flag"' like this:

        set "safe_copy" "${#imapflags}";
        removeflag "Flag";
        keep :globalflags
        setflag "${safe_copy}";

the downside to this proposal is that it introduces a new syntax for
system variables since we don't want to allow assignment to them.
I'll also note that the first snippet will misbehave with no
diagnostics if the user has forgotten 'require "variables"'.

my suggestion is to add a another action instead, only available when
the variables extension is in effect:

        STOREFLAGS <variable: string>

the result of this is that the variable contains the list of flags
currently set.  the code to replace ':globalflags_plus "Flag"' becomes
a little more verbose:

        storeflags "curr_flags";
        keep :flags "${curr_flags} Flag";

the code for ':globalflag_minus "Flag"' is essentially the same.

        storeflags "safe_copy";
        removeflag "Flag";
        keep :globalflags
        setflag "${safe_copy}";

for both proposals, :globalflags is sufficient, but verbosity is
slightly increased without :flags <flag-list>.  more radically, we
could remove both and make :globalflags the default and mandatory
behaviour for KEEP/FILEINTO.  this way the implicit keep can set flags
on the message.  I think that would be useful.

-- 
Kjetil T.

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 11 May 2003 04:14:18 +0200
Sender: [email-masked]

I'll try to summarise in a new thread, so just a simple answer here.

[Marc Mutz]:
>
>   > an ISO 8601 date comparator is not needed, since a normal string
>   > compare is enough.
>
>   Please explain why "a normal string compare" is enough to determine 
>   which of the two dates
>     Sat, 10 May 2003 21:18:44 +0200
>     Sat, 10 May 2003 20:18:44 +0000
>   is earlier.

those aren't on ISO 8601 format.  still, I forgot about time zones,
you'd need to normalise those before doing the string compare.

-- 
Kjetil T.

To: [email-masked]
Subject: Re: variables and side-effects of tests
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 11 May 2003 04:04:28 +0200
Sender: [email-masked]

[Ned Freed]:
>
>   [Marc Mutz]:
>   > So the variables extension, as it stands, violates this MUST,
>   > since the numerical variables are set as a side-effect of a test
>   > with :matches match type. :-(
>   >
>   > What now?
>
>   We have to reach consensus to revise the specification and remove
>   the MUST.

I don't object to a revision of RFC 3028, but an alternative may be to
replace the numerical variables with

  SETMATCH <variables: string-list>

the :matches will change the internal state, and this is still
observable by a user, so it may not be good enough to satisfy language
lawyers.

Example:
  if address :all :matches "To" "*+*@*.ifi.uio.no" {
     setmatch [ "user", "", "host" ];
  }

(the empty string means throw the value away.)  at least it gets rid
of this problematic code:

  if header :matches "Subject" [ "make * fast", "free ${1}!" ] {
     ...
  }

the behaviour of the above code is an open issue.  I assumed "free
${1}!" would be interpolated _before_ the test was executed, but then
Jutta came up with replaceheader where an argument string is
interpolated _after_ the tests built in to the action have been
performed.  the variables draft does not contain any definitive
wording on this issue.

-- 
Kjetil T.

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sun, 11 May 2003 03:44:29 +0200
Sender: [email-masked]

-- 
Kjetil T.

Date: Sat, 10 May 2003 15:31:00 -0700 (PDT)
From: [email-masked]
Subject: Re: variables and side-effects of tests
To: Marc Mutz <[email-masked]>
Cc: [email-masked]
Content-description: signed data
Sender: [email-masked]

> Hi!

> From RFC3028:
> 2.5.     Tests

>    Tests MUST NOT have side effects.  That is, a test cannot affect the
>    state of the filter or message.  No tests in this specification have
>    side effects, and side effects are forbidden in extension tests as
>    well.

>    The rationale for this is that tests with side effects impair
>    readability and maintainability and are difficult to represent in a
>    graphic interface for generating scripts.  Side effects are confined
>    to actions where they are clearer.

> So the variables extension, as it stands, violates this MUST, since the
> numerical variables are set as a side-effect of a test with :matches
> match type. :-(

> What now?

We have to reach consensus to revise the specification and remove the MUST. 

				Ned

Date: Sat, 10 May 2003 15:13:21 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

set var "foo"

				Ned

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 23:50:30 +0200
Sender: [email-masked]

not exactly.

-- 
Kjetil T.

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: variables and side-effects of tests
Date: Sat, 10 May 2003 21:44:53 +0200
Sender: [email-masked]

--Boundary-02=_+aVv+ug3v+RR58H
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

Hi!

=46rom RFC3028:
2.5.     Tests

<snip>

   Tests MUST NOT have side effects.  That is, a test cannot affect the
   state of the filter or message.  No tests in this specification have
   side effects, and side effects are forbidden in extension tests as
   well.

   The rationale for this is that tests with side effects impair
   readability and maintainability and are difficult to represent in a
   graphic interface for generating scripts.  Side effects are confined
   to actions where they are clearer.

So the variables extension, as it stands, violates this MUST, since the=20
numerical variables are set as a side-effect of a test with :matches=20
match type. :-(

What now?

Marc

=2D-=20
If you read this Mail while moving in traffic, you could hit
lantern stakes or end up knocked over by trucks.
                       -- freely adapted from iX 11/2002 editorial

--Boundary-02=_+aVv+ug3v+RR58H
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_+aVv+ug3v+RR58H--

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Sat, 10 May 2003 21:46:25 +0200
Sender: [email-masked]

--Boundary-02=_ScVv+Df2ZRh52mg
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Saturday 10 May 2003 04:14, Kjetil Torgrim Homme wrote:
<snip>
> >   This might work - for imapflags. What about ${#currentdate},
> >   ${#size} (see below).
>
> these aren't suitable for a list type, are they?
<snip>

That's not the point. The point is that list variables solve this=20
particular problem (if at all, see the duplicates problem). I've seen=20
no other example of their usefulness outside the "nice to have"=20
department.

System variables solve all "internal variable" problems, not only=20
imapflags. And with a common concept.

> you'd need to
>    set "foo" [ "@{undef}" ];
> to reset foo to the empty list.
<snip>

If at all, I'd prefer an unset action for that.

> >>   set "flags" "${flags} \\Flagged ";  # note trailing space
> >>   if string :matches "${flags}" "* \\Flagged *" {
> >>     set "flags" "${1} ${2}";
> >>   }
> >
> >   Ugh. Too subtle. It's a open invitation for bugs in scripts.
>
> possibly.  if this code is given in the documentation, that would
> reduce the error rate quite a bit.=20
<snip>

Unlike the matches-set idiom, this is not generally applicable and I=20
still maintain the view that it's too fragile.

> >   >   # now we want to restore the flags to the values before the
> >   > if. # or do we?
> >
> >   <snip>
>
> you snipped my "sorry, I can't come up with a credible example",
> which is what "or do we" is about.  why would we ever need to restore
> the flags?
<snip>

To get rid of the numerous fileinto tagged arguments that imapflags=20
defines. There would be only :globalflags semantics left. "local" flags=20
can be had with

set "saveflags" "${#imapflags}";
setflags "\\Seen";
fileinto "low-priority";
setflags "${saveflags}";

This is more flexible than the multitude of :flags +-=3D :globalflags +-=3D=
=20
tags, which are unified into the common concept of manipulating=20
variables.

> an ISO 8601 date comparator is not
> needed, since a normal string compare is enough.

Please explain why "a normal string compare" is enough to determine=20
which of the two dates
  Sat, 10 May 2003 21:18:44 +0200
  Sat, 10 May 2003 20:18:44 +0000
is earlier.

> > Should every foo test
> > have a corresponding setfoo action that optimizes away the
> > otherwise necessary matches-set idioms?
>
> that's the big question, isn't it?  the matches-set idiom seems
> clunky, a hack.  I don't want to base the design of the language on
> it.
<snip>

The problem is again the already defined tests. It's one thing to let=20
the variables extension add system variables for basic Sieve or the=20
extensions already implemented/out. It's another thing to let it add a=20
load of _actions_, whose availability is determined by a combination of=20
extensions.

> >   [2] And one which the variables spec needs to define, so it
> >   addresses Ned's concerns about defining a concept and not
> >   instantly making use of it.
>
> which concept?

That of system variables.

> >   What do you mean with "generic mechanism"? Isn't "system
> > variable" such a generic mechanism?
>
> every system variable will behave differently.  I don't think that's
> "generic".

Define "behavior". Different system variables behave as differently as=20
different actions do, as different tests do. Still, you wouldn't say=20
that "action" or "test" is not a generic concept, or would you?

Marc

=2D-=20
Eternal vigilance is the price of liberty   -- Thomas Jefferson

--Boundary-02=_ScVv+Df2ZRh52mg
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_ScVv+Df2ZRh52mg--

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 22:18:05 +0200
Sender: [email-masked]

-- 
Kjetil T.

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: Re: Variables : list : scope : system : const
Date: Sat, 10 May 2003 21:02:48 +0200
Sender: [email-masked]

--Boundary-02=_pzUv+gvuBGZvi5U
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

<snip>

Let's defer list variables for the time being. String variables are=20
useful enough in itself to get this out as RFC rather sooner; the more=20
so as other drafts in the pipeline probably want to make use of it.

We can always come back to try hammering out a listvariables extension=20
later, if anyone cares enough.

Marc

=2D-=20

--Boundary-02=_pzUv+gvuBGZvi5U
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_pzUv+gvuBGZvi5U--

Date: Sat, 10 May 2003 12:36:19 -0700 (PDT)
From: [email-masked]
Subject: Re: Limitation of editheader
To: Jutta Degener <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> On Sat, May 03, 2003 at 02:50:05PM -0700, [email-masked] wrote
> in response to Simon Josefsson:

> > Prepending would be a better default.

> The application I had in mind for "addheader" is very simple
> flagging or priorization of messages by a user.  Just going by
> "the important stuff first" aesthetics, I'd place such user
> headers after the standard message headers, certainly not in
> the middle of the Received: chronicle of delivery, even though
> that would be temporally accurate.

> But it's correct that both Resent-* and adding of Received:
> headers don't work this way, and that it may be useful to
> see where in the delivery process a header was inserted.

> How about I change the default to insertion in front, and add
> an optional :last that forces an append?

Works for me.

> I've thought about adding more detailed positional parameters
> to addheader, but couldn't come up with anything that wasn't
> too complicated given the rarity of its use.

Exactly right IMO.

				Ned

Date: Sat, 10 May 2003 11:28:56 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

set "var" "meep";

Exactly.

				Ned

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: Re: Variables : list : scope : system : const
Date: Sat, 10 May 2003 19:36:24 +0200
Sender: [email-masked]

--Boundary-02=_9iTv+KsYhf1pHgw
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

> >  >   ["@{var} @{var}!"] =3D> ["meep", " ", "meep", "!"]

  [ "@{var} @{var}!" ] =3D> [ "meep meep!" ].
    =3D> [ "foo foo!", "foo bar!", "bar foo!", "bar bar!" ]

<snip>

That would introduce a catch22:
=2D You need the grammar to parse "require" commands
=2D You need to parse the "require" commands to find out the currently
  valid grammar.

At least in my implementation the grammar is hard-coded and any=20
extension-dependent code kicks in after parsing into=20
command/test/command-list/etc has taken place.

Marc

=2D-=20

--Boundary-02=_9iTv+KsYhf1pHgw
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_9iTv+KsYhf1pHgw--

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 15:07:35 +0200
Sender: [email-masked]

-- 
Kjetil T.

Date: Sat, 10 May 2003 02:01:19 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

Correct.

				Ned

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 04:36:32 +0200
Sender: [email-masked]

my current suggestion:

  set "var" "meep";
  set "var" ["meep"];

I agree.

-- 
Kjetil T.

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: Sat, 10 May 2003 04:14:45 +0200
Sender: [email-masked]

[Marc Mutz]:
>
>   I want a single syntactical element for a common concept instead
>   of every extension brewing it's own thing. (Notify: $subject$ etc
>   substitution in the :message text; Imapflags: :(global)flags;
>   Variables: setdate).
>
>   The connection is the general variables syntax: ${}.

I see.

>   This might work - for imapflags. What about ${#currentdate},
>   ${#size} (see below).

these aren't suitable for a list type, are they?  list variables are
useful in other contexts as well, like

   set "hdrs" ["To", "Cc"];
   set "domains" ["njus.no", "usenet.no", "uio.no"];
   if allof (address :localpart ["@{hdrs}"] "kjetilho",
             address :domain ["@{hdrs}"] ["@{domains}"]) { ... }
   if allof (address :localpart ["@{hdrs}"] "postmaster",
             address :domain ["@{hdrs}"] ["@{domains}"]) { ... }

("@" may not be the best character, but I'll just leave it for now as
an example.)

>   And what about initial content?

an empty list seems best.  interestingly, since this is an error:
   set "foo" [ ];
you'd need to
   set "foo" [ "@{undef}" ];
to reset foo to the empty list.

>   <sidenote>
>   I don't understand why imapflags requires it's internal set to be empty 
>   on startup. Wouldn't it be more useful to allow implementations to 
>   preload the set with the current flags? E.g. when applying scripts to a 
>   selection of messages from an IMAP folder or in sequential script 
>   ("multiscript" environment) execution?
>   </sidenote>

your view sounds reasonable to me.

>> 2)
>> part of me thinks strings will work OK.  it just needs careful
>> programming to keep everything padded with spaces.
>>
>>   set "flags" "${flags} \\Flagged ";  # note trailing space
>>   if string :matches "${flags}" "* \\Flagged *" {
>>     set "flags" "${1} ${2}";
>>   }
>
>   Ugh. Too subtle. It's a open invitation for bugs in scripts.

possibly.  if this code is given in the documentation, that would
reduce the error rate quite a bit.  btw, it's wrong to blindly add a
flag, you need to make sure there are no duplicates (or else removing
flags becomes very difficult).  shame on me ...

>   >   # now we want to restore the flags to the values before the if.
>   >   # or do we?
>   <snip>

you snipped my "sorry, I can't come up with a credible example", which
is what "or do we" is about.  why would we ever need to restore the
flags?

>   My feeling is that trying to use an unknown timezone should flag
>   an error.

perhaps that is best.  in the usual case, the argument to SETDATE is a
static string, and the error can be flagged at upload, which is nice.
I'll add that as a SHOULD in my draft.

>> >   Together with the relational extension and a rfc822-date
>> >   comparator,
>>
>> please make it an ISO 8601 date comparator ...
>   <snip>
>
>   But rfc822 messages have rfc822 dates? Or is it the engine's job
>   to convert the string representation of the dates into ISO 8601
>   strings?

it is the engine's job to normalise the string representation into one
common format.  but forget it, an ISO 8601 date comparator is not
needed, since a normal string compare is enough.

>   > I think your date test and SETDATE can co-exist.
>
>   I don't like setdate polluting the "user namespace" with unused 
>   variables. How many people will need ${second} compared to ${year} and 
>   ${month}? If setdate remains, I'd like to see it write to ${#year} etc, 
>   but I'd really prefer to have it's functionality go into a date test, 
>   else why do we have a setdate and no setenvelope, setaddress, 
>   setheader, setsize[1]? With a date test, all but the last are equally 
>   superfluous. Should every foo test have a corresponding setfoo action 
>   that optimizes away the otherwise necessary matches-set idioms?

that's the big question, isn't it?  the matches-set idiom seems
clunky, a hack.  I don't want to base the design of the language on
it.

>   It's simply needed. Size is just another example[2]. Whether it's good 
>   or not that there are such variables, they're (implicitly) there even 
>   in the Sieve base spec.
>
>   [2] And one which the variables spec needs to define, so it
>   addresses Ned's concerns about defining a concept and not
>   instantly making use of it.

which concept?

>   What do you mean with "generic mechanism"? Isn't "system variable"
>   such a generic mechanism?

every system variable will behave differently.  I don't think that's
"generic".

-- 
Kjetil T.			|  read and make up your own mind
				|  http://www.cactus48.com/truth.html

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Thu, 8 May 2003 21:46:40 +0200
Sender: [email-masked]

--Boundary-02=_uQru+FSXtWWEkd6
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Wednesday 07 May 2003 22:59, Kjetil Torgrim Homme wrote:
<snip>
> I agree [system variables] need to be recognisable by syntax alone.
> one type of prefix could be "date." or "imap.", ie. the distinguishing
> feature is the presence of one or more periods.
<snip>

That was my first reaction, too, but I thought that would be too verbose=20
and I remembered there was opposition to "named namespaces", so I chose=20
a single character prefix.

> > there was objection
> > to define extensions that have any kind of "internal variable" (as
> > imapflags once called it) without first having variables for Sieve
> > in general.
>
> I don't understand this reasoning.  you want "internal variables"
> with properties very different from those of the general variables.=20
> where is the connection between the features?
<snip>

I want a single syntactical element for a common concept instead of=20
every extension brewing it's own thing. (Notify: $subject$ etc=20
substitution in the :message text; Imapflags: :(global)flags;=20
Variables: setdate).

The connection is the general variables syntax: ${}.

> ok.  I honestly don't know what's the best way to handle the IMAP
> flags.
>
> 1)
> part of me wants to add a list variable type.  the main sticky point
> is the interpolation.  my suggestion was:
>
>   set "foo" [ "one", "two", "three" ];
>   set "bar" [ "${foo}", "four" ];
>   set "zot" [ "woo${foo}hoo" ];
>
> now ${bar} is [ "one", "two", "three", "four" ] and ${zot} is [
> "woo", "one", "two", "three", "four", "hoo" ].  this makes
> "${foo}${bar}" equivalent with [ "${foo}", "${bar}" ].
<snip failure discussion>
> string with a single space as delimiter or something.

This might work - for imapflags. What about ${#currentdate}, ${#size}=20
(see below). And what about initial content?
<sidenote>
I don't understand why imapflags requires it's internal set to be empty=20
on startup. Wouldn't it be more useful to allow implementations to=20
preload the set with the current flags? E.g. when applying scripts to a=20
selection of messages from an IMAP folder or in sequential script=20
("multiscript" environment) execution?
</sidenote>

> 2)
> part of me thinks strings will work OK.  it just needs careful
> programming to keep everything padded with spaces.
>
>   set "flags" "${flags} \\Flagged ";  # note trailing space
>   if string :matches "${flags}" "* \\Flagged *" {
>     set "flags" "${1} ${2}";
>   }

Ugh. Too subtle. It's a open invitation for bugs in scripts.

> 3)
> part of me thinks an intrinsic variable is the solution which appears
> cleanest to the user.
>
> why do you need variables at all for IMAP flags?  to keep state, it
> seems.

Yes. And I want to use a system variable for that because that's the=20
_concept_ behind it. A new imapflags draft revision can then say:
"fileinto uses the current content of the ${#imapflags} system variable=20
as the set of flags to use" or "Scoping? See variables!" So this:

>   if something {
>      addflag "\\Flagged";
>      if somethingelse {
>         addflag "Big";
>         keep :globalflags;
>      }
>   }
>   # now we want to restore the flags to the values before the if.
>   # or do we?
<snip>

Won't confuse anyone anymore, since the set of flags is scoped like any=20
other variable (ie. globally - currently). If you want to restore the=20
imapflags from before the first if, save them away in a user variable=20
and restore them after the fact:

set "myimapflags" "${#imapflags}";
if .... {
  ....
}
setflags "${myimapflags}";

> >   if date :current :timezone "+0200" :hour :matches "*" {
> >     set "${hour}" "${1}";
> >   }
>
> you won't get daylight savings time handled correctly this way.
> compare with
>
>   setdate "Europe/Oslo";
>   if not string "${timezone}" "Europe/Oslo" {
>     setdate "+0100";
>   }
>
> ie., use Europe/Oslo if the server recognises the time zone,
> otherwise fall back to +0100.

My feeling is that trying to use an unknown timezone should flag an=20
error.

I used :numericzone to work around the tagged argument name clash. Maybe =20
instead of the :timezone parameter there should be a ${#timezone}=20
variable that can be altered by a settimezone action?

settimezone "Europe/Berlin";
if not string "${#timezone}" "Europe/Berlin" {
  settimezone "+0100";
}

That introduces the question of which date tests should honour the=20
timezone? Maybe all, since you can always say:

set "mytimezone" "${#timezone}";
settimezone ""; # use local (:current) or implied (else) timezone
if date :comparator "i;ascii-numeric" :sent :hour :value "lt" "6" {
  set "prefix" "Oh, my. Nightshift again? ";
}
vacation
    "${prefix}I'm on vacation (Hawaii :-P) until the end of the month.";
settimezone "${mytimezone}";

> >   where the date test syntax would be:
> >
> >   date [COMPARATOR] WHICH [TIMEZONE] [DATE-PART] [MATCH-TYPE]
> >        <key-list: string-list>
> >
> >   WHICH :=3D ":current" / ":received" / ":sent"
> >            / ( ":header" <headerfield: string-list> )
> >   DATE-PART :=3D ":weekday" / ":day" / ":month" / ":year"
> >            / ":hour" / ":minute" / ":second" / ":numericzone"
> >            / ":all" / ... ; ":all" is default
> >   TIMEZONE :=3D ":timezone" <timezone: string>
> >        ; server's local timezone is default for :current,
> >          timezone specified in the header is default for :received,
> > :sent and :header.
>
> I think this is nice.  but for a user simply wanting to save his mail
> in different folders depending on month would write
>
>   setdate;
>   fileinto "INBOX.${year}-${month}";
>
> which now becomes
>
>   if date :current :year :matches "*" {
>      set "year" "${1}";
>   }
>   if date :current :month :matches "*" {
>      fileinfo "INBOX.${year}-${1}";
>   }
>
> >   Together with the relational extension and a rfc822-date
> >   comparator,
>
> please make it an ISO 8601 date comparator ...
<snip>

But rfc822 messages have rfc822 dates? Or is it the engine's job to=20
convert the string representation of the dates into ISO 8601 strings? I=20
think not, since it's also not the engine's job to convert strings to=20
numbers before handing them off to i;ascii-numeric.

> the existence of a global "setdate" script is not guaranteed.  isn't
> it better to make the support for this certain?

It would be cute to have a "sieve stdlib". However, just providing this=20
script on the homepage or as an appendix to the variables rfc would=20
probably suffice. Those that want it can either include it or=20
cut'n'paste it into their scripts.

> I think your date=20
> test and SETDATE can co-exist.
<snip>

I don't like setdate polluting the "user namespace" with unused=20
variables. How many people will need ${second} compared to ${year} and=20
${month}? If setdate remains, I'd like to see it write to ${#year} etc,=20
but I'd really prefer to have it's functionality go into a date test,=20
else why do we have a setdate and no setenvelope, setaddress,=20
setheader, setsize[1]? With a date test, all but the last are equally=20
superfluous. Should every foo test have a corresponding setfoo action=20
that optimizes away the otherwise necessary matches-set idioms?

[1] This might even be useful, since the relational extension failed to=20
redefine the size test to take a [MATCH-TYPE] parameter, so you=20
_cannot_ say
  if size :matches "*" {
    set "size" "${1}";
  }
So ${#size} perfectly fits the definition of a system variable :-):

It affects the size test, ...
> >   is read-only, it's natural representation is not a string,
> >   it's value can't be computed with Sieve and it can't be extracted
> >   from the message.
>
> but is this a good thing,

It's simply needed. Size is just another example[2]. Whether it's good=20
or not that there are such variables, they're (implicitly) there even=20
in the Sieve base spec.

[2] And one which the variables spec needs to define, so it addresses=20
Ned's concerns about defining a concept and not instantly making use of=20
it.

> or do we want to make generic mechanisms=20
> instead?

What do you mean with "generic mechanism"? Isn't "system variable" such=20
a generic mechanism?

Marc

=2D-=20
I am Bush of USA. You will be pacified. Resistance is futile.

--Boundary-02=_uQru+FSXtWWEkd6
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_uQru+FSXtWWEkd6--

Date: Fri, 09 May 2003 08:47:21 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: [email-masked]
Cc: Kjetil Torgrim Homme <[email-masked]>, [email-masked]
Sender: [email-masked]

...

				Ned

Date: Fri, 09 May 2003 08:21:10 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

				Ned

To: [email-masked]
Subject: Re: Draft for script 'include' capability
From: Kjetil Torgrim Homme <[email-masked]>
Date: 09 May 2003 14:13:16 +0200
Lines: 38
Sender: [email-masked]

[Marc Mutz]:
>
>   I think that the draft should forbid recursion in the description
>   of the include action, where it talks about policy limits on
>   nesting depth. I don't see the point in having it the Security
>   Considerations section.  I'd also make it explicit that a script
>   MUST NOT include itself:

spoilsport!  how am I going to do Towers of Hanoi in Sieve now?

oh well, I guess I must support your suggestion...

>   [nifty code snipped]

I'm glad to see :length being put to good use :-)

> > another issue is case-sensitivity for filenames.  this is glossed
> > over in fileinto as well, but I think it should be stated explicitly.
> > something like:
> >
> >     The filename is case sensitive.  An implementation MAY refuse to
> >     allow two files whose names only differ in case.
> >
> > actually, this should go into the Managesieve draft as well.
> 
> I'd prefer to leave that up to the implementation since it touches
> on OS or even filesystem/database dependence.

that's my point.  if we don't say anything, the script generator must
assume the lowest common denominator, which is case-sensitive (Unix)
and upper-case (DOS and others).  perhaps even more restrictions must
be put in place (allowable characters, length).  passing the work of
finding this lowest common denominator on the script writer is wrong,
I think the draft needs to put in some ground rules for what
implementations _must_ support.

-- 
Kjetil T.

To: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Kjetil Torgrim Homme <[email-masked]>
Date: 09 May 2003 13:55:59 +0200
Lines: 106
Sender: [email-masked]

I agree.

what do you mean?

-- 
Kjetil T.

To: Jutta Degener <[email-masked]>
Cc: [email-masked]
Subject: Re: Variables : list : scope : system : const
From: Simon Josefsson <[email-masked]>
Date: Fri, 09 May 2003 00:40:27 +0200
Sender: [email-masked]

Date: Thu, 8 May 2003 14:33:16 -0700
From: Jutta Degener <[email-masked]>
To: [email-masked]
Subject: Re: Variables : list : scope : system : const
Sender: [email-masked]

Jutta

Date: Thu, 08 May 2003 11:13:34 -0700 (PDT)
From: [email-masked]
Subject: Re: Variables : list : scope : system : const
To: Nigel Swinson <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

				Ned

From: "Nigel Swinson" <[email-masked]>
To: <[email-masked]>
Subject: Variables : list : scope : system : const 
Date: Thu, 8 May 2003 18:49:04 +0100
Sender: [email-masked]

Nigel

From: Marc Mutz <[email-masked]>
Organization: KDE
To: [email-masked]
Subject: Re: Draft for script 'include' capability
Date: Wed, 7 May 2003 18:06:18 +0200
Sender: [email-masked]

--Boundary-02=_K8Su+m93ikRFVtT
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

Hi!

I think that the draft should forbid recursion in the description of the=20
include action, where it talks about policy limits on nesting depth. I=20
don't see the point in having it the Security Considerations section.=20
I'd also make it explicit that a script MUST NOT include itself:

    SIEVE implementations MUST ensure that recursive includes are not
=2D   possible. i.e. if script "A" includes script "B", and script "B"
=2D   includes script "A" an error MUST be generated either when the
+   possible. e.g. if script "A" includes script "B", and script "B"
+   includes script "A" or a script includes itself an error MUST be
+   generated either when the
    script is uploaded to the SIEVE repository, or when the script is
    executed.  If such an error is detected whilst processing a SIEVE
    script, an implicit "keep" action MUST be executed to prevent loss
    of any messages.

Just for fun, here's a fact() "function" implemented with "variables"=20
and "include" (with only include-self allowed) :-)

=2D-usage--
if header :matches "X-SPAM-LEVEL" "*" {
  set "arg" "${1}";
}
include "fact";
addheader "X-SPAM-FACULTY-OF-LEVEL" "${fact}";
=2D-end usage--

=2D-script fact--
require [ "variables", "include" ];

# returns in ${fact} the faculty of ${arg} (expressed as the length of
# the string: "#####" =3D 5, "foo" =3D 3, etc. ${fact} is a "real" integer

if anyof( string "${arg}" "", string :matches "${arg}" "?" ) {
  set "arg" "#";  # 0! =3D 1! =3D 1
  set "fact" "1";
  return;
} else {
  # return arg * fact( arg - 1 )
  set "arg1" "${arg}";
  # --arg
  if string :matches "?*" "${arg}"; {
    set "arg" ${2};
  }
  # arg =3D fact( arg )
  include "fact";
  # result =3D arg1 * arg
  set "arg2" "${arg}";
  include "multiply";
  set "arg" "${result}";
  set :length "fact" "${arg};
  return;
}
=2D-end script fact--

=2D-script multiply--
require [ "variables", "include" ];

# returns a string in ${result} that contains ${arg1} * ${arg2} times
# the hash # character:

if string "${arg1}" "" {
  set "result" "";
} elsif string :matches "${arg1}" "?" ) {
  set "result" "${arg2}";
} else {
  # --arg1:
  if string :matches "?*" "${arg1}" {
    set "arg1" "${2}";
  }
  include "multiply";
  # arg1 * arg2 =3D=3D ( arg1 - 1 ) * arg2 + arg2
  set "result" "${result}${arg2}"
}
=2D-end script multiply--

On Monday 14 April 2003 05:20, Kjetil Torgrim Homme wrote:
<snip>
> another issue is case-sensitivity for filenames.  this is glossed
> over in fileinto as well, but I think it should be stated explicitly.
> something like:
>
>     The filename is case sensitive.  An implementation MAY refuse to
>     allow two files whose names only differ in case.
>
> actually, this should go into the Managesieve draft as well.

I'd prefer to leave that up to the implementation since it touches on OS=20
or even filesystem/database dependence.

Marc

=2D-=20
If free-software authors lose the right to disclaim all warranties and
find themselves getting sued over the performance of the programs
they've written, they'll stop contributing free software to the world.
 -- Bruce Perens: Open Sources: Voices from the Open Source Revolution

--Boundary-02=_K8Su+m93ikRFVtT
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_K8Su+m93ikRFVtT--

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 07 May 2003 22:59:17 +0200
Sender: [email-masked]

[Marc Mutz]:
>
>   It's a side-effect of executing setdate. I call this a
>   side-effect, since it's not an explicit "set".

alright.

>> (easily fixed, just specify lexical sorting of the "set" at all
>> times.)
>
>   I always wondered why imapflags didn't include a flag test. It
>   should, esp. if one considers the include extension where a
>   included script doesn't know what it's parent has already done and
>   might want to know.

agreed.  the current method is to use a :matches, which will work, but
is a bit roundabout.

>> wouldn't it be better to make generic functions for this?
>>     addflag "flag" "\\Flagged";
>> changes ${flag}.  you'd now use
>>     keep :flags "${flag}";
>> rather than
>>     keep :globalflags;
>> and
>>     keep :flags "${flag} \\Flagged";
>> instead of
>>     keep :globalflags_plus "\\Flagged";
>>
>> some scripts may become more verbose, but the complexity of the
>> IMAPFLAGS is decreased, IMO.  (ADDFLAG and REMOVEFLAG could move to a
>> separate extension.)
>
>   One could always add the feature to accept multiple flags per string 
>   argument, so you can use
>
>   set "myflags" "${#imapflags}" # save away
>   # ...
>   setflags "${myflags}" # restore
>
>   I fact, you already used that feature in the code above.

yep.

>   Also, keep in mind that imapflags implementations already exist
>   and you can imagine what a confusion would arise if addflags' et
>   al. syntax would be changed.

I don't think this is something we should worry too much about.
deploying features before they are standardised is a gamble.

>> >   mark
>> >   - equivalent to addflag "\\Flagged"
>> >
>> >   unmark
>> >   - equivalent to removeflag "\\Flagged"
>>
>> what is the rationale for MARK and UNMARK?  according to the draft
>> these actions are _not_ equivalent to ADDFLAG "\\Flagged".
>
>   But IIRC, it's suggested they are. However, this is nitpicking
>   again, since it's clear that the exact (un)mark semantics are
>   implementation-defined. I've just chosen the obvious one.

I was just wondering why their effect is allowed to be implementation
defined.  I can see that MARK/UNMARK are useful shorthands.

>   Ok, here's what I think are properties of system variables, why
>   they are needed and why they should be prefixed:

I agree they need to be recognisable by syntax alone.  one type of
prefix could be "date." or "imap.", ie. the distinguishing feature is
the presence of one or more periods.

>   3. They may represent concepts that are not naturally represented
>      as strings.
>
>      Examples: ${#imapflags} is a set of imap flags, not a string.
>                ${#currentdate} is a number or struct tm or whatever, not a
>                string.
>
>      There are many issues if you allow ${#imapflags} to be "set" by
>      the user (duplicate or invalid flags, ordering
>      issues). However, it's a variable (remember that imapflags was
>      what prompted a variables extension in the first place, even if
>      you were not personally driven by that) and there was objection
>      to define extensions that have any kind of "internal variable"
>      (as imapflags once called it) without first having variables
>      for Sieve in general.

I don't understand this reasoning.  you want "internal variables" with
properties very different from those of the general variables.  where
is the connection between the features?

>      It is, however, potentially very useful to allow interpolation
>      into strings, and a system variable declaration must specify
>      the string representation to use. I've already given an example
>      for ${#imapflags}, ${#currentdate} could be interpolated in
>      rfc2822 format.
>
>      So system variables could be handled internally in their
>      natural representation, with a string representation being
>      required only on expansion into strings, which should be pretty
>      rare.

ok.  I honestly don't know what's the best way to handle the IMAP
flags.

1)
part of me wants to add a list variable type.  the main sticky point
is the interpolation.  my suggestion was:

  set "foo" [ "one", "two", "three" ];
  set "bar" [ "${foo}", "four" ];
  set "zot" [ "woo${foo}hoo" ];

now ${bar} is [ "one", "two", "three", "four" ] and ${zot} is [ "woo",
"one", "two", "three", "four", "hoo" ].  this makes "${foo}${bar}"
equivalent with [ "${foo}", "${bar}" ].

to go with this, we'd need a prune command to remove an element from a
list.  actually we need two, one by position, and one by value.  for
imapflags, an append command would be useful as well (it's not worth
worrying about duplicates.)

one problem with this is run-time syntax errors, like

  set "folder" [ "foo", "bar" ];
  fileinto "${folder}";

2)
part of me thinks strings will work OK.  it just needs careful
programming to keep everything padded with spaces.

  set "flags" "${flags} \\Flagged ";  # note trailing space
  if string :matches "${flags}" "* \\Flagged *" {
    set "flags" "${1} ${2}";
  }

3)
part of me thinks an intrinsic variable is the solution which appears
cleanest to the user.

why do you need variables at all for IMAP flags?  to keep state, it
seems.

  if something {
     addflag "\\Flagged";
     if somethingelse {
        addflag "Big";
        keep :globalflags;
     }
  }
  # now we want to restore the flags to the values before the if.
  # or do we?

sorry, I can't come up with a credible example.

>   4. Their values may not be computed with Sieve.
>
>      You said setdate would be a macro for a list of set's. But it
>      isn't.  You can't represent setdate "+0200" like that.

I meant as seen from an optimising Sieve parser, not from a user
perspective.  it wasn't a good comparison, anyway.

>      E.g. a hypothetical ${#envelope_sender} system variable would
>      not be a good idea, since you can always obtain that by using
>      the matches-set idiom:
>        if :envelope :all :matches "from" "*" {
>          set "myenvelopesender" "${1}";
>        }

it seems to me Sieve scripts will soon consist of mostly matches-set
idioms ... ;-)

>   5. Their values may not be available from the message.
>
>      Example: ${#currentdate}
>
>   I now agree that ${year} as specified is not a proper system
>   variable.
>
>   Here's a proposal for a date test (which is important to have in
>   it own right) that makes setdate superfluous by using the
>   matches-set idiom to explicitly set the setdate variables:
>
>   if date :current :timezone "+0200" :hour :matches "*" {
>     set "${hour}" "${1}";
>   }

you won't get daylight savings time handled correctly this way.
compare with

  setdate "Europe/Oslo";
  if not string "${timezone}" "Europe/Oslo" {
    setdate "+0100";
  }

ie., use Europe/Oslo if the server recognises the time zone, otherwise
fall back to +0100.

>   where the date test syntax would be:
>
>   date [COMPARATOR] WHICH [TIMEZONE] [DATE-PART] [MATCH-TYPE]
>        <key-list: string-list>
>
>   WHICH := ":current" / ":received" / ":sent"
>            / ( ":header" <headerfield: string-list> )
>   DATE-PART := ":weekday" / ":day" / ":month" / ":year"
>            / ":hour" / ":minute" / ":second" / ":numericzone"
>            / ":all" / ... ; ":all" is default
>   TIMEZONE := ":timezone" <timezone: string>
>        ; server's local timezone is default for :current,
>          timezone specified in the header is default for :received, :sent
>          and :header.

I think this is nice.  but for a user simply wanting to save his mail
in different folders depending on month would write

  setdate;
  fileinto "INBOX.${year}-${month}";

which now becomes

  if date :current :year :matches "*" {
     set "year" "${1}";
  }
  if date :current :month :matches "*" {
     fileinfo "INBOX.${year}-${1}";
  }

>   Together with the relational extension and a rfc822-date
>   comparator,

please make it an ISO 8601 date comparator ...

>   And for those that want "old" setdate:
>
>     set "setdate_timezone" "CEST"
>     include :global "setdate";
>
>   where "setdate" includes the matches-set idioms:

:-)

the existence of a global "setdate" script is not guaranteed.  isn't
it better to make the support for this certain?  I think your date
test and SETDATE can co-exist.

>   In this scenario, ${#currentdate} would be a system variable with
>   all properties as defined above: It affects the date :current
>   test, is read-only, it's natural representation is not a string,
>   it's value can't be computed with Sieve and it can't be extracted
>   from the message.

but is this a good thing, or do we want to make generic mechanisms
instead?

-- 
Kjetil T.			|  read and make up your own mind
				|  http://www.cactus48.com/truth.html

From: Marc Mutz <[email-masked]>
Organization: "Old Europe" - and proud
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Tue, 6 May 2003 14:27:55 +0200
Sender: [email-masked]

--Boundary-02=_vp6t+tgpaITmhKH
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Monday 05 May 2003 00:19, Kjetil Torgrim Homme wrote:
<snip>
> I don't agree that ${year} is a system variable.

Granted it's not a clean-cut one, but see below.

> >   > I think an explicit action to set them is easier to handle
> >   > than a multitude of variables which can change as side-effects.
> >
> >   But they _do_ change as side effects. You can't go and pretend
> >   they don't.
>
> I wouldn't say it's a side effect, SETDATE is quite explicit.=20
> nothing else will change the variables.
<snip>

It's a side-effect of executing setdate. I call this a side-effect,=20
since it's not an explicit "set".

> >   >   SET "hour" "${_internal_startup_hour}";
<snip>
> ${_internal_startup_hour} doesn't exist...

So how can SET "hour" ${_internal_startup_hour}" work?

> >   > what actions can influence ${_imapflags}?  how will its value
> >   > change?
> >
> >   ${#imapflags} is (if resolved) a space-separated list of imap
> >   flags. The order of flags in ${#imapflags} is undefined (it's
> >   conceptually a set).
>
> so
>
>   if string :matches "${#imapflags}" "*\\Answered*\\Flagged*" {
>     ...
>   }
>
> may or may not match even if both flags are set.

Correct.

> (easily fixed, just=20
> specify lexical sorting of the "set" at all times.)

I always wondered why imapflags didn't include a flag test. It should,=20
esp. if one considers the include extension where a included script=20
doesn't know what it's parent has already done and might want to know.

> >   addflag
> >   - adds the flags given as arg to #imapflags then
> >   - removes all duplicates.
> >
> >   removeflag
> >   - removes all flags given as arg from #imapflags
>
> wouldn't it be better to make generic functions for this?
>     addflag "flag" "\\Flagged";
> changes ${flag}.  you'd now use
>     keep :flags "${flag}";
> rather than
>     keep :globalflags;
> and
>     keep :flags "${flag} \\Flagged";
> instead of
>     keep :globalflags_plus "\\Flagged";
>
> some scripts may become more verbose, but the complexity of the
> IMAPFLAGS is decreased, IMO.  (ADDFLAG and REMOVEFLAG could move to a
> separate extension.)

One could always add the feature to accept multiple flags per string=20
argument, so you can use

set "myflags" "${#imapflags}" # save away
# ...
setflags "${myflags}" # restore

I fact, you already used that feature in the code above. Also, keep in=20
mind that imapflags implementations already exist and you can imagine=20
what a confusion would arise if addflags' et al. syntax would be=20
changed.

> >   setflag
> >   - clears #imapflags
> >   - executes addflag <args>
>
> what would SETFLAG "${#imapflags}" do?  just one of those things
> which need to be clarified.

With the above feature, it would be a no-op.

> >   mark
> >   - equivalent to addflag "\\Flagged"
> >
> >   unmark
> >   - equivalent to removeflag "\\Flagged"
>
> what is the rationale for MARK and UNMARK?  according to the draft
> these actions are _not_ equivalent to ADDFLAG "\\Flagged".

But IIRC, it's suggested they are. However, this is nitpicking again,=20
since it's clear that the exact (un)mark semantics are=20
implementation-defined. I've just chosen the obvious one.

> >   Summary:
> >   There are two issues, let's separate them:
> >   1. Should set be allowed to act on system variables?
> >      I hope to have made it clear that that's a bad idea.
>
> I strongly agree!

:-)

> >   2. Should system variables be prefixed?
> >      Well, why _not_? If it makes set's job easier to determine if
> >      something is read-only or not? If it makes system variable
> >      references more explicit?
>
> I don't want to introduce system variables at all.  I haven't seen a
> compelling case for adding them.

Ok, here's what I think are properties of system variables, why they are=20
needed and why they should be prefixed:

1. System variables are implicitly changed by certain commands and vice
   versa: the value of system variables affects certain commands.

   Examples: ${#imapflags} is changed by {add,remove,set}flags,
             {un,}mark. It implicitly affects keep, fileinto and a
             potential "flags" test.

             A ${#currentdate} variable may affect the to-be-defined
             date test.

   This is the most visible property of system variables, though in
   itself alone it doesn't require system variables to be handled
   specially.

2. They are read-only to "set".

   Examples: ${#imapflags} should be written to only by the imapflags
             actions.
             ${#currentdate} should never be written to.

   So the set action needs to raise an error if the script author tries
   to assign to a system variable. That begs the question of how set
   should know what is a system variable and what isn't. You could keep
   a list of system variables, which is complicated by the fact that (as
   you said) it should be perfectly OK to use the name of a system
   variable as long as the corresponding extension is not "required".
   But apart from the fact that you forgot about the include extension
   (where variables have global, but require'd extensions local scope),
   it would be much easier (for user _and_ engine) if system variables
   had a common prefix to detect them.

3. They may represent concepts that are not naturally represented as
   strings.

   Examples: ${#imapflags} is a set of imap flags, not a string.
             ${#currentdate} is a number or struct tm or whatever, not a
             string.

   There are many issues if you allow ${#imapflags} to be "set" by the
   user (duplicate or invalid flags, ordering issues). However, it's a
   variable (remember that imapflags was what prompted a variables
   extension in the first place, even if you were not personally driven
   by that) and there was objection to define extensions that have any
   kind of "internal variable" (as imapflags once called it) without
   first having variables for Sieve in general.

   It is, however, potentially very useful to allow interpolation into
   strings, and a system variable declaration must specify the string
   representation to use. I've already given an example for
   ${#imapflags}, ${#currentdate} could be interpolated in rfc2822
   format.

   So system variables could be handled internally in their natural
   representation, with a string representation being required only on
   expansion into strings, which should be pretty rare.

4. Their values may not be computed with Sieve.

   You said setdate would be a macro for a list of set's. But it isn't.
   You can't represent setdate "+0200" like that.

   E.g. a hypothetical ${#envelope_sender} system variable would not be
   a good idea, since you can always obtain that by using the
   matches-set idiom:
     if :envelope :all :matches "from" "*" {
       set "myenvelopesender" "${1}";
     }

5. Their values may not be available from the message.

   Example: ${#currentdate}

I now agree that ${year} as specified is not a proper system variable.

Here's a proposal for a date test (which is important to have in it own=20
right) that makes setdate superfluous by using the matches-set idiom to=20
explicitly set the setdate variables:

if date :current :timezone "+0200" :hour :matches "*" {
  set "${hour}" "${1}";
}

where the date test syntax would be:

date [COMPARATOR] WHICH [TIMEZONE] [DATE-PART] [MATCH-TYPE]
     <key-list: string-list>

WHICH :=3D ":current" / ":received" / ":sent"
         / ( ":header" <headerfield: string-list> )
DATE-PART :=3D ":weekday" / ":day" / ":month" / ":year"
         / ":hour" / ":minute" / ":second" / ":numericzone"
         / ":all" / ... ; ":all" is default
TIMEZONE :=3D ":timezone" <timezone: string>
     ; server's local timezone is default for :current,
       timezone specified in the header is default for :received, :sent
       and :header.

Together with the relational extension and a rfc822-date comparator, you=20
could check for dates:

  if allof( date :comparator "rfc822-date"
      :current :value "ge" "Tue, 06 May 2003 14:02:33 +0200",
            date :comparator "rfc822-date"
      :current :all :value "le" "Tue, 13 May 2003 14:04:05 +0200" )
  {
    vacation "blah, blubb";
  }

And for those that want "old" setdate:

  set "setdate_timezone" "CEST"
  include :global "setdate";

where "setdate" includes the matches-set idioms:

=2D-begin datevariables--
require [ "variables", "dates" ];

if date :current :timezone "${datevariables_get_timezone}"
   :year :matches "*" {
  set "year" "${1}";
}
if date ... # and so on for any other setdate variable
=2D-end datevariables--

In this scenario, ${#currentdate} would be a system variable with all=20
properties as defined above: It affects the date :current test, is=20
read-only, it's natural representation is not a string, it's value=20
can't be computed with Sieve and it can't be extracted from the=20
message.

Marc

=2D-=20
Ein Grundrecht auf Sicherheit steht bewusst nicht in der Verfassung.
  -- Sabine Leutheusser-Schnarrenberger (ehem. Bundesjustizministerin)

--Boundary-02=_vp6t+tgpaITmhKH
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_vp6t+tgpaITmhKH--

Date: Mon, 05 May 2003 23:21:04 -0700 (PDT)
From: [email-masked]
Subject: Re: variables: open issue a) date variables
To: Tony Hansen <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> No one but Kjetil responded on this the last time I brought it up. I
> suggested that names be allowed to be compound, and the DATE extension
> put everything into the "time." namespace. So, instead of "${year}",
> we'd have "${time.year}".

I actually prefer a system variable scheme or something similar to this.

				Ned

Date: Mon, 05 May 2003 23:27:13 -0400
From: Tony Hansen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: variables: open issue a) date variables
Sender: [email-masked]

No one but Kjetil responded on this the last time I brought it up. I 
suggested that names be allowed to be compound, and the DATE extension 
put everything into the "time." namespace. So, instead of "${year}", 
we'd have "${time.year}".

	Tony Hansen
	[email-masked]

[email-masked] wrote:
>>On Friday 02 May 2003 01:04, Kjetil Torgrim Homme wrote:
>>Short comment: Define a namespace for magic variables and move the date
>>variables out into the to-be-written draft defining the date test.
> 
> I have no problem with defining a magic variable namespace. In fact
> in some way I'd prefer it. That being said, however, I have a big
> problem with defining it and not using it in the same document. All
> too often this leads to issues being left open and problems not being
> spotted until too late.

To: Jutta Degener <[email-masked]>
Cc: [email-masked]
Subject: Re: Message uniqueness in editheader
From: Simon Josefsson <[email-masked]>
Date: Mon, 05 May 2003 21:25:06 +0200
Sender: [email-masked]

Jutta Degener <[email-masked]> writes:

> The reason that I stayed so vague was worry that a message
> store could consider two messages with the same ID to the
> same user "the same message" and drop one of them.  I wouldn't
> want to disrupt sieve compatibility with such a message store.
> But maybe I should wait until faced with one; I don't know if
> anyone actually does this.

I see.  I think this is a good point.  I believe my Cyrus installation
prune duplicates, based on Message-Id, although it is an extra
feature, not part of the message store.

Perhaps include a discussion of this?  A warning to the user that
filing the message several times, even if they are altered by the
add/delete/replace commands, may not do what she expects, because of
Message-Id duplication checks.

> But rather than writing it like this
>
>> |    A message modified by addheader, deleteheader, or replaceheader
>> |    MUST NOT be considered the same as the original message.
>
> I'd rather go by results and say that a message that is textually
> different from the original is a different message and mustn't be
> dropped.
>
> That would allow a message store to drop the second and following
> of an exactly identical message, which is something I'd still
> like to allow.

Sounds good.

Date: Mon, 5 May 2003 11:42:53 -0700
From: Jutta Degener <[email-masked]>
To: [email-masked]
Subject: Re: Message uniqueness in editheader
Sender: [email-masked]

It's a good idea to supply an example, and I think your
choice of always considering different messages different
is a good one.

The reason that I stayed so vague was worry that a message
store could consider two messages with the same ID to the
same user "the same message" and drop one of them.  I wouldn't
want to disrupt sieve compatibility with such a message store.
But maybe I should wait until faced with one; I don't know if
anyone actually does this.

But rather than writing it like this

> |    A message modified by addheader, deleteheader, or replaceheader
> |    MUST NOT be considered the same as the original message.

I'd rather go by results and say that a message that is textually
different from the original is a different message and mustn't be
dropped.

That would allow a message store to drop the second and following
of an exactly identical message, which is something I'd still
like to allow.

Jutta

Date: Mon, 5 May 2003 11:14:03 -0700
From: Jutta Degener <[email-masked]>
To: [email-masked]
Subject: Re: Editorial editheader comment
Sender: [email-masked]

On Sat, May 03, 2003 at 10:12:18PM +0200, Simon Josefsson wrote:
> 3028 terminology.  3028 sometimes
> uses the word "header" as referring to (in RFC 2822 terminology)
> header fields.  E.g., from "2.4.2.2 Headers":
> 
>    A header name never contains a colon.  The "From" header refers to a
>    line beginning "From:" (or "From   :", etc.).  No header will match
>    the string "From:" due to the trailing colon.
> 
> Given this, it was unclear (to me) which definition was used.  But if
> it is clear for everyone else, ignore my message.

I think you have a point; there's a clash of vocabularies going
on, and a sentence of clarification wouldn't be too much to ask.

+  The term "header field" is used here as in [RFC-2822] to mean
+  a logical line of an email message header.

Privately, I will almost always use "header" to mean a line of an
email message header, just like RFC 3028 does.  I did that unthinkingly
and consistently in the very first internal version of the draft, but
was then asked by one of my internal reviewers to use the more accurate,
but to me unfamiliar, "header field" language from RFC 2822.

It is a source of quiet satisfaction to me that this has now created
misunderstandings in the _other_ direction.

Jutta

To: [email-masked]
Cc: [email-masked]
Subject: Re: Limitation of editheader
From: Simon Josefsson <[email-masked]>
Date: Mon, 05 May 2003 19:58:31 +0200
Sender: [email-masked]

[email-masked] writes:

>> The draft says:
>
>> ,----
>> | 3. Action addheader
>> |
>> |    Syntax:
>> |    	"addheader" <name: string> <value: string>
>> |
>> |    The addheader action appends a header field to the existing
>> |    message header.
>> `----
>
> Prepending would be a better default.

This sounds like a simple and reasonable solution; I'm for it.

I didn't think of this simple solution, so instead I imagined
something similar to deleteheader's :index and :last.  The addheader
:index would indicate the position the new header would be inserted,
at, and :last would start counting from the end instead.  This
solution appears more flexible, but I can't come up with an example
where prepending doesn't work.  I'm not convinced there aren't any,
though.

Date: Mon, 5 May 2003 10:48:28 -0700
From: Jutta Degener <[email-masked]>
To: [email-masked]
Subject: Re: Limitation of editheader
Sender: [email-masked]

On Sat, May 03, 2003 at 02:50:05PM -0700, [email-masked] wrote
in response to Simon Josefsson:

> Prepending would be a better default.

The application I had in mind for "addheader" is very simple
flagging or priorization of messages by a user.  Just going by
"the important stuff first" aesthetics, I'd place such user
headers after the standard message headers, certainly not in
the middle of the Received: chronicle of delivery, even though
that would be temporally accurate.

But it's correct that both Resent-* and adding of Received:
headers don't work this way, and that it may be useful to
see where in the delivery process a header was inserted.

How about I change the default to insertion in front, and add
an optional :last that forces an append?

I've thought about adding more detailed positional parameters
to addheader, but couldn't come up with anything that wasn't
too complicated given the rarity of its use.

Jutta

Date: Mon, 05 May 2003 08:12:38 -0700 (PDT)
From: [email-masked]
Subject: Re: Last Call: Sieve -- Subaddress Extension to Proposed Standard
To: Ken Murchison <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> I was just asked what would happen if this extension is used on an
> address such as:

> [email-masked]

> I'm proposing adding the following text to the draft:

> "NOTE: If the separator character occurs more than once in the
> local-part,
> then the address SHOULD be split at the left-most separator."

> Does this seem sufficient?  Should this be a MUST instead of a SHOULD?
> Any other thoughts?

SHOULD seems sufficient. I see no reason we cannot add this during the
author 48 hour editing pass.

				Ned

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 05 May 2003 00:19:13 +0200
Lines: 103
Sender: [email-masked]

[Marc Mutz]:
>
>   Come on. I thought we agreed that writing to system variables is a
>   bad idea...

I don't agree that ${year} is a system variable.

>   > I think an explicit action to set them is easier to handle 
>   > than a multitude of variables which can change as side-effects.
>   
>   But they _do_ change as side effects. You can't go and pretend
>   they don't.

I wouldn't say it's a side effect, SETDATE is quite explicit.  nothing
else will change the variables.

>   > we've already seen that this is a tricky issue with the match
>   > back references.  there the gain for easy access seems great
>   > enough to be worth it.  SETDATE can be seen as a macro for
>   >
>   >   SET "hour" "${_internal_startup_hour}";
>   >   SET "minute" "${_internal_startup_minute}";
>   >   SET "second" "${_internal_startup_second}";
>   >   ...
>   >
>   > pretty straightforward to analyse, IMO.
>   
>   So what should ${_internal_startup_hour} be other than a system
>   variable?

${_internal_startup_hour} doesn't exist...

>   > what actions can influence ${_imapflags}?  how will its value
>   > change?
>   
>   ${#imapflags} is (if resolved) a space-separated list of imap
>   flags. The order of flags in ${#imapflags} is undefined (it's
>   conceptually a set).

so

  if string :matches "${#imapflags}" "*\\Answered*\\Flagged*" {
    ...
  }

may or may not match even if both flags are set.  (easily fixed, just
specify lexical sorting of the "set" at all times.)

>   addflag
>   - adds the flags given as arg to #imapflags then
>   - removes all duplicates.
>   
>   removeflag
>   - removes all flags given as arg from #imapflags

wouldn't it be better to make generic functions for this?
    addflag "flag" "\\Flagged";
changes ${flag}.  you'd now use
    keep :flags "${flag}";
rather than
    keep :globalflags;
and
    keep :flags "${flag} \\Flagged";
instead of
    keep :globalflags_plus "\\Flagged";

some scripts may become more verbose, but the complexity of the
IMAPFLAGS is decreased, IMO.  (ADDFLAG and REMOVEFLAG could move to a
separate extension.)

>   setflag
>   - clears #imapflags
>   - executes addflag <args>

what would SETFLAG "${#imapflags}" do?  just one of those things which
need to be clarified.

>   mark
>   - equivalent to addflag "\\Flagged"
>   
>   unmark
>   - equivalent to removeflag "\\Flagged"

what is the rationale for MARK and UNMARK?  according to the draft
these actions are _not_ equivalent to ADDFLAG "\\Flagged".

>   Summary:
>   There are two issues, let's separate them:
>   1. Should set be allowed to act on system variables?
>      I hope to have made it clear that that's a bad idea.

I strongly agree!

>   2. Should system variables be prefixed?
>      Well, why _not_? If it makes set's job easier to determine if
>      something is read-only or not? If it makes system variable
>      references more explicit?

I don't want to introduce system variables at all.  I haven't seen a
compelling case for adding them.

-- 
Kjetil T.

From: Marc Mutz <[email-masked]>
Organization: "Old Europe" - and proud
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Sun, 4 May 2003 14:41:03 +0200
Sender: [email-masked]

--Boundary-02=_+pQt+IvIdQaiIWs
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Sunday 04 May 2003 00:15, Kjetil Torgrim Homme wrote:
<snip>
> if you want a prefix character, use "_".  it's part of the identifier
> grammar, and has seen similar use in other contexts.

"_" is as good as "#". I've chosen "#" b/c of the #namespace convention=20
of IMAP and #... would be a kind of system namespace, like std:: in=20
c++.

I don't see why variable names have to match the identifier production.=20
num-variable certainly doesn't, either.

> >   The reasons against magic/system variables Kjetil mentioned were:
> >
> >   1. Year, month, etc. are not really magic, you can write them via
> > set and they only spring into life on setdate.
> >
> >   I think we agree that writing to those variables is not a good
> >   idea, since it's not needed, makes program flow analysis more
> >   complex
>
> really?

Come one. I thought we agreed that writing to system variables is a bad=20
idea...

> I think an explicit action to set them is easier to handle=20
> than a multitude of variables which can change as side-effects.

But they _do_ change as side effects. You can't go and pretend they=20
don't.

> we've already seen that this is a tricky issue with the match back
> references.  there the gain for easy access seems great enough to be
> worth it.  SETDATE can be seen as a macro for
>
>   SET "hour" "${_internal_startup_hour}";
>   SET "minute" "${_internal_startup_minute}";
>   SET "second" "${_internal_startup_second}";
>   ...
>
> pretty straightforward to analyse, IMO.

So what should ${_internal_startup_hour} be other than a system=20
variable? Also, how do the equivalent set commands look like for
  setdate "CEST";
? ;-)

> what actions can influence ${_imapflags}?  how will its value change?
<snip>

${#imapflags} is (if resolved) a space-separated list of imap flags. The=20
order of flags in ${#imapflags} is undefined (it's conceptually a set).

addflag
=2D adds the flags given as arg to #imapflags then
=2D removes all duplicates.

removeflag
=2D removes all flags given as arg from #imapflags

setflag
=2D clears #imapflags
=2D executes addflag <args>

mark
=2D equivalent to addflag "\\Flagged"

unmark
=2D equivalent to removeflag "\\Flagged"

> >   Furthermore, that they spring into life as a side-effect of a
> >   command is proof, not disproof, of their magicness.
>
> see macro comment.

The macro comment only shifts the problem from ${year} to=20
${_internal_startup_year}. Instead of ${year}, you make=20
${_internal_startup_year} magic.

> >   3. We'd need a registry for them, since extensions could
> > conflict.
> >
> >   They can still conflict if we don't make system variables
> > explicit in the variables extension. Prefixing them at least avoids
> > conflicts with user variables.
>
> no, there can be no conflict, two extensions can use the same
> variable name and co-exist peacefully. e.g., you could have two=20
> actions putting its return value in ${result}.  the user would just
> have to do a SET to save it in another variable before doing the next
> action, and he only has to do that if he needs to use the two actions
> in the same Sieve run.  with your truly magic variables, this can't
> be done.
<snip>

Granted, but where's the usefulness in allowing that in the first place?=20
I don't see why such behaviour is needed, the more so as it's very=20
inconvenient for the user (oh - and GUI editors!) to have to save away=20
${result} themselves.

Summary:
There are two issues, let's separate them:
1. Should set be allowed to act on system variables?
   I hope to have made it clear that that's a bad idea.
2. Should system variables be prefixed?
   Well, why _not_? If it makes set's job easier to determine if
   something is read-only or not? If it makes system variable references
   more explicit?

Marc

=2D-=20
I am Bush of USA. You will be pacified. Resistance is futile.

--Boundary-02=_+pQt+IvIdQaiIWs
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_+pQt+IvIdQaiIWs--

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 04 May 2003 02:06:30 +0200
Lines: 48
Sender: [email-masked]

[Ned Freed]:
>
>   > do you mean a variable containing the complete
>   > "2003-05-03T05:03:12.00+02:00" ?  (I think I would leave out the
>   > time zone for this application, actually.)
>   
>   Yes, that's what I meant.

okay, here's what I got now:

-----
   The action setdate initialises a few variables:

      ${year}     => the year, "0000" .. "9999"
      ${month}    => the month, "01" .. "12"
      ${day}      => the day, "01" .. "31"
      ${hour}     => the hour, "00" .. "23"
      ${minute}   => the minute, "00" .. "59"
|     ${second}   => the second, "00" .. "60",
+                    60 is only used for leap seconds, see [TIMESTAMPS].
+     ${weekday}  => the day of week, "1" .. "7",
+                    1 is Monday
+     ${week}     => the week, "01" .. "53",
+                    numbered according to [ISO8601]
+     ${weekyear} => the year ${week} is part of, "0000" .. "9999",
+                    according to [ISO8601].  (e.g, 2000-01-01 is
+                    in week 52 of 1999)
+     ${date}     => a lexically ordered timestamp.  Using the above
+                    definitions, it is set to the expansion of
+                    "${year}-${month}-${day}T${hour}:${minute}:${second}".
      ${timezone} => the time zone in use.  If the user specified a
                     time zone which was recognised, ${timezone} will
                     contain the name given.  Otherwise, the value
                     MUST be the server's default time zone in offset
                     format.
-----

I changed the name of "isoyear" to "weekyear", I think that's more
accurate.  after all, the year in ISO 8601 is the same as ${year}.
it's only in week context it's different.

>   It isn't clear to me that weeks are referred to by number enough
>   to matter.  At least not in the US.

week numbers are used quite a bit here in Europe.

-- 
Kjetil T.

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 04 May 2003 00:57:12 +0200
Lines: 50
Sender: [email-masked]

[Marc Mutz]:
>
>   So you say that
>     set "year" "${year} AD";
>   is possible, but that
>     set "imapflags" "${imapflags} \\Seen";
>   is too magic and writing to system variables should be disallowed?

I don't like that writing to some variables has side effects.  it
seems to me that you want this SET to actually change the global IMAP
flags?  I want to require an explicit action to do so.  side effects
must be kept to a minimum.

>   I agree with the latter, but not with the former. I don't see the 
>   usefulness of
>     set "year" "${year} AD";

me neither :-)

perhaps if you want to change ${year} to use the Mayan calendar?
(that'd be fun -- lots of regexp matching should do it :-)

>   since
>   1. It can lead to hard-to-find bugs if, in a large script, you 
>      alter the value of "year" and way down the script test against
>      ${year}.

don't do that then.

my point is that the user can use ${year} himself if he doesn't use
SETDATE.  e.g.,

    if header :matches "Date" "*, * * * *:*:* *" {
        set "day" "${2}";
        set "year" "${4}";
        # ...
    }

this is particularily important since SETDATE is part of the
"variables" extension, but I don't like to add "reserved variables"
when we don't need to, and I'm not convinced we do.

>   and more importantly,
>   2. What does the above solve that can't be solved by
>        set "myyear" "${year} AD";

nothing.

-- 
Kjetil T.

Date: Sat, 03 May 2003 15:10:54 -0700 (PDT)
From: [email-masked]
Subject: Re: Three new drafts and a question
To: Marc Mutz <[email-masked]>
Cc: [email-masked]
Content-description: signed data
Sender: [email-masked]

> On Wednesday 30 April 2003 01:22, [email-masked] wrote:
> <snip>
> > That's not an excuse for turning sieve so heavy that many if not most
> > service providers will be unwilling to provide support for it.
> <snip>

> Not every ISP will want to support all Sieve extensions. They don't need
> to. Many ISP will probably refrain from allowing regex use, of
> variables, for that matter. I think it's perfectly OK to create Sieve
> extensions that are explicitely not suitable for all Sieve use cases.
> E.g. the envelope extension can't be used in MUAs since that
> information in general is lost on final delivery. OTOH, people will
> want to have execute command/play sound actions (the latter could be
> added to notify) on the client.

If it were only this simple... If an extension is defined that proves to be
wildely popular ISPs will have to choose between supporting a dialect of
sieve that presents as an ongoing series of support isses and not supporting
sieve at all. I deal with these people on a daily basis, and I can assure
you that given such a choice they will go with the latter. And given their
economic reality I'd probably make the same choice.

Like it or not, there's real cost associated with expanding sieve into a
full-fledged language. And that cost may end up being irrelevance.

> I agree, though, that the I-Ds currently floating around should first be
> advanced to RFCs before proposing even more extensions. It's a bit too
> much currently ;-)

It certainly is.

				Ned

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 04 May 2003 00:15:24 +0200
Lines: 83
Sender: [email-masked]

[Marc Mutz]:
>
>   [Ned Freed]:
>   > I do note, however, that nobody else is jumping in to support
>   > this prefix idea. I'm ambivalent about it myself.
>   
>   It's weekend ;-)

weekends seem to be more active than workdays on this list.

if you want a prefix character, use "_".  it's part of the identifier
grammar, and has seen similar use in other contexts.

>   The reasons against magic/system variables Kjetil mentioned were:
>   
>   1. Year, month, etc. are not really magic, you can write them via set
>      and they only spring into life on setdate.
>   
>   I think we agree that writing to those variables is not a good
>   idea, since it's not needed, makes program flow analysis more
>   complex

really?  I think an explicit action to set them is easier to handle
than a multitude of variables which can change as side-effects.  we've
already seen that this is a tricky issue with the match back
references.  there the gain for easy access seems great enough to be
worth it.  SETDATE can be seen as a macro for 

  SET "hour" "${_internal_startup_hour}";
  SET "minute" "${_internal_startup_minute}";
  SET "second" "${_internal_startup_second}";
  ...

pretty straightforward to analyse, IMO.

what actions can influence ${_imapflags}?  how will its value change?

>   and produces potentially invalid system variable content that
>   leads to a new class of runtime errors not present with read-only
>   system variables.

it's not a system variable.  it will not lead to more errors than we
get from making changable variables available to the user.

>   Furthermore, that they spring into life as a side-effect of a
>   command is proof, not disproof, of their magicness.

see macro comment.

>   2. There's no good use for them.
>   
>   Well, the fact that they are being defined (whether called system
>   variables or not) in itself is proof of their
>   usefulness. imapflags could also make good use of system
>   variables. In fact, IIRC, the imapflags extension was was what
>   prompted the variables extension in the first place.

I wasn't around then.  my motivation for writing the draft was to
enable match back references in the argument to fileinto.

>   3. We'd need a registry for them, since extensions could conflict.
>   
>   They can still conflict if we don't make system variables explicit
>   in the variables extension. Prefixing them at least avoids
>   conflicts with user variables.

no, there can be no conflict, two extensions can use the same variable
name and co-exist peacefully.  e.g., you could have two actions
putting its return value in ${result}.  the user would just have to do
a SET to save it in another variable before doing the next action, and
he only has to do that if he needs to use the two actions in the same
Sieve run.  with your truly magic variables, this can't be done.

>   As to requiring a registry. Sure, that would be nice, but isn't a
>   necessity. We don't have a registry for command or test
>   identifiers either, even though they could also conflict between
>   extensions.  Instead, we believe the extension authors and the
>   review process to avoid these conflicts.

fair enough.

-- 
Kjetil T.

Date: Sat, 03 May 2003 14:50:05 -0700 (PDT)
From: [email-masked]
Subject: Re: Limitation of editheader
To: Simon Josefsson <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> The draft says:

> ,----
> | 3. Action addheader
> |
> |    Syntax:
> |    	"addheader" <name: string> <value: string>
> |
> |    The addheader action appends a header field to the existing
> |    message header.
> `----

Prepending would be a better default.

> It is problematic that the user cannot control where the header is
> inserted.  I mentioned Received: earlier, but apparently that example
> wasn't credible, so here is another, different, example.  There are
> more headers that are order dependent, in case this isn't a credible
> example either.

> Consider the following usage:

> set mydatestring ...;
> set mymsgidstring ...;
> if header :contains "Subject" "libidn" {
>   addheader "Resent-From" "Simon Josefsson <[email-masked]>";
>   addheader "Resent-Date" $mydatestring;
>   addheader "Resent-Message-ID" $mymsgidstring;
>   redirect "[email-masked]";
> }

I'm not sure I think this is legitimate usage, but in any case, I'd argue for
prepend since that's how headers are normally added.

				Ned

From: Marc Mutz <[email-masked]>
Organization: "Old Europe" - and proud
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Sat, 3 May 2003 22:27:20 +0200
Sender: [email-masked]

--Boundary-02=_2YCt+P41Q58Dym/
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Saturday 03 May 2003 17:08, [email-masked] wrote:
<snip>
> I do note, however, that nobody
> else is jumping in to support this prefix idea. I'm ambivalent about
> it myself.
<snip>

It's weekend ;-)

As we _have_ magic variables in the current draft (year, month, etc)=20
that are implicitly changed by commands (as opposed to be set with=20
"set"), no matter if we call them thus or not, it would be a good idea=20
to make that explicit and to enable other extensions to define their=20
own magic/system variables.

The reasons against magic/system variables Kjetil mentioned were:

1. Year, month, etc. are not really magic, you can write them via set
   and they only spring into life on setdate.

I think we agree that writing to those variables is not a good idea,=20
since it's not needed, makes program flow analysis more complex and=20
produces potentially invalid system variable content that leads to a=20
new class of runtime errors not present with read-only system=20
variables.

=46urthermore, that they spring into life as a side-effect of a command is=
=20
proof, not disproof, of their magicness.

2. There's no good use for them.

Well, the fact that they are being defined (whether called system=20
variables or not) in itself is proof of their usefulness. imapflags=20
could also make good use of system variables. In fact, IIRC, the=20
imapflags extension was was what prompted the variables extension in=20
the first place.

3. We'd need a registry for them, since extensions could conflict.

They can still conflict if we don't make system variables explicit in=20
the variables extension. Prefixing them at least avoids conflicts with=20
user variables.

As to requiring a registry. Sure, that would be nice, but isn't a=20
necessity. We don't have a registry for command or test identifiers=20
either, even though they could also conflict between extensions.=20
Instead, we believe the extension authors and the review process to=20
avoid these conflicts.

Marc

=2D-=20
We do not believe it draws the proper balance in a democratic society
for the activities of government to be concealed from public scrutiny
while the private activities of citizens are made open to government.
       -- EPIC letter to the President of the EU Council of Ministers

--Boundary-02=_2YCt+P41Q58Dym/
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_2YCt+P41Q58Dym/--

From: Marc Mutz <[email-masked]>
Organization: "Old Europe" - and proud
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Sat, 3 May 2003 22:38:16 +0200
Sender: [email-masked]

--Boundary-02=_4iCt+c63jbZE6FE
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Saturday 03 May 2003 02:06, Kjetil Torgrim Homme wrote:
<snip>
> they are not really magic.  you can do
>   SETDATE;
>   SET "year" "${year} AD";
<snip>
> >     set "#imapflags" "${#imapflags} \\Seen"
<snip>
> that's just _too_ magic :-)
<snip>
> > Alternatively, assignment to system variables could be
> > disallowed.  I think would be the the best way to handle it,
> > since it also prevents users from defining variables in the system
> > namespace (which wouldn't be too problematic, given that system
> > variables spring into life only if their extension is require'd).
>
> yes.

So you say that
  set "year" "${year} AD";
is possible, but that
  set "imapflags" "${imapflags} \\Seen";
is too magic and writing to system variables should be disallowed?

I agree with the latter, but not with the former. I don't see the=20
usefulness of
  set "year" "${year} AD";
since
1. It can lead to hard-to-find bugs if, in a large script, you alter the
   value of "year" and way down the script test against ${year}.
and more importantly,
2. What does the above solve that can't be solved by
     set "myyear" "${year} AD";

Marc

=2D-=20
Military justice is to justice what military music is to music.
                                                  -- Groucho Marx

--Boundary-02=_4iCt+c63jbZE6FE
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_4iCt+c63jbZE6FE--

To: Philip Guenther <[email-masked]>
Cc: [email-masked]
Subject: Re: Editorial editheader comment
From: Simon Josefsson <[email-masked]>
Date: Sat, 03 May 2003 22:12:18 +0200
Sender: [email-masked]

Philip Guenther <[email-masked]> writes:

>>4) A message header that looks like:
>>...
>>To: [email-masked]
>>...
>>To: [email-masked]
>>...
>>To: [email-masked]
>
> Given the meanings layed out in rfc2822 for "header" and "header field",
> I don't see how the text can be interpreted as permitting anything
> besides #4.  The other choices all either appended text *to* a header
> field or added the header field somewhere besides the end of the
> existing header, neither of which match the text of the draft.

I agree, but I was using the RFC 3028 terminology.  3028 sometimes
uses the word "header" as referring to (in RFC 2822 terminology)
header fields.  E.g., from "2.4.2.2 Headers":

   A header name never contains a colon.  The "From" header refers to a
   line beginning "From:" (or "From   :", etc.).  No header will match
   the string "From:" due to the trailing colon.

Given this, it was unclear (to me) which definition was used.  But if
it is clear for everyone else, ignore my message.

To: Simon Josefsson <[email-masked]>
Cc: [email-masked]
From: Philip Guenther <[email-masked]>
Subject: Re: Editorial editheader comment 
Date: Sat, 03 May 2003 12:43:49 -0700
Sender: [email-masked]

Simon Josefsson <[email-masked]> writes:
>
>Quoting the document:
>
>,----
>| 3. Action addheader
>| 
>|    Syntax:
>|    	"addheader" <name: string> <value: string>
>| 
>|    The addheader action appends a header field to the existing
>|    message header.
>...
>|    The header field MUST be added at the end of the existing
>|    header.
>`----
>
>It isn't clear if a Sieve statement
>
>   addheader "To" "[email-masked]";
>
>invoked on a message that have a header
>
>...
>To: [email-masked]
>...
>To: [email-masked]
>...
>
>will result in:
...
>4) A message header that looks like:
>...
>To: [email-masked]
>...
>To: [email-masked]
>...
>To: [email-masked]

Given the meanings layed out in rfc2822 for "header" and "header field",
I don't see how the text can be interpreted as permitting anything
besides #4.  The other choices all either appended text *to* a header
field or added the header field somewhere besides the end of the
existing header, neither of which match the text of the draft.

Philip Guenther
[email-masked]

To: [email-masked]
Subject: Editorial editheader comment
From: Simon Josefsson <[email-masked]>
Date: Sat, 03 May 2003 20:48:47 +0200
Sender: [email-masked]

Quoting the document:

,----
| 3. Action addheader
| 
|    Syntax:
|    	"addheader" <name: string> <value: string>
| 
|    The addheader action appends a header field to the existing
|    message header.
...
|    The header field MUST be added at the end of the existing
|    header.
`----

It isn't clear if a Sieve statement

   addheader "To" "[email-masked]";

invoked on a message that have a header

...
To: [email-masked]
...
To: [email-masked]
...

will result in:

1) A message header that looks like:

...
To: [email-masked] [email-masked]
...
To: [email-masked]
...

2) A message header that looks like: (somewhat unrealistic interpretation)

...
To: [email-masked], [email-masked]
...
To: [email-masked]
...

3) A message header that looks like:

...
To: [email-masked]
To: [email-masked]
...
To: [email-masked]
...

4) A message header that looks like:

...
To: [email-masked]
...
To: [email-masked]
...
To: [email-masked]

Honestly I'm not sure what is intended, otherwise I would have
provided clarified text.

I suspect 4 may be what was intended, but I'm not sure it is the best
choice.

To: [email-masked]
Subject: Limitation of editheader
From: Simon Josefsson <[email-masked]>
Date: Sat, 03 May 2003 20:33:32 +0200
Sender: [email-masked]

The draft says:

,----
| 3. Action addheader
| 
|    Syntax:
|    	"addheader" <name: string> <value: string>
| 
|    The addheader action appends a header field to the existing
|    message header.
`----

It is problematic that the user cannot control where the header is
inserted.  I mentioned Received: earlier, but apparently that example
wasn't credible, so here is another, different, example.  There are
more headers that are order dependent, in case this isn't a credible
example either.

Consider the following usage:

set mydatestring ...;
set mymsgidstring ...;
if header :contains "Subject" "libidn" {
  addheader "Resent-From" "Simon Josefsson <[email-masked]>";
  addheader "Resent-Date" $mydatestring;
  addheader "Resent-Message-ID" $mymsgidstring;
  redirect "[email-masked]";
}

If the incoming message happens to contain Resent headers, this would
violate RFC 2822 which says the most recently introduced Resent-*
header block is the first one:

,----
| 3.6.6. Resent fields
|
|    Resent fields SHOULD be added to any message that is reintroduced by
|    a user into the transport system.  A separate set of resent fields
|    SHOULD be added each time this is done.  All of the resent fields
|    corresponding to a particular resending of the message SHOULD be
|    together.  Each new set of resent fields is prepended to the message;
|    that is, the most recent set of resent fields appear earlier in the
|    message.
`----

To: [email-masked]
Subject: Message uniqueness in editheader
From: Simon Josefsson <[email-masked]>
Date: Sat, 03 May 2003 20:12:49 +0200
Sender: [email-masked]

RFC 3028 says:

,----
| 2.10.3.  Message Uniqueness in a Mailbox
| 
|    Implementations SHOULD NOT deliver a message to the same folder more
|    than once, even if a script explicitly asks for a message to be
|    written to a mailbox twice.
| 
|    The test for equality of two messages is implementation-defined.
| 
|    If a script asks for a message to be written to a mailbox twice, it
|    MUST NOT be treated as an error.
`----

It may be useful to discuss message uniqueness in the editheader
draft, so that the following code is well defined:

fileinto "INBOX.foo"
addheader "X-Sieve-Filtered" "<[email-masked]>";
fileinto "INBOX.foo"

Otherwise, it seems to me that

1) add original message to INBOX.foo
2) add modified message to INBOX.foo
3) add both original and modified message to INBOX.foo

can all be valid behavior, which I believe is unfortunate.

Suggested new section 6 below.

Note the wording that says which order the fileinto commands must be
executed.  Perhaps the specification doesn't have to guarantee the
order messages are inserted.

,----
| 6. Interaction with Sieve and other Sieve Extensions
| 
|    Tests and actions such as "exist" or "header" that examine
|    headers MUST examine the current state of a header as modified
|    by any actions that have taken place so far.  As an example,
|    in the following example, the if statement will always be true,
|    regardless of whether the message contains the header or not.
| 
|      addheader "X-Hello" "World";
|      if header :contains "X-Hello" "World" {
|        fileinto "INBOX.hi"
|      }
| 
|    Actions that create messages in storage or in transport to
|    MTAs MUST store and send messages with the current set of
|    header fields.
| 
|    A message modified by addheader, deleteheader, or replaceheader
|    MUST NOT be considered the same as the original message.  Compare
|    the discussion about message uniqueness in section 2.10.3 of RFC
|    3028.  This means the following code will always insert two
|    messages, the first one is always the original message and the
|    second the modified one.
| 
|      fileinto "INBOX.foo"
|      addheader "X-Sieve-Filtered" "<[email-masked]>";
|      fileinto "INBOX.foo"
`----

Date: Sat, 03 May 2003 08:08:17 -0700 (PDT)
From: [email-masked]
Subject: Re: variables: open issue a) date variables
To: Marc Mutz <[email-masked]>
Cc: [email-masked]
Content-description: signed data
Sender: [email-masked]

> On Saturday 03 May 2003 02:06, [email-masked] wrote:
> <snip>
> > I have no problem with defining a magic variable namespace. In fact
> > in some way I'd prefer it. That being said, however, I have a big
> > problem with defining it and not using it in the same document. All
> > too often this leads to issues being left open and problems not being
> > spotted until too late.
> <snip>

> We'd have the date and imapflags (and notify) extensions to go along
> with variables. The variables draft also defines the special
> num-variable "system variables" for the regex extension and :matches.

> > I'd somewhat prefer to use $ for the prefix, but it works either way.
> <snip>

> I don't like $ being the prefix since it's also the variable special
> character. "${$year}" smells like resolving references (or pointers if
> you're C) to a perl programmer. Also, the variables draft mentions this
> example:

That's sufficient reason, I guess. I do note, however, that nobody else is
jumping in to support this prefix idea. I'm ambivalent about it myself.

> >         "${company}" => "ACME"
> >         "${President, ${Company} Inc.}"
> >                      => "${President, ACME Inc.}"

> Which makes (if kept, I'm not sure it should be, personally, I think it
> should give an error) the interpretation of

> set "year" "2000";
> fileinto "archive.${$year}"

> ambiguous (could either file into "archive.${2000}" or "archive.2003" if
> 2003 was the current year).

I don't see any ambiguity here. This is clearly a reference to $year, not
year, and the result will be 2003. The brackets are mandatory in variable
references, not optional.

				Ned

Date: Sat, 03 May 2003 08:04:19 -0700 (PDT)
From: [email-masked]
Subject: Re: variables: open issue a) date variables
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> [Ned Freed]:
> >
> >   There definitely needs to be a way to specify a day of the week. I
> >   really don't care if it is 1/2/3/4 or mon/tue/wed or whatever, but
> >   it needs to be there because people want to take certain actions
> >   based on the day of week.
> >
> >   FWIW, RFC 2445 uses
> >
> >   weekday    = "SU" / "MO" / "TU" / "WE" / "TH" / "FR" / "SA"

> numeric values seem more versatile, but textual values make for more
> readable scripts.

> >   Beyond that, an iso8601 option would be useful for checking
> >   boundaries in a single operation. RFC 3339 can be referenced for
> >   this.

> thanks, I hadn't read that RFC (shame on me), but I don't quite get
> the reference?  do you mean a variable containing the complete
> "2003-05-03T05:03:12.00+02:00" ?  (I think I would leave out the time
> zone for this application, actually.)

Yes, that's what I meant.

> >   > for the weekday, while 0..6 with 0 meaning Sunday may seem alien
> >   > to many users (including me :), at least the information is
> >   > available in an unambigious fashion.
> >
> >   If you use 1/2/3 you _are_ using the Islamic days of the week ;-)

> well, I'm a member of the The Net Atheists, but on the seventh day He
> resteth or something like that, so it always struck me as odd to
> number Sunday as the first day of the week.  but I digress ;-)

> >   > so my suggestion is to augment the list with the following:
> >   >
> >   >   ${weekday}   (0..6)       %w: 0 is Sunday
> >   >   ${week}      (01..53)     %V: week number according to ISO 8601
> >
> >   These are both fine as far as I'm concerned.

> now I'm not so sure.  RFC 3339 follows ISO 8601 and uses

>    date-wday       =  DIGIT  ; 1-7  ; 1 is Monday, 7 is Sunday

Then that's what we'd better use.

> >   >   ${isoyear}   (0000..9999) %G: year number according to ISO 8601.
> >   >                                 needed since Jan 1 2000 is in week 52
> >   >                                 of 1999
> >
> >   The last doesn't necessarily follow. If you're using week numbers
> >   you may not care about the specific year.

> I see two main uses for ${week}.  one is vacation message: "I'll be
> out of town during week 45 and 46".  the other is for fileinto, and
> then we need ${isoyear} to get a strict ordering.

> the problem with the former use is that US weeks doesn't match ISO
> weeks.  we Europeans will be fine (I think -- Norwegians for sure ;-).

It isn't clear to me that weeks are referred to by number enough to matter.
At least not in the US.

				Ned

From: Marc Mutz <[email-masked]>
Organization: "Old Europe" - and proud
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Sat, 3 May 2003 12:27:53 +0200
Sender: [email-masked]

--Boundary-02=_0m5s+GdBHOeB/IC
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Saturday 03 May 2003 02:06, [email-masked] wrote:
<snip>
> I have no problem with defining a magic variable namespace. In fact
> in some way I'd prefer it. That being said, however, I have a big
> problem with defining it and not using it in the same document. All
> too often this leads to issues being left open and problems not being
> spotted until too late.
<snip>

We'd have the date and imapflags (and notify) extensions to go along=20
with variables. The variables draft also defines the special=20
num-variable "system variables" for the regex extension and :matches.

> I'd somewhat prefer to use $ for the prefix, but it works either way.
<snip>

I don't like $ being the prefix since it's also the variable special=20
character. "${$year}" smells like resolving references (or pointers if=20
you're C) to a perl programmer. Also, the variables draft mentions this=20
example:

>         "${company}" =3D> "ACME"
>         "${President, ${Company} Inc.}"
>                      =3D> "${President, ACME Inc.}"

Which makes (if kept, I'm not sure it should be, personally, I think it=20
should give an error) the interpretation of

set "year" "2000";
fileinto "archive.${$year}"

ambiguous (could either file into "archive.${2000}" or "archive.2003" if=20
2003 was the current year).

Marc

=2D-=20
We notice things that don't work. We don't notice things that do.
We notice computers, we don't notice pennies.
We notice e-book readers, we don't notice books.
                                          -- Douglas Adams

--Boundary-02=_0m5s+GdBHOeB/IC
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_0m5s+GdBHOeB/IC--

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 03 May 2003 05:36:53 +0200
Lines: 60
Sender: [email-masked]

[Ned Freed]:
>
>   There definitely needs to be a way to specify a day of the week. I
>   really don't care if it is 1/2/3/4 or mon/tue/wed or whatever, but
>   it needs to be there because people want to take certain actions
>   based on the day of week.
>   
>   FWIW, RFC 2445 uses
>   
>   weekday    = "SU" / "MO" / "TU" / "WE" / "TH" / "FR" / "SA"

numeric values seem more versatile, but textual values make for more
readable scripts.

>   Beyond that, an iso8601 option would be useful for checking
>   boundaries in a single operation. RFC 3339 can be referenced for
>   this.

thanks, I hadn't read that RFC (shame on me), but I don't quite get
the reference?  do you mean a variable containing the complete
"2003-05-03T05:03:12.00+02:00" ?  (I think I would leave out the time
zone for this application, actually.)

>   > for the weekday, while 0..6 with 0 meaning Sunday may seem alien
>   > to many users (including me :), at least the information is
>   > available in an unambigious fashion.
>   
>   If you use 1/2/3 you _are_ using the Islamic days of the week ;-)

well, I'm a member of the The Net Atheists, but on the seventh day He
resteth or something like that, so it always struck me as odd to
number Sunday as the first day of the week.  but I digress ;-)

>   > so my suggestion is to augment the list with the following:
>   >
>   >   ${weekday}   (0..6)       %w: 0 is Sunday
>   >   ${week}      (01..53)     %V: week number according to ISO 8601
>   
>   These are both fine as far as I'm concerned.

now I'm not so sure.  RFC 3339 follows ISO 8601 and uses

   date-wday       =  DIGIT  ; 1-7  ; 1 is Monday, 7 is Sunday

>   >   ${isoyear}   (0000..9999) %G: year number according to ISO 8601.
>   >                                 needed since Jan 1 2000 is in week 52
>   >                                 of 1999
>   
>   The last doesn't necessarily follow. If you're using week numbers
>   you may not care about the specific year.

I see two main uses for ${week}.  one is vacation message: "I'll be
out of town during week 45 and 46".  the other is for fileinto, and
then we need ${isoyear} to get a strict ordering.

the problem with the former use is that US weeks doesn't match ISO
weeks.  we Europeans will be fine (I think -- Norwegians for sure ;-).

-- 
Kjetil T.

Date: Fri, 02 May 2003 17:13:53 -0700 (PDT)
From: [email-masked]
Subject: Re: variables: open issue a) date variables
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> there are a few open issues mentioned in the draft.  I'm sure there
> should be more listed, so please remind me of what I have forgotten or
> glossed over.  in any case, I think it is time to try to get them
> resolved.  the first one is quite trivial:

>   a) should we include more predefined variables to access the time?
>      (weekday, week number (US, EU and/or ISO?), name of month, name
>      of weekday, ...)  could use strftime(3c) as a list of what to
>      offer, but the names should be in English.

> the current list is ${year} ${month} ${day} ${hour} ${minute}
> ${second} ${timezone}, all numeric and zero-padded.

There definitely needs to be a way to specify a day of the week. I really don't
care if it is 1/2/3/4 or mon/tue/wed or whatever, but it needs to be there
because people want to take certain actions based on the day of week.

FWIW, RFC 2445 uses

weekday    = "SU" / "MO" / "TU" / "WE" / "TH" / "FR" / "SA"

Beyond that, an iso8601 option would be useful for checking boundaries
in a single operation. RFC 3339 can be referenced for this.

> I don't like adding variables with values in natural language, since
> that adds internationalisation problems for little gain.
> unfortunately, both weekday and week have localisation problems.

You've already bought in to the localization issue to some extent by using
Gregorian months. I think worrying about the localization issues for
days of the week is silly.

The same issue was faced in calshd. Not to put too fine a point on it, but if
users of other calendars want their weeks, months and so on represtented they
are free to write up a draft specifying such things. 

> strftime(3c) solves this by supplying many conversion specifiers,
> i.e. %U, %V and %W for three methods of computing the week number.  I
> feel we would need to select one of these (and then the ISO
> definitions is the obvious candidate), after all having one is better
> than none.

Assuming we even need week number, I'd suggest %V. That's what ISO 8601
uses and what RFC 2445 picked.

> for the weekday, while 0..6 with 0 meaning Sunday may seem alien to
> many users (including me :), at least the information is available in
> an unambigious fashion.

If you use 1/2/3 you _are_ using the Islamic days of the week ;-)

> so my suggestion is to augment the list with the following:

>   ${weekday}   (0..6)       %w: 0 is Sunday
>   ${week}      (01..53)     %V: week number according to ISO 8601

These are both fine as far as I'm concerned.

>   ${isoyear}   (0000..9999) %G: year number according to ISO 8601.
>                                 needed since Jan 1 2000 is in week 52
>                                 of 1999

The last doesn't necessarily follow. If you're using week numbers you may not
care about the specific year.

> my reasoning is that this is useful information for some people, and
> it is easy to implement.

Sure. Doesn't everyone have a copy of _Calendrical Calculation_ on their desk?
I certainly do.

				Ned

Date: Fri, 02 May 2003 17:06:57 -0700 (PDT)
From: [email-masked]
Subject: Re: variables: open issue a) date variables
To: Marc Mutz <[email-masked]>
Cc: [email-masked]
Content-description: signed data
Sender: [email-masked]

> On Friday 02 May 2003 01:04, Kjetil Torgrim Homme wrote:
> <snip>
> > the current list is ${year} ${month} ${day} ${hour} ${minute}
> > ${second} ${timezone}, all numeric and zero-padded.

> There are two issues I have with this:
> 1. We define magic variables and don't provide a mechanism for other
> extensions to register their own magic variables.
> 2. We combine two basically unrelated extensions ("variables" and
> "dates") into one.

> Short comment: Define a namespace for magic variables and move the date
> variables out into the to-be-written draft defining the date test.

I have no problem with defining a magic variable namespace. In fact
in some way I'd prefer it. That being said, however, I have a big
problem with defining it and not using it in the same document. All
too often this leads to issues being left open and problems not being
spotted until too late.

> Long comments:
> Let's assume for the moment that "variables" didn't include "dates", but
> defined a syntax for magic variables[1]. "dates" would encompass the
> date test Ned wanted to write an I-D for and - in the presence of the
> "variables" extension - would define certain magic variables "#year",
> "#month" etc.

> ad 1: Change the variable-name production to read
>   variable-name := num-variable / magic-variable / user-variable
>   magic-variable := "#" identifier
>   user-variable := identifier
> and require every Sieve extension to define their magic-variables and
> which commands change their state and how. This is more like how common
> script languages (bash, awk, perl,...) work and it provides a namespace
> for magic (or "system") variables alike to perl's $<non-alnum>
> variables.

I'd somewhat prefer to use $ for the prefix, but it works either way.

> ad 2: The "dates" extension would then use this mechanism to declare
> #year, #month, #day, #hour, #minute, #second and #timezone to be system
> variables that are initialized to their respective values of the time
> of script execution start (local timezone). Since the "setdate" action
> now didn't really set the date variables anymore, it would better be
> named "settimezone". If executed, these system variable values are
> recomputed, based on the new timezone given to "settimezone", but still
> representing the time of script execution start.

Works for me.

> To see the benefit of the system variable notion, consider the imapflags
> extension. This would now declare the #imapflags system variable that
> holds the current flags (e.g. as a list of imap flag names separated by
> spaces). addflags, setflags and removeflags would then act on the
> #imapflags variable, but
>   set "#imapflags" "${#imapflags} \\Seen"
> would be OK, too, of you accept the fact that you can cause problems if
> you forget the space before "\Seen". Basically, if you assign to system
> variables, it's your own fault. Alternatively, assignment to system
> variables could be disallowed. I think would be the the best way to
> handle it, since it also prevents users from defining variables in the
> system namespace (which wouldn't be too problematic, given that system
> variables spring into life only if their extension is require'd).
> Fileinto would then use the value of #imapflags to determine the flags
> to set on (e.g.) IMAP APPEND.

I kind of like the idea of not being able to assign to system variables.

				Ned

To: [email-masked]
Subject: Re: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 03 May 2003 02:06:41 +0200
Lines: 67
Sender: [email-masked]

[Marc Mutz]:
>
>   On Friday 02 May 2003 01:04, Kjetil Torgrim Homme wrote:
>   <snip>
>   > the current list is ${year} ${month} ${day} ${hour} ${minute}
>   > ${second} ${timezone}, all numeric and zero-padded.
>   
>   There are two issues I have with this:
>   1. We define magic variables and don't provide a mechanism for other 
>   extensions to register their own magic variables.

they are not really magic.  you can do

  SETDATE;
  SET "year" "${year} AD";

unless you call SETDATE, no variables in your name space will be
touched.

>   2. We combine two basically unrelated extensions ("variables" and
>   "dates") into one.

yes.  but do we gain anything by splitting the two?  "dates" would
require "variables", and the implementation impact of "dates" is quite
low, especially compared to "variables".

>   Short comment: Define a namespace for magic variables and move the
>   date variables out into the to-be-written draft defining the date
>   test.

I don't think we need magic variables.  I had them in my very first
draft, but threw them out since I couldn't see a good use for them.
we'd need a registry for known magic variable names, since extensions
could conflict.  actions updating variables doesn't need that -- if
they stomp on each others variables, the user will know from the spec
and take whatever action is necessary to save aside any data needed.

>   To see the benefit of the system variable notion, consider the
>   imapflags extension. This would now declare the #imapflags system
>   variable that holds the current flags (e.g. as a list of imap flag
>   names separated by spaces).

nice, but you can get the same capability by adding an action
IMAPFLAGS which updates the value of ${imapflag}.  (this might be a
good idea, actually.)

>   addflags, setflags and removeflags would then act on the
>   #imapflags variable, but
>
>     set "#imapflags" "${#imapflags} \\Seen"
>
>   would be OK, too, of you accept the fact that you can cause
>   problems if you forget the space before "\Seen".

that's just _too_ magic :-)

>   Basically, if you assign to system variables, it's your own
>   fault. Alternatively, assignment to system variables could be
>   disallowed.  I think would be the the best way to handle it, since
>   it also prevents users from defining variables in the system
>   namespace (which wouldn't be too problematic, given that system
>   variables spring into life only if their extension is require'd).

yes.

-- 
Kjetil T.

To: [email-masked], Gisle Aas <[email-masked]>
Subject: Re: Are extension names case insensitive
From: Kjetil Torgrim Homme <[email-masked]>
Date: 03 May 2003 01:49:02 +0200
Lines: 45
Sender: [email-masked]

[Ned Freed]:
>
>   > but an implementation that allows that will create an
>   > incompatibility which didn't exist before.
>   
>   Incompatible only in the sense that something that has only one
>   reasonable interpretation wasn't interpreted in a reasonable way.

hah! :-)

to answer Gisle's question, though: a Sieve generator needs to use the
lower case capability string, simply due to the sizable market
penetration of Cyrus.  a Sieve processor should do case-insensitive
matching.

>   Command arguments are a very different matter than command
>   names. There is a lengthy tradition in the IETF of treating
>   command names and capability strings and so on in a
>   case-insensitive fashion. The exact opposite is true of arguments:
>   They need to preserve case, and if appropriate act in a
>   case-sensitive fashion.

this is an argument I accept, but I won't accept that the wording in
the Sieve RFC mandates this.  luckily, this isn't a big problem, I
can't think of any actions other than REQUIRE and FILEINTO which are
affected by this.  going quickly through the drafts: INCLUDE might use
a note about this issue; NOTIFY is explicit on :id but not :method;
IMAPFLAGS is constrained by IMAP, but doesn't state the case
sensitivity explicitly; RELATIONAL is OK, since literal strings in
ABNF are case-insensitive.

(it seems to me it would have been better to use tags rather than
strings as capability identifiers.  that would restrict the allowable
names nicely.  water under the bridge, anyhow.)

>   > if "INBOX.foo" exists and you do FILEINTO "Inbox.Foo", Cyrus
>   > will file the message into "INBOX".
>   
>   A perfectly reasonable action on its part INO. Another reasonable
>   alternative would have been to create the folder "Inbox.Foo".

yes, that was my point.

-- 
Kjetil T.

From: Marc Mutz <[email-masked]>
Organization: "Old Europe" - and proud
To: [email-masked]
Subject: Re: variables: open issue a) date variables
Date: Sat, 3 May 2003 00:47:32 +0200
Sender: [email-masked]

--Boundary-02=_VWvs+g/HO1fkac8
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Content-Description: signed data
Content-Disposition: inline

On Friday 02 May 2003 01:04, Kjetil Torgrim Homme wrote:
<snip>
> the current list is ${year} ${month} ${day} ${hour} ${minute}
> ${second} ${timezone}, all numeric and zero-padded.

There are two issues I have with this:
1. We define magic variables and don't provide a mechanism for other=20
extensions to register their own magic variables.
2. We combine two basically unrelated extensions ("variables" and=20
"dates") into one.

Short comment: Define a namespace for magic variables and move the date=20
variables out into the to-be-written draft defining the date test.

Long comments:
Let's assume for the moment that "variables" didn't include "dates", but=20
defined a syntax for magic variables[1]. "dates" would encompass the=20
date test Ned wanted to write an I-D for and - in the presence of the=20
"variables" extension - would define certain magic variables "#year",=20
"#month" etc.

ad 1: Change the variable-name production to read
  variable-name :=3D num-variable / magic-variable / user-variable
  magic-variable :=3D "#" identifier
  user-variable :=3D identifier
and require every Sieve extension to define their magic-variables and=20
which commands change their state and how. This is more like how common=20
script languages (bash, awk, perl,...) work and it provides a namespace=20
for magic (or "system") variables alike to perl's $<non-alnum>=20
variables.

ad 2: The "dates" extension would then use this mechanism to declare=20
#year, #month, #day, #hour, #minute, #second and #timezone to be system=20
variables that are initialized to their respective values of the time=20
of script execution start (local timezone). Since the "setdate" action=20
now didn't really set the date variables anymore, it would better be=20
named "settimezone". If executed, these system variable values are=20
recomputed, based on the new timezone given to "settimezone", but still=20
representing the time of script execution start.

To see the benefit of the system variable notion, consider the imapflags=20
extension. This would now declare the #imapflags system variable that=20
holds the current flags (e.g. as a list of imap flag names separated by=20
spaces). addflags, setflags and removeflags would then act on the=20
#imapflags variable, but
  set "#imapflags" "${#imapflags} \\Seen"
would be OK, too, of you accept the fact that you can cause problems if=20
you forget the space before "\Seen". Basically, if you assign to system=20
variables, it's your own fault. Alternatively, assignment to system=20
variables could be disallowed. I think would be the the best way to=20
handle it, since it also prevents users from defining variables in the=20
system namespace (which wouldn't be too problematic, given that system=20
variables spring into life only if their extension is require'd).
=46ileinto would then use the value of #imapflags to determine the flags=20
to set on (e.g.) IMAP APPEND.

Comments?

Marc

[1] Those that are either implicitly changed by commands or are=20
automatic in the sense that the environment provides them w/o explicit=20
instruction (with can be rephrased as: "are set implicitely by the=20
require command requesting the corresponding extension"). For=20
concreteness, let's assume they must all start with a hash '#'.

=2D-=20
If this were a dictatorship, it'd be a heck of a lot easier...just as
long as I'm the dictator...
=2D- George W. Bush, Washington, DC, Dec 18, 2000,
   during his first trip to Washington as President-Elect

--Boundary-02=_VWvs+g/HO1fkac8
Content-Type: application/pgp-signature
Content-Description: signature

--Boundary-02=_VWvs+g/HO1fkac8--

Date: Fri, 02 May 2003 13:57:14 -0700 (PDT)
From: [email-masked]
Subject: Re: Are extension names case insensitive
To: Kjetil Torgrim Homme <[email-masked]>
Cc: [email-masked], Gisle Aas <[email-masked]>
Sender: [email-masked]

> [Ned Freed]:
> >
> >   [Kjetil Torgrim Homme]:
> >   > [Gisle Aas]:
> >   > >   Should this work?
> >   > >
> >   > >      REQUIRE "FILEINTO";
> >   > >      fileinto "FOO";
> >   >
> >   > I'd say no.
> >
> >   I'd say yes. It isn't like we're going to define FILEINTO to have
> >   a different meaning than fileinto, so implementations get maximum
> >   interop by performing a case-insensitive comparison in this
> >   context.

> but an implementation that allows that will create an incompatibility
> which didn't exist before.

Incompatible only in the sense that something that has only one reasonable
interpretation wasn't interpreted in a reasonable way.

> or perhaps it already exists?  Cyrus does
> case-sensitive comparison on the capability string.  (added whitespace
> also makes the comparison fail.)

And my implementation does it in a case-insensitive fashion. And I have
seen scripts written that take advange of this fact.

I see no reaosn to expect that added whitespace would be allowed, however.

> how about FILEINTO itself?  the folder name is case sensitive in IMAP
> (with the exception of the prefix "INBOX").  shouldn't Sieve do
> case-insensitive matching on existing folders to find the correct one,
> and then store into it?

That would depend on the characteristics of the underlying store. IMAP allows
for either case-sensitive or case-insensitive names. It is even possible for
the same IMAP server to handle different parts of its folder namespace in
different ways.

The way this should work is that sieve should act in a case-preserving fashion.
Once the message hits the message store the rules for folder name in that store
should apply. 

There's also the issue of internationalized mailbox names. The IMAP
specifications defines a modified form of UTF-7 for this. Sieve, on the other
hand, uses UTF-8. A case-preserving conversion from UTF-8 to modified UTF-7
is therefore appropriate in some cases.

Command arguments are a very different matter than command names. There is a
lengthy tradition in the IETF of treating command names and capability strings
and so on in a case-insensitive fashion. The exact opposite is true of
arguments: They need to preserve case, and if appropriate act in a
case-sensitive fashion. 

> if "INBOX.foo" exists and you do FILEINTO "Inbox.Foo", Cyrus will file
> the message into "INBOX".

A perfectly reasonable action on its part INO. Another reasonable alternative
would have been to create the folder "Inbox.Foo".

				Ned

To: [email-masked]
Cc: Gisle Aas <[email-masked]>
Subject: Re: Are extension names case insensitive
From: Kjetil Torgrim Homme <[email-masked]>
Date: 02 May 2003 22:17:43 +0200
Lines: 45
Sender: [email-masked]

[Ned Freed]:
>
>   [Kjetil Torgrim Homme]:
>   > [Gisle Aas]:
>   > >   Should this work?
>   > >
>   > >      REQUIRE "FILEINTO";
>   > >      fileinto "FOO";
>   >
>   > I'd say no.
>   
>   I'd say yes. It isn't like we're going to define FILEINTO to have
>   a different meaning than fileinto, so implementations get maximum
>   interop by performing a case-insensitive comparison in this
>   context.

but an implementation that allows that will create an incompatibility
which didn't exist before.  or perhaps it already exists?  Cyrus does
case-sensitive comparison on the capability string.  (added whitespace
also makes the comparison fail.)

  line 1: unsupported feature

what do others do?

>   Case preservation does not imply case sensitivity on comparisons.

agreed, I only showed that literal strings in Sieve preserve case.

the question is whether a match is implied.

>   We have case-insensitive tokens and string comparisons are
>   case-insensitive by default. I'm sorry, but what's dubious is the
>   assrtion that require should be case-sensitive.

how about FILEINTO itself?  the folder name is case sensitive in IMAP
(with the exception of the prefix "INBOX").  shouldn't Sieve do
case-insensitive matching on existing folders to find the correct one,
and then store into it?

if "INBOX.foo" exists and you do FILEINTO "Inbox.Foo", Cyrus will file
the message into "INBOX".

-- 
Kjetil T.

To: [email-masked]
Subject: Re: variables: open issue b) fileinto :create
From: Kjetil Torgrim Homme <[email-masked]>
Date: 02 May 2003 21:33:14 +0200
Lines: 28
Sender: [email-masked]

[Jutta Degener]:
>
>   [Kjetil Torgrim Homme]:
>   > Jutta said she might write up a draft on this.  I'm thinking it
>   > could fit into her copy extension, IMHO it is good to reduce the
>   > number of extensions when possible.  the only downside to it I
>   > can see is that the extension name ("copy") doesn't fit very
>   > well :-)
>   
>   I think it's a better fit in your "variables" extension than in
>   copy; it has nothing to do with the functionality of :copy
>   (retaining the implicit "keep").

ah, but your "copy" changes the behaviour of "fileinto", and the
impact on an implementation of explicitly stating how to handle "new
folder" is very small...

it certainly does not fit into the "variables" extension :-).  it
could fit into the draft, as a separate extension, but the connection
between the two extensions would be flimsy.

I don't know how we will go forward with this eventually.  a lot of
drafts have been written, and quite a few of them should become
official extensions to Sieve, IMHO.  will there be an RFC for each?
is, say, a dozen RFCs for a dozen extensions unproblematic for IESG?

-- 
Kjetil T.

Date: Fri, 02 May 2003 11:36:56 -0700 (PDT)
From: [email-masked]
Subject: Re: Are extension names case insensitive
To: Kjetil Torgrim Homme <[email-masked]>
Cc: Gisle Aas <[email-masked]>, [email-masked]
Sender: [email-masked]

> [Gisle Aas]:
> >
> >   I was not able to tell from RFC 3028 if the extension names
> >   provided with the require statement are case-insensitive or not.
> >   Should this work?
> >
> >      REQUIRE "FILEINTO";
> >      fileinto "FOO";

> I'd say no.

I'd say yes. It isn't like we're going to define FILEINTO to have a different
meaning than fileinto, so implementations get maximum interop by performing
a case-insensitive comparison in this context.

> >   The closest thing I find is section 2.1 that says:

> >      Tokens in the ASCII range are considered case-insensitive.

> >   Should strings (as the argument to require) be considered tokens
> >   in this regard?

> I agree it isn't clearly stated, but strings are called "literal
> data", and you can do case sensitive matching (using the "i;octet"
> comparator) on literal strings.  from this I conclude that strings
> preserve case unless otherwise stated.  but then there is this tidbit:

Case preservation does not imply case sensitivity on comparisons.

> [2.7.3 Comparators]:
> |  In order to allow for language-independent, case-independent matches,
> |  the match type may be coupled with a comparator name.  [...]
> |
> |  All implementations MUST support the "i;octet" comparator (simply
> |  compares octets) and the "i;ascii-casemap" comparator (which treats
> |  uppercase and lowercase characters in the ASCII subset of UTF-8 as
> |  the same).  If left unspecified, the default is "i;ascii-casemap".

> I find it dubious to apply this default comparator on actions where no
> match type can be specified or is explicitly specified.  in other
> words, I don't think it is right to say that REQUIRE has an implicit
> :IS match type.

We have case-insensitive tokens and string comparisons are case-insensitive
by default. I'm sorry, but what's dubious is the assrtion that require should
be case-sensitive.

				Ned

To: Gisle Aas <[email-masked]>
Cc: [email-masked]
Subject: Re: Are extension names case insensitive
From: Kjetil Torgrim Homme <[email-masked]>
Date: 02 May 2003 19:24:20 +0200
Lines: 39
Sender: [email-masked]

[Gisle Aas]:
>
>   I was not able to tell from RFC 3028 if the extension names
>   provided with the require statement are case-insensitive or not.
>   Should this work?
>   
>      REQUIRE "FILEINTO";
>      fileinto "FOO";

I'd say no.

>   The closest thing I find is section 2.1 that says:
>   
>      Tokens in the ASCII range are considered case-insensitive.
>   
>   Should strings (as the argument to require) be considered tokens
>   in this regard?

I agree it isn't clearly stated, but strings are called "literal
data", and you can do case sensitive matching (using the "i;octet"
comparator) on literal strings.  from this I conclude that strings
preserve case unless otherwise stated.  but then there is this tidbit:

[2.7.3 Comparators]:
|  In order to allow for language-independent, case-independent matches,
|  the match type may be coupled with a comparator name.  [...]
|
|  All implementations MUST support the "i;octet" comparator (simply
|  compares octets) and the "i;ascii-casemap" comparator (which treats
|  uppercase and lowercase characters in the ASCII subset of UTF-8 as
|  the same).  If left unspecified, the default is "i;ascii-casemap".

I find it dubious to apply this default comparator on actions where no
match type can be specified or is explicitly specified.  in other
words, I don't think it is right to say that REQUIRE has an implicit
:IS match type.

-- 
Kjetil T.

Date: Fri, 2 May 2003 05:30:43 -0700
From: Jutta Degener <[email-masked]>
To: [email-masked]
Subject: Re: variables: open issue c) NOTIFY compatibility
Sender: [email-masked]

I liked the rest of your comments, but this one I disagree with:

On Fri, May 02, 2003 at 01:46:17AM +0200, Kjetil Torgrim Homme wrote:
> the one thing missing from variables is the ability to
> truncate string values, which is done in NOTIFY using the syntax
> $var[n]$.  in variables, you can simulate it using regex:
> 
>   require ["variables", "regex"];
>   if header "Subject" :regex "^(.{,200})" {
>     set "short" "${1}";
>   }
> 
> this does seem a bit hackish.  we could instead do something like
> 
>   if header "Subject" :matches "*" {
>     set :maxlength 200 "short" "${1}";
>   }

I'd be happier to just use the existing regex mechanism.

It's what someone familiar with regular expressions would
turn to naturally and already remembers from other languages;
the new mechanism would be unfamiliar to all.

Jutta

Date: Fri, 2 May 2003 05:12:16 -0700
From: Jutta Degener <[email-masked]>
To: [email-masked]
Subject: Re: variables: open issue b) fileinto :create
Sender: [email-masked]

On Fri, May 02, 2003 at 01:39:34AM +0200, Kjetil Torgrim Homme wrote:
> 
>   b) this extension is particularily useful if fileinto creates new
>      folders on demand.  [SIEVE] doesn't prohibit this, and currently
>      some implementations will create new folders automatically,
>      others won't.
> 
> Jutta said she might write up a draft on this.  I'm thinking it could
> fit into her copy extension, IMHO it is good to reduce the number of
> extensions when possible.  the only downside to it I can see is that
> the extension name ("copy") doesn't fit very well :-)

I think it's a better fit in your "variables" extension than
in copy; it has nothing to do with the functionality of :copy
(retaining the implicit "keep").

Jutta

To: [email-masked]
Subject: Are extension names case insensitive
From: Gisle Aas <[email-masked]>
Date: 01 May 2003 18:36:14 -0700
Lines: 17
Sender: [email-masked]

I was not able to tell from RFC 3028 if the extension names provided
with the require statement are case-insensitive or not.  Should this
work?

   REQUIRE "FILEINTO";
   fileinto "FOO";

The closest thing I find is section 2.1 that says:

   Tokens in the ASCII range are considered case-insensitive.

Should strings (as the argument to require) be considered tokens in
this regard?

Regards,
Gisle Aas,
ActiveState

To: [email-masked]
Subject: variables: open issue c) NOTIFY compatibility
From: Kjetil Torgrim Homme <[email-masked]>
Date: 02 May 2003 01:46:17 +0200
Lines: 37
Sender: [email-masked]

the text in the draft reads:

  c) the NOTIFY draft has variables, too.  it would be nice if the
     syntax for the two extensions agreed.  a changed NOTIFY can be
     used on its own without the variables extension, the user simply
     won't be able to configure the notification message to include
     different snippets from the message.

the NOTIFY draft has expired, so it's not a particularily important
issue.  the one thing missing from variables is the ability to
truncate string values, which is done in NOTIFY using the syntax
$var[n]$.  in variables, you can simulate it using regex:

  require ["variables", "regex"];
  if header "Subject" :regex "^(.{,200})" {
    set "short" "${1}";
  }

this does seem a bit hackish.  we could instead do something like

  if header "Subject" :matches "*" {
    set :maxlength 200 "short" "${1}";
  }

(aside: it seems to me that optimising the pattern "*" to do no
matching, only set up ${1}, will be worthwhile, this looks to be a
common idiom for all sorts of things...)

the rationale for the truncation operation in NOTIFY is obviously the
ability to send small excerpts of the message through limited mediums
such as SMS or ICQ.  it may be useful for vacation as well.

regular expressions are better suited for doing general substrings
(which I doubt have much general utility anyway), so I won't suggest
an ":offset N".
-- 
Kjetil T.

To: [email-masked]
Subject: variables: open issue b) fileinto :create
From: Kjetil Torgrim Homme <[email-masked]>
Date: 02 May 2003 01:39:34 +0200
Lines: 12
Sender: [email-masked]

  b) this extension is particularily useful if fileinto creates new
     folders on demand.  [SIEVE] doesn't prohibit this, and currently
     some implementations will create new folders automatically,
     others won't.

Jutta said she might write up a draft on this.  I'm thinking it could
fit into her copy extension, IMHO it is good to reduce the number of
extensions when possible.  the only downside to it I can see is that
the extension name ("copy") doesn't fit very well :-)

-- 
Kjetil T.

To: [email-masked]
Subject: variables: open issue a) date variables
From: Kjetil Torgrim Homme <[email-masked]>
Date: 02 May 2003 01:04:16 +0200
Lines: 40
Sender: [email-masked]

there are a few open issues mentioned in the draft.  I'm sure there
should be more listed, so please remind me of what I have forgotten or
glossed over.  in any case, I think it is time to try to get them
resolved.  the first one is quite trivial:

  a) should we include more predefined variables to access the time?
     (weekday, week number (US, EU and/or ISO?), name of month, name
     of weekday, ...)  could use strftime(3c) as a list of what to
     offer, but the names should be in English.

the current list is ${year} ${month} ${day} ${hour} ${minute}
${second} ${timezone}, all numeric and zero-padded.

I don't like adding variables with values in natural language, since
that adds internationalisation problems for little gain.
unfortunately, both weekday and week have localisation problems.

strftime(3c) solves this by supplying many conversion specifiers,
i.e. %U, %V and %W for three methods of computing the week number.  I
feel we would need to select one of these (and then the ISO
definitions is the obvious candidate), after all having one is better
than none.

for the weekday, while 0..6 with 0 meaning Sunday may seem alien to
many users (including me :), at least the information is available in
an unambigious fashion.

so my suggestion is to augment the list with the following:

  ${weekday}   (0..6)       %w: 0 is Sunday
  ${week}      (01..53)     %V: week number according to ISO 8601
  ${isoyear}   (0000..9999) %G: year number according to ISO 8601.  
                                needed since Jan 1 2000 is in week 52 
                                of 1999

my reasoning is that this is useful information for some people, and
it is easy to implement.

-- 
Kjetil T.

Date: Thu, 01 May 2003 08:21:56 -0700 (PDT)
From: [email-masked]
Subject: Re: variables: greedy or non-greedy matching
To: Jutta Degener <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

> On Tue, Apr 29, 2003 at 06:01:19PM -0700, [email-masked] wrote:
> > > [Ned Freed]:
> > > >
> > > >   [Jutta Degener]:
> > > >   > So let's stay greedy; thanks for setting me straight!
> > > >
> > > >   No problem ;-) And let's also remember that a necessary corollary
> > > >   here is that the regexp we end up choosing needs to support both
> > > >   greedy and non-greedy matching.
> >
> > > why?  I thought we only needed that if :matches was non-greedy.
> >
> > > I wouldn't mind having non-greedy regexps, but they're not in POSIX.2
> > > which the regex draft refers to.
> >
> > Jutta's argument, which I believe is valid, is that expectations can only
> > be met by having both types available.

> Most cases of it can also be met with character classes.

> That's why I said "let's stay greedy" in the piece you quote above --
> I realized that I wanted to solve problems with non-greed that can
> be solved with regexes.

Fair enough. It isn't a point we need to address in the context of the
variables draft in any case.

				Ned