Subject: Re: variables extension redux

From: [email-masked]
Date: Tue, 03 Aug 2004 10:51:47 -0700 (PDT)

> On fre, 2004-07-30 at 16:31 +0100, Alexey Melnikov wrote:
> > If this helps to get Sieve variables published (I am being selfish here,
> > of course :-)), than I agree.

> I'm sorry, but I've lost track.  what should be done about it?  there
> are some open issues in it, but they don't seem very hard problems,
> really.  I apologise if I have left out criticisms, it is not
> intentional, my memory isn't so good.

> === excerpt ===
> 0.3.  Open Issues

> b)   this extension is particularily useful if fileinto creates new
>      folders on demand.  [SIEVE] doesn't prohibit this, and currently
>      some implementations will create new folders automatically, others
>      won't.

> [out of scope, an explicit :createfolder for fileinto should be a
> separate extension.  this can be closed.]

Agreed.

> d)   the EDITHEADER draft includes an action that needs the unexpanded
>      string to be passed to the procedure, since the action first per-
>      forms matching which may influence numeric variable references in
>      the argument.  this is can be seen as a layering violation, and the
>      variable draft should state explicitly whether such extensions are
>      possible.

Vacation also needs access to the unexpanded strings for its internal hash
computations. This means that implementations need to allow for argument
evaluation to be done in two steps, first get the argument, then expand it.
Given that you have to allow for this anyway I don't see a problem with letting
future extensions do this sort of thing explicitly.

Banning variable substitutions in body parameters may also be something
implementations want to do, but this is a somewhat different issue.

> e)   the numeric variables are causing many headaches since they may
>      change spontaneously when running tests or even _during_ actions.

I don't see this as causing real problems in practice. If you need reliable
access to these variables you need to code your tests and actions accordingly.

>      an alternative approach is SETMATCH ["var1", "var2", "var3"] which
>      stores the first three match components into the listed variables.
>      (an empty string as a variable name means skip storing that match.)
>      this approach makes REPLACEHEADER less powerful.

> [this is a big change to the draft, but I think it makes the semantics
> clearer and reduces the chances of pitfalls.  the REPLACEHEADER magic
> was too magic, IMHO (it was removed from the draft anyway, but someone
> else may want something similar in the future)]

This would be easy for me to implement but I really don't care for it. In
particular, I find

     setmatch ["", "", "", "foo"];

to be pretty awful.

I'd feel differently if this actually solved the value changing problem during
compound tests, but it doesn't.

>      a related question is what happens when a match fails.  the draft
>      currently says that the numeric variables are reset, but this may
>      be inconvenient.

I'd prefer to have the values remain unchanged, but I can live with it
either way.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Wed, 04 Aug 2004 10:13:36 +0200

> future extensions do this sort of thing explicitly.

"vacation" is a good example.  perhaps a implementor's hint is off-topic
for the draft, but I suggest adding this to the end of 3 (just after the
next excerpt), before 3.1.

        Tests or actions in future extensions may want to access the
        unexpanded version of the string argument and do the expansion
        after, e.g., setting variables in its namespace.  The design of
        the implementation should allow this.

or more tersely:

        Future extensions may require access to the unexpanded version
        of the string argument.

(it's a given the implementation provides an internal function for
expanding a string...)

> implementations want to do, but this is a somewhat different issue.

right, there is already this wording:

        Strings where no variable substitutions take place are referred
        to as constant strings.  Future extensions may specify that
        passing non-constant strings as arguments to its actions or
        tests is an error.

> > [SETMATCH]

okay, I'll kill the issue and stick to numeric variables unless someone
else weighs in.

> I'd prefer to have the values remain unchanged, but I can live with it

I'm currently favouring the same, mostly due to precedence of this
behaviour in Perl.  I note that Sendmail claims to have implemented
"variables" already, and this is one of those small things which can
bite script writers if we change it around, although I think most
scripts will be straightforward and use a "SET" immediately after the ":
matches".  My conclusion is that we can do the change.

-- 
Kjetil T.

---

From: [email-masked]
Date: Wed, 04 Aug 2004 08:50:20 -0700 (PDT)

> (it's a given the implementation provides an internal function for

I prefer the wordier of these two.

> behaviour in Perl.

Yes, good point.

> I note that Sendmail claims to have implemented

FWIW, we have also implemented variables in our product, but we haven't
documented its availability yet and you have to enable it. Nor does our GUI
sieve interface generate sieves that depend on it (yet). We may be forceed to
change this in the not-so-distant future, however, since customer calls for the
functionality variables gives them are pretty common.

				Ned

---

From: Kjetil Torgrim Homme <[email-masked]>
Date: Thu, 05 Aug 2004 16:04:06 +0200

thank you for your feedback, Ned, I've updated my copy of the draft, and
closed all the open issues.

I'd like to bring up a new open issue, though:

6.  Implementation Limits

   An implementation of this draft MUST support at least 128 distinct
   variables.  The supported length of variable names MUST be at least
   32 characters.  Each variable MUST be able to hold at least 4000
   characters.  Attempts to set the variable to a value larger than what
   the implementation supports MUST be treated as an error.

   Numeric variables ${1} through ${9} MUST be supported.  Referencing
   higher indices than is supported is a syntax error which MUST be dis-
   covered at compile-time.  If the string matching a wildcard or a
   regex group operator exceeds the maximum variable size, the implemen-
   tation SHOULD truncate it and MUST NOT treat it as an error.

I'm not entirely happy with this since in a minimal implementaion it
makes

   if header :matches "Subject" "*" {
      set "subject" "I'm gone (was: ${1})";
   }

a run-time error whenever Subject is more than 4000 characters long:  $1
is truncated, but the expanded string is too long.  the result is Sieve
aborting, and the e-mail is implicitly kept.  if the desired action is a
redirect, this can be as bad as losing e-mail.

it *is* possible to code defensively:

   require ["relational", "variables", "i;ascii-numeric"];

   if header :matches "Subject" "*" {
      set "orig_subject" "${1}";
      set :length "length" "${orig_subject}";
      if string :value "ge" :comparator "i;ascii-numeric"
                "${length}" "3983" {
         set "orig_subject" "[excessively long Subject]";
      }
      set "subject" "I'm gone (was: ${orig_subject})";
   }

but clearly this isn't very practical :-).  there is an alternative
method which is more readable:

   require ["regex", "variables"];

   if header :regex "Subject" "(.{,3983})" {
      set "subject" "I'm gone (was: ${1})";
   }

but I'm thinking I specified the wrong behaviour when the limit is
exceeded in the first place.  it doesn't really solve any problems to
treat it as an error.  I think simply truncating silently is better.
users who are worried by this, can code defensively.  such long strings
are after all quite uncommon, and will probably mostly occur in the
context of vacation.  a truncated vacation message is better than no
message at all, IMO.
-- 
Kjetil T.

---

From: [email-masked]
Date: Thu, 05 Aug 2004 07:34:20 -0700 (PDT)

> context of vacation.  a truncated vacation message is better than no

I agree with your analysis. Truncation is the better option.

				Ned