Subject: Re: [sieve] Fwd: New Version Notification for draft-bosch-sieve-duplicate-01.txt

From: Stephan Bosch <[email-masked]>
Date: Sat, 09 Mar 2013 12:08:41 +0100

Hi,

On 2/10/2013 1:12 AM, Stephan Bosch wrote:
> Hi,
>
> I made a new version of the "duplicate" draft. It should address the 
> comments by Ned and Alexey.

Any thoughts about this?

Regards,

Stephan.

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 10 Mar 2013 14:01:20 +0000

On 9 Mar 2013, at 11:08, Stephan Bosch <[email-masked]> wrote:

>> I made a new version of the "duplicate" draft. It should address the comments by Ned and Alexey.
> 
> Any thoughts about this?

Hi Stephan,
This looks much better, thank you.

One new issue is that I don't understand how :value works. Maybe it is a Sunday morning effect. Maybe you can show an example or two?

---

From: Stephan Bosch <[email-masked]>
Date: Tue, 12 Mar 2013 22:14:06 +0100

Hi Alexey,

> One new issue is that I don't understand how :value works. Maybe it is a Sunday morning effect. Maybe you can show an example or two?

To detect duplicates, some more or less unique identifying string value 
must be extracted from the message and compared between deliveries. For 
most cases, the contents of the Message-ID header is the best choice; 
that is what it is meant for. However, to accommodate more complex uses, 
the duplicate test has the flexibility to choose a different string 
value for comparison. Using the :value argument, the string value can be 
set manually to an arbitrary string. Typically, one would compose this 
string using variable substitutions. One can for example compose it from 
parts of a header, the envelope, multiple headers, parts of the message 
body or even the current date; whatever is appropriate for the application.

There is one example of the use of :value in the draft (the second one). 
I should perhaps make the subject header structure a bit more complex to 
justify its use in this example (because :header would work here as well).

Another example would be the following scenario. A mail user receives 
XMPP notifications about new mail through Sieve. Sometimes a single 
contact sends many messages in a short period of time and the user wants 
to prevent being notified of all of those messages. He wants to be 
notified about messages from each person at most once per 30 minutes. He 
writes the following script:

require ["variables", "envelope","enotify", "duplicate"];

if envelope :matches "from" "*" { set "sender" "${1}"; }
if header :matches "subject" "*" { set "subject" "${1}"; }

if not duplicate :seconds 1800 :value "${sender}") {
     notify :message "[SIEVE] ${from}: ${subject}" 
"xmpp:[email-masked]";
}

So this example uses the message envelope sender rather than the 
Message-ID header as an identifier for duplicate checking.

This example can be extended to allow multiple messages from the same 
sender in close succession as long as the discussed topic is different. 
This could be done as follows:

require ["variables", "envelope","enotify", "duplicate"];

if envelope :matches "from" "*" { set "sender" "${1}"; }
if header :matches "subject" "*" { set "subject" "${1}"; }

if string :comparator "i;ascii-casemap" :matches "${subject}" "Re:*" {
     set "topic" "${1}";
} else {
     set "topic" "${subject}";
}

if not duplicate :seconds 1800 :value "${sender} ${topic}") {
     notify :message "[SIEVE] ${from}: ${topic}" "xmpp:[email-masked]";
}

Regards,

Stephan.

---

From: [email-masked]
Date: Fri, 15 Mar 2013 10:39:36 -0700 (PDT)

> Any thoughts about this?

Sorry to take so long to respond; I finally managed to complete my
implementation yesterday.

In implementing this, I saw nothing about the semantics of the test I would
like to see changed. I really like the clever way the single word "duplicate"
ends up doing the right thing in most cases.

The draft also seems pretty complete, although a couple of clarifications
are needed:

(0) A clear execution model is really important here. In particular, the way
    this has to be implemented is that during sieve execution queries
    whatever back end database you're using but does *not* update it.
    Only after the sieve has been evalauted without error and the actions
    it specified have been perform should the database be updated.

    I note that the same applies in environments that support multiple sieves -
    you evaluate them all, apply the results, and for the subset of sieves
    that evaluated without error, perform the updates. I'll also note 
    that how duplicate tests in multiple scripts interact is tricky. I
    eventually decided to qualify duplicate with the sieve "owner", which
    then acts like another handle.

    I don't see any reason to get into the multiple scripts bit in the draft;
    I'm just noting it in case someone is interested.

(1) The draft says nothing about what happens when the specified header
    field does not exist. The test should fail unconditionally in this case.

(2) The draft says nothing about what happens when there are more than one
    of the specified fields present. It should say that the first one
    that is found is used.

(3) The draft probably should say that duplicate does not support either the
    index or mime extensions directly (:index, :mime:, and :anychild), and
    that if you want to perform such checks you can do them with variables
    and :value.

And finally, a few nits:

(1) The "MUST throw a compile time error if both :header and :value are
    specified" is a bit strong. For one thing, such a check is inappropriate
    if ihave is in effect. And for another, while I was able to implement
    it, it may not be convenient for all implementations to perform this
    check at compile time.

(2) I'm wondering if it wouldn't be best to suggest a default timeout.
    12 hours? A day?

(3) A MUST be case-sensitive is probably a good idea, perhaps with a note
    as to how you can use the set action to muck around with case.

That's it! Thanks for producing such a good draft!

				Ned

---

From: Kristin Hubner <[email-masked]>
Date: Fri, 15 Mar 2013 14:09:52 -0700

A minor comment: I'm not crazy about the ":value" tag name, as I find it
less self-explanatory than I'd prefer.  I'd propose instead ":key", or 
or maybe ":uniqueid".  

Regards,

Kristin

On Mar 12, 2013, at 2:14 PM, Stephan Bosch wrote:

> To detect duplicates, some more or less unique identifying string value must be extracted from the message and compared between deliveries. For most cases, the contents of the Message-ID header is the best choice; that is what it is meant for. However, to accommodate more complex uses, the duplicate test has the flexibility to choose a different string value for comparison. Using the :value argument, the string value can be set manually to an arbitrary string. Typically, one would compose this string using variable substitutions. One can for example compose it from parts of a header, the envelope, multiple headers, parts of the message body or even the current date; whatever is appropriate for the application.
> 
> There is one example of the use of :value in the draft (the second one). I should perhaps make the subject header structure a bit more complex to justify its use in this example (because :header would work here as well).
> 
> Another example would be the following scenario. A mail user receives XMPP notifications about new mail through Sieve. Sometimes a single contact sends many messages in a short period of time and the user wants to prevent being notified of all of those messages. He wants to be notified about messages from each person at most once per 30 minutes. He writes the following script:
> 
>    notify :message "[SIEVE] ${from}: ${subject}" "xmpp:[email-masked]";
> }
> 
> So this example uses the message envelope sender rather than the Message-ID header as an identifier for duplicate checking.
> 
> This example can be extended to allow multiple messages from the same sender in close succession as long as the discussed topic is different. This could be done as follows:
> 
> _______________________________________________
> sieve mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sieve

---

From: Stephan Bosch <[email-masked]>
Date: Thu, 21 Mar 2013 13:06:01 +0100

Hi Ned,

>    it specified have been perform should the database be updated.

Ok, I'll improve that a bit.

>    I'm just noting it in case someone is interested.

Agreed. Perhaps when someone finishes the multiscript draft at some 
point, it could be mentioned there.

>    field does not exist. The test should fail unconditionally in this 

Agreed.

>    of the specified fields present. It should say that the first one

Yes, good catch. This is what my current implementation does too.

>    that if you want to perform such checks you can do them with variables

Yes, always good to give hints in the right direction.

>    it, it may not be convenient for all implementations to perform this

I'm not sure what you mean here. Do you want me to make the kind of 
error more open, e.g. by adopting the wording used for the :zone and 
:originalzone arguments for the date extension?

`It is an error to specify both :zone and :originalzone.'

Which basically leaves the kind of error up to the implementer. I have no problem with that.

> (2) I'm wondering if it wouldn't be best to suggest a default timeout.

It can be useful. Anyone else got suggestions or comments about this?

>    as to how you can use the set action to muck around with case.

Yes, otherwise users could get surprised at some point.

> That's it! Thanks for producing such a good draft!

Thanks. I'll make a new version some time next week.

Regards,

Stephan.

---

From: Stephan Bosch <[email-masked]>
Date: Thu, 21 Mar 2013 13:11:25 +0100

> less self-explanatory than I'd prefer.  I'd propose instead ":key", or

Ok, I understand, but I don't really have a preference; either ":key" or 
":uniqueid" makes sense. Anyone else got suggestions or strong opinions 
about this? :)

Regards,

Stephan.

---

From: [email-masked]
Date: Thu, 21 Mar 2013 07:37:29 -0700 (PDT)

> Which basically leaves the kind of error up to the implementer. I have no problem with that.

I was unclear, sorry. I was talking about having to catch the error at compile
time. I think catching it at execution time should be allowed.

> It can be useful. Anyone else got suggestions or comments about this?

Yeah, this is one where additional comments would really be nice.

> Thanks. I'll make a new version some time next week.

Excellent.

				Ned