Subject: [sieve] Duplicate test

From: Stephan Bosch <[email-masked]>
Date: Mon, 14 Jan 2013 13:23:35 +0100

Hi,

A while back I built a non-standard Sieve extension for detecting 
duplicate message deliveries, e.g. as seen when people cross-post over 
multiple mailing lists. This functionality is very similar to what could 
be achieved with Procmail using formail -D. Someone requested this at 
some point and in a moment of boredom I built the initial 
implementation. Last year I extended it a bit and I made it more 
generic. I never intended for this to be standardized, but - now that I 
look at it again - I think this may yet be of some interest to other 
Sieve implementers as well.

I make a habit of documenting my custom Sieve extensions in a familiar 
RFC-like manner:

http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/doc/rfc/spec-bosch-sieve-duplicate.txt

What do you think? Would you be interested in having functionality like 
this standardized? Should I submit this as a draft?

Regards,

Stephan.

---

From: [email-masked]
Date: Tue, 15 Jan 2013 19:10:13 -0800 (PST)

> this standardized? Should I submit this as a draft?

I think the underlying idea is excellent and deserves to be standardized in one
form or another. I also like the way the optional parameters make the test easy
to use in the default way, but provide the flexibility to test arbitrary string
parameters.

About the only thing that's missing is the ability to do things like count
occurances, and I'm pretty strongly inclined not to add that.

So yes, please submit this and let's work on getting it standardized.

				Ned

---

From: Stephan Bosch <[email-masked]>
Date: Thu, 17 Jan 2013 12:53:09 +0100

Hi Ned,

> So yes, please submit this and let's work on getting it standardized.

Ok, I turned it into an Internet draft.

Here is the submission:

https://datatracker.ietf.org/doc/draft-bosch-sieve-duplicate/
http://tools.ietf.org/html/draft-bosch-sieve-duplicate-00

Regards,

Stephan.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 16 Jan 2013 22:36:19 +0000

On 14/01/2013 12:23, Stephan Bosch wrote:
> multiple mailing lists. This functionality is very similar to what 
> could be achieved with Procmail using formail -D. Someone requested 
> this at some point and in a moment of boredom I built the initial 
> generic. I never intended for this to be standardized, but - now that 
> I look at it again - I think this may yet be of some interest to other 
> What do you think? Would you be interested in having functionality 
> like this standardized? Should I submit this as a draft?
I think this would be worth standardizing. Some comments on the document:

1). I think the text motivating existence of this extension needs a bit 
of restructuring: it first introduces a very generic concept of matching 
and only later it talks about duplicate suppression. I would reverse 
that: describe some specific use cases first, then generalize.

2). I found that "the side effect of the test takes force at the end of 
a successful script execution" to be odd/awkward. Maybe we can have an 
explicit action? Or maybe leaving this as is is Ok... Need to think a 
bit more about that.

Otherwise this looks quite good to me.

Best Regards,
Alexey

---

From: [email-masked]
Date: Fri, 18 Jan 2013 11:22:30 -0800 (PST)

> explicit action? Or maybe leaving this as is is Ok... Need to think a

I'm not wild about this either, mostly because when you do this as two steps
there's a window where overlapping script executions can cause a duplicate to
be missed. Of course locking can be used to prevent this, but given the scale
we operate at any use of locking has to be carefully considered.

That said, the problem where a script goes wonky and ends up marking something
as a duplicate is much more severe, so IMO the draft is handling this the right
way.

tl;dr: Missing a duplicate is preferable to marking a message as a duplicate
incorrectly.

				Ned

---

From: Stephan Bosch <[email-masked]>
Date: Sun, 20 Jan 2013 13:57:20 +0100

On 1/16/2013 11:36 PM, Alexey Melnikov wrote:
>> duplicate message deliveries, e.g. as seen when people cross-post 
>> over multiple mailing lists. This functionality is very similar to 
>> what could be achieved with Procmail using formail -D. Someone 
>> requested this at some point and in a moment of boredom I built the 
>> initial implementation. Last year I extended it a bit and I made it 
>> more generic. I never intended for this to be standardized, but - now 
>> that I look at it again - I think this may yet be of some interest to 
>> other Sieve implementers as well.
>>
>> I make a habit of documenting my custom Sieve extensions in a 
>> familiar RFC-like manner:
>>
> 1). I think the text motivating existence of this extension needs a 
> bit of restructuring: it first introduces a very generic concept of 
> matching and only later it talks about duplicate suppression. I would 
> reverse that: describe some specific use cases first, then generalize.

Agreed. I'll revise that in the next version.

Regards,

Stephan.

---

From: Stephan Bosch <[email-masked]>
Date: Sun, 20 Jan 2013 14:00:36 +0100

> tl;dr: Missing a duplicate is preferable to marking a message as a 

Exactly.

Perhaps I should mention these considerations more explicitly in the draft?

Regards,

Stephan.

---

From: [email-masked]
Date: Sun, 20 Jan 2013 14:27:53 -0800 (PST)

> Perhaps I should mention these considerations more explicitly in the draft?

Yes, I think that's a good idea.

				Ned