Subject: RE: [lemonade] IMAP-SIEVE: Per-Client Sieve Script

From: <[email-masked]>
Date: Mon, 3 Sep 2007 11:27:30 +0300

Thank You Randall,
I have further questions/comments.

>>  What is the execution order of the scripts (last modification 
>> date/alphabetical/etc)? I am asking this because one script 
>might move 
>> an email while another might notify it - which one happens first...
>
>My suggested text included:
>
>>  After all Sieve scripts have run, the implementation MAY 
>then expunge 
>> the original message
>
>This way, the message will remain in the folder while all 
>scripts run, so multiple scripts may move it, resulting in the 
>message effectively being copied.  As for modifications, 
>existing text in the document says modifications don't show up 
>to the script, so in effect they likewise take effect after 
>all scripts have run.

But I see no text saying that the message MUST NOT be expunged until all
scripts have finished. Perhaps it would contradict a MUST somewhere in
the SIEVE specs?

>>  What if there are two scripts and: both want to send a 
>notification, 
>> and to the same/different addresses? Which will be sent? Both?
>
>Yes.  Each scripts needs to operate independently.  
>Separately, as an optimization, an implementation could 
>coalesce multiple notifications to the same address into a 
>single notification.

Ok, so let' take this a step further.
There is a new mail arriving, and we have scripts A, B and C from
clients A, B and C respectively. The new mail triggers both A and B
scripts.
The result of script A is to delete the email. The result of script B is
to move (copy+delete, right?) the email to a different mailbox. The
result of script C is to notify about new mail. So,  A copy of the email
is placed into another mailbox and finally the mail is deleted and C is
notified. The deposit of the new mail to the other mailbox (the copy
action of script B) triggers at least script A (since it was script B
making the change, I assume that it will not trigger itself to avoid
loops) and C. The result of script C is to notify about new mail. The
result of script A is to delete the email (same result as before).
Script A triggers script B and C. B does nothing, C is notified again.
Problems:
 -  the email that B wanted to move somewhere else is deleted
 - C receives four notifications when "nothing" really happened (a new
mail arrived, copied, but deleted both). This can be optimized though.
 - If C wanted a copy of the mail (justl like B), script A would have
deleted it.
 - What if scripts A and B moved (copy+delete) the email between two
different mailboxes continuously (infinite loop)?

>>  The original text refers to these problems as: " complexity and 
>> possible ambiguity involved with coordinating the results of 
>> multiple scripts".
>>  Should these problems be elaborated in more detail and - hopefully 
>> - provide a standardized behavior?
>
>I have attempted to provide a standardized behavior.  If you feel 
>more detail is needed, or that there is still ambiguity, please point 
>out where.

There is nothing wrong with the text - I just got the feeling that
something is missing. I can't tell what, but something does not feel
right. Perhaps after we went through a few use cases, we can come up
with a solid proposal.

>>  One more question if I may:
>>  How do You identify (uniquely) which script belongs to which 
>> client? I mean if we can't do this and clients keep overwriting 
>> each others' scripts (because they don't know which one belongs to 
>> them), we did not achieve anything, did we?
>
>My suggested text included:
>
>At 3:40 PM -0700 8/13/07, Randall Gellens wrote:
>>     The entry name below ""/IMAPSieve/Script/client/" SHOULD be
>>     named <client-id>.<script-id>.  For example, "Eudora.FC2302".
>
>Hence, each client is assigned a <client-id> through an IANA 
>registry, perhaps an existing one, or a new one can be created.  Each 
>client is free to choose a <script-id> of its own creation.
> I was 
>thinking that any identifier for the device could be used, but even a 
>random number with a reasonable range would suffice.

I guess a client could figure out the allocated random IDs by doing a
simple query to the server - so it could figure out a unique random ID
for itself if needed.

Could we add <random> then (with a requirement saying that it must be
unique within the scope of the client-id; could be any number of hexa
digits, case insensitive? 0-9, a-f, A-F)?
So the whole ID would look like: <client-id>.<random-id>.<script-id>

A requirement might be also needed to store the <random id> in a safe
place. Some clients overwrite/delete user data when the
OS/client/firmware is updated. If the random id is lost, there will be
no way to identify which client owned the script and it wil be hanging
there for an eternity.

Could this ID be re-used? Well, at least the <client-id>.<random-id>
part of it - for example, to store client-specific preferences on the
server?

>  There was never 
>any intention of allowing clients to "keep overwriting each others' 
>scripts (because they don't know which one belongs to them)" -- 
>instead, the idea is to err on the side of possible extra scripts, 
>perhaps because a client has forgotten its identifier.  Those 
>obsolete scripts would eventually be reaped by the server, as 
>described in my suggested text.

Yes, this is my goal as well. Just trying to make it fool-proof.

>-- 
>Randall Gellens
>Opinions are personal;    facts are suspect;    I speak for myself only
>-------------- Randomly-selected tag: ---------------
>The number of licorice gumballs you get out of a gumball machine
>increases in direct proportion to how much you hate licorice.
>
>
>_______________________________________________
>lemonade mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/lemonade
>Supplemental Web Site:
>http://www.standardstrack.com/ietf/lemonade
>

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade
Supplemental Web Site:
http://www.standardstrack.com/ietf/lemonade

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 04 Sep 2007 10:03:11 -0400

Hi,

>> coalesce multiple notifications to the same address into a

I think as Zoltan's examples show this is getting way too complicated. 
Having multiple "competing" scripts being executed like this is a recipe 
for disaster. Lets drop this aspect of the spec and see whether the rest 
will actually get implemented and be interoperable first. Then, if there 
really is demand, we can consider per-client scripts as an extension.

-- 
Cyrus Daboo

---

From: Dan Karp <[email-masked]>
Date: Tue, 4 Sep 2007 07:26:13 -0700 (PDT)

> really is demand, we can consider per-client scripts as an extension.

Hear, hear.

---

From: <[email-masked]>
Date: Wed, 5 Sep 2007 10:22:35 +0300

Hi Cyrus,
Randall has a valid point and sooner or later we are going to tackle the multi-client issue.
I think it would be beneficial to address these issues - if possible, before the Lemonade release.

Best regards: Zoltán Ördögh
E-mail: zoltan dot ordogh at nokia dot com
Phone: +358 50 386 0566

>-----Original Message-----
>From: ext Cyrus Daboo [mailto:[email-masked]] 
>Sent: 04 September, 2007 17:03
>To: Ordogh Zoltan (Nokia-TP-MSW/Tampere); [email-masked]; 
>[email-masked]
>Cc: [email-masked]
>Subject: RE: [lemonade] IMAP-SIEVE: Per-Client Sieve Script
>
>Hi,
>
>>> Separately, as an optimization, an implementation could coalesce 
>>> multiple notifications to the same address into a single 
>>> notification.
>
>Having multiple "competing" scripts being executed like this 
>is a recipe for disaster. Lets drop this aspect of the spec 
>and see whether the rest will actually get implemented and be 
>interoperable first. Then, if there really is demand, we can 
>consider per-client scripts as an extension.
>
>--
>Cyrus Daboo
>
>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 5 Sep 2007 09:46:14 +0200

> before the Lemonade release.

Then let's drop them and discuss them separately, and if it's resolved 
before WGLC we can always put them back in.

Personally I have absolutely no idea how to do this, but I see Randall's 
point, so I'm afraid I cannot contribute right now. I will if I think 
of something.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Wed, 05 Sep 2007 09:35:35 -0400

Hi Zoltan,

--On September 5, 2007 10:22:35 AM +0300 [email-masked] wrote:

> Randall has a valid point and sooner or later we are going to tackle the
> multi-client issue. I think it would be beneficial to address these
> issues - if possible, before the Lemonade release.

I disagree - I question whether there is a need to do it this way at all. I 
think an adequate solution for most cases is to allow manual triggering of 
scripts by clients. That way clients can apply their own scripts when they 
actually interact with the server.

The whole business of allowing clients to upload their own scripts which 
get automatically handled and considered "private" to each client is 
awkward. For example, what happens if a user stops using a client? How does 
the script for that client get disabled? Does there have to be some sort of 
script management UI in every client so users can manage other client's 
scripts use etc?

I think we should remove this for now, finish the spec with the single 
script behavior, deploy it and then see what the practical needs of clients 
are rather than the current theoretical needs.

-- 
Cyrus Daboo

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 Sep 2007 13:22:11 -0700 (PDT)

> really is demand, we can consider per-client scripts as an extension.

+1

I have actually implemented handling of multiple script handling in the context
of delivery processing. There are all sorts of ways to do it, but the goal has
to be that things manage to work in the way users expect. I eventually arrived
at a result that was remarkably close to the recommendations Jutta Degener did
(she wrote a droaft some time back on how to handle multiple delivery time
sieves).

However, delivery time processing is a better defined problem (it is, after
all, the place Sieve was deisned to operate) and there are some simplifying
assumptions having to do with ranking of multiple sieves that I don't think are
possible in the context of IMAP. I therefore support dropping this for now.

				Ned

---

From: <[email-masked]>
Date: Thu, 6 Sep 2007 12:01:12 +0300

Ned, Cyrus, Arnt,
I think we all agree Randall does have a valid point - what we disagree about is the means to deal with the problem. Is this a valid assessment so far?

My worry here is that we might release a genie from the bottle.
I mean if Lemonade bis was released/deployed as is (with a single script setup), we might not be able to change that in the future - well, at least not easily and quickly because we are going to have to provide backwards compatibility until the old services are phase out.
Would it be possible to build in some sort of safeguard so that we can change this later on without making it incompatible with existing Lemonade deployment? Some sort of future-proof-ish solution?

>From: ext Ned Freed [mailto:[email-masked]] 
>Sent: 05 September, 2007 23:22
>To: Cyrus Daboo
>Cc: Ordogh Zoltan (Nokia-TP-MSW/Tampere); [email-masked]; 
>[email-masked]; [email-masked]
>> Having multiple "competing" scripts being executed like this is a 
>> recipe for disaster. Lets drop this aspect of the spec and 
>see whether 
>> the rest will actually get implemented and be interoperable first. 
>> Then, if there really is demand, we can consider per-client 
>scripts as an extension.
>
>+1
>
>I have actually implemented handling of multiple script 
>handling in the context of delivery processing. There are all 
>sorts of ways to do it, but the goal has to be that things 
>manage to work in the way users expect. I eventually arrived 
>at a result that was remarkably close to the recommendations 
>Jutta Degener did (she wrote a droaft some time back on how to 
>handle multiple delivery time sieves).
>
>However, delivery time processing is a better defined problem 
>(it is, after all, the place Sieve was deisned to operate) and 
>there are some simplifying assumptions having to do with 
>ranking of multiple sieves that I don't think are possible in 
>the context of IMAP. I therefore support dropping this for now.
>
>				Ned
>