Subject: Re: [imapext] Pete Resnick's Discuss on draft-ietf-imapmove-command-03: (with DISCUSS and COMMENT)

From: Barry Leiba <[email-masked]>
Date: Sat, 24 Nov 2012 09:42:42 -0500

I'm adding the working group mailing list to this, because I think it
needs to be aired there.

> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> I have not been following this WG, and I might have the Sieve bit wrong,
> so I apologize in advance if this issue has been considered and addressed
> by the WG; if so, I will remove this DISCUSS immediately. But this causes
> me concern:
>
> Section 4.5:
>
>    Because MOVE does not cause flags
>    to be changed, a MOVE command will not result in a script invocation
>    with the imap.cause set to "FLAG".
>
> I think this is either wrong or needs more explanation. Section section
> 3.3 is clear that doing a move "has the same effect for each message
> as...[UID] STORE +FLAGS.SILENT \DELETED", which the above seems to
> contradict. But even this is intended, I would think this would at the
> very least cause significant user surprise: In a current client, if I
> choose the "move" command in the UI, I'm going to get an invocation for
> the COPY on the destination message *and* an invocation for the STORE
> \DELETED on the source message. You are saying that an upgraded client
> that starts using "MOVE" for it's "move" command will suddenly stop
> invoking for the deletion. If you *don't* want Sieve to invoke for the
> flag change, I think you're going to have to add a new imap.cause of
> either MOVE or EXPUNGE, and folks will have to go rewrite their scripts.
> But I would think the easier route would be to get a second invocation
> from MOVE with imap.cause set to "FLAG".

There appear to be two points here:

1. The MOVE spec is saying that in the end, the MOVE command has the
same effect as the COPY/STORE/EXPUNGE sequence that clients are doing
today.  It does NOT mean that the MOVE command is equivalent to that
sequence in every way, including intermediate states.  It's precisely
the purpose of making it a single command that we get away from the
intermediate state problem, and allow servers to optimize a MOVE over
a COPY/DELETE function.

We absolutely do NOT want to make servers present that "\Deleted flag
is set" state in any context.

2. We did discuss, though only very briefly, whether to have a
separate imapsieve cause for this, and decided that it wasn't worth
the complexity.  No one has a use case for needing it.  If you do,
we'd like to hear it..

Perhaps there might be a use case if, say, one wanted to use a Sieve
script to take any message that was being removed from the inbox and
do "fileinto Archive" on it.  That use case would be marginal now,
because there's no imapsieve event for expunge (only for the flag
change on \Deleted), and no way in Sieve to remove a message from
Archive, so constantly setting and resetting the \Deleted flag,
without ever expunging, would just cause the message to be repeatedly
re-filed into Archive.  But anyway, if we accept some use case like
that, we might need something here.

But it gets complicated.

The COPY cause is run against the target mailbox only.  We'd have to
have a new cause that's run against the source mailbox in *addition*
to that.  Say, a MOVE (out of) cause that's run against the source
mailbox's script before the move, and then a COPY (into) cause that's
run against the target mailbox's script after the move.  And then we'd
have to ask whether there's a need for a new environment variable to
name the other mailbox that's involved (we had no use case to do this
for the COPY cause, so we didn't), because we might want to know where
it was being moved TO, or where it's being copied FROM, for processing
in the script.

As I said, no one wanted the extra complication, and no one had a use
case for needing it.

> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> Section 1:
>
> Purely editorial nonsense:
>
>    clients have instead had to use a combination of the UID STORE, UID
>    COPY and EXPUNGE commands to perform this very common operation.
>
> Seems to me that you should list the commands without the UID (since
> they're not required) and in the right order. So instead:
>
>    clients have instead had to use a combination of the COPY, STORE,
>    and EXPUNGE commands to perform this very common operation.
>
> And if you have any reason for keeping the original, do so. Just one AD's
> OCD.

That's probably an appropriate change -- the UID versions are a
vestige of the original spec, which ONLY had UID MOVE, and did not
define a non-UID version.

Barry

---

From: Ned Freed <[email-masked]>
Date: Sat, 24 Nov 2012 11:36:45 -0800 (PST)

> intermediate state problem, and allow servers to optimize a MOVE over

Yes, and we've been struggling with how to present this from the start.
The fact that Pete misunderstood this suggests that the document still
isn't clear about some aspects of this. (See below for some suggested text.)

> is set" state in any context.

Absolutely. Perhaps we need to add something like this before the third
paragraph in section 3.3:

   Although the effect of MOVE is the same as the preceeding steps,
   the intermediate states produced by those steps do not occur. In
   particular, at no time will the \DELETED flag be seen to be set.

> the complexity.  No one has a use case for needing it.  If you do,

Specifically, I brought up the idea of a "MOVE" cause. Nobody seemed interested
in it.

> As I said, no one wanted the extra complication, and no one had a use

Agreed on all points. I'm not especially happy with this result, but I'm
nowhere near convinced that doing something like, say, applying a Sieve to the
source mailbox with a "movefrom" cause then applying a Sieve to the target
mailbox with a "moveto" cause is worth it. I am convinced, however, that
exposing intemediate state, which a "flags" cause absolutely would do, would be
very bad.

I'll also note that Sieve in IMAP has always been a tough nut semantically.
(And this really has little to do with Sieve; the semantics of IMAP weren't
designed with coherent generation of mailbox events in mind.) While I think
we've done a good job on Sieve in IMAP, if usage takes off I have a feeling
we'll need to revisit some of the choices we've made.

> define a non-UID version.

Agreed. I need to do another revision anyway; this will be in it.

				Ned

---

From: Jan Kundrát <[email-masked]>
Date: Mon, 26 Nov 2012 16:13:30 +0100

>    particular, at no time will the \DELETED flag be seen to be set.

(Replying just to imapext, please let me know if this should've gone to all recipients instead.)

Does it mean that servers SHOULD NOT send the FLAGS \Deleted message to (this, or any other parallel) IMAP session, or does it mean that they MUST NOT do so? It is not clear to me from this wording.

With kind regards,
Jan

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 26 Nov 2012 15:55:43 +0000

Hi Jan,

On 26/11/2012 15:13, Jan Kundrát wrote:
> Does it mean that servers SHOULD NOT send the FLAGS \Deleted message 
> to (this, or any other parallel) IMAP session, or does it mean that 
> they MUST NOT do so? It is not clear to me from this wording.

I don't think it actually matters. If a server mistakenly sends \Deleted 
on MOVE, it doesn't look like any harm will occur. Feel free to disagree 
with me, of course.

---

From: Pete Resnick <[email-masked]>
Date: Mon, 26 Nov 2012 10:29:08 -0600

>     particular, at no time will the \DELETED flag be seen to be set.

If you're going this route, maybe even a bit more would be useful:

    Although the effect of the MOVE is the same as the preceding steps,
    the semantics are not identical: The intermediate states produced by
    those steps do not occur, and the response codes are different. In
    particular, though the COPY and EXPUNGE response codes will return,
    no response code for a STORE will be generated nor will the \DELETED
    flag be set for any message.

Too much?

> Specifically, I brought up the idea of a "MOVE" cause. Nobody seemed interested

And it's tricky: I mentioned an EXPUNGE cause above, but that would have 
to trigger before the message was actually gone from the original 
mailbox (because otherwise you wouldn't have access to the message that 
just got expunged). A MOVE clause is a little better since you would 
have access to the destination message. But if nobody is interested, so 
be it.

> exposing intemediate state, which a "flags" cause absolutely would do, would be

I presume you don't want the intermediate state because it would make 
the command non-atomic? Yup, I guess so. :-/

OK, I'll clear the DISCUSS and note the change in the COMMENT. Thanks.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Cyrus Daboo <[email-masked]>
Date: Mon, 26 Nov 2012 11:44:23 -0500

Hi Alexey,

> on MOVE, it doesn't look like any harm will occur. Feel free to disagree

If we can I think we need to say "servers MUST NOT send unsolicited FLAGS 
\Deleted messages in other client sessions that have the mailbox selected, 
only EXPUNGED or VANISHED are sent". Why? Well scalability - if a user 
moves 1000 messages (something I occasionally do when "archiving" 
messages), having 1000 unsolicited deletes go to other clients is a waste 
of bandwidth/cpu.

-- 
Cyrus Daboo

---

From: Ned Freed <[email-masked]>
Date: Mon, 26 Nov 2012 08:10:04 -0800 (PST)

> Does it mean that servers SHOULD NOT send the FLAGS \Deleted message to
> (this, or any other parallel) IMAP session, or does it mean that they MUST NOT
> do so? It is not clear to me from this wording.
                                                       
I have no idea what "send" a "message" means in this context. The text says
that the \Deleted flag must not be visibly set at any time during the
operation. We can use compliance language for that to make it cleaer, but what
happens inside the server is not our concern and should not be specified. 

				Ned

---

From: Jan Kundrát <[email-masked]>
Date: Mon, 26 Nov 2012 18:27:59 +0100

> happens inside the server is not our concern and should not be specified. 

Using SHOULD NOT or MUST NOT is what I was requesting.

And for the record, my understanding of draft-ietf-imapmove-command-03 (as posted on IETF's website) is that it does not say anything about this visibility of the immediate state (which means that it was allowed). I support this change and hope it isn't a too strict requirement for the implementors. In particular, as a client author, I'd prefer to send UID MOVE to a server which returns me back useless FLAGS \Deleted response than send three command myself. From my point of view, having a SHOULD NOT is fine with me.

With kind regards,
Jan

---

From: Ned Freed <[email-masked]>
Date: Mon, 26 Nov 2012 11:31:14 -0800 (PST)

> on MOVE, it doesn't look like any harm will occur. Feel free to disagree

Cyrus described one case where it matters. Another is the Sieve interfation:
Sending out a \DELETED flag change without running the Sieve with a flag
change cause creates a silly state.

I'm strongly inclined to go with a MUST NOT here.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 26 Nov 2012 22:33:03 +0000

Hi Ned,

On 26 Nov 2012, at 19:31, Ned Freed <[email-masked]> wrote:

> Cyrus described one case where it matters.

Yeah, I wanted to say that there was wasted bandwidth.

> Another is the Sieve interfation:

No objection from me (speaking as a participant).

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Tue, 27 Nov 2012 00:17:01 +0000

------ Original Message ------
From: "Ned Freed" <[email-masked]>
>>to 
>I have no idea what "send" a "message" means in this context.

I guess we should use language like "emit a PDU", which I take to be 
the meaning here.

Regards

Adrien

> The text says 
>_______________________________________________ 
>imapext mailing list 
>[email-masked] 
>https://www.ietf.org/mailman/listinfo/imapext

---

From: Ned Freed <[email-masked]>
Date: Mon, 26 Nov 2012 17:35:55 -0800 (PST)

> I guess we should use language like "emit a PDU", which I take to be

I've since realized it probably refers to responses being sent in this and
other sessions.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Mon, 26 Nov 2012 19:09:29 -0800 (PST)

>     flag be set for any message.

I've opted for a variant of this that uses MUST NOT compliance language.

> have access to the destination message. But if nobody is interested, so

That seems to be the case. As I've commented before, we may have to revisit
various things about Sieve in IMAP if it sees significant deployment.

				Ned