Subject: [imapext] Preparing for WGLC on IMAP MOVE: AD review

From: Barry Leiba <[email-masked]>
Date: Sun, 16 Sep 2012 17:17:13 -0400

Because it looks like we're close to WGLC on the MOVE spec, I thought
I would do an early AD review on it, so we can get any changes
resulting from that into -02 and WGLC *that*.  Consider what's below
in addition to my earlier comments about de-emphasizing the COPY +
STORE + EXPUNGE bit, and not making that the *definition* of MOVE
(here: http://www.ietf.org/mail-archive/web/imapext/current/msg04694.html
).

-- Abstract --
The abstract needs to stand alone and give a little bit of context,
because people will just see the title and abstract sometimes.  I
suggest fixing the Overview as below, and then copying it verbatim
into the Abstract.  Also, please move the Abstract to be the first
thing in the document, before the boilerplate.  That matches the
current I-D format.  (It will also make it look less silly to have the
Abstract and the Overview identical.)

-- Overview --
First, please move this to be Section 1, and make the "conventions"
section be either 1.1 or 2.  The RFC Editor is likely to make that
change anyway, and other reviewers will appreciate it.

Then:
OLD
   clients have to use a combination of UID
   STORE, UID COPY and EXPUNGE, and cope with partial failures and side
   effects.
NEW
   clients have had to use a combination of UID
   STORE, UID COPY and EXPUNGE commands to create that function.
   This has meant needing to cope with partial failures and side effects.

-- Section 3 --
RFC 3501 has a consistent way to define commands, and many of the
extensions we write use that format.  I would like to use it here, as
well, to match up with the base IMAP spec.  I suggest something like
this:

----------
3. MOVE and UID MOVE

3.1  MOVE Command

   Arguments:  sequence set
               mailbox name

   Responses:  no specific responses for this command

   Result:     OK - move completed
               NO - move error: can't move those messages or to that
                    name
               BAD - command unknown or arguments invalid

3.2  UID MOVE Command

This extends the first form of the UID command (see [RFC3501], Section
6.4.8) to add the MOVE command, defined above, as a valid argument.

3.3  How They Work

... [Etc, as you have in Section 3 now, but see below.]
----------

With that up front, you can now word the explanation of how things
work in terms of "MOVE", without having to say "UID MOVE" -- that was
an artifact of having done just UID MOVE originally anyway.  So, let's
take this paragraph by paragraph:

Paragraph 1:

   The MOVE command takes two arguments: a message set (sequence
   numbers for MOVE, UIDs for UID MOVE) and a named mailbox.  Each
   message included in the set is moved, rather than copied, from
   the selected (source) mailbox to the named (target) mailbox.

Paragraph 2:

   This means that a new message is created in the target mailbox,
   with a new UID, the original message is removed from the source
   mailbox, and it appears to the client as a single action.  This
   has the same effect for each message as this sequence:
   1. [UID] COPY
   2. [UID] STORE +FLAGS.SILENT \DELETED
   3. UID EXPUNGE
   Each message SHOULD either be moved or unaffected. The server
   MUST NOT leave a message in neither mailbox, and SHOULD NOT
   leave a message in both mailboxes afterwards, even if the server
   returns a tagged NO response.

Paragraph 3:
I suggest leaving all extension stuff for Section 4, and only say this here:

   Because of the similarity of MOVE to COPY, extensions that
   affect COPY affect MOVE in the same way.  Response codes such
   TRYCREATE (see [RFC3501] Section 6.4.7), as well as those
   defined by extensions, are sent as appropriate.  See Section 4
   for more information about certain IMAP extensions.

Paragraph 4 (the example):
No change needed.

Paragraph 5; I suggest a small clarification:

   Note that the server may send EXPUNGEs for other messages as well,
   if any happen to have been expunged at the same time; this is normal
   IMAP operation.

Paragraph 6; I suggest a small clarification:

   Implementers will need to read [RFC4315] to understand what UID
   EXPUNGE does, though full implementation of [RFC4315] is not necessary.

Paragraph 7:
I can just about guarantee that this will get questions from some ADs,
as it's currently worded.  How about this?:

   Note that moving messages to the current mailbox (that is, where the
   source and target mailboxes are the same) is well-defined, so that MOVE
   is well defined for all the cases where COPY is.

Paragraph 8:

   The server may send EXPUNGE (or VANISHED) messages before the tagged
   response, so the client cannot safely send more commands with message
   sequence number arguments while the server is processing MOVE.  The
   UID MOVE command does not have this limitation.

-- Section 4 --

Add a new Section:

   4.5.  IMAP Events in Sieve

   MOVE applies to IMAP events in Sieve [reference] in the same way
   as COPY does.  Therefore, MOVE can cause a Sieve script to be
   invoked with the imap.cause set to "COPY".  Because MOVE does not
   cause flags to be changed, a MOVE command will not result in a
   script invocation with the imap.cause set to "FLAG".

The [reference] for IMAP Events in Sieve is
draft-ietf-sieve-imap-sieve-09, which should be in the RFC Editor
queue any day now (so it won't hold this document up).

-- Section 6 --
I suggest some more thought and some re-wording on the Security
Considerations.  I'd be surprised if this didn't get pushback from the
Sec ADs as it is now.  Alexey can probably help here.

--
Barry, Responsible AD

---

From: Ned Freed <[email-masked]>
Date: Mon, 17 Sep 2012 11:30:30 -0700 (PDT)

> because people will just see the title and abstract sometimes.

I hear this all the time and while it's certainly true for paywalled material
I sometimes wonder how true it is for RFCs. But rules are rules, and your
point is well taken that some changes are needed to conform.

> I

If I understand you correctly, this has the effect of changing the Abstract
from:

   The MOVE extension provides commands to move one or more messages
   from the selected mailbox to a named mailbox.

to:

   This document defines an IMAP extension to move messages from one
   mailbox to another. This function (very common in MUA UIs) is not
   provided by stock IMAP, and clients have to use a combination of UID
   STORE, UID COPY and EXPUNGE, and cope with partial failures and side
   effects.

The one thing this loses that was there before is the idea that this is done
with new commands. In IMAP this is important because a lot of extensions add
stuff to existing commands. I would therefore suggest a change to something
like:

   This document defines an IMAP extension consisting of two new commands, MOVE
   and UID MOVE, that are used to move messages from one mailbox to another.
   This function (very common in MUA UIs) is not provided by stock IMAP, and
   clients have to use a combination of UID STORE, UID COPY and EXPUNGE, and
   cope with partial failures and side effects.

> ... [Etc, as you have in Section 3 now, but see below.]

Sorry, I should have caught this one. I agree we should use this convention.

> ...

These changes all look fine to me.

>    script invocation with the imap.cause set to "FLAG".

I think this is the right thing to do, but I have to ask: Given the timing
of the Sieve in IMAP spec we could define a new "MOVE" cause in the base
specification if we wanted to. Do we want to do this?

> Sec ADs as it is now.  Alexey can probably help here.

Is this the sort of thing you're looking for?

     MOVE does not introduce any new capabilities to IMAP, and this limits
     the security. However, the transactional semantics of MOVE may
     interact with specific implementations in ways that could have
     unexpected consequences. For example, moving messages between folders
     under the quota root may require temporary suspension of quota checking.

     An additional area of concern is interaction with antispam, antivirus,
     and other security scanning and auditing mechanisms. Different folders
     may have different security policies which could interact with MOVE
     in complex ways. Scanning with updated rules may also be required
     when messages are moved even with the underlying policy has not changed.

					Ned

---

From: Barry Leiba <[email-masked]>
Date: Mon, 17 Sep 2012 22:17:45 -0400

> point is well taken that some changes are needed to conform.

Actually, the title and abstract are all that go out in the
announcement message, and the RFC announcement list feeds other lists.
 It does have some value to have the RFC announcements have a useful
abstract.  Also, the RFC search features allow showing the abstract in
the search response.  You're right that it's not a big deal, and if it
were a pain in the ass to conform or if there were a good reason not
to, I'd be happy to push back on it.  But as it's easy....

>    cope with partial failures and side effects.

Yes, that's better; thanks.

> specification if we wanted to. Do we want to do this?

I thought about that when I wrote the above, and I can't find a good
reason to distinguish MOVE from COPY in imap-sieve.  It also wouldn't
be as simple as just defining a new cause: we'd have to decide whether
only the target-mailbox script is run, or whether the source-mailbox
script is run as well.  My sense is that it's not an important
distinction, and we don't want to go there.

(I will note, though, that it makes a notable difference between
copy/store/expunge and move: the former sequence causes a "flags"
script to be run in the source mailbox, in addition to triggering a
"copy" script to be run in the target mailbox.  I'm proposing only the
latter for move.  But I'm just noting that here; again, I think that's
beyond where we want to or should go.)

>      when messages are moved even with the underlying policy has not changed.

Yes, I like that proposal (with fixing the first sentence to "limits
the security effects", or some such).  Arnt, there's a good starting
point (and perhaps ending point as well).

Barry

---

From: Ned Freed <[email-masked]>
Date: Mon, 17 Sep 2012 19:29:06 -0700 (PDT)

> reason to distinguish MOVE from COPY in imap-sieve.

Neither can I, but I thought I'd ask the list to see if anyone else
disagrees.

> It also wouldn't

> beyond where we want to or should go.)

Certainly WFM.

				Ned

---

From: Barry Leiba <[email-masked]>
Date: Mon, 24 Sep 2012 20:32:37 -0400

Responses to this from anyone else (besides Ned)?
Arnt?  Bron?  Jan?  Timo?  Dan?
Should Arnt just roll in my two reviews, modulo Ned's responses, and
we're ready to go?  Or what?

Barry

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Tue, 25 Sep 2012 00:37:58 +0000

I don't have any concerns.  We'll modify our implementation when it's 
available, I think the only affected clients that I'm aware of don't 
use the COPYUID response anyway.

Regards

Adrien

------ Original Message ------
From: "Barry Leiba" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 25/09/2012 12:32:37 p.m.
Subject: Re: [imapext] Preparing for WGLC on IMAP MOVE: AD review
>_______________________________________________
>imapext mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/imapext
>
>

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 25 Sep 2012 03:49:36 +0300

Sorry, busy :( But I read through this thread and I don't see any problems.

Security section might want to mention the possibility of deadlock if the server works by locking both of the mailboxes.

---

From: Ned Freed <[email-masked]>
Date: Mon, 24 Sep 2012 17:58:08 -0700 (PDT)

> Security section might want to mention the possibility of deadlock if the server works by locking both of the mailboxes.

Not clear on what you're getting at here. If you're talking about someone
doing a completely crap implementation that allows a deadlock condition to
occur on completely legitimate operations, then that's an implementation
detail, not a security consideration. OTOH, if there's a series of commands
that can be sent to a carelessly done implementation that will cause a
deadlock, then yes, that absolutely does need to be in the security
considerations.

				Ned

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 25 Sep 2012 04:24:17 +0300

> deadlock, then yes, that absolutely does need to be in the security

MOVE is the only command in IMAP protocol that modifies two mailboxes at the same time. So compared to all other commands it could require special attention in some server implementations. Especially servers that try to make MOVE fully atomic without a storage/database that does row-level locking could easily deadlock (or detect it and fail)..

---

From: Ned Freed <[email-masked]>
Date: Mon, 24 Sep 2012 18:29:48 -0700 (PDT)

> MOVE is the only command in IMAP protocol that modifies two mailboxes at the
> same time. So compared to all other commands it could require special attention
> in some server implementations. Especially servers that try to make MOVE fully
> atomic without a storage/database that does row-level locking could easily
> deadlock (or detect it and fail)..

I'm still thinking that's a implementation consideration, not a security
consideration. An implementation that deadlocks is broken, not insecure.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 24 Sep 2012 21:22:20 +0100

Hi Barry,
I am personally happy with all your other suggestions (and Ned's 
corrections/extra changes). One small thing:

>     is well defined for all the cases where COPY is.

Maybe it is me, but this still gets me very confused. This doesn't 
actually tell me anything useful. How about saying that "moving a 
message to the currently selected mailbox is allowed when copying the 
message to the currently selected mailbox is also allowed". Or something 
like that.

---

From: Jan Kundrát <[email-masked]>
Date: Wed, 26 Sep 2012 09:18:51 +0200

> we're ready to go?  Or what?

I've tried to follow the discussion and don't have any objection to
various changes which have been proposed. I also think that it's time
for an updated draft -- and I'll be happy to read it thoroughly.

With kind regards,
Jan

-- 
Trojita, a fast e-mail client -- http://trojita.flaska.net/