Subject: [imapext] MOVEUID

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 3 Jul 2012 11:45:16 +0200

On 06/29/2012 11:36 PM, Jan Kundrát wrote:
> I recognize that the UID COPY & UID STORE & UID EXPUNGE has the very
> same problem as I'm presenting here, but I'd like to (re)propose an
> untagged variant of the COPYUID response code

I think you're right. It feels hackish, but I think you're right.

I'll add it to the draft and repost in a few days, unless anyone has a 
strong opinion on the matter?

I think what I'll add is a MOVEUID that's sent untagged, defined from 
first principles, does not say things like "if both UIDPLUS and MOVE", 
may be sent by the server at its discretion, and if sent should be sent 
before any EXPUNGEs.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 3 Jul 2012 14:24:16 +0300

On 3.7.2012, at 12.45, Arnt Gulbrandsen wrote:

> I'll add it to the draft and repost in a few days, unless anyone has a strong opinion on the matter?
> 
> I think what I'll add is a MOVEUID that's sent untagged, defined from first principles, does not say things like "if both UIDPLUS and MOVE", may be sent by the server at its discretion, and if sent should be sent before any EXPUNGEs.

Looks like there was no problem implementing this change to Dovecot. I don't know if other servers might have extra trouble with delaying sending EXPUNGE replies.

Probably shouldn't give server the choice of sending COPYUID instead of MOVEUID (I guess you didn't mean that but just to be sure).

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 3 Jul 2012 13:35:47 +0200

> Looks like there was no problem implementing this change to Dovecot. I don't know if other servers might have extra trouble with delaying sending EXPUNGE replies.

I would think not.

> Probably shouldn't give server the choice of sending COPYUID instead of MOVEUID (I guess you didn't mean that but just to be sure).

Haven't done the detailed work yet. My basic inclination is to suggest 
sending MOVEUID when practically possible, and to not send COPYUID if 
MOVEUID has been sent. But I'll read the UIDPLUS document before I write 
the text.

Arnt

---

From: Dan Karp <[email-masked]>
Date: Tue, 3 Jul 2012 07:55:16 -0700 (PDT)

> untagged variant of the COPYUID response code working like this one:
> 
> S: * OK [MOVEUID a 432432 1202:1205] Messages moved
> 
> "Client implementors shall keep in mind that the EXPUNGE or VANISHED
> responses precede delivery the COPYUID response code.  In case the
> clients want to make use of the returned information, they should
> postpone removing the affected messages from their cache until the
> tagged response is processed."

Why is MOVEUID necessary?  UIDPLUS-capable clients already handle
COPYUID for their UID COPY operations, and UID COPY allows for sending
untagged EXPUNGEs -- including those of the copied messages.  Isn't
UID MOVE just a specific case where the EXPUNGE responses are (much)
more likely?

- Dan

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 3 Jul 2012 17:02:29 +0200

> UID MOVE just a specific case where the EXPUNGE responses are (much)

It's also a case where UIDCOPY invariably arrives after the EXPUNGE.

In unix terms: You can't create a new hardlink if you've just removed 
the only known hardlink to that file.

Arnt

---

From: Dan Karp <[email-masked]>
Date: Tue, 3 Jul 2012 08:12:30 -0700 (PDT)

> > COPYUID for their UID COPY operations, and UID COPY allows for
> > sending untagged EXPUNGEs -- including those of the copied messages.
> > Isn't UID MOVE just a specific case where the EXPUNGE responses are
> > (much) more likely?
> 

Yes, but COPYUID already can arrive after an untagged EXPUNGE in UID
COPY.  Clients already need to handle this case.

> the only known hardlink to that file.

True.  This is the same problem that already exists for for EXPUNGE/
COPYUID.

- Dan

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 3 Jul 2012 17:29:58 +0200

> COPY.  Clients already need to handle this case.

It can, but it rarely happens with COPY, so all it does is add a few 
cache misses. With MOVE t happens every time, so the cache hit ratio 
drops to zero, and COPYUID is pointless.

It's possible to get around it. The client can make a new reference to 
each message when it issues the move command and hold the references in 
a command issuer object. When copyuid arrives, the client can create new 
cache references in the target mailbox based on the references held by 
the move command issuer rather than based on the references in the 
source mailbox cache.

That complicated explanation puts me off.

Arnt

---

From: Dan Karp <[email-masked]>
Date: Tue, 3 Jul 2012 09:43:17 -0700 (PDT)

> It's possible to get around it. The client can make a new reference
> to each message when it issues the move command and hold the references
> in a command issuer object. When copyuid arrives, the client can create
> new cache references in the target mailbox based on the references held
> by the move command issuer rather than based on the references in the
> source mailbox cache.

You've got a good point.  There are legitimate response ordering issues
with putting MOVEUID last.

It just kinda puts me off from an aesthetic perspective that COPYUID and
APPENDUID come on the tagged OK, while MOVEUID would come on an untagged
response mid-stream with the tag inlined in the response code.  It'd be
good to hew to the precedent set by UIDPLUS unless the burden on client
implementors is too great.

(Honestly, I'd even lean towards reusing COPYUID for MOVE responses.  But
that won't work if we need the tag in the response code.)

- Dan

---

From: Timo Sirainen <[email-masked]>
Date: Tue, 3 Jul 2012 23:42:09 +0300

> that won't work if we need the tag in the response code.)

The COPYUID/MOVEUID specifies both source and destination UIDs. There's no need to match them to a specific tagged command. Although there should then be a MUST that if either of them is sent as untagged, the MOVE command MUST reply with OK.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 4 Jul 2012 09:25:33 +0200

How about this text?

"COPYUID is normally sent as part of the tagged OK. Since some clients 
have problems using COPYUID after EXPUNGE (they remove the messages from 
their cache at EXPUNGE, and so cannot update the cache at COPYUID), the 
server SHOULD send COPYUID in an untagged OK just before it sends the 
the first EXPUNGE."

Well, it needs wordsmithing. I can see that.

Arnt

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 05 Jul 2012 08:15:20 +0000

Hi

we're currently already sending COPYUID on the tagged response to the 
UID MOVE, and have done this for about a year.  I don't know if clients 
rely on the COPYUID response.  I think there is a bug report against TB 
for not using it (it rediscovers).

26975 29670 5/07/2012 18:10:12.551 121.74.33.191 Activation IMAP4 
Server 888 19736 Debug 0 C=>: 13 uid move 21030 "Trash"
26976 29671 5/07/2012 18:10:12.566 121.74.33.191 Activation IMAP4 
Server 888 19736 Debug 0 <=S: * 4537 EXPUNGE
26978 29673 5/07/2012 18:10:12.769 121.74.33.191 Activation IMAP4 
Server 888 19736 Debug 0 <=S: 13 OK [COPYUID 1301566381 21030 20068] 
command completed

So you're saying this should change to MOVEUID with tag so that it can 
come in an untagged response prior to EXPUNGE responses

just to make life a little easier for client devs who use a particular 
method?

What about server devs?  It's much easier to re-use the existing copy 
etc processing code, and therefore leave the COPYUID as is.

Why can't the client devs just know they did a move so defer EXPUNGE 
processing if they can't just cope with it?

Also, if this is going to be called MOVE, then how do we as a server 
deal with clients that use the old MOVE vs the new one?

We'd need to call this new method NEWMOVE or something.

Adrien

------ Original Message ------
From: "Arnt Gulbrandsen" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 4/07/2012 7:25:33 p.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>have problems using COPYUID after EXPUNGE (they remove the messages 
>from their cache at EXPUNGE, and so cannot update the cache at 
>COPYUID), the server SHOULD send COPYUID in an untagged OK just before 
>it sends the the first EXPUNGE." 
>
>_______________________________________________ 
>imapext mailing list 
>[email-masked] 
>https://www.ietf.org/mailman/listinfo/imapext

---

From: Dan Karp <[email-masked]>
Date: Thu, 5 Jul 2012 10:03:01 -0700 (PDT)

> We'd need to call this new method NEWMOVE or something.

You really shouldn't have implemented a command called "MOVE" without
having gone through the standardization process.  You needed to call 
it "XMOVE" or the like.  As RFC 3501 section 6.5.1 says:

      Any command prefixed with an X is an experimental command.
      Commands which are not part of this specification, a standard or
      standards-track revision of this specification, or an
      IESG-approved experimental protocol, MUST use the X prefix.

- Dan

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 00:19:28 +0000

that's a fair comment, however you saw the log, it came from 
thunderbird.  You could say the same thing to them.

But that's a big problem, since there are a lot of thunderbird installs 
out there.  

Basically anything you do to MOVE which breaks compatibility will 
potentially have some impact on those using TB (e.g. if their server 
advertises MOVE - new or old).

We just saw the list of IMAP extensions supported by TB and went from 
there.  I don't know if other server vendors did the same or not.

https://wiki.mozilla.org/MailNews:Supported_IMAP_extensions

which points to 

http://tools.ietf.org/html/draft-krecicki-imap-move

So I don't really know how big a problem it is, but I do know that at 
least TB uses it as currently defined in Witold's draft.

Personally I don't see what this previous draft is really missing, and 
why it needs to be re-written.

It's possible changing the tagged response with COPYUID to an untagged 
MOVEUID will have no impact.  But I don't think we know that.

Adrien

------ Original Message ------
From: "Dan Karp" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Arnt Gulbrandsen" <[email-masked]>;"[email-masked]" 
<[email-masked]>
Sent: 6/07/2012 5:03:01 a.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 04:42:26 +0000

hi all

I've been thinking a bit more about this.

If we are to allow the potential for other connected clients to delete 
messages from a mailbox whilst one client is doing a move on multiple 
messages, then it's possible that some messages may not be movable at 
the time the server tries to move them.

The problem then is that in either the COPYUID or MOVEUID proposal, the 
client can't tell which source messages mapped to the UIDs specified, 
since COPYUID/MOVEUID only provides a new UID sequence for the messages 
that were moved, and if the size of this sequence is not the same as 
the size of the original requested sequence, then there's no way to 
tell which one didn't get moved.

ALSO, since it will be a real pain in the neck for a server to defer 
sending EXPUNGE to the mover until after it collected all the UIDs for 
the target mailbox, and since this is new, why don't we do something 
entirely different.

E.g. 

t UID MOVE 1:2 "target"
* MOVED 1 "target" <uidvalidity> <uid>
* MOVED 2 <uid>
t OK move completed, 2 messages moved

to other connected clients we'd just send the EXPUNGE responses.
This way it's completely explicit what was moved and what wasn't.  

Note:

1. to distinguish new MOVE (this) from old style, client must ENABLE 
MOVE
2. only send target and uidvalidity on first untagged MOVED response 
(since it is the same for all messages in the move).
3. clients must not pipeline move commands so that there's no need to 
send the tag in each untagged response.
4. If a client indicates it supports move (ENABLE MOVE), then can 
indicate MOVED responses instead of EXPUNGED responses.

Adrien

------ Original Message ------
From: "Adrien W. de Croy" <[email-masked]>
To: "Dan Karp" <[email-masked]>
Cc: "Arnt Gulbrandsen" <[email-masked]>;"[email-masked]" 
<[email-masked]>
Sent: 6/07/2012 12:19:28 p.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>But that's a big problem, since there are a lot of thunderbird 
>installs out there.

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 08:16:25 +0300

On 6.7.2012, at 7.42, Adrien W. de Croy wrote:

> The problem then is that in either the COPYUID or MOVEUID proposal, the client can't tell which source messages mapped to the UIDs specified, since COPYUID/MOVEUID only provides a new UID sequence for the messages that were moved, and if the size of this sequence is not the same as the size of the original requested sequence, then there's no way to tell which one didn't get moved.

MOVE is specified as COPY+STORE+EXPUNGE. COPY already requires that the server MUST abort in such situation:

      If the COPY command is unsuccessful for any reason, server
      implementations MUST restore the destination mailbox to its state
      before the COPY attempt.

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 08:19:16 +0300

On 6.7.2012, at 3.19, Adrien W. de Croy wrote:

> Basically anything you do to MOVE which breaks compatibility will potentially have some impact on those using TB (e.g. if their server advertises MOVE - new or old).

Didn't you just say that TB doesn't use COPYUID with MOVE? If we change COPYUID behavior how is that making anything worse?

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 05:38:11 +0000

as at April 2011 it didn't, I don't know about now

Adrien

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Dan Karp" <[email-masked]>;"Arnt Gulbrandsen" 
<[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 6/07/2012 5:19:16 p.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 05:45:32 +0000

Well...

how useful is this?

If I select a bunch of messages and drag them onto another folder, I 
want it to happen.  

If one message won't move for some reason, I want the rest to move.  
Otherwise you put a big burden back on the user to sort out issues that 
the server is perfectly capable of handling in a useful fashion.

I also think that only sending EXPUNGED to other connected clients is 
lame.  Why not just send them everything they need to know so they can 
know what happened (as long as they indicated support for it).  
Otherwise all they know is some mail disappeared.  They don't know it 
was moved.  To them the mail ils gone, not moved.  That's why I suggest 
making what we tell other clients as useful as possible.

Whether or not a server mandates that all moves must succeed or none 
will, should be an implementation decision.  the protocol should not 
mandate it either way IMO.  A server author that wants to make life 
easier for their mail users should not be prohibited from doing so just 
because it was deemed to hard to do something useful in this case by 
the spec writers?

E.g. you select 1000 messages and drag them somewhere.  1 fails.  Do 
you really want to have to figure out which one by trial and error 
(move smaller and smaller selections until they succeed)?  If the other 
999 were moved, it would become blindingly obvious which one, then the 
user can do something about it.

Otherwise we set up an awful user experience.

Adrien

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Arnt Gulbrandsen" <[email-masked]>;"[email-masked]" 
<[email-masked]>;"Dan Karp" <[email-masked]>
Sent: 6/07/2012 5:16:25 p.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 09:41:28 +0200

> as at April 2011 it didn't, I don't know about now

Let me guess: It processes the EXPUNGEs as they arrive, and by the time 
the COPYUID arrives there's nothing left to process.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 10:00:39 +0200

> for not using it (it rediscovers).

I'm not well acquainted with the tb bug tracker. Could you look up that 
but and see if has been fixed?

> processing if they can't just cope with it?

The tb developers are well placed to answer that. Jan probably can 
comment, too.

Arnt

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 10:01:24 +0000

------ Original Message ------
From: "Alexey Melnikov" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Timo Sirainen" <[email-masked]>;"Arnt Gulbrandsen" 
<[email-masked]>;"Dan Karp" 
<[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 6/07/2012 8:55:59 p.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>Not a protocol problem: MOVE one message at a time if you care about this, or MOVE many and handle error cases gracefully. This probably deserves some text in the document.
>
>On 6 Jul 2012, at 06:45, "Adrien W. de Croy" <[email-masked]> wrote:
>
>
>>
>>
>>Well...
>>
>>how useful is this?
>>
>>If I select a bunch of messages and drag them onto another folder, I want it to happen.
>>If one message won't move for some reason, I want the rest to move.  Otherwise you put a big burden back on the user to sort out issues that the server is perfectly capable of handling in a useful fashion.
>>
>>I also think that only sending EXPUNGED to other connected clients is lame.  Why not just send them everything they need to know so they can know what happened (as long as they indicated support for it).  Otherwise all they know is some mail disappeared.  They don't know it was moved.  To them the mail ils gone, not moved.  That's why I suggest making what we tell other clients as useful as possible.
>>
>>Whether or not a server mandates that all moves must succeed or none will, should be an implementation decision.  the protocol should not mandate it either way IMO.  A server author that wants to make life easier for their mail users should not be prohibited from doing so just because it was deemed to hard to do something useful in this case by the spec writers?
>>
>
>
>Disagree. We need to define specific behaviour (don't particularly care which).
>

at the end of the day, whether the client requests a move of single 
messages at a time, or multiple, the server cannot say.

If the client wants the server to just move everything and report 
whatever it couldn't, then that's a reasonable client wish.
If the client wants to take on that burden itself, that's also 
reasonable.

We don't need to decide beforehand (in the spec) one way or another 
whether any single move in a range of messages should fail the whole 
bunch.

Why not make it explicit, the client indicates error handling, e.g.

t UID move 1:1000000 "target" ON_ERROR_REVERT

The poor server will start moving files.  If it gets to 999999 and the 
last one fails, does it really really have to move everything back?  
that's 99.999% likely to be a complete waste of server and client 
effort and operator time.  the operator already clearly indicated a 
desire for the messages to be moved.  Failure of 1, is extremely 
unlikely IME to alter that desire.

As long as errors can be properly reported in a way that allows a 
client to recover and know what happened, then taking the optimistic 
approach by default should be ok?

This is equally a problem with COPY.  If the server cannot properly 
report errors (e.g. which messages failed) then sure, it only has 1 
option, and that's to roll back.  But this is new.  We can make it so 
that it reports errors properly.

Adrien
>>
>>
>>E.g. you select 1000 messages and drag them somewhere.  1 fails.  Do you really want to have to figure out which one by trial and error (move smaller and smaller selections until they succeed)?  If the other 999 were moved, it would become blindingly obvious which one, then the user can do something about it.
>>
>>Cc: "Arnt Gulbrandsen" <[email-masked]>;"[email-masked]" <[email-masked]>;"Dan Karp" <[email-masked]>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 12:12:21 +0200

On 07/05/2012 07:03 PM, Dan Karp wrote:
> You needed to call
> it "XMOVE" or the like

To join the other four extensions by that name? Formally it's correct, 
but I dare say that in this case following the rules does not confer 
good karma.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 14:53:16 +0200

On 07/06/2012 10:55 AM, Alexey Melnikov wrote:
> This probably deserves some text in the document.

Will add.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 14:54:59 +0200

> t UID move 1:1000000 "target" ON_ERROR_REVERT

I've seen almost ten different move proposals and x- implementations. 
None of them contained this feature, which makes me think that it 
probably would not be used very often.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 06 Jul 2012 09:45:09 -0400

Hi Adrien,

--On July 6, 2012 4:42:26 AM +0000 "Adrien W. de Croy" <[email-masked]> 
wrote:

> that were moved, and if the size of this sequence is not the same as the
> size of the original requested sequence, then there's no way to tell
> which one didn't get moved.

That is wrong, from UIDPLUS (RFC2359):

   The COPYUID response code contains as an argument the UIDVALIDITY of
   the appended-to mailbox, a message set containing the UIDs of the
   messages copied to the destination mailbox, in the order they were
   copied, and a message containing the UIDs assigned to the copied
   messages, in the order they were assigned.

   resp_code_copy  ::= "COPYUID" SPACE nz_number SPACE set SPACE set

So the server gives the client an explicit mapping of "old UID" to "new 
UID". So if some messages were not moved they would simply not appear in 
the "old UID" set and there would be no corresponding entry in the "new 
UID" set.

There is no problem here!

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 06 Jul 2012 09:58:40 -0400

Hi Adrien,

> t OK move completed, 2 messages moved

OK, so there is one big benefit to having MOVE responses - the server could 
send those unsolicited to other clients that have a mailbox selected, where 
another client is doing a MOVE. That way the other clients also have the 
option of moving the messages in their local cache, rather than doing a 
local cache expunge, followed sometime later by a fetch of the new messages 
in the target mailbox. For people accessing their IMAP store from multiple 
devices, that would be a big benefit.

In order for that to work and be backwards compatible the server would have 
to send a MOVE followed by an EXPUNGE for the moved messages. I think it 
would make sense to design the MOVE (or MOVEUID) response to look like 
COPYUID in that a single response should be able to identify moves of 
multiple messages (to cut down on the total number of responses).

-- 
Cyrus Daboo

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 14:19:22 +0000

Hi
------ Original Message ------
From: "Cyrus Daboo" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>;"Dan Karp" <[email-masked]>
Cc: "Arnt Gulbrandsen" <[email-masked]>;"[email-masked]" 
<[email-masked]>
Sent: 7/07/2012 1:58:40 a.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>Hi Adrien, 
>
>--On July 6, 2012 4:42:26 AM +0000 "Adrien W. de Croy" 
><[email-masked]> wrote: 
>
>>t UID MOVE 1:2 "target" 
>OK, so there is one big benefit to having MOVE responses - the server 
>could send those unsolicited to other clients that have a mailbox 
>selected, where another client is doing a MOVE. That way the other 
>clients also have the option of moving the messages in their local 
>cache, rather than doing a local cache expunge, followed sometime 
>later by a fetch of the new messages in the target mailbox. For people 
>accessing their IMAP store from multiple devices, that would be a big 
>benefit. 
yes, this happens a lot.

Most moves are from inbox (junk or filter processing).  This is also 
the mailbox that most often clients have connections to for a long 
time. So I think if a client supported unsolicited MOVED responses, 
there could be some good benefit.

If you have a lot of mailboxes, and one client moves a message, it can 
be effectively lost to the other clients.  You basically need to go to 
the client that did the move to see where it went (unless the other 
clients regularly poll ALL mailboxes for changes).

>
>
>In order for that to work and be backwards compatible the server would 
>have to send a MOVE followed by an EXPUNGE for the moved messages. I 
>think it would make sense to design the MOVE (or MOVEUID) response to 
>look like COPYUID in that a single response should be able to identify 
>moves of multiple messages (to cut down on the total number of 
>responses). 
OK, so it would need to be untagged so it could be unsolicited to 
others.

e.g. 

     t UID MOVE 1,3:10,46,48:900 "target"
     * MOVED UID 1,3:10,48:898,900 "target" <uidvalidity> 2:863
     t MOVE 861 messages moved, 2 messages vanished

no expunges back to the requestor are required, all the information is 
there.  And all the information is in the untagged response for any 
other observer to perform the same move in their cache.

Adrien

p.s. point taken on my misinterpretation of the UIDPLUS COPYUID 
response.  I shouldn't rely on my memory it seems!

>-- Cyrus Daboo

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 17:25:40 +0300

On 6.7.2012, at 16.58, Cyrus Daboo wrote:

> OK, so there is one big benefit to having MOVE responses - the server could send those unsolicited to other clients that have a mailbox selected, where another client is doing a MOVE. That way the other clients also have the option of moving the messages in their local cache, rather than doing a local cache expunge, followed sometime later by a fetch of the new messages in the target mailbox. For people accessing their IMAP store from multiple devices, that would be a big benefit.

Way, way too much work to be worth it. I very much wouldn't like to implement that.

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 14:31:48 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Cyrus Daboo" <[email-masked]>
Cc: "Adrien W. de Croy" <[email-masked]>;"Arnt Gulbrandsen" 
<[email-masked]>;"[email-masked]" <[email-masked]>;"Dan 
Karp" <[email-masked]>
Sent: 7/07/2012 2:25:40 a.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>Way, way too much work to be worth it. I very much wouldn't like to implement that.

really?

Would be hardly any work for us.  Anyway I prefer Cyrus' proposal of a 
single agreggated MOVED response.  Which in fact is no more or less 
work than current proposal (except to allow unsolicited moved responses 
to others who indicate support.  But that's trivially simple with 
ENABLE).

As for benefit, I guess you don't have many mail folders or use many 
clients then.  It would save me a bunch of work.  But I'm not a typical 
user.

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 06 Jul 2012 10:39:49 -0400

Hi Timo,

--On July 6, 2012 5:25:40 PM +0300 Timo Sirainen <[email-masked]> wrote:

>> cache, rather than doing a local cache expunge, followed sometime later
>> by a fetch of the new messages in the target mailbox. For people

I figured that would be the reaction from server developers :-) I would 
like to hear whether client implementors think unsolicited MOVE/MOVEUID 
would be beneficial - if not then we can punt on it entirely. However, the 
more we do to reduce the need for clients to re-download messages, the 
better.

-- 
Cyrus Daboo

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 17:42:01 +0300

On 6.7.2012, at 17.31, Adrien de Croy wrote:

> Would be hardly any work for us.

Sounds like you have a single process multithreaded server. Dovecot can run in a cluster of multiple servers, each server possibly handling the same user. All communication goes through the filesystem. What this would need is that for each MOVE command it would have to write about it to mailbox's log file (or alternatively a new method of communicating between servers - simply for this feature). Possible, but I think it would be mostly a waste of effort.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 16:43:48 +0200

Timo answers Cyrus:
>>> t UID MOVE 1:2 "target"
>>> * MOVED 1 "target"<uidvalidity>  <uid>
>>> * MOVED 2<uid>

Easy in some architectures, but I don't think it's a win no matter how 
easy it is. How often to people read messages one device, then move 
messages from other mailboxes on another device, then change back to the 
first device and read them again?

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 17:45:20 +0300

On 6.7.2012, at 17.39, Cyrus Daboo wrote:

> However, the more we do to reduce the need for clients to re-download messages, the better.

This could be done in a more generic way by adding a GUID extension to IMAP protocol. APPEND returns you GUIDs of saved mails. When you see new mails you could FETCH GUID for them and if the GUID is already cached you wouldn't have to download it.

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 14:48:26 +0000

------ Original Message ------
From: "Cyrus Daboo" <[email-masked]>
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>>I figured that would be the reaction from server developers :-) I 
>>would like to hear whether client implementors think unsolicited 
>>MOVE/MOVEUID would be beneficial - if not then we can punt on it 
>>entirely. However, the more we do to reduce the need for clients to 
>>re-download messages, the better. 
>>
Actually it's a worse problem than that.

I have something like 600 folders/mailboxes in my login.  I have filter 
rules to sort the mail incoming into these folders.

If something moves a message to another folder, basically it can be 
months before other clients would by chance open that folder and see 
it.  Unless I go to the client that did the move, I generally don't see 
it.  To all intents and purposes it may as well have been deleted.

This means I close my work email when I leave, and also means I have to 
constantly keep maintaining rules across multiple email clients.  This 
is a pain in the neck.  I know server-side SEIVE is supposed to solve 
that, but that actually just means it would effectively ALWAYS be 
something else doing the move, and none of the clients would know where 
the mail went.

So telling everyone enough information about the move seems like a 
no-brainer to me.

And it would also reduce bandwidth consumption, since bystanders 
wouldn't get a zillion EXPUNGE untagged responses any more (if they 
supported the untagged moved one).  For mobile clients especially it 
would be good.

But maybe some server architectures make this more difficult than 
others.
Adrien

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 14:51:36 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>Sounds like you have a single process multithreaded server.
>

you got me there.

Haven't gone to clustering yet.

Adrien.
>Dovecot can run in a cluster of multiple servers, each server possibly handling the same user. All communication goes through the filesystem. What this would need is that for each MOVE command it would have to write about it to mailbox's log file (or alternatively a new method of communicating between servers - simply for this feature). Possible, but I think it would be mostly a waste of effort.
>

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 14:53:51 +0000

------ Original Message ------
From: "Arnt Gulbrandsen" <[email-masked]>
bject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>>>OK, so there is one big benefit to having MOVE responses - the 
>>>server could send those unsolicited to other clients that have a 
>>>mailbox selected, where another client is doing a MOVE. That way the 
>>>other clients also have the option of moving the messages in their 
>>>local cache, rather than doing a local cache expunge, followed 
>>>sometime later by a fetch of the new messages in the target mailbox. 
>>>For people accessing their IMAP store from multiple devices, that 
>>>would be a big benefit. 
>>
>messages from other mailboxes on another device, then change back to 
>the first device and read them again? 

for me all the time.

I have my office desktop, home desktop and iPhone.  All hitting the 
same account.

With the rise of "smart" phones, more and more people have multiple 
agents hitting the same mailbox.

So it would actually be useful for me.

Regards

Adrien

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 06 Jul 2012 10:55:45 -0400

Hi Adrien,

--On July 6, 2012 2:48:26 PM +0000 Adrien de Croy <[email-masked]> wrote:

> months before other clients would by chance open that folder and see it.
> Unless I go to the client that did the move, I generally don't see it.
> To all intents and purposes it may as well have been deleted.
>
> constantly keep maintaining rules across multiple email clients.  This is
> a pain in the neck.  I know server-side SEIVE is supposed to solve that,
> but that actually just means it would effectively ALWAYS be something
> else doing the move, and none of the clients would know where the mail
> went.
>

Well one could argue that clients and servers should support some existing 
extensions to help with that. Most notably the NOTIFY extension (RFC 5465). 
Or perhaps status-in-list (RFC 5819). Both of those would allow a client to 
monitor multiple (or all) mailboxes for changes.

Perhaps rather than having unsolicited MOVE/MOVEUID, we define a new NOTIFY 
event type for "MessageMove" and then make use of that. Is NOTIFY widely 
implemented?

-- 
Cyrus Daboo

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 14:57:49 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Cyrus Daboo" <[email-masked]>
Cc: "Adrien W. de Croy" <[email-masked]>;"Arnt Gulbrandsen" 
<[email-masked]>;"[email-masked]" <[email-masked]>;"Dan 
Karp" <[email-masked]>
Sent: 7/07/2012 2:45:20 a.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>This could be done in a more generic way by adding a GUID extension to IMAP protocol. APPEND returns you GUIDs of saved mails. When you see new mails you could FETCH GUID for them and if the GUID is…

GUID would be great for detecting moves after the fact, but unless the 
client looks at the target mailbox at some stage, they won't notice the 
new messages there.

what about if the MOVE extension were split in 2.

1 for single client mode, and another extension for the unsolicited 
responses to others.

E.g advertise MOVE and MOVEPLUS.  I know it's major CAPABILITY 
bloat.... and hate it for that... but it solves the problem for servers 
whose architecture doesn't lend itself easily to notifying other 
clients of moves.  They then can do MOVE.

Or maybe the notification part of it should be handled by the 
notification RFC.

Adrien

---

From: Dan Karp <[email-masked]>
Date: Fri, 6 Jul 2012 08:05:38 -0700 (PDT)

> Perhaps rather than having unsolicited MOVE/MOVEUID, we define a new
> NOTIFY event type for "MessageMove" and then make use of that. Is
> NOTIFY widely implemented?

Unfortunately, I don't believe it is.  Neither by clients nor servers,
though there's chickens and eggs there.

- Dan

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 17:08:12 +0200

On 07/06/2012 04:53 PM, Adrien de Croy wrote:
> for me all the time.
>
> I have my office desktop, home desktop and iPhone.  All hitting the same
> account.

AOL, but once I've read a message on one, I hardly ever reread on the 
others.

So the others would update their caches devoutly, but uselessly.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 18:08:35 +0300

On 6.7.2012, at 17.55, Cyrus Daboo wrote:

> Perhaps rather than having unsolicited MOVE/MOVEUID, we define a new NOTIFY event type for "MessageMove" and then make use of that.

There is no MessageCopy either.

> Is NOTIFY widely implemented?

No. I've so far scanned 34% of public IPs' IMAP ports and zero servers have advertised NOTIFY. Although I've almost finished writing it for Dovecot. http://openemailsurvey.org/ if anyone's interested :)

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 15:08:46 +0000

the filtering happens before it's read.

now, if the email clients could filter after reading/replying then 
sure... 

------ Original Message ------
From: "Arnt Gulbrandsen" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 7/07/2012 3:08:12 a.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>>same

---

From: Dan Karp <[email-masked]>
Date: Fri, 6 Jul 2012 08:13:03 -0700 (PDT)

> > The problem then is that in either the COPYUID or MOVEUID proposal,
> > the client can't tell which source messages mapped to the UIDs
> > specified, since COPYUID/MOVEUID only provides a new UID sequence
> > for the messages that were moved, and if the size of this sequence
> > is not the same as the size of the original requested sequence,
> > then there's no way to tell which one didn't get moved.

Actually, COPYUID contains both the source and target UIDs, in order.

      Followed by the UIDVALIDITY of the destination mailbox, a UID set
      containing the UIDs of the message(s) in the source mailbox that
      were copied to the destination mailbox and containing the UIDs
      assigned to the copied message(s) in the destination mailbox,
      indicates that the message(s) have been copied to the destination
      mailbox with the stated UID(s).

      The source UID set is in the order the message(s) were copied; the
      destination UID set corresponds to the source UID set and is in
      the same order.  Neither of the UID sets may contain extraneous
      UIDs or the symbol "*".

>       implementations MUST restore the destination mailbox to its
>       state before the COPY attempt.

The MOVE extension does not give this guarantee.

   First, each message SHOULD either be moved or unaffected. The server
   SHOULD NOT leave a message in neither or both mailboxes afterwards
   (even if the server returns a tagged NO response).

I asked about whether you could have a mix of moved and non-moved
messages, and Bron answered:

| > Can there be a situation where some messages in the set are moved and
| > others are unaffected?
| 
| I suspect it's going to be implementation dependent.  Worst cases are
| always things like crashes and filesystem corruption, where all bets
| are a bit off...

- Dan

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 17:16:20 +0200

So it's not really about client cache update at all, it's about the 
particular way in which you're doing delivery. Would it be fair to say 
that you're really after a server-push equivalent of STATUS (UNSEEN)?

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 18:19:49 +0300

>   (even if the server returns a tagged NO response).

I think this text was about what happens if the (atomic) COPY part succeeds but EXPUNGE part fails. But I guess if servers wanted to implement MOVE using rename()-like functions the problem would be different. So I guess it should be explicitly mentioned also.

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 15:24:00 +0000

Hi
------ Original Message ------
From: "Arnt Gulbrandsen" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 7/07/2012 3:16:20 a.m.
Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt
>that you're really after a server-push equivalent of STATUS (UNSEEN)? 
I'm not sure.

Basically if you have multiple clients sitting on the INBOX, when a 
message is delivered to it (e.g. incoming from SMTP) the race is on.

All the clients will get the notification about the new message, and 
one or more will try and "filter" it.

So it's possible that it could be a cache update for the clients who 
did learn of its existence but didn't move it.

But server push of STATUS could also be very interesting.  That 
probably is covered in NOTIFY.

If Dovecot is going to roll out NOTIFY, and some clients pick it up, 
we'd definitely look at rolling it in as well.  The more clients can be 
told about things rather than polling the better, and if they can be 
told about everything over a single connection, that greatly reduces 
the number of average connections a client needs to keep open to the 
server.

I don't know enough about NOTIFY though.

Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 18:26:06 +0300

> I think this text was about what happens if the (atomic) COPY part succeeds but EXPUNGE part fails. But I guess if servers wanted to implement MOVE using rename()-like functions the problem would b…

Although really: It's going to be extremely rare for the MOVE to partially fail. There's no good reason for that normally. Other than filesystem corruption, running out of disk space, software bugs and such, the only reason I can think of MOVE to partially fail in normally operating systems is if it's implemented as filesystem rename(), filesystem quotas are used and the destination directory needs to grow and that would bring the user over the quota.

So I don't think we need to worry about partial MOVE case much. Specify that it SHOULD NOT happen, and that's about it.

---

From: "Adrien de Croy" <[email-masked]>
Date: Fri, 06 Jul 2012 15:28:28 +0000

------ Original Message ------
>Although really: It's going to be extremely rare for the MOVE to partially fail. There's no good reason for that normally. Other than filesystem corruption, running out of disk space, software bugs…

I think it's more likely that move would fail because some other agent 
expunges something in the meantime.

Regards

Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 6 Jul 2012 18:48:33 +0300

On 6.7.2012, at 18.28, Adrien de Croy wrote:

> I think it's more likely that move would fail because some other agent expunges something in the meantime.

Oh, right, I keep forgetting that since I implemented MOVE as copy+expunge, and if copy is missing a message it reverts everything.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 6 Jul 2012 18:24:01 +0200

On 07/06/2012 04:55 PM, Cyrus Daboo wrote:
> NOTIFY event type for "MessageMove" and then make use of that.

Perhaps, although I lean towards Timo's suggestion of a GUID.

> widely implemented?

No, not at all.

http://rant.gulbrandsen.priv.no/good-bad-rfc

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 6 Jul 2012 09:55:59 +0100

---

From: Bron Gondwana <[email-masked]>
Date: Sat, 28 Jul 2012 14:19:29 +0200

On Thu, Jun 28, 2012, at 11:15 PM, [email-masked] wrote:
> A new Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the IMAP MOVE extension Working Group of the IETF.
> 
>     Title         : The IMAP Move Extension
>     Author(s)     : A. Gulbrandsen
>     Filename      : draft-ietf-imapmove-command
>     Pages         : 7 
>     Date          : June 28, 2012 
>     
> The MOVE extension provides a new command, UID MOVE, which moves one
>    or more messages from the selected mailbox to a named mailbox.

Here's an interesting one I just ran into in production.  A bug in our web client code caused it to try to MOVE a message from "Junk Mail" to "Junk Mail".  This caused an assertion failure when it tried to open the already open mailbox, which
is a Cyrus IMAPd bug, and I'll fix it.

I'm wondering - is it worth explicitly mentioning this in the RFC as "returns BAD" - it doesn't make sense to move messages to the same folder as you're currently selecting.

Bron.

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 28 Jul 2012 15:32:16 +0300

> I'm wondering - is it worth explicitly mentioning this in the RFC as "returns BAD" - it doesn't make sense to move messages to the same folder as you're currently selecting.

It doesn't make much sense, but I don't know if it should be disallowed either. MOVE is defined as combination of COPY+STORE+EXPUNGE, all of which allow source and destination being the same. One potential use case could be to give messages new UIDs, although I don't know how useful that is really. :)

Probably worth mentioning anyway that server implementers should check it.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 28 Jul 2012 15:46:37 +0200

It isn't bad: the client may not know the mailbox is the same. In your code, inbox has two names, right?

But i will add a note.

---

From: Barry Leiba <[email-masked]>
Date: Sat, 28 Jul 2012 12:57:15 -0400

>
> It isn't bad: the client may not know the mailbox is the same.

It isn't "BAD", perhaps.  I continue to think that it was an unnecessary
and perhaps confusing decision to make a distinction between BAD and NO.
 Does any client actually behave differently for those two responses types?

I suggest that  the text say that servers MAY respond NO in this case, and
that they SHOULD include a suitable response code to indicate the
situation.  Perhaps ALREADYEXISTS makes sense, or you can use CANNOT... or
else create a new code and register it.

Barry, as participant

---

From: Bron Gondwana <[email-masked]>
Date: Sat, 28 Jul 2012 21:52:00 +0200

> It isn't bad: the client may not know the mailbox is the same. In your code, inbox has two names, right?

Actually, no.  It's the same name.

Though: more investigation shows it wasn't our interface:

2012-07-27T23:16:16.397496-04:00 imap15 sloti15d3p1/imap[8927]: login: frontend1.nyi.mail.srv.osa [10.202.2.160] *redacted* plaintext User logged in SESSIONID=<sloti15d3p1-8927-1343445376-1>
2012-07-27T23:16:16.912373-04:00 imap15 sloti15d3p1/imap[8927]: client id: "name" "SeaMonkey" "version" "2.11"
[...]

> But i will add a note.

And here's part of the stack trace (we dump core on assertion errors
in FastMail production to help us track down bugs faster):

#9  0x000000000041f15d in cmd_copy (tag=0x18f93b0 "18", 
    sequence=0x18f9bd0 "14350,14353,14355,14357:14359", 
    name=0x18fa410 "inbox.Junk Mail", usinguid=1, ismove=1)

Unfortunately, I don't have a record of the original name passed to
the SELECT command earlier, because we don't run telemetry logging
by default on users who haven't reported any issues.

So what do you suggest? Just return:

TAG NO can not move messages into same folder

?

Bron.
-- 
  Bron Gondwana
  [email-masked]

---

From: Bron Gondwana <[email-masked]>
Date: Sat, 28 Jul 2012 21:53:14 +0200

> Probably worth mentioning anyway that server implementers should check it.

Hmm... yeah, that's a point actually.

I guess I just have to detect that special case and deal with it correctly :)

Bron.
-- 
  Bron Gondwana
  [email-masked]

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 28 Jul 2012 21:53:54 +0200

>   Does any client actually behave differently for those two responses types?

No idea.

(A digression. IMO, BAD is misnamed, but otherwise okay. It should have 
been broken down into the two cases NO [SYNTAXERROR] and NO [CANNOT].)

> I suggest that  the text say that servers MAY respond NO in this case,
> and that they SHOULD include a suitable response code to indicate the
> situation.  Perhaps ALREADYEXISTS makes sense, or you can use CANNOT...
> or else create a new code and register it.
>
> Barry, as participant

I lean towards adding this sentence: "If the target mailbox is actually 
the same as the source (perhaps by a different name), the server MAY 
treat the operation as a no-op and simply return OK without doing any 
work, or it MAY answer NO [CANNOT]."

Is it worth defining a new response code, NOOP? I think not.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 28 Jul 2012 23:07:31 +0300

On 28.7.2012, at 22.53, Arnt Gulbrandsen wrote:

> I lean towards adding this sentence: "If the target mailbox is actually the same as the source (perhaps by a different name), the server MAY treat the operation as a no-op and simply return OK without doing any work, or it MAY answer NO [CANNOT]."

I don't think it should return OK if it didn't change the UID.

> Is it worth defining a new response code, NOOP? I think not.

No.

---

From: Barry Leiba <[email-masked]>
Date: Sat, 28 Jul 2012 13:53:30 -0700

> been broken down into the two cases NO [SYNTAXERROR] and NO [CANNOT].)

I believe that CANNOT is not a subcase of BAD.  BAD, as it's intended,
means "the client should have known that it couldn't do that."  In other
words, the client is misusing the protocol.  CANNOT can mean other things.

> the same as the source (perhaps by a different name), the server MAY treat
> the operation as a no-op and simply return OK without doing any work, or it
> MAY answer NO [CANNOT]."
>

I'm with Timo: I don't like the first option.  Either you do *something* --
actually do the move in some way, or recognize that it's silly and just
give it a new UID (and you can put text in to that effect) -- or you
respond NO [CANNOT].  How about this text?:

"If the target mailbox is actually the same as the source (perhaps by a
different name), the server MAY optimize for this situation.  If it does
so, it MUST either ensure that the message gets a new UID (and respond OK),
or respond NO [CANNOT]."

Barry, as participant

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 28 Jul 2012 23:02:24 +0200

God, a digression thread. But I'll digress with the best of you!

> words, the client is misusing the protocol.  CANNOT can mean other things.

3501 says (on almost every page):

     BAD - command unknown or arguments invalid

The arguments invalid bit often is CANNOT land. Legal by the protocol, 
sometimes the client could know, but more often the user knew.

     C: a create ""

Did I tell you about my time as listmaster? I had the thingy set to 
forward all unparsable -request mail to me, and one day I received the 
following two-line missive:

    subscribe
    crash

I subscribed the originator and sent a polite message saying so, and I 
also said "command not supported". Then he told me that quite a few 
mailing list processors did indeed carry out his complete instructions.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Sun, 29 Jul 2012 00:09:59 +0300

On 29.7.2012, at 0.02, Arnt Gulbrandsen wrote:

> The arguments invalid bit often is CANNOT land. Legal by the protocol, sometimes the client could know, but more often the user knew.
> 
>    C: a create ""

Is your point that it would be legal to reply BAD for this command? I would disagree.

---

From: Barry Leiba <[email-masked]>
Date: Sat, 28 Jul 2012 14:25:59 -0700

> sometimes the client could know, but more often the user knew.

I'm not going to argue with you here, because it doesn't matter.  I urge
everyone else not to argue about this here either.

Barry

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 29 Jul 2012 09:02:59 +0200

> Is your point that it would be legal to reply BAD for this command? I would disagree.

Actually, I think that's a conservative/liberal point. The RFC doesn't 
define validity, so you should be conservative and assume that BAD only 
covers syntactical invalidity, but accept that others think BAD also 
covers implementation-specific invalidity.

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 29 Jul 2012 07:44:09 -0700

On 28 Jul 2012, at 13:53, Barry Leiba <[email-masked]> wrote:

> (A digression. IMO, BAD is misnamed, but otherwise okay. It should have been broken down into the two cases NO [SYNTAXERROR] and NO [CANNOT].)
> 
> I believe that CANNOT is not a subcase of BAD.  BAD, as it's intended, means "the client should have known that it couldn't do that."  In other words, the client is misusing the protocol.  CANNOT can mean other things.
>  
> I'm with Timo: I don't like the first option.  Either you do *something* -- actually do the move in some way, or recognize that it's silly and just give it a new UID (and you can put text in to that effect) -- or you respond NO [CANNOT].  How about this text?:
> 
> "If the target mailbox is actually the same as the source (perhaps by a different name), the server MAY optimize for this situation.  If it does so, it MUST either ensure that the message gets a new UID (and respond OK), or respond NO [CANNOT]."

As a participant: why would the client want a new set of UIDs? I would rather do nothing or reject the command with a NO.

---

From: Barry Leiba <[email-masked]>
Date: Sun, 29 Jul 2012 10:58:03 -0400

>
> I'm with Timo: I don't like the first option.  Either you do *something*
> -- actually do the move in some way, or recognize that it's silly and just
> As a participant: why would the client want a new set of UIDs? I would
> rather do nothing or reject the command with a NO.
>

Indeed, you're right.  So maybe we should invert this, preferring NO:
<<
If the target mailbox is actually the same as the source (perhaps by a
different name), the server MAY optimize for this situation.  If it does
so, it does it in one of the following two ways:
1. It SHOULD take no action, and respond "NO [CANNOT]".
2. Otherwise, it MUST ensure that the message gets a new UID, and respond
"OK".

Barry, as participant

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 29 Jul 2012 19:01:15 +0200

I can't say why anyone would want that (other than a test suite looking 
for a really fast way to generate lots of \recent messages), but for 
some servers it would happen naturally, so allowing it is as harmless as 
it is useless.

I added this sentence: "The behaviour when the target mailbox is the 
same as the source is undefined." Perhaps I should add "At least one 
server is known to issue new UIDs and set \recent, another is known to 
fail the command with NO."?

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 29 Jul 2012 10:17:33 -0700

On 29 Jul 2012, at 10:01, Arnt Gulbrandsen <[email-masked]> wrote:

> I can't say why anyone would want that (other than a test suite looking for a really fast way to generate lots of \recent messages), but for some servers it would happen naturally, so allowing it is as harmless as it is useless.
> 
> I added this sentence: "The behaviour when the target mailbox is the same as the source is undefined." Perhaps I should add "At least one server is known to issue new UIDs and set \recent, another is known to fail the command with NO."?
> 

I would much prefer earlier suggestions explicitly stating allowed choices (e.g. Barry's proposal)

---

From: Barry Leiba <[email-masked]>
Date: Sun, 29 Jul 2012 13:54:47 -0400

> same as the source is undefined." Perhaps I should add "At least one server
> is known to issue new UIDs and set \recent, another is known to fail the
> command with NO."?
>

Yes, I very strongly prefer normative language here, rather than leaving it
entirely unspecified.  There are too many vague implementation choices in
IMAP already, which cause problems for client developers.

Barry

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 29 Jul 2012 22:27:42 +0200

On 07/29/2012 07:54 PM, Barry Leiba wrote:
> Yes, I very strongly prefer normative language here, rather than leaving
> it entirely unspecified.  There are too many vague implementation
> choices in IMAP already, which cause problems for client developers.

I very strongly dislike writing things like "servers may optimise 
unimportant corner cases" (I rephrased to bring the issue out more 
clearly). Yes, they may, but I don't think good RFCs should say things 
like that, and rewording to shroud the issue doesn't remove the problem.

I thought about possible rules that would satisfy both you, Alexey and 
myself, and I changed my own position. I now think that the only 
sensible behaviour is to mandate OK.

Arguments:

First, copystoreexpunge works that way. Do we have a good reason to fail 
in cases copystoreexpunge OKs? I fail to see any. Performance in corner 
cases doesn't matter, and the locking issue Bron brings up is a cyrus bug.

Second, other move-capable systems I use support noop moves, and it's 
convenient. SQL servers do not reject an UPDATE ... WHERE when zero rows 
would be affected, they say "OK, 0 rows affected". The unix mv command 
also accepts noop moves, and many, many shell scripts and state machines 
depend on it. "mv $a $basedir/unprocessed" works even if $a happens to 
be in unprocessed already, "mv $(find ...) foo" works, etc.

Arnt