Subject: Re: [imap5] Should unsolicited EXPUNGE responses be returned during UID MOVE?

From: Bron Gondwana <[email-masked]>
Date: Sat, 2 Jun 2012 00:26:58 +0200

On Thu, May 31, 2012 at 12:11:35PM +0200, Jan Kundrát wrote:
> On 05/31/12 11:52, Timo Sirainen wrote:
> > Another possibility:
> >      C: a UID MOVE 42:45 forble
> >      S: * OK [COPYUID 432432 1202:1205] Messages moved
> >      S: * 15 EXPUNGE
> >      S: * 15 EXPUNGE
> >      S: * 15 EXPUNGE
> >      S: * 15 EXPUNGE
> >      S: a OK Done
> 
> The untagged OK with COPYUID doesn't specify to which UID MOVE it is
> related, unfortunately, and will therefore break with concurrent UID
> MOVE operations. My GUI will happily send concurrent UID MOVEs when the
> connection is slow and user moves her mouse fast enough, so I believe
> it's a real problem.

Huh?  Who sends untagged COPYUID responses?  Cyrus puts the COPYUID
in the tagged OK.  The docs back me up:

http://www.faqs.org/rfcs/rfc2359.html

4.3. COPYUID response code

   Successful COPY and UID COPY commands return a COPYUID response code
   in the tagged OK response [...]

Bron.

---

From: Jan Kundrát <[email-masked]>
Date: Sat, 02 Jun 2012 00:54:04 +0200

>    in the tagged OK response [...]

When my client sees an EXPUNGE, it removes the message from its cache,
almost immediately. If the UID MOVE command sent regular EXPUNGEs
followed by regular tagged OK COPYUID, my client would have lost the
cached data, effectively negating any benefits of UIDPLUS.

Either you postpone the EXPUNGEs until the tagged OK with COPYUID
arrives (which looks very un-IMAPy to me), or you somehow report the
COPYUID before the untagged EXpUNGEs *and* the tagged OK gets sent.

With kind regards,
Jan

-- 
Trojita, a fast e-mail client -- http://trojita.flaska.net/

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 01 Jun 2012 22:59:04 +0000

------ Original Message ------
From: "Jan Kundrát" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 2/06/2012 10:54:04 a.m.
Subject: Re: [imap5] Should unsolicited EXPUNGE responses be returned 
during UID MOVE?
>COPYUID before the untagged EXpUNGEs *and* the tagged OK gets sent.
  
Or, your client could - knowing it is in a MOVE command - defer the 
processing of those expunges, or actually probably ignore them, since 
the information required is in the COPYUID response (tells you which 
messages were moved and are therefore gone from source folder).
  
Maybe we should make MOVE depend on existence of UIDPLUS...
  
What do other clients do here?
  
Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 2 Jun 2012 02:06:06 +0300

On 2.6.2012, at 1.59, Adrien W. de Croy wrote:

> Or, your client could - knowing it is in a MOVE command - defer the processing of those expunges, or actually probably ignore them, since the information required is in the COPYUID response (tells you which messages were moved and are therefore gone from source folder).

That's possible of course, but also pretty kludgy. IMAP in general doesn't require clients to do such things.

And anyway it shouldn't ignore them entirely, because there might be other EXPUNGEs reported that didn't come from a MOVE.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 01 Jun 2012 23:19:10 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Jan Kundrát" <[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 2/06/2012 11:06:06 a.m.
Subject: Re: [imap5] Should unsolicited EXPUNGE responses be returned 
during UID MOVE?
>That's possible of course, but also pretty kludgy. IMAP in general doesn't require clients to do such things.
  
The flip-side to that is that implementation design decisions shouldn't 
necessarily force the protocol.
  
Other clients don't have this problem, so can you blame the protocol.
>And anyway it shouldn't ignore them entirely, because there might be other EXPUNGEs reported that didn't come from a MOVE.

Not sure that's possible/legal.  I don't think a server is allowed to 
send unsolicited EXPUNGE responses whilst in another command.

Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 2 Jun 2012 02:25:46 +0300

On 2.6.2012, at 2.19, Adrien W. de Croy wrote:

> The flip-side to that is that implementation design decisions shouldn't necessarily force the protocol.

I am talking about the protocol, not the client implementation. The protocol should try to behave predictably and without special exceptions as much as possible. Jan's client behaves the way an "ideal IMAP client" would behave. Saying "other IMAP clients don't have this problem" doesn't matter, because pretty much all IMAP clients behave badly or horribly badly. The protocol shouldn't be designed for those.

>  Not sure that's possible/legal.  I don't think a server is allowed to send unsolicited EXPUNGE responses whilst in another command.

It depends on the command. The way UID MOVE is currently defined it is possible for other EXPUNGEs to be sent. (And I think it should stay that way, otherwise it would be an exception to how the protocol normally behaves. Exceptions are bad.)

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 2 Jun 2012 17:23:15 +0200

This makes a lot of sense... but I think the basic problem here is one 
we cannot fix any more: COPYUID cannot be parsed without knowledge of 
the command that caused it.

So, I think that a client which issues MOVE has to add an extra 
reference to the cached message(s) when it issues MOVE, and when it 
parses the COPYUID it looks at the reference(s) held by the command.

MOVE makes this much more acute, but I note that the problem exists with 
copystoreexpunge too:

C: a COPY 1000:1999 foo
S: * 1000 EXPUNGE
S: * 1500 EXPUNGE
S: a OK [COPYUID 4132 1000:1499,1501:1999 1000:1998] done

(One of the two messages had been copied by the time it was expunged, 
the other was expunged before it could be copied.)

Arnt