Subject: Re: [imapext] I-D ACTION:draft-ietf-imapmove-command-00.txt

From: Bron Gondwana <[email-masked]>
Date: Wed, 01 Aug 2012 14:10:40 +0200

On Sun, Jul 29, 2012, at 10:27 PM, Arnt Gulbrandsen wrote:
> On 07/29/2012 07:54 PM, Barry Leiba wrote:
> > Yes, I very strongly prefer normative language here, rather than leaving
> > it entirely unspecified.  There are too many vague implementation
> > choices in IMAP already, which cause problems for client developers.
> 
> I very strongly dislike writing things like "servers may optimise 
> unimportant corner cases" (I rephrased to bring the issue out more 
> clearly). Yes, they may, but I don't think good RFCs should say things 
> like that, and rewording to shroud the issue doesn't remove the problem.
> 
> I thought about possible rules that would satisfy both you, Alexey and 
> myself, and I changed my own position. I now think that the only 
> sensible behaviour is to mandate OK.
> 
> Arguments:
> 
> First, copystoreexpunge works that way. Do we have a good reason to fail 
> in cases copystoreexpunge OKs? I fail to see any. Performance in corner 
> cases doesn't matter, and the locking issue Bron brings up is a cyrus bug.

I agree - it's a Cyrus bug.  I'm going to fix Cyrus.  Sorry I even
brought the topic up!  Let's behave like COPYSTOREEXPUNGE and not
screw around.

> Second, other move-capable systems I use support noop moves, and it's 
> convenient. SQL servers do not reject an UPDATE ... WHERE when zero rows 
> would be affected, they say "OK, 0 rows affected". The unix mv command 
> also accepts noop moves, and many, many shell scripts and state machines 
> depend on it. "mv $a $basedir/unprocessed" works even if $a happens to 
> be in unprocessed already, "mv $(find ...) foo" works, etc.

What would COPYUID say?

Bron.
-- 
  Bron Gondwana
  [email-masked]

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 1 Aug 2012 14:24:43 +0200

> What would COPYUID say?

Something that describes reality.

If the server doesn't reissue UIDs, COPYUID has to say something like 
42:69 42:69 (ie. unity mapping). If the server reissues, then it has to 
say 42:69 12342:12369 or whatever. Or the server might not send COPYUID, 
wich would cause some unnecessary cache misses, but who cares about 
cache efficiency in such uncommon cases?

Arnt

---

From: Bron Gondwana <[email-masked]>
Date: Wed, 01 Aug 2012 14:57:51 +0200

> cache efficiency in such uncommon cases?

Is a client going to cope with unity mapping?  I guess they will actually,
since they presumably think they are different folders for some reason.

Makes sense.

Bron.
-- 
  Bron Gondwana
  [email-masked]

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 1 Aug 2012 14:59:56 +0200

> since they presumably think they are different folders for some reason.

Maybe I ought to put in a hint that this case warrants testing (and for 
clients, both with and without the extension).

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 09 Aug 2012 00:12:41 -0700

>> sensible behaviour is to mandate OK.
(Speaking as a developer, not as a WG co-chair)
If you mean that a compliant server would have to return COPYUID with 
renumbered messages, than I am unhappy to implement additional code (UID 
renumbering) if nobody can think of a reason how this is useful.
>> be in unprocessed already, "mv $(find ...) foo" works, etc.
I have the same question. But if the server can just omit COPYUID in 
this case, I would be Ok with that.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 9 Aug 2012 15:28:26 +0200

> this case, I would be Ok with that.

As I see it, omitting COPYUID is acceptable. But I think that's really a 
UIDPLUS question. I don't see either a requirement there to send COPYUID 
for all UID COPY commands nor an explicit permission to omit it.

Clients have to deal with servers that don't support UIDPLUS, so I would 
be very surprised if omitting COPYUID were to lead to interoperability 
problems.

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 10 Aug 2012 15:37:57 +0100

On 09/08/2012 14:28, Arnt Gulbrandsen wrote:
> As I see it, omitting COPYUID is acceptable. But I think that's really 
> a UIDPLUS question. I don't see either a requirement there to send 
> COPYUID for all UID COPY commands nor an explicit permission to omit it.
>
> Clients have to deal with servers that don't support UIDPLUS, so I 
> would be very surprised if omitting COPYUID were to lead to 
> interoperability problems.

Ok. Can you tell me what is the currently allowed behavior (or currently 
allowed choices) about MOVE to the currently selected mailbox? I think 
this still needs clarification in the draft and I am a bit confused 
about where we stand on this.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 10 Aug 2012 17:06:05 +0200

> about where we stand on this.

The draft says MOVE works as COPYSTOREEXPUNGE. I think that's an 
excellent principle, and we should deviate from it only when we have 
good reason to. It's good that a client needs no new if() when adding MOVE.

In this case, what COPYSTOREEXPUNGE does is assign new UIDs for the 
messages in question. It would be possible to optimise that to not 
assign new UIDs, but do we have any good reasons to do so? I see none.

I could add some text, but I fear it would sound a little stupid. "In 
this case, like in all others, UID MOVE has the same effect as a 
sequence of UID COPY, UID STORE and UID EXPUNGE."

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 10 Aug 2012 17:35:02 +0200

I answered Alexey:
> assign new UIDs,

I thought some more about this optimisation, and I don't want to go there.

Today, if a client moves a message (whether with or without the 
extension), the message is \recent afterwards. Preserving that rule with 
this optimisation is unpleasant. Complicating a one-sentence \recent 
rule in order to support an obscure optimisation is IMO disproportionate.

It's also possible that some clients depend on UID allocation being 
monotonic to a somewhat higher degree than they perhaps should.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Fri, 10 Aug 2012 09:20:51 -0700 (PDT)

> monotonic to a somewhat higher degree than they perhaps should.

Agreed on all points.

				Ned

---

From: Barry Leiba <[email-masked]>
Date: Fri, 10 Aug 2012 14:20:02 -0400

> monotonic to a somewhat higher degree than they perhaps should.

I'm confused: are you saying that clients should be willing to accept a new
message with a UID that is *less* than those of some existing ones, and
that if they don't accept that, they're being overly rigorous?

The IMAP spec is very clear on this point, and there are good
implementation reasons for it -- such as being able to find all new
messages by remembering the highest UID seen, and then fetching n:*.

Barry

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 10 Aug 2012 22:21:16 +0200

On 08/10/2012 08:20 PM, Barry Leiba wrote:
> I'm confused: are you saying that clients should be willing to accept a
> new message with a UID that is *less* than those of some existing ones,
> and that if they don't accept that, they're being overly rigorous?

I'm saying that this optimisation isn't valuable enough that we should 
even be discussing the question. And it's not a new message, it's a UID 
mentioned in COPYUID, not quite the same thing. Do you want to discuss 
that difference or just skip the optimisation?

Arnt

---

From: Barry Leiba <[email-masked]>
Date: Fri, 10 Aug 2012 16:58:47 -0400

> I'm saying that this optimisation isn't valuable enough that we should even
> be discussing the question. And it's not a new message, it's a UID mentioned
> in COPYUID, not quite the same thing. Do you want to discuss that difference
> or just skip the optimisation?

No, I want to know what you mean by this, which you did not address in

Barry

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 10 Aug 2012 23:09:36 +0200

>> >monotonic to a somewhat higher degree than they perhaps should.

I was too oblique. Sorry.

Clearly, if a server assigns a new UID, the newly assigned is has to be 
greater than or equal to the advertised uidnext value for that mailbox. 
But is a client permitted to assume that any UID mentioned in the target 
set of a COPYUID response code is newly assigned?

I believe that most clients are either written by programmers who expect 
every UID there to be newly assigned, or have only been tested with 
newly assigned UIDs, or both. In light of this, I think that the 
optimisation is not worth even discussing.

Arnt

---

From: Barry Leiba <[email-masked]>
Date: Fri, 10 Aug 2012 17:16:03 -0400

> greater than or equal to the advertised uidnext value for that mailbox. But
> is a client permitted to assume that any UID mentioned in the target set of
> a COPYUID response code is newly assigned?

I believe so, yes.  The sense is that the "new" copy is, indeed, new.

> every UID there to be newly assigned, or have only been tested with newly
> assigned UIDs, or both.

Yes.

> In light of this, I think that the optimisation is not worth even discussing.

OK.  Thanks for clarifying.

b

---

From: Brandon Long <[email-masked]>
Date: Fri, 10 Aug 2012 15:07:44 -0700

On Fri, Aug 10, 2012 at 2:09 PM, Arnt Gulbrandsen
<[email-masked]>wrote:

> assigned UIDs, or both. In light of this, I think that the optimisation is
> not worth even discussing.

I'm pretty sure that Gmail violates that.  Since a COPY translates to
adding a label, and there is only ever one "copy" of a message, if you copy
a message to a folder that the message is already in, you'll get the
original UID back.

If you're copying from the same folder to itself, we will give all of the
messages new UIDs.

This and other reasons also mean that the destination UIDs in the COPYUID
are necessarily in numeric order, either.  Though, it also wasn't clear to
me from the spec whether or not the source UIDs are supposed to be in the
order requested or not (we re-order the source UIDs to be in numeric order).

Nobody's complained yet, at least that I know of.

Brandon