Subject: collations: mismatch between equality and ordering
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Date: Fri, 24 Feb 2006 18:11:56 +0100
List-Id: <ietf-imapext.imc.org>

Hi,

in the current collation draft, it is possible to sort strings that 
cannot be tested for equality: A collation has a domain. It can test 
any pair of strings for equality, provided both strings are within its 
domain. And it can order pair of strings, whether they are in its 
domain or not. I find that last bit highly dubious, and want to strike 
it.

I think that if a server wants to sort a set of strings and the chosen 
collation can't handle some, the server can fall back to i;octet for 
the rest. The collation itself does not need to define any behaviour 
outside its domain.

Striking it would also permit the number of dependences on i;octet to 
decrease. I believe I mentioned that a few weeks ago. I think that with 
this change, it should be possible to write a server which uses 
exclusively unicode internally and does not implement i;octet. (Not an 
IMAP server, of course. An IMAP server has to do i;octet.)

Comments? Pretty please?

In general news: Alexey Melnikov, Frank Ellermann and Mark Davis/the 
Unicode Technical Committee have gone over the draft. Long comments 
from all three. Good stuff. This issue is one I came across while 
pondering Mark's mail. I'll go on for another few hours today, 
hopefully I'll have a -07 before the end of the day.

Arnt

Date: Wed, 22 Feb 2006 19:51:20 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Timo Sirainen <[email-masked]>, [email-masked]
Subject: Re: ACL once more
Sender: [email-masked]

Dave Cridland wrote:

> On Tue Feb 21 14:23:13 2006, Timo Sirainen wrote:
>
>> On Tue, 2006-02-21 at 14:12 +0000, Dave Cridland wrote:
>> > On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
>> > > > > Is SETACL id -<all rights> the same as DELETEACL id?
>> > > > > > > > No. SETACL id -<all rights> means id is denied all 
>> rights in some > specific manner.
>>
>> Isn't that SETACL -id <all rights>?
>>
>> Minusing rights only removes the rights from the ACL, so I'd think once
>> all of them are removed, the whole ACL could be deleted since there's
>> nothing left.
>
> Ah. Hmmm. Quite so.
>
> Options:
>
> 1) It's equivalent then to SETACL id "", because it's removing any 
> rights from the ACL entry for the id. Which might mean the same as 
> "this identifier has no rights", or might mean "the rights of this 
> identifier are not affected by this line", which is basically 
> DELETEACL id, depending on whether "most specific" ACL entires are 
> used, or "union" applies.
>
> 2) According to the first para of Section 3.1, it might mean the same 
> as SETACL -id lrswipkxeta anyway, because the access control list as a 
> whole is changed such that the identifier does not have the rights 
> removed.

No, the text in section 3.1 is saying that "SETACL id -<rights>" only 
affects the entry for "id", not the whole ACL. So interpretation #1 is 
the correct one.

Alexey

Date: Tue, 21 Feb 2006 16:57:18 +0000
Subject: Re: ACL once more
From: Dave Cridland <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue Feb 21 14:23:13 2006, Timo Sirainen wrote:
> On Tue, 2006-02-21 at 14:12 +0000, Dave Cridland wrote:
> > On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
> > > > > Is SETACL id -<all rights> the same as DELETEACL id?
> > > > > > > > No. SETACL id -<all rights> means id is denied all 
> rights in some > specific manner.
> 
> Isn't that SETACL -id <all rights>?
> 
> Minusing rights only removes the rights from the ACL, so I'd think 
> once
> all of them are removed, the whole ACL could be deleted since 
> there's
> nothing left.
> 
> 
> 
Ah. Hmmm. Quite so.

Options:

1) It's equivalent then to SETACL id "", because it's removing any 
rights from the ACL entry for the id. Which might mean the same as 
"this identifier has no rights", or might mean "the rights of this 
identifier are not affected by this line", which is basically 
DELETEACL id, depending on whether "most specific" ACL entires are 
used, or "union" applies.

2) According to the first para of Section 3.1, it might mean the same 
as SETACL -id lrswipkxeta anyway, because the access control list as 
a whole is changed such that the identifier does not have the rights 
removed.

So, erm, I really don't know. :-)

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Subject: Re: ACL once more
From: Timo Sirainen <[email-masked]>
To: Dave Cridland <[email-masked]>
Cc: [email-masked]
Date: Tue, 21 Feb 2006 16:23:13 +0200
Sender: [email-masked]

On Tue, 2006-02-21 at 14:12 +0000, Dave Cridland wrote:
> On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
> > 
> > Is SETACL id -<all rights> the same as DELETEACL id?
> > 
> > 
> > 
> No. SETACL id -<all rights> means id is denied all rights in some 
> specific manner.

Isn't that SETACL -id <all rights>?

Minusing rights only removes the rights from the ACL, so I'd think once
all of them are removed, the whole ACL could be deleted since there's
nothing left.

Date: Tue, 21 Feb 2006 14:12:33 +0000
Subject: Re: ACL once more
From: Dave Cridland <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
> 
> Is SETACL id -<all rights> the same as DELETEACL id?
> 
> 
> 
No. SETACL id -<all rights> means id is denied all rights in some 
specific manner.

DELETEACL id means that id is not mentioned specifically, but may 
have rights due to being part of a group which is.

Mind you, SETACL id "" may, on some servers, have the same effect as 
DELETEACL id *or* SETACL id -<all rights>, depending on how the ACL 
entries result in a rights list.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Subject: ACL once more
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Tue, 21 Feb 2006 13:07:53 +0200
Sender: [email-masked]

Is SETACL id -<all rights> the same as DELETEACL id?

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-condstore-09.txt 
Date: Wed, 15 Feb 2006 10:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: IMAP Extension for Conditional STORE operation
	Author(s)	: A. Melnikov, S. Hole
	Filename	: draft-ietf-imapext-condstore-09.txt
	Pages		: 0
	Date		: 2006-2-13
	
Often, multiple IMAP (RFC 3501) clients need to coordinate changes to
a common IMAP mailbox. Examples include different clients working on
behalf of the same user, and multiple users accessing shared mailboxes.
These clients need a mechanism to synchronize state changes for messages within the mailbox. They must be able to guarantee that only one client can change message state (e.g., message flags) at any time.  An example of such an application is use of an IMAP mailbox as a message queue with multiple dequeueing clients.

The Conditional Store facility provides a protected update mechanism for message state information that can detect and resolve conflicts between multiple writing mail clients.

The Conditional Store facility also allows a client to quickly resynchronize mailbox flag changes.

This document defines an extension to IMAP (RFC 3501).

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-09.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-imapext-condstore-09.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-imapext-condstore-09.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-condstore-09.txt 
Date: Mon, 13 Feb 2006 15:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: IMAP Extension for Conditional STORE operation
	Author(s)	: A. Melnikov, S. Hole
	Filename	: draft-ietf-imapext-condstore-09.txt
	Pages		: 0
	Date		: 2006-2-13
	
Often, multiple IMAP (RFC 3501) clients need to coordinate changes to
a common IMAP mailbox. Examples include different clients working on
behalf of the same user, and multiple users accessing shared mailboxes.
These clients need a mechanism to synchronize state changes for messages within the mailbox. They must be able to guarantee that only one client can change message state (e.g., message flags) at any time.  An example of such an application is use of an IMAP mailbox as a message queue with multiple dequeueing clients.

The Conditional Store facility provides a protected update mechanism for message state information that can detect and resolve conflicts between multiple writing mail clients.

The Conditional Store facility also allows a client to quickly resynchronize mailbox flag changes.

This document defines an extension to IMAP (RFC 3501).

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-09.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-imapext-condstore-09.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-imapext-condstore-09.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Mon, 06 Feb 2006 12:11:19 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Lisa Dusseault <[email-masked]>, Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Dave Cridland wrote:

> On Thu Feb  2 22:32:50 2006, Lisa Dusseault wrote:
>
>> Dave said "the deficient server in this case causes no damage other 
>>  than a spurious round-trip from a client.", and Mark said something  
>> about guns and preschoolers :)
>
> Yeah. I'm talking specifically the case where a server increments a 
> MODSEQ for a message when the message is not visibly changed. Which is 
> the same as, for instance, a change to an ANNOTATion when the client 
> doesn't speak ANNOTATE - the client isn't damaged by it as such, and 
> has to expect it.

It is also the same as if another client set a flag and then unset it.

Date: Fri, 03 Feb 2006 09:02:00 +0000
Subject: Re: draft-newman-comparator updated
From: Dave Cridland <[email-masked]>
To: Martin Duerst <[email-masked]>
Cc: Arnt Gulbrandsen <[email-masked]>, <[email-masked]>, <[email-masked]>
Sender: [email-masked]

On Fri Feb  3 03:10:17 2006, Martin Duerst wrote:
> 
> At 23:18 06/02/02, Arnt Gulbrandsen wrote:
> >
> >Hi,
> >
> >the draft is lying silently in the internet-draft editor queue. 
> However, I would appreciate reviews, speedily if possible. If you 
> have the time and it isn't yet in your internet-drafts mirror, I 
> can send a copy.
> >
> >There are a few changes.
> >
> >The basic and biggest change is that collation is defined in terms 
> of octet strings, with explicit notes that implementations may 
> choose to use character strings, and notes that in that case, e.g. 
> i;octet may not be possible to implement. Various minor changes 
> changes result.
> 
> This may work out, or it may not work out. I'll check this 
> carefully,
> but don't have any time this (and probably next) week, sorry.
> 
> The main problem is that octet-based collations (except possibly for
> i;octet) need an encoding, but character-based collations obviously
> don't. So if I have some French collation defined over characters,
> and some French collation defined over iso-8859-1, how will they
> be distinguished? How can we make sure registrations can be
> as compact and easy as possible?

I thought quite a few of the interested parties had essentially 
agreed on this, but I could be wrong.

I thought there was a general feel of consensus for specifying that 
all comparators operated on octets, and that any comparators using 
character strings used the same encoding of UTF-8.

In principle, this means that given a character string, feeding it 
into "i;octet" means first encoding it into UTF-8. However, for 
collation, I believe that UTF-8 collation using "+i;octet" and 
unicode code-point order are the same thing, hence there's an obvious 
shortcut to be made.

Even more obvious shortcuts apply to "i;ascii-casemap", and 
"i;ascii-numeric", since both *can* operate on UTF-8 encoded octet 
strings, *or* unicode codepoints, and have the same effect very 
easily. They might say "ascii" in the name, but we can retrofit a 
"ASCII range" meaning to that without breaking semantics.

And that effect is the key - all comparators would be defined in 
terms of UTF-8, and whether or not the implementation could deal 
directly with some other representation of Unicode character strings, 
the overall effect must be identical.

So a French collation is only defined on characters in as much as it 
is defined in terms of the UTF-8 encoding of those characters. It's 
simply not allowed to deal with ISO-8859-15, nor ISO-8859-1, and 
"which character set" is removed from the comparator library 
altogether. This makes quite a lot of sense - you don't want to 
potentially have different collating order depending on the character 
set of the data, do you?

For real-world implementations, it's quite likely that a library will 
have entry points for bare octet strings, some native Unicode (like 
UCS-2 or whatever), and possibly octet string and encoding pairs - 
and that's absolutely fine, as long as the effect is identical to 
using UTF-8 encoded octet strings.

That doesn't mean that "i;octet" makes sense for all protocols, of 
course, and I cannot see the point of using is as a tie-break, when 
some existing protocols using comparators already provide a means for 
the user to select any tie break if required. This practise is 
useful, so feel free to suggest it in protocol support.

I'll put together some other suggestions for handling the various 
non-compatible changes between ACAP/SIEVE style comparators and these 
ones over the next few days.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Fri, 03 Feb 2006 12:10:17 +0900
To: Arnt Gulbrandsen <[email-masked]>, IMAP Extensions WG <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: draft-newman-comparator updated
Sender: [email-masked]

At 23:18 06/02/02, Arnt Gulbrandsen wrote:
 >
 >Hi,
 >
 >the draft is lying silently in the internet-draft editor queue. However, 
I would appreciate reviews, speedily if possible. If you have the time and 
it isn't yet in your internet-drafts mirror, I can send a copy.
 >
 >There are a few changes.
 >
 >The basic and biggest change is that collation is defined in terms of 
octet strings, with explicit notes that implementations may choose to use 
character strings, and notes that in that case, e.g. i;octet may not be 
possible to implement. Various minor changes changes result.

This may work out, or it may not work out. I'll check this carefully,
but don't have any time this (and probably next) week, sorry.

The main problem is that octet-based collations (except possibly for
i;octet) need an encoding, but character-based collations obviously
don't. So if I have some French collation defined over characters,
and some French collation defined over iso-8859-1, how will they
be distinguished? How can we make sure registrations can be
as compact and easy as possible?

Regards,    Martin.

 >There are also more definitions and requirements. For example, equality 
has to be implemented if ordering is, and if ordering returns equal, 
equality must return true. That sort of thing.
 >
 >I am unhappy with one thing: i;octet is still used a little as a last 
resort to resolve equality/order, which brings it very close to 
mandatory-to-implement. This doesn't seem necessary from an architectural 
point of view. I just haven't found a solution I really like.
 >
 >Arnt
 > 

Date: Thu, 02 Feb 2006 23:29:16 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Lisa Dusseault <[email-masked]>
Cc: Mark Crispin <[email-masked]>, <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 22:32:50 2006, Lisa Dusseault wrote:
> Dave said "the deficient server in this case causes no damage other 
>  than a spurious round-trip from a client.", and Mark said 
> something  about guns and preschoolers :)
> 
> 
Yeah. I'm talking specifically the case where a server increments a 
MODSEQ for a message when the message is not visibly changed. Which 
is the same as, for instance, a change to an ANNOTATion when the 
client doesn't speak ANNOTATE - the client isn't damaged by it as 
such, and has to expect it.

What would happen is that the client would see a changed 
HIGHESTMODSEQ, and accordingly issue a FETCH  ... CHANGEDSINCE, which 
would yield some response, which would provide data the client 
already had.

Where the HIGHESTMODSEQ is the same, the FETCH can be eliminated 
(and, optionally, even the SELECT).

Obviously the latter is more efficient and faster, and therefore a 
server which incremements the Modification sequence when no 
Modification has taken place is not as good as it could be.

Obviously the client will remain in sync without resorting to any 
kind of workaround - resync just happens to be more costly - so the 
server is not broken as such, and such behaviour could be the result 
of a modification the client is not capable enough to see.

> Were you two talking about different cases?  Would the outcome of 
> bad  client use of MODSEQ would be dire or simply the client's own 
> problem?

Mark's suggesting that, because all known uses of MODSEQ involve 
ignoring both its numeric nature and its status as a FETCH item, then 
it's potentially inviting doom by having MODSEQs have a numeric, 
ascending, per-message-accessible nature. This is a different case to 
the above. At least I think so.

I can't see this being the case - a client expecting some specific 
meaning to the difference of two MODSEQ numbers is probably asking 
for trouble (except in two vanishingly rare cases, where the 
difference is exactly 1), but for the most part, it won't cause any 
damage other than to itself, and potentially the user's data bill. 
Other clients I cannot see being affected by that, except 
beneficially.

You see, I could assume that an increment of 2 to HIGHESTMODSEQ 
implied there were two modifications, and having found two modified 
messages, I could decide that I'd found any modified messages, and 
consider myself resynchronized. Sometimes, I might even *be* 
resynchronized. Trouble is, any modification may encompass as many or 
few messages as possible, hence that's an invalid assumption, and the 
client may actually be out of sync. But, this doesn't effect another 
client. What can happen, however, is that the out of sync client can 
mark deleted, and subsequently expunge, the wrong message. But any 
out-of-sync client can do that perfectly well without CONDSTORE, and 
I don't think it makes the problem markedly worse.

You can cause wasteful round-trips for other clients by toggling a 
flag on and off across all messages, but I can't see this being done 
by accident much, and it merely degenerates CONDSTORE to uselessness, 
rather than actively breaking it.

I'm sure we'll eventually find a use for access to the MODSEQ of 
messages, probably something to do with archiving away old messages, 
or watching the changes to status of work items in a shared mailbox - 
I could probably come up with plenty of uses. I can't come up with 
much in the way of harm.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Thu, 2 Feb 2006 14:44:03 -0800
From: Mark Crispin <[email-masked]>
To: Lisa Dusseault <[email-masked]>
cc: Dave Cridland <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Thu, 2 Feb 2006, Lisa Dusseault wrote:
> Were you two talking about different cases?  Would the outcome of bad client 
> use of MODSEQ would be dire or simply the client's own problem?

That's the problem.  I don't know; and in such cases like this pessimism 
seems to be the more prudent course.

There's the harm in an implementation which uses empirical testing instead 
of the specification to define behavior.

There's the harm of making unwarranted assumptions, and decreeing that 
those who do not conform to those assumptions are "defective".  Consider 
those clients that assume that all names are "dual-use", all mailboxes 
support shared access, all servers support such-and-such extension, etc.

There's the harm of making a bizarre interpretation of the specification, 
claiming that interpretation is correct simply because it is not 
explicitly forbidden (never mind that it contracts something else), and 
then behaving badly against other implementations.

I've seen all of the above happen; and as a result I get nervous when 
there is wiggle-room in the specification or statements that it's 
"obvious" that it's such-and-such way.

I'm not saying that this applies here.  But has anyone made sure that it 
does not?

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Cc: Mark Crispin <[email-masked]>, <[email-masked]>
From: Lisa Dusseault <[email-masked]>
Subject: Re: MODSEQ in CONDSTORE
Date: Thu, 2 Feb 2006 14:32:50 -0800
To: Dave Cridland <[email-masked]>
Sender: [email-masked]

Dave said "the deficient server in this case causes no damage other  
than a spurious round-trip from a client.", and Mark said something  
about guns and preschoolers :)

Were you two talking about different cases?  Would the outcome of bad  
client use of MODSEQ would be dire or simply the client's own problem?

lisa

Date: Thu, 02 Feb 2006 22:01:37 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 17:23:04 2006, Mark Crispin wrote:
> On Thu, 2 Feb 2006, Dave Cridland wrote:
>> A user setting \Seen (possibly implicitly in a FETCH BODY[...]) 
>> causes the MODSEQ to be bumped for that message. If the 
>> implementation is poor, then this can cause the MODSEQ seen by 
>> other users to be changed, too. That's undesirable, but as a 
>> client implementor, I infinitely prefer that level of 
>> implementation to none at all.
> 
> But that's only if (1) the mailbox has per-user flags, (2) the 
> mailbox has MODSEQ.  Right?
> 
> 
Exactly, that's precisely the circumstance I'm describing.

> Personally, I never thought that per-user flags were worth the 
> complexity, even though I was guilty of creating them.  But it was 
> only as a grim necessity for bboards and newsgroups; and even then 
> that was just a single flag.  The idea of using MODSEQ for such a 
> thing staggers me.
> 
> 
It seems quite logical to me. I'm told it's a little tricky to get 
right/efficient with CONDSTORE, mind.

>> Otherwise, you're effectively lowering the width of the UID space, 
>> down to 30, or 22, bits. That's still probably plenty of space, 
>> but there's something philosophically wrong about an extension 
>> lowering the scalability of the protocol it extends.
> 
> This argument only applies when you are monotonically allocating 
> UIDs (otherwise it's your implementation that reduces the UID 
> space, not MODSEQ).  And if you are monotonically allocating UIDs, 
> then there is no reason not to monotonically allocate MODSEQ.  The 
> notion of there being over 4 billion updates to one mailbox is 
> simply not credible.
> 
> 
Probably not right now, but the notion that if you allow for 
4,294,967,295 messages in a mailbox in the protocol - and never mind 
that existing servers will collapse long before that - and the 
arrival of each counts as a modification, and there can be further 
modifications, therefore that the maximum number of modifications 
before an overflow event should be higher than 4,294,967,295 seems 
perfectly sensible to me, especially when no "real-world" server has 
to support more than 4,294,967,295, it's only the protocol.

>>> I disagree.  Either MODSEQ reports that something touched the 
>>> message, or it reports that something changed the mesage.  As 
>>> matters stand, it is ambiguous.  A client may care.
>> Well, let me put it this way - a client which actually does care 
>> is broken. A server which claims there to be a change when there 
>> is not is deficient.
> 
> [sarcasm] And the IMAP community never, ever, produces broken 
> clients and deficient servers (or deficient clients and broken 
> servers). [/sarcasm]
> 
> We should have learned the painful lesson by now.
> 
> 
Sure... But the deficient server in this case causes no damage other 
than a spurious round-trip from a client.

I don't see how a client could assume that a MODSEQ change to a 
message represented a visible change to the FLAGS, but even if it 
could, what good would it do? Surely in finding the MODSEQ, it could 
easily enough find the FLAGS at the same time? And if it did make 
such an assumption somehow, what would it then do which would affect 
other clients?

>>>> Well, I don't think it's much of a problem. Most of the 
>>>> computers I work with these days increasingly use "unsigned 
>>>> long" for 64-bit values - even my laptop is 64-bit, but the vast 
>>>> majority of the 32-bit systems I use seem to support some kind 
>>>> of "unsigned long long" or equivalent.
>>> That hasn't been my experience... :-(
>> Get a better compiler? :-)
> 
> Sure, please write it.  And give it away for free, along with free 
> technical support.
> 
> 
To be fair, there's not much point competing with the GCC community, 
who seem to be doing a marvellous job there.

> I have a difficult time tolerating the "my system has 
> such-and-such, and it's your fault if your system does not have it 
> too" attitude.  Either we care about interoperability and 
> implementation costs, or we should forget this IETF nonsense and go 
> home.
> 
> 
Mark, I obviously care about interoperability, as we're right here 
discussing it. I obviously care about implementation costs, because I 
provided an estimate. I'm also already at home, come to that.

I'm simply saying that many systems do, indeed, have 64-bit unsigned 
integer datatypes available, and that it appears to be an increasing 
trend, and I actually find it very hard to believe that there exists 
any current desktop system which does not have 64-bit support in the 
compiler and libraries.

There are indeed some systems that don't, such as Symbian for one, 
but in my considered opinion, this doesn't matter, because it only 
affects client implementations, and client implementations can cope 
perfectly well with using 3 strings, for only a slight overhead.

[Aside: Symbian's compiler is GCC, and does indeed have unsigned long 
long in at least some versions, but the descriptor library has no 
support for it, only limited support for a C++-implemented signed 
64-bit int.]

The implementation cost is therefore very slight indeed, unless the 
client stores every MODSEQ's value and/or performs arithmetic or 
comparisons upon them, and this is unlikely to be beneficial to the 
client, and it is certainly not required for the purposes of faster 
resynchronization, which is the obvious use-case for CONDSTORE on 
limited platforms anyway.

On those platforms where it's possible to build a usable server which 
could overflow a 32-bit MODSEQ, whether by an astonishing throughput 
of STORE commands or a truly impressive amount of exceedingly fast 
disk, I refuse to believe that a compiler with 64-bit support is 
going to be a problem to find.

>> I don't believe clients ever need to handle more than 3 MODSEQ 
>> values at any time, and only need to store 2 permanently. If I 
>> needed to store each message's MODSEQ, I'd probably feel 
>> differently, but I don't think that's ever the case.
> 
> Are you certain?  IMHO, the client ability to fetch a message's 
> MODSEQ directly is like placing a loaded pistol with a hair trigger 
> in a room of preschoolers.  MODSEQ's ascending property makes it 
> nice and shiny too.
> 
> 
Well, no, I'm not certain there are no cases where being able to 
treat the 64-bit MODSEQ as a 64-bit number is useful to a client, 
merely that I've not found one useful enough to implement yet. Nor 
have I found the ability to directly find MODSEQ values for messages 
useful, as yet. There's no harm in doing so, of course.

However, I've implemented the client side for the uses of fast 
resynchronization, and I use it exclusively and daily to read my 
mail, and it works perfectly well, dealing with only three MODSEQ 
values per mailbox. Nor can I see anywhere in my code where instead 
of using a true 64-bit integer and comparing, I couldn't just as 
effectively compare strings for equality. I'm very confident that 
adding the conditional STORE itself would not cause me to change that 
assessment.

I have spotted two possible cases for the rare circumstance where a 
difference of one is noted between HIGHESTMODSEQ values which 
represent a potential gain, when also comparing UIDNEXT and EXISTS, 
but it's so rare I've never bothered implementing it, and I strongly 
suspect that, again, this is possible to handle with strings.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Cc: IMAP Extensions WG <[email-masked]>, [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Document store (was: Re: draft-newman-comparator updated)
Date: Thu, 2 Feb 2006 09:58:48 -0800
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

FYI, If anybody wants to post documents-in-progress to http:// 
ietf.webdav.org/imapext,  I can give out accounts and permissions.   
For the documents I edit, I use it to post the XML, keep a version  
history, and let people subscribe to notifications of changes.

lisa

On Feb 2, 2006, at 6:18 AM, Arnt Gulbrandsen wrote:

>
> Hi,
>
> the draft is lying silently in the internet-draft editor queue.  
> However, I would appreciate reviews, speedily if possible. If you  
> have the time and it isn't yet in your internet-drafts mirror, I  
> can send a copy.
>
> There are a few changes.
>
> The basic and biggest change is that collation is defined in terms  
> of octet strings, with explicit notes that implementations may  
> choose to use character strings, and notes that in that case, e.g.  
> i;octet may not be possible to implement. Various minor changes  
> changes result.
>
> There are also more definitions and requirements. For example,  
> equality has to be implemented if ordering is, and if ordering  
> returns equal, equality must return true. That sort of thing.
>
> I am unhappy with one thing: i;octet is still used a little as a  
> last resort to resolve equality/order, which brings it very close  
> to mandatory-to-implement. This doesn't seem necessary from an  
> architectural point of view. I just haven't found a solution I  
> really like.
>
> Arnt
>

Date: Thu, 2 Feb 2006 09:23:04 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Dave Cridland wrote:
> A user setting \Seen (possibly implicitly in a FETCH BODY[...]) causes the 
> MODSEQ to be bumped for that message. If the implementation is poor, then 
> this can cause the MODSEQ seen by other users to be changed, too. That's 
> undesirable, but as a client implementor, I infinitely prefer that level of 
> implementation to none at all.

But that's only if (1) the mailbox has per-user flags, (2) the mailbox has 
MODSEQ.  Right?

Personally, I never thought that per-user flags were worth the complexity, 
even though I was guilty of creating them.  But it was only as a grim 
necessity for bboards and newsgroups; and even then that was just a single 
flag.  The idea of using MODSEQ for such a thing staggers me.

> Otherwise, you're effectively lowering the width of the UID space, down to 
> 30, or 22, bits. That's still probably plenty of space, but there's something 
> philosophically wrong about an extension lowering the scalability of the 
> protocol it extends.

This argument only applies when you are monotonically allocating UIDs 
(otherwise it's your implementation that reduces the UID space, not 
MODSEQ).  And if you are monotonically allocating UIDs, then there is no 
reason not to monotonically allocate MODSEQ.  The notion of there being 
over 4 billion updates to one mailbox is simply not credible.

Now, if you have per-user and global flags in a mailbox that has MODSEQ, 
that may be a different matter.  But if you have that, that's a hole that 
you dug for yourself.  I see no reason to have such a thing; and if it was 
that important you should have defined two MODSEQs.

>> I disagree.  Either MODSEQ reports that something touched the message, or 
>> it reports that something changed the mesage.  As matters stand, it is 
>> ambiguous.  A client may care.
> Well, let me put it this way - a client which actually does care is broken. A 
> server which claims there to be a change when there is not is deficient.

[sarcasm] And the IMAP community never, ever, produces broken clients and 
deficient servers (or deficient clients and broken servers). [/sarcasm]

We should have learned the painful lesson by now.

>>> Well, I don't think it's much of a problem. Most of the computers I work 
>>> with these days increasingly use "unsigned long" for 64-bit values - even 
>>> my laptop is 64-bit, but the vast majority of the 32-bit systems I use 
>>> seem to support some kind of "unsigned long long" or equivalent.
>> That hasn't been my experience... :-(
> Get a better compiler? :-)

Sure, please write it.  And give it away for free, along with free 
technical support.

I have a difficult time tolerating the "my system has such-and-such, and 
it's your fault if your system does not have it too" attitude.  Either we 
care about interoperability and implementation costs, or we should forget 
this IETF nonsense and go home.

> I don't believe clients ever need to handle more than 3 MODSEQ values at any 
> time, and only need to store 2 permanently. If I needed to store each 
> message's MODSEQ, I'd probably feel differently, but I don't think that's 
> ever the case.

Are you certain?  IMHO, the client ability to fetch a message's MODSEQ 
directly is like placing a loaded pistol with a hair trigger in a room of 
preschoolers.  MODSEQ's ascending property makes it nice and shiny too.

Perhaps it would have been better to have created an alternative means to 
do those functionalities that doesn't leave temptation in the way.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 02 Feb 2006 16:52:36 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin wrote:

> On Thu, 2 Feb 2006, Alexey Melnikov wrote:
>
>>>> Clients, of course, have to deal with it as a 64-bit unsigned 
>>>> integer for all servers.
>>>
>>> That's a problem.  If you don't have a 64-bit integer type available 
>>> in then compiler, then you have to do the work by hand.  There seems 
>>> to me to be no good reason for this.
>>
>> Clients don't need to do any math on modsequences, they should be 
>> treated as opaque values.
>
> Are you sure?  It seems to me that the ascending property of modseqs 
> is something that clients may want to use, even though there's a 
> server operation for it.

No :-). One clever guy pointed out that a client might want to compare 
the cached modseq with the value returned in STATUS HIGHESTMODSEQ, for 
example.

Date: Thu, 2 Feb 2006 08:51:19 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked]
Subject: Re: collation: COMPARATOR/COLLATION?
Sender: [email-masked]

On Thu, 2 Feb 2006, Arnt Gulbrandsen wrote:
> A show of hands, please? I think we should go with whatever term suits most 
> IMAP people.

I agree with COMPARATOR.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 08:49:13 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Dave Cridland <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Alexey Melnikov wrote:
>>> Clients, of course, have to deal with it as a 64-bit unsigned integer for 
>>> all servers.
>> That's a problem.  If you don't have a 64-bit integer type available in 
>> then compiler, then you have to do the work by hand.  There seems to me to 
>> be no good reason for this.
> Clients don't need to do any math on modsequences, they should be treated as 
> opaque values.

Are you sure?  It seems to me that the ascending property of modseqs is 
something that clients may want to use, even though there's a server 
operation for it.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 08:44:41 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Alexey Melnikov wrote:
>> The examples imply MODSEQ as a date/time, but as the document says you 
>> can't actually do that.
> Because time() is not granular enough for the task (multiple flag changes can 
> happen in one second), and it may also go back when time on the computer 
> changes.

Yes, of course, I understand that.  But the use of times in the examples 
implies otherwise.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 15:18:10 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>, [email-masked]
Subject: draft-newman-comparator updated
Sender: [email-masked]

Hi,

the draft is lying silently in the internet-draft editor queue. However, 
I would appreciate reviews, speedily if possible. If you have the time 
and it isn't yet in your internet-drafts mirror, I can send a copy.

There are a few changes.

The basic and biggest change is that collation is defined in terms of 
octet strings, with explicit notes that implementations may choose to 
use character strings, and notes that in that case, e.g. i;octet may 
not be possible to implement. Various minor changes changes result.

There are also more definitions and requirements. For example, equality 
has to be implemented if ordering is, and if ordering returns equal, 
equality must return true. That sort of thing.

I am unhappy with one thing: i;octet is still used a little as a last 
resort to resolve equality/order, which brings it very close to 
mandatory-to-implement. This doesn't seem necessary from an 
architectural point of view. I just haven't found a solution I really 
like.

Arnt

Date: Thu, 2 Feb 2006 15:03:09 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: collation: COMPARATOR/COLLATION?
Sender: [email-masked]

Dave Cridland writes:
> On Thu Feb  2 13:50:19 2006, Arnt Gulbrandsen wrote:
>> draft-ietf-imapext-i18n has no substantive changes. But I am 
>> considering renaming the COMPARATOR extension to COLLATION to better 
>> suit the terminology in draft-newman-comparator. Opinions on that 
>> change? I know that sieve uses the term comparator.
>
> As does ACAP. I actually prefer the term comparator, and thoroughly 
> dislike the concept that they must collate.

A show of hands, please? I think we should go with whatever term suits 
most IMAP people.

Arnt

Date: Thu, 02 Feb 2006 13:49:50 +0000
Subject: Re: collation: COMPARATOR/COLLATION?
From: Dave Cridland <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 13:50:19 2006, Arnt Gulbrandsen wrote:
> a week and a few hours ago, I finished an update to the collation 
> drafts. Not sure why it hasn't been published yet. If anyone will 
> review before the i-d editor has done the necessary, I'll happily 
> send out copies via private email. (It's a little large for the 
> list, I suppose. 100k.)
> 
> 
Yes please.

> draft-ietf-imapext-i18n has no substantive changes. But I am 
> considering renaming the COMPARATOR extension to COLLATION to 
> better suit the terminology in draft-newman-comparator. Opinions on 
> that change? I know that sieve uses the term comparator.
> 
> 
As does ACAP. I actually prefer the term comparator, and thoroughly 
dislike the concept that they must collate.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Thu, 2 Feb 2006 14:50:19 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: collation: COMPARATOR/COLLATION?
Sender: [email-masked]

Hi,

a week and a few hours ago, I finished an update to the collation 
drafts. Not sure why it hasn't been published yet. If anyone will 
review before the i-d editor has done the necessary, I'll happily send 
out copies via private email. (It's a little large for the list, I 
suppose. 100k.)

draft-ietf-imapext-i18n has no substantive changes. But I am considering 
renaming the COMPARATOR extension to COLLATION to better suit the 
terminology in draft-newman-comparator. Opinions on that change? I know 
that sieve uses the term comparator.

draft-newman-comparator has changes. Review _needed_.

Arnt

Date: Thu, 02 Feb 2006 13:23:05 +0000
Subject: Re: ACL LISTRIGHTS
From: Dave Cridland <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 13:11:01 2006, Timo Sirainen wrote:
> But what I'm wondering is what does the "required rights" parameter 
> mean
> in LISTRIGHTS reply? Is it the same as what rights the user 
> currently
> has (ie. for "LISTRIGHTS INBOX <myusername>" it's the same as 
> "MYRIGHTS
> INBOX"?
> 
> 
No, it's the inalienable rights the user has. For instance, a server 
might require that a user always has full rights over their INBOX, or 
all their personal folders. It's impossible to remove those rights 
via an ACL command.

> Also it looks like the "optional rights" doesn't necessarily list 
> all
> the possible rights that the RFC defines. Is there a reason why 
> server
> wouldn't list all of them? Except possibly if it doesn't support the
> feature at all?
> 
> 
Exactly that - it may be impossible to grant certain users 
administrative rights, for instance.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Subject: ACL LISTRIGHTS
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Thu, 02 Feb 2006 15:11:01 +0200
Sender: [email-masked]

Now that I'm beginning to implement ACLs for my server, I finally got
around to reading rfc4314 (unfortunately I didn't really have time while
it was still in draft).

I noticed one bug in it:

   Example:    C: a005 listrights archive/imap anyone
               S: * LISTRIGHTS archive.imap anyone ""

But what I'm wondering is what does the "required rights" parameter mean
in LISTRIGHTS reply? Is it the same as what rights the user currently
has (ie. for "LISTRIGHTS INBOX <myusername>" it's the same as "MYRIGHTS
INBOX"?

Also it looks like the "optional rights" doesn't necessarily list all
the possible rights that the RFC defines. Is there a reason why server
wouldn't list all of them? Except possibly if it doesn't support the
feature at all?

Date: Thu, 02 Feb 2006 10:37:34 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Dave Cridland wrote:

> On Thu Feb  2 00:46:03 2006, Mark Crispin wrote:
>
>> On Thu, 2 Feb 2006, Dave Cridland wrote:
>>
>>> If you make UIDs 32-bit, then you have to allow for more space in 
>>> something expected to have more than one increment per UID. 
>>> Moreover, this gives enough space for poor implementations to show 
>>> MODSEQ changes from users' private flags. It's not ideal, but it's 
>>> better than having no CONDSTORE at all.
>>
>> I don't understand this statement at all.
>
> A user setting \Seen (possibly implicitly in a FETCH BODY[...]) causes 
> the MODSEQ to be bumped for that message. If the implementation is 
> poor, then this can cause the MODSEQ seen by other users to be 
> changed, too.

Indeed.

> That's undesirable, but as a client implementor, I infinitely prefer 
> that level of implementation to none at all.
>
>> Both UIDs and sequence numbers are also 32-bit.  I don't see what is 
>> so special about MODSEQ that it needs 9 orders of magnitude more 
>> value space. A typical message gets its flags changed fewer than 4 
>> times.
>
> Sure, we could make it 34-bit and probably be safe. Even if you take 
> into account the private flags situation, and give yourself 15,000 
> users, you could get away with 42 bits. I think 64-bit is simply the 
> next logical choice above 32-bit. It needs *more* space than 32-bit, 
> how much more is rather moot, since the next logical size up is 64-bit.

Exactly.

> Otherwise, you're effectively lowering the width of the UID space, 
> down to 30, or 22, bits. That's still probably plenty of space, but 
> there's something philosophically wrong about an extension lowering 
> the scalability of the protocol it extends.
>
>>> If you mean setting a flag where the flag is already set, then from 
>>> a practical standpoint it simply doesn't matter if the MODSEQ for 
>>> the message is bumped or not. It's much better if it isn't, but it 
>>> doesn't break anything if it is.
>>
>> I disagree.  Either MODSEQ reports that something touched the 
>> message, or it reports that something changed the mesage.  As matters 
>> stand, it is ambiguous.  A client may care.
>
> Well, let me put it this way - a client which actually does care is 
> broken. A server which claims there to be a change when there is not 
> is deficient.

I concur.

[...]

>>> Well, I don't think it's much of a problem. Most of the computers I 
>>> work with these days increasingly use "unsigned long" for 64-bit 
>>> values - even my laptop is 64-bit, but the vast majority of the 
>>> 32-bit systems I use seem to support some kind of "unsigned long 
>>> long" or equivalent.
>>
>> That hasn't been my experience... :-(
>
> Get a better compiler? :-)
>
> FWIW, I strongly suspect I could get away with using the textual 
> representation of the MODSEQ/HIGHESTMODSEQ throughout my 
> implementation. It'd waste 36 octets, which is a figure I can live 
> with. (Probably more likely to be 74 octets including string handling 
> overhead, but that's still well within my tolerance - the code 
> overhead of parsing and formatting them would be much higher).

Or one can use a struct with 2 32bit fields.

Date: Thu, 02 Feb 2006 10:37:22 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin wrote:

> The examples imply MODSEQ as a date/time, but as the document says you 
> can't actually do that.

Because time() is not granular enough for the task (multiple flag 
changes can happen in one second), and it may also go back when time on 
the computer changes.

Date: Thu, 02 Feb 2006 10:22:10 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: Dave Cridland <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin wrote:

>> Clients, of course, have to deal with it as a 64-bit unsigned integer 
>> for all servers.
>
> That's a problem.  If you don't have a 64-bit integer type available 
> in then compiler, then you have to do the work by hand.  There seems 
> to me to be no good reason for this.

Clients don't need to do any math on modsequences, they should be 
treated as opaque values.

And clients have an implementation option to treat modseqs as strings.

Date: Thu, 02 Feb 2006 10:19:04 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Dave Cridland wrote:

>> What's more, I didn't observe any definition in the CONDSTORE 
>> document of what is supposed to happen if a STORE results in no change.
>
> I don't think there's anything which states explicitly that if a STORE 
> results in no change then no MODSEQ increases happen, but I'd assume 
> that was implicit in the definition.

That was always the intent and this is how I've implemented this. If you 
think this is not clear, now might be a good time to add a clarifying 
sentence (CONDSTORE -08 is going to IESG shortly).

>> I am implementing a new mailbox format which has a per-message 
>> modification sequence.  For my purpose, it's easiest to have it be 
>> monotonically increasing.  Also, I don't want to change it if STORE 
>> results in no change.  I was going to make this available as a MODSEQ 
>> for a possible future CONDSTORE implementation, but I see no point in 
>> keeping it as a 64-bit value other than bloating the data structures 
>> further. I want to see if I am missing something.
>
> FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ value. 
> There's nothing *forcing* the value to be 64-bit for servers, but if 
> you overflowed a 32-bit, you'd have to reset the UIDVALIDITY in order 
> to recover.

That is correct.

> Clients, of course, have to deal with it as a 64-bit unsigned integer 
> for all servers.

Date: Thu, 02 Feb 2006 09:37:34 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 00:46:03 2006, Mark Crispin wrote:
> On Thu, 2 Feb 2006, Dave Cridland wrote:
>> If you make UIDs 32-bit, then you have to allow for more space in 
>> something expected to have more than one increment per UID. 
>> Moreover, this gives enough space for poor implementations to show 
>> MODSEQ changes from users' private flags. It's not ideal, but it's 
>> better than having no CONDSTORE at all.
> 
> I don't understand this statement at all.
> 
> 
A user setting \Seen (possibly implicitly in a FETCH BODY[...]) 
causes the MODSEQ to be bumped for that message. If the 
implementation is poor, then this can cause the MODSEQ seen by other 
users to be changed, too. That's undesirable, but as a client 
implementor, I infinitely prefer that level of implementation to none 
at all.

> Both UIDs and sequence numbers are also 32-bit.  I don't see what 
> is so special about MODSEQ that it needs 9 orders of magnitude more 
> value space. A typical message gets its flags changed fewer than 4 
> times.
> 
> 
Sure, we could make it 34-bit and probably be safe. Even if you take 
into account the private flags situation, and give yourself 15,000 
users, you could get away with 42 bits. I think 64-bit is simply the 
next logical choice above 32-bit. It needs *more* space than 32-bit, 
how much more is rather moot, since the next logical size up is 
64-bit.

Otherwise, you're effectively lowering the width of the UID space, 
down to 30, or 22, bits. That's still probably plenty of space, but 
there's something philosophically wrong about an extension lowering 
the scalability of the protocol it extends.

>> If you mean setting a flag where the flag is already set, then 
>> from a practical standpoint it simply doesn't matter if the MODSEQ 
>> for the message is bumped or not. It's much better if it isn't, 
>> but it doesn't break anything if it is.
> 
> I disagree.  Either MODSEQ reports that something touched the 
> message, or it reports that something changed the mesage.  As 
> matters stand, it is ambiguous.  A client may care.
> 
> 
Well, let me put it this way - a client which actually does care is 
broken. A server which claims there to be a change when there is not 
is deficient.

>> Not very likely, no. But equally, it's not very likely that you'd 
>> overflow UIDs, but that option is handled by the protocol. If it 
>> were an onerous thing, I'd worry about it, but it's really not.
> 
> But the same mechanism resets the MODSEQ as well.
> 
> 
Sure, but why make things worse?

>> Well, I don't think it's much of a problem. Most of the computers 
>> I work with these days increasingly use "unsigned long" for 64-bit 
>> values - even my laptop is 64-bit, but the vast majority of the 
>> 32-bit systems I use seem to support some kind of "unsigned long 
>> long" or equivalent.
> 
> That hasn't been my experience... :-(

Get a better compiler? :-)

FWIW, I strongly suspect I could get away with using the textual 
representation of the MODSEQ/HIGHESTMODSEQ throughout my 
implementation. It'd waste 36 octets, which is a figure I can live 
with. (Probably more likely to be 74 octets including string handling 
overhead, but that's still well within my tolerance - the code 
overhead of parsing and formatting them would be much higher).

I don't believe clients ever need to handle more than 3 MODSEQ values 
at any time, and only need to store 2 permanently. If I needed to 
store each message's MODSEQ, I'd probably feel differently, but I 
don't think that's ever the case.

Servers, of course, need to store at minimum one for every message 
plus one for the mailbox, but as I've said, they can use a 32-bit 
value if they prefer.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Wed, 1 Feb 2006 16:46:03 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Dave Cridland wrote:
> If you make UIDs 32-bit, then you have to allow for more space in something 
> expected to have more than one increment per UID. Moreover, this gives enough 
> space for poor implementations to show MODSEQ changes from users' private 
> flags. It's not ideal, but it's better than having no CONDSTORE at all.

I don't understand this statement at all.

Both UIDs and sequence numbers are also 32-bit.  I don't see what is so 
special about MODSEQ that it needs 9 orders of magnitude more value space. 
A typical message gets its flags changed fewer than 4 times.

> If you mean setting a flag where the flag is already set, then from a 
> practical standpoint it simply doesn't matter if the MODSEQ for the message 
> is bumped or not. It's much better if it isn't, but it doesn't break anything 
> if it is.

I disagree.  Either MODSEQ reports that something touched the message, or 
it reports that something changed the mesage.  As matters stand, it is 
ambiguous.  A client may care.

> Not very likely, no. But equally, it's not very likely that you'd overflow 
> UIDs, but that option is handled by the protocol. If it were an onerous 
> thing, I'd worry about it, but it's really not.

But the same mechanism resets the MODSEQ as well.

> Well, I don't think it's much of a problem. Most of the computers I work with 
> these days increasingly use "unsigned long" for 64-bit values - even my 
> laptop is 64-bit, but the vast majority of the 32-bit systems I use seem to 
> support some kind of "unsigned long long" or equivalent.

That hasn't been my experience... :-(

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 02 Feb 2006 00:30:42 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Wed Feb  1 23:50:22 2006, Mark Crispin wrote:
> On Wed, 1 Feb 2006, Dave Cridland wrote:
>> Two stores on different messages generate two different MODSEQ 
>> values, though. So if it were 32-bit, then just filling a mailbox 
>> would be enough to exhaust it. You actually only need 2^32-1 
>> STORES on a single message - the arrival of the single message 
>> itself increments the MODSEQ.
> 
> Yes, but how likely is a mailbox to have anything near the order of 
> magnitude of 2^32-1 messages (which would exhaust the UID space)?  
> Even given a ridiculous number of messages in the mailbox, there 
> should be tens of thousands of updates possible for each message 
> before a 32-bit MODSEQ space runs out.
> 
> 
If you make UIDs 32-bit, then you have to allow for more space in 
something expected to have more than one increment per UID. Moreover, 
this gives enough space for poor implementations to show MODSEQ 
changes from users' private flags. It's not ideal, but it's better 
than having no CONDSTORE at all.

>>> The examples imply MODSEQ as a date/time, but as the document 
>>> says you can't actually do that.
>> Indeed, but it's possible to come close. ACAP uses modtimes that 
>> can't actually be times, they just look remarkably like them.
> 
> I don't consider that to be a feature.  Some client author will 
> depend upon that characteristic, and it will come back to bite you.
> 
> 
Yes, it's probably not wise to exclusively use that method in the 
examples.

>> I don't think there's anything which states explicitly that if a 
>> STORE results in no change then no MODSEQ increases happen, but 
>> I'd assume that was implicit in the definition.
> 
> Well, that's a bug in the definition.
> 
> 
Actually, it depends on what you mean.

If you mean setting a flag where the flag is already set, then from a 
practical standpoint it simply doesn't matter if the MODSEQ for the 
message is bumped or not. It's much better if it isn't, but it 
doesn't break anything if it is.

If you mean a UID STORE that affects no messages, then there are no 
messages to bump the MODSEQ on - a server which bumped the 
HIGHESTMODSEQ is poor, but again, it doesn't break anything.

Given the definition of MODSEQ, I think it's pretty clear what should 
happen in both cases, though.

>> FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ 
>> value. There's nothing *forcing* the value to be 64-bit for 
>> servers, but if you overflowed a 32-bit, you'd have to reset the 
>> UIDVALIDITY in order to recover.
> 
> Not likely to happen, otherwise that option wouldn't exist.
> 
> 
Not very likely, no. But equally, it's not very likely that you'd 
overflow UIDs, but that option is handled by the protocol. If it were 
an onerous thing, I'd worry about it, but it's really not.

>> Clients, of course, have to deal with it as a 64-bit unsigned 
>> integer for all servers.
> 
> That's a problem.  If you don't have a 64-bit integer type 
> available in then compiler, then you have to do the work by hand.  
> There seems to me to be no good reason for this.
> 
> 
Well, I don't think it's much of a problem. Most of the computers I 
work with these days increasingly use "unsigned long" for 64-bit 
values - even my laptop is 64-bit, but the vast majority of the 
32-bit systems I use seem to support some kind of "unsigned long 
long" or equivalent.

Besides which, you only need to parse, format, and compare them, 
anyway. All of which I have to do quite soon, on a purely 32-bit 
platform, and I have to say I'm not particularly concerned about it, 
compared to all the other code I have to write.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Wed, 1 Feb 2006 15:50:22 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Wed, 1 Feb 2006, Dave Cridland wrote:
> Two stores on different messages generate two different MODSEQ values, 
> though. So if it were 32-bit, then just filling a mailbox would be enough to 
> exhaust it. You actually only need 2^32-1 STORES on a single message - the 
> arrival of the single message itself increments the MODSEQ.

Yes, but how likely is a mailbox to have anything near the order of 
magnitude of 2^32-1 messages (which would exhaust the UID space)?  Even 
given a ridiculous number of messages in the mailbox, there should be tens 
of thousands of updates possible for each message before a 32-bit MODSEQ 
space runs out.

>> The examples imply MODSEQ as a date/time, but as the document says you 
>> can't actually do that.
> Indeed, but it's possible to come close. ACAP uses modtimes that can't 
> actually be times, they just look remarkably like them.

I don't consider that to be a feature.  Some client author will depend 
upon that characteristic, and it will come back to bite you.

> I don't think there's anything which states explicitly that if a STORE 
> results in no change then no MODSEQ increases happen, but I'd assume that was 
> implicit in the definition.

Well, that's a bug in the definition.

> FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ value. There's 
> nothing *forcing* the value to be 64-bit for servers, but if you overflowed a 
> 32-bit, you'd have to reset the UIDVALIDITY in order to recover.

Not likely to happen, otherwise that option wouldn't exist.

> Clients, of course, have to deal with it as a 64-bit unsigned integer for all 
> servers.

That's a problem.  If you don't have a 64-bit integer type available in 
then compiler, then you have to do the work by hand.  There seems to me to 
be no good reason for this.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Wed, 01 Feb 2006 23:37:34 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Wed Feb  1 23:20:18 2006, Mark Crispin wrote:
> 
> I know that it's probably much too late to complain about this now, 
> but can someplace please explain why the MODSEQ is a 64-bit value?  
> Is there any serious possibility of there being more than 2^32 
> STOREs being performed on a single message in a mailbox?
> 
> 
Two stores on different messages generate two different MODSEQ 
values, though. So if it were 32-bit, then just filling a mailbox 
would be enough to exhaust it. You actually only need 2^32-1 STORES 
on a single message - the arrival of the single message itself 
increments the MODSEQ.

> The examples imply MODSEQ as a date/time, but as the document says 
> you can't actually do that.
> 
> 
Indeed, but it's possible to come close. ACAP uses modtimes that 
can't actually be times, they just look remarkably like them.

> What's more, I didn't observe any definition in the CONDSTORE 
> document of what is supposed to happen if a STORE results in no 
> change.
> 
> 
I don't think there's anything which states explicitly that if a 
STORE results in no change then no MODSEQ increases happen, but I'd 
assume that was implicit in the definition.

> I am implementing a new mailbox format which has a per-message 
> modification sequence.  For my purpose, it's easiest to have it be 
> monotonically increasing.  Also, I don't want to change it if STORE 
> results in no change.  I was going to make this available as a 
> MODSEQ for a possible future CONDSTORE implementation, but I see no 
> point in keeping it as a 64-bit value other than bloating the data 
> structures further. I want to see if I am missing something.
> 
> 
FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ value. 
There's nothing *forcing* the value to be 64-bit for servers, but if 
you overflowed a 32-bit, you'd have to reset the UIDVALIDITY in order 
to recover.

Clients, of course, have to deal with it as a 64-bit unsigned integer 
for all servers.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Wed, 1 Feb 2006 15:36:40 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Arnt Gulbrandsen wrote:
> Mark Crispin writes:
>> I know that it's probably much too late to complain about this now, but can 
>> someplace please explain why the MODSEQ is a 64-bit value?
> It's desirable for modseq to be comparable across multiple messages. For 
> example, 'search for messages with modseq higher than x' while updating an 
> offline client's cache.

And how does a 64-bit MODSEQ accomplish this in a way that a 32-bit MODSEQ 
does not?

The examples imply the usage of a date/time, but you can't actually use 
such; but there seems to be a nudge-nudge wink-wink implication that 
clients can assume that.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 00:40:07 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin writes:
> I know that it's probably much too late to complain about this now, 
> but can someplace please explain why the MODSEQ is a 64-bit value?

It's desirable for modseq to be comparable across multiple messages. For 
example, 'search for messages with modseq higher than x' while updating 
an offline client's cache.

Arnt

Date: Wed, 1 Feb 2006 15:20:18 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: MODSEQ in CONDSTORE
Sender: [email-masked]

I know that it's probably much too late to complain about this now, but 
can someplace please explain why the MODSEQ is a 64-bit value?  Is there 
any serious possibility of there being more than 2^32 STOREs being 
performed on a single message in a mailbox?

The examples imply MODSEQ as a date/time, but as the document says you 
can't actually do that.

What's more, I didn't observe any definition in the CONDSTORE document of 
what is supposed to happen if a STORE results in no change.

I am implementing a new mailbox format which has a per-message 
modification sequence.  For my purpose, it's easiest to have it be 
monotonically increasing.  Also, I don't want to change it if STORE 
results in no change.  I was going to make this available as a MODSEQ for 
a possible future CONDSTORE implementation, but I see no point in keeping 
it as a 64-bit value other than bloating the data structures further. 
I want to see if I am missing something.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Fri, 24 Feb 2006 18:11:56 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: collations: mismatch between equality and ordering
Sender: [email-masked]

Hi,

in the current collation draft, it is possible to sort strings that 
cannot be tested for equality: A collation has a domain. It can test 
any pair of strings for equality, provided both strings are within its 
domain. And it can order pair of strings, whether they are in its 
domain or not. I find that last bit highly dubious, and want to strike 
it.

I think that if a server wants to sort a set of strings and the chosen 
collation can't handle some, the server can fall back to i;octet for 
the rest. The collation itself does not need to define any behaviour 
outside its domain.

Striking it would also permit the number of dependences on i;octet to 
decrease. I believe I mentioned that a few weeks ago. I think that with 
this change, it should be possible to write a server which uses 
exclusively unicode internally and does not implement i;octet. (Not an 
IMAP server, of course. An IMAP server has to do i;octet.)

Comments? Pretty please?

In general news: Alexey Melnikov, Frank Ellermann and Mark Davis/the 
Unicode Technical Committee have gone over the draft. Long comments 
from all three. Good stuff. This issue is one I came across while 
pondering Mark's mail. I'll go on for another few hours today, 
hopefully I'll have a -07 before the end of the day.

Arnt

Date: Wed, 22 Feb 2006 19:51:20 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Timo Sirainen <[email-masked]>, [email-masked]
Subject: Re: ACL once more
Sender: [email-masked]

Dave Cridland wrote:

> On Tue Feb 21 14:23:13 2006, Timo Sirainen wrote:
>
>> On Tue, 2006-02-21 at 14:12 +0000, Dave Cridland wrote:
>> > On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
>> > > > > Is SETACL id -<all rights> the same as DELETEACL id?
>> > > > > > > > No. SETACL id -<all rights> means id is denied all 
>> rights in some > specific manner.
>>
>> Isn't that SETACL -id <all rights>?
>>
>> Minusing rights only removes the rights from the ACL, so I'd think once
>> all of them are removed, the whole ACL could be deleted since there's
>> nothing left.
>
> Ah. Hmmm. Quite so.
>
> Options:
>
> 1) It's equivalent then to SETACL id "", because it's removing any 
> rights from the ACL entry for the id. Which might mean the same as 
> "this identifier has no rights", or might mean "the rights of this 
> identifier are not affected by this line", which is basically 
> DELETEACL id, depending on whether "most specific" ACL entires are 
> used, or "union" applies.
>
> 2) According to the first para of Section 3.1, it might mean the same 
> as SETACL -id lrswipkxeta anyway, because the access control list as a 
> whole is changed such that the identifier does not have the rights 
> removed.

No, the text in section 3.1 is saying that "SETACL id -<rights>" only 
affects the entry for "id", not the whole ACL. So interpretation #1 is 
the correct one.

Alexey

Date: Tue, 21 Feb 2006 16:57:18 +0000
Subject: Re: ACL once more
From: Dave Cridland <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue Feb 21 14:23:13 2006, Timo Sirainen wrote:
> On Tue, 2006-02-21 at 14:12 +0000, Dave Cridland wrote:
> > On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
> > > > > Is SETACL id -<all rights> the same as DELETEACL id?
> > > > > > > > No. SETACL id -<all rights> means id is denied all 
> rights in some > specific manner.
> 
> Isn't that SETACL -id <all rights>?
> 
> Minusing rights only removes the rights from the ACL, so I'd think 
> once
> all of them are removed, the whole ACL could be deleted since 
> there's
> nothing left.
> 
> 
> 
Ah. Hmmm. Quite so.

Options:

1) It's equivalent then to SETACL id "", because it's removing any 
rights from the ACL entry for the id. Which might mean the same as 
"this identifier has no rights", or might mean "the rights of this 
identifier are not affected by this line", which is basically 
DELETEACL id, depending on whether "most specific" ACL entires are 
used, or "union" applies.

2) According to the first para of Section 3.1, it might mean the same 
as SETACL -id lrswipkxeta anyway, because the access control list as 
a whole is changed such that the identifier does not have the rights 
removed.

So, erm, I really don't know. :-)

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Subject: Re: ACL once more
From: Timo Sirainen <[email-masked]>
To: Dave Cridland <[email-masked]>
Cc: [email-masked]
Date: Tue, 21 Feb 2006 16:23:13 +0200
Sender: [email-masked]

On Tue, 2006-02-21 at 14:12 +0000, Dave Cridland wrote:
> On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
> > 
> > Is SETACL id -<all rights> the same as DELETEACL id?
> > 
> > 
> > 
> No. SETACL id -<all rights> means id is denied all rights in some 
> specific manner.

Isn't that SETACL -id <all rights>?

Minusing rights only removes the rights from the ACL, so I'd think once
all of them are removed, the whole ACL could be deleted since there's
nothing left.

Date: Tue, 21 Feb 2006 14:12:33 +0000
Subject: Re: ACL once more
From: Dave Cridland <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue Feb 21 11:07:53 2006, Timo Sirainen wrote:
> 
> Is SETACL id -<all rights> the same as DELETEACL id?
> 
> 
> 
No. SETACL id -<all rights> means id is denied all rights in some 
specific manner.

DELETEACL id means that id is not mentioned specifically, but may 
have rights due to being part of a group which is.

Mind you, SETACL id "" may, on some servers, have the same effect as 
DELETEACL id *or* SETACL id -<all rights>, depending on how the ACL 
entries result in a rights list.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Subject: ACL once more
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Tue, 21 Feb 2006 13:07:53 +0200
Sender: [email-masked]

Is SETACL id -<all rights> the same as DELETEACL id?

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-condstore-09.txt 
Date: Wed, 15 Feb 2006 10:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: IMAP Extension for Conditional STORE operation
	Author(s)	: A. Melnikov, S. Hole
	Filename	: draft-ietf-imapext-condstore-09.txt
	Pages		: 0
	Date		: 2006-2-13
	
Often, multiple IMAP (RFC 3501) clients need to coordinate changes to
a common IMAP mailbox. Examples include different clients working on
behalf of the same user, and multiple users accessing shared mailboxes.
These clients need a mechanism to synchronize state changes for messages within the mailbox. They must be able to guarantee that only one client can change message state (e.g., message flags) at any time.  An example of such an application is use of an IMAP mailbox as a message queue with multiple dequeueing clients.

The Conditional Store facility provides a protected update mechanism for message state information that can detect and resolve conflicts between multiple writing mail clients.

The Conditional Store facility also allows a client to quickly resynchronize mailbox flag changes.

This document defines an extension to IMAP (RFC 3501).

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-09.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-imapext-condstore-09.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-imapext-condstore-09.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-condstore-09.txt 
Date: Mon, 13 Feb 2006 15:50:02 -0500
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: IMAP Extension for Conditional STORE operation
	Author(s)	: A. Melnikov, S. Hole
	Filename	: draft-ietf-imapext-condstore-09.txt
	Pages		: 0
	Date		: 2006-2-13
	
Often, multiple IMAP (RFC 3501) clients need to coordinate changes to
a common IMAP mailbox. Examples include different clients working on
behalf of the same user, and multiple users accessing shared mailboxes.
These clients need a mechanism to synchronize state changes for messages within the mailbox. They must be able to guarantee that only one client can change message state (e.g., message flags) at any time.  An example of such an application is use of an IMAP mailbox as a message queue with multiple dequeueing clients.

The Conditional Store facility provides a protected update mechanism for message state information that can detect and resolve conflicts between multiple writing mail clients.

The Conditional Store facility also allows a client to quickly resynchronize mailbox flag changes.

This document defines an extension to IMAP (RFC 3501).

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-09.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-imapext-condstore-09.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-imapext-condstore-09.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-imapext-condstore-09.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Mon, 06 Feb 2006 12:11:19 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Lisa Dusseault <[email-masked]>, Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Dave Cridland wrote:

> On Thu Feb  2 22:32:50 2006, Lisa Dusseault wrote:
>
>> Dave said "the deficient server in this case causes no damage other 
>>  than a spurious round-trip from a client.", and Mark said something  
>> about guns and preschoolers :)
>
> Yeah. I'm talking specifically the case where a server increments a 
> MODSEQ for a message when the message is not visibly changed. Which is 
> the same as, for instance, a change to an ANNOTATion when the client 
> doesn't speak ANNOTATE - the client isn't damaged by it as such, and 
> has to expect it.

It is also the same as if another client set a flag and then unset it.

Date: Fri, 03 Feb 2006 09:02:00 +0000
Subject: Re: draft-newman-comparator updated
From: Dave Cridland <[email-masked]>
To: Martin Duerst <[email-masked]>
Cc: Arnt Gulbrandsen <[email-masked]>, <[email-masked]>, <[email-masked]>
Sender: [email-masked]

On Fri Feb  3 03:10:17 2006, Martin Duerst wrote:
> 
> At 23:18 06/02/02, Arnt Gulbrandsen wrote:
> >
> >Hi,
> >
> >the draft is lying silently in the internet-draft editor queue. 
> However, I would appreciate reviews, speedily if possible. If you 
> have the time and it isn't yet in your internet-drafts mirror, I 
> can send a copy.
> >
> >There are a few changes.
> >
> >The basic and biggest change is that collation is defined in terms 
> of octet strings, with explicit notes that implementations may 
> choose to use character strings, and notes that in that case, e.g. 
> i;octet may not be possible to implement. Various minor changes 
> changes result.
> 
> This may work out, or it may not work out. I'll check this 
> carefully,
> but don't have any time this (and probably next) week, sorry.
> 
> The main problem is that octet-based collations (except possibly for
> i;octet) need an encoding, but character-based collations obviously
> don't. So if I have some French collation defined over characters,
> and some French collation defined over iso-8859-1, how will they
> be distinguished? How can we make sure registrations can be
> as compact and easy as possible?

I thought quite a few of the interested parties had essentially 
agreed on this, but I could be wrong.

I thought there was a general feel of consensus for specifying that 
all comparators operated on octets, and that any comparators using 
character strings used the same encoding of UTF-8.

In principle, this means that given a character string, feeding it 
into "i;octet" means first encoding it into UTF-8. However, for 
collation, I believe that UTF-8 collation using "+i;octet" and 
unicode code-point order are the same thing, hence there's an obvious 
shortcut to be made.

Even more obvious shortcuts apply to "i;ascii-casemap", and 
"i;ascii-numeric", since both *can* operate on UTF-8 encoded octet 
strings, *or* unicode codepoints, and have the same effect very 
easily. They might say "ascii" in the name, but we can retrofit a 
"ASCII range" meaning to that without breaking semantics.

And that effect is the key - all comparators would be defined in 
terms of UTF-8, and whether or not the implementation could deal 
directly with some other representation of Unicode character strings, 
the overall effect must be identical.

So a French collation is only defined on characters in as much as it 
is defined in terms of the UTF-8 encoding of those characters. It's 
simply not allowed to deal with ISO-8859-15, nor ISO-8859-1, and 
"which character set" is removed from the comparator library 
altogether. This makes quite a lot of sense - you don't want to 
potentially have different collating order depending on the character 
set of the data, do you?

For real-world implementations, it's quite likely that a library will 
have entry points for bare octet strings, some native Unicode (like 
UCS-2 or whatever), and possibly octet string and encoding pairs - 
and that's absolutely fine, as long as the effect is identical to 
using UTF-8 encoded octet strings.

That doesn't mean that "i;octet" makes sense for all protocols, of 
course, and I cannot see the point of using is as a tie-break, when 
some existing protocols using comparators already provide a means for 
the user to select any tie break if required. This practise is 
useful, so feel free to suggest it in protocol support.

I'll put together some other suggestions for handling the various 
non-compatible changes between ACAP/SIEVE style comparators and these 
ones over the next few days.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Fri, 03 Feb 2006 12:10:17 +0900
To: Arnt Gulbrandsen <[email-masked]>, IMAP Extensions WG <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: draft-newman-comparator updated
Sender: [email-masked]

At 23:18 06/02/02, Arnt Gulbrandsen wrote:
 >
 >Hi,
 >
 >the draft is lying silently in the internet-draft editor queue. However, 
I would appreciate reviews, speedily if possible. If you have the time and 
it isn't yet in your internet-drafts mirror, I can send a copy.
 >
 >There are a few changes.
 >
 >The basic and biggest change is that collation is defined in terms of 
octet strings, with explicit notes that implementations may choose to use 
character strings, and notes that in that case, e.g. i;octet may not be 
possible to implement. Various minor changes changes result.

This may work out, or it may not work out. I'll check this carefully,
but don't have any time this (and probably next) week, sorry.

The main problem is that octet-based collations (except possibly for
i;octet) need an encoding, but character-based collations obviously
don't. So if I have some French collation defined over characters,
and some French collation defined over iso-8859-1, how will they
be distinguished? How can we make sure registrations can be
as compact and easy as possible?

Regards,    Martin.

 >There are also more definitions and requirements. For example, equality 
has to be implemented if ordering is, and if ordering returns equal, 
equality must return true. That sort of thing.
 >
 >I am unhappy with one thing: i;octet is still used a little as a last 
resort to resolve equality/order, which brings it very close to 
mandatory-to-implement. This doesn't seem necessary from an architectural 
point of view. I just haven't found a solution I really like.
 >
 >Arnt
 > 

Date: Thu, 02 Feb 2006 23:29:16 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Lisa Dusseault <[email-masked]>
Cc: Mark Crispin <[email-masked]>, <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 22:32:50 2006, Lisa Dusseault wrote:
> Dave said "the deficient server in this case causes no damage other 
>  than a spurious round-trip from a client.", and Mark said 
> something  about guns and preschoolers :)
> 
> 
Yeah. I'm talking specifically the case where a server increments a 
MODSEQ for a message when the message is not visibly changed. Which 
is the same as, for instance, a change to an ANNOTATion when the 
client doesn't speak ANNOTATE - the client isn't damaged by it as 
such, and has to expect it.

What would happen is that the client would see a changed 
HIGHESTMODSEQ, and accordingly issue a FETCH  ... CHANGEDSINCE, which 
would yield some response, which would provide data the client 
already had.

Where the HIGHESTMODSEQ is the same, the FETCH can be eliminated 
(and, optionally, even the SELECT).

Obviously the latter is more efficient and faster, and therefore a 
server which incremements the Modification sequence when no 
Modification has taken place is not as good as it could be.

Obviously the client will remain in sync without resorting to any 
kind of workaround - resync just happens to be more costly - so the 
server is not broken as such, and such behaviour could be the result 
of a modification the client is not capable enough to see.

> Were you two talking about different cases?  Would the outcome of 
> bad  client use of MODSEQ would be dire or simply the client's own 
> problem?

Mark's suggesting that, because all known uses of MODSEQ involve 
ignoring both its numeric nature and its status as a FETCH item, then 
it's potentially inviting doom by having MODSEQs have a numeric, 
ascending, per-message-accessible nature. This is a different case to 
the above. At least I think so.

I can't see this being the case - a client expecting some specific 
meaning to the difference of two MODSEQ numbers is probably asking 
for trouble (except in two vanishingly rare cases, where the 
difference is exactly 1), but for the most part, it won't cause any 
damage other than to itself, and potentially the user's data bill. 
Other clients I cannot see being affected by that, except 
beneficially.

You see, I could assume that an increment of 2 to HIGHESTMODSEQ 
implied there were two modifications, and having found two modified 
messages, I could decide that I'd found any modified messages, and 
consider myself resynchronized. Sometimes, I might even *be* 
resynchronized. Trouble is, any modification may encompass as many or 
few messages as possible, hence that's an invalid assumption, and the 
client may actually be out of sync. But, this doesn't effect another 
client. What can happen, however, is that the out of sync client can 
mark deleted, and subsequently expunge, the wrong message. But any 
out-of-sync client can do that perfectly well without CONDSTORE, and 
I don't think it makes the problem markedly worse.

You can cause wasteful round-trips for other clients by toggling a 
flag on and off across all messages, but I can't see this being done 
by accident much, and it merely degenerates CONDSTORE to uselessness, 
rather than actively breaking it.

I'm sure we'll eventually find a use for access to the MODSEQ of 
messages, probably something to do with archiving away old messages, 
or watching the changes to status of work items in a shared mailbox - 
I could probably come up with plenty of uses. I can't come up with 
much in the way of harm.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Thu, 2 Feb 2006 14:44:03 -0800
From: Mark Crispin <[email-masked]>
To: Lisa Dusseault <[email-masked]>
cc: Dave Cridland <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Thu, 2 Feb 2006, Lisa Dusseault wrote:
> Were you two talking about different cases?  Would the outcome of bad client 
> use of MODSEQ would be dire or simply the client's own problem?

That's the problem.  I don't know; and in such cases like this pessimism 
seems to be the more prudent course.

There's the harm in an implementation which uses empirical testing instead 
of the specification to define behavior.

There's the harm of making unwarranted assumptions, and decreeing that 
those who do not conform to those assumptions are "defective".  Consider 
those clients that assume that all names are "dual-use", all mailboxes 
support shared access, all servers support such-and-such extension, etc.

There's the harm of making a bizarre interpretation of the specification, 
claiming that interpretation is correct simply because it is not 
explicitly forbidden (never mind that it contracts something else), and 
then behaving badly against other implementations.

I've seen all of the above happen; and as a result I get nervous when 
there is wiggle-room in the specification or statements that it's 
"obvious" that it's such-and-such way.

I'm not saying that this applies here.  But has anyone made sure that it 
does not?

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Cc: Mark Crispin <[email-masked]>, <[email-masked]>
From: Lisa Dusseault <[email-masked]>
Subject: Re: MODSEQ in CONDSTORE
Date: Thu, 2 Feb 2006 14:32:50 -0800
To: Dave Cridland <[email-masked]>
Sender: [email-masked]

Dave said "the deficient server in this case causes no damage other  
than a spurious round-trip from a client.", and Mark said something  
about guns and preschoolers :)

Were you two talking about different cases?  Would the outcome of bad  
client use of MODSEQ would be dire or simply the client's own problem?

lisa

Date: Thu, 02 Feb 2006 22:01:37 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 17:23:04 2006, Mark Crispin wrote:
> On Thu, 2 Feb 2006, Dave Cridland wrote:
>> A user setting \Seen (possibly implicitly in a FETCH BODY[...]) 
>> causes the MODSEQ to be bumped for that message. If the 
>> implementation is poor, then this can cause the MODSEQ seen by 
>> other users to be changed, too. That's undesirable, but as a 
>> client implementor, I infinitely prefer that level of 
>> implementation to none at all.
> 
> But that's only if (1) the mailbox has per-user flags, (2) the 
> mailbox has MODSEQ.  Right?
> 
> 
Exactly, that's precisely the circumstance I'm describing.

> Personally, I never thought that per-user flags were worth the 
> complexity, even though I was guilty of creating them.  But it was 
> only as a grim necessity for bboards and newsgroups; and even then 
> that was just a single flag.  The idea of using MODSEQ for such a 
> thing staggers me.
> 
> 
It seems quite logical to me. I'm told it's a little tricky to get 
right/efficient with CONDSTORE, mind.

>> Otherwise, you're effectively lowering the width of the UID space, 
>> down to 30, or 22, bits. That's still probably plenty of space, 
>> but there's something philosophically wrong about an extension 
>> lowering the scalability of the protocol it extends.
> 
> This argument only applies when you are monotonically allocating 
> UIDs (otherwise it's your implementation that reduces the UID 
> space, not MODSEQ).  And if you are monotonically allocating UIDs, 
> then there is no reason not to monotonically allocate MODSEQ.  The 
> notion of there being over 4 billion updates to one mailbox is 
> simply not credible.
> 
> 
Probably not right now, but the notion that if you allow for 
4,294,967,295 messages in a mailbox in the protocol - and never mind 
that existing servers will collapse long before that - and the 
arrival of each counts as a modification, and there can be further 
modifications, therefore that the maximum number of modifications 
before an overflow event should be higher than 4,294,967,295 seems 
perfectly sensible to me, especially when no "real-world" server has 
to support more than 4,294,967,295, it's only the protocol.

>>> I disagree.  Either MODSEQ reports that something touched the 
>>> message, or it reports that something changed the mesage.  As 
>>> matters stand, it is ambiguous.  A client may care.
>> Well, let me put it this way - a client which actually does care 
>> is broken. A server which claims there to be a change when there 
>> is not is deficient.
> 
> [sarcasm] And the IMAP community never, ever, produces broken 
> clients and deficient servers (or deficient clients and broken 
> servers). [/sarcasm]
> 
> We should have learned the painful lesson by now.
> 
> 
Sure... But the deficient server in this case causes no damage other 
than a spurious round-trip from a client.

I don't see how a client could assume that a MODSEQ change to a 
message represented a visible change to the FLAGS, but even if it 
could, what good would it do? Surely in finding the MODSEQ, it could 
easily enough find the FLAGS at the same time? And if it did make 
such an assumption somehow, what would it then do which would affect 
other clients?

>>>> Well, I don't think it's much of a problem. Most of the 
>>>> computers I work with these days increasingly use "unsigned 
>>>> long" for 64-bit values - even my laptop is 64-bit, but the vast 
>>>> majority of the 32-bit systems I use seem to support some kind 
>>>> of "unsigned long long" or equivalent.
>>> That hasn't been my experience... :-(
>> Get a better compiler? :-)
> 
> Sure, please write it.  And give it away for free, along with free 
> technical support.
> 
> 
To be fair, there's not much point competing with the GCC community, 
who seem to be doing a marvellous job there.

> I have a difficult time tolerating the "my system has 
> such-and-such, and it's your fault if your system does not have it 
> too" attitude.  Either we care about interoperability and 
> implementation costs, or we should forget this IETF nonsense and go 
> home.
> 
> 
Mark, I obviously care about interoperability, as we're right here 
discussing it. I obviously care about implementation costs, because I 
provided an estimate. I'm also already at home, come to that.

I'm simply saying that many systems do, indeed, have 64-bit unsigned 
integer datatypes available, and that it appears to be an increasing 
trend, and I actually find it very hard to believe that there exists 
any current desktop system which does not have 64-bit support in the 
compiler and libraries.

There are indeed some systems that don't, such as Symbian for one, 
but in my considered opinion, this doesn't matter, because it only 
affects client implementations, and client implementations can cope 
perfectly well with using 3 strings, for only a slight overhead.

[Aside: Symbian's compiler is GCC, and does indeed have unsigned long 
long in at least some versions, but the descriptor library has no 
support for it, only limited support for a C++-implemented signed 
64-bit int.]

The implementation cost is therefore very slight indeed, unless the 
client stores every MODSEQ's value and/or performs arithmetic or 
comparisons upon them, and this is unlikely to be beneficial to the 
client, and it is certainly not required for the purposes of faster 
resynchronization, which is the obvious use-case for CONDSTORE on 
limited platforms anyway.

On those platforms where it's possible to build a usable server which 
could overflow a 32-bit MODSEQ, whether by an astonishing throughput 
of STORE commands or a truly impressive amount of exceedingly fast 
disk, I refuse to believe that a compiler with 64-bit support is 
going to be a problem to find.

>> I don't believe clients ever need to handle more than 3 MODSEQ 
>> values at any time, and only need to store 2 permanently. If I 
>> needed to store each message's MODSEQ, I'd probably feel 
>> differently, but I don't think that's ever the case.
> 
> Are you certain?  IMHO, the client ability to fetch a message's 
> MODSEQ directly is like placing a loaded pistol with a hair trigger 
> in a room of preschoolers.  MODSEQ's ascending property makes it 
> nice and shiny too.
> 
> 
Well, no, I'm not certain there are no cases where being able to 
treat the 64-bit MODSEQ as a 64-bit number is useful to a client, 
merely that I've not found one useful enough to implement yet. Nor 
have I found the ability to directly find MODSEQ values for messages 
useful, as yet. There's no harm in doing so, of course.

However, I've implemented the client side for the uses of fast 
resynchronization, and I use it exclusively and daily to read my 
mail, and it works perfectly well, dealing with only three MODSEQ 
values per mailbox. Nor can I see anywhere in my code where instead 
of using a true 64-bit integer and comparing, I couldn't just as 
effectively compare strings for equality. I'm very confident that 
adding the conditional STORE itself would not cause me to change that 
assessment.

I have spotted two possible cases for the rare circumstance where a 
difference of one is noted between HIGHESTMODSEQ values which 
represent a potential gain, when also comparing UIDNEXT and EXISTS, 
but it's so rare I've never bothered implementing it, and I strongly 
suspect that, again, this is possible to handle with strings.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Cc: IMAP Extensions WG <[email-masked]>, [email-masked]
From: Lisa Dusseault <[email-masked]>
Subject: Document store (was: Re: draft-newman-comparator updated)
Date: Thu, 2 Feb 2006 09:58:48 -0800
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

FYI, If anybody wants to post documents-in-progress to http:// 
ietf.webdav.org/imapext,  I can give out accounts and permissions.   
For the documents I edit, I use it to post the XML, keep a version  
history, and let people subscribe to notifications of changes.

lisa

On Feb 2, 2006, at 6:18 AM, Arnt Gulbrandsen wrote:

>
> Hi,
>
> the draft is lying silently in the internet-draft editor queue.  
> However, I would appreciate reviews, speedily if possible. If you  
> have the time and it isn't yet in your internet-drafts mirror, I  
> can send a copy.
>
> There are a few changes.
>
> The basic and biggest change is that collation is defined in terms  
> of octet strings, with explicit notes that implementations may  
> choose to use character strings, and notes that in that case, e.g.  
> i;octet may not be possible to implement. Various minor changes  
> changes result.
>
> There are also more definitions and requirements. For example,  
> equality has to be implemented if ordering is, and if ordering  
> returns equal, equality must return true. That sort of thing.
>
> I am unhappy with one thing: i;octet is still used a little as a  
> last resort to resolve equality/order, which brings it very close  
> to mandatory-to-implement. This doesn't seem necessary from an  
> architectural point of view. I just haven't found a solution I  
> really like.
>
> Arnt
>

Date: Thu, 2 Feb 2006 09:23:04 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Dave Cridland wrote:
> A user setting \Seen (possibly implicitly in a FETCH BODY[...]) causes the 
> MODSEQ to be bumped for that message. If the implementation is poor, then 
> this can cause the MODSEQ seen by other users to be changed, too. That's 
> undesirable, but as a client implementor, I infinitely prefer that level of 
> implementation to none at all.

But that's only if (1) the mailbox has per-user flags, (2) the mailbox has 
MODSEQ.  Right?

Personally, I never thought that per-user flags were worth the complexity, 
even though I was guilty of creating them.  But it was only as a grim 
necessity for bboards and newsgroups; and even then that was just a single 
flag.  The idea of using MODSEQ for such a thing staggers me.

> Otherwise, you're effectively lowering the width of the UID space, down to 
> 30, or 22, bits. That's still probably plenty of space, but there's something 
> philosophically wrong about an extension lowering the scalability of the 
> protocol it extends.

This argument only applies when you are monotonically allocating UIDs 
(otherwise it's your implementation that reduces the UID space, not 
MODSEQ).  And if you are monotonically allocating UIDs, then there is no 
reason not to monotonically allocate MODSEQ.  The notion of there being 
over 4 billion updates to one mailbox is simply not credible.

Now, if you have per-user and global flags in a mailbox that has MODSEQ, 
that may be a different matter.  But if you have that, that's a hole that 
you dug for yourself.  I see no reason to have such a thing; and if it was 
that important you should have defined two MODSEQs.

>> I disagree.  Either MODSEQ reports that something touched the message, or 
>> it reports that something changed the mesage.  As matters stand, it is 
>> ambiguous.  A client may care.
> Well, let me put it this way - a client which actually does care is broken. A 
> server which claims there to be a change when there is not is deficient.

[sarcasm] And the IMAP community never, ever, produces broken clients and 
deficient servers (or deficient clients and broken servers). [/sarcasm]

We should have learned the painful lesson by now.

>>> Well, I don't think it's much of a problem. Most of the computers I work 
>>> with these days increasingly use "unsigned long" for 64-bit values - even 
>>> my laptop is 64-bit, but the vast majority of the 32-bit systems I use 
>>> seem to support some kind of "unsigned long long" or equivalent.
>> That hasn't been my experience... :-(
> Get a better compiler? :-)

Sure, please write it.  And give it away for free, along with free 
technical support.

I have a difficult time tolerating the "my system has such-and-such, and 
it's your fault if your system does not have it too" attitude.  Either we 
care about interoperability and implementation costs, or we should forget 
this IETF nonsense and go home.

> I don't believe clients ever need to handle more than 3 MODSEQ values at any 
> time, and only need to store 2 permanently. If I needed to store each 
> message's MODSEQ, I'd probably feel differently, but I don't think that's 
> ever the case.

Are you certain?  IMHO, the client ability to fetch a message's MODSEQ 
directly is like placing a loaded pistol with a hair trigger in a room of 
preschoolers.  MODSEQ's ascending property makes it nice and shiny too.

Perhaps it would have been better to have created an alternative means to 
do those functionalities that doesn't leave temptation in the way.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 02 Feb 2006 16:52:36 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin wrote:

> On Thu, 2 Feb 2006, Alexey Melnikov wrote:
>
>>>> Clients, of course, have to deal with it as a 64-bit unsigned 
>>>> integer for all servers.
>>>
>>> That's a problem.  If you don't have a 64-bit integer type available 
>>> in then compiler, then you have to do the work by hand.  There seems 
>>> to me to be no good reason for this.
>>
>> Clients don't need to do any math on modsequences, they should be 
>> treated as opaque values.
>
> Are you sure?  It seems to me that the ascending property of modseqs 
> is something that clients may want to use, even though there's a 
> server operation for it.

No :-). One clever guy pointed out that a client might want to compare 
the cached modseq with the value returned in STATUS HIGHESTMODSEQ, for 
example.

Date: Thu, 2 Feb 2006 08:51:19 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked]
Subject: Re: collation: COMPARATOR/COLLATION?
Sender: [email-masked]

On Thu, 2 Feb 2006, Arnt Gulbrandsen wrote:
> A show of hands, please? I think we should go with whatever term suits most 
> IMAP people.

I agree with COMPARATOR.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 08:49:13 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Dave Cridland <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Alexey Melnikov wrote:
>>> Clients, of course, have to deal with it as a 64-bit unsigned integer for 
>>> all servers.
>> That's a problem.  If you don't have a 64-bit integer type available in 
>> then compiler, then you have to do the work by hand.  There seems to me to 
>> be no good reason for this.
> Clients don't need to do any math on modsequences, they should be treated as 
> opaque values.

Are you sure?  It seems to me that the ascending property of modseqs is 
something that clients may want to use, even though there's a server 
operation for it.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 08:44:41 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Alexey Melnikov wrote:
>> The examples imply MODSEQ as a date/time, but as the document says you 
>> can't actually do that.
> Because time() is not granular enough for the task (multiple flag changes can 
> happen in one second), and it may also go back when time on the computer 
> changes.

Yes, of course, I understand that.  But the use of times in the examples 
implies otherwise.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 15:18:10 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>, [email-masked]
Subject: draft-newman-comparator updated
Sender: [email-masked]

Hi,

the draft is lying silently in the internet-draft editor queue. However, 
I would appreciate reviews, speedily if possible. If you have the time 
and it isn't yet in your internet-drafts mirror, I can send a copy.

There are a few changes.

The basic and biggest change is that collation is defined in terms of 
octet strings, with explicit notes that implementations may choose to 
use character strings, and notes that in that case, e.g. i;octet may 
not be possible to implement. Various minor changes changes result.

There are also more definitions and requirements. For example, equality 
has to be implemented if ordering is, and if ordering returns equal, 
equality must return true. That sort of thing.

I am unhappy with one thing: i;octet is still used a little as a last 
resort to resolve equality/order, which brings it very close to 
mandatory-to-implement. This doesn't seem necessary from an 
architectural point of view. I just haven't found a solution I really 
like.

Arnt

Date: Thu, 2 Feb 2006 15:03:09 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: collation: COMPARATOR/COLLATION?
Sender: [email-masked]

Dave Cridland writes:
> On Thu Feb  2 13:50:19 2006, Arnt Gulbrandsen wrote:
>> draft-ietf-imapext-i18n has no substantive changes. But I am 
>> considering renaming the COMPARATOR extension to COLLATION to better 
>> suit the terminology in draft-newman-comparator. Opinions on that 
>> change? I know that sieve uses the term comparator.
>
> As does ACAP. I actually prefer the term comparator, and thoroughly 
> dislike the concept that they must collate.

A show of hands, please? I think we should go with whatever term suits 
most IMAP people.

Arnt

Date: Thu, 02 Feb 2006 13:49:50 +0000
Subject: Re: collation: COMPARATOR/COLLATION?
From: Dave Cridland <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 13:50:19 2006, Arnt Gulbrandsen wrote:
> a week and a few hours ago, I finished an update to the collation 
> drafts. Not sure why it hasn't been published yet. If anyone will 
> review before the i-d editor has done the necessary, I'll happily 
> send out copies via private email. (It's a little large for the 
> list, I suppose. 100k.)
> 
> 
Yes please.

> draft-ietf-imapext-i18n has no substantive changes. But I am 
> considering renaming the COMPARATOR extension to COLLATION to 
> better suit the terminology in draft-newman-comparator. Opinions on 
> that change? I know that sieve uses the term comparator.
> 
> 
As does ACAP. I actually prefer the term comparator, and thoroughly 
dislike the concept that they must collate.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Thu, 2 Feb 2006 14:50:19 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: collation: COMPARATOR/COLLATION?
Sender: [email-masked]

Hi,

a week and a few hours ago, I finished an update to the collation 
drafts. Not sure why it hasn't been published yet. If anyone will 
review before the i-d editor has done the necessary, I'll happily send 
out copies via private email. (It's a little large for the list, I 
suppose. 100k.)

draft-ietf-imapext-i18n has no substantive changes. But I am considering 
renaming the COMPARATOR extension to COLLATION to better suit the 
terminology in draft-newman-comparator. Opinions on that change? I know 
that sieve uses the term comparator.

draft-newman-comparator has changes. Review _needed_.

Arnt

Date: Thu, 02 Feb 2006 13:23:05 +0000
Subject: Re: ACL LISTRIGHTS
From: Dave Cridland <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 13:11:01 2006, Timo Sirainen wrote:
> But what I'm wondering is what does the "required rights" parameter 
> mean
> in LISTRIGHTS reply? Is it the same as what rights the user 
> currently
> has (ie. for "LISTRIGHTS INBOX <myusername>" it's the same as 
> "MYRIGHTS
> INBOX"?
> 
> 
No, it's the inalienable rights the user has. For instance, a server 
might require that a user always has full rights over their INBOX, or 
all their personal folders. It's impossible to remove those rights 
via an ACL command.

> Also it looks like the "optional rights" doesn't necessarily list 
> all
> the possible rights that the RFC defines. Is there a reason why 
> server
> wouldn't list all of them? Except possibly if it doesn't support the
> feature at all?
> 
> 
Exactly that - it may be impossible to grant certain users 
administrative rights, for instance.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Subject: ACL LISTRIGHTS
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Thu, 02 Feb 2006 15:11:01 +0200
Sender: [email-masked]

Now that I'm beginning to implement ACLs for my server, I finally got
around to reading rfc4314 (unfortunately I didn't really have time while
it was still in draft).

I noticed one bug in it:

   Example:    C: a005 listrights archive/imap anyone
               S: * LISTRIGHTS archive.imap anyone ""

But what I'm wondering is what does the "required rights" parameter mean
in LISTRIGHTS reply? Is it the same as what rights the user currently
has (ie. for "LISTRIGHTS INBOX <myusername>" it's the same as "MYRIGHTS
INBOX"?

Also it looks like the "optional rights" doesn't necessarily list all
the possible rights that the RFC defines. Is there a reason why server
wouldn't list all of them? Except possibly if it doesn't support the
feature at all?

Date: Thu, 02 Feb 2006 10:37:34 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Dave Cridland wrote:

> On Thu Feb  2 00:46:03 2006, Mark Crispin wrote:
>
>> On Thu, 2 Feb 2006, Dave Cridland wrote:
>>
>>> If you make UIDs 32-bit, then you have to allow for more space in 
>>> something expected to have more than one increment per UID. 
>>> Moreover, this gives enough space for poor implementations to show 
>>> MODSEQ changes from users' private flags. It's not ideal, but it's 
>>> better than having no CONDSTORE at all.
>>
>> I don't understand this statement at all.
>
> A user setting \Seen (possibly implicitly in a FETCH BODY[...]) causes 
> the MODSEQ to be bumped for that message. If the implementation is 
> poor, then this can cause the MODSEQ seen by other users to be 
> changed, too.

Indeed.

> That's undesirable, but as a client implementor, I infinitely prefer 
> that level of implementation to none at all.
>
>> Both UIDs and sequence numbers are also 32-bit.  I don't see what is 
>> so special about MODSEQ that it needs 9 orders of magnitude more 
>> value space. A typical message gets its flags changed fewer than 4 
>> times.
>
> Sure, we could make it 34-bit and probably be safe. Even if you take 
> into account the private flags situation, and give yourself 15,000 
> users, you could get away with 42 bits. I think 64-bit is simply the 
> next logical choice above 32-bit. It needs *more* space than 32-bit, 
> how much more is rather moot, since the next logical size up is 64-bit.

Exactly.

> Otherwise, you're effectively lowering the width of the UID space, 
> down to 30, or 22, bits. That's still probably plenty of space, but 
> there's something philosophically wrong about an extension lowering 
> the scalability of the protocol it extends.
>
>>> If you mean setting a flag where the flag is already set, then from 
>>> a practical standpoint it simply doesn't matter if the MODSEQ for 
>>> the message is bumped or not. It's much better if it isn't, but it 
>>> doesn't break anything if it is.
>>
>> I disagree.  Either MODSEQ reports that something touched the 
>> message, or it reports that something changed the mesage.  As matters 
>> stand, it is ambiguous.  A client may care.
>
> Well, let me put it this way - a client which actually does care is 
> broken. A server which claims there to be a change when there is not 
> is deficient.

I concur.

[...]

>>> Well, I don't think it's much of a problem. Most of the computers I 
>>> work with these days increasingly use "unsigned long" for 64-bit 
>>> values - even my laptop is 64-bit, but the vast majority of the 
>>> 32-bit systems I use seem to support some kind of "unsigned long 
>>> long" or equivalent.
>>
>> That hasn't been my experience... :-(
>
> Get a better compiler? :-)
>
> FWIW, I strongly suspect I could get away with using the textual 
> representation of the MODSEQ/HIGHESTMODSEQ throughout my 
> implementation. It'd waste 36 octets, which is a figure I can live 
> with. (Probably more likely to be 74 octets including string handling 
> overhead, but that's still well within my tolerance - the code 
> overhead of parsing and formatting them would be much higher).

Or one can use a struct with 2 32bit fields.

Date: Thu, 02 Feb 2006 10:37:22 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin wrote:

> The examples imply MODSEQ as a date/time, but as the document says you 
> can't actually do that.

Because time() is not granular enough for the task (multiple flag 
changes can happen in one second), and it may also go back when time on 
the computer changes.

Date: Thu, 02 Feb 2006 10:22:10 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: Dave Cridland <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin wrote:

>> Clients, of course, have to deal with it as a 64-bit unsigned integer 
>> for all servers.
>
> That's a problem.  If you don't have a 64-bit integer type available 
> in then compiler, then you have to do the work by hand.  There seems 
> to me to be no good reason for this.

Clients don't need to do any math on modsequences, they should be 
treated as opaque values.

And clients have an implementation option to treat modseqs as strings.

Date: Thu, 02 Feb 2006 10:19:04 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Dave Cridland wrote:

>> What's more, I didn't observe any definition in the CONDSTORE 
>> document of what is supposed to happen if a STORE results in no change.
>
> I don't think there's anything which states explicitly that if a STORE 
> results in no change then no MODSEQ increases happen, but I'd assume 
> that was implicit in the definition.

That was always the intent and this is how I've implemented this. If you 
think this is not clear, now might be a good time to add a clarifying 
sentence (CONDSTORE -08 is going to IESG shortly).

>> I am implementing a new mailbox format which has a per-message 
>> modification sequence.  For my purpose, it's easiest to have it be 
>> monotonically increasing.  Also, I don't want to change it if STORE 
>> results in no change.  I was going to make this available as a MODSEQ 
>> for a possible future CONDSTORE implementation, but I see no point in 
>> keeping it as a 64-bit value other than bloating the data structures 
>> further. I want to see if I am missing something.
>
> FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ value. 
> There's nothing *forcing* the value to be 64-bit for servers, but if 
> you overflowed a 32-bit, you'd have to reset the UIDVALIDITY in order 
> to recover.

That is correct.

> Clients, of course, have to deal with it as a 64-bit unsigned integer 
> for all servers.

Date: Thu, 02 Feb 2006 09:37:34 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Thu Feb  2 00:46:03 2006, Mark Crispin wrote:
> On Thu, 2 Feb 2006, Dave Cridland wrote:
>> If you make UIDs 32-bit, then you have to allow for more space in 
>> something expected to have more than one increment per UID. 
>> Moreover, this gives enough space for poor implementations to show 
>> MODSEQ changes from users' private flags. It's not ideal, but it's 
>> better than having no CONDSTORE at all.
> 
> I don't understand this statement at all.
> 
> 
A user setting \Seen (possibly implicitly in a FETCH BODY[...]) 
causes the MODSEQ to be bumped for that message. If the 
implementation is poor, then this can cause the MODSEQ seen by other 
users to be changed, too. That's undesirable, but as a client 
implementor, I infinitely prefer that level of implementation to none 
at all.

> Both UIDs and sequence numbers are also 32-bit.  I don't see what 
> is so special about MODSEQ that it needs 9 orders of magnitude more 
> value space. A typical message gets its flags changed fewer than 4 
> times.
> 
> 
Sure, we could make it 34-bit and probably be safe. Even if you take 
into account the private flags situation, and give yourself 15,000 
users, you could get away with 42 bits. I think 64-bit is simply the 
next logical choice above 32-bit. It needs *more* space than 32-bit, 
how much more is rather moot, since the next logical size up is 
64-bit.

Otherwise, you're effectively lowering the width of the UID space, 
down to 30, or 22, bits. That's still probably plenty of space, but 
there's something philosophically wrong about an extension lowering 
the scalability of the protocol it extends.

>> If you mean setting a flag where the flag is already set, then 
>> from a practical standpoint it simply doesn't matter if the MODSEQ 
>> for the message is bumped or not. It's much better if it isn't, 
>> but it doesn't break anything if it is.
> 
> I disagree.  Either MODSEQ reports that something touched the 
> message, or it reports that something changed the mesage.  As 
> matters stand, it is ambiguous.  A client may care.
> 
> 
Well, let me put it this way - a client which actually does care is 
broken. A server which claims there to be a change when there is not 
is deficient.

>> Not very likely, no. But equally, it's not very likely that you'd 
>> overflow UIDs, but that option is handled by the protocol. If it 
>> were an onerous thing, I'd worry about it, but it's really not.
> 
> But the same mechanism resets the MODSEQ as well.
> 
> 
Sure, but why make things worse?

>> Well, I don't think it's much of a problem. Most of the computers 
>> I work with these days increasingly use "unsigned long" for 64-bit 
>> values - even my laptop is 64-bit, but the vast majority of the 
>> 32-bit systems I use seem to support some kind of "unsigned long 
>> long" or equivalent.
> 
> That hasn't been my experience... :-(

Get a better compiler? :-)

FWIW, I strongly suspect I could get away with using the textual 
representation of the MODSEQ/HIGHESTMODSEQ throughout my 
implementation. It'd waste 36 octets, which is a figure I can live 
with. (Probably more likely to be 74 octets including string handling 
overhead, but that's still well within my tolerance - the code 
overhead of parsing and formatting them would be much higher).

I don't believe clients ever need to handle more than 3 MODSEQ values 
at any time, and only need to store 2 permanently. If I needed to 
store each message's MODSEQ, I'd probably feel differently, but I 
don't think that's ever the case.

Servers, of course, need to store at minimum one for every message 
plus one for the mailbox, but as I've said, they can use a 32-bit 
value if they prefer.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Wed, 1 Feb 2006 16:46:03 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Dave Cridland wrote:
> If you make UIDs 32-bit, then you have to allow for more space in something 
> expected to have more than one increment per UID. Moreover, this gives enough 
> space for poor implementations to show MODSEQ changes from users' private 
> flags. It's not ideal, but it's better than having no CONDSTORE at all.

I don't understand this statement at all.

Both UIDs and sequence numbers are also 32-bit.  I don't see what is so 
special about MODSEQ that it needs 9 orders of magnitude more value space. 
A typical message gets its flags changed fewer than 4 times.

> If you mean setting a flag where the flag is already set, then from a 
> practical standpoint it simply doesn't matter if the MODSEQ for the message 
> is bumped or not. It's much better if it isn't, but it doesn't break anything 
> if it is.

I disagree.  Either MODSEQ reports that something touched the message, or 
it reports that something changed the mesage.  As matters stand, it is 
ambiguous.  A client may care.

> Not very likely, no. But equally, it's not very likely that you'd overflow 
> UIDs, but that option is handled by the protocol. If it were an onerous 
> thing, I'd worry about it, but it's really not.

But the same mechanism resets the MODSEQ as well.

> Well, I don't think it's much of a problem. Most of the computers I work with 
> these days increasingly use "unsigned long" for 64-bit values - even my 
> laptop is 64-bit, but the vast majority of the 32-bit systems I use seem to 
> support some kind of "unsigned long long" or equivalent.

That hasn't been my experience... :-(

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 02 Feb 2006 00:30:42 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Wed Feb  1 23:50:22 2006, Mark Crispin wrote:
> On Wed, 1 Feb 2006, Dave Cridland wrote:
>> Two stores on different messages generate two different MODSEQ 
>> values, though. So if it were 32-bit, then just filling a mailbox 
>> would be enough to exhaust it. You actually only need 2^32-1 
>> STORES on a single message - the arrival of the single message 
>> itself increments the MODSEQ.
> 
> Yes, but how likely is a mailbox to have anything near the order of 
> magnitude of 2^32-1 messages (which would exhaust the UID space)?  
> Even given a ridiculous number of messages in the mailbox, there 
> should be tens of thousands of updates possible for each message 
> before a 32-bit MODSEQ space runs out.
> 
> 
If you make UIDs 32-bit, then you have to allow for more space in 
something expected to have more than one increment per UID. Moreover, 
this gives enough space for poor implementations to show MODSEQ 
changes from users' private flags. It's not ideal, but it's better 
than having no CONDSTORE at all.

>>> The examples imply MODSEQ as a date/time, but as the document 
>>> says you can't actually do that.
>> Indeed, but it's possible to come close. ACAP uses modtimes that 
>> can't actually be times, they just look remarkably like them.
> 
> I don't consider that to be a feature.  Some client author will 
> depend upon that characteristic, and it will come back to bite you.
> 
> 
Yes, it's probably not wise to exclusively use that method in the 
examples.

>> I don't think there's anything which states explicitly that if a 
>> STORE results in no change then no MODSEQ increases happen, but 
>> I'd assume that was implicit in the definition.
> 
> Well, that's a bug in the definition.
> 
> 
Actually, it depends on what you mean.

If you mean setting a flag where the flag is already set, then from a 
practical standpoint it simply doesn't matter if the MODSEQ for the 
message is bumped or not. It's much better if it isn't, but it 
doesn't break anything if it is.

If you mean a UID STORE that affects no messages, then there are no 
messages to bump the MODSEQ on - a server which bumped the 
HIGHESTMODSEQ is poor, but again, it doesn't break anything.

Given the definition of MODSEQ, I think it's pretty clear what should 
happen in both cases, though.

>> FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ 
>> value. There's nothing *forcing* the value to be 64-bit for 
>> servers, but if you overflowed a 32-bit, you'd have to reset the 
>> UIDVALIDITY in order to recover.
> 
> Not likely to happen, otherwise that option wouldn't exist.
> 
> 
Not very likely, no. But equally, it's not very likely that you'd 
overflow UIDs, but that option is handled by the protocol. If it were 
an onerous thing, I'd worry about it, but it's really not.

>> Clients, of course, have to deal with it as a 64-bit unsigned 
>> integer for all servers.
> 
> That's a problem.  If you don't have a 64-bit integer type 
> available in then compiler, then you have to do the work by hand.  
> There seems to me to be no good reason for this.
> 
> 
Well, I don't think it's much of a problem. Most of the computers I 
work with these days increasingly use "unsigned long" for 64-bit 
values - even my laptop is 64-bit, but the vast majority of the 
32-bit systems I use seem to support some kind of "unsigned long 
long" or equivalent.

Besides which, you only need to parse, format, and compare them, 
anyway. All of which I have to do quite soon, on a purely 32-bit 
platform, and I have to say I'm not particularly concerned about it, 
compared to all the other code I have to write.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Wed, 1 Feb 2006 15:50:22 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Wed, 1 Feb 2006, Dave Cridland wrote:
> Two stores on different messages generate two different MODSEQ values, 
> though. So if it were 32-bit, then just filling a mailbox would be enough to 
> exhaust it. You actually only need 2^32-1 STORES on a single message - the 
> arrival of the single message itself increments the MODSEQ.

Yes, but how likely is a mailbox to have anything near the order of 
magnitude of 2^32-1 messages (which would exhaust the UID space)?  Even 
given a ridiculous number of messages in the mailbox, there should be tens 
of thousands of updates possible for each message before a 32-bit MODSEQ 
space runs out.

>> The examples imply MODSEQ as a date/time, but as the document says you 
>> can't actually do that.
> Indeed, but it's possible to come close. ACAP uses modtimes that can't 
> actually be times, they just look remarkably like them.

I don't consider that to be a feature.  Some client author will depend 
upon that characteristic, and it will come back to bite you.

> I don't think there's anything which states explicitly that if a STORE 
> results in no change then no MODSEQ increases happen, but I'd assume that was 
> implicit in the definition.

Well, that's a bug in the definition.

> FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ value. There's 
> nothing *forcing* the value to be 64-bit for servers, but if you overflowed a 
> 32-bit, you'd have to reset the UIDVALIDITY in order to recover.

Not likely to happen, otherwise that option wouldn't exist.

> Clients, of course, have to deal with it as a 64-bit unsigned integer for all 
> servers.

That's a problem.  If you don't have a 64-bit integer type available in 
then compiler, then you have to do the work by hand.  There seems to me to 
be no good reason for this.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Wed, 01 Feb 2006 23:37:34 +0000
Subject: Re: MODSEQ in CONDSTORE
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Wed Feb  1 23:20:18 2006, Mark Crispin wrote:
> 
> I know that it's probably much too late to complain about this now, 
> but can someplace please explain why the MODSEQ is a 64-bit value?  
> Is there any serious possibility of there being more than 2^32 
> STOREs being performed on a single message in a mailbox?
> 
> 
Two stores on different messages generate two different MODSEQ 
values, though. So if it were 32-bit, then just filling a mailbox 
would be enough to exhaust it. You actually only need 2^32-1 STORES 
on a single message - the arrival of the single message itself 
increments the MODSEQ.

> The examples imply MODSEQ as a date/time, but as the document says 
> you can't actually do that.
> 
> 
Indeed, but it's possible to come close. ACAP uses modtimes that 
can't actually be times, they just look remarkably like them.

> What's more, I didn't observe any definition in the CONDSTORE 
> document of what is supposed to happen if a STORE results in no 
> change.
> 
> 
I don't think there's anything which states explicitly that if a 
STORE results in no change then no MODSEQ increases happen, but I'd 
assume that was implicit in the definition.

> I am implementing a new mailbox format which has a per-message 
> modification sequence.  For my purpose, it's easiest to have it be 
> monotonically increasing.  Also, I don't want to change it if STORE 
> results in no change.  I was going to make this available as a 
> MODSEQ for a possible future CONDSTORE implementation, but I see no 
> point in keeping it as a 64-bit value other than bloating the data 
> structures further. I want to see if I am missing something.
> 
> 
FWIW, Ken Murchison's patch for Cyrus can use a 32-bit MODSEQ value. 
There's nothing *forcing* the value to be 64-bit for servers, but if 
you overflowed a 32-bit, you'd have to reset the UIDVALIDITY in order 
to recover.

Clients, of course, have to deal with it as a 64-bit unsigned integer 
for all servers.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Wed, 1 Feb 2006 15:36:40 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

On Thu, 2 Feb 2006, Arnt Gulbrandsen wrote:
> Mark Crispin writes:
>> I know that it's probably much too late to complain about this now, but can 
>> someplace please explain why the MODSEQ is a 64-bit value?
> It's desirable for modseq to be comparable across multiple messages. For 
> example, 'search for messages with modseq higher than x' while updating an 
> offline client's cache.

And how does a 64-bit MODSEQ accomplish this in a way that a 32-bit MODSEQ 
does not?

The examples imply the usage of a date/time, but you can't actually use 
such; but there seems to be a nudge-nudge wink-wink implication that 
clients can assume that.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Thu, 2 Feb 2006 00:40:07 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: MODSEQ in CONDSTORE
Sender: [email-masked]

Mark Crispin writes:
> I know that it's probably much too late to complain about this now, 
> but can someplace please explain why the MODSEQ is a 64-bit value?

It's desirable for modseq to be comparable across multiple messages. For 
example, 'search for messages with modseq higher than x' while updating 
an offline client's cache.

Arnt

Date: Wed, 1 Feb 2006 15:20:18 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: MODSEQ in CONDSTORE
Sender: [email-masked]

I know that it's probably much too late to complain about this now, but 
can someplace please explain why the MODSEQ is a 64-bit value?  Is there 
any serious possibility of there being more than 2^32 STOREs being 
performed on a single message in a mailbox?

The examples imply MODSEQ as a date/time, but as the document says you 
can't actually do that.

What's more, I didn't observe any definition in the CONDSTORE document of 
what is supposed to happen if a STORE results in no change.

I am implementing a new mailbox format which has a per-message 
modification sequence.  For my purpose, it's easiest to have it be 
monotonically increasing.  Also, I don't want to change it if STORE 
results in no change.  I was going to make this available as a MODSEQ for 
a possible future CONDSTORE implementation, but I see no point in keeping 
it as a 64-bit value other than bloating the data structures further. 
I want to see if I am missing something.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.