Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
From: Barry Leiba <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Date: Tue, 25 Mar 2008 17:21:46 -0400
List-Id: <ietf-imapext.imc.org>

Cyrus Daboo said the following:
> So perhaps we need a slight variant here:
> 
> /public/comment
> /private/comment
> 
> With these two top-level entry segments, we have a clear distinction 
> between what is shared and what is private. Registered annotations would 
> have to define annotations in each as appropriate. Clients could create 
> unregistered entries in either to achieve the behavior they need. A server 
> that does not support per-user annotations simply does not support /private.

I'm a bit late with this comment, but: That variant works for me.

Barry

Date: Mon, 17 Mar 2008 22:59:22 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>What happened to this?
>
My editorial copy contains the following text:

      Use of system flags or user defined keywords with
      "UNCHANGEDSINCE 0" MUST be treated as an error; the server MUST
      return tagged BAD response to any such request. For all other
      metadata item types, use of UNCHANGEDSINCE with a modification
      sequence of 0 always fails if the metadata item exists.

>Your implementation seems to just ignore
>UNCHANGEDSINCE 0 instead of failing the update.
>
I will fix that.

Date: Mon, 17 Mar 2008 22:52:18 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>One more thing:
>
>   When a message is appended to a mailbox (via the IMAP APPEND command,
>   COPY to the mailbox, or using an external mechanism) the server
>   generates a new modification sequence that is higher than the highest
>   modification sequence of all messages in the mailbox and assigns it
>   to the appended message.
>
>This doesn't say if it's OK to give the same new modseq to new messages
>when copying/appending multiple messages. Since it's atomic I think it
>is.  
>
As it turned out, I've already made this change in my copy of rfc4551bis.

From: The IESG <[email-masked]>
To: IETF-Announce <[email-masked]>
Cc: Internet Architecture Board <[email-masked]>, RFC Editor <[email-masked]>, imapext mailing list <[email-masked]>, imapext chair <[email-masked]>
Subject: Protocol Action: 'Internet Message Access Protocol  Internationalization' to Proposed Standard 
Date: Mon, 17 Mar 2008 15:35:51 -0700 (PDT)
Sender: [email-masked]

The IESG has approved the following document:

- 'Internet Message Access Protocol Internationalization '
   <draft-ietf-imapext-i18n-15.txt> as a Proposed Standard

This document is the product of the Internet Message Access Protocol 
Extension Working Group. 

The IESG contact persons are Lisa Dusseault and Chris Newman.

A URL of this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-i18n-15.txt

Technical Summary
 
This document provides the internationalization building blocks for
several IMAP extensions. It specifies (1) how to return non-English,
non-US-ASCII human-readable text in replies to IMAP commands; (2) how to
provide translations for NAMESPACE prefixes; and  (3) how to use
internationalized comparators in the IMAP SEARCH, SORT, and THREAD
commands.

Working Group Summary

This document is a product of the IMAP Extensions (IMAPEXT) Working Group
and has been extensively reviewed. In addition to agreement on the
technical aspects of the document, the result of discussion was consensus
to separate the LANGUAGE extension (for human-readable text replies and
NAMESPACE prefixes) from the use of comparators and the separation of two
levels of comparator support.

Protocol Quality

The protocol is the final piece in the puzzle for the IMAP SEARCH, SORT,
and THREAD commands. Implementation for those has been ongoing, and the
result of this work has satisified implementors of those extensions.

 Lisa Dusseault reviewed this for the IESG.

Note to RFC Editor

In Section 2, add to the end of the last paragraph:

The I18NLEVEL=1 extension updates SEARCH/SORT/THREAD to use
i;unicode-casemap comparator, as defined in [UCM]. I18NLEVEL=1 is
a simpler version of I18NLEVEL=2 with no ability to select a different
collation.

In Section 3.1, add the following sentence to the end of 2nd paragraph:

IMAP servers SHOULD NOT advertise the LANGUAGE extension
if they discover that they only support "i-default".

In Section 3.1, last paragraph, second sentence:

OLD:
 Clients are urged to issue LANGUAGE before authentication, since some
servers send
         ^^^^^^^^^^^^
 valuable user information as part of authentication (e.g. "password
 is correct, but expired").

NEW:
 Clients SHOULD issue LANGUAGE before authentication, since some servers
send
         ^^^^^^
 valuable user information as part of authentication (e.g. "password
 is correct, but expired").

In Section 3.2, add a new sentence to the end of paragraph 9:

OLD:
 If there aren't any arguments, the server SHOULD send an untagged
 LANGUAGE response listing the languages it supports.  If the server
 is unable to enumerate the list of languages it supports it MAY
 return a tagged NO response to the enumeration request.

NEW:
 If there aren't any arguments, the server SHOULD send an untagged
 LANGUAGE response listing the languages it supports.  If the server
 is unable to enumerate the list of languages it supports it MAY
 return a tagged NO response to the enumeration request.
 If, after receiving a LANGUAGE request, the server discovers
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 that it doesn't support any language other than i-default,
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 it MUST return a tagged NO response to the enumeration request.
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Subject: Re: SEARCHRES
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: ietf-imapext <[email-masked]>
Date: Sun, 16 Mar 2008 18:17:07 +0200
Sender: [email-masked]

--=-Soei3ES6z+W1ny1+6WmQ
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Sun, 2008-03-16 at 14:54 +0000, Alexey Melnikov wrote:

> >>In some cases, the server MAY refuse to save a SEARCH (SAVE)
> >>        result, for example if an internal limit on the number of saved
> >>        results is reached.
> >>   =20
> >>
> >I don't see the point, it just complicates the protocol.
> >
> Some reviewers wanted to have a way to handle DoS on SEARCHRES gracefully=
.
>=20
> While this is allowed, it is up to server implementors to decide if the=20
> server should ever do that.

I just don't see how this can be considered a DoS in any way. If server
thinks there are too many SEARCHRES results stored, it shouldn't have
allowed the user to log in that many times. I can think of a lot easier
ways to DoS an IMAP compliant server than it. Wonder how much memory a
server would take parsing a 10MB mail with maximal number of MIME parts,
or something similar..

> Folks, the document is past AUTH48 already.

Why do I always read these too late.. :)

--=-Soei3ES6z+W1ny1+6WmQ
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-Soei3ES6z+W1ny1+6WmQ--

Date: Sun, 16 Mar 2008 15:22:18 +0000
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Timo Sirainen <[email-masked]>
Subject: Re: SEARCHRES
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> Alexey Melnikov writes:
>
>> It makes no much difference. If we clear the response, the client can 
>> issue "SEARCH NOT $" and produce the same result after that.
>
> Does this mean that
>
>    a login alexey yexela
>    b select inbox

At this point $ = {} (empty set).

>    c search not $

This is legal, even if $ is {}. So this will not return BAD.

> can cause anything other than
>
>    c BAD $ not defined
>
> ?
>
> (I thought I'd ignore this extension, but here I am, arguing about it.)

Date: Sun, 16 Mar 2008 16:18:46 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: SEARCHRES
Cc: Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> It makes no much difference. If we clear the response, the client can 
> issue "SEARCH NOT $" and produce the same result after that.

Does this mean that

    a login alexey yexela
    b select inbox
    c search not $

can cause anything other than

    c BAD $ not defined

?

(I thought I'd ignore this extension, but here I am, arguing about it.)

Arnt

Date: Sun, 16 Mar 2008 15:18:30 +0000
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Timo Sirainen <[email-masked]>
Subject: Re: SEARCHRES
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> Alexey Melnikov writes:
>
>> Timo Sirainen wrote:
>>
>>>>   Any of the following SEARCH command MUST NOT change the search 
>>>> result
>>>>   variable:
>>>>
>>>>    o A SEARCH command that caused the server to return the BAD tagged
>>>>      response,
>>>
>>> I think it's less dangerous to clear the variable.
>>
>> BAD response typically means that the server couldn't parse the 
>> command. A server implementation might not even know that it was a 
>> SEARCH command.
>
> Suggested language: When the server sends a BAD or NO response, it 
> MUST clear the search result variable.

Folks, the document is past AUTH48 already. This is not a kind of change 
that is worth it so late in the process.

Date: Sun, 16 Mar 2008 16:15:01 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: SEARCHRES
Cc: Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> Timo Sirainen wrote:
>>>   Any of the following SEARCH command MUST NOT change the search result
>>>   variable:
>>>
>>>    o A SEARCH command that caused the server to return the BAD tagged
>>>      response,
>>
>> I think it's less dangerous to clear the variable.
>
> BAD response typically means that the server couldn't parse the 
> command. A server implementation might not even know that it was a 
> SEARCH command.

Suggested language: When the server sends a BAD or NO response, it MUST 
clear the search result variable.

Arnt

Date: Sun, 16 Mar 2008 14:54:50 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: ietf-imapext <[email-masked]>
Subject: Re: SEARCHRES
Sender: [email-masked]

Timo Sirainen wrote:

>(wonder which of the IMAP lists would be the most appropriate for this..)
>  
>
>>   Any of the following SEARCH command MUST NOT change the search result
>>   variable:
>>
>>    o A SEARCH command that caused the server to return the BAD tagged
>>      response,
>>    
>>
>I think it's less dangerous to clear the variable.
>
BAD response typically means that the server couldn't parse the command. 
A server implementation might not even know that it was a SEARCH command.

>For example:
>
>1 search return (save) all
>2 fetch $ flags
>
>3 search return (save) subject hello world
>4 store $ +flags.silent \deleted
>5 expunge
>
>3 exposes a buggy client and returns BAD. 5 then expunges all messages.
>  
>
It makes no much difference. If we clear the response, the client can 
issue "SEARCH NOT $" and produce the same result after that.

>>In some cases, the server MAY refuse to save a SEARCH (SAVE)
>>        result, for example if an internal limit on the number of saved
>>        results is reached.
>>    
>>
>I don't see the point, it just complicates the protocol.
>
Some reviewers wanted to have a way to handle DoS on SEARCHRES gracefully.

While this is allowed, it is up to server implementors to decide if the 
server should ever do that.
Isode's server never do this.

>The result
>can't be larger than the number of messages in a mailbox. The server has
>to keep all kinds of data related to the messages already, so one search
>result shouldn't be that big of a deal. I can think of two usable ways
>to store the result, the worst case space usages would be:
>
>1. Array of { uint32_t uid1, uid2 }: 2 * 4 * message_count bytes = 78kB
>for 10000 message mailbox.
>
>2. Bitmask of sequences: message_count/8 bytes = 1,2 kB for 10000
>message mailbox.
>  
>

Subject: SEARCHRES
From: Timo Sirainen <[email-masked]>
To: ietf-imapext <[email-masked]>
Date: Sun, 16 Mar 2008 09:36:00 +0200
Sender: [email-masked]

--=-6M7b+vc4Yv4EYsR32Hv1
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

(wonder which of the IMAP lists would be the most appropriate for this..)

>    Any of the following SEARCH command MUST NOT change the search result
>    variable:
>=20
>     o A SEARCH command that caused the server to return the BAD tagged
>       response,

I think it's less dangerous to clear the variable. For example:

1 search return (save) all
2 fetch $ flags

3 search return (save) subject hello world
4 store $ +flags.silent \deleted
5 expunge

3 exposes a buggy client and returns BAD. 5 then expunges all messages.

> In some cases, the server MAY refuse to save a SEARCH (SAVE)
>         result, for example if an internal limit on the number of saved
>         results is reached.

I don't see the point, it just complicates the protocol. The result
can't be larger than the number of messages in a mailbox. The server has
to keep all kinds of data related to the messages already, so one search
result shouldn't be that big of a deal. I can think of two usable ways
to store the result, the worst case space usages would be:

1. Array of { uint32_t uid1, uid2 }: 2 * 4 * message_count bytes =3D 78kB
for 10000 message mailbox.

2. Bitmask of sequences: message_count/8 bytes =3D 1,2 kB for 10000
message mailbox.

--=-6M7b+vc4Yv4EYsR32Hv1
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-6M7b+vc4Yv4EYsR32Hv1--

Subject: Re: CONDSTORE + ESEARCH
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Sun, 16 Mar 2008 08:24:00 +0200
Sender: [email-masked]

--=-yvu7WnmY+AzFhbkbaqvq
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Sun, 2008-03-16 at 08:14 +0200, Timo Sirainen wrote:
> Don't seem to work well together?=20
>=20
> 1 SEARCH RETURN (ALL) MODSEQ 1
> * ESEARCH (tag "1") ALL 1:10
>=20
> MODSEQ <highest> is missing in the reply.

Sorry, it is defined there after all. It's just Alexey's implementation
that's not sending it. :)

--=-yvu7WnmY+AzFhbkbaqvq
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-yvu7WnmY+AzFhbkbaqvq--

Subject: CONDSTORE + ESEARCH
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Sun, 16 Mar 2008 08:14:42 +0200
Sender: [email-masked]

--=-r49VEd19tK4a15FeUhrb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Don't seem to work well together?=20

1 SEARCH RETURN (ALL) MODSEQ 1
* ESEARCH (tag "1") ALL 1:10

MODSEQ <highest> is missing in the reply.

--=-r49VEd19tK4a15FeUhrb
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-r49VEd19tK4a15FeUhrb--

Date: Sat, 15 Mar 2008 23:40:12 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Alexey,
>
> --On March 14, 2008 9:50:30 PM +0000 Alexey Melnikov 
> <[email-masked]> wrote:
>
>>> If you want searching, then write an extension to this extension that
>>> does use LISTEXT.
>>
>> But it is possible to search for everything else in IMAP (headers, text
>> in body parts, flags, etc.), so lack of search would be awkward.
>
> That maybe true at a message level, but its not true at a mailbox 
> level. i.e. without LISTEXT there is no way to search for mailboxes 
> containing unseen messages. Even with LISTEXT there is no way to 
> search for mailboxes on which I have the 'w' ACL right.

That is a bad example, as ACLs are not exposed in LISTEXT. Besides I 
don't see much point searching for access rights, they are different 
objects from human readable strings.

> So I don't agree that lack of search in this case is a hinderance. If 
> it is, then we extend METADATA with LISTEXT, but that won't be present 
> in the "base" METADATA spec.

I am not sure I will survive this change again :-).
Note that that searching for metadata doesn't necessarily need LISTEXT.

Date: Sat, 15 Mar 2008 13:04:41 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Hi Alexey,

--On March 14, 2008 9:50:30 PM +0000 Alexey Melnikov 
<[email-masked]> wrote:

>> If you want searching, then write an extension to this extension that
>> does use LISTEXT.
>
> But it is possible to search for everything else in IMAP (headers, text
> in body parts, flags, etc.), so lack of search would be awkward.

That maybe true at a message level, but its not true at a mailbox level. 
i.e. without LISTEXT there is no way to search for mailboxes containing 
unseen messages. Even with LISTEXT there is no way to search for mailboxes 
on which I have the 'w' ACL right. So I don't agree that lack of search in 
this case is a hinderance. If it is, then we extend METADATA with LISTEXT, 
but that won't be present in the "base" METADATA spec.

-- 
Cyrus Daboo

Date: Sat, 15 Mar 2008 16:24:49 +0000
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> Let me propose some RFC editor notes based on discussions in this 
> thread and some other comments from IESG:

As there were no objections to my RFC editor notes, I will recommend 
them to Lisa, so that we can get the document published. Thank you everyone.

Date: Sat, 15 Mar 2008 13:51:13 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>On Thu, 2008-03-13 at 18:31 +0200, Timo Sirainen wrote:
>  
>
>>On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>>    
>>
>>>Timo Sirainen wrote:
>>>      
>>>
>>>>What happened to this?
>>>>        
>>>>
>>>I need to publish CONDSTOREbis draft at some point which contains  
>>>the fix. I don't have time or energy to do this now. I should find  
>>>some time once all my Lemonade drafts are approved for publication.
>>>      
>>>
>>Are there going to be other changes?
>>    
>>
>One more thing:
>
>   When a message is appended to a mailbox (via the IMAP APPEND command,
>   COPY to the mailbox, or using an external mechanism) the server
>   generates a new modification sequence that is higher than the highest
>   modification sequence of all messages in the mailbox and assigns it
>   to the appended message.
>
>This doesn't say if it's OK to give the same new modseq to new messages
>when copying/appending multiple messages. Since it's atomic I think it
>is.
>  
>
Yes, it is Ok to assign the same modseq to multiple messages 
injectec/appended atomically. This was the intent.

Date: Sat, 15 Mar 2008 06:30:17 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>, Cyrus Daboo <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

> > If you want searching, then write an extension to this extension 
> > that does use LISTEXT.
> 
> But it is possible to search for everything else in IMAP (headers,
> text in body parts, flags, etc.), so lack of search would be awkward.

I see the point, although I honestly don't see the use case.

But even if you do introduce search functionality (and the LISTEXT
stuff from the previous draft went way beyond the simple "find this text
string" searching available on all those other pieces of data), you'd
still have to make awkward exceptions for non-text data.

(And, before we risk going there, I'd also like to avoid requiring
I18NLEVEL=2.)

Date: Sat, 15 Mar 2008 02:01:16 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

> On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>
>> Timo Sirainen wrote:
>>
>>> What happened to this?
>>
>> I need to publish CONDSTOREbis draft at some point which contains  
>> the fix. I don't have time or energy to do this now. I should find  
>> some time once all my Lemonade drafts are approved for publication.
>
> Are there going to be other changes? The main problem I currently 
> have  implementing CONDSTORE is that I'd rather not want to figure 
> out  HIGHESTMODSEQ value when selecting a mailbox unless I know the 
> client  wants to use CONDSTORE. I started to wonder if it would work 
> nicely if  a SELECT mailbox returned [NOMODSEQ] but the first 
> CONDSTORE enabling  command would cause * OK [HIGHESTMODSEQ n] to be 
> sent.

That should be Ok, especially if clients try to use "ENABLE QRESYNC" or 
"ENABLE CONDSTORE".

However I don't think a client that sees NOMODSEQ is going to issue any 
CONDSTORE enabling command.

> Which also brings to my mind.. I didn't really understand why the * 
> OK  [HIGHESTMODSEQ n] would be sent again after it was already sent as 
> a  reply to SELECT. This is described in the spec example 5 and also  
> happens in your implementation.

Because HIGHESTMODSEQ might have changed between the moment when the 
mailbox was opened and the moment when the client issued a CONDSTORE 
enabling command.

Date: Fri, 14 Mar 2008 21:45:36 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

>Hi,
>So I have been thinking some more about the issue of removing attributes 
>from METADATA. I think that will introduce some problems with how we can 
>then do .priv and .shared. My original idea was to allow ".priv" and 
>".shared" to be tacked on the end of an entry name to have the appropriate 
>value returned. This is ugly in that it then requires the somewhat 
>arbitrary restriction that entry names themselves cannot end in .priv or 
>.shared.
>
>So here is a more radical proposal: do away with .priv and .shared as well. 
>Instead each annotation would define whether it is per-user or shared. So 
>on a mailbox we would initially define:
>
>/comment-public
>/comment-private
>
>Where "public" would be "shared" and "private" would be per-user.
>  
>
I really really really dislike that, as now there is no way to 
algorithmically determine which annotation is public and which one is 
private. Unless you meant that each annotation will have either 
"-public" or "-private"?
A better way would be to define two separate hierarchies: /public and 
/private, e.g. /private/comment.

Date: Sat, 15 Mar 2008 01:29:52 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: Randall Gellens <[email-masked]>, Dan Karp <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Removing attributes, was Re: Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

>Hi Randall,
>
>--On March 14, 2008 7:17:47 AM -0700 Randall Gellens <[email-masked]> 
>wrote:
>
>  
>
>>We were trying to maintain some level of similarity with ANNOTATE, and
>>also provide the client with a way to see the size before fetching
>>something.
>>    
>>
>Well ANNOTATE is only experimental and I'm not sure anyone (except perhaps 
>Ken @ CMU) has done anything with it. So limiting ourselves to ANNOTATE 
>syntax should not be a requirement IMHO.
>
>As far as the size goes, we could easily add an option to GETMETADATA that 
>would allow a client to limit the size of data returned and add an option 
>to the response to indicate when a truncation has occurred, e.g.:
>
>           C: a GETMETADATA (MAXSIZE 1024) /comment
>           S: * METADATA (TRUNCATED 2048) /comment "My comment ...."
>           S: a OK GETMETADATA complete
>  
>
I can live with that, as long as this is a part of 
METADATA/METADATA-SERVER and not yet another extension.

>So in this example the client asks the server to return no more than 1024 
>octets. The /comment value returned is truncated at 1024 octets, and the 
>server also tells the client that the actual size of /comment is 2048.
>
>I think this is more efficient if you make the assumption that for the most 
>part annotations are going to be small. i.e. the client doesn't have to do 
>two roundtrips to get an annotation (one to check the size, the other to 
>get the data).
>
That is a good point.

>Instead it just does one with MAXSIZE and then, only if 
>there was truncation, can do another get if it really want to.
>  
>

Date: Fri, 14 Mar 2008 21:50:30 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Alexey,
>
> --On March 13, 2008 10:03:44 PM +0000 Alexey Melnikov 
> <[email-masked]> wrote:
>
>>> 1) Remove the dependency on LISTEXT. This basically means turning the
>>> clock  back to the original ANNOTATEMORE extension. So the GETMETADATA
>>> command  would be used to retrieve mailbox annotations as well as 
>>> server
>>> annotations.
>>
>> That is fine with me.
>>
>> You forgot handling on comparators in GETMETADATA.
>
> Actually I am not proposing that GETMETADATA provide a way to search 
> annotations (which is what LISTEXT does).

But LISTEXT is not relevant to per-server annotations (i.e. 
METADATA-SERVER).

> If you want searching, then write an extension to this extension that 
> does use LISTEXT.

But it is possible to search for everything else in IMAP (headers, text 
in body parts, flags, etc.), so lack of search would be awkward.

Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Sat, 15 Mar 2008 05:54:28 +0200
Sender: [email-masked]

--=-8S8FqGkVyj9chmQv2Bku
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Thu, 2008-03-13 at 18:31 +0200, Timo Sirainen wrote:
> On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>=20
> > Timo Sirainen wrote:
> >
> >> What happened to this?
> >>
> > I need to publish CONDSTOREbis draft at some point which contains =20
> > the fix. I don't have time or energy to do this now. I should find =20
> > some time once all my Lemonade drafts are approved for publication.
>=20
> Are there going to be other changes?

One more thing:

   When a message is appended to a mailbox (via the IMAP APPEND command,
   COPY to the mailbox, or using an external mechanism) the server
   generates a new modification sequence that is higher than the highest
   modification sequence of all messages in the mailbox and assigns it
   to the appended message.

This doesn't say if it's OK to give the same new modseq to new messages
when copying/appending multiple messages. Since it's atomic I think it
is.

--=-8S8FqGkVyj9chmQv2Bku
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-8S8FqGkVyj9chmQv2Bku--

Date: Fri, 14 Mar 2008 13:26:25 -0400
From: Cyrus Daboo <[email-masked]>
To: Dan Karp <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

Hi Dan,

--On March 14, 2008 8:12:34 AM -0700 Dan Karp <[email-masked]> wrote:

>>   /comment-public
>>   /comment-private
>>
>> Where "public" would be "shared" and "private" would be per-user.
>
> I really like this, as there are very, very few entries that are
> meaningful as both private and public.
>
> How does this affect the statement from section 3.3 of the old document:
>
>    If the METADATA extension is present, support for shared annotations
>    is REQUIRED, whilst support for private annotations is OPTIONAL.
>

If server vendors still feel they need to have the option to deny per-user 
annotations, then we need to add some suitable response codes that would 
indicate lack of support for a registered per-user annotation.

Another thing: with this new approach there is no way for a client to 
create arbitrary (unregistered) entry names on a shared or per-user basis, 
given that the server determines what is shared and per-user based on 
registrations.

So perhaps we need a slight variant here:

/public/comment
/private/comment

With these two top-level entry segments, we have a clear distinction 
between what is shared and what is private. Registered annotations would 
have to define annotations in each as appropriate. Clients could create 
unregistered entries in either to achieve the behavior they need. A server 
that does not support per-user annotations simply does not support /private.

-- 
Cyrus Daboo

Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
From: Curtis King <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Date: Fri, 14 Mar 2008 10:44:15 -0600
Sender: [email-masked]

On 14-Mar-08, at 9:05 AM, Cyrus Daboo wrote:

> Hi,
> So I have been thinking some more about the issue of removing  
> attributes
> from METADATA. I think that will introduce some problems with how we  
> can
> then do .priv and .shared. My original idea was to allow ".priv" and
> ".shared" to be tacked on the end of an entry name to have the  
> appropriate
> value returned. This is ugly in that it then requires the somewhat
> arbitrary restriction that entry names themselves cannot end  
> in .priv or
> .shared.
>
> So here is a more radical proposal: do away with .priv and .shared  
> as well.
> Instead each annotation would define whether it is per-user or  
> shared. So
> on a mailbox we would initially define:
>
> /comment-public
> /comment-private
>
> Where "public" would be "shared" and "private" would be per-user.

I really like this idea.

ck

Date: Fri, 14 Mar 2008 08:12:34 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

> So here is a more radical proposal: do away with .priv and .shared as
> well.  Instead each annotation would define whether it is per-user or
> shared.  So on a mailbox we would initially define:
> 
>   /comment-public
>   /comment-private
> 
> Where "public" would be "shared" and "private" would be per-user.

I really like this, as there are very, very few entries that are
meaningful as both private and public.

How does this affect the statement from section 3.3 of the old document:

   If the METADATA extension is present, support for shared annotations
   is REQUIRED, whilst support for private annotations is OPTIONAL.

Date: Fri, 14 Mar 2008 11:05:19 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Even more drastic cut-backs in METADATA
Sender: [email-masked]

Hi,
So I have been thinking some more about the issue of removing attributes 
from METADATA. I think that will introduce some problems with how we can 
then do .priv and .shared. My original idea was to allow ".priv" and 
".shared" to be tacked on the end of an entry name to have the appropriate 
value returned. This is ugly in that it then requires the somewhat 
arbitrary restriction that entry names themselves cannot end in .priv or 
.shared.

So here is a more radical proposal: do away with .priv and .shared as well. 
Instead each annotation would define whether it is per-user or shared. So 
on a mailbox we would initially define:

/comment-public
/comment-private

Where "public" would be "shared" and "private" would be per-user.

-- 
Cyrus Daboo

Date: Fri, 14 Mar 2008 10:57:32 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Hi Alexey,

--On March 13, 2008 10:03:44 PM +0000 Alexey Melnikov 
<[email-masked]> wrote:

>> 1) Remove the dependency on LISTEXT. This basically means turning the
>> clock  back to the original ANNOTATEMORE extension. So the GETMETADATA
>> command  would be used to retrieve mailbox annotations as well as server
>> annotations.
>>
>>
> That is fine with me.
>
> You forgot handling on comparators in GETMETADATA.

Actually I am not proposing that GETMETADATA provide a way to search 
annotations (which is what LISTEXT does). If you want searching, then write 
an extension to this extension that does use LISTEXT.

-- 
Cyrus Daboo

Date: Fri, 14 Mar 2008 10:54:41 -0400
From: Cyrus Daboo <[email-masked]>
To: Randall Gellens <[email-masked]>, Dan Karp <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Removing attributes, was Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Hi Randall,

--On March 14, 2008 7:17:47 AM -0700 Randall Gellens <[email-masked]> 
wrote:

> We were trying to maintain some level of similarity with ANNOTATE, and
> also provide the client with a way to see the size before fetching
> something.

Well ANNOTATE is only experimental and I'm not sure anyone (except perhaps 
Ken @ CMU) has done anything with it. So limiting ourselves to ANNOTATE 
syntax should not be a requirement IMHO.

As far as the size goes, we could easily add an option to GETMETADATA that 
would allow a client to limit the size of data returned and add an option 
to the response to indicate when a truncation has occurred, e.g.:

           C: a GETMETADATA (MAXSIZE 1024) /comment
           S: * METADATA (TRUNCATED 2048) /comment "My comment ...."
           S: a OK GETMETADATA complete

So in this example the client asks the server to return no more than 1024 
octets. The /comment value returned is truncated at 1024 octets, and the 
server also tells the client that the actual size of /comment is 2048.

I think this is more efficient if you make the assumption that for the most 
part annotations are going to be small. i.e. the client doesn't have to do 
two roundtrips to get an annotation (one to check the size, the other to 
get the data). Instead it just does one with MAXSIZE and then, only if 
there was truncation, can do another get if it really want to.

-- 
Cyrus Daboo

Date: Fri, 14 Mar 2008 07:47:28 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Randall Gellens <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>, Randall Gellens <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

> I'm not saying these are must-have elements, but I'm not sure taking 
> them out will make much difference in ease of implementation.  Are 
> you saying it will?  Is this because of parsing responses, or 
> something else?

Attributes make the protocol a moderate amount more complex, with
relatively little benefit back to the client.  (They also give a false
sense of extensibility.)

Date: Fri, 14 Mar 2008 15:37:05 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: [lemonade] Taking the knife to METADATA
Cc: Randall Gellens <[email-masked]>, Dan Karp <[email-masked]>, Cyrus Daboo <[email-masked]>
Sender: [email-masked]

Randall Gellens writes:
> I'm not saying these are must-have elements, but I'm not sure taking 
> them out will make much difference in ease of implementation.  Are 
> you saying it will?

Matching size was difficult, the rest was easy. And AFAICT 
searching/matching doesn't apply to metadata, only to annotate.

Arnt

Date: Fri, 14 Mar 2008 07:17:47 -0700
To: Dan Karp <[email-masked]>, Randall Gellens <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Cc: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Sender: [email-masked]

At 6:48 AM -0700 3/14/08, Dan Karp wrote:

>   > >  2) Remove attributes. i.e. 'value' and 'size' attributes will 
> be removed.
>>  >  There will only be a value. The .priv and .shared suffixes will then be
>>  >  applied to the entry names.
>>
>>  Does this really make a difference?
>
>  If there's no real difference, then simpler is better, no?

We were trying to maintain some level of similarity with ANNOTATE, 
and also provide the client with a way to see the size before 
fetching something.

I'm not saying these are must-have elements, but I'm not sure taking 
them out will make much difference in ease of implementation.  Are 
you saying it will?  Is this because of parsing responses, or 
something else?

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Auntie Em, Hate you; Hate Kansas; Taking the dog.  --Dorothy.

Date: Fri, 14 Mar 2008 06:48:36 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Randall Gellens <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

> >  2) Remove attributes. i.e. 'value' and 'size' attributes will be removed.
> >  There will only be a value. The .priv and .shared suffixes will then be
> >  applied to the entry names.
> 
> Does this really make a difference?

If there's no real difference, then simpler is better, no?

Date: Fri, 14 Mar 2008 05:06:36 -0700
To: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

At 5:54 PM -0400 3/13/08, Cyrus Daboo wrote:

>  1) Remove the dependency on LISTEXT. This basically means turning the clock
>  back to the original ANNOTATEMORE extension. So the GETMETADATA command
>  would be used to retrieve mailbox annotations as well as server annotations.

As I recall, the original concern was that using GETMETADATA for 
mailbox annotations would be confusing?  I have no problem with the 
change, but I wanted to be sure we understood why it was changed in 
the first place.

>  2) Remove attributes. i.e. 'value' and 'size' attributes will be removed.
>  There will only be a value. The .priv and .shared suffixes will then be
>  applied to the entry names.

Does this really make a difference?
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
We like because, we love although.

Cc: Alexey Melnikov <[email-masked]>, IMAP Extensions WG <[email-masked]>
From: Timo Sirainen <[email-masked]>
To: Bron Gondwana <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Date: Fri, 14 Mar 2008 03:46:33 +0200
Sender: [email-masked]

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--Apple-Mail-19--481272957
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

On Mar 14, 2008, at 2:15 AM, Bron Gondwana wrote:

>> Are there going to be other changes? The main problem I currently  
>> have
>> implementing CONDSTORE is that I'd rather not want to figure out
>> HIGHESTMODSEQ value when selecting a mailbox unless I know the  
>> client wants
>> to use CONDSTORE. I started to wonder if it would work nicely if a  
>> SELECT
>> mailbox returned [NOMODSEQ] but the first CONDSTORE enabling  
>> command would
>> cause * OK [HIGHESTMODSEQ n] to be sent.
>
> How do you calculate the MODSEQ if you're not using some sort of fast,
> atomically incrementable counter?

Dovecot writes all metadata changes to a transaction log. I use the  
log file number + offset for the change as the modseq. When CONDSTORE  
is first enabled I go through the existing log file(s) and calculate  
modseqs for each message. Modseqs are sometimes later stored to main  
index so this scanning doesn't have to be done all the time.

So the main problem is that before CONDSTORE is enabled, I don't keep  
track of the offset of last flag change (or expunge after QRESYNC is  
implemented) and I can't return a reliable HIGHESTMODSEQ. This is of  
course fixable in a few different ways, but if there are no real  
clients that delay announcing their CONDSTORE needs it seems kind of a  
waste of effort.

--Apple-Mail-19--481272957
content-type: application/pgp-signature; x-mac-type=70674453;
	name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

--Apple-Mail-19--481272957--

Date: Fri, 14 Mar 2008 11:15:35 +1100
From: Bron Gondwana <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Organization: brong.net
Sender: [email-masked]

On Thu, Mar 13, 2008 at 06:31:34PM +0200, Timo Sirainen wrote:
> On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>
>> Timo Sirainen wrote:
>>
>>> What happened to this?
>>>
>> I need to publish CONDSTOREbis draft at some point which contains the fix. 
>> I don't have time or energy to do this now. I should find some time once 
>> all my Lemonade drafts are approved for publication.
>
> Are there going to be other changes? The main problem I currently have 
> implementing CONDSTORE is that I'd rather not want to figure out 
> HIGHESTMODSEQ value when selecting a mailbox unless I know the client wants 
> to use CONDSTORE. I started to wonder if it would work nicely if a SELECT 
> mailbox returned [NOMODSEQ] but the first CONDSTORE enabling command would 
> cause * OK [HIGHESTMODSEQ n] to be sent.

How do you calculate the MODSEQ if you're not using some sort of fast,
atomically incrementable counter?

I guess it would be viable to store a "changed timestamp" or something,
and then lazily calculate a modseq only when something using CONDSTORE
viewed the mailbox - but that sounds pretty messy and potentially
expensive.  I've only read the Cyrus implementation, but it makes sense
in that it has a per-mailbox highestmodseq which gets updated while
the mailbox index is locked (hence the atomic bit) and the index records
themselves all have a modseq value which gets created/updated on any
store/append action.

My initial reaction to CONDSTORE when I saw it go in Cyrus was lukewarm,
but having recently actually tried to figure out the distribution of
bandwidth use on our servers (FastMail.FM) we have discovered that IMAP
TAG UID FETCH 1:* (FLAGS) calls are a huge part of our traffic.  It's a
big pain that in vanilla IMAP there's no way for a client to say "tell
me what has changed" on a 200k message mailbox.  I'd like to see
CONDSTORE used more generally now!

Regards,

Bron.

Date: Thu, 13 Mar 2008 22:03:44 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

>Hi folks,
>There have been several discussions at the IETF meeting about METADATA. Its 
>clear that implementers want a "lighter" version. Here are some proposals 
>based on those discussions and other comments during last call:
>
>1) Remove the dependency on LISTEXT. This basically means turning the clock 
>back to the original ANNOTATEMORE extension. So the GETMETADATA command 
>would be used to retrieve mailbox annotations as well as server annotations.
>  
>
That is fine with me.

You forgot handling on comparators in GETMETADATA.

>2) Remove attributes. i.e. 'value' and 'size' attributes will be removed. 
>There will only be a value. The .priv and .shared suffixes will then be 
>applied to the entry names.
>  
>
I am not convinced this is needed. I am really concerned about people 
putting big annotations on the server and then clients being flooded 
with traffic.

>3) Entries are case insensitive.
>  
>
I am neutral on this one. I can be convinced both ways.

>4) Wildcards not allowed in SETMETADATA.
>  
>
That would be fine with me.

>5) Only register /comment for both server and mailbox annotations, and 
>/admin for server annotations in this document.
>  
>
+1.

>Please comment on these so we can move forward with this extension, thanks.
>  
>

Some other things that need to be addressed:

Clarify access controls for various metadata items.

Date: Thu, 13 Mar 2008 17:54:56 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Taking the knife to METADATA
Sender: [email-masked]

Hi folks,
There have been several discussions at the IETF meeting about METADATA. Its 
clear that implementers want a "lighter" version. Here are some proposals 
based on those discussions and other comments during last call:

1) Remove the dependency on LISTEXT. This basically means turning the clock 
back to the original ANNOTATEMORE extension. So the GETMETADATA command 
would be used to retrieve mailbox annotations as well as server annotations.

2) Remove attributes. i.e. 'value' and 'size' attributes will be removed. 
There will only be a value. The .priv and .shared suffixes will then be 
applied to the entry names.

3) Entries are case insensitive.

4) Wildcards not allowed in SETMETADATA.

5) Only register /comment for both server and mailbox annotations, and 
/admin for server annotations in this document.

Please comment on these so we can move forward with this extension, thanks.

-- 
Cyrus Daboo

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-sort-20.txt 
Date: Thu, 13 Mar 2008 14:30:01 -0700 (PDT)
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS
	Author(s)	: M. Crispin, K. Murchison
	Filename	: draft-ietf-imapext-sort-20.txt
	Pages		: 0
	Date		: 2008-3-13
	
This document describes the base-level server-based sorting and
   threading extensions to the [IMAP] protocol.  These extensions
   provide substantial performance improvements for IMAP clients which
   offer sorted and threaded views.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-20.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-imapext-sort-20.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--NextPart--

Cc: IMAP Extensions WG <[email-masked]>
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Date: Thu, 13 Mar 2008 18:31:34 +0200
Sender: [email-masked]

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--Apple-Mail-18--514571583
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:

> Timo Sirainen wrote:
>
>> What happened to this?
>>
> I need to publish CONDSTOREbis draft at some point which contains  
> the fix. I don't have time or energy to do this now. I should find  
> some time once all my Lemonade drafts are approved for publication.

Are there going to be other changes? The main problem I currently have  
implementing CONDSTORE is that I'd rather not want to figure out  
HIGHESTMODSEQ value when selecting a mailbox unless I know the client  
wants to use CONDSTORE. I started to wonder if it would work nicely if  
a SELECT mailbox returned [NOMODSEQ] but the first CONDSTORE enabling  
command would cause * OK [HIGHESTMODSEQ n] to be sent.

Which also brings to my mind.. I didn't really understand why the * OK  
[HIGHESTMODSEQ n] would be sent again after it was already sent as a  
reply to SELECT. This is described in the spec example 5 and also  
happens in your implementation.

--Apple-Mail-18--514571583
content-type: application/pgp-signature; x-mac-type=70674453;
	name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

--Apple-Mail-18--514571583--

Date: Thu, 13 Mar 2008 15:57:21 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>What happened to this?
>
I need to publish CONDSTOREbis draft at some point which contains the 
fix. I don't have time or energy to do this now. I should find some time 
once all my Lemonade drafts are approved for publication.

>Your implementation seems to just ignore UNCHANGEDSINCE 0 instead of failing the update.
>  
>
It is possible, I need to check the code.

Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Thu, 13 Mar 2008 10:49:41 +0200
Sender: [email-masked]

--=-blAZ9GWAVWOpj816ro5X
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

What happened to this? Your implementation seems to just ignore
UNCHANGEDSINCE 0 instead of failing the update.

On Thu, 2006-11-09 at 19:15 -0800, Alexey Melnikov wrote:
> I think the following text in section 3.2 of RFC 4551
>=20
>  >      Use of UNCHANGEDSINCE with a modification sequence of 0 always
>  >      fails if the metadata item exists.  A system flag MUST always be
>  >      considered existent, whether it was set or not.
>=20
> is broken:
>=20
> 1) It is not clear why system flags have to be special
> 2) A flag existence is the same as have the flag set. Testing for flag=20
> existence can already be done using the FETCH (FLAGS) command.
>=20
> There are two ways to fix it:
>=20
> 1). Remove the second sentence to remove special casing for system=20
> flags. This command would allow people to test if a particular flag or a=20
> set of flags is set on a message. However this would leave STORE -FLAGS=20
> UNCHANGEDSINCE somewhat meaningless.
>=20
> 2). Replace the second sentence to say that this command always fail=20
> with tagged NO for both system and user defined flags. Note that this=20
> doesn't affect annotations.
>=20
> The rough consensus during the IMAPEXT meeting was to go with option 2.
> If you have any objections to this, please speak up now.
>=20
> Regards,
> Alexey

--=-blAZ9GWAVWOpj816ro5X
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-blAZ9GWAVWOpj816ro5X--

Date: Tue, 11 Mar 2008 21:25:17 +0000
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Let me propose some RFC editor notes based on discussions in this thread 
and some other comments from IESG:

In Section 2, add to the end of the last paragraph:

  The I18NLEVEL=1 extension updates SEARCH/SORT/THREAD to use
  i;unicode-casemap comparator, as defined in [UCM]. I18NLEVEL=1 is
  a simpler version of I18NLEVEL=2 with no ability to select a different
  collation.

In Section 3.1, add the following sentence to the end of 2nd paragraph:

 IMAP servers SHOULD NOT advertise the LANGUAGE extension
 if they discover that they only support "i-default".

In Section 3.1, last paragraph, second sentence:

OLD:
   Clients are urged to issue LANGUAGE before authentication, since some 
servers send
           ^^^^^^^^^^^^
   valuable user information as part of authentication (e.g. "password
   is correct, but expired").

NEW:
   Clients SHOULD issue LANGUAGE before authentication, since some 
servers send
           ^^^^^^
   valuable user information as part of authentication (e.g. "password
   is correct, but expired").

In Section 3.2, add a new sentence to the end of paragraph 9:

OLD:
   If there aren't any arguments, the server SHOULD send an untagged
   LANGUAGE response listing the languages it supports.  If the server
   is unable to enumerate the list of languages it supports it MAY
   return a tagged NO response to the enumeration request.

NEW:
   If there aren't any arguments, the server SHOULD send an untagged
   LANGUAGE response listing the languages it supports.  If the server
   is unable to enumerate the list of languages it supports it MAY
   return a tagged NO response to the enumeration request.
   If, after receiving a LANGUAGE request, the server discovers
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   that it doesn't support any language other than i-default,
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   it MUST return a tagged NO response to the enumeration request.
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Date: Sat, 08 Mar 2008 18:21:21 +0000
From: Alexey Melnikov <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Pete Resnick wrote:

> On 3/7/08 at 1:03 PM +0000, Alexey Melnikov wrote:
>
>> I've suggested addition of the following text:
>>   If the server discovers that it doesn't support any language other
>>   than i-default, it MUST return a tagged NO response to the
>>   enumeration request.
>
> If we're going to add that, also saying that one MUST NOT advertise 
> language if you don't support other than i-default is silly.

It is not silly. It is a better server behavior towards clients. I 
understand it might be too strong for some implementation. What people 
think about saying "servers SHOULD NOT advertise the LANGUAGE extension 
..."?

> If the client must deal with the case of a NO response to the 
> enumeration request, then it won't have a problem.

True.

> And I think Ned's point about not knowing until after CAPABILITIES 
> time that you don't have more than one language is well taken.

> Again, I don't see the big deal about a client dealing with a LANGUAGE 
> response containing only i-default, but if we really don't like that, 
> I think the above is a sufficient change.

Sufficient - yes. Ideal - no :-).
I guess I don't care that strongly about the requirements on LANGUAGE 
capability advertisement.

Date: Sat, 08 Mar 2008 18:13:35 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> On Sat, 8 Mar 2008, Alexey Melnikov wrote:
>
>> Can this text being read as saying that I18NLEVEL=1 requirements 
>> override I18NLEVEL=2 requirements (in the case when I18NLEVEL=2 is 
>> also implemented by the server)?
>
> Oh, all right.  How about this addition:
>
> 67c67,68
> <    I18NLEVEL=1 extension.
> ---
>
>>    I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>>    I18NLEVEL=2 (or higher) and comply with the rules of that level.
>
> 743c744,745
> <    the I18NLEVEL=1 extension.
> ---
>
>>    the I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>>    I18NLEVEL=2 (or higher) and comply with the rules of that level.
>
This looks good to me.

Date: Sat, 8 Mar 2008 09:12:08 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

On Sat, 8 Mar 2008, Alexey Melnikov wrote:
> Can this text being read as saying that I18NLEVEL=1 requirements override 
> I18NLEVEL=2 requirements (in the case when I18NLEVEL=2 is also implemented by 
> the server)?

Oh, all right.  How about this addition:

67c67,68
<    I18NLEVEL=1 extension.
---
>    I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>    I18NLEVEL=2 (or higher) and comply with the rules of that level.
743c744,745
<    the I18NLEVEL=1 extension.
---
>    the I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>    I18NLEVEL=2 (or higher) and comply with the rules of that level.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Date: Sat, 08 Mar 2008 16:19:00 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> On Fri, 7 Mar 2008, Alexey Melnikov wrote:
>
>> But this contradict the text you already have in section 7:
>
> OK, how about this in section 7:
>
>    As stated in the introduction, the rules of I18NLEVEL=1 as described
>    in [IMAP-I18N] MUST be followed; that is, the SORT and THREAD
>    extensions MUST collate strings according to the i;unicode-casemap
>    collation described in [UNICASEMAP].  Servers SHOULD also advertise
>    the I18NLEVEL=1 extension.
>
>    As discussed in [IMAP-I18N] section 4.5, all server implementations
>    should eventually be updated to support the [IMAP-I18N] I18NLEVEL=2
>    extension.

This looks better.

Can this text being read as saying that I18NLEVEL=1 requirements 
override I18NLEVEL=2 requirements (in the case when I18NLEVEL=2 is also 
implemented by the server)?

Cc: [email-masked], Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Date: Sat, 08 Mar 2008 03:44:49 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Ned Freed writes:
> >> Ned Freed writes:
> >> >> Ok, so you are talking about misconfiguration case, while I was thinking
> >> >> about implementing LANGUAGE without implementing any languages case.
> >> >
> >> > I realize with all the attention i18n gets in the IETF it's likely to
> >> > be considered an aberrant attitude, but there are plenty of cases
> >> > where people don't want any i18n support on their systems. So I
> >> > wouldn't call it misconfiguration per se.
> >>
> >> Unless the software in question... hm... there are two ways to finish
> >> this sentence...
> >>    - sucks enough to break subtly when it can't find it i18n files.
> >>    - is documented to require its i18n files.
> >> Personally I lean towards the first phrasing. And if we implement
> >> LANGUAGE we'll check that we have >1 language before announcing it.
> >
> > I'm sorry, but I quite simply disagree with this assessment.

> Are you sure?

I'm completely sure.

> Let me put it another way. If a piece of software needs a file and the
> file isn't there, then IMNSHO there are exactly three possibilities:
> Either the software says so at startup and quits, or it says so and
> adjusts its configuration so it doesn't need the file, or it sucks.

Hogwash. Our product uses literally hundreds of files in dozens of directories,
there are many similar products that use thousands. Messaging servers typically
spend more time running that they does being shut down, and full-on startups
are in any case rare - in our case we've had systems remain continuously
available for periods most easily measured in years.

Files can be deleted, moved, or modified at any time, and in practice these
sorts of actions happen routinely as a part of normal product reconfiguration.
But for every legitimate reconfiguration actitivity a vast number of harmful or
nonsensical actions are possible, and sufficient user cluelessness to screw
things up abounds despite numerous warnings and whatnot in the documeentation.

Indeed, only yesterday we had a case of someone creating files with
inappropriate names in a reserved directory that tricked a check we
periodically do into some bad behavior.  (And please note the irony - this was
a check we specifically make to see if certain files are present that went
wonky, not a problem caused by a check we didn't do.)

Code written to check and see if all is well at the filesystem level at
frequent enough intervals to catch all such problems would never do anything
else. (Actually, it's worse than that. There are cases where figuring the set
of files necessary is at least NP-hard, if not halting problem hard.) It
follows that it is entirely reasonable for code to only perform file presence
checks when ther's a compelling argument for doing so. In fact it's a practical
necessity that things be coded this way.

In this particular case the argument, such as it is, for this particular check
is that doing it prevents a wierd corner case from arising that most readers of
a specification aren't going to catch. And that in turn meands absent a
specific admonition in the specification it is entirely reasonable to expect
developers not too perform such a check.

> Breaking subtly when it's time to use a particular feature puts it into
> the third category.

Any the sorts of checks necessary to truly insure the sort of safety you're
talking about would put it in a fourth category - pararnoiaware. (Although if
you manage to crack NP in the process, it would definitely be worth it...)

				Ned

Date: Sat, 8 Mar 2008 09:46:04 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

Ned Freed writes:
>> Ned Freed writes:
>> >> Ok, so you are talking about misconfiguration case, while I was =
thinking
>> >> about implementing LANGUAGE without implementing any languages =
case.
>> >
>> > I realize with all the attention i18n gets in the IETF it's likely =
to
>> > be considered an aberrant attitude, but there are plenty of cases
>> > where people don't want any i18n support on their systems. So I
>> > wouldn't call it misconfiguration per se.
>>
>> Unless the software in question... hm... there are two ways to finish
>> this sentence...
>>    - sucks enough to break subtly when it can't find it i18n files.
>>    - is documented to require its i18n files.
>> Personally I lean towards the first phrasing. And if we implement
>> LANGUAGE we'll check that we have >1 language before announcing it.
>
> I'm sorry, but I quite simply disagree with this assessment.

Are you sure?

Let me put it another way. If a piece of software needs a file and the=20
file isn't there, then IMNSHO there are exactly three possibilities:=20
Either the software says so at startup and quits, or it says so and=20
adjusts its configuration so it doesn't need the file, or it sucks.

Breaking subtly when it's time to use a particular feature puts it into=20
the third category.

> I think what's being required of implementations is borderline=20
> unreasonablle and absent specific cautions in the specification I=20
> would not blame an implementation in
> the slightest for getting this "wrong".
>
> That said, it's not like we haven't required far more unreasonable =
stuff from
> implementations on numerous past occasions. So, as I said when I =
brought this
> up, I really don't care how  this ends up as long as the issue is=20
> mentioned in
> the text.

Yes. Text necessary.

Arnt

Cc: [email-masked], Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Date: Fri, 07 Mar 2008 10:56:15 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Ned Freed writes:
> >> Ok, so you are talking about misconfiguration case, while I was thinking
> >> about implementing LANGUAGE without implementing any languages case.
> >
> > I realize with all the attention i18n gets in the IETF it's likely to
> > be considered an aberrant attitude, but there are plenty of cases
> > where people don't want any i18n support on their systems. So I
> > wouldn't call it misconfiguration per se.

> Unless the software in question... hm... there are two ways to finish
> this sentence...
>    - sucks enough to break subtly when it can't find it i18n files.
>    - is documented to require its i18n files.
> Personally I lean towards the first phrasing. And if we implement
> LANGUAGE we'll check that we have >1 language before announcing it.

I'm sorry, but I quite simply disagree with this assessment. I think what's
being required of implementations is borderline unreasonablle and absent
specific cautions in the specification I would not blame an implementation in
the slightest for getting this "wrong".

That said, it's not like we haven't required far more unreasonable stuff from
implementations on numerous past occasions. So, as I said when I brought this
up, I really don't care how  this ends up as long as the issue is mentioned in
the text.

				Ned

Date: Fri, 7 Mar 2008 15:41:39 -0600
To: Alexey Melnikov <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 3/7/08 at 1:03 PM +0000, Alexey Melnikov wrote:

>I've suggested addition of the following text:
>   If the server discovers that it doesn't support any language other
>   than i-default, it MUST return a tagged NO response to the
>   enumeration request.

If we're going to add that, also saying that one MUST NOT advertise 
language if you don't support other than i-default is silly. If the 
client must deal with the case of a NO response to the enumeration 
request, then it won't have a problem. And I think Ned's point about 
not knowing until after CAPABILITIES time that you don't have more 
than one language is well taken.

Again, I don't see the big deal about a client dealing with a 
LANGUAGE response containing only i-default, but if we really don't 
like that, I think the above is a sufficient change.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 07 Mar 2008 17:14:06 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> On Fri, 7 Mar 2008, Alexey Melnikov wrote:
>
>>>    A server which implements the SORT and/or THREAD extensions MUST
>>>    collate strings in accordance with the requirements of I18NLEVEL=1,
>>>    as described in [IMAP-I18N], and SHOULD implement and advertise the
>>>    I18NLEVEL=1 extension.
>>
>> Typo: the last I18NLEVEL=1 should be I18NLEVEL=2.
>
> I accidentally deleted your previous message where you asked about this.
>
> I18NLEVEL=1 is correct.  Currently, there are servers that implement 
> SORT/THREAD in accordance with the requirements of I18NLEVEL=1, but do 
> not advertise I18NLEVEL=1.  We want those servers to upgrade sooner, 
> rather than later, when imap-i18n (finally) comes out; because there 
> is an ambiguity that a server might advertise SORT/THREAD without 
> complying with I18NLEVEL=1.
>
> Note the next paragraph:
>       Discussion: the SORT and THREAD extensions predate [IMAP-I18N] by
>       several years.  At the time of this writing, all known server
>       implementations of SORT and THREAD comply with the rules of
>       I18NLEVEL=1, but do not necessarily advertise it.  As discussed
>       in [IMAP-I18N] section 4.5, all server implementations should
>       eventually be updated to comply with the I18NLEVEL=2 extension.
>
> I don't think that SORT/THREAD can mandate I18NLEVEL=2 right now, even 
> with a SHOULD.

But this contradict the text you already have in section 7:

   As stated in the introduction, the server SHOULD support the
   [IMAP-I18N] I18NLEVEL=2 extension and follow its rules to perform
   collations in the SORT and THREAD extensions.

   If the server does not support I18NLEVEL=2, the rules of I18NLEVEL=1
   MUST be followed; that is, strings MUST be collated according to the
   i;unicode-casemap collation described in [UNICASEMAP].

Date: Fri, 7 Mar 2008 10:14:58 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

On Fri, 7 Mar 2008, Alexey Melnikov wrote:
> But this contradict the text you already have in section 7:

OK, how about this in section 7:

    As stated in the introduction, the rules of I18NLEVEL=1 as described
    in [IMAP-I18N] MUST be followed; that is, the SORT and THREAD
    extensions MUST collate strings according to the i;unicode-casemap
    collation described in [UNICASEMAP].  Servers SHOULD also advertise
    the I18NLEVEL=1 extension.

    As discussed in [IMAP-I18N] section 4.5, all server implementations
    should eventually be updated to support the [IMAP-I18N] I18NLEVEL=2
    extension.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Fri, 7 Mar 2008 09:00:42 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

On Fri, 7 Mar 2008, Alexey Melnikov wrote:
>>    A server which implements the SORT and/or THREAD extensions MUST
>>    collate strings in accordance with the requirements of I18NLEVEL=1,
>>    as described in [IMAP-I18N], and SHOULD implement and advertise the
>>    I18NLEVEL=1 extension.
> Typo: the last I18NLEVEL=1 should be I18NLEVEL=2.

I accidentally deleted your previous message where you asked about this.

I18NLEVEL=1 is correct.  Currently, there are servers that implement 
SORT/THREAD in accordance with the requirements of I18NLEVEL=1, but do not 
advertise I18NLEVEL=1.  We want those servers to upgrade sooner, rather 
than later, when imap-i18n (finally) comes out; because there is an 
ambiguity that a server might advertise SORT/THREAD without complying with 
I18NLEVEL=1.

Note the next paragraph:
       Discussion: the SORT and THREAD extensions predate [IMAP-I18N] by
       several years.  At the time of this writing, all known server
       implementations of SORT and THREAD comply with the rules of
       I18NLEVEL=1, but do not necessarily advertise it.  As discussed
       in [IMAP-I18N] section 4.5, all server implementations should
       eventually be updated to comply with the I18NLEVEL=2 extension.

I don't think that SORT/THREAD can mandate I18NLEVEL=2 right now, even 
with a SHOULD.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Fri, 07 Mar 2008 13:18:07 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> I've tried to get all the Last Call review comments together.  I was 
> going to fix these during AUTH48, but am considering pushing out a -20 
> on March 10 when the ID repository opens again.
>
> Here's what I have now.  Note the expansion of section 2.2, the 
> removal of date handling comments in the text in favor of references 
> to section 2.2, and the two new paragraphs in the security 
> considerations.

The new text in the Security Considerations looks good to me. I haven't 
checked new text in 2.2 very attentively.

> 1. Introduction

 [...]

>    A server which implements the SORT and/or THREAD extensions MUST
>    collate strings in accordance with the requirements of I18NLEVEL=1,
>    as described in [IMAP-I18N], and SHOULD implement and advertise the
>    I18NLEVEL=1 extension.

Typo: the last I18NLEVEL=1 should be I18NLEVEL=2.

Date: Fri, 07 Mar 2008 13:03:58 +0000
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Ned Freed wrote:

>> Ned Freed wrote:
>
 [...]

>> Ok, so you are talking about misconfiguration case, while I was thinking
>> about implementing LANGUAGE without implementing any languages case.
>
> I realize with all the attention i18n gets in the IETF it's likely to be
> considered an aberrant attitude, but there are plenty of cases where 
> people
> don't want any i18n support on their systems. So I wouldn't call it
> misconfiguration per se.

Sure. See my text below.

>> I think it is worth adding some text to describe correct behaviour for
>> your case.
>
> IMO that's an acceptable way to handle it.

I've suggested addition of the following text:
   If the server discovers that it doesn't support any language other
   than i-default, it MUST return a tagged NO response to the
   enumeration request.

At the end of the following paragraph:
   If there aren't any arguments, the server SHOULD send an untagged
   LANGUAGE response listing the languages it supports.  If the server
   is unable to enumerate the list of languages it supports it MAY
   return a tagged NO response to the enumeration request.

I can also suggest some additional text saying that the LANGUAGE 
extension shouldn't be advertised if no language other than i-default is 
available.

> Of course one way to deal with it
> would be to hard code one additionaal language code other than 
> i-default for
> whatever the built in language support is.

Indeed.

Date: Fri, 7 Mar 2008 12:23:28 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

Ned Freed writes:
>> Ok, so you are talking about misconfiguration case, while I was thinking
>> about implementing LANGUAGE without implementing any languages case.
>
> I realize with all the attention i18n gets in the IETF it's likely to 
> be considered an aberrant attitude, but there are plenty of cases 
> where people don't want any i18n support on their systems. So I 
> wouldn't call it misconfiguration per se.

Unless the software in question... hm... there are two ways to finish 
this sentence...
   - sucks enough to break subtly when it can't find it i18n files.
   - is documented to require its i18n files.
Personally I lean towards the first phrasing. And if we implement 
LANGUAGE we'll check that we have >1 language before announcing it.

Arnt

Date: Thu, 6 Mar 2008 16:05:03 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: SORT document, draft -20
Sender: [email-masked]

I've tried to get all the Last Call review comments together.  I was going 
to fix these during AUTH48, but am considering pushing out a -20 on March 
10 when the ID repository opens again.

Here's what I have now.  Note the expansion of section 2.2, the removal of 
date handling comments in the text in favor of references to section 2.2, 
and the two new paragraphs in the security considerations.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

IMAP Extensions Working Group                                 M. Crispin
Internet-Draft                                              K. Murchison
Intended status: Proposed Standard                        March 10, 2008
Expires: September 10, 2008
Document: internet-drafts/draft-ietf-imapext-sort-20.txt

      INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS

Status of this Memo

    By submitting this Internet-Draft, each author represents that
    any applicable patent or other IPR claims of which he or she is
    aware have been or will be disclosed, and any of which he or she
    becomes aware will be disclosed, in accordance with Section 6 of
    BCP 79.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as
    Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six months
    and may be updated, replaced, or obsoleted by other documents at any
    time.  It is inappropriate to use Internet-Drafts as reference
    material or to cite them other than as "work in progress."

    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/ietf/1id-abstracts.txt

    The list of Internet-Draft Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html.

    A revised version of this draft document will be submitted to the RFC
    editor as a Proposed Standard for the Internet Community.  Discussion
    and suggestions for improvement are requested, and should be sent to
    [email-masked].

    Distribution of this memo is unlimited.

Abstract

    This document describes the base-level server-based sorting and
    threading extensions to the [IMAP] protocol.  These extensions
    provide substantial performance improvements for IMAP clients which
    offer sorted and threaded views.

1. Introduction

    The SORT and THREAD extensions to the [IMAP] protocol provide a means
    of server-based sorting and threading of messages, without requiring
    that the client download the necessary data to do so itself.  This is
    particularly useful for online clients as described in [IMAP-MODELS].

    A server which supports the base-level SORT extension indicates this
    with a capability name which starts with "SORT".  Future,
    upwards-compatible extensions to the SORT extension will all start
    with "SORT", indicating support for this base level.

    A server which supports the THREAD extension indicates this with one
    or more capability names consisting of "THREAD=" followed by a
    supported threading algorithm name as described in this document.
    This provides for future upwards-compatible extensions.

    A server which implements the SORT and/or THREAD extensions MUST
    collate strings in accordance with the requirements of I18NLEVEL=1,
    as described in [IMAP-I18N], and SHOULD implement and advertise the
    I18NLEVEL=1 extension.

       Discussion: the SORT and THREAD extensions predate [IMAP-I18N] by
       several years.  At the time of this writing, all known server
       implementations of SORT and THREAD comply with the rules of
       I18NLEVEL=1, but do not necessarily advertise it.  As discussed
       in [IMAP-I18N] section 4.5, all server implementations should
       eventually be updated to comply with the I18NLEVEL=2 extension.

    Historical note: the REFERENCES threading algorithm is based on the
    [THREADING] algorithm written used in "Netscape Mail and News"
    versions 2.0 through 3.0.

2. Terminology

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
    document are to be interpreted as described in [KEYWORDS].

    The word "can" (not "may") is used to refer to a possible
    circumstance or situation, as opposed to an optional facility of the
    protocol.

    "User" is used to refer to a human user, whereas "client" refers to
    the software being run by the user.

    In examples, "C:" and "S:" indicate lines sent by the client and
    server respectively.

2.1 Base Subject

    Subject sorting and threading use the "base subject," which has
    specific subject artifacts removed.  Due to the complexity of these
    artifacts, the formal syntax for the subject extraction rules is
    ambiguous.  The following procedure is followed to determine the
    "base subject", using the [ABNF] formal syntax rules described in
    section 5:

         (1) Convert any RFC 2047 encoded-words in the subject to
         UTF-8 as described in "internationalization
         considerations."  Convert all tabs and continuations to
         space.  Convert all multiple spaces to a single space.

         (2) Remove all trailing text of the subject that matches
         the subj-trailer ABNF, repeat until no more matches are
         possible.

         (3) Remove all prefix text of the subject that matches the
         subj-leader ABNF.

         (4) If there is prefix text of the subject that matches the
         subj-blob ABNF, and removing that prefix leaves a non-empty
         subj-base, then remove the prefix text.

         (5) Repeat (3) and (4) until no matches remain.

    Note: it is possible to defer step (2) until step (6), but this
    requires checking for subj-trailer in step (4).

         (6) If the resulting text begins with the subj-fwd-hdr ABNF
         and ends with the subj-fwd-trl ABNF, remove the
         subj-fwd-hdr and subj-fwd-trl and repeat from step (2).

         (7) The resulting text is the "base subject" used in the
         SORT.

    All servers and disconnected (as described in [IMAP-MODELS]) clients
    MUST use exactly this algorithm to determine the "base subject".
    Otherwise there is potential for a user to get inconsistent results
    based on whether they are running in connected or disconnected mode.

2.2 Sent Date

    As used in this document, the term "sent date" refers to the date and
    time from the Date: header, adjusted by time zone to normalize to
    UTC.  For example, "31 Dec 2000 16:01:33 -0800" is equivalent to the
    UTC date and time of "1 Jan 2001 00:01:33 +0000".

    If the time zone is invalid, the date and time SHOULD be treated as
    UTC.  If the time is also invalid, the time SHOULD be treated as
    00:00:00.  If there is no valid date or time, the date and time
    SHOULD be treated as 00:00:00 on the earliest possible date.

    This differs from the date-related criteria in the SEARCH command
    (described in [IMAP] section 6.4.4), which use just the date and not
    the time, and are not adjusted by time zone.

    If the sent date can not be determined (a Date: header is missing or
    can not be parsed), the INTERNALDATE for that message is used as the
    sent date.

    When comparing two sent dates that match exactly, the order in which
    the two messages appear in the mailbox (that is, by sequence number)
    is used as a tie-breaker to determine the order.

3. Additional Commands

    These commands are extension to the [IMAP] base protocol.

    The section headings are intended to correspond with where they would
    be located in the main document if they were part of the base
    specification.

BASE.6.4.SORT. SORT Command

    Arguments:  sort program
                charset specification
                searching criteria (one or more)

    Data:       untagged responses: SORT

    Result:     OK - sort completed
                NO - sort error: can't sort that charset or
                     criteria
                BAD - command unknown or arguments invalid

       The SORT command is a variant of SEARCH with sorting semantics for
       the results.  Sort has two arguments before the searching criteria
       argument; a parenthesized list of sort criteria, and the searching
       charset.

       The charset argument is mandatory (unlike SEARCH) and indicates
       the [CHARSET] of the strings that appear in the searching
       criteria.  The US-ASCII and UTF-8 charsets MUST be implemented.
       All other charsets are optional.

       There is also a UID SORT command which returns unique identifiers
       instead of message sequence numbers.  Note that there are separate
       searching criteria for message sequence numbers and UIDs; thus the
       arguments to UID SORT are interpreted the same as in SORT.  This
       is analogous to the behavior of UID SEARCH, as opposed to UID
       COPY, UID FETCH, or UID STORE.

       The SORT command first searches the mailbox for messages that
       match the given searching criteria using the charset argument for
       the interpretation of strings in the searching criteria.  It then
       returns the matching messages in an untagged SORT response, sorted
       according to one or more sort criteria.

       Sorting is in ascending order.  Earlier dates sort before later
       dates; smaller sizes sort before larger sizes; and strings are
       sorted according to ascending values established by their
       collation algorithm (see under "Internationalization
       Considerations").

       If two or more messages exactly match according to the sorting
       criteria, these messages are sorted according to the order in
       which they appear in the mailbox.  In other words, there is an
       implicit sort criterion of "sequence number".

       When multiple sort criteria are specified, the result is sorted in
       the priority order that the criteria appear.  For example,
       (SUBJECT DATE) will sort messages in order by their base subject
       text; and for messages with the same base subject text will sort
       by their sent date.

       Untagged EXPUNGE responses are not permitted while the server is
       responding to a SORT command, but are permitted during a UID SORT
       command.

       The defined sort criteria are as follows.  Refer to the Formal
       Syntax section for the precise syntactic definitions of the
       arguments.  If the associated RFC-822 header for a particular
       criterion is absent, it is treated as the empty string.  The empty
       string always collates before non-empty strings.

       ARRIVAL
          Internal date and time of the message.  This differs from the
          ON criteria in SEARCH, which uses just the internal date.

       CC
          [IMAP] addr-mailbox of the first "cc" address.

       DATE
          Sent date and time, as described in section 2.2.

       FROM
          [IMAP] addr-mailbox of the first "From" address.

       REVERSE
          Followed by another sort criterion, has the effect of that
          criterion but in reverse (descending) order.
             Note: REVERSE only reverses a single criterion, and does not
             affect the implicit "sequence number" sort criterion if all
             other criteria are identicial.  Consequently, a sort of
             REVERSE SUBJECT is not the same as a reverse ordering of a
             SUBJECT sort.  This can be avoided by use of additional
             criteria, e.g. SUBJECT DATE vs. REVERSE SUBJECT REVERSE
             DATE.  In general, however, it's better (and faster, if the
             client has a "reverse current ordering" command) to reverse
             the results in the client instead of issuing a new SORT.

       SIZE
          Size of the message in octets.

       SUBJECT
          Base subject text.

       TO
          [IMAP] addr-mailbox of the first "To" address.

    Example:    C: A282 SORT (SUBJECT) UTF-8 SINCE 1-Feb-1994
                S: * SORT 2 84 882
                S: A282 OK SORT completed
                C: A283 SORT (SUBJECT REVERSE DATE) UTF-8 ALL
                S: * SORT 5 3 4 1 2
                S: A283 OK SORT completed
                C: A284 SORT (SUBJECT) US-ASCII TEXT "not in mailbox"
                S: * SORT
                S: A284 OK SORT completed

BASE.6.4.THREAD. THREAD Command

Arguments:  threading algorithm
             charset specification
             searching criteria (one or more)

Data:       untagged responses: THREAD

Result:     OK - thread completed
             NO - thread error: can't thread that charset or
                  criteria
             BAD - command unknown or arguments invalid

       The THREAD command is a variant of SEARCH with threading semantics
       for the results.  Thread has two arguments before the searching
       criteria argument; a threading algorithm, and the searching
       charset.

       The charset argument is mandatory (unlike SEARCH) and indicates
       the [CHARSET] of the strings that appear in the searching
       criteria.  The US-ASCII and UTF-8 charsets MUST be implemented.
       All other charsets are optional.

       There is also a UID THREAD command which returns unique
       identifiers instead of message sequence numbers.  Note that there
       are separate searching criteria for message sequence numbers and
       UIDs; thus the arguments to UID THREAD are interpreted the same as
       in THREAD.  This is analogous to the behavior of UID SEARCH, as
       opposed to UID COPY, UID FETCH, or UID STORE.

       The THREAD command first searches the mailbox for messages that
       match the given searching criteria using the charset argument for
       the interpretation of strings in the searching criteria.  It then
       returns the matching messages in an untagged THREAD response,
       threaded according to the specified threading algorithm.

       All collation is in ascending order.  Earlier dates collate before
       later dates and strings are collated according to ascending values
       established by their collation algorithm (see under
       "Internationalization Considerations").

       Untagged EXPUNGE responses are not permitted while the server is
       responding to a THREAD command, but are permitted during a UID
       THREAD command.

       The defined threading algorithms are as follows:

       ORDEREDSUBJECT

          The ORDEREDSUBJECT threading algorithm is also referred to as
          "poor man's threading."  The searched messages are sorted by
          base subject and then by the sent date.  The messages are then
          split into separate threads, with each thread containing
          messages with the same base subject text.  Finally, the threads
          are sorted by the sent date of the first message in the thread.

          The first message of each thread are siblings of each other
          (the "root").  The second message of a thread is the child of
          the first message, and subsequent messages of the thread are
          siblings of the second message and hence children of the
          message at the root.  Hence, there are no grandchildren in
          ORDEREDSUBJECT threading.

          Children in ORDEREDSUBJECT threading do not have descendents.
          Client implementations SHOULD treat descendents of a child in
          a server response as being siblings of that child.

       REFERENCES

          The REFERENCES threading algorithm threads the searched
          messages by grouping them together in parent/child
          relationships based on which messages are replies to others.
          The parent/child relationships are built using two methods:
          reconstructing a message's ancestry using the references
          contained within it; and checking the original (not base)
          subject of a message to see if it is a reply to (or forward of)
          another message.

             Note: "Message ID" in the following description refers to a
             normalized form of the msg-id in [RFC-2822].  The actual
             text in an RFC 2822 may use quoting, resulting in multiple
             ways of expressing the same Message ID.  Implementations of
             the REFERENCES threading algorithm MUST normalize any msg-id
             in order to avoid false non-matches due to differences in
             quoting.

             For example, the msg-id
                <"01KF8JCEOCBS0045PS"@xxx.yyy.com>
             and the msg-id
                <[email-masked]>
             MUST be interpreted as being the same Message ID.

          The references used for reconstructing a message's ancestry are
          found using the following rules:

             If a message contains a References header line, then use the
             Message IDs in the References header line as the references.

             If a message does not contain a References header line, or
             the References header line does not contain any valid
             Message IDs, then use the first (if any) valid Message ID
             found in the In-Reply-To header line as the only reference
             (parent) for this message.

                Note: Although [RFC-2822] permits multiple Message IDs in
                the In-Reply-To header, in actual practice this
                discipline has not been followed.  For example,
                In-Reply-To headers have been observed with message
                addresses after the Message ID, and there are no good
                heuristics for software to determine the difference.
                This is not a problem with the References header however.

             If a message does not contain an In-Reply-To header line, or
             the In-Reply-To header line does not contain a valid Message
             ID, then the message does not have any references (NIL).

          A message is considered to be a reply or forward if the base
          subject extraction rules, applied to the original subject,
          remove any of the following: a subj-refwd, a "(fwd)"
          subj-trailer, or a subj-fwd-hdr and subj-fwd-trl.

          The REFERENCES algorithm is significantly more complex than
          ORDEREDSUBJECT and consists of six main steps.  These steps are
          outlined in detail below.

          (1) For each searched message:

             (A) Using the Message IDs in the message's references, link
             the corresponding messages (those whose Message-ID header
             line contains the given reference Message ID) together as
             parent/child.  Make the first reference the parent of the
             second (and the second a child of the first), the second the
             parent of the third (and the third a child of the second),
             etc.  The following rules govern the creation of these
             links:

                If a message does not contain a Message-ID header line,
                or the Message-ID header line does not contain a valid
                Message ID, then assign a unique Message ID to this
                message.

                If two or more messages have the same Message ID, then
                only use that Message ID in the first (lowest sequence
                number) message, and assign a unique Message ID to each
                of the subsequent messages with a duplicate of that
                Message ID.

                If no message can be found with a given Message ID,
                create a dummy message with this ID.  Use this dummy
                message for all subsequent references to this ID.

                If a message already has a parent, don't change the
                existing link.  This is done because the References
                header line may have been truncated by a MUA.  As a
                result, there is no guarantee that the messages
                corresponding to adjacent Message IDs in the References
                header line are parent and child.

                Do not create a parent/child link if creating that link
                would introduce a loop.  For example, before making
                message A the parent of B, make sure that A is not a
                descendent of B.

                   Note: Message ID comparisons are case-sensitive.

             (B) Create a parent/child link between the last reference
             (or NIL if there are no references) and the current message.
             If the current message already has a parent, it is probably
             the result of a truncated References header line, so break
             the current parent/child link before creating the new
             correct one.  As in step 1.A, do not create the parent/child
             link if creating that link would introduce a loop.  Note
             that if this message has no references, that it will now
             have no parent.

                Note: The parent/child links created in steps 1.A and 1.B
                MUST be kept consistent with one another at ALL times.

          (2) Gather together all of the messages that have no parents
          and make them all children (siblings of one another) of a dummy
          parent (the "root").  These messages constitute the first
          (head) message of the threads created thus far.

          (3) Prune dummy messages from the thread tree.  Traverse each
          thread under the root, and for each message:

             If it is a dummy message with NO children, delete it.

             If it is a dummy message with children, delete it, but
             promote its children to the current level.  In other words,
             splice them in with the dummy's siblings.

             Do not promote the children if doing so would make them
             children of the root, unless there is only one child.

          (4) Sort the messages under the root (top-level siblings only)
          by sent date as described in section 2.2.  In the case of a
          dummy message, sort its children by sent date and then use the
          first child for the top-level sort.

          (5) Gather together messages under the root that have the same
          base subject text.

             (A) Create a table for associating base subjects with
             messages, called the subject table.

             (B) Populate the subject table with one message per each
             base subject.  For each child of the root:

                (i) Find the subject of this thread, by using the base
                subject from either the current message or its first
                child if the current message is a dummy.  This is the
                thread subject.

                (ii) If the thread subject is empty, skip this message.

                (iii) Look up the message associated with the thread
                subject in the subject table.

                (iv) If there is no message in the subject table with the
                thread subject, add the current message and the thread
                subject to the subject table.

                Otherwise, if the message in the subject table is not a
                dummy, AND either of the following criteria are true:

                   The current message is a dummy, OR

                   The message in the subject table is a reply or forward
                   and the current message is not.

             then replace the message in the subject table with the
             current message.

             (C) Merge threads with the same thread subject.  For each
             child of the root:

                (i) Find the message's thread subject as in step 5.B.i
                above.

                (ii) If the thread subject is empty, skip this message.

                (iii) Lookup the message associated with this thread
                subject in the subject table.

                (iv) If the message in the subject table is the current
                message, skip this message.

                Otherwise, merge the current message with the one in the
                subject table using the following rules:

                   If both messages are dummies, append the current
                   message's children to the children of the message in
                   the subject table (the children of both messages
                   become siblings), and then delete the current message.

                   If the message in the subject table is a dummy and the
                   current message is not, make the current message a
                   child of the message in the subject table (a sibling
                   of its children).

                   If the current message is a reply or forward and the
                   message in the subject table is not, make the current
                   message a child of the message in the subject table (a
                   sibling of its children).

                   Otherwise, create a new dummy message and make both
                   the current message and the message in the subject
                   table children of the dummy.  Then replace the message
                   in the subject table with the dummy message.

                      Note: Subject comparisons are case-insensitive, as
                      described under "Internationalization
                      Considerations."

          (6) Traverse the messages under the root and sort each set of
          siblings by sent date as described in section 2.2.  Traverse
          the messages in such a way that the "youngest" set of siblings
          are sorted first, and the "oldest" set of siblings are sorted
          last (grandchildren are sorted before children, etc).  In the
          case of a dummy message (which can only occur with top-level
          siblings), use its first child for sorting.

    Example:    C: A283 THREAD ORDEREDSUBJECT UTF-8 SINCE 5-MAR-2000
                S: * THREAD (166)(167)(168)(169)(172)(170)(171)
                   (173)(174 (175)(176)(178)(181)(180))(179)(177
                   (183)(182)(188)(184)(185)(186)(187)(189))(190)
                   (191)(192)(193)(194 195)(196 (197)(198))(199)
                   (200 202)(201)(203)(204)(205)(206 207)(208)
                S: A283 OK THREAD completed
                C: A284 THREAD ORDEREDSUBJECT US-ASCII TEXT "gewp"
                S: * THREAD
                S: A284 OK THREAD completed
                C: A285 THREAD REFERENCES UTF-8 SINCE 5-MAR-2000
                S: * THREAD (166)(167)(168)(169)(172)((170)(179))
                   (171)(173)((174)(175)(176)(178)(181)(180))
                   ((177)(183)(182)(188 (184)(189))(185 186)(187))
                   (190)(191)(192)(193)((194)(195 196))(197 198)
                   (199)(200 202)(201)(203)(204)(205 206 207)(208)
                S: A285 OK THREAD completed

         Note: The line breaks in the first and third server
         responses are for editorial clarity and do not appear in
         real THREAD responses.

4. Additional Responses

    These responses are extensions to the [IMAP] base protocol.

    The section headings of these responses are intended to correspond
    with where they would be located in the main document.

BASE.7.2.SORT. SORT Response

    Data:       zero or more numbers

       The SORT response occurs as a result of a SORT or UID SORT
       command.  The number(s) refer to those messages that match the
       search criteria.  For SORT, these are message sequence numbers;
       for UID SORT, these are unique identifiers.  Each number is
       delimited by a space.

    Example:    S: * SORT 2 3 6

BASE.7.2.THREAD. THREAD Response

    Data:       zero or more threads

       The THREAD response occurs as a result of a THREAD or UID THREAD
       command.  It contains zero or more threads.  A thread consists of
       a parenthesized list of thread members.

       Thread members consist of zero or more message numbers, delimited
       by spaces, indicating successive parent and child.  This continues
       until the thread splits into multiple sub-threads, at which point
       the thread nests into multiple sub-threads with the first member
       of each subthread being siblings at this level.  There is no limit
       to the nesting of threads.

       The messages numbers refer to those messages that match the search
       criteria.  For THREAD, these are message sequence numbers; for UID
       THREAD, these are unique identifiers.

    Example:    S: * THREAD (2)(3 6 (4 23)(44 7 96))

       The first thread consists only of message 2.  The second thread
       consists of the messages 3 (parent) and 6 (child), after which it
       splits into two subthreads; the first of which contains messages 4
       (child of 6, sibling of 44) and 23 (child of 4), and the second of
       which contains messages 44 (child of 6, sibling of 4), 7 (child of
       44), and 96 (child of 7).  Since some later messages are parents
       of earlier messages, the messages were probably moved from some
       other mailbox at different times.

       -- 2

       -- 3
          \-- 6
              |-- 4
              |   \-- 23
              |
              \-- 44
                   \-- 7
                       \-- 96

    Example:    S: * THREAD ((3)(5))

       In this example, 3 and 5 are siblings of a parent which does not
       match the search criteria (and/or does not exist in the mailbox);
       however they are members of the same thread.

5. Formal Syntax of SORT and THREAD Commands and Responses

    The following syntax specification uses the Augmented Backus-Naur
    Form (ABNF) notation as specified in [ABNF].  It also uses [ABNF]
    rules defined in [IMAP].

sort            = ["UID" SP] "SORT" SP sort-criteria SP search-criteria

sort-criteria   = "(" sort-criterion *(SP sort-criterion) ")"

sort-criterion  = ["REVERSE" SP] sort-key

sort-key        = "ARRIVAL" / "CC" / "DATE" / "FROM" / "SIZE" /
                   "SUBJECT" / "TO"

thread          = ["UID" SP] "THREAD" SP thread-alg SP search-criteria

thread-alg      = "ORDEREDSUBJECT" / "REFERENCES" / thread-alg-ext

thread-alg-ext  = atom
                     ; New algorithms MUST be registered with IANA

search-criteria = charset 1*(SP search-key)

charset         = atom / quoted
                     ; CHARSET values MUST be registered with IANA

sort-data       = "SORT" *(SP nz-number)

thread-data     = "THREAD" [SP 1*thread-list]

thread-list     = "(" (thread-members / thread-nested) ")"

thread-members  = nz-number *(SP nz-number) [SP thread-nested]

thread-nested   = 2*thread-list

    The following syntax describes base subject extraction rules (2)-(6):

subject         = *subj-leader [subj-middle] *subj-trailer

subj-refwd      = ("re" / ("fw" ["d"])) *WSP [subj-blob] ":"

subj-blob       = "[" *BLOBCHAR "]" *WSP

subj-fwd        = subj-fwd-hdr subject subj-fwd-trl

subj-fwd-hdr    = "[fwd:"

subj-fwd-trl    = "]"

subj-leader     = (*subj-blob subj-refwd) / WSP

subj-middle     = *subj-blob (subj-base / subj-fwd)
                     ; last subj-blob is subj-base if subj-base would
                     ; otherwise be empty

subj-trailer    = "(fwd)" / WSP

subj-base       = NONWSP *(*WSP NONWSP)
                     ; can be a subj-blob

BLOBCHAR        = %x01-5a / %x5c / %x5e-ff
                     ; any CHAR8 except '[' and ']'

NONWSP          = %x01-08 / %x0a-1f / %x21-ff
                     ; any CHAR8 other than WSP

6. Security Considerations

    The SORT and THREAD extensions do not raise any security
    considerations that are not present in the base [IMAP] protocol, and
    these issues are discussed in [IMAP].  Nevertheless, it is important
    to remember that [IMAP] protocol transactions, including message
    data, are sent in the clear over the network unless protection from
    snooping is negotiated, either by the use of STARTTLS, privacy
    protection is negotiated in the AUTHENTICATE command, or some other
    protection mechanism.

    Although not a security consideration, it is important to recognize
    that sorting by REFERENCES can lead to misleading threading trees.
    For example, a message with false References: header data will cause
    a thread to be incorporated into another thread.

    The process of extracting the base subject may lead to incorrect
    collation if the extracted data was significant text as opposed to
    a subject artifact.

7. Internationalization Considerations

    As stated in the introduction, the server SHOULD support the
    [IMAP-I18N] I18NLEVEL=2 extension and follow its rules to perform
    collations in the SORT and THREAD extensions.

    If the server does not support I18NLEVEL=2, the rules of I18NLEVEL=1
    MUST be followed; that is, strings MUST be collated according to the
    i;unicode-casemap collation described in [UNICASEMAP].

    Translations of the "re" or "fw"/"fwd" tokens are not specified for
    removal in the base subject extraction process.  An attempt to add
    such translated tokens would result in a geometrically complex, and
    ultimately unimplementable, task.

    Instead, note that [RFC-2822] section 3.6.5 recommends that "re:"
    (from the Latin "res", in the matter of) be used to identify a reply.
    Although it is evident that, from the multiple forms of token to
    identify a forwarded message, there is considerable variation found
    in the wild, the variations are (still) manageable.  Consequently, it
    is suggested that "re:" and one of the variations of the tokens for
    forward supported by the base subject extraction rules be adopted for
    Internet mail messages, since doing so makes it a simple display time
    task to localize the token language for the user.

8. IANA Considerations

    [IMAP] capabilities are registered by publishing a standards track or
    IESG approved experimental RFC.  This document constitutes
    registration of the SORT and THREAD capabilities in the [IMAP]
    capabilities registry.

    This document creates a new [IMAP] threading algorithms registry,
    which registers threading algorithms by publishing a standards track
    or IESG approved experimental RFC.  This document constitutes
    registration of the ORDEREDSUBJECT and REFERENCES algorithms in that
    registry.

9. Normative References

    The following documents are normative to this document:

    [ABNF]                Crocker, D. and Overell, P. "Augmented BNF
                          for Syntax Specifications: ABNF", RFC 5234
                          January 2008

    [CHARSET]             Freed, N. and Postel, J. "IANA Character Set
                          Registration Procedures", RFC 2978, October
                          2000.

    [IMAP]                Crispin, M. "Internet Message Access Protocol -
                          Version 4rev1", RFC 3501, March 2003.

    [IMAP-I18N]           Newman, C. and Gulbrandsen, A. "Internet
                          Message Access Protocol Internationalization",
                          Work in Progress.

    [KEYWORDS]            Bradner, S. "Key words for use in RFCs to
                          Indicate Requirement Levels", BCP 14, RFC 2119,
                          March 1997.

    [RFC-2822]            Resnick, P. "Internet Message Format", RFC
                          2822, April 2001.

    [UNICASEMAP]          Crispin, M. "i;unicode-casemap - Simple Unicode
                          Collation Algorithm", RFC 5051.

10. Informative References

    The following documents are informative to this document:

    [IMAP-MODELS]         Crispin, M. "Distributed Electronic Mail Models
                          in IMAP4", RFC 1733, December 1994.

    [THREADING]           Zawinski, J. "Message Threading",
                          http://www.jwz.org/doc/threading.html,
                          1997-2002.

Appendices

Author's Address

    Mark R. Crispin
    Networks and Distributed Computing
    University of Washington
    4545 15th Avenue NE
    Seattle, WA  98105-4527

    Phone: +1 (206) 543-5762

    EMail: [email-masked]

    Kenneth Murchison
    Carnegie Mellon University
    5000 Forbes Avenue
    Cyert Hall 285
    Pittsburgh, PA  15213

    Phone: +1 (412) 268-2638
    Email: [email-masked]

Full Copyright Statement

    Copyright (C) The IETF Trust (2008).

    This document is subject to the rights, licenses and restrictions
    contained in BCP 78, and except as set forth therein, the authors
    retain all their rights.

    This document and the information contained herein are provided on an
    "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
    OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
    THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
    OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
    THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
    WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

    The IETF takes no position regarding the validity or scope of any
    Intellectual Property Rights or other rights that might be claimed to
    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; nor does it represent that it has
    made any independent effort to identify any such rights.  Information
    on the procedures with respect to rights in RFC documents can be
    found in BCP 78 and BCP 79.

    Copies of IPR disclosures made to the IETF Secretariat and any
    assurances of licenses to be made available, or the result of an
    attempt made to obtain a general license or permission for the use of
    such proprietary rights by implementers or users of this
    specification can be obtained from the IETF on-line IPR repository at
    http://www.ietf.org/ipr.

    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights that may cover technology that may be required to implement
    this standard.  Please address the information to the IETF at ietf-
    [email-masked].

Acknowledgement

    Funding for the RFC Editor function is currently provided by the
    Internet Society.

Date: Thu, 6 Mar 2008 16:02:26 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Tobias Gondrom <[email-masked]>
cc: [email-masked], [email-masked], [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: secdir review of draft-ietf-imapext-sort-19
Organization: UW Technology
Sender: [email-masked]

Thank you for your thoughtful comments.

I am working on update that addresses your issues.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Cc: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Thu, 06 Mar 2008 13:14:47 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> Ned Freed wrote:

> >> > Now, if this corner case does exist the cleanest solution would be to
> >> > have two separate responses.
> >
> >> Right.
> >
> >> > But that might break existing implementations.
> >
> >> Indeed.
> >
> >> After reading all responses on this subject I still tend to think that
> >> Tim's proposal is the best one: I can't see any good reason why a server
> >> would implement LANGUAGE command and only support i-default, as this is
> >> pointless.
> >
> > Hmm. Well, the one possibility that occurs to me os that quite a lot of
> > software implements i18n support by having a separate
> > directory/file/container
> > for each supported language. That way new languages can be added or
> > removed on
> > the fly. And when a given language is requested a check is made to see if
> > there's a corrresponding directory/file/container for it.
> >
> > This makes it fairly easy to end up in a situation where the only
> > directory/file/container is the i-default one - in fact the usual way
> > this
> > works is i-default is the built-in default that doesn't have a
> > directory/file/container associated with it, so this corresponds to
> > the fairly
> > common "I deleted all those nasty i18n files in order to save  space"
> > behavior.

> Ok, so you are talking about misconfiguration case, while I was thinking
> about implementing LANGUAGE without implementing any languages case.

I realize with all the attention i18n gets in the IETF it's likely to be
considered an aberrant attitude, but there are plenty of cases where people
don't want any i18n support on their systems. So I wouldn't call it
misconfiguration per se.

> I think it is worth adding some text to describe correct behaviour for
> your case.

IMO that's an acceptable way to handle it. Of course one way to deal with it
would be to hard code one additionaal language code other than i-default for
whatever the built in language support is.

> > This requirement would force such an implementation to check and see
> > whether or
> > not non-i-default material is available before even offering the
> > extension. (Or
> > you can have a extension enable/disable configuration variable, but of
> > course
> > this can lead to a silly state.)

> Yes. This is similar to STARTTLS, which shouldn't be advertised unless
> server can create a TLS context.

Exactly, and unfortunately what we've seen in practice is plenty of server
implementations that do not check this and offer STARTTLS even when they don't
have the necessary certificate(s) installed. This effectively killed
opportunistic use of TLS in SMTP back when it was first standardized -  we had
to advise all our customers not to enable "use TLS if available" mode on
Internet-facing relay clients because it blocked message tranfer in too many
cases. Overall email security didn't exactly benefit from this...

Now, of course this odd corner case isn't remotely as important as the lost
opportunity to encrypt a significant fraction of email on the wire, But it
still needs to be dealt with.

				Ned

Date: Wed, 05 Mar 2008 21:12:31 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Pete Resnick <[email-masked]>, [email-masked], Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Dave Cridland wrote:

> On Tue Mar  4 19:30:02 2008, Pete Resnick wrote:
>
>> No, I think Tim's concern is different, but I still think there is  
>> nothing (besides perhaps a little wordsmithing) that needs to be  
>> done: Tim's concern is that if you get back a LANGUAGE response  with 
>> only 1 language tag, you don't know (without saving some  context 
>> about what command you issued) whether that means the  server is 
>> sending back its list of available languages (which only  contains 1 
>> item) or is telling you that it is now using the  language it just 
>> sent back. But I think this is a distinction  without a difference: 
>> If the server only supports i-default, it is  *always* using 
>> i-default and will *always* return a single language  tag (of 
>> i-default).
>>
>> I'm open for additional text at the end of the paragraph to clarify.
>
> I had to think hard about this - Pete's got a good point here - I  
> just have to disagree.
>
> What Pete's saying is that in the case where there is only one  
> language, which MUST be i-default, there is no semantic difference  
> between the server saying "I only use i-default" and "I am using  
> i-default" - whatever happens, the server is now using i-default. The  
> trouble is that if a client wishes to kick the server into telling it  
> what languages it supports, it instead gets a message back saying  
> that the language has been set - I have a feeling I'd still need to  
> involve semantics at too low a level to be comfortable with. It's  
> painful enough with unextended SEARCH, I wouldn't want to  
> special-case anything else.
>
> What Ned says in another message is that there is considerable work  
> in the server knowing it only has i-default available prior to  
> offering the extension.
>
> On this basis, I think I'll change my mind again - we don't mandate  
> multiple languages, since the choice to remove additional language  
> support from the implementation can be a deployment-stage decision.  
> Instead, we really do need to change the syntax.

I am not convinced that this is necessary.

> Maybe change one to be a response code:
>
> C001 LANGUAGE DE
> * OK [LANGUAGE DE] Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
> C001 OK Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
>
> Or we could change the response to always carry the current language,  
> when enumerating languages:
>
> * LANGUAGE i-default (EN DE IT i-default)
>
> Syntax:
>
> language-data = "LANGUAGE" SP lang-tag-quoted [ "(" lang-tag-quoted  
> *(SP lang-tag-quoted) ")" ]
>
> Or, when enumerating languages, i-default MUST NOT be included:
>
> A01 LANGUAGE
> * LANGUAGE ()
> A01 OK I skipped all my electives at college
>
> And finally, we could say that if you only support i-default for some  
> reason, then you MUST say NO to LANGUAGE.

I prefer the last alternative, as it means no syntactic changes to the 
LANGUAGE response for deployed servers and clients.

> I don't care which, but I vaguely prefer excluding it from the list -  
> it's noise anyway, since it's mandated to be supported.

Date: Wed, 05 Mar 2008 21:06:55 +0000
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Ned Freed wrote:

>> > Now, if this corner case does exist the cleanest solution would be to
>> > have two separate responses.
>
>> Right.
>
>> > But that might break existing implementations.
>
>> Indeed.
>
>> After reading all responses on this subject I still tend to think that
>> Tim's proposal is the best one: I can't see any good reason why a server
>> would implement LANGUAGE command and only support i-default, as this is
>> pointless.
>
> Hmm. Well, the one possibility that occurs to me os that quite a lot of
> software implements i18n support by having a separate 
> directory/file/container
> for each supported language. That way new languages can be added or 
> removed on
> the fly. And when a given language is requested a check is made to see if
> there's a corrresponding directory/file/container for it.
>
> This makes it fairly easy to end up in a situation where the only
> directory/file/container is the i-default one - in fact the usual way 
> this
> works is i-default is the built-in default that doesn't have a
> directory/file/container associated with it, so this corresponds to 
> the fairly
> common "I deleted all those nasty i18n files in order to save  space" 
> behavior.

Ok, so you are talking about misconfiguration case, while I was thinking 
about implementing LANGUAGE without implementing any languages case.

I think it is worth adding some text to describe correct behaviour for 
your case.

> This requirement would force such an implementation to check and see 
> whether or
> not non-i-default material is available before even offering the 
> extension. (Or
> you can have a extension enable/disable configuration variable, but of 
> course
> this can lead to a silly state.)

Yes. This is similar to STARTTLS, which shouldn't be advertised unless 
server can create a TLS context.

> Anyway, the inconvenience of doing this may be unavoidable, but I did 
> want to
> point out that there are reasonable ways the i-default-only case can 
> arise in
> practice.

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
From: Timo Sirainen <[email-masked]>
To: Dave Cridland <[email-masked]>
Cc: Pete Resnick <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked], Ned Freed <[email-masked]>
Date: Thu, 06 Mar 2008 04:49:28 +0200
Sender: [email-masked]

--=-pBM0/G3Ovp74wEkKhuIb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2008-03-05 at 16:01 +0000, Dave Cridland wrote:
> What Ned says in another message is that there is considerable work =20
> in the server knowing it only has i-default available prior to =20
> offering the extension.
>=20
> On this basis, I think I'll change my mind again - we don't mandate =20
> multiple languages, since the choice to remove additional language =20
> support from the implementation can be a deployment-stage decision. =20
> Instead, we really do need to change the syntax.

I was also getting concerned about this, but there is an easy workaroud:

1. If some i18n dirs are found at server startup, advertise the
extension.

2. When IMAP session is started and/or LANGUAGE is used and no i18n
files are found anymore, just disconnect the client and log an error.
Sysadmin will eventually restart the server which fixes the problem.

It's not like i18n files are constantly being added and removed..

--=-pBM0/G3Ovp74wEkKhuIb
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-pBM0/G3Ovp74wEkKhuIb--

Date: Wed, 5 Mar 2008 15:41:26 -0600
To: Dave Cridland <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

On 3/5/08 at 3:36 PM -0600, Pete Resnick wrote:

>if(returned languages contains i-default)
>     set supported languages to returned languages

Whoops. That's a mistake. You'd have to say:

if(returned languages contains i-default and supported languages is NULL)
     set supported languages to returned languages

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Wed, 5 Mar 2008 15:36:36 -0600
To: Dave Cridland <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

On 3/5/08 at 4:01 PM +0000, Dave Cridland wrote:

>What Pete's saying is that in the case where there is only one 
>language, which MUST be i-default, there is no semantic difference 
>between the server saying "I only use i-default" and "I am using 
>i-default" - whatever happens, the server is now using i-default.

Correct.

>The trouble is that if a client wishes to kick the server into 
>telling it what languages it supports, it instead gets a message 
>back saying that the language has been set - I have a feeling I'd 
>still need to involve semantics at too low a level to be comfortable 
>with.

I assume that the client will keep a list of languages that the 
server supports, and keep track of what language the server is using. 
Let's imagine each of these start as NULL. If the supported languages 
list is NULL, the client is going to request the list. If the current 
language is NULL, the client is going to consult the supported 
languages list, choose one and set it. Either way, the logic for 
handling the LANGUAGE response is:

if(returned languages contains i-default)
     set supported languages to returned languages
if(number of returned languages is 1)
     set currently set language to returned language

>On this basis, I think I'll change my mind again - we don't mandate 
>multiple languages, since the choice to remove additional language 
>support from the implementation can be a deployment-stage decision. 
>Instead, we really do need to change the syntax.

I don't see why.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Wed, 05 Mar 2008 16:01:32 +0000
From: Dave Cridland <[email-masked]>
To: Pete Resnick <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

On Tue Mar  4 19:30:02 2008, Pete Resnick wrote:
> 
> On 3/4/08 at 9:41 AM -0800, Ned Freed wrote:
> 
>>> Tim Polk have noticed the following issue with the COMPARATOR  
>>> response:
>>> 
>>>>     A LANGUAGE response with a list containing a single language  
>>>> tag
>>>>     indicates that the server is now using that language.  A  
>>>> LANGUAGE
>>>>     response with a list containing multiple language tags  
>>>> indicates the
>>>>     server is communicating a list of available languages to the  
>>>> client,
>>>>     and no change in the active language has been made.
>> 
>> Maybe I'm not reading the draft properly, but I'm not convinced  
>> this corner
>> case exists. There are two cases: A LANGUAGE command with an  
>> argument and a
>> LANGUAGE command without an arugment. The former always gets a  
>> response
>> consisting of the single language that was selected (or an error)  
>> while the
>> latter always gets a response specifying what languages are  
>> available.
>> 
>> In particular, I don't see a way for the LANGUAGE command with an  
>> argument
>> to return the available language list.
> 
> No, I think Tim's concern is different, but I still think there is  
> nothing (besides perhaps a little wordsmithing) that needs to be  
> done: Tim's concern is that if you get back a LANGUAGE response  
> with only 1 language tag, you don't know (without saving some  
> context about what command you issued) whether that means the  
> server is sending back its list of available languages (which only  
> contains 1 item) or is telling you that it is now using the  
> language it just sent back. But I think this is a distinction  
> without a difference: If the server only supports i-default, it is  
> *always* using i-default and will *always* return a single language  
> tag (of i-default).
> 
> I'm open for additional text at the end of the paragraph to clarify.

I had to think hard about this - Pete's got a good point here - I  
just have to disagree.

What Pete's saying is that in the case where there is only one  
language, which MUST be i-default, there is no semantic difference  
between the server saying "I only use i-default" and "I am using  
i-default" - whatever happens, the server is now using i-default. The  
trouble is that if a client wishes to kick the server into telling it  
what languages it supports, it instead gets a message back saying  
that the language has been set - I have a feeling I'd still need to  
involve semantics at too low a level to be comfortable with. It's  
painful enough with unextended SEARCH, I wouldn't want to  
special-case anything else.

What Ned says in another message is that there is considerable work  
in the server knowing it only has i-default available prior to  
offering the extension.

On this basis, I think I'll change my mind again - we don't mandate  
multiple languages, since the choice to remove additional language  
support from the implementation can be a deployment-stage decision.  
Instead, we really do need to change the syntax.

Maybe change one to be a response code:

C001 LANGUAGE DE
* OK [LANGUAGE DE] Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
C001 OK Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt

Or we could change the response to always carry the current language,  
when enumerating languages:

* LANGUAGE i-default (EN DE IT i-default)

Syntax:

language-data = "LANGUAGE" SP lang-tag-quoted [ "(" lang-tag-quoted  
*(SP lang-tag-quoted) ")" ]

Or, when enumerating languages, i-default MUST NOT be included:

A01 LANGUAGE
* LANGUAGE ()
A01 OK I skipped all my electives at college

And finally, we could say that if you only support i-default for some  
reason, then you MUST say NO to LANGUAGE.

I don't care which, but I vaguely prefer excluding it from the list -  
it's noise anyway, since it's mandated to be supported.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Wed, 5 Mar 2008 07:53:03 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Pete Resnick <[email-masked]>, Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

+1

On Wed, 5 Mar 2008, Alexey Melnikov wrote:
> The new sentence is:
>
>  IMAP servers that support this extension MUST support
>  at least one language in addition to "i-default".

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Cc: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Wed, 05 Mar 2008 07:00:41 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> > In particular, I don't see a way for the LANGUAGE command with an argument
> > to return the available language list.

> You are correct regarding LANGUAGE responses in response to various
> forms of the LANGUAGE command. However the LANGUAGE response is an
> untagged response and in theory can be returned by the server in
> response to any other command. So there is still an issue of identifying
> the exact meaning of the response if it is sent in unexpected context.

I wasn't entirely buying the "it's hard to track the context of the response
without complicating your parser" assertions but this justification makes
complete sense to me.

> > Now, if this corner case does exist the cleanest solution would be to
> > have two separate responses.

> Right.

> > But that might break existing implementations.

> Indeed.

> After reading all responses on this subject I still tend to think that
> Tim's proposal is the best one: I can't see any good reason why a server
> would implement LANGUAGE command and only support i-default, as this is
> pointless.

Hmm. Well, the one possibility that occurs to me os that quite a lot of
software implements i18n support by having a separate directory/file/container
for each supported language. That way new languages can be added or removed on
the fly. And when a given language is requested a check is made to see if
there's a corrresponding directory/file/container for it.

This makes it fairly easy to end up in a situation where the only
directory/file/container is the i-default one - in fact the usual way this
works is i-default is the built-in default that doesn't have a
directory/file/container associated with it, so this corresponds to the fairly
common "I deleted all those nasty i18n files in order to save  space" behavior.

This requirement would force such an implementation to check and see whether or
not non-i-default material is available before even offering the extension. (Or
you can have a extension enable/disable configuration variable, but of course
this can lead to a silly state.)

Anyway, the inconvenience of doing this may be unavoidable, but I did want to
point out that there are reasonable ways the i-default-only case can arise in
practice.

				Ned

Date: Wed, 05 Mar 2008 14:23:53 +0000
From: Alexey Melnikov <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Pete Resnick wrote:

> On 3/4/08 at 9:41 AM -0800, Ned Freed wrote:
>
>>> Tim Polk have noticed the following issue with the COMPARATOR response:
>>>
>>>>     A LANGUAGE response with a list containing a single language tag
>>>>     indicates that the server is now using that language.  A LANGUAGE
>>>>     response with a list containing multiple language tags 
>>>> indicates the
>>>>     server is communicating a list of available languages to the 
>>>> client,
>>>>     and no change in the active language has been made.
>>>
>> Maybe I'm not reading the draft properly, but I'm not convinced this 
>> corner
>> case exists. There are two cases: A LANGUAGE command with an argument 
>> and a
>> LANGUAGE command without an arugment. The former always gets a response
>> consisting of the single language that was selected (or an error) 
>> while the
>> latter always gets a response specifying what languages are available.
>>
>> In particular, I don't see a way for the LANGUAGE command with an 
>> argument
>> to return the available language list.
>
> No, I think Tim's concern is different, but I still think there is 
> nothing (besides perhaps a little wordsmithing) that needs to be done: 
> Tim's concern is that if you get back a LANGUAGE response with only 1 
> language tag, you don't know (without saving some context about what 
> command you issued) whether that means the server is sending back its 
> list of available languages (which only contains 1 item) or is telling 
> you that it is now using the language it just sent back. But I think 
> this is a distinction without a difference: If the server only 
> supports i-default, it is *always* using i-default and will *always* 
> return a single language tag (of i-default).
>
> I'm open for additional text at the end of the paragraph to clarify.

I think section 3.3 is good as is. I suggest adding a new sentence at 
the end of the following paragraph in Section 3.1 (LANGUAGE Extension 
Requirements):

   A server that advertises this extension MUST use the language "i-
   default" as described in [RFC2277] as its default language until
   another supported language is negotiated by the client. A server
   MUST include "i-default" as one of its supported languages.

The new sentence is:

   IMAP servers that support this extension MUST support
   at least one language in addition to "i-default".

Date: Wed, 05 Mar 2008 11:44:59 +0000
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Ned Freed wrote:

>> Tim Polk have noticed the following issue with the COMPARATOR response:
>
>> >The LANGUAGE response is ambiguous for the corner case where the 
>> LANGUAGE
>> >extension
>> >is supported but only the i-default langauge is supported.
>> >Specifically, Section 3.3 states:
>> >
>> >    A LANGUAGE response with a list containing a single language tag
>> >    indicates that the server is now using that language.  A LANGUAGE
>> >    response with a list containing multiple language tags indicates 
>> the
>> >    server is communicating a list of available languages to the 
>> client,
>> >    and no change in the active language has been made.
>> >
>> >However, for the corner case the server's list of langauges is just 
>> i-default.
>> >
>> >Adding a requirements that "IMAP servers that support this extension 
>> MUST
>> >support at least one langauge in addition to i-default" would 
>> correct this by
>> >avoiding the corner case.   (Just an observation, I'm not set on any 
>> particular
>> >solution.)
>> >
>> So we can follow's Tim suggestion. Alternatively we can have 2 separate
>> responses to distinguish 2 cases. Which alternative is preferred by 
>> the WG?
>
> Maybe I'm not reading the draft properly, but I'm not convinced this 
> corner
> case exists. There are two cases: A LANGUAGE command with an argument 
> and a
> LANGUAGE command without an argument. The former always gets a response
> consisting of the single language that was selected (or an error) 
> while the
> latter always gets a response specifying what languages are available.
>
> In particular, I don't see a way for the LANGUAGE command with an 
> argument
> to return the available language list.

You are correct regarding LANGUAGE responses in response to various 
forms of the LANGUAGE command. However the LANGUAGE response is an 
untagged response and in theory can be returned by the server in 
response to any other command. So there is still an issue of identifying 
the exact meaning of the response if it is sent in unexpected context.

> Now, if this corner case does exist the cleanest solution would be to 
> have
> two separate responses.

Right.

> But that might break existing implementations.

Indeed.
After reading all responses on this subject I still tend to think that 
Tim's proposal is the best one: I can't see any good reason why a server 
would implement LANGUAGE command and only support i-default, as this is 
pointless.

> But other fixes are possib;e: The obvious one would be to eliminate 
> the case
> where a LANGUAGE command with an argument returns a list of available
> languages.

Date: Tue, 4 Mar 2008 13:30:02 -0600
To: Ned Freed <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 3/4/08 at 9:41 AM -0800, Ned Freed wrote:

>>Tim Polk have noticed the following issue with the COMPARATOR response:
>>
>>>     A LANGUAGE response with a list containing a single language tag
>>>     indicates that the server is now using that language.  A LANGUAGE
>>>     response with a list containing multiple language tags indicates the
>>>     server is communicating a list of available languages to the client,
>>>     and no change in the active language has been made.
>
>Maybe I'm not reading the draft properly, but I'm not convinced this corner
>case exists. There are two cases: A LANGUAGE command with an argument and a
>LANGUAGE command without an arugment. The former always gets a response
>consisting of the single language that was selected (or an error) while the
>latter always gets a response specifying what languages are available.
>
>In particular, I don't see a way for the LANGUAGE command with an argument
>to return the available language list.

No, I think Tim's concern is different, but I still think there is 
nothing (besides perhaps a little wordsmithing) that needs to be 
done: Tim's concern is that if you get back a LANGUAGE response with 
only 1 language tag, you don't know (without saving some context 
about what command you issued) whether that means the server is 
sending back its list of available languages (which only contains 1 
item) or is telling you that it is now using the language it just 
sent back. But I think this is a distinction without a difference: If 
the server only supports i-default, it is *always* using i-default 
and will *always* return a single language tag (of i-default).

I'm open for additional text at the end of the paragraph to clarify.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Tue, 04 Mar 2008 19:12:41 +0000
From: Dave Cridland <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: <[email-masked]>, Alexey Melnikov <[email-masked]>
Sender: [email-masked]

On Tue Mar  4 17:41:14 2008, Ned Freed wrote:
> Maybe I'm not reading the draft properly, but I'm not convinced  
> this corner
> case exists. There are two cases: A LANGUAGE command with an  
> argument and a
> LANGUAGE command without an arugment. The former always gets a  
> response
> consisting of the single language that was selected (or an error)  
> while the
> latter always gets a response specifying what languages are  
> available.
> 
> In particular, I don't see a way for the LANGUAGE command with an  
> argument
> to return the available language list.
> 
> 
Client implementors really like to be able to handle responses  
without knowing which command emitted them. Makes for cleaner designs.

So there *is* a corner case, just not an obvious one.

Dave.

> Now, if this corner case does exist the cleanest solution would be  
> to have
> two separate responses. But that might break existing  
> implementations. But
> other fixes are possib;e: The obvious one would be to eliminate the  
> case
> where a LANGUAGE command with an argument returns a list of  
> available
> languages.
> 
> 				Ned
> 
> 
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Tue, 04 Mar 2008 19:08:39 +0000
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>, Alexey Melnikov <[email-masked]>, Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

On Tue Mar  4 18:52:20 2008, Mark Crispin wrote:
> Can someone explain to me why the LANGUAGE extension is be offered  
> at all if i-default is the only defined extension?  Can't we do an  
> end run by stating that an implementation of the LANGUAGE extension  
> MUST allow at least one language other than i-default?
> 
> I have no sympathy for implementions that want to "support  
> LANGUAGE" as a checklist item, yet not actually implement any  
> language.

+1 - I wasn't thinking about semantics, only syntax.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Tue, 4 Mar 2008 11:07:47 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked], Alexey Melnikov <[email-masked]>, Dave Cridland <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

On Tue, 4 Mar 2008, Arnt Gulbrandsen wrote:
> Mark Crispin writes:
>> Can't we do an end run by stating that an implementation of the LANGUAGE 
>> extension MUST allow at least one language other than i-default?
> Isn't that what Tim suggests?

I think so, but I haven't kept close enough track.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Tue, 4 Mar 2008 20:04:01 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Mark Crispin <[email-masked]>, Dave Cridland <[email-masked]>
Sender: [email-masked]

Mark Crispin writes:
> Can't we do an end run by stating that an implementation of the 
> LANGUAGE extension MUST allow at least one language other than 
> i-default?

Isn't that what Tim suggests?

Arnt

Date: Tue, 4 Mar 2008 11:00:25 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Stephane Bortzmeyer <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS   PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Organization: UW Technology
Sender: [email-masked]

On Tue, 4 Mar 2008, Stephane Bortzmeyer wrote:
>> The actual correct collation, assuming(!) surname-first collation and
>> Latin character ordering(!!), is:
>> due to where the surname is located in various cultures.
> Is it a good idea to sort on the ordering of the sender's culture? If
> the ordering is to be useful for the human user, it should be
> according the receiver's culture, no?

My answer to this question is "yes to all of the above".

That is, a truly correct collation uses knowledge of both the sender's 
culture *AND* the receiver's culture; and the ways that these interact are 
not necessarily obvious.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Date: Tue, 4 Mar 2008 10:52:20 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked], Alexey Melnikov <[email-masked]>, Dave Cridland <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Organization: UW Technology
Sender: [email-masked]

On Tue, 4 Mar 2008, Arnt Gulbrandsen wrote:
>> * LANGUAGE i-default // I am now using i-default
>> * LANGUAGE (i-default) // I can only do i-default
> A little subtle, don't you think? Demanding of tired brains? I had to look 
> twice...

I agree.  IMAP's architecture may be subtle at times, but not THAT subtle.

> Tim's suggestion to only offer the choice when you're offering more than one 
> alternative makes more sense to me.

Can someone explain to me why the LANGUAGE extension is be offered at all 
if i-default is the only defined extension?  Can't we do an end run by 
stating that an implementation of the LANGUAGE extension MUST allow at 
least one language other than i-default?

I have no sympathy for implementions that want to "support LANGUAGE" as a 
checklist item, yet not actually implement any language.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Cc: IMAP Extensions WG <[email-masked]>
Date: Tue, 04 Mar 2008 09:41:14 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> Tim Polk have noticed the following issue with the COMPARATOR response:

> >The LANGUAGE response is ambiguous for the corner case where the LANGUAGE
> >extension
> >is supported but only the i-default langauge is supported.
> >Specifically, Section 3.3 states:
> >
> >    A LANGUAGE response with a list containing a single language tag
> >    indicates that the server is now using that language.  A LANGUAGE
> >    response with a list containing multiple language tags indicates the
> >    server is communicating a list of available languages to the client,
> >    and no change in the active language has been made.
> >
> >However, for the corner case the server's list of langauges is just i-default.
> >
> >Adding a requirements that "IMAP servers that support this extension MUST
> >support at least one langauge in addition to i-default" would correct this by
> >avoiding the corner case.   (Just an observation, I'm not set on any particular
> >solution.)
> >
> So we can follow's Tim suggestion. Alternatively we can have 2 separate
> responses to distinguish 2 cases. Which alternative is preferred by the WG?

Maybe I'm not reading the draft properly, but I'm not convinced this corner
case exists. There are two cases: A LANGUAGE command with an argument and a
LANGUAGE command without an arugment. The former always gets a response
consisting of the single language that was selected (or an error) while the
latter always gets a response specifying what languages are available.

In particular, I don't see a way for the LANGUAGE command with an argument
to return the available language list.

Now, if this corner case does exist the cleanest solution would be to have
two separate responses. But that might break existing implementations. But
other fixes are possib;e: The obvious one would be to eliminate the case
where a LANGUAGE command with an argument returns a list of available
languages.

				Ned

Date: Tue, 4 Mar 2008 12:49:19 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Cc: Mark Crispin <[email-masked]>, Stephane Bortzmeyer <[email-masked]>
Sender: [email-masked]

Stephane Bortzmeyer writes:
> On Wed, Feb 27, 2008 at 10:26:44AM -0800,
>  Mark Crispin <[email-masked]> wrote a message of 162 lines which said:
>
>>  The actual correct collation, assuming(!) surname-first collation 
>>  and Latin character ordering(!!), is:
> ...
>>  due to where the surname is located in various cultures.
>
> Is it a good idea to sort on the ordering of the sender's culture? If 
> the ordering is to be useful for the human user, it should be 
> according the receiver's culture, no?

There's support for that. You have to read a few drafts and RFCs, though.

The draft in question defines a sort command and some sort keys, and it 
permits defining more sort keys. Other documents allow defining 
collations (sort orders) and using them im IMAP.

Sorting by From field the way most MUAs do requires defining a new 
collation and a new sort key, since this draft's "from" key uses only 
the localpart ('arnt' in the case of this message). Personally I think 
that sort key isn't at all fortunate, but it's too late to change that. 
There's decade-old running code already and the most important thing is 
to document the deployed extension.

Arnt

Date: Tue, 4 Mar 2008 12:19:57 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Dave Cridland <[email-masked]>
Sender: [email-masked]

Dave Cridland writes:
> So you'd get:
>
> * LANGUAGE i-default // I am now using i-default
> * LANGUAGE (i-default) // I can only do i-default

A little subtle, don't you think? Demanding of tired brains? I had to 
look twice...

Tim's suggestion to only offer the choice when you're offering more than 
one alternative makes more sense to me.

Arnt

Date: Tue, 4 Mar 2008 12:13:59 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Alexey Melnikov writes:
>> Adding a requirements that "IMAP servers that support this extension MUST
>> support at least one langauge in addition to i-default" would correct 
>> this by
>> avoiding the corner case.   (Just an observation, I'm not set on any 
>> particular
>> solution.)
>
> So we can follow's Tim suggestion. Alternatively we can have 2 
> separate responses to distinguish 2 cases. Which alternative is 
> preferred by the WG?

Tim's suggestion. What he's found is that LANGUAGE without languages has 
syntactic problems. I think it's likely that there are other problems 
too, which his suggestion avoids at low or no cost.

Arnt

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Tue, 04 Mar 2008 10:42:56 +0000
From: Dave Cridland <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Mon Mar  3 21:26:11 2008, Alexey Melnikov wrote:
> 
> Tim Polk have noticed the following issue with the COMPARATOR  
> response:
> 
>> The LANGUAGE response is ambiguous for the corner case where the  
>> LANGUAGE
>> extension
>> is supported but only the i-default langauge is supported.
>> Specifically, Section 3.3 states:
>> 
>>    A LANGUAGE response with a list containing a single language tag
>>    indicates that the server is now using that language.  A  
>> LANGUAGE
>>    response with a list containing multiple language tags  
>> indicates the
>>    server is communicating a list of available languages to the  
>> client,
>>    and no change in the active language has been made.
>> 
>> However, for the corner case the server's list of langauges is  
>> just i-default.
>> 
>> Adding a requirements that "IMAP servers that support this  
>> extension MUST
>> support at least one langauge in addition to i-default" would  
>> correct this by
>> avoiding the corner case.   (Just an observation, I'm not set on  
>> any particular
>> solution.)
>> 
> So we can follow's Tim suggestion. Alternatively we can have 2  
> separate responses to distinguish 2 cases. Which alternative is  
> preferred by the WG?

I like the idea of sticking the available languages in parentheses.  
It's the simplest change I can think of.

So you'd get:

* LANGUAGE i-default // I am now using i-default
* LANGUAGE (i-default) // I can only do i-default

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Tue, 4 Mar 2008 11:32:16 +0100
From: Stephane Bortzmeyer <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>, [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS  	PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Organization: NIC France
Sender: [email-masked]

On Wed, Feb 27, 2008 at 10:26:44AM -0800,
 Mark Crispin <[email-masked]> wrote 
 a message of 162 lines which said:

> The actual correct collation, assuming(!) surname-first collation and 
> Latin character ordering(!!), is:
...
> due to where the surname is located in various cultures.

Is it a good idea to sort on the ordering of the sender's culture? If
the ordering is to be useful for the human user, it should be
according the receiver's culture, no?

Date: Mon, 03 Mar 2008 21:26:11 +0000
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Tim Polk have noticed the following issue with the COMPARATOR response:

>The LANGUAGE response is ambiguous for the corner case where the LANGUAGE
>extension
>is supported but only the i-default langauge is supported.
>Specifically, Section 3.3 states:
>
>    A LANGUAGE response with a list containing a single language tag
>    indicates that the server is now using that language.  A LANGUAGE
>    response with a list containing multiple language tags indicates the
>    server is communicating a list of available languages to the client,
>    and no change in the active language has been made.
>
>However, for the corner case the server's list of langauges is just i-default.
>
>Adding a requirements that "IMAP servers that support this extension MUST
>support at least one langauge in addition to i-default" would correct this by
>avoiding the corner case.   (Just an observation, I'm not set on any particular
>solution.)
>
So we can follow's Tim suggestion. Alternatively we can have 2 separate 
responses to distinguish 2 cases. Which alternative is preferred by the WG?

Date: Mon, 3 Mar 2008 17:42:58 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, [email-masked], Dan Karp <[email-masked]>, Alexey Melnikov <[email-masked]>, Messaging Organization <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Organization: UW Technology
Sender: [email-masked]

On Mon, 3 Mar 2008, Dave Cridland wrote:
> I can think of circumstances where sorting by the full email address is 
> useful (which would also catch the above), and where sorting by the full name 
> is also useful.

I agree with all this; which is why the SORT/THREAD specification is 
extensible.  I have said many times that I would support a follow-on 
specification that adds new sort keys and thread algorithms.  As long as 
the keys are not preposterous I would seriously consider implementing 
them.  I might even implement a preposterous key if the implementation is 
straightforward.

My objections are not -- and have never been -- about adding new keys. 
My objections are to ademand that existing keys with a long history be 
removed.

If the keys in question are difficult to implement in a server, then the 
server is fundamentally flawed in its frameworks.  The most plausible 
explanation would be a server that uses a database back end that doesn't 
have ready access to the necessary addr-mailbox components.  That's no 
excuse.  The same data are needed to deliver satisfactory ENVELOPE 
performance; if the server can not do that well then it is not doing IMAP 
well.

Then there is the postulate that the presence of "useless" keys will make 
it more difficult to add "more useful" keys.  That would happen in only 
two scenarios:
  (1) The existing keys are good enough.
and/or
  (2) Nobody cares enough about the issue.

Neither is a justification to remove functionality.

If something is "good enough", then it should win especially when it is 
entrenched for decades.  The ridiculous notion that "good enough" must 
yield to "perfect" has led to the current situation where the IETF never 
gets anything done.

And, if nobody cared enough about the issue, then the "more useful" keys 
are a solution in search of a problem and should die.

Put another way, we should add new sort keys iff the current set of keys 
are not "good enough" AND some consituency cares.  I consider both 
conditions to be plausible; but my stake is that these new keys are 
additions, not replacements, for the existing keys.

> Threads "changing" is indeed a problem for many cases - I'm not sure I see 
> the value in ordering by subject to address it though - could you explain a 
> bit more? (I'm not saying there's no value, I just can't visualize how this 
> helps).

Typically, the thread-changer thinks that he has broken the thread by 
changing the subject, but did not do anything about the references links. 
A SUBJECT sort or ORDEREDSUBJECT thread breaks the thread, since neither 
uses the references links at all.

Now, there are also cases where the subject changes but the thread is 
legitimately the same.  That's why I don't use just one collation; I 
normally use THREAD=REFERENCES by default but switch to other collations 
when the situation is such that I think one of them will do a better job.

> Hard - one has to consider what the flag's precedence is, and how to order 
> flag combinations. Some cases are easy - one might say that (\Flagged) > 
> (\Flagged \Seen) > () > (\Seen), but where user-defined keywords are heavily 
> used, any sort order really needs to be client-defined.

I agree.  If flag sorting is done at all, it needs to be engineered 
carefully and probably via a client-defined mechanism.

> I've seen people doing both "Sent" and "Received" date sorts.

Strong agreement.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Date: Mon, 03 Mar 2008 20:59:54 +0000
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, <[email-masked]>, Dan Karp <[email-masked]>, Alexey Melnikov <[email-masked]>, Messaging Organization <[email-masked]>
Sender: [email-masked]

Dropped ietf@ietf from the CC list, and added [email-masked]... I  
think we're moving into the realms of what could usefully be  
discussed for future extension. The Mail ORGanization list was  
created to act as a venue some time ago, and I've been remiss in not  
trying to stimulate discussion. (Partly because I was waiting for  
things like this draft to be pushed through to RFC, in my defense).

On Mon Mar  3 18:44:35 2008, Mark Crispin wrote:
> On Mon, 3 Mar 2008, Alexey Melnikov wrote:
>>> (In some clients, the sorted list is scrolled to whichever   
>>> message was previously selected, so it's a fast way of finding  
>>> other  messages by the same person).
>> Yea, I do this frequently in Thunderbird.
> 
> As do I.
> 
> Note that an address sort that collates all "mrc" messages together  
> is more likely to do the right thing that one that collates by name  
> and has to figure out that "Mark Crispin", "Mark R. Crispin",  
> "Crispin, Mark", "M Crispin", "Mr. Mark Crispin", "IMAP Support",  
> etc. are all the same.
> 
> 
I can think of circumstances where sorting by the full email address  
is useful (which would also catch the above), and where sorting by  
the full name is also useful.

The case where sorting by only the mailbox is useful is where this is  
common across domains to mean the same person - true for you, not so  
for others - including myself. On the other hand, people rarely vary  
the full name they use in their From header field, even between  
accounts, in my experience - this may be true for most people and  
false for you, of course.

Equally, if you take subaddressing into account, then the local-part  
varies for a single user.

FWIW, I note that full names are sometimes varied for use in distinct  
roles ("IMAP Support"), which might even be a positive point for  
sorting only by full names.

>> I frequently sort by subject, because many clients still don't  
>> support proper threading (threads can become broken due to lack of  
>> the References header).
> 
> I also sort by Subject (although more commonly ORDERSUBJECT  
> threading than direct SUBJECT sorting) in cases where the thread  
> tree has become so complex that it has lost value.  A classic  
> example is when a new thread is started by replying to a message in  
> a thread that has little or no relationship to the thread.  Think  
> USENET.
> 
> 
Threads "changing" is indeed a problem for many cases - I'm not sure  
I see the value in ordering by subject to address it though - could  
you explain a bit more? (I'm not saying there's no value, I just  
can't visualize how this helps).

>> I also sort by IMAP flags (i.e. I want to see all important  
>> messages), size and rarely by date.
> 
> Usually for flags I would do a filter rather than a SORT, but it  
> seems reasonable to me to add such a thing in the first round of  
> extensions.
> 
> 
Hard - one has to consider what the flag's precedence is, and how to  
order flag combinations. Some cases are easy - one might say that  
(\Flagged) > (\Flagged \Seen) > () > (\Seen), but where user-defined  
keywords are heavily used, any sort order really needs to be  
client-defined.

> Date sorting seems to be pretty common with our users.  Some people  
> who prefer their mail "in order" use it instead of sequence order  
> in the mailbox (no sort) which tends to be an INTERNALDATE sort.

I've seen people doing both "Sent" and "Received" date sorts.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Mon, 3 Mar 2008 11:06:29 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, IETF Discussion <[email-masked]>, [email-masked], Dan Karp <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

On Mon, 3 Mar 2008, Dave Cridland wrote:
> Still, though, the presence of useless SORT keys would only be a significant 
> problem if they were especially difficult to implement, and the large 
> deployments of SORT indicate that they're not. Even if it's only PINE, that's 
> still a pretty vast number of users who may be relying on this behaviour.

+1.

It amazes me that this is even questioned, much less that the entire 
[email-masked] list has been dragged into the fray.  This discussion 
belongs either in ietf-imapext, or if an appeal is to be lodged it should 
be made with the IESG.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Mon, 3 Mar 2008 10:44:35 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Dave Cridland <[email-masked]>, Cyrus Daboo <[email-masked]>, [email-masked], IETF Discussion <[email-masked]>, Dan Karp <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort          (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to         Proposed Standard
Sender: [email-masked]

On Mon, 3 Mar 2008, Alexey Melnikov wrote:
>> (In some 
>> clients, the sorted list is scrolled to whichever  message was previously 
>> selected, so it's a fast way of finding other  messages by the same 
>> person).
> Yea, I do this frequently in Thunderbird.

As do I.

Note that an address sort that collates all "mrc" messages together is 
more likely to do the right thing that one that collates by name and has 
to figure out that "Mark Crispin", "Mark R. Crispin", "Crispin, Mark", "M 
Crispin", "Mr. Mark Crispin", "IMAP Support", etc. are all the same.

> I frequently sort by subject, because many clients still don't support proper 
> threading (threads can become broken due to lack of the References header).

I also sort by Subject (although more commonly ORDERSUBJECT threading than 
direct SUBJECT sorting) in cases where the thread tree has become so 
complex that it has lost value.  A classic example is when a new thread is 
started by replying to a message in a thread that has little or no 
relationship to the thread.  Think USENET.

> I also sort by IMAP flags (i.e. I want to see all important messages), size 
> and rarely by date.

Usually for flags I would do a filter rather than a SORT, but it seems 
reasonable to me to add such a thing in the first round of extensions.

Date sorting seems to be pretty common with our users.  Some people who 
prefer their mail "in order" use it instead of sequence order in the 
mailbox (no sort) which tends to be an INTERNALDATE sort.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Mon, 3 Mar 2008 10:27:16 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort          (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS)         to Proposed Standard
Sender: [email-masked]

I agree and will/have address[ed] these in AUTH48.

On Mon, 3 Mar 2008, Alexey Melnikov wrote:
> The IESG wrote:
>
>> The IESG is considering the following document again now that important 
>> dependencies are ready:
>> 
>> - 'INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS'
>>   <draft-ietf-imapext-sort-19.txt> as a Proposed Standard
>> 
> Hi Mark,
> I've just reread section 7 of -19 and found some relatively minor things that 
> need fixing:
>
>> 7. Internationalization Considerations
>>
>>     As stated in the introduction, the server SHOULD support the
>>     [IMAP-I18N] COMPARATOR extension and follow its rules to perform
>>     collations in the SORT and THREAD extensions.
>
> As per your recent suggestion in the IMAPEXT WG, the COMPARATOR extension was 
> renamed to I18NLEVEL=2.
> The last sentence in section 1 needs to be updated as well.
>
>>     If the server does not support COMPARATOR, strings MUST be collated
>>     according to the i;unicode-casemap collation described in
>>     [UNICASEMAP].
>
> I think you also mean that support for at least I18NLEVEL=1 is REQUIREd.
>
>>     As described in [IMAP-I18N], strings in charsets other than US-ASCII
>>     and UTF-8 MUST be converted to UTF-8 and compared in ascending order
>>     according to the selected or active collation algorithm.  If the
>>     server does not support the [IMAP-I18N] COMPARATOR extension, the
>>     collation algorithm used is the "en;ascii-casemap" collation
>>     described in [COMPARATOR].
>
> The last sentence contradict the second paragraph quoted above. I think you 
> forgot to delete it in -19.
> If for some reason you want to keep it, then please change "en;ascii-casemap" 
> to "i;ascii-casemap", as "en;ascii-casemap" is invalid.
>
> I can recommend some replacement text, if you want.
>
>

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Mon, 03 Mar 2008 16:23:01 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, [email-masked], Mark Crispin <[email-masked]>, IETF Discussion <[email-masked]>, Dan Karp <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort  (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

Speaking as email client user:

Dave Cridland wrote:

>On Mon Mar  3 15:03:20 2008, Cyrus Daboo wrote:
>  
>
>>--On March 2, 2008 11:07:50 PM -0800 Dan Karp <[email-masked]>  
>>wrote:
>>    
>>
>>>The purpose of sorting in mail clients is so that users can find
>>>messages they're looking for.
>>>      
>>>
>>Actually you need to look at your use cases in more detail because  
>>a lot of times searching is a much better solution than sorting.  
>>e.g. the case of trying to find email from a particular person - a  
>>search is much better. Yes you do have to do a little more work to  
>>setup the search (type something in) rather than the single click  
>>sorting requires, but on large mailboxes you will usually see what  
>>you want immediately without having to scroll through the sorted  
>>list looking for what you want.
>>    
>>
>I don't think Dan's saying that using SORT for searching is a good  
>idea, but it's certainly what people do based on my observations,  
>too. (In some clients, the sorted list is scrolled to whichever  
>message was previously selected, so it's a fast way of finding other  
>messages by the same person).
>  
>
Yea, I do this frequently in Thunderbird.

>>For the most part I question why anyone would really want to sort  
>>on any of the text fields. The only sort I have found useful is  
>>sort by size - and then only to help in culling large messages when  
>>my quota-limited mailboxes get full.
>>    
>>
>My client does no sorting at all, so I'm hard pressed to give any  
>alternative use-cases. Generally, people seem to like sorting by date  
>fields, though again not a text field. I've also seen people who like  
>to sort by subject, essentially as a grouping thing - better done  
>with threading, but still.
>  
>
I frequently sort by subject, because many clients still don't support 
proper threading (threads can become broken due to lack of the 
References header).

I also sort by IMAP flags (i.e. I want to see all important messages), 
size and rarely by date.

Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Date: Mon, 03 Mar 2008 16:14:57 +0000
From: Dave Cridland <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Cc: IETF Discussion <[email-masked]>, <[email-masked]>, Dan Karp <[email-masked]>, Mark Crispin <[email-masked]>
Sender: [email-masked]

On Mon Mar  3 15:03:20 2008, Cyrus Daboo wrote:
> --On March 2, 2008 11:07:50 PM -0800 Dan Karp <[email-masked]>  
> wrote:
> 
>> The purpose of sorting in mail clients is so that users can find
>> messages they're looking for.
> 
> Actually you need to look at your use cases in more detail because  
> a lot of times searching is a much better solution than sorting.  
> e.g. the case of trying to find email from a particular person - a  
> search is much better. Yes you do have to do a little more work to  
> setup the search (type something in) rather than the single click  
> sorting requires, but on large mailboxes you will usually see what  
> you want immediately without having to scroll through the sorted  
> list looking for what you want.
> 
> 
I don't think Dan's saying that using SORT for searching is a good  
idea, but it's certainly what people do based on my observations,  
too. (In some clients, the sorted list is scrolled to whichever  
message was previously selected, so it's a fast way of finding other  
messages by the same person).

> For the most part I question why anyone would really want to sort  
> on any of the text fields. The only sort I have found useful is  
> sort by size - and then only to help in culling large messages when  
> my quota-limited mailboxes get full.

My client does no sorting at all, so I'm hard pressed to give any  
alternative use-cases. Generally, people seem to like sorting by date  
fields, though again not a text field. I've also seen people who like  
to sort by subject, essentially as a grouping thing - better done  
with threading, but still.

> View filtering can often be done with a single click (e.g.  
> option-click on the from address of a message currently in view  
> causes the view to show only those messages with that address).  
> This is far superior to sorting.

My client does the same (albeit with different clicks involved), and  
I too find it very useful. I'm not sure I'd say "far superior to  
sorting", since I don't see them as related, but I get your drift.

Still, though, the presence of useless SORT keys would only be a  
significant problem if they were especially difficult to implement,  
and the large deployments of SORT indicate that they're not. Even if  
it's only PINE, that's still a pretty vast number of users who may be  
relying on this behaviour.

When I last brought this up, I got the impression that it was  
considered by the vast majority that it was simply too late to change  
FROM to be useful.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Mon, 03 Mar 2008 10:03:20 -0500
From: Cyrus Daboo <[email-masked]>
To: Dan Karp <[email-masked]>, Mark Crispin <[email-masked]>
cc: [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

Hi Dan,

--On March 2, 2008 11:07:50 PM -0800 Dan Karp <[email-masked]> wrote:

> The purpose of sorting in mail clients is so that users can find
> messages they're looking for.

Actually you need to look at your use cases in more detail because a lot of 
times searching is a much better solution than sorting. e.g. the case of 
trying to find email from a particular person - a search is much better. 
Yes you do have to do a little more work to setup the search (type 
something in) rather than the single click sorting requires, but on large 
mailboxes you will usually see what you want immediately without having to 
scroll through the sorted list looking for what you want.

For the most part I question why anyone would really want to sort on any of 
the text fields. The only sort I have found useful is sort by size - and 
then only to help in culling large messages when my quota-limited mailboxes 
get full. The rest of the time I either use no sort, or threading, or view 
filtering (view only the results of a search). View filtering can often be 
done with a single click (e.g. option-click on the from address of a 
message currently in view causes the view to show only those messages with 
that address). This is far superior to sorting. At least that works well 
for me. Others may have different ways of processes their email where 
sorting may make more sense.

-- 
Cyrus Daboo

Date: Mon, 03 Mar 2008 11:22:14 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort  (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

The IESG wrote:

>The IESG is considering the following document again now that 
>important dependencies are ready:
>
>- 'INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS'
>   <draft-ietf-imapext-sort-19.txt> as a Proposed Standard
>  
>
Hi Mark,
I've just reread section 7 of -19 and found some relatively minor things 
that need fixing:

> 7. Internationalization Considerations
>
>     As stated in the introduction, the server SHOULD support the
>     [IMAP-I18N] COMPARATOR extension and follow its rules to perform
>     collations in the SORT and THREAD extensions.

As per your recent suggestion in the IMAPEXT WG, the COMPARATOR 
extension was renamed to I18NLEVEL=2.
The last sentence in section 1 needs to be updated as well.

>     If the server does not support COMPARATOR, strings MUST be collated
>     according to the i;unicode-casemap collation described in
>     [UNICASEMAP].

I think you also mean that support for at least I18NLEVEL=1 is REQUIREd.

>     As described in [IMAP-I18N], strings in charsets other than US-ASCII
>     and UTF-8 MUST be converted to UTF-8 and compared in ascending order
>     according to the selected or active collation algorithm.  If the
>     server does not support the [IMAP-I18N] COMPARATOR extension, the
>     collation algorithm used is the "en;ascii-casemap" collation
>     described in [COMPARATOR].

The last sentence contradict the second paragraph quoted above. I think 
you forgot to delete it in -19.
If for some reason you want to keep it, then please change 
"en;ascii-casemap" to "i;ascii-casemap", as "en;ascii-casemap" is invalid.

I can recommend some replacement text, if you want.

Date: Mon, 3 Mar 2008 11:40:45 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Cc: Dan Karp <[email-masked]>, Mark Crispin <[email-masked]>
Sender: [email-masked]

Dan:

The draft can't be changed any more.

But there's good news. Like you, I have a database, and I've implemented 
SORT. It took less than a day. A part of those hours was wasted time, 
because as you point out, only Mark's client wants the kind of address 
sorting which the draft offers. But it was less than a day. Wasting 
part of a day isn't really something to get excited about. Worse things 
happen every month.

Arnt

Date: Sun, 2 Mar 2008 23:53:04 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dan Karp <[email-masked]>
cc: [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

On Sun, 2 Mar 2008, Dan Karp wrote:
> Once the draft makes it to Proposed Standard, getting a second draft out
> with a variant version of these sorts becomes exponentially harder.

When there is both a de facto standard that has existed for over 10 years, 
and a completely incompatible de jure standard created because some guy 
didn't like the de facto standard, then getting any standard becomes 
impossible.

The SORT specification is infinitely extensible.  The original keys are 
there, and remain there forever.  However, nothing prevents the creation 
of FROM2, FROM.PLUS, ZIMBRAFROM, etc. as extensions to the keys.  That is, 
all new forms of SORT would be a superset.

The only thing that a client need concern itself about with a server is if 
the server implements (at least) the SORT level it desires.  If the client 
just cares about the original, 10+ year old, keys, then any server SORT 
will work, because all SORT is a superset of the original.

Should you get your way, there would be now two incompatible sets of keys. 
There would be the set that has been around for more than a decade, and 
the new incompatible set that *subsets* the original set instead of 
*supersets* it.

Clients would have to know which set is implemented by the server.  If the 
server implements the wrong set, then the client will have to do all the 
work in the client.  The end result is sufficient fear, uncertainty, and 
doubt that nobody will use server-based SORT at all.

> There is a clear consensus in the mail user agent marketplace for how
> address sorts should behave.

Empirical testing of a few clients does not constitute a "concensus".

> New revisions of the legacy servers would advertise both the legacy
> "SORT" and the standard "SORT=BASE".  Legacy clients would see "SORT"
> and behave exactly as before.  New clients would see both "SORT" and
> "SORT=BASE" and act accordingly.

I already explained why this can not work.

SORT=BASE, by definition in the current SORT specification, is a 
*superset* of the 10+ year SORT.

    A server which supports the base-level SORT extension indicates this
    with a capability name which starts with "SORT".  Future,
    upwards-compatible extensions to the SORT extension will all start
    with "SORT", indicating support for this base level.

Consequently, current software does
    if (strcasencmp (cap,"SORT",4)) hasSort = 1;

Your demand has the effect of breaking this software.  The new extension 
would have to be called something like ZIMBRASORT, since the 10+ year old 
SORT specification reserves the entire SORT* extension namespace for 
upwards-compatible extensions.  ZIMBRASORT, by being a subset (and hence 
incompatible) must therefore have a different name.

The result is that, with IMAP sorting forked into two incompatible forms 
of sort, nobody will implement either.

> I believe that your client already does client-side FROM sorting for
> servers that don't advertise SORT.

Yes, and it is as slow as cold molasses in January when such a server is 
encountered.  Users generally don't even try to sort or thread on such 
deficient servers since it is entirely too painful.

I do not understand why you are so insistant upon demanding that 
something that has been in use for 10+ years be removed.  Forks in 
specifications are bad things.

Let me take that back.  Forks in specifications are not bad things; forks 
in specifications are VERY bad things.  The Achilles' heel of UNIX has 
been the multiplicity of forks; and it's not just the forks of SysV vs. 
BSD vs. Linux but incompatible forks within SysV, BSD, and Linux.

When IMAP forked in the past the result was incredible chaos and confusion 
that should never be repeated.  Ever since IMAP has gone to considerable 
effort to maintain upwards compatibility and not fork.  That is why, among 
other things, we have BODY and BODYSTRUCTURE.  That is why IMAP has at 
times chosen ugly syntax and mechanisms, because doing so provided the 
functionality without creating a fork.

If it was difficult to implement the keys in question, that would be one 
thing; but it is utterly trivial to do so.  The necessary framework is 
provided by virtue of the ENVELOPE (a base specification requirement since 
the very beginnings of IMAP) and SUBJECT sorting.  A high school kid can 
implement these keys with that framework in place.

Nothing prevents the addition of the "better" keys that you advocate, 
other than your vaguely expressed fear that the "better" keys won't happen 
if the "not better" keys are allowed to survive.  If the "better" keys 
can't survive that modest hurdle then maybe they really aren't "better" 
after all -- or maybe the entire argument is pointless and you should just 
let sleeping dogs lie.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Sun, 2 Mar 2008 23:07:50 -0800 (PST)
From: Dan Karp <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

Let's step back for a moment.

The purpose of sorting in mail clients is so that users can find
messages they're looking for.

After sorting on INTERNALDATE, the user skips down the "Received" column
to the range that their target messages fall in.  And, voila!  The
messages are there.

After sorting on SUBJECT, the user skips through the results to the set
of messages with the correct normalized subject.  This works because the
user mentally normalizes the Subject header in the same way that the
SORT extension does.  (The transform's easy to do, and years of mail
client use have trained them to do this; they're not looking under 'R'
for "re: Your Brains".)  And, voila!  They find the messages.

But sorting on FROM?  That's where things break down for the draft.

I took a look at 7 different mail clients: Gmail, mutt, Outlook Express,
PC-Pine, Thunderbird, Yahoo! Mail, and Zimbra.  In every single one of
these clients, there is a list of mail messages.  And in that list,
every client has a "From" column.  And every one of those "From" columns
contains the RFC2822 display-name of the From header of the relevant
message, or the RFC2822 addr-spec if there's no display-name.  Every
single one.

Gmail ("search, not sort") doesn't allow you to sort on "From".  But
every other client offers a "From" sort.  And every client *but* PC-Pine
does it exactly the same way: A straight text sort on the data displayed
in the "From" column.  And this makes sense, because a user knows the
name of the author of the message they're trying to find, and they can
skip to the right part of the list.  Things work because (a) the user
knows the sort key of the message, (b) the sort key is displayed in the
user interface, and (c) the list is ordered on that displayed sort key.
It's just as intuitive as the received-date search.

PC-Pine is different.  Looking at the first page of results when I point
it at the INBOX of my test account, I see the following "FROM sorted"
list (names but not sort order have been changed to protect the
innocent):

  N  21 Jun 21 Betsy Avertain                  (787) ...
  N  22 Sep 24 Hurricane Electric, Inc.       (3465) ...
  N  23 Sep 19 Waverly Bindle                  (11K) ...
  N  24 Dec  7 [email-masked]         (12K) ...
  N  25 Feb 23 Brian Pearson                  (5878) ...
  N  26 Jan 24 patrick walton                  (66K) ...
  N  27 Nov  9 build                          (2022) ...
  N  28 Nov 18 Calvin Poutine                 (6275) ...
     29 Feb  8 Leonardo Merman                (2535) ...

Those are the messages that PC-Pine, using the FROM sort rules as
defined in the draft, has decided to sort in the "B-C" range.  Note that
the sort key itself (the local-part of the RFC2822 addr-spec) isn't
displayed at all.  And it's not just PC-Pine: NO CLIENT that I know of
displays that information by default in the message listing; most if not
all can't even be configured to show it.

So if I'm looking for messages from Pat Walton, I'm going to first look
under "P" and not find anything, then wonder if I haven't actually filed
his messages into a different mailbox, then rack my brain until if I'm
lucky I remember that his username is actually "bpw", then skip to
somewhere in the "B"s and start trying to remember if Brian Pearson's
username was "brianp" (in which case I need to scroll up to find the
block of messages from Pat) or "bpearson" (in which case I need to
scroll down).

In order to find a message in this "sorted" list, you have to first
remember a piece of information that the client will not display.  If
you can't remember this piece of information -- and, in the world of
display-name-based autocomplete and correspondents you rarely send
mail to, that's a large and growing probability -- you're completely
hosed.  You then have to guess roughly the right position in the list to
jump to, and then in order to scroll/page you have to again remember a
non-displayed piece of information for each sender in the page you're
looking at.

This is not a joke case or an outlier.  This is reality.  I've
personally had usernames that started with "y.", "li", "da", "dk", "t-",
and "ka", most of them not by choice.

There is a reason that your cell phone's address book sorts by name,
not by phone number.

> I see no reason to remove/deprecate the FROM, TO, and CC sorts.
> 
> You have presented no reason other than your dislike of their
> definition.  The provision of FROM, TO, and CC sorts in no way blocks
> future extensions that offer other forms of sorting these (and other
> fields).

Once the draft makes it to Proposed Standard, getting a second draft out
with a variant version of these sorts becomes exponentially harder.
Getting server and client uptake for that revision is even less likely.
It is better for "the ecosystem" (I cannot believe I just typed that) to
solve the problem just once and to do it as correctly as possible.

There is a clear consensus in the mail user agent marketplace for how
address sorts should behave.  Intentionally specifying a different
algorithm, especially one whose results can be as counterintuitive as
the one in the draft, seems unwise.

> SORT is implemented by at least two clients and at least four servers.
> 
> All of these would have to change to support the proposed crippled
> SORT.  Worse, my client would require substantial new, special case
> code to restore the functionality lost due to crippled SORT; and after
> this work will have worse performance.

Legacy servers such as yours would advertise "SORT".  Legacy clients
looking for "SORT" would find it and use it.  Nothing would break.

New revisions of the legacy servers would advertise both the legacy
"SORT" and the standard "SORT=BASE".  Legacy clients would see "SORT"
and behave exactly as before.  New clients would see both "SORT" and
"SORT=BASE" and act accordingly.

I believe that your client already does client-side FROM sorting for
servers that don't advertise SORT.  Determining whether to do client- or
server-side FROM sorting based on whether the server advertised "SORT"
or just "SORT=BASE" is probably on the order of 10 lines of code.

Date: Tue, 25 Mar 2008 17:21:46 -0400
From: Barry Leiba <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

Cyrus Daboo said the following:
> So perhaps we need a slight variant here:
> 
> /public/comment
> /private/comment
> 
> With these two top-level entry segments, we have a clear distinction 
> between what is shared and what is private. Registered annotations would 
> have to define annotations in each as appropriate. Clients could create 
> unregistered entries in either to achieve the behavior they need. A server 
> that does not support per-user annotations simply does not support /private.

I'm a bit late with this comment, but: That variant works for me.

Barry

Date: Mon, 17 Mar 2008 22:59:22 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>What happened to this?
>
My editorial copy contains the following text:

      Use of system flags or user defined keywords with
      "UNCHANGEDSINCE 0" MUST be treated as an error; the server MUST
      return tagged BAD response to any such request. For all other
      metadata item types, use of UNCHANGEDSINCE with a modification
      sequence of 0 always fails if the metadata item exists.

>Your implementation seems to just ignore
>UNCHANGEDSINCE 0 instead of failing the update.
>
I will fix that.

Date: Mon, 17 Mar 2008 22:52:18 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>One more thing:
>
>   When a message is appended to a mailbox (via the IMAP APPEND command,
>   COPY to the mailbox, or using an external mechanism) the server
>   generates a new modification sequence that is higher than the highest
>   modification sequence of all messages in the mailbox and assigns it
>   to the appended message.
>
>This doesn't say if it's OK to give the same new modseq to new messages
>when copying/appending multiple messages. Since it's atomic I think it
>is.  
>
As it turned out, I've already made this change in my copy of rfc4551bis.

From: The IESG <[email-masked]>
To: IETF-Announce <[email-masked]>
Cc: Internet Architecture Board <[email-masked]>, RFC Editor <[email-masked]>, imapext mailing list <[email-masked]>, imapext chair <[email-masked]>
Subject: Protocol Action: 'Internet Message Access Protocol  Internationalization' to Proposed Standard 
Date: Mon, 17 Mar 2008 15:35:51 -0700 (PDT)
Sender: [email-masked]

The IESG has approved the following document:

- 'Internet Message Access Protocol Internationalization '
   <draft-ietf-imapext-i18n-15.txt> as a Proposed Standard

This document is the product of the Internet Message Access Protocol 
Extension Working Group. 

The IESG contact persons are Lisa Dusseault and Chris Newman.

A URL of this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-i18n-15.txt

Technical Summary
 
This document provides the internationalization building blocks for
several IMAP extensions. It specifies (1) how to return non-English,
non-US-ASCII human-readable text in replies to IMAP commands; (2) how to
provide translations for NAMESPACE prefixes; and  (3) how to use
internationalized comparators in the IMAP SEARCH, SORT, and THREAD
commands.

Working Group Summary

This document is a product of the IMAP Extensions (IMAPEXT) Working Group
and has been extensively reviewed. In addition to agreement on the
technical aspects of the document, the result of discussion was consensus
to separate the LANGUAGE extension (for human-readable text replies and
NAMESPACE prefixes) from the use of comparators and the separation of two
levels of comparator support.

Protocol Quality

The protocol is the final piece in the puzzle for the IMAP SEARCH, SORT,
and THREAD commands. Implementation for those has been ongoing, and the
result of this work has satisified implementors of those extensions.

 Lisa Dusseault reviewed this for the IESG.

Note to RFC Editor

In Section 2, add to the end of the last paragraph:

The I18NLEVEL=1 extension updates SEARCH/SORT/THREAD to use
i;unicode-casemap comparator, as defined in [UCM]. I18NLEVEL=1 is
a simpler version of I18NLEVEL=2 with no ability to select a different
collation.

In Section 3.1, add the following sentence to the end of 2nd paragraph:

IMAP servers SHOULD NOT advertise the LANGUAGE extension
if they discover that they only support "i-default".

In Section 3.1, last paragraph, second sentence:

OLD:
 Clients are urged to issue LANGUAGE before authentication, since some
servers send
         ^^^^^^^^^^^^
 valuable user information as part of authentication (e.g. "password
 is correct, but expired").

NEW:
 Clients SHOULD issue LANGUAGE before authentication, since some servers
send
         ^^^^^^
 valuable user information as part of authentication (e.g. "password
 is correct, but expired").

In Section 3.2, add a new sentence to the end of paragraph 9:

OLD:
 If there aren't any arguments, the server SHOULD send an untagged
 LANGUAGE response listing the languages it supports.  If the server
 is unable to enumerate the list of languages it supports it MAY
 return a tagged NO response to the enumeration request.

NEW:
 If there aren't any arguments, the server SHOULD send an untagged
 LANGUAGE response listing the languages it supports.  If the server
 is unable to enumerate the list of languages it supports it MAY
 return a tagged NO response to the enumeration request.
 If, after receiving a LANGUAGE request, the server discovers
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 that it doesn't support any language other than i-default,
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 it MUST return a tagged NO response to the enumeration request.
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Subject: Re: SEARCHRES
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: ietf-imapext <[email-masked]>
Date: Sun, 16 Mar 2008 18:17:07 +0200
Sender: [email-masked]

--=-Soei3ES6z+W1ny1+6WmQ
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Sun, 2008-03-16 at 14:54 +0000, Alexey Melnikov wrote:

> >>In some cases, the server MAY refuse to save a SEARCH (SAVE)
> >>        result, for example if an internal limit on the number of saved
> >>        results is reached.
> >>   =20
> >>
> >I don't see the point, it just complicates the protocol.
> >
> Some reviewers wanted to have a way to handle DoS on SEARCHRES gracefully=
.
>=20
> While this is allowed, it is up to server implementors to decide if the=20
> server should ever do that.

I just don't see how this can be considered a DoS in any way. If server
thinks there are too many SEARCHRES results stored, it shouldn't have
allowed the user to log in that many times. I can think of a lot easier
ways to DoS an IMAP compliant server than it. Wonder how much memory a
server would take parsing a 10MB mail with maximal number of MIME parts,
or something similar..

> Folks, the document is past AUTH48 already.

Why do I always read these too late.. :)

--=-Soei3ES6z+W1ny1+6WmQ
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-Soei3ES6z+W1ny1+6WmQ--

Date: Sun, 16 Mar 2008 15:22:18 +0000
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Timo Sirainen <[email-masked]>
Subject: Re: SEARCHRES
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> Alexey Melnikov writes:
>
>> It makes no much difference. If we clear the response, the client can 
>> issue "SEARCH NOT $" and produce the same result after that.
>
> Does this mean that
>
>    a login alexey yexela
>    b select inbox

At this point $ = {} (empty set).

>    c search not $

This is legal, even if $ is {}. So this will not return BAD.

> can cause anything other than
>
>    c BAD $ not defined
>
> ?
>
> (I thought I'd ignore this extension, but here I am, arguing about it.)

Date: Sun, 16 Mar 2008 16:18:46 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: SEARCHRES
Cc: Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> It makes no much difference. If we clear the response, the client can 
> issue "SEARCH NOT $" and produce the same result after that.

Does this mean that

    a login alexey yexela
    b select inbox
    c search not $

can cause anything other than

    c BAD $ not defined

?

(I thought I'd ignore this extension, but here I am, arguing about it.)

Arnt

Date: Sun, 16 Mar 2008 15:18:30 +0000
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], Timo Sirainen <[email-masked]>
Subject: Re: SEARCHRES
Sender: [email-masked]

Arnt Gulbrandsen wrote:

> Alexey Melnikov writes:
>
>> Timo Sirainen wrote:
>>
>>>>   Any of the following SEARCH command MUST NOT change the search 
>>>> result
>>>>   variable:
>>>>
>>>>    o A SEARCH command that caused the server to return the BAD tagged
>>>>      response,
>>>
>>> I think it's less dangerous to clear the variable.
>>
>> BAD response typically means that the server couldn't parse the 
>> command. A server implementation might not even know that it was a 
>> SEARCH command.
>
> Suggested language: When the server sends a BAD or NO response, it 
> MUST clear the search result variable.

Folks, the document is past AUTH48 already. This is not a kind of change 
that is worth it so late in the process.

Date: Sun, 16 Mar 2008 16:15:01 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: SEARCHRES
Cc: Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> Timo Sirainen wrote:
>>>   Any of the following SEARCH command MUST NOT change the search result
>>>   variable:
>>>
>>>    o A SEARCH command that caused the server to return the BAD tagged
>>>      response,
>>
>> I think it's less dangerous to clear the variable.
>
> BAD response typically means that the server couldn't parse the 
> command. A server implementation might not even know that it was a 
> SEARCH command.

Suggested language: When the server sends a BAD or NO response, it MUST 
clear the search result variable.

Arnt

Date: Sun, 16 Mar 2008 14:54:50 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: ietf-imapext <[email-masked]>
Subject: Re: SEARCHRES
Sender: [email-masked]

Timo Sirainen wrote:

>(wonder which of the IMAP lists would be the most appropriate for this..)
>  
>
>>   Any of the following SEARCH command MUST NOT change the search result
>>   variable:
>>
>>    o A SEARCH command that caused the server to return the BAD tagged
>>      response,
>>    
>>
>I think it's less dangerous to clear the variable.
>
BAD response typically means that the server couldn't parse the command. 
A server implementation might not even know that it was a SEARCH command.

>For example:
>
>1 search return (save) all
>2 fetch $ flags
>
>3 search return (save) subject hello world
>4 store $ +flags.silent \deleted
>5 expunge
>
>3 exposes a buggy client and returns BAD. 5 then expunges all messages.
>  
>
It makes no much difference. If we clear the response, the client can 
issue "SEARCH NOT $" and produce the same result after that.

>>In some cases, the server MAY refuse to save a SEARCH (SAVE)
>>        result, for example if an internal limit on the number of saved
>>        results is reached.
>>    
>>
>I don't see the point, it just complicates the protocol.
>
Some reviewers wanted to have a way to handle DoS on SEARCHRES gracefully.

While this is allowed, it is up to server implementors to decide if the 
server should ever do that.
Isode's server never do this.

>The result
>can't be larger than the number of messages in a mailbox. The server has
>to keep all kinds of data related to the messages already, so one search
>result shouldn't be that big of a deal. I can think of two usable ways
>to store the result, the worst case space usages would be:
>
>1. Array of { uint32_t uid1, uid2 }: 2 * 4 * message_count bytes = 78kB
>for 10000 message mailbox.
>
>2. Bitmask of sequences: message_count/8 bytes = 1,2 kB for 10000
>message mailbox.
>  
>

Subject: SEARCHRES
From: Timo Sirainen <[email-masked]>
To: ietf-imapext <[email-masked]>
Date: Sun, 16 Mar 2008 09:36:00 +0200
Sender: [email-masked]

--=-6M7b+vc4Yv4EYsR32Hv1
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

(wonder which of the IMAP lists would be the most appropriate for this..)

>    Any of the following SEARCH command MUST NOT change the search result
>    variable:
>=20
>     o A SEARCH command that caused the server to return the BAD tagged
>       response,

I think it's less dangerous to clear the variable. For example:

1 search return (save) all
2 fetch $ flags

3 search return (save) subject hello world
4 store $ +flags.silent \deleted
5 expunge

3 exposes a buggy client and returns BAD. 5 then expunges all messages.

> In some cases, the server MAY refuse to save a SEARCH (SAVE)
>         result, for example if an internal limit on the number of saved
>         results is reached.

I don't see the point, it just complicates the protocol. The result
can't be larger than the number of messages in a mailbox. The server has
to keep all kinds of data related to the messages already, so one search
result shouldn't be that big of a deal. I can think of two usable ways
to store the result, the worst case space usages would be:

1. Array of { uint32_t uid1, uid2 }: 2 * 4 * message_count bytes =3D 78kB
for 10000 message mailbox.

2. Bitmask of sequences: message_count/8 bytes =3D 1,2 kB for 10000
message mailbox.

--=-6M7b+vc4Yv4EYsR32Hv1
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-6M7b+vc4Yv4EYsR32Hv1--

Subject: Re: CONDSTORE + ESEARCH
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Sun, 16 Mar 2008 08:24:00 +0200
Sender: [email-masked]

--=-yvu7WnmY+AzFhbkbaqvq
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Sun, 2008-03-16 at 08:14 +0200, Timo Sirainen wrote:
> Don't seem to work well together?=20
>=20
> 1 SEARCH RETURN (ALL) MODSEQ 1
> * ESEARCH (tag "1") ALL 1:10
>=20
> MODSEQ <highest> is missing in the reply.

Sorry, it is defined there after all. It's just Alexey's implementation
that's not sending it. :)

--=-yvu7WnmY+AzFhbkbaqvq
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-yvu7WnmY+AzFhbkbaqvq--

Subject: CONDSTORE + ESEARCH
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Date: Sun, 16 Mar 2008 08:14:42 +0200
Sender: [email-masked]

--=-r49VEd19tK4a15FeUhrb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Don't seem to work well together?=20

1 SEARCH RETURN (ALL) MODSEQ 1
* ESEARCH (tag "1") ALL 1:10

MODSEQ <highest> is missing in the reply.

--=-r49VEd19tK4a15FeUhrb
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-r49VEd19tK4a15FeUhrb--

Date: Sat, 15 Mar 2008 23:40:12 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Alexey,
>
> --On March 14, 2008 9:50:30 PM +0000 Alexey Melnikov 
> <[email-masked]> wrote:
>
>>> If you want searching, then write an extension to this extension that
>>> does use LISTEXT.
>>
>> But it is possible to search for everything else in IMAP (headers, text
>> in body parts, flags, etc.), so lack of search would be awkward.
>
> That maybe true at a message level, but its not true at a mailbox 
> level. i.e. without LISTEXT there is no way to search for mailboxes 
> containing unseen messages. Even with LISTEXT there is no way to 
> search for mailboxes on which I have the 'w' ACL right.

That is a bad example, as ACLs are not exposed in LISTEXT. Besides I 
don't see much point searching for access rights, they are different 
objects from human readable strings.

> So I don't agree that lack of search in this case is a hinderance. If 
> it is, then we extend METADATA with LISTEXT, but that won't be present 
> in the "base" METADATA spec.

I am not sure I will survive this change again :-).
Note that that searching for metadata doesn't necessarily need LISTEXT.

Date: Sat, 15 Mar 2008 13:04:41 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Hi Alexey,

--On March 14, 2008 9:50:30 PM +0000 Alexey Melnikov 
<[email-masked]> wrote:

>> If you want searching, then write an extension to this extension that
>> does use LISTEXT.
>
> But it is possible to search for everything else in IMAP (headers, text
> in body parts, flags, etc.), so lack of search would be awkward.

That maybe true at a message level, but its not true at a mailbox level. 
i.e. without LISTEXT there is no way to search for mailboxes containing 
unseen messages. Even with LISTEXT there is no way to search for mailboxes 
on which I have the 'w' ACL right. So I don't agree that lack of search in 
this case is a hinderance. If it is, then we extend METADATA with LISTEXT, 
but that won't be present in the "base" METADATA spec.

-- 
Cyrus Daboo

Date: Sat, 15 Mar 2008 16:24:49 +0000
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> Let me propose some RFC editor notes based on discussions in this 
> thread and some other comments from IESG:

As there were no objections to my RFC editor notes, I will recommend 
them to Lisa, so that we can get the document published. Thank you everyone.

Date: Sat, 15 Mar 2008 13:51:13 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>On Thu, 2008-03-13 at 18:31 +0200, Timo Sirainen wrote:
>  
>
>>On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>>    
>>
>>>Timo Sirainen wrote:
>>>      
>>>
>>>>What happened to this?
>>>>        
>>>>
>>>I need to publish CONDSTOREbis draft at some point which contains  
>>>the fix. I don't have time or energy to do this now. I should find  
>>>some time once all my Lemonade drafts are approved for publication.
>>>      
>>>
>>Are there going to be other changes?
>>    
>>
>One more thing:
>
>   When a message is appended to a mailbox (via the IMAP APPEND command,
>   COPY to the mailbox, or using an external mechanism) the server
>   generates a new modification sequence that is higher than the highest
>   modification sequence of all messages in the mailbox and assigns it
>   to the appended message.
>
>This doesn't say if it's OK to give the same new modseq to new messages
>when copying/appending multiple messages. Since it's atomic I think it
>is.
>  
>
Yes, it is Ok to assign the same modseq to multiple messages 
injectec/appended atomically. This was the intent.

Date: Sat, 15 Mar 2008 06:30:17 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>, Cyrus Daboo <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

> > If you want searching, then write an extension to this extension 
> > that does use LISTEXT.
> 
> But it is possible to search for everything else in IMAP (headers,
> text in body parts, flags, etc.), so lack of search would be awkward.

I see the point, although I honestly don't see the use case.

But even if you do introduce search functionality (and the LISTEXT
stuff from the previous draft went way beyond the simple "find this text
string" searching available on all those other pieces of data), you'd
still have to make awkward exceptions for non-text data.

(And, before we risk going there, I'd also like to avoid requiring
I18NLEVEL=2.)

Date: Sat, 15 Mar 2008 02:01:16 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

> On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>
>> Timo Sirainen wrote:
>>
>>> What happened to this?
>>
>> I need to publish CONDSTOREbis draft at some point which contains  
>> the fix. I don't have time or energy to do this now. I should find  
>> some time once all my Lemonade drafts are approved for publication.
>
> Are there going to be other changes? The main problem I currently 
> have  implementing CONDSTORE is that I'd rather not want to figure 
> out  HIGHESTMODSEQ value when selecting a mailbox unless I know the 
> client  wants to use CONDSTORE. I started to wonder if it would work 
> nicely if  a SELECT mailbox returned [NOMODSEQ] but the first 
> CONDSTORE enabling  command would cause * OK [HIGHESTMODSEQ n] to be 
> sent.

That should be Ok, especially if clients try to use "ENABLE QRESYNC" or 
"ENABLE CONDSTORE".

However I don't think a client that sees NOMODSEQ is going to issue any 
CONDSTORE enabling command.

> Which also brings to my mind.. I didn't really understand why the * 
> OK  [HIGHESTMODSEQ n] would be sent again after it was already sent as 
> a  reply to SELECT. This is described in the spec example 5 and also  
> happens in your implementation.

Because HIGHESTMODSEQ might have changed between the moment when the 
mailbox was opened and the moment when the client issued a CONDSTORE 
enabling command.

Date: Fri, 14 Mar 2008 21:45:36 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

>Hi,
>So I have been thinking some more about the issue of removing attributes 
>from METADATA. I think that will introduce some problems with how we can 
>then do .priv and .shared. My original idea was to allow ".priv" and 
>".shared" to be tacked on the end of an entry name to have the appropriate 
>value returned. This is ugly in that it then requires the somewhat 
>arbitrary restriction that entry names themselves cannot end in .priv or 
>.shared.
>
>So here is a more radical proposal: do away with .priv and .shared as well. 
>Instead each annotation would define whether it is per-user or shared. So 
>on a mailbox we would initially define:
>
>/comment-public
>/comment-private
>
>Where "public" would be "shared" and "private" would be per-user.
>  
>
I really really really dislike that, as now there is no way to 
algorithmically determine which annotation is public and which one is 
private. Unless you meant that each annotation will have either 
"-public" or "-private"?
A better way would be to define two separate hierarchies: /public and 
/private, e.g. /private/comment.

Date: Sat, 15 Mar 2008 01:29:52 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: Randall Gellens <[email-masked]>, Dan Karp <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Removing attributes, was Re: Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

>Hi Randall,
>
>--On March 14, 2008 7:17:47 AM -0700 Randall Gellens <[email-masked]> 
>wrote:
>
>  
>
>>We were trying to maintain some level of similarity with ANNOTATE, and
>>also provide the client with a way to see the size before fetching
>>something.
>>    
>>
>Well ANNOTATE is only experimental and I'm not sure anyone (except perhaps 
>Ken @ CMU) has done anything with it. So limiting ourselves to ANNOTATE 
>syntax should not be a requirement IMHO.
>
>As far as the size goes, we could easily add an option to GETMETADATA that 
>would allow a client to limit the size of data returned and add an option 
>to the response to indicate when a truncation has occurred, e.g.:
>
>           C: a GETMETADATA (MAXSIZE 1024) /comment
>           S: * METADATA (TRUNCATED 2048) /comment "My comment ...."
>           S: a OK GETMETADATA complete
>  
>
I can live with that, as long as this is a part of 
METADATA/METADATA-SERVER and not yet another extension.

>So in this example the client asks the server to return no more than 1024 
>octets. The /comment value returned is truncated at 1024 octets, and the 
>server also tells the client that the actual size of /comment is 2048.
>
>I think this is more efficient if you make the assumption that for the most 
>part annotations are going to be small. i.e. the client doesn't have to do 
>two roundtrips to get an annotation (one to check the size, the other to 
>get the data).
>
That is a good point.

>Instead it just does one with MAXSIZE and then, only if 
>there was truncation, can do another get if it really want to.
>  
>

Date: Fri, 14 Mar 2008 21:50:30 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Alexey,
>
> --On March 13, 2008 10:03:44 PM +0000 Alexey Melnikov 
> <[email-masked]> wrote:
>
>>> 1) Remove the dependency on LISTEXT. This basically means turning the
>>> clock  back to the original ANNOTATEMORE extension. So the GETMETADATA
>>> command  would be used to retrieve mailbox annotations as well as 
>>> server
>>> annotations.
>>
>> That is fine with me.
>>
>> You forgot handling on comparators in GETMETADATA.
>
> Actually I am not proposing that GETMETADATA provide a way to search 
> annotations (which is what LISTEXT does).

But LISTEXT is not relevant to per-server annotations (i.e. 
METADATA-SERVER).

> If you want searching, then write an extension to this extension that 
> does use LISTEXT.

But it is possible to search for everything else in IMAP (headers, text 
in body parts, flags, etc.), so lack of search would be awkward.

Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Sat, 15 Mar 2008 05:54:28 +0200
Sender: [email-masked]

--=-8S8FqGkVyj9chmQv2Bku
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Thu, 2008-03-13 at 18:31 +0200, Timo Sirainen wrote:
> On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>=20
> > Timo Sirainen wrote:
> >
> >> What happened to this?
> >>
> > I need to publish CONDSTOREbis draft at some point which contains =20
> > the fix. I don't have time or energy to do this now. I should find =20
> > some time once all my Lemonade drafts are approved for publication.
>=20
> Are there going to be other changes?

One more thing:

   When a message is appended to a mailbox (via the IMAP APPEND command,
   COPY to the mailbox, or using an external mechanism) the server
   generates a new modification sequence that is higher than the highest
   modification sequence of all messages in the mailbox and assigns it
   to the appended message.

This doesn't say if it's OK to give the same new modseq to new messages
when copying/appending multiple messages. Since it's atomic I think it
is.

--=-8S8FqGkVyj9chmQv2Bku
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-8S8FqGkVyj9chmQv2Bku--

Date: Fri, 14 Mar 2008 13:26:25 -0400
From: Cyrus Daboo <[email-masked]>
To: Dan Karp <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

Hi Dan,

--On March 14, 2008 8:12:34 AM -0700 Dan Karp <[email-masked]> wrote:

>>   /comment-public
>>   /comment-private
>>
>> Where "public" would be "shared" and "private" would be per-user.
>
> I really like this, as there are very, very few entries that are
> meaningful as both private and public.
>
> How does this affect the statement from section 3.3 of the old document:
>
>    If the METADATA extension is present, support for shared annotations
>    is REQUIRED, whilst support for private annotations is OPTIONAL.
>

If server vendors still feel they need to have the option to deny per-user 
annotations, then we need to add some suitable response codes that would 
indicate lack of support for a registered per-user annotation.

Another thing: with this new approach there is no way for a client to 
create arbitrary (unregistered) entry names on a shared or per-user basis, 
given that the server determines what is shared and per-user based on 
registrations.

So perhaps we need a slight variant here:

/public/comment
/private/comment

With these two top-level entry segments, we have a clear distinction 
between what is shared and what is private. Registered annotations would 
have to define annotations in each as appropriate. Clients could create 
unregistered entries in either to achieve the behavior they need. A server 
that does not support per-user annotations simply does not support /private.

-- 
Cyrus Daboo

Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
From: Curtis King <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Date: Fri, 14 Mar 2008 10:44:15 -0600
Sender: [email-masked]

On 14-Mar-08, at 9:05 AM, Cyrus Daboo wrote:

> Hi,
> So I have been thinking some more about the issue of removing  
> attributes
> from METADATA. I think that will introduce some problems with how we  
> can
> then do .priv and .shared. My original idea was to allow ".priv" and
> ".shared" to be tacked on the end of an entry name to have the  
> appropriate
> value returned. This is ugly in that it then requires the somewhat
> arbitrary restriction that entry names themselves cannot end  
> in .priv or
> .shared.
>
> So here is a more radical proposal: do away with .priv and .shared  
> as well.
> Instead each annotation would define whether it is per-user or  
> shared. So
> on a mailbox we would initially define:
>
> /comment-public
> /comment-private
>
> Where "public" would be "shared" and "private" would be per-user.

I really like this idea.

ck

Date: Fri, 14 Mar 2008 08:12:34 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Even more drastic cut-backs in METADATA
Sender: [email-masked]

> So here is a more radical proposal: do away with .priv and .shared as
> well.  Instead each annotation would define whether it is per-user or
> shared.  So on a mailbox we would initially define:
> 
>   /comment-public
>   /comment-private
> 
> Where "public" would be "shared" and "private" would be per-user.

I really like this, as there are very, very few entries that are
meaningful as both private and public.

How does this affect the statement from section 3.3 of the old document:

   If the METADATA extension is present, support for shared annotations
   is REQUIRED, whilst support for private annotations is OPTIONAL.

Date: Fri, 14 Mar 2008 11:05:19 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Even more drastic cut-backs in METADATA
Sender: [email-masked]

Hi,
So I have been thinking some more about the issue of removing attributes 
from METADATA. I think that will introduce some problems with how we can 
then do .priv and .shared. My original idea was to allow ".priv" and 
".shared" to be tacked on the end of an entry name to have the appropriate 
value returned. This is ugly in that it then requires the somewhat 
arbitrary restriction that entry names themselves cannot end in .priv or 
.shared.

So here is a more radical proposal: do away with .priv and .shared as well. 
Instead each annotation would define whether it is per-user or shared. So 
on a mailbox we would initially define:

/comment-public
/comment-private

Where "public" would be "shared" and "private" would be per-user.

-- 
Cyrus Daboo

Date: Fri, 14 Mar 2008 10:57:32 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Hi Alexey,

--On March 13, 2008 10:03:44 PM +0000 Alexey Melnikov 
<[email-masked]> wrote:

>> 1) Remove the dependency on LISTEXT. This basically means turning the
>> clock  back to the original ANNOTATEMORE extension. So the GETMETADATA
>> command  would be used to retrieve mailbox annotations as well as server
>> annotations.
>>
>>
> That is fine with me.
>
> You forgot handling on comparators in GETMETADATA.

Actually I am not proposing that GETMETADATA provide a way to search 
annotations (which is what LISTEXT does). If you want searching, then write 
an extension to this extension that does use LISTEXT.

-- 
Cyrus Daboo

Date: Fri, 14 Mar 2008 10:54:41 -0400
From: Cyrus Daboo <[email-masked]>
To: Randall Gellens <[email-masked]>, Dan Karp <[email-masked]>
cc: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Removing attributes, was Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Hi Randall,

--On March 14, 2008 7:17:47 AM -0700 Randall Gellens <[email-masked]> 
wrote:

> We were trying to maintain some level of similarity with ANNOTATE, and
> also provide the client with a way to see the size before fetching
> something.

Well ANNOTATE is only experimental and I'm not sure anyone (except perhaps 
Ken @ CMU) has done anything with it. So limiting ourselves to ANNOTATE 
syntax should not be a requirement IMHO.

As far as the size goes, we could easily add an option to GETMETADATA that 
would allow a client to limit the size of data returned and add an option 
to the response to indicate when a truncation has occurred, e.g.:

           C: a GETMETADATA (MAXSIZE 1024) /comment
           S: * METADATA (TRUNCATED 2048) /comment "My comment ...."
           S: a OK GETMETADATA complete

So in this example the client asks the server to return no more than 1024 
octets. The /comment value returned is truncated at 1024 octets, and the 
server also tells the client that the actual size of /comment is 2048.

I think this is more efficient if you make the assumption that for the most 
part annotations are going to be small. i.e. the client doesn't have to do 
two roundtrips to get an annotation (one to check the size, the other to 
get the data). Instead it just does one with MAXSIZE and then, only if 
there was truncation, can do another get if it really want to.

-- 
Cyrus Daboo

Date: Fri, 14 Mar 2008 07:47:28 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Randall Gellens <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>, Randall Gellens <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

> I'm not saying these are must-have elements, but I'm not sure taking 
> them out will make much difference in ease of implementation.  Are 
> you saying it will?  Is this because of parsing responses, or 
> something else?

Attributes make the protocol a moderate amount more complex, with
relatively little benefit back to the client.  (They also give a false
sense of extensibility.)

Date: Fri, 14 Mar 2008 15:37:05 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: [lemonade] Taking the knife to METADATA
Cc: Randall Gellens <[email-masked]>, Dan Karp <[email-masked]>, Cyrus Daboo <[email-masked]>
Sender: [email-masked]

Randall Gellens writes:
> I'm not saying these are must-have elements, but I'm not sure taking 
> them out will make much difference in ease of implementation.  Are 
> you saying it will?

Matching size was difficult, the rest was easy. And AFAICT 
searching/matching doesn't apply to metadata, only to annotate.

Arnt

Date: Fri, 14 Mar 2008 07:17:47 -0700
To: Dan Karp <[email-masked]>, Randall Gellens <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Cc: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Sender: [email-masked]

At 6:48 AM -0700 3/14/08, Dan Karp wrote:

>   > >  2) Remove attributes. i.e. 'value' and 'size' attributes will 
> be removed.
>>  >  There will only be a value. The .priv and .shared suffixes will then be
>>  >  applied to the entry names.
>>
>>  Does this really make a difference?
>
>  If there's no real difference, then simpler is better, no?

We were trying to maintain some level of similarity with ANNOTATE, 
and also provide the client with a way to see the size before 
fetching something.

I'm not saying these are must-have elements, but I'm not sure taking 
them out will make much difference in ease of implementation.  Are 
you saying it will?  Is this because of parsing responses, or 
something else?

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Auntie Em, Hate you; Hate Kansas; Taking the dog.  --Dorothy.

Date: Fri, 14 Mar 2008 06:48:36 -0700 (PDT)
From: Dan Karp <[email-masked]>
To: Randall Gellens <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

> >  2) Remove attributes. i.e. 'value' and 'size' attributes will be removed.
> >  There will only be a value. The .priv and .shared suffixes will then be
> >  applied to the entry names.
> 
> Does this really make a difference?

If there's no real difference, then simpler is better, no?

Date: Fri, 14 Mar 2008 05:06:36 -0700
To: Cyrus Daboo <[email-masked]>, IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
From: Randall Gellens <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

At 5:54 PM -0400 3/13/08, Cyrus Daboo wrote:

>  1) Remove the dependency on LISTEXT. This basically means turning the clock
>  back to the original ANNOTATEMORE extension. So the GETMETADATA command
>  would be used to retrieve mailbox annotations as well as server annotations.

As I recall, the original concern was that using GETMETADATA for 
mailbox annotations would be confusing?  I have no problem with the 
change, but I wanted to be sure we understood why it was changed in 
the first place.

>  2) Remove attributes. i.e. 'value' and 'size' attributes will be removed.
>  There will only be a value. The .priv and .shared suffixes will then be
>  applied to the entry names.

Does this really make a difference?
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
We like because, we love although.

Cc: Alexey Melnikov <[email-masked]>, IMAP Extensions WG <[email-masked]>
From: Timo Sirainen <[email-masked]>
To: Bron Gondwana <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Date: Fri, 14 Mar 2008 03:46:33 +0200
Sender: [email-masked]

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--Apple-Mail-19--481272957
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

On Mar 14, 2008, at 2:15 AM, Bron Gondwana wrote:

>> Are there going to be other changes? The main problem I currently  
>> have
>> implementing CONDSTORE is that I'd rather not want to figure out
>> HIGHESTMODSEQ value when selecting a mailbox unless I know the  
>> client wants
>> to use CONDSTORE. I started to wonder if it would work nicely if a  
>> SELECT
>> mailbox returned [NOMODSEQ] but the first CONDSTORE enabling  
>> command would
>> cause * OK [HIGHESTMODSEQ n] to be sent.
>
> How do you calculate the MODSEQ if you're not using some sort of fast,
> atomically incrementable counter?

Dovecot writes all metadata changes to a transaction log. I use the  
log file number + offset for the change as the modseq. When CONDSTORE  
is first enabled I go through the existing log file(s) and calculate  
modseqs for each message. Modseqs are sometimes later stored to main  
index so this scanning doesn't have to be done all the time.

So the main problem is that before CONDSTORE is enabled, I don't keep  
track of the offset of last flag change (or expunge after QRESYNC is  
implemented) and I can't return a reliable HIGHESTMODSEQ. This is of  
course fixable in a few different ways, but if there are no real  
clients that delay announcing their CONDSTORE needs it seems kind of a  
waste of effort.

--Apple-Mail-19--481272957
content-type: application/pgp-signature; x-mac-type=70674453;
	name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

--Apple-Mail-19--481272957--

Date: Fri, 14 Mar 2008 11:15:35 +1100
From: Bron Gondwana <[email-masked]>
To: Timo Sirainen <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Organization: brong.net
Sender: [email-masked]

On Thu, Mar 13, 2008 at 06:31:34PM +0200, Timo Sirainen wrote:
> On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:
>
>> Timo Sirainen wrote:
>>
>>> What happened to this?
>>>
>> I need to publish CONDSTOREbis draft at some point which contains the fix. 
>> I don't have time or energy to do this now. I should find some time once 
>> all my Lemonade drafts are approved for publication.
>
> Are there going to be other changes? The main problem I currently have 
> implementing CONDSTORE is that I'd rather not want to figure out 
> HIGHESTMODSEQ value when selecting a mailbox unless I know the client wants 
> to use CONDSTORE. I started to wonder if it would work nicely if a SELECT 
> mailbox returned [NOMODSEQ] but the first CONDSTORE enabling command would 
> cause * OK [HIGHESTMODSEQ n] to be sent.

How do you calculate the MODSEQ if you're not using some sort of fast,
atomically incrementable counter?

I guess it would be viable to store a "changed timestamp" or something,
and then lazily calculate a modseq only when something using CONDSTORE
viewed the mailbox - but that sounds pretty messy and potentially
expensive.  I've only read the Cyrus implementation, but it makes sense
in that it has a per-mailbox highestmodseq which gets updated while
the mailbox index is locked (hence the atomic bit) and the index records
themselves all have a modseq value which gets created/updated on any
store/append action.

My initial reaction to CONDSTORE when I saw it go in Cyrus was lukewarm,
but having recently actually tried to figure out the distribution of
bandwidth use on our servers (FastMail.FM) we have discovered that IMAP
TAG UID FETCH 1:* (FLAGS) calls are a huge part of our traffic.  It's a
big pain that in vanilla IMAP there's no way for a client to say "tell
me what has changed" on a 200k message mailbox.  I'd like to see
CONDSTORE used more generally now!

Regards,

Bron.

Date: Thu, 13 Mar 2008 22:03:44 +0000
From: Alexey Melnikov <[email-masked]>
To: Cyrus Daboo <[email-masked]>
CC: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Re: [lemonade] Taking the knife to METADATA
Sender: [email-masked]

Cyrus Daboo wrote:

>Hi folks,
>There have been several discussions at the IETF meeting about METADATA. Its 
>clear that implementers want a "lighter" version. Here are some proposals 
>based on those discussions and other comments during last call:
>
>1) Remove the dependency on LISTEXT. This basically means turning the clock 
>back to the original ANNOTATEMORE extension. So the GETMETADATA command 
>would be used to retrieve mailbox annotations as well as server annotations.
>  
>
That is fine with me.

You forgot handling on comparators in GETMETADATA.

>2) Remove attributes. i.e. 'value' and 'size' attributes will be removed. 
>There will only be a value. The .priv and .shared suffixes will then be 
>applied to the entry names.
>  
>
I am not convinced this is needed. I am really concerned about people 
putting big annotations on the server and then clients being flooded 
with traffic.

>3) Entries are case insensitive.
>  
>
I am neutral on this one. I can be convinced both ways.

>4) Wildcards not allowed in SETMETADATA.
>  
>
That would be fine with me.

>5) Only register /comment for both server and mailbox annotations, and 
>/admin for server annotations in this document.
>  
>
+1.

>Please comment on these so we can move forward with this extension, thanks.
>  
>

Some other things that need to be addressed:

Clarify access controls for various metadata items.

Date: Thu, 13 Mar 2008 17:54:56 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAPEXT <[email-masked]>, Lemonade <[email-masked]>
Subject: Taking the knife to METADATA
Sender: [email-masked]

Hi folks,
There have been several discussions at the IETF meeting about METADATA. Its 
clear that implementers want a "lighter" version. Here are some proposals 
based on those discussions and other comments during last call:

1) Remove the dependency on LISTEXT. This basically means turning the clock 
back to the original ANNOTATEMORE extension. So the GETMETADATA command 
would be used to retrieve mailbox annotations as well as server annotations.

2) Remove attributes. i.e. 'value' and 'size' attributes will be removed. 
There will only be a value. The .priv and .shared suffixes will then be 
applied to the entry names.

3) Entries are case insensitive.

4) Wildcards not allowed in SETMETADATA.

5) Only register /comment for both server and mailbox annotations, and 
/admin for server annotations in this document.

Please comment on these so we can move forward with this extension, thanks.

-- 
Cyrus Daboo

To: [email-masked]
Cc: [email-masked]
From: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-sort-20.txt 
Date: Thu, 13 Mar 2008 14:30:01 -0700 (PDT)
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS
	Author(s)	: M. Crispin, K. Murchison
	Filename	: draft-ietf-imapext-sort-20.txt
	Pages		: 0
	Date		: 2008-3-13
	
This document describes the base-level server-based sorting and
   threading extensions to the [IMAP] protocol.  These extensions
   provide substantial performance improvements for IMAP clients which
   offer sorted and threaded views.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-20.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Message/External-body;
	name="draft-ietf-imapext-sort-20.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--NextPart--

Cc: IMAP Extensions WG <[email-masked]>
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Date: Thu, 13 Mar 2008 18:31:34 +0200
Sender: [email-masked]

This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--Apple-Mail-18--514571583
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit

On Mar 13, 2008, at 5:57 PM, Alexey Melnikov wrote:

> Timo Sirainen wrote:
>
>> What happened to this?
>>
> I need to publish CONDSTOREbis draft at some point which contains  
> the fix. I don't have time or energy to do this now. I should find  
> some time once all my Lemonade drafts are approved for publication.

Are there going to be other changes? The main problem I currently have  
implementing CONDSTORE is that I'd rather not want to figure out  
HIGHESTMODSEQ value when selecting a mailbox unless I know the client  
wants to use CONDSTORE. I started to wonder if it would work nicely if  
a SELECT mailbox returned [NOMODSEQ] but the first CONDSTORE enabling  
command would cause * OK [HIGHESTMODSEQ n] to be sent.

Which also brings to my mind.. I didn't really understand why the * OK  
[HIGHESTMODSEQ n] would be sent again after it was already sent as a  
reply to SELECT. This is described in the spec example 5 and also  
happens in your implementation.

--Apple-Mail-18--514571583
content-type: application/pgp-signature; x-mac-type=70674453;
	name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit

--Apple-Mail-18--514571583--

Date: Thu, 13 Mar 2008 15:57:21 +0000
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
Sender: [email-masked]

Timo Sirainen wrote:

>What happened to this?
>
I need to publish CONDSTOREbis draft at some point which contains the 
fix. I don't have time or energy to do this now. I should find some time 
once all my Lemonade drafts are approved for publication.

>Your implementation seems to just ignore UNCHANGEDSINCE 0 instead of failing the update.
>  
>
It is possible, I need to check the code.

Subject: Re: CONDSTORE: STORE UNCHANGEDSINCE 0
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Thu, 13 Mar 2008 10:49:41 +0200
Sender: [email-masked]

--=-blAZ9GWAVWOpj816ro5X
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

What happened to this? Your implementation seems to just ignore
UNCHANGEDSINCE 0 instead of failing the update.

On Thu, 2006-11-09 at 19:15 -0800, Alexey Melnikov wrote:
> I think the following text in section 3.2 of RFC 4551
>=20
>  >      Use of UNCHANGEDSINCE with a modification sequence of 0 always
>  >      fails if the metadata item exists.  A system flag MUST always be
>  >      considered existent, whether it was set or not.
>=20
> is broken:
>=20
> 1) It is not clear why system flags have to be special
> 2) A flag existence is the same as have the flag set. Testing for flag=20
> existence can already be done using the FETCH (FLAGS) command.
>=20
> There are two ways to fix it:
>=20
> 1). Remove the second sentence to remove special casing for system=20
> flags. This command would allow people to test if a particular flag or a=20
> set of flags is set on a message. However this would leave STORE -FLAGS=20
> UNCHANGEDSINCE somewhat meaningless.
>=20
> 2). Replace the second sentence to say that this command always fail=20
> with tagged NO for both system and user defined flags. Note that this=20
> doesn't affect annotations.
>=20
> The rough consensus during the IMAPEXT meeting was to go with option 2.
> If you have any objections to this, please speak up now.
>=20
> Regards,
> Alexey

--=-blAZ9GWAVWOpj816ro5X
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-blAZ9GWAVWOpj816ro5X--

Date: Tue, 11 Mar 2008 21:25:17 +0000
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Let me propose some RFC editor notes based on discussions in this thread 
and some other comments from IESG:

In Section 2, add to the end of the last paragraph:

  The I18NLEVEL=1 extension updates SEARCH/SORT/THREAD to use
  i;unicode-casemap comparator, as defined in [UCM]. I18NLEVEL=1 is
  a simpler version of I18NLEVEL=2 with no ability to select a different
  collation.

In Section 3.1, add the following sentence to the end of 2nd paragraph:

 IMAP servers SHOULD NOT advertise the LANGUAGE extension
 if they discover that they only support "i-default".

In Section 3.1, last paragraph, second sentence:

OLD:
   Clients are urged to issue LANGUAGE before authentication, since some 
servers send
           ^^^^^^^^^^^^
   valuable user information as part of authentication (e.g. "password
   is correct, but expired").

NEW:
   Clients SHOULD issue LANGUAGE before authentication, since some 
servers send
           ^^^^^^
   valuable user information as part of authentication (e.g. "password
   is correct, but expired").

In Section 3.2, add a new sentence to the end of paragraph 9:

OLD:
   If there aren't any arguments, the server SHOULD send an untagged
   LANGUAGE response listing the languages it supports.  If the server
   is unable to enumerate the list of languages it supports it MAY
   return a tagged NO response to the enumeration request.

NEW:
   If there aren't any arguments, the server SHOULD send an untagged
   LANGUAGE response listing the languages it supports.  If the server
   is unable to enumerate the list of languages it supports it MAY
   return a tagged NO response to the enumeration request.
   If, after receiving a LANGUAGE request, the server discovers
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   that it doesn't support any language other than i-default,
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   it MUST return a tagged NO response to the enumeration request.
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Date: Sat, 08 Mar 2008 18:21:21 +0000
From: Alexey Melnikov <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Pete Resnick wrote:

> On 3/7/08 at 1:03 PM +0000, Alexey Melnikov wrote:
>
>> I've suggested addition of the following text:
>>   If the server discovers that it doesn't support any language other
>>   than i-default, it MUST return a tagged NO response to the
>>   enumeration request.
>
> If we're going to add that, also saying that one MUST NOT advertise 
> language if you don't support other than i-default is silly.

It is not silly. It is a better server behavior towards clients. I 
understand it might be too strong for some implementation. What people 
think about saying "servers SHOULD NOT advertise the LANGUAGE extension 
..."?

> If the client must deal with the case of a NO response to the 
> enumeration request, then it won't have a problem.

True.

> And I think Ned's point about not knowing until after CAPABILITIES 
> time that you don't have more than one language is well taken.

> Again, I don't see the big deal about a client dealing with a LANGUAGE 
> response containing only i-default, but if we really don't like that, 
> I think the above is a sufficient change.

Sufficient - yes. Ideal - no :-).
I guess I don't care that strongly about the requirements on LANGUAGE 
capability advertisement.

Date: Sat, 08 Mar 2008 18:13:35 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> On Sat, 8 Mar 2008, Alexey Melnikov wrote:
>
>> Can this text being read as saying that I18NLEVEL=1 requirements 
>> override I18NLEVEL=2 requirements (in the case when I18NLEVEL=2 is 
>> also implemented by the server)?
>
> Oh, all right.  How about this addition:
>
> 67c67,68
> <    I18NLEVEL=1 extension.
> ---
>
>>    I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>>    I18NLEVEL=2 (or higher) and comply with the rules of that level.
>
> 743c744,745
> <    the I18NLEVEL=1 extension.
> ---
>
>>    the I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>>    I18NLEVEL=2 (or higher) and comply with the rules of that level.
>
This looks good to me.

Date: Sat, 8 Mar 2008 09:12:08 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

On Sat, 8 Mar 2008, Alexey Melnikov wrote:
> Can this text being read as saying that I18NLEVEL=1 requirements override 
> I18NLEVEL=2 requirements (in the case when I18NLEVEL=2 is also implemented by 
> the server)?

Oh, all right.  How about this addition:

67c67,68
<    I18NLEVEL=1 extension.
---
>    I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>    I18NLEVEL=2 (or higher) and comply with the rules of that level.
743c744,745
<    the I18NLEVEL=1 extension.
---
>    the I18NLEVEL=1 extension.  Alternatively, a server MAY implement
>    I18NLEVEL=2 (or higher) and comply with the rules of that level.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Date: Sat, 08 Mar 2008 16:19:00 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> On Fri, 7 Mar 2008, Alexey Melnikov wrote:
>
>> But this contradict the text you already have in section 7:
>
> OK, how about this in section 7:
>
>    As stated in the introduction, the rules of I18NLEVEL=1 as described
>    in [IMAP-I18N] MUST be followed; that is, the SORT and THREAD
>    extensions MUST collate strings according to the i;unicode-casemap
>    collation described in [UNICASEMAP].  Servers SHOULD also advertise
>    the I18NLEVEL=1 extension.
>
>    As discussed in [IMAP-I18N] section 4.5, all server implementations
>    should eventually be updated to support the [IMAP-I18N] I18NLEVEL=2
>    extension.

This looks better.

Can this text being read as saying that I18NLEVEL=1 requirements 
override I18NLEVEL=2 requirements (in the case when I18NLEVEL=2 is also 
implemented by the server)?

Cc: [email-masked], Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Date: Sat, 08 Mar 2008 03:44:49 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Ned Freed writes:
> >> Ned Freed writes:
> >> >> Ok, so you are talking about misconfiguration case, while I was thinking
> >> >> about implementing LANGUAGE without implementing any languages case.
> >> >
> >> > I realize with all the attention i18n gets in the IETF it's likely to
> >> > be considered an aberrant attitude, but there are plenty of cases
> >> > where people don't want any i18n support on their systems. So I
> >> > wouldn't call it misconfiguration per se.
> >>
> >> Unless the software in question... hm... there are two ways to finish
> >> this sentence...
> >>    - sucks enough to break subtly when it can't find it i18n files.
> >>    - is documented to require its i18n files.
> >> Personally I lean towards the first phrasing. And if we implement
> >> LANGUAGE we'll check that we have >1 language before announcing it.
> >
> > I'm sorry, but I quite simply disagree with this assessment.

> Are you sure?

I'm completely sure.

> Let me put it another way. If a piece of software needs a file and the
> file isn't there, then IMNSHO there are exactly three possibilities:
> Either the software says so at startup and quits, or it says so and
> adjusts its configuration so it doesn't need the file, or it sucks.

Hogwash. Our product uses literally hundreds of files in dozens of directories,
there are many similar products that use thousands. Messaging servers typically
spend more time running that they does being shut down, and full-on startups
are in any case rare - in our case we've had systems remain continuously
available for periods most easily measured in years.

Files can be deleted, moved, or modified at any time, and in practice these
sorts of actions happen routinely as a part of normal product reconfiguration.
But for every legitimate reconfiguration actitivity a vast number of harmful or
nonsensical actions are possible, and sufficient user cluelessness to screw
things up abounds despite numerous warnings and whatnot in the documeentation.

Indeed, only yesterday we had a case of someone creating files with
inappropriate names in a reserved directory that tricked a check we
periodically do into some bad behavior.  (And please note the irony - this was
a check we specifically make to see if certain files are present that went
wonky, not a problem caused by a check we didn't do.)

Code written to check and see if all is well at the filesystem level at
frequent enough intervals to catch all such problems would never do anything
else. (Actually, it's worse than that. There are cases where figuring the set
of files necessary is at least NP-hard, if not halting problem hard.) It
follows that it is entirely reasonable for code to only perform file presence
checks when ther's a compelling argument for doing so. In fact it's a practical
necessity that things be coded this way.

In this particular case the argument, such as it is, for this particular check
is that doing it prevents a wierd corner case from arising that most readers of
a specification aren't going to catch. And that in turn meands absent a
specific admonition in the specification it is entirely reasonable to expect
developers not too perform such a check.

> Breaking subtly when it's time to use a particular feature puts it into
> the third category.

Any the sorts of checks necessary to truly insure the sort of safety you're
talking about would put it in a fourth category - pararnoiaware. (Although if
you manage to crack NP in the process, it would definitely be worth it...)

				Ned

Date: Sat, 8 Mar 2008 09:46:04 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

Ned Freed writes:
>> Ned Freed writes:
>> >> Ok, so you are talking about misconfiguration case, while I was =
thinking
>> >> about implementing LANGUAGE without implementing any languages =
case.
>> >
>> > I realize with all the attention i18n gets in the IETF it's likely =
to
>> > be considered an aberrant attitude, but there are plenty of cases
>> > where people don't want any i18n support on their systems. So I
>> > wouldn't call it misconfiguration per se.
>>
>> Unless the software in question... hm... there are two ways to finish
>> this sentence...
>>    - sucks enough to break subtly when it can't find it i18n files.
>>    - is documented to require its i18n files.
>> Personally I lean towards the first phrasing. And if we implement
>> LANGUAGE we'll check that we have >1 language before announcing it.
>
> I'm sorry, but I quite simply disagree with this assessment.

Are you sure?

Let me put it another way. If a piece of software needs a file and the=20
file isn't there, then IMNSHO there are exactly three possibilities:=20
Either the software says so at startup and quits, or it says so and=20
adjusts its configuration so it doesn't need the file, or it sucks.

Breaking subtly when it's time to use a particular feature puts it into=20
the third category.

> I think what's being required of implementations is borderline=20
> unreasonablle and absent specific cautions in the specification I=20
> would not blame an implementation in
> the slightest for getting this "wrong".
>
> That said, it's not like we haven't required far more unreasonable =
stuff from
> implementations on numerous past occasions. So, as I said when I =
brought this
> up, I really don't care how  this ends up as long as the issue is=20
> mentioned in
> the text.

Yes. Text necessary.

Arnt

Cc: [email-masked], Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Date: Fri, 07 Mar 2008 10:56:15 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

> Ned Freed writes:
> >> Ok, so you are talking about misconfiguration case, while I was thinking
> >> about implementing LANGUAGE without implementing any languages case.
> >
> > I realize with all the attention i18n gets in the IETF it's likely to
> > be considered an aberrant attitude, but there are plenty of cases
> > where people don't want any i18n support on their systems. So I
> > wouldn't call it misconfiguration per se.

> Unless the software in question... hm... there are two ways to finish
> this sentence...
>    - sucks enough to break subtly when it can't find it i18n files.
>    - is documented to require its i18n files.
> Personally I lean towards the first phrasing. And if we implement
> LANGUAGE we'll check that we have >1 language before announcing it.

I'm sorry, but I quite simply disagree with this assessment. I think what's
being required of implementations is borderline unreasonablle and absent
specific cautions in the specification I would not blame an implementation in
the slightest for getting this "wrong".

That said, it's not like we haven't required far more unreasonable stuff from
implementations on numerous past occasions. So, as I said when I brought this
up, I really don't care how  this ends up as long as the issue is mentioned in
the text.

				Ned

Date: Fri, 7 Mar 2008 15:41:39 -0600
To: Alexey Melnikov <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 3/7/08 at 1:03 PM +0000, Alexey Melnikov wrote:

>I've suggested addition of the following text:
>   If the server discovers that it doesn't support any language other
>   than i-default, it MUST return a tagged NO response to the
>   enumeration request.

If we're going to add that, also saying that one MUST NOT advertise 
language if you don't support other than i-default is silly. If the 
client must deal with the case of a NO response to the enumeration 
request, then it won't have a problem. And I think Ned's point about 
not knowing until after CAPABILITIES time that you don't have more 
than one language is well taken.

Again, I don't see the big deal about a client dealing with a 
LANGUAGE response containing only i-default, but if we really don't 
like that, I think the above is a sufficient change.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 07 Mar 2008 17:14:06 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> On Fri, 7 Mar 2008, Alexey Melnikov wrote:
>
>>>    A server which implements the SORT and/or THREAD extensions MUST
>>>    collate strings in accordance with the requirements of I18NLEVEL=1,
>>>    as described in [IMAP-I18N], and SHOULD implement and advertise the
>>>    I18NLEVEL=1 extension.
>>
>> Typo: the last I18NLEVEL=1 should be I18NLEVEL=2.
>
> I accidentally deleted your previous message where you asked about this.
>
> I18NLEVEL=1 is correct.  Currently, there are servers that implement 
> SORT/THREAD in accordance with the requirements of I18NLEVEL=1, but do 
> not advertise I18NLEVEL=1.  We want those servers to upgrade sooner, 
> rather than later, when imap-i18n (finally) comes out; because there 
> is an ambiguity that a server might advertise SORT/THREAD without 
> complying with I18NLEVEL=1.
>
> Note the next paragraph:
>       Discussion: the SORT and THREAD extensions predate [IMAP-I18N] by
>       several years.  At the time of this writing, all known server
>       implementations of SORT and THREAD comply with the rules of
>       I18NLEVEL=1, but do not necessarily advertise it.  As discussed
>       in [IMAP-I18N] section 4.5, all server implementations should
>       eventually be updated to comply with the I18NLEVEL=2 extension.
>
> I don't think that SORT/THREAD can mandate I18NLEVEL=2 right now, even 
> with a SHOULD.

But this contradict the text you already have in section 7:

   As stated in the introduction, the server SHOULD support the
   [IMAP-I18N] I18NLEVEL=2 extension and follow its rules to perform
   collations in the SORT and THREAD extensions.

   If the server does not support I18NLEVEL=2, the rules of I18NLEVEL=1
   MUST be followed; that is, strings MUST be collated according to the
   i;unicode-casemap collation described in [UNICASEMAP].

Date: Fri, 7 Mar 2008 10:14:58 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

On Fri, 7 Mar 2008, Alexey Melnikov wrote:
> But this contradict the text you already have in section 7:

OK, how about this in section 7:

    As stated in the introduction, the rules of I18NLEVEL=1 as described
    in [IMAP-I18N] MUST be followed; that is, the SORT and THREAD
    extensions MUST collate strings according to the i;unicode-casemap
    collation described in [UNICASEMAP].  Servers SHOULD also advertise
    the I18NLEVEL=1 extension.

    As discussed in [IMAP-I18N] section 4.5, all server implementations
    should eventually be updated to support the [IMAP-I18N] I18NLEVEL=2
    extension.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Fri, 7 Mar 2008 09:00:42 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

On Fri, 7 Mar 2008, Alexey Melnikov wrote:
>>    A server which implements the SORT and/or THREAD extensions MUST
>>    collate strings in accordance with the requirements of I18NLEVEL=1,
>>    as described in [IMAP-I18N], and SHOULD implement and advertise the
>>    I18NLEVEL=1 extension.
> Typo: the last I18NLEVEL=1 should be I18NLEVEL=2.

I accidentally deleted your previous message where you asked about this.

I18NLEVEL=1 is correct.  Currently, there are servers that implement 
SORT/THREAD in accordance with the requirements of I18NLEVEL=1, but do not 
advertise I18NLEVEL=1.  We want those servers to upgrade sooner, rather 
than later, when imap-i18n (finally) comes out; because there is an 
ambiguity that a server might advertise SORT/THREAD without complying with 
I18NLEVEL=1.

Note the next paragraph:
       Discussion: the SORT and THREAD extensions predate [IMAP-I18N] by
       several years.  At the time of this writing, all known server
       implementations of SORT and THREAD comply with the rules of
       I18NLEVEL=1, but do not necessarily advertise it.  As discussed
       in [IMAP-I18N] section 4.5, all server implementations should
       eventually be updated to comply with the I18NLEVEL=2 extension.

I don't think that SORT/THREAD can mandate I18NLEVEL=2 right now, even 
with a SHOULD.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Fri, 07 Mar 2008 13:18:07 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: SORT document, draft -20
Sender: [email-masked]

Mark Crispin wrote:

> I've tried to get all the Last Call review comments together.  I was 
> going to fix these during AUTH48, but am considering pushing out a -20 
> on March 10 when the ID repository opens again.
>
> Here's what I have now.  Note the expansion of section 2.2, the 
> removal of date handling comments in the text in favor of references 
> to section 2.2, and the two new paragraphs in the security 
> considerations.

The new text in the Security Considerations looks good to me. I haven't 
checked new text in 2.2 very attentively.

> 1. Introduction

 [...]

>    A server which implements the SORT and/or THREAD extensions MUST
>    collate strings in accordance with the requirements of I18NLEVEL=1,
>    as described in [IMAP-I18N], and SHOULD implement and advertise the
>    I18NLEVEL=1 extension.

Typo: the last I18NLEVEL=1 should be I18NLEVEL=2.

Date: Fri, 07 Mar 2008 13:03:58 +0000
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Ned Freed wrote:

>> Ned Freed wrote:
>
 [...]

>> Ok, so you are talking about misconfiguration case, while I was thinking
>> about implementing LANGUAGE without implementing any languages case.
>
> I realize with all the attention i18n gets in the IETF it's likely to be
> considered an aberrant attitude, but there are plenty of cases where 
> people
> don't want any i18n support on their systems. So I wouldn't call it
> misconfiguration per se.

Sure. See my text below.

>> I think it is worth adding some text to describe correct behaviour for
>> your case.
>
> IMO that's an acceptable way to handle it.

I've suggested addition of the following text:
   If the server discovers that it doesn't support any language other
   than i-default, it MUST return a tagged NO response to the
   enumeration request.

At the end of the following paragraph:
   If there aren't any arguments, the server SHOULD send an untagged
   LANGUAGE response listing the languages it supports.  If the server
   is unable to enumerate the list of languages it supports it MAY
   return a tagged NO response to the enumeration request.

I can also suggest some additional text saying that the LANGUAGE 
extension shouldn't be advertised if no language other than i-default is 
available.

> Of course one way to deal with it
> would be to hard code one additionaal language code other than 
> i-default for
> whatever the built in language support is.

Indeed.

Date: Fri, 7 Mar 2008 12:23:28 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

Ned Freed writes:
>> Ok, so you are talking about misconfiguration case, while I was thinking
>> about implementing LANGUAGE without implementing any languages case.
>
> I realize with all the attention i18n gets in the IETF it's likely to 
> be considered an aberrant attitude, but there are plenty of cases 
> where people don't want any i18n support on their systems. So I 
> wouldn't call it misconfiguration per se.

Unless the software in question... hm... there are two ways to finish 
this sentence...
   - sucks enough to break subtly when it can't find it i18n files.
   - is documented to require its i18n files.
Personally I lean towards the first phrasing. And if we implement 
LANGUAGE we'll check that we have >1 language before announcing it.

Arnt

Date: Thu, 6 Mar 2008 16:05:03 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: SORT document, draft -20
Sender: [email-masked]

I've tried to get all the Last Call review comments together.  I was going 
to fix these during AUTH48, but am considering pushing out a -20 on March 
10 when the ID repository opens again.

Here's what I have now.  Note the expansion of section 2.2, the removal of 
date handling comments in the text in favor of references to section 2.2, 
and the two new paragraphs in the security considerations.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

IMAP Extensions Working Group                                 M. Crispin
Internet-Draft                                              K. Murchison
Intended status: Proposed Standard                        March 10, 2008
Expires: September 10, 2008
Document: internet-drafts/draft-ietf-imapext-sort-20.txt

      INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS

Status of this Memo

    By submitting this Internet-Draft, each author represents that
    any applicable patent or other IPR claims of which he or she is
    aware have been or will be disclosed, and any of which he or she
    becomes aware will be disclosed, in accordance with Section 6 of
    BCP 79.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as
    Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six months
    and may be updated, replaced, or obsoleted by other documents at any
    time.  It is inappropriate to use Internet-Drafts as reference
    material or to cite them other than as "work in progress."

    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/ietf/1id-abstracts.txt

    The list of Internet-Draft Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html.

    A revised version of this draft document will be submitted to the RFC
    editor as a Proposed Standard for the Internet Community.  Discussion
    and suggestions for improvement are requested, and should be sent to
    [email-masked].

    Distribution of this memo is unlimited.

Abstract

    This document describes the base-level server-based sorting and
    threading extensions to the [IMAP] protocol.  These extensions
    provide substantial performance improvements for IMAP clients which
    offer sorted and threaded views.

1. Introduction

    The SORT and THREAD extensions to the [IMAP] protocol provide a means
    of server-based sorting and threading of messages, without requiring
    that the client download the necessary data to do so itself.  This is
    particularly useful for online clients as described in [IMAP-MODELS].

    A server which supports the base-level SORT extension indicates this
    with a capability name which starts with "SORT".  Future,
    upwards-compatible extensions to the SORT extension will all start
    with "SORT", indicating support for this base level.

    A server which supports the THREAD extension indicates this with one
    or more capability names consisting of "THREAD=" followed by a
    supported threading algorithm name as described in this document.
    This provides for future upwards-compatible extensions.

    A server which implements the SORT and/or THREAD extensions MUST
    collate strings in accordance with the requirements of I18NLEVEL=1,
    as described in [IMAP-I18N], and SHOULD implement and advertise the
    I18NLEVEL=1 extension.

       Discussion: the SORT and THREAD extensions predate [IMAP-I18N] by
       several years.  At the time of this writing, all known server
       implementations of SORT and THREAD comply with the rules of
       I18NLEVEL=1, but do not necessarily advertise it.  As discussed
       in [IMAP-I18N] section 4.5, all server implementations should
       eventually be updated to comply with the I18NLEVEL=2 extension.

    Historical note: the REFERENCES threading algorithm is based on the
    [THREADING] algorithm written used in "Netscape Mail and News"
    versions 2.0 through 3.0.

2. Terminology

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
    document are to be interpreted as described in [KEYWORDS].

    The word "can" (not "may") is used to refer to a possible
    circumstance or situation, as opposed to an optional facility of the
    protocol.

    "User" is used to refer to a human user, whereas "client" refers to
    the software being run by the user.

    In examples, "C:" and "S:" indicate lines sent by the client and
    server respectively.

2.1 Base Subject

    Subject sorting and threading use the "base subject," which has
    specific subject artifacts removed.  Due to the complexity of these
    artifacts, the formal syntax for the subject extraction rules is
    ambiguous.  The following procedure is followed to determine the
    "base subject", using the [ABNF] formal syntax rules described in
    section 5:

         (1) Convert any RFC 2047 encoded-words in the subject to
         UTF-8 as described in "internationalization
         considerations."  Convert all tabs and continuations to
         space.  Convert all multiple spaces to a single space.

         (2) Remove all trailing text of the subject that matches
         the subj-trailer ABNF, repeat until no more matches are
         possible.

         (3) Remove all prefix text of the subject that matches the
         subj-leader ABNF.

         (4) If there is prefix text of the subject that matches the
         subj-blob ABNF, and removing that prefix leaves a non-empty
         subj-base, then remove the prefix text.

         (5) Repeat (3) and (4) until no matches remain.

    Note: it is possible to defer step (2) until step (6), but this
    requires checking for subj-trailer in step (4).

         (6) If the resulting text begins with the subj-fwd-hdr ABNF
         and ends with the subj-fwd-trl ABNF, remove the
         subj-fwd-hdr and subj-fwd-trl and repeat from step (2).

         (7) The resulting text is the "base subject" used in the
         SORT.

    All servers and disconnected (as described in [IMAP-MODELS]) clients
    MUST use exactly this algorithm to determine the "base subject".
    Otherwise there is potential for a user to get inconsistent results
    based on whether they are running in connected or disconnected mode.

2.2 Sent Date

    As used in this document, the term "sent date" refers to the date and
    time from the Date: header, adjusted by time zone to normalize to
    UTC.  For example, "31 Dec 2000 16:01:33 -0800" is equivalent to the
    UTC date and time of "1 Jan 2001 00:01:33 +0000".

    If the time zone is invalid, the date and time SHOULD be treated as
    UTC.  If the time is also invalid, the time SHOULD be treated as
    00:00:00.  If there is no valid date or time, the date and time
    SHOULD be treated as 00:00:00 on the earliest possible date.

    This differs from the date-related criteria in the SEARCH command
    (described in [IMAP] section 6.4.4), which use just the date and not
    the time, and are not adjusted by time zone.

    If the sent date can not be determined (a Date: header is missing or
    can not be parsed), the INTERNALDATE for that message is used as the
    sent date.

    When comparing two sent dates that match exactly, the order in which
    the two messages appear in the mailbox (that is, by sequence number)
    is used as a tie-breaker to determine the order.

3. Additional Commands

    These commands are extension to the [IMAP] base protocol.

    The section headings are intended to correspond with where they would
    be located in the main document if they were part of the base
    specification.

BASE.6.4.SORT. SORT Command

    Arguments:  sort program
                charset specification
                searching criteria (one or more)

    Data:       untagged responses: SORT

    Result:     OK - sort completed
                NO - sort error: can't sort that charset or
                     criteria
                BAD - command unknown or arguments invalid

       The SORT command is a variant of SEARCH with sorting semantics for
       the results.  Sort has two arguments before the searching criteria
       argument; a parenthesized list of sort criteria, and the searching
       charset.

       The charset argument is mandatory (unlike SEARCH) and indicates
       the [CHARSET] of the strings that appear in the searching
       criteria.  The US-ASCII and UTF-8 charsets MUST be implemented.
       All other charsets are optional.

       There is also a UID SORT command which returns unique identifiers
       instead of message sequence numbers.  Note that there are separate
       searching criteria for message sequence numbers and UIDs; thus the
       arguments to UID SORT are interpreted the same as in SORT.  This
       is analogous to the behavior of UID SEARCH, as opposed to UID
       COPY, UID FETCH, or UID STORE.

       The SORT command first searches the mailbox for messages that
       match the given searching criteria using the charset argument for
       the interpretation of strings in the searching criteria.  It then
       returns the matching messages in an untagged SORT response, sorted
       according to one or more sort criteria.

       Sorting is in ascending order.  Earlier dates sort before later
       dates; smaller sizes sort before larger sizes; and strings are
       sorted according to ascending values established by their
       collation algorithm (see under "Internationalization
       Considerations").

       If two or more messages exactly match according to the sorting
       criteria, these messages are sorted according to the order in
       which they appear in the mailbox.  In other words, there is an
       implicit sort criterion of "sequence number".

       When multiple sort criteria are specified, the result is sorted in
       the priority order that the criteria appear.  For example,
       (SUBJECT DATE) will sort messages in order by their base subject
       text; and for messages with the same base subject text will sort
       by their sent date.

       Untagged EXPUNGE responses are not permitted while the server is
       responding to a SORT command, but are permitted during a UID SORT
       command.

       The defined sort criteria are as follows.  Refer to the Formal
       Syntax section for the precise syntactic definitions of the
       arguments.  If the associated RFC-822 header for a particular
       criterion is absent, it is treated as the empty string.  The empty
       string always collates before non-empty strings.

       ARRIVAL
          Internal date and time of the message.  This differs from the
          ON criteria in SEARCH, which uses just the internal date.

       CC
          [IMAP] addr-mailbox of the first "cc" address.

       DATE
          Sent date and time, as described in section 2.2.

       FROM
          [IMAP] addr-mailbox of the first "From" address.

       REVERSE
          Followed by another sort criterion, has the effect of that
          criterion but in reverse (descending) order.
             Note: REVERSE only reverses a single criterion, and does not
             affect the implicit "sequence number" sort criterion if all
             other criteria are identicial.  Consequently, a sort of
             REVERSE SUBJECT is not the same as a reverse ordering of a
             SUBJECT sort.  This can be avoided by use of additional
             criteria, e.g. SUBJECT DATE vs. REVERSE SUBJECT REVERSE
             DATE.  In general, however, it's better (and faster, if the
             client has a "reverse current ordering" command) to reverse
             the results in the client instead of issuing a new SORT.

       SIZE
          Size of the message in octets.

       SUBJECT
          Base subject text.

       TO
          [IMAP] addr-mailbox of the first "To" address.

    Example:    C: A282 SORT (SUBJECT) UTF-8 SINCE 1-Feb-1994
                S: * SORT 2 84 882
                S: A282 OK SORT completed
                C: A283 SORT (SUBJECT REVERSE DATE) UTF-8 ALL
                S: * SORT 5 3 4 1 2
                S: A283 OK SORT completed
                C: A284 SORT (SUBJECT) US-ASCII TEXT "not in mailbox"
                S: * SORT
                S: A284 OK SORT completed

BASE.6.4.THREAD. THREAD Command

Arguments:  threading algorithm
             charset specification
             searching criteria (one or more)

Data:       untagged responses: THREAD

Result:     OK - thread completed
             NO - thread error: can't thread that charset or
                  criteria
             BAD - command unknown or arguments invalid

       The THREAD command is a variant of SEARCH with threading semantics
       for the results.  Thread has two arguments before the searching
       criteria argument; a threading algorithm, and the searching
       charset.

       The charset argument is mandatory (unlike SEARCH) and indicates
       the [CHARSET] of the strings that appear in the searching
       criteria.  The US-ASCII and UTF-8 charsets MUST be implemented.
       All other charsets are optional.

       There is also a UID THREAD command which returns unique
       identifiers instead of message sequence numbers.  Note that there
       are separate searching criteria for message sequence numbers and
       UIDs; thus the arguments to UID THREAD are interpreted the same as
       in THREAD.  This is analogous to the behavior of UID SEARCH, as
       opposed to UID COPY, UID FETCH, or UID STORE.

       The THREAD command first searches the mailbox for messages that
       match the given searching criteria using the charset argument for
       the interpretation of strings in the searching criteria.  It then
       returns the matching messages in an untagged THREAD response,
       threaded according to the specified threading algorithm.

       All collation is in ascending order.  Earlier dates collate before
       later dates and strings are collated according to ascending values
       established by their collation algorithm (see under
       "Internationalization Considerations").

       Untagged EXPUNGE responses are not permitted while the server is
       responding to a THREAD command, but are permitted during a UID
       THREAD command.

       The defined threading algorithms are as follows:

       ORDEREDSUBJECT

          The ORDEREDSUBJECT threading algorithm is also referred to as
          "poor man's threading."  The searched messages are sorted by
          base subject and then by the sent date.  The messages are then
          split into separate threads, with each thread containing
          messages with the same base subject text.  Finally, the threads
          are sorted by the sent date of the first message in the thread.

          The first message of each thread are siblings of each other
          (the "root").  The second message of a thread is the child of
          the first message, and subsequent messages of the thread are
          siblings of the second message and hence children of the
          message at the root.  Hence, there are no grandchildren in
          ORDEREDSUBJECT threading.

          Children in ORDEREDSUBJECT threading do not have descendents.
          Client implementations SHOULD treat descendents of a child in
          a server response as being siblings of that child.

       REFERENCES

          The REFERENCES threading algorithm threads the searched
          messages by grouping them together in parent/child
          relationships based on which messages are replies to others.
          The parent/child relationships are built using two methods:
          reconstructing a message's ancestry using the references
          contained within it; and checking the original (not base)
          subject of a message to see if it is a reply to (or forward of)
          another message.

             Note: "Message ID" in the following description refers to a
             normalized form of the msg-id in [RFC-2822].  The actual
             text in an RFC 2822 may use quoting, resulting in multiple
             ways of expressing the same Message ID.  Implementations of
             the REFERENCES threading algorithm MUST normalize any msg-id
             in order to avoid false non-matches due to differences in
             quoting.

             For example, the msg-id
                <"01KF8JCEOCBS0045PS"@xxx.yyy.com>
             and the msg-id
                <[email-masked]>
             MUST be interpreted as being the same Message ID.

          The references used for reconstructing a message's ancestry are
          found using the following rules:

             If a message contains a References header line, then use the
             Message IDs in the References header line as the references.

             If a message does not contain a References header line, or
             the References header line does not contain any valid
             Message IDs, then use the first (if any) valid Message ID
             found in the In-Reply-To header line as the only reference
             (parent) for this message.

                Note: Although [RFC-2822] permits multiple Message IDs in
                the In-Reply-To header, in actual practice this
                discipline has not been followed.  For example,
                In-Reply-To headers have been observed with message
                addresses after the Message ID, and there are no good
                heuristics for software to determine the difference.
                This is not a problem with the References header however.

             If a message does not contain an In-Reply-To header line, or
             the In-Reply-To header line does not contain a valid Message
             ID, then the message does not have any references (NIL).

          A message is considered to be a reply or forward if the base
          subject extraction rules, applied to the original subject,
          remove any of the following: a subj-refwd, a "(fwd)"
          subj-trailer, or a subj-fwd-hdr and subj-fwd-trl.

          The REFERENCES algorithm is significantly more complex than
          ORDEREDSUBJECT and consists of six main steps.  These steps are
          outlined in detail below.

          (1) For each searched message:

             (A) Using the Message IDs in the message's references, link
             the corresponding messages (those whose Message-ID header
             line contains the given reference Message ID) together as
             parent/child.  Make the first reference the parent of the
             second (and the second a child of the first), the second the
             parent of the third (and the third a child of the second),
             etc.  The following rules govern the creation of these
             links:

                If a message does not contain a Message-ID header line,
                or the Message-ID header line does not contain a valid
                Message ID, then assign a unique Message ID to this
                message.

                If two or more messages have the same Message ID, then
                only use that Message ID in the first (lowest sequence
                number) message, and assign a unique Message ID to each
                of the subsequent messages with a duplicate of that
                Message ID.

                If no message can be found with a given Message ID,
                create a dummy message with this ID.  Use this dummy
                message for all subsequent references to this ID.

                If a message already has a parent, don't change the
                existing link.  This is done because the References
                header line may have been truncated by a MUA.  As a
                result, there is no guarantee that the messages
                corresponding to adjacent Message IDs in the References
                header line are parent and child.

                Do not create a parent/child link if creating that link
                would introduce a loop.  For example, before making
                message A the parent of B, make sure that A is not a
                descendent of B.

                   Note: Message ID comparisons are case-sensitive.

             (B) Create a parent/child link between the last reference
             (or NIL if there are no references) and the current message.
             If the current message already has a parent, it is probably
             the result of a truncated References header line, so break
             the current parent/child link before creating the new
             correct one.  As in step 1.A, do not create the parent/child
             link if creating that link would introduce a loop.  Note
             that if this message has no references, that it will now
             have no parent.

                Note: The parent/child links created in steps 1.A and 1.B
                MUST be kept consistent with one another at ALL times.

          (2) Gather together all of the messages that have no parents
          and make them all children (siblings of one another) of a dummy
          parent (the "root").  These messages constitute the first
          (head) message of the threads created thus far.

          (3) Prune dummy messages from the thread tree.  Traverse each
          thread under the root, and for each message:

             If it is a dummy message with NO children, delete it.

             If it is a dummy message with children, delete it, but
             promote its children to the current level.  In other words,
             splice them in with the dummy's siblings.

             Do not promote the children if doing so would make them
             children of the root, unless there is only one child.

          (4) Sort the messages under the root (top-level siblings only)
          by sent date as described in section 2.2.  In the case of a
          dummy message, sort its children by sent date and then use the
          first child for the top-level sort.

          (5) Gather together messages under the root that have the same
          base subject text.

             (A) Create a table for associating base subjects with
             messages, called the subject table.

             (B) Populate the subject table with one message per each
             base subject.  For each child of the root:

                (i) Find the subject of this thread, by using the base
                subject from either the current message or its first
                child if the current message is a dummy.  This is the
                thread subject.

                (ii) If the thread subject is empty, skip this message.

                (iii) Look up the message associated with the thread
                subject in the subject table.

                (iv) If there is no message in the subject table with the
                thread subject, add the current message and the thread
                subject to the subject table.

                Otherwise, if the message in the subject table is not a
                dummy, AND either of the following criteria are true:

                   The current message is a dummy, OR

                   The message in the subject table is a reply or forward
                   and the current message is not.

             then replace the message in the subject table with the
             current message.

             (C) Merge threads with the same thread subject.  For each
             child of the root:

                (i) Find the message's thread subject as in step 5.B.i
                above.

                (ii) If the thread subject is empty, skip this message.

                (iii) Lookup the message associated with this thread
                subject in the subject table.

                (iv) If the message in the subject table is the current
                message, skip this message.

                Otherwise, merge the current message with the one in the
                subject table using the following rules:

                   If both messages are dummies, append the current
                   message's children to the children of the message in
                   the subject table (the children of both messages
                   become siblings), and then delete the current message.

                   If the message in the subject table is a dummy and the
                   current message is not, make the current message a
                   child of the message in the subject table (a sibling
                   of its children).

                   If the current message is a reply or forward and the
                   message in the subject table is not, make the current
                   message a child of the message in the subject table (a
                   sibling of its children).

                   Otherwise, create a new dummy message and make both
                   the current message and the message in the subject
                   table children of the dummy.  Then replace the message
                   in the subject table with the dummy message.

                      Note: Subject comparisons are case-insensitive, as
                      described under "Internationalization
                      Considerations."

          (6) Traverse the messages under the root and sort each set of
          siblings by sent date as described in section 2.2.  Traverse
          the messages in such a way that the "youngest" set of siblings
          are sorted first, and the "oldest" set of siblings are sorted
          last (grandchildren are sorted before children, etc).  In the
          case of a dummy message (which can only occur with top-level
          siblings), use its first child for sorting.

    Example:    C: A283 THREAD ORDEREDSUBJECT UTF-8 SINCE 5-MAR-2000
                S: * THREAD (166)(167)(168)(169)(172)(170)(171)
                   (173)(174 (175)(176)(178)(181)(180))(179)(177
                   (183)(182)(188)(184)(185)(186)(187)(189))(190)
                   (191)(192)(193)(194 195)(196 (197)(198))(199)
                   (200 202)(201)(203)(204)(205)(206 207)(208)
                S: A283 OK THREAD completed
                C: A284 THREAD ORDEREDSUBJECT US-ASCII TEXT "gewp"
                S: * THREAD
                S: A284 OK THREAD completed
                C: A285 THREAD REFERENCES UTF-8 SINCE 5-MAR-2000
                S: * THREAD (166)(167)(168)(169)(172)((170)(179))
                   (171)(173)((174)(175)(176)(178)(181)(180))
                   ((177)(183)(182)(188 (184)(189))(185 186)(187))
                   (190)(191)(192)(193)((194)(195 196))(197 198)
                   (199)(200 202)(201)(203)(204)(205 206 207)(208)
                S: A285 OK THREAD completed

         Note: The line breaks in the first and third server
         responses are for editorial clarity and do not appear in
         real THREAD responses.

4. Additional Responses

    These responses are extensions to the [IMAP] base protocol.

    The section headings of these responses are intended to correspond
    with where they would be located in the main document.

BASE.7.2.SORT. SORT Response

    Data:       zero or more numbers

       The SORT response occurs as a result of a SORT or UID SORT
       command.  The number(s) refer to those messages that match the
       search criteria.  For SORT, these are message sequence numbers;
       for UID SORT, these are unique identifiers.  Each number is
       delimited by a space.

    Example:    S: * SORT 2 3 6

BASE.7.2.THREAD. THREAD Response

    Data:       zero or more threads

       The THREAD response occurs as a result of a THREAD or UID THREAD
       command.  It contains zero or more threads.  A thread consists of
       a parenthesized list of thread members.

       Thread members consist of zero or more message numbers, delimited
       by spaces, indicating successive parent and child.  This continues
       until the thread splits into multiple sub-threads, at which point
       the thread nests into multiple sub-threads with the first member
       of each subthread being siblings at this level.  There is no limit
       to the nesting of threads.

       The messages numbers refer to those messages that match the search
       criteria.  For THREAD, these are message sequence numbers; for UID
       THREAD, these are unique identifiers.

    Example:    S: * THREAD (2)(3 6 (4 23)(44 7 96))

       The first thread consists only of message 2.  The second thread
       consists of the messages 3 (parent) and 6 (child), after which it
       splits into two subthreads; the first of which contains messages 4
       (child of 6, sibling of 44) and 23 (child of 4), and the second of
       which contains messages 44 (child of 6, sibling of 4), 7 (child of
       44), and 96 (child of 7).  Since some later messages are parents
       of earlier messages, the messages were probably moved from some
       other mailbox at different times.

       -- 2

       -- 3
          \-- 6
              |-- 4
              |   \-- 23
              |
              \-- 44
                   \-- 7
                       \-- 96

    Example:    S: * THREAD ((3)(5))

       In this example, 3 and 5 are siblings of a parent which does not
       match the search criteria (and/or does not exist in the mailbox);
       however they are members of the same thread.

5. Formal Syntax of SORT and THREAD Commands and Responses

    The following syntax specification uses the Augmented Backus-Naur
    Form (ABNF) notation as specified in [ABNF].  It also uses [ABNF]
    rules defined in [IMAP].

sort            = ["UID" SP] "SORT" SP sort-criteria SP search-criteria

sort-criteria   = "(" sort-criterion *(SP sort-criterion) ")"

sort-criterion  = ["REVERSE" SP] sort-key

sort-key        = "ARRIVAL" / "CC" / "DATE" / "FROM" / "SIZE" /
                   "SUBJECT" / "TO"

thread          = ["UID" SP] "THREAD" SP thread-alg SP search-criteria

thread-alg      = "ORDEREDSUBJECT" / "REFERENCES" / thread-alg-ext

thread-alg-ext  = atom
                     ; New algorithms MUST be registered with IANA

search-criteria = charset 1*(SP search-key)

charset         = atom / quoted
                     ; CHARSET values MUST be registered with IANA

sort-data       = "SORT" *(SP nz-number)

thread-data     = "THREAD" [SP 1*thread-list]

thread-list     = "(" (thread-members / thread-nested) ")"

thread-members  = nz-number *(SP nz-number) [SP thread-nested]

thread-nested   = 2*thread-list

    The following syntax describes base subject extraction rules (2)-(6):

subject         = *subj-leader [subj-middle] *subj-trailer

subj-refwd      = ("re" / ("fw" ["d"])) *WSP [subj-blob] ":"

subj-blob       = "[" *BLOBCHAR "]" *WSP

subj-fwd        = subj-fwd-hdr subject subj-fwd-trl

subj-fwd-hdr    = "[fwd:"

subj-fwd-trl    = "]"

subj-leader     = (*subj-blob subj-refwd) / WSP

subj-middle     = *subj-blob (subj-base / subj-fwd)
                     ; last subj-blob is subj-base if subj-base would
                     ; otherwise be empty

subj-trailer    = "(fwd)" / WSP

subj-base       = NONWSP *(*WSP NONWSP)
                     ; can be a subj-blob

BLOBCHAR        = %x01-5a / %x5c / %x5e-ff
                     ; any CHAR8 except '[' and ']'

NONWSP          = %x01-08 / %x0a-1f / %x21-ff
                     ; any CHAR8 other than WSP

6. Security Considerations

    The SORT and THREAD extensions do not raise any security
    considerations that are not present in the base [IMAP] protocol, and
    these issues are discussed in [IMAP].  Nevertheless, it is important
    to remember that [IMAP] protocol transactions, including message
    data, are sent in the clear over the network unless protection from
    snooping is negotiated, either by the use of STARTTLS, privacy
    protection is negotiated in the AUTHENTICATE command, or some other
    protection mechanism.

    Although not a security consideration, it is important to recognize
    that sorting by REFERENCES can lead to misleading threading trees.
    For example, a message with false References: header data will cause
    a thread to be incorporated into another thread.

    The process of extracting the base subject may lead to incorrect
    collation if the extracted data was significant text as opposed to
    a subject artifact.

7. Internationalization Considerations

    As stated in the introduction, the server SHOULD support the
    [IMAP-I18N] I18NLEVEL=2 extension and follow its rules to perform
    collations in the SORT and THREAD extensions.

    If the server does not support I18NLEVEL=2, the rules of I18NLEVEL=1
    MUST be followed; that is, strings MUST be collated according to the
    i;unicode-casemap collation described in [UNICASEMAP].

    Translations of the "re" or "fw"/"fwd" tokens are not specified for
    removal in the base subject extraction process.  An attempt to add
    such translated tokens would result in a geometrically complex, and
    ultimately unimplementable, task.

    Instead, note that [RFC-2822] section 3.6.5 recommends that "re:"
    (from the Latin "res", in the matter of) be used to identify a reply.
    Although it is evident that, from the multiple forms of token to
    identify a forwarded message, there is considerable variation found
    in the wild, the variations are (still) manageable.  Consequently, it
    is suggested that "re:" and one of the variations of the tokens for
    forward supported by the base subject extraction rules be adopted for
    Internet mail messages, since doing so makes it a simple display time
    task to localize the token language for the user.

8. IANA Considerations

    [IMAP] capabilities are registered by publishing a standards track or
    IESG approved experimental RFC.  This document constitutes
    registration of the SORT and THREAD capabilities in the [IMAP]
    capabilities registry.

    This document creates a new [IMAP] threading algorithms registry,
    which registers threading algorithms by publishing a standards track
    or IESG approved experimental RFC.  This document constitutes
    registration of the ORDEREDSUBJECT and REFERENCES algorithms in that
    registry.

9. Normative References

    The following documents are normative to this document:

    [ABNF]                Crocker, D. and Overell, P. "Augmented BNF
                          for Syntax Specifications: ABNF", RFC 5234
                          January 2008

    [CHARSET]             Freed, N. and Postel, J. "IANA Character Set
                          Registration Procedures", RFC 2978, October
                          2000.

    [IMAP]                Crispin, M. "Internet Message Access Protocol -
                          Version 4rev1", RFC 3501, March 2003.

    [IMAP-I18N]           Newman, C. and Gulbrandsen, A. "Internet
                          Message Access Protocol Internationalization",
                          Work in Progress.

    [KEYWORDS]            Bradner, S. "Key words for use in RFCs to
                          Indicate Requirement Levels", BCP 14, RFC 2119,
                          March 1997.

    [RFC-2822]            Resnick, P. "Internet Message Format", RFC
                          2822, April 2001.

    [UNICASEMAP]          Crispin, M. "i;unicode-casemap - Simple Unicode
                          Collation Algorithm", RFC 5051.

10. Informative References

    The following documents are informative to this document:

    [IMAP-MODELS]         Crispin, M. "Distributed Electronic Mail Models
                          in IMAP4", RFC 1733, December 1994.

    [THREADING]           Zawinski, J. "Message Threading",
                          http://www.jwz.org/doc/threading.html,
                          1997-2002.

Appendices

Author's Address

    Mark R. Crispin
    Networks and Distributed Computing
    University of Washington
    4545 15th Avenue NE
    Seattle, WA  98105-4527

    Phone: +1 (206) 543-5762

    EMail: [email-masked]

    Kenneth Murchison
    Carnegie Mellon University
    5000 Forbes Avenue
    Cyert Hall 285
    Pittsburgh, PA  15213

    Phone: +1 (412) 268-2638
    Email: [email-masked]

Full Copyright Statement

    Copyright (C) The IETF Trust (2008).

    This document is subject to the rights, licenses and restrictions
    contained in BCP 78, and except as set forth therein, the authors
    retain all their rights.

    This document and the information contained herein are provided on an
    "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
    OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
    THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
    OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
    THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
    WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

    The IETF takes no position regarding the validity or scope of any
    Intellectual Property Rights or other rights that might be claimed to
    pertain to the implementation or use of the technology described in
    this document or the extent to which any license under such rights
    might or might not be available; nor does it represent that it has
    made any independent effort to identify any such rights.  Information
    on the procedures with respect to rights in RFC documents can be
    found in BCP 78 and BCP 79.

    Copies of IPR disclosures made to the IETF Secretariat and any
    assurances of licenses to be made available, or the result of an
    attempt made to obtain a general license or permission for the use of
    such proprietary rights by implementers or users of this
    specification can be obtained from the IETF on-line IPR repository at
    http://www.ietf.org/ipr.

    The IETF invites any interested party to bring to its attention any
    copyrights, patents or patent applications, or other proprietary
    rights that may cover technology that may be required to implement
    this standard.  Please address the information to the IETF at ietf-
    [email-masked].

Acknowledgement

    Funding for the RFC Editor function is currently provided by the
    Internet Society.

Date: Thu, 6 Mar 2008 16:02:26 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Tobias Gondrom <[email-masked]>
cc: [email-masked], [email-masked], [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: secdir review of draft-ietf-imapext-sort-19
Organization: UW Technology
Sender: [email-masked]

Thank you for your thoughtful comments.

I am working on update that addresses your issues.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Cc: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Thu, 06 Mar 2008 13:14:47 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> Ned Freed wrote:

> >> > Now, if this corner case does exist the cleanest solution would be to
> >> > have two separate responses.
> >
> >> Right.
> >
> >> > But that might break existing implementations.
> >
> >> Indeed.
> >
> >> After reading all responses on this subject I still tend to think that
> >> Tim's proposal is the best one: I can't see any good reason why a server
> >> would implement LANGUAGE command and only support i-default, as this is
> >> pointless.
> >
> > Hmm. Well, the one possibility that occurs to me os that quite a lot of
> > software implements i18n support by having a separate
> > directory/file/container
> > for each supported language. That way new languages can be added or
> > removed on
> > the fly. And when a given language is requested a check is made to see if
> > there's a corrresponding directory/file/container for it.
> >
> > This makes it fairly easy to end up in a situation where the only
> > directory/file/container is the i-default one - in fact the usual way
> > this
> > works is i-default is the built-in default that doesn't have a
> > directory/file/container associated with it, so this corresponds to
> > the fairly
> > common "I deleted all those nasty i18n files in order to save  space"
> > behavior.

> Ok, so you are talking about misconfiguration case, while I was thinking
> about implementing LANGUAGE without implementing any languages case.

I realize with all the attention i18n gets in the IETF it's likely to be
considered an aberrant attitude, but there are plenty of cases where people
don't want any i18n support on their systems. So I wouldn't call it
misconfiguration per se.

> I think it is worth adding some text to describe correct behaviour for
> your case.

IMO that's an acceptable way to handle it. Of course one way to deal with it
would be to hard code one additionaal language code other than i-default for
whatever the built in language support is.

> > This requirement would force such an implementation to check and see
> > whether or
> > not non-i-default material is available before even offering the
> > extension. (Or
> > you can have a extension enable/disable configuration variable, but of
> > course
> > this can lead to a silly state.)

> Yes. This is similar to STARTTLS, which shouldn't be advertised unless
> server can create a TLS context.

Exactly, and unfortunately what we've seen in practice is plenty of server
implementations that do not check this and offer STARTTLS even when they don't
have the necessary certificate(s) installed. This effectively killed
opportunistic use of TLS in SMTP back when it was first standardized -  we had
to advise all our customers not to enable "use TLS if available" mode on
Internet-facing relay clients because it blocked message tranfer in too many
cases. Overall email security didn't exactly benefit from this...

Now, of course this odd corner case isn't remotely as important as the lost
opportunity to encrypt a significant fraction of email on the wire, But it
still needs to be dealt with.

				Ned

Date: Wed, 05 Mar 2008 21:12:31 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Pete Resnick <[email-masked]>, [email-masked], Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Dave Cridland wrote:

> On Tue Mar  4 19:30:02 2008, Pete Resnick wrote:
>
>> No, I think Tim's concern is different, but I still think there is  
>> nothing (besides perhaps a little wordsmithing) that needs to be  
>> done: Tim's concern is that if you get back a LANGUAGE response  with 
>> only 1 language tag, you don't know (without saving some  context 
>> about what command you issued) whether that means the  server is 
>> sending back its list of available languages (which only  contains 1 
>> item) or is telling you that it is now using the  language it just 
>> sent back. But I think this is a distinction  without a difference: 
>> If the server only supports i-default, it is  *always* using 
>> i-default and will *always* return a single language  tag (of 
>> i-default).
>>
>> I'm open for additional text at the end of the paragraph to clarify.
>
> I had to think hard about this - Pete's got a good point here - I  
> just have to disagree.
>
> What Pete's saying is that in the case where there is only one  
> language, which MUST be i-default, there is no semantic difference  
> between the server saying "I only use i-default" and "I am using  
> i-default" - whatever happens, the server is now using i-default. The  
> trouble is that if a client wishes to kick the server into telling it  
> what languages it supports, it instead gets a message back saying  
> that the language has been set - I have a feeling I'd still need to  
> involve semantics at too low a level to be comfortable with. It's  
> painful enough with unextended SEARCH, I wouldn't want to  
> special-case anything else.
>
> What Ned says in another message is that there is considerable work  
> in the server knowing it only has i-default available prior to  
> offering the extension.
>
> On this basis, I think I'll change my mind again - we don't mandate  
> multiple languages, since the choice to remove additional language  
> support from the implementation can be a deployment-stage decision.  
> Instead, we really do need to change the syntax.

I am not convinced that this is necessary.

> Maybe change one to be a response code:
>
> C001 LANGUAGE DE
> * OK [LANGUAGE DE] Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
> C001 OK Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
>
> Or we could change the response to always carry the current language,  
> when enumerating languages:
>
> * LANGUAGE i-default (EN DE IT i-default)
>
> Syntax:
>
> language-data = "LANGUAGE" SP lang-tag-quoted [ "(" lang-tag-quoted  
> *(SP lang-tag-quoted) ")" ]
>
> Or, when enumerating languages, i-default MUST NOT be included:
>
> A01 LANGUAGE
> * LANGUAGE ()
> A01 OK I skipped all my electives at college
>
> And finally, we could say that if you only support i-default for some  
> reason, then you MUST say NO to LANGUAGE.

I prefer the last alternative, as it means no syntactic changes to the 
LANGUAGE response for deployed servers and clients.

> I don't care which, but I vaguely prefer excluding it from the list -  
> it's noise anyway, since it's mandated to be supported.

Date: Wed, 05 Mar 2008 21:06:55 +0000
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Ned Freed wrote:

>> > Now, if this corner case does exist the cleanest solution would be to
>> > have two separate responses.
>
>> Right.
>
>> > But that might break existing implementations.
>
>> Indeed.
>
>> After reading all responses on this subject I still tend to think that
>> Tim's proposal is the best one: I can't see any good reason why a server
>> would implement LANGUAGE command and only support i-default, as this is
>> pointless.
>
> Hmm. Well, the one possibility that occurs to me os that quite a lot of
> software implements i18n support by having a separate 
> directory/file/container
> for each supported language. That way new languages can be added or 
> removed on
> the fly. And when a given language is requested a check is made to see if
> there's a corrresponding directory/file/container for it.
>
> This makes it fairly easy to end up in a situation where the only
> directory/file/container is the i-default one - in fact the usual way 
> this
> works is i-default is the built-in default that doesn't have a
> directory/file/container associated with it, so this corresponds to 
> the fairly
> common "I deleted all those nasty i18n files in order to save  space" 
> behavior.

Ok, so you are talking about misconfiguration case, while I was thinking 
about implementing LANGUAGE without implementing any languages case.

I think it is worth adding some text to describe correct behaviour for 
your case.

> This requirement would force such an implementation to check and see 
> whether or
> not non-i-default material is available before even offering the 
> extension. (Or
> you can have a extension enable/disable configuration variable, but of 
> course
> this can lead to a silly state.)

Yes. This is similar to STARTTLS, which shouldn't be advertised unless 
server can create a TLS context.

> Anyway, the inconvenience of doing this may be unavoidable, but I did 
> want to
> point out that there are reasonable ways the i-default-only case can 
> arise in
> practice.

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
From: Timo Sirainen <[email-masked]>
To: Dave Cridland <[email-masked]>
Cc: Pete Resnick <[email-masked]>, Alexey Melnikov <[email-masked]>, [email-masked], Ned Freed <[email-masked]>
Date: Thu, 06 Mar 2008 04:49:28 +0200
Sender: [email-masked]

--=-pBM0/G3Ovp74wEkKhuIb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Wed, 2008-03-05 at 16:01 +0000, Dave Cridland wrote:
> What Ned says in another message is that there is considerable work =20
> in the server knowing it only has i-default available prior to =20
> offering the extension.
>=20
> On this basis, I think I'll change my mind again - we don't mandate =20
> multiple languages, since the choice to remove additional language =20
> support from the implementation can be a deployment-stage decision. =20
> Instead, we really do need to change the syntax.

I was also getting concerned about this, but there is an easy workaroud:

1. If some i18n dirs are found at server startup, advertise the
extension.

2. When IMAP session is started and/or LANGUAGE is used and no i18n
files are found anymore, just disconnect the client and log an error.
Sysadmin will eventually restart the server which fixes the problem.

It's not like i18n files are constantly being added and removed..

--=-pBM0/G3Ovp74wEkKhuIb
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-pBM0/G3Ovp74wEkKhuIb--

Date: Wed, 5 Mar 2008 15:41:26 -0600
To: Dave Cridland <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

On 3/5/08 at 3:36 PM -0600, Pete Resnick wrote:

>if(returned languages contains i-default)
>     set supported languages to returned languages

Whoops. That's a mistake. You'd have to say:

if(returned languages contains i-default and supported languages is NULL)
     set supported languages to returned languages

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Wed, 5 Mar 2008 15:36:36 -0600
To: Dave Cridland <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

On 3/5/08 at 4:01 PM +0000, Dave Cridland wrote:

>What Pete's saying is that in the case where there is only one 
>language, which MUST be i-default, there is no semantic difference 
>between the server saying "I only use i-default" and "I am using 
>i-default" - whatever happens, the server is now using i-default.

Correct.

>The trouble is that if a client wishes to kick the server into 
>telling it what languages it supports, it instead gets a message 
>back saying that the language has been set - I have a feeling I'd 
>still need to involve semantics at too low a level to be comfortable 
>with.

I assume that the client will keep a list of languages that the 
server supports, and keep track of what language the server is using. 
Let's imagine each of these start as NULL. If the supported languages 
list is NULL, the client is going to request the list. If the current 
language is NULL, the client is going to consult the supported 
languages list, choose one and set it. Either way, the logic for 
handling the LANGUAGE response is:

if(returned languages contains i-default)
     set supported languages to returned languages
if(number of returned languages is 1)
     set currently set language to returned language

>On this basis, I think I'll change my mind again - we don't mandate 
>multiple languages, since the choice to remove additional language 
>support from the implementation can be a deployment-stage decision. 
>Instead, we really do need to change the syntax.

I don't see why.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Wed, 05 Mar 2008 16:01:32 +0000
From: Dave Cridland <[email-masked]>
To: Pete Resnick <[email-masked]>
Cc: Alexey Melnikov <[email-masked]>, <[email-masked]>, Ned Freed <[email-masked]>
Sender: [email-masked]

On Tue Mar  4 19:30:02 2008, Pete Resnick wrote:
> 
> On 3/4/08 at 9:41 AM -0800, Ned Freed wrote:
> 
>>> Tim Polk have noticed the following issue with the COMPARATOR  
>>> response:
>>> 
>>>>     A LANGUAGE response with a list containing a single language  
>>>> tag
>>>>     indicates that the server is now using that language.  A  
>>>> LANGUAGE
>>>>     response with a list containing multiple language tags  
>>>> indicates the
>>>>     server is communicating a list of available languages to the  
>>>> client,
>>>>     and no change in the active language has been made.
>> 
>> Maybe I'm not reading the draft properly, but I'm not convinced  
>> this corner
>> case exists. There are two cases: A LANGUAGE command with an  
>> argument and a
>> LANGUAGE command without an arugment. The former always gets a  
>> response
>> consisting of the single language that was selected (or an error)  
>> while the
>> latter always gets a response specifying what languages are  
>> available.
>> 
>> In particular, I don't see a way for the LANGUAGE command with an  
>> argument
>> to return the available language list.
> 
> No, I think Tim's concern is different, but I still think there is  
> nothing (besides perhaps a little wordsmithing) that needs to be  
> done: Tim's concern is that if you get back a LANGUAGE response  
> with only 1 language tag, you don't know (without saving some  
> context about what command you issued) whether that means the  
> server is sending back its list of available languages (which only  
> contains 1 item) or is telling you that it is now using the  
> language it just sent back. But I think this is a distinction  
> without a difference: If the server only supports i-default, it is  
> *always* using i-default and will *always* return a single language  
> tag (of i-default).
> 
> I'm open for additional text at the end of the paragraph to clarify.

I had to think hard about this - Pete's got a good point here - I  
just have to disagree.

What Pete's saying is that in the case where there is only one  
language, which MUST be i-default, there is no semantic difference  
between the server saying "I only use i-default" and "I am using  
i-default" - whatever happens, the server is now using i-default. The  
trouble is that if a client wishes to kick the server into telling it  
what languages it supports, it instead gets a message back saying  
that the language has been set - I have a feeling I'd still need to  
involve semantics at too low a level to be comfortable with. It's  
painful enough with unextended SEARCH, I wouldn't want to  
special-case anything else.

What Ned says in another message is that there is considerable work  
in the server knowing it only has i-default available prior to  
offering the extension.

On this basis, I think I'll change my mind again - we don't mandate  
multiple languages, since the choice to remove additional language  
support from the implementation can be a deployment-stage decision.  
Instead, we really do need to change the syntax.

Maybe change one to be a response code:

C001 LANGUAGE DE
* OK [LANGUAGE DE] Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
C001 OK Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt

Or we could change the response to always carry the current language,  
when enumerating languages:

* LANGUAGE i-default (EN DE IT i-default)

Syntax:

language-data = "LANGUAGE" SP lang-tag-quoted [ "(" lang-tag-quoted  
*(SP lang-tag-quoted) ")" ]

Or, when enumerating languages, i-default MUST NOT be included:

A01 LANGUAGE
* LANGUAGE ()
A01 OK I skipped all my electives at college

And finally, we could say that if you only support i-default for some  
reason, then you MUST say NO to LANGUAGE.

I don't care which, but I vaguely prefer excluding it from the list -  
it's noise anyway, since it's mandated to be supported.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Wed, 5 Mar 2008 07:53:03 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Pete Resnick <[email-masked]>, Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

+1

On Wed, 5 Mar 2008, Alexey Melnikov wrote:
> The new sentence is:
>
>  IMAP servers that support this extension MUST support
>  at least one language in addition to "i-default".

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Cc: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Wed, 05 Mar 2008 07:00:41 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> > In particular, I don't see a way for the LANGUAGE command with an argument
> > to return the available language list.

> You are correct regarding LANGUAGE responses in response to various
> forms of the LANGUAGE command. However the LANGUAGE response is an
> untagged response and in theory can be returned by the server in
> response to any other command. So there is still an issue of identifying
> the exact meaning of the response if it is sent in unexpected context.

I wasn't entirely buying the "it's hard to track the context of the response
without complicating your parser" assertions but this justification makes
complete sense to me.

> > Now, if this corner case does exist the cleanest solution would be to
> > have two separate responses.

> Right.

> > But that might break existing implementations.

> Indeed.

> After reading all responses on this subject I still tend to think that
> Tim's proposal is the best one: I can't see any good reason why a server
> would implement LANGUAGE command and only support i-default, as this is
> pointless.

Hmm. Well, the one possibility that occurs to me os that quite a lot of
software implements i18n support by having a separate directory/file/container
for each supported language. That way new languages can be added or removed on
the fly. And when a given language is requested a check is made to see if
there's a corrresponding directory/file/container for it.

This makes it fairly easy to end up in a situation where the only
directory/file/container is the i-default one - in fact the usual way this
works is i-default is the built-in default that doesn't have a
directory/file/container associated with it, so this corresponds to the fairly
common "I deleted all those nasty i18n files in order to save  space" behavior.

This requirement would force such an implementation to check and see whether or
not non-i-default material is available before even offering the extension. (Or
you can have a extension enable/disable configuration variable, but of course
this can lead to a silly state.)

Anyway, the inconvenience of doing this may be unavoidable, but I did want to
point out that there are reasonable ways the i-default-only case can arise in
practice.

				Ned

Date: Wed, 05 Mar 2008 14:23:53 +0000
From: Alexey Melnikov <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: Ned Freed <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Pete Resnick wrote:

> On 3/4/08 at 9:41 AM -0800, Ned Freed wrote:
>
>>> Tim Polk have noticed the following issue with the COMPARATOR response:
>>>
>>>>     A LANGUAGE response with a list containing a single language tag
>>>>     indicates that the server is now using that language.  A LANGUAGE
>>>>     response with a list containing multiple language tags 
>>>> indicates the
>>>>     server is communicating a list of available languages to the 
>>>> client,
>>>>     and no change in the active language has been made.
>>>
>> Maybe I'm not reading the draft properly, but I'm not convinced this 
>> corner
>> case exists. There are two cases: A LANGUAGE command with an argument 
>> and a
>> LANGUAGE command without an arugment. The former always gets a response
>> consisting of the single language that was selected (or an error) 
>> while the
>> latter always gets a response specifying what languages are available.
>>
>> In particular, I don't see a way for the LANGUAGE command with an 
>> argument
>> to return the available language list.
>
> No, I think Tim's concern is different, but I still think there is 
> nothing (besides perhaps a little wordsmithing) that needs to be done: 
> Tim's concern is that if you get back a LANGUAGE response with only 1 
> language tag, you don't know (without saving some context about what 
> command you issued) whether that means the server is sending back its 
> list of available languages (which only contains 1 item) or is telling 
> you that it is now using the language it just sent back. But I think 
> this is a distinction without a difference: If the server only 
> supports i-default, it is *always* using i-default and will *always* 
> return a single language tag (of i-default).
>
> I'm open for additional text at the end of the paragraph to clarify.

I think section 3.3 is good as is. I suggest adding a new sentence at 
the end of the following paragraph in Section 3.1 (LANGUAGE Extension 
Requirements):

   A server that advertises this extension MUST use the language "i-
   default" as described in [RFC2277] as its default language until
   another supported language is negotiated by the client. A server
   MUST include "i-default" as one of its supported languages.

The new sentence is:

   IMAP servers that support this extension MUST support
   at least one language in addition to "i-default".

Date: Wed, 05 Mar 2008 11:44:59 +0000
From: Alexey Melnikov <[email-masked]>
To: Ned Freed <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Ned Freed wrote:

>> Tim Polk have noticed the following issue with the COMPARATOR response:
>
>> >The LANGUAGE response is ambiguous for the corner case where the 
>> LANGUAGE
>> >extension
>> >is supported but only the i-default langauge is supported.
>> >Specifically, Section 3.3 states:
>> >
>> >    A LANGUAGE response with a list containing a single language tag
>> >    indicates that the server is now using that language.  A LANGUAGE
>> >    response with a list containing multiple language tags indicates 
>> the
>> >    server is communicating a list of available languages to the 
>> client,
>> >    and no change in the active language has been made.
>> >
>> >However, for the corner case the server's list of langauges is just 
>> i-default.
>> >
>> >Adding a requirements that "IMAP servers that support this extension 
>> MUST
>> >support at least one langauge in addition to i-default" would 
>> correct this by
>> >avoiding the corner case.   (Just an observation, I'm not set on any 
>> particular
>> >solution.)
>> >
>> So we can follow's Tim suggestion. Alternatively we can have 2 separate
>> responses to distinguish 2 cases. Which alternative is preferred by 
>> the WG?
>
> Maybe I'm not reading the draft properly, but I'm not convinced this 
> corner
> case exists. There are two cases: A LANGUAGE command with an argument 
> and a
> LANGUAGE command without an argument. The former always gets a response
> consisting of the single language that was selected (or an error) 
> while the
> latter always gets a response specifying what languages are available.
>
> In particular, I don't see a way for the LANGUAGE command with an 
> argument
> to return the available language list.

You are correct regarding LANGUAGE responses in response to various 
forms of the LANGUAGE command. However the LANGUAGE response is an 
untagged response and in theory can be returned by the server in 
response to any other command. So there is still an issue of identifying 
the exact meaning of the response if it is sent in unexpected context.

> Now, if this corner case does exist the cleanest solution would be to 
> have
> two separate responses.

Right.

> But that might break existing implementations.

Indeed.
After reading all responses on this subject I still tend to think that 
Tim's proposal is the best one: I can't see any good reason why a server 
would implement LANGUAGE command and only support i-default, as this is 
pointless.

> But other fixes are possib;e: The obvious one would be to eliminate 
> the case
> where a LANGUAGE command with an argument returns a list of available
> languages.

Date: Tue, 4 Mar 2008 13:30:02 -0600
To: Ned Freed <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 3/4/08 at 9:41 AM -0800, Ned Freed wrote:

>>Tim Polk have noticed the following issue with the COMPARATOR response:
>>
>>>     A LANGUAGE response with a list containing a single language tag
>>>     indicates that the server is now using that language.  A LANGUAGE
>>>     response with a list containing multiple language tags indicates the
>>>     server is communicating a list of available languages to the client,
>>>     and no change in the active language has been made.
>
>Maybe I'm not reading the draft properly, but I'm not convinced this corner
>case exists. There are two cases: A LANGUAGE command with an argument and a
>LANGUAGE command without an arugment. The former always gets a response
>consisting of the single language that was selected (or an error) while the
>latter always gets a response specifying what languages are available.
>
>In particular, I don't see a way for the LANGUAGE command with an argument
>to return the available language list.

No, I think Tim's concern is different, but I still think there is 
nothing (besides perhaps a little wordsmithing) that needs to be 
done: Tim's concern is that if you get back a LANGUAGE response with 
only 1 language tag, you don't know (without saving some context 
about what command you issued) whether that means the server is 
sending back its list of available languages (which only contains 1 
item) or is telling you that it is now using the language it just 
sent back. But I think this is a distinction without a difference: If 
the server only supports i-default, it is *always* using i-default 
and will *always* return a single language tag (of i-default).

I'm open for additional text at the end of the paragraph to clarify.

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Tue, 04 Mar 2008 19:12:41 +0000
From: Dave Cridland <[email-masked]>
To: Ned Freed <[email-masked]>
Cc: <[email-masked]>, Alexey Melnikov <[email-masked]>
Sender: [email-masked]

On Tue Mar  4 17:41:14 2008, Ned Freed wrote:
> Maybe I'm not reading the draft properly, but I'm not convinced  
> this corner
> case exists. There are two cases: A LANGUAGE command with an  
> argument and a
> LANGUAGE command without an arugment. The former always gets a  
> response
> consisting of the single language that was selected (or an error)  
> while the
> latter always gets a response specifying what languages are  
> available.
> 
> In particular, I don't see a way for the LANGUAGE command with an  
> argument
> to return the available language list.
> 
> 
Client implementors really like to be able to handle responses  
without knowing which command emitted them. Makes for cleaner designs.

So there *is* a corner case, just not an obvious one.

Dave.

> Now, if this corner case does exist the cleanest solution would be  
> to have
> two separate responses. But that might break existing  
> implementations. But
> other fixes are possib;e: The obvious one would be to eliminate the  
> case
> where a LANGUAGE command with an argument returns a list of  
> available
> languages.
> 
> 				Ned
> 
> 
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Tue, 04 Mar 2008 19:08:39 +0000
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: <[email-masked]>, Alexey Melnikov <[email-masked]>, Arnt Gulbrandsen <[email-masked]>
Sender: [email-masked]

On Tue Mar  4 18:52:20 2008, Mark Crispin wrote:
> Can someone explain to me why the LANGUAGE extension is be offered  
> at all if i-default is the only defined extension?  Can't we do an  
> end run by stating that an implementation of the LANGUAGE extension  
> MUST allow at least one language other than i-default?
> 
> I have no sympathy for implementions that want to "support  
> LANGUAGE" as a checklist item, yet not actually implement any  
> language.

+1 - I wasn't thinking about semantics, only syntax.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Tue, 4 Mar 2008 11:07:47 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked], Alexey Melnikov <[email-masked]>, Dave Cridland <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

On Tue, 4 Mar 2008, Arnt Gulbrandsen wrote:
> Mark Crispin writes:
>> Can't we do an end run by stating that an implementation of the LANGUAGE 
>> extension MUST allow at least one language other than i-default?
> Isn't that what Tim suggests?

I think so, but I haven't kept close enough track.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Tue, 4 Mar 2008 20:04:01 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Mark Crispin <[email-masked]>, Dave Cridland <[email-masked]>
Sender: [email-masked]

Mark Crispin writes:
> Can't we do an end run by stating that an implementation of the 
> LANGUAGE extension MUST allow at least one language other than 
> i-default?

Isn't that what Tim suggests?

Arnt

Date: Tue, 4 Mar 2008 11:00:25 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Stephane Bortzmeyer <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS   PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Organization: UW Technology
Sender: [email-masked]

On Tue, 4 Mar 2008, Stephane Bortzmeyer wrote:
>> The actual correct collation, assuming(!) surname-first collation and
>> Latin character ordering(!!), is:
>> due to where the surname is located in various cultures.
> Is it a good idea to sort on the ordering of the sender's culture? If
> the ordering is to be useful for the human user, it should be
> according the receiver's culture, no?

My answer to this question is "yes to all of the above".

That is, a truly correct collation uses knowledge of both the sender's 
culture *AND* the receiver's culture; and the ways that these interact are 
not necessarily obvious.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Date: Tue, 4 Mar 2008 10:52:20 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: [email-masked], Alexey Melnikov <[email-masked]>, Dave Cridland <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Organization: UW Technology
Sender: [email-masked]

On Tue, 4 Mar 2008, Arnt Gulbrandsen wrote:
>> * LANGUAGE i-default // I am now using i-default
>> * LANGUAGE (i-default) // I can only do i-default
> A little subtle, don't you think? Demanding of tired brains? I had to look 
> twice...

I agree.  IMAP's architecture may be subtle at times, but not THAT subtle.

> Tim's suggestion to only offer the choice when you're offering more than one 
> alternative makes more sense to me.

Can someone explain to me why the LANGUAGE extension is be offered at all 
if i-default is the only defined extension?  Can't we do an end run by 
stating that an implementation of the LANGUAGE extension MUST allow at 
least one language other than i-default?

I have no sympathy for implementions that want to "support LANGUAGE" as a 
checklist item, yet not actually implement any language.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Cc: IMAP Extensions WG <[email-masked]>
Date: Tue, 04 Mar 2008 09:41:14 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
To: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

> Tim Polk have noticed the following issue with the COMPARATOR response:

> >The LANGUAGE response is ambiguous for the corner case where the LANGUAGE
> >extension
> >is supported but only the i-default langauge is supported.
> >Specifically, Section 3.3 states:
> >
> >    A LANGUAGE response with a list containing a single language tag
> >    indicates that the server is now using that language.  A LANGUAGE
> >    response with a list containing multiple language tags indicates the
> >    server is communicating a list of available languages to the client,
> >    and no change in the active language has been made.
> >
> >However, for the corner case the server's list of langauges is just i-default.
> >
> >Adding a requirements that "IMAP servers that support this extension MUST
> >support at least one langauge in addition to i-default" would correct this by
> >avoiding the corner case.   (Just an observation, I'm not set on any particular
> >solution.)
> >
> So we can follow's Tim suggestion. Alternatively we can have 2 separate
> responses to distinguish 2 cases. Which alternative is preferred by the WG?

Maybe I'm not reading the draft properly, but I'm not convinced this corner
case exists. There are two cases: A LANGUAGE command with an argument and a
LANGUAGE command without an arugment. The former always gets a response
consisting of the single language that was selected (or an error) while the
latter always gets a response specifying what languages are available.

In particular, I don't see a way for the LANGUAGE command with an argument
to return the available language list.

Now, if this corner case does exist the cleanest solution would be to have
two separate responses. But that might break existing implementations. But
other fixes are possib;e: The obvious one would be to eliminate the case
where a LANGUAGE command with an argument returns a list of available
languages.

				Ned

Date: Tue, 4 Mar 2008 12:49:19 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Cc: Mark Crispin <[email-masked]>, Stephane Bortzmeyer <[email-masked]>
Sender: [email-masked]

Stephane Bortzmeyer writes:
> On Wed, Feb 27, 2008 at 10:26:44AM -0800,
>  Mark Crispin <[email-masked]> wrote a message of 162 lines which said:
>
>>  The actual correct collation, assuming(!) surname-first collation 
>>  and Latin character ordering(!!), is:
> ...
>>  due to where the surname is located in various cultures.
>
> Is it a good idea to sort on the ordering of the sender's culture? If 
> the ordering is to be useful for the human user, it should be 
> according the receiver's culture, no?

There's support for that. You have to read a few drafts and RFCs, though.

The draft in question defines a sort command and some sort keys, and it 
permits defining more sort keys. Other documents allow defining 
collations (sort orders) and using them im IMAP.

Sorting by From field the way most MUAs do requires defining a new 
collation and a new sort key, since this draft's "from" key uses only 
the localpart ('arnt' in the case of this message). Personally I think 
that sort key isn't at all fortunate, but it's too late to change that. 
There's decade-old running code already and the most important thing is 
to document the deployed extension.

Arnt

Date: Tue, 4 Mar 2008 12:19:57 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Cc: Alexey Melnikov <[email-masked]>, Dave Cridland <[email-masked]>
Sender: [email-masked]

Dave Cridland writes:
> So you'd get:
>
> * LANGUAGE i-default // I am now using i-default
> * LANGUAGE (i-default) // I can only do i-default

A little subtle, don't you think? Demanding of tired brains? I had to 
look twice...

Tim's suggestion to only offer the choice when you're offering more than 
one alternative makes more sense to me.

Arnt

Date: Tue, 4 Mar 2008 12:13:59 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Alexey Melnikov writes:
>> Adding a requirements that "IMAP servers that support this extension MUST
>> support at least one langauge in addition to i-default" would correct 
>> this by
>> avoiding the corner case.   (Just an observation, I'm not set on any 
>> particular
>> solution.)
>
> So we can follow's Tim suggestion. Alternatively we can have 2 
> separate responses to distinguish 2 cases. Which alternative is 
> preferred by the WG?

Tim's suggestion. What he's found is that LANGUAGE without languages has 
syntactic problems. I think it's likely that there are other problems 
too, which his suggestion avoids at low or no cost.

Arnt

Subject: Re: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Date: Tue, 04 Mar 2008 10:42:56 +0000
From: Dave Cridland <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Mon Mar  3 21:26:11 2008, Alexey Melnikov wrote:
> 
> Tim Polk have noticed the following issue with the COMPARATOR  
> response:
> 
>> The LANGUAGE response is ambiguous for the corner case where the  
>> LANGUAGE
>> extension
>> is supported but only the i-default langauge is supported.
>> Specifically, Section 3.3 states:
>> 
>>    A LANGUAGE response with a list containing a single language tag
>>    indicates that the server is now using that language.  A  
>> LANGUAGE
>>    response with a list containing multiple language tags  
>> indicates the
>>    server is communicating a list of available languages to the  
>> client,
>>    and no change in the active language has been made.
>> 
>> However, for the corner case the server's list of langauges is  
>> just i-default.
>> 
>> Adding a requirements that "IMAP servers that support this  
>> extension MUST
>> support at least one langauge in addition to i-default" would  
>> correct this by
>> avoiding the corner case.   (Just an observation, I'm not set on  
>> any particular
>> solution.)
>> 
> So we can follow's Tim suggestion. Alternatively we can have 2  
> separate responses to distinguish 2 cases. Which alternative is  
> preferred by the WG?

I like the idea of sticking the available languages in parentheses.  
It's the simplest change I can think of.

So you'd get:

* LANGUAGE i-default // I am now using i-default
* LANGUAGE (i-default) // I can only do i-default

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Tue, 4 Mar 2008 11:32:16 +0100
From: Stephane Bortzmeyer <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>, [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS  	PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Organization: NIC France
Sender: [email-masked]

On Wed, Feb 27, 2008 at 10:26:44AM -0800,
 Mark Crispin <[email-masked]> wrote 
 a message of 162 lines which said:

> The actual correct collation, assuming(!) surname-first collation and 
> Latin character ordering(!!), is:
...
> due to where the surname is located in various cultures.

Is it a good idea to sort on the ordering of the sender's culture? If
the ordering is to be useful for the human user, it should be
according the receiver's culture, no?

Date: Mon, 03 Mar 2008 21:26:11 +0000
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Tim Polk's DISCUSS on draft-ietf-imapext-i18n-15.txt
Sender: [email-masked]

Tim Polk have noticed the following issue with the COMPARATOR response:

>The LANGUAGE response is ambiguous for the corner case where the LANGUAGE
>extension
>is supported but only the i-default langauge is supported.
>Specifically, Section 3.3 states:
>
>    A LANGUAGE response with a list containing a single language tag
>    indicates that the server is now using that language.  A LANGUAGE
>    response with a list containing multiple language tags indicates the
>    server is communicating a list of available languages to the client,
>    and no change in the active language has been made.
>
>However, for the corner case the server's list of langauges is just i-default.
>
>Adding a requirements that "IMAP servers that support this extension MUST
>support at least one langauge in addition to i-default" would correct this by
>avoiding the corner case.   (Just an observation, I'm not set on any particular
>solution.)
>
So we can follow's Tim suggestion. Alternatively we can have 2 separate 
responses to distinguish 2 cases. Which alternative is preferred by the WG?

Date: Mon, 3 Mar 2008 17:42:58 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, [email-masked], Dan Karp <[email-masked]>, Alexey Melnikov <[email-masked]>, Messaging Organization <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Organization: UW Technology
Sender: [email-masked]

On Mon, 3 Mar 2008, Dave Cridland wrote:
> I can think of circumstances where sorting by the full email address is 
> useful (which would also catch the above), and where sorting by the full name 
> is also useful.

I agree with all this; which is why the SORT/THREAD specification is 
extensible.  I have said many times that I would support a follow-on 
specification that adds new sort keys and thread algorithms.  As long as 
the keys are not preposterous I would seriously consider implementing 
them.  I might even implement a preposterous key if the implementation is 
straightforward.

My objections are not -- and have never been -- about adding new keys. 
My objections are to ademand that existing keys with a long history be 
removed.

If the keys in question are difficult to implement in a server, then the 
server is fundamentally flawed in its frameworks.  The most plausible 
explanation would be a server that uses a database back end that doesn't 
have ready access to the necessary addr-mailbox components.  That's no 
excuse.  The same data are needed to deliver satisfactory ENVELOPE 
performance; if the server can not do that well then it is not doing IMAP 
well.

Then there is the postulate that the presence of "useless" keys will make 
it more difficult to add "more useful" keys.  That would happen in only 
two scenarios:
  (1) The existing keys are good enough.
and/or
  (2) Nobody cares enough about the issue.

Neither is a justification to remove functionality.

If something is "good enough", then it should win especially when it is 
entrenched for decades.  The ridiculous notion that "good enough" must 
yield to "perfect" has led to the current situation where the IETF never 
gets anything done.

And, if nobody cared enough about the issue, then the "more useful" keys 
are a solution in search of a problem and should die.

Put another way, we should add new sort keys iff the current set of keys 
are not "good enough" AND some consituency cares.  I consider both 
conditions to be plausible; but my stake is that these new keys are 
additions, not replacements, for the existing keys.

> Threads "changing" is indeed a problem for many cases - I'm not sure I see 
> the value in ordering by subject to address it though - could you explain a 
> bit more? (I'm not saying there's no value, I just can't visualize how this 
> helps).

Typically, the thread-changer thinks that he has broken the thread by 
changing the subject, but did not do anything about the references links. 
A SUBJECT sort or ORDEREDSUBJECT thread breaks the thread, since neither 
uses the references links at all.

Now, there are also cases where the subject changes but the thread is 
legitimately the same.  That's why I don't use just one collation; I 
normally use THREAD=REFERENCES by default but switch to other collations 
when the situation is such that I think one of them will do a better job.

> Hard - one has to consider what the flag's precedence is, and how to order 
> flag combinations. Some cases are easy - one might say that (\Flagged) > 
> (\Flagged \Seen) > () > (\Seen), but where user-defined keywords are heavily 
> used, any sort order really needs to be client-defined.

I agree.  If flag sorting is done at all, it needs to be engineered 
carefully and probably via a client-defined mechanism.

> I've seen people doing both "Sent" and "Received" date sorts.

Strong agreement.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Date: Mon, 03 Mar 2008 20:59:54 +0000
From: Dave Cridland <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, <[email-masked]>, Dan Karp <[email-masked]>, Alexey Melnikov <[email-masked]>, Messaging Organization <[email-masked]>
Sender: [email-masked]

Dropped ietf@ietf from the CC list, and added [email-masked]... I  
think we're moving into the realms of what could usefully be  
discussed for future extension. The Mail ORGanization list was  
created to act as a venue some time ago, and I've been remiss in not  
trying to stimulate discussion. (Partly because I was waiting for  
things like this draft to be pushed through to RFC, in my defense).

On Mon Mar  3 18:44:35 2008, Mark Crispin wrote:
> On Mon, 3 Mar 2008, Alexey Melnikov wrote:
>>> (In some clients, the sorted list is scrolled to whichever   
>>> message was previously selected, so it's a fast way of finding  
>>> other  messages by the same person).
>> Yea, I do this frequently in Thunderbird.
> 
> As do I.
> 
> Note that an address sort that collates all "mrc" messages together  
> is more likely to do the right thing that one that collates by name  
> and has to figure out that "Mark Crispin", "Mark R. Crispin",  
> "Crispin, Mark", "M Crispin", "Mr. Mark Crispin", "IMAP Support",  
> etc. are all the same.
> 
> 
I can think of circumstances where sorting by the full email address  
is useful (which would also catch the above), and where sorting by  
the full name is also useful.

The case where sorting by only the mailbox is useful is where this is  
common across domains to mean the same person - true for you, not so  
for others - including myself. On the other hand, people rarely vary  
the full name they use in their From header field, even between  
accounts, in my experience - this may be true for most people and  
false for you, of course.

Equally, if you take subaddressing into account, then the local-part  
varies for a single user.

FWIW, I note that full names are sometimes varied for use in distinct  
roles ("IMAP Support"), which might even be a positive point for  
sorting only by full names.

>> I frequently sort by subject, because many clients still don't  
>> support proper threading (threads can become broken due to lack of  
>> the References header).
> 
> I also sort by Subject (although more commonly ORDERSUBJECT  
> threading than direct SUBJECT sorting) in cases where the thread  
> tree has become so complex that it has lost value.  A classic  
> example is when a new thread is started by replying to a message in  
> a thread that has little or no relationship to the thread.  Think  
> USENET.
> 
> 
Threads "changing" is indeed a problem for many cases - I'm not sure  
I see the value in ordering by subject to address it though - could  
you explain a bit more? (I'm not saying there's no value, I just  
can't visualize how this helps).

>> I also sort by IMAP flags (i.e. I want to see all important  
>> messages), size and rarely by date.
> 
> Usually for flags I would do a filter rather than a SORT, but it  
> seems reasonable to me to add such a thing in the first round of  
> extensions.
> 
> 
Hard - one has to consider what the flag's precedence is, and how to  
order flag combinations. Some cases are easy - one might say that  
(\Flagged) > (\Flagged \Seen) > () > (\Seen), but where user-defined  
keywords are heavily used, any sort order really needs to be  
client-defined.

> Date sorting seems to be pretty common with our users.  Some people  
> who prefer their mail "in order" use it instead of sequence order  
> in the mailbox (no sort) which tends to be an INTERNALDATE sort.

I've seen people doing both "Sent" and "Received" date sorts.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Mon, 3 Mar 2008 11:06:29 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dave Cridland <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, IETF Discussion <[email-masked]>, [email-masked], Dan Karp <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

On Mon, 3 Mar 2008, Dave Cridland wrote:
> Still, though, the presence of useless SORT keys would only be a significant 
> problem if they were especially difficult to implement, and the large 
> deployments of SORT indicate that they're not. Even if it's only PINE, that's 
> still a pretty vast number of users who may be relying on this behaviour.

+1.

It amazes me that this is even questioned, much less that the entire 
[email-masked] list has been dragged into the fray.  This discussion 
belongs either in ietf-imapext, or if an appeal is to be lodged it should 
be made with the IESG.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Mon, 3 Mar 2008 10:44:35 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Dave Cridland <[email-masked]>, Cyrus Daboo <[email-masked]>, [email-masked], IETF Discussion <[email-masked]>, Dan Karp <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort          (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to         Proposed Standard
Sender: [email-masked]

On Mon, 3 Mar 2008, Alexey Melnikov wrote:
>> (In some 
>> clients, the sorted list is scrolled to whichever  message was previously 
>> selected, so it's a fast way of finding other  messages by the same 
>> person).
> Yea, I do this frequently in Thunderbird.

As do I.

Note that an address sort that collates all "mrc" messages together is 
more likely to do the right thing that one that collates by name and has 
to figure out that "Mark Crispin", "Mark R. Crispin", "Crispin, Mark", "M 
Crispin", "Mr. Mark Crispin", "IMAP Support", etc. are all the same.

> I frequently sort by subject, because many clients still don't support proper 
> threading (threads can become broken due to lack of the References header).

I also sort by Subject (although more commonly ORDERSUBJECT threading than 
direct SUBJECT sorting) in cases where the thread tree has become so 
complex that it has lost value.  A classic example is when a new thread is 
started by replying to a message in a thread that has little or no 
relationship to the thread.  Think USENET.

> I also sort by IMAP flags (i.e. I want to see all important messages), size 
> and rarely by date.

Usually for flags I would do a filter rather than a SORT, but it seems 
reasonable to me to add such a thing in the first round of extensions.

Date sorting seems to be pretty common with our users.  Some people who 
prefer their mail "in order" use it instead of sequence order in the 
mailbox (no sort) which tends to be an INTERNALDATE sort.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Mon, 3 Mar 2008 10:27:16 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort          (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS)         to Proposed Standard
Sender: [email-masked]

I agree and will/have address[ed] these in AUTH48.

On Mon, 3 Mar 2008, Alexey Melnikov wrote:
> The IESG wrote:
>
>> The IESG is considering the following document again now that important 
>> dependencies are ready:
>> 
>> - 'INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS'
>>   <draft-ietf-imapext-sort-19.txt> as a Proposed Standard
>> 
> Hi Mark,
> I've just reread section 7 of -19 and found some relatively minor things that 
> need fixing:
>
>> 7. Internationalization Considerations
>>
>>     As stated in the introduction, the server SHOULD support the
>>     [IMAP-I18N] COMPARATOR extension and follow its rules to perform
>>     collations in the SORT and THREAD extensions.
>
> As per your recent suggestion in the IMAPEXT WG, the COMPARATOR extension was 
> renamed to I18NLEVEL=2.
> The last sentence in section 1 needs to be updated as well.
>
>>     If the server does not support COMPARATOR, strings MUST be collated
>>     according to the i;unicode-casemap collation described in
>>     [UNICASEMAP].
>
> I think you also mean that support for at least I18NLEVEL=1 is REQUIREd.
>
>>     As described in [IMAP-I18N], strings in charsets other than US-ASCII
>>     and UTF-8 MUST be converted to UTF-8 and compared in ascending order
>>     according to the selected or active collation algorithm.  If the
>>     server does not support the [IMAP-I18N] COMPARATOR extension, the
>>     collation algorithm used is the "en;ascii-casemap" collation
>>     described in [COMPARATOR].
>
> The last sentence contradict the second paragraph quoted above. I think you 
> forgot to delete it in -19.
> If for some reason you want to keep it, then please change "en;ascii-casemap" 
> to "i;ascii-casemap", as "en;ascii-casemap" is invalid.
>
> I can recommend some replacement text, if you want.
>
>

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Mon, 03 Mar 2008 16:23:01 +0000
From: Alexey Melnikov <[email-masked]>
To: Dave Cridland <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, [email-masked], Mark Crispin <[email-masked]>, IETF Discussion <[email-masked]>, Dan Karp <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort  (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

Speaking as email client user:

Dave Cridland wrote:

>On Mon Mar  3 15:03:20 2008, Cyrus Daboo wrote:
>  
>
>>--On March 2, 2008 11:07:50 PM -0800 Dan Karp <[email-masked]>  
>>wrote:
>>    
>>
>>>The purpose of sorting in mail clients is so that users can find
>>>messages they're looking for.
>>>      
>>>
>>Actually you need to look at your use cases in more detail because  
>>a lot of times searching is a much better solution than sorting.  
>>e.g. the case of trying to find email from a particular person - a  
>>search is much better. Yes you do have to do a little more work to  
>>setup the search (type something in) rather than the single click  
>>sorting requires, but on large mailboxes you will usually see what  
>>you want immediately without having to scroll through the sorted  
>>list looking for what you want.
>>    
>>
>I don't think Dan's saying that using SORT for searching is a good  
>idea, but it's certainly what people do based on my observations,  
>too. (In some clients, the sorted list is scrolled to whichever  
>message was previously selected, so it's a fast way of finding other  
>messages by the same person).
>  
>
Yea, I do this frequently in Thunderbird.

>>For the most part I question why anyone would really want to sort  
>>on any of the text fields. The only sort I have found useful is  
>>sort by size - and then only to help in culling large messages when  
>>my quota-limited mailboxes get full.
>>    
>>
>My client does no sorting at all, so I'm hard pressed to give any  
>alternative use-cases. Generally, people seem to like sorting by date  
>fields, though again not a text field. I've also seen people who like  
>to sort by subject, essentially as a grouping thing - better done  
>with threading, but still.
>  
>
I frequently sort by subject, because many clients still don't support 
proper threading (threads can become broken due to lack of the 
References header).

I also sort by IMAP flags (i.e. I want to see all important messages), 
size and rarely by date.

Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Date: Mon, 03 Mar 2008 16:14:57 +0000
From: Dave Cridland <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Cc: IETF Discussion <[email-masked]>, <[email-masked]>, Dan Karp <[email-masked]>, Mark Crispin <[email-masked]>
Sender: [email-masked]

On Mon Mar  3 15:03:20 2008, Cyrus Daboo wrote:
> --On March 2, 2008 11:07:50 PM -0800 Dan Karp <[email-masked]>  
> wrote:
> 
>> The purpose of sorting in mail clients is so that users can find
>> messages they're looking for.
> 
> Actually you need to look at your use cases in more detail because  
> a lot of times searching is a much better solution than sorting.  
> e.g. the case of trying to find email from a particular person - a  
> search is much better. Yes you do have to do a little more work to  
> setup the search (type something in) rather than the single click  
> sorting requires, but on large mailboxes you will usually see what  
> you want immediately without having to scroll through the sorted  
> list looking for what you want.
> 
> 
I don't think Dan's saying that using SORT for searching is a good  
idea, but it's certainly what people do based on my observations,  
too. (In some clients, the sorted list is scrolled to whichever  
message was previously selected, so it's a fast way of finding other  
messages by the same person).

> For the most part I question why anyone would really want to sort  
> on any of the text fields. The only sort I have found useful is  
> sort by size - and then only to help in culling large messages when  
> my quota-limited mailboxes get full.

My client does no sorting at all, so I'm hard pressed to give any  
alternative use-cases. Generally, people seem to like sorting by date  
fields, though again not a text field. I've also seen people who like  
to sort by subject, essentially as a grouping thing - better done  
with threading, but still.

> View filtering can often be done with a single click (e.g.  
> option-click on the from address of a message currently in view  
> causes the view to show only those messages with that address).  
> This is far superior to sorting.

My client does the same (albeit with different clicks involved), and  
I too find it very useful. I'm not sure I'd say "far superior to  
sorting", since I don't see them as related, but I get your drift.

Still, though, the presence of useless SORT keys would only be a  
significant problem if they were especially difficult to implement,  
and the large deployments of SORT indicate that they're not. Even if  
it's only PINE, that's still a pretty vast number of users who may be  
relying on this behaviour.

When I last brought this up, I got the impression that it was  
considered by the vast majority that it was simply too late to change  
FROM to be useful.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

Date: Mon, 03 Mar 2008 10:03:20 -0500
From: Cyrus Daboo <[email-masked]>
To: Dan Karp <[email-masked]>, Mark Crispin <[email-masked]>
cc: [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

Hi Dan,

--On March 2, 2008 11:07:50 PM -0800 Dan Karp <[email-masked]> wrote:

> The purpose of sorting in mail clients is so that users can find
> messages they're looking for.

Actually you need to look at your use cases in more detail because a lot of 
times searching is a much better solution than sorting. e.g. the case of 
trying to find email from a particular person - a search is much better. 
Yes you do have to do a little more work to setup the search (type 
something in) rather than the single click sorting requires, but on large 
mailboxes you will usually see what you want immediately without having to 
scroll through the sorted list looking for what you want.

For the most part I question why anyone would really want to sort on any of 
the text fields. The only sort I have found useful is sort by size - and 
then only to help in culling large messages when my quota-limited mailboxes 
get full. The rest of the time I either use no sort, or threading, or view 
filtering (view only the results of a search). View filtering can often be 
done with a single click (e.g. option-click on the from address of a 
message currently in view causes the view to show only those messages with 
that address). This is far superior to sorting. At least that works well 
for me. Others may have different ways of processes their email where 
sorting may make more sense.

-- 
Cyrus Daboo

Date: Mon, 03 Mar 2008 11:22:14 +0000
From: Alexey Melnikov <[email-masked]>
To: Mark Crispin <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort  (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

The IESG wrote:

>The IESG is considering the following document again now that 
>important dependencies are ready:
>
>- 'INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS'
>   <draft-ietf-imapext-sort-19.txt> as a Proposed Standard
>  
>
Hi Mark,
I've just reread section 7 of -19 and found some relatively minor things 
that need fixing:

> 7. Internationalization Considerations
>
>     As stated in the introduction, the server SHOULD support the
>     [IMAP-I18N] COMPARATOR extension and follow its rules to perform
>     collations in the SORT and THREAD extensions.

As per your recent suggestion in the IMAPEXT WG, the COMPARATOR 
extension was renamed to I18NLEVEL=2.
The last sentence in section 1 needs to be updated as well.

>     If the server does not support COMPARATOR, strings MUST be collated
>     according to the i;unicode-casemap collation described in
>     [UNICASEMAP].

I think you also mean that support for at least I18NLEVEL=1 is REQUIREd.

>     As described in [IMAP-I18N], strings in charsets other than US-ASCII
>     and UTF-8 MUST be converted to UTF-8 and compared in ascending order
>     according to the selected or active collation algorithm.  If the
>     server does not support the [IMAP-I18N] COMPARATOR extension, the
>     collation algorithm used is the "en;ascii-casemap" collation
>     described in [COMPARATOR].

The last sentence contradict the second paragraph quoted above. I think 
you forgot to delete it in -19.
If for some reason you want to keep it, then please change 
"en;ascii-casemap" to "i;ascii-casemap", as "en;ascii-casemap" is invalid.

I can recommend some replacement text, if you want.

Date: Mon, 3 Mar 2008 11:40:45 +0100
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Cc: Dan Karp <[email-masked]>, Mark Crispin <[email-masked]>
Sender: [email-masked]

Dan:

The draft can't be changed any more.

But there's good news. Like you, I have a database, and I've implemented 
SORT. It took less than a day. A part of those hours was wasted time, 
because as you point out, only Mark's client wants the kind of address 
sorting which the draft offers. But it was less than a day. Wasting 
part of a day isn't really something to get excited about. Worse things 
happen every month.

Arnt

Date: Sun, 2 Mar 2008 23:53:04 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Dan Karp <[email-masked]>
cc: [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

On Sun, 2 Mar 2008, Dan Karp wrote:
> Once the draft makes it to Proposed Standard, getting a second draft out
> with a variant version of these sorts becomes exponentially harder.

When there is both a de facto standard that has existed for over 10 years, 
and a completely incompatible de jure standard created because some guy 
didn't like the de facto standard, then getting any standard becomes 
impossible.

The SORT specification is infinitely extensible.  The original keys are 
there, and remain there forever.  However, nothing prevents the creation 
of FROM2, FROM.PLUS, ZIMBRAFROM, etc. as extensions to the keys.  That is, 
all new forms of SORT would be a superset.

The only thing that a client need concern itself about with a server is if 
the server implements (at least) the SORT level it desires.  If the client 
just cares about the original, 10+ year old, keys, then any server SORT 
will work, because all SORT is a superset of the original.

Should you get your way, there would be now two incompatible sets of keys. 
There would be the set that has been around for more than a decade, and 
the new incompatible set that *subsets* the original set instead of 
*supersets* it.

Clients would have to know which set is implemented by the server.  If the 
server implements the wrong set, then the client will have to do all the 
work in the client.  The end result is sufficient fear, uncertainty, and 
doubt that nobody will use server-based SORT at all.

> There is a clear consensus in the mail user agent marketplace for how
> address sorts should behave.

Empirical testing of a few clients does not constitute a "concensus".

> New revisions of the legacy servers would advertise both the legacy
> "SORT" and the standard "SORT=BASE".  Legacy clients would see "SORT"
> and behave exactly as before.  New clients would see both "SORT" and
> "SORT=BASE" and act accordingly.

I already explained why this can not work.

SORT=BASE, by definition in the current SORT specification, is a 
*superset* of the 10+ year SORT.

    A server which supports the base-level SORT extension indicates this
    with a capability name which starts with "SORT".  Future,
    upwards-compatible extensions to the SORT extension will all start
    with "SORT", indicating support for this base level.

Consequently, current software does
    if (strcasencmp (cap,"SORT",4)) hasSort = 1;

Your demand has the effect of breaking this software.  The new extension 
would have to be called something like ZIMBRASORT, since the 10+ year old 
SORT specification reserves the entire SORT* extension namespace for 
upwards-compatible extensions.  ZIMBRASORT, by being a subset (and hence 
incompatible) must therefore have a different name.

The result is that, with IMAP sorting forked into two incompatible forms 
of sort, nobody will implement either.

> I believe that your client already does client-side FROM sorting for
> servers that don't advertise SORT.

Yes, and it is as slow as cold molasses in January when such a server is 
encountered.  Users generally don't even try to sort or thread on such 
deficient servers since it is entirely too painful.

I do not understand why you are so insistant upon demanding that 
something that has been in use for 10+ years be removed.  Forks in 
specifications are bad things.

Let me take that back.  Forks in specifications are not bad things; forks 
in specifications are VERY bad things.  The Achilles' heel of UNIX has 
been the multiplicity of forks; and it's not just the forks of SysV vs. 
BSD vs. Linux but incompatible forks within SysV, BSD, and Linux.

When IMAP forked in the past the result was incredible chaos and confusion 
that should never be repeated.  Ever since IMAP has gone to considerable 
effort to maintain upwards compatibility and not fork.  That is why, among 
other things, we have BODY and BODYSTRUCTURE.  That is why IMAP has at 
times chosen ugly syntax and mechanisms, because doing so provided the 
functionality without creating a fork.

If it was difficult to implement the keys in question, that would be one 
thing; but it is utterly trivial to do so.  The necessary framework is 
provided by virtue of the ENVELOPE (a base specification requirement since 
the very beginnings of IMAP) and SUBJECT sorting.  A high school kid can 
implement these keys with that framework in place.

Nothing prevents the addition of the "better" keys that you advocate, 
other than your vaguely expressed fear that the "better" keys won't happen 
if the "not better" keys are allowed to survive.  If the "better" keys 
can't survive that modest hurdle then maybe they really aren't "better" 
after all -- or maybe the entire argument is pointless and you should just 
let sleeping dogs lie.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Sun, 2 Mar 2008 23:07:50 -0800 (PST)
From: Dan Karp <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: [email-masked], IMAP Extensions WG <[email-masked]>
Subject: Re: Last Call: draft-ietf-imapext-sort (INTERNET MESSAGE ACCESS PROTOCOL - SORT AND THREAD EXTENSIONS) to Proposed Standard
Sender: [email-masked]

Let's step back for a moment.

The purpose of sorting in mail clients is so that users can find
messages they're looking for.

After sorting on INTERNALDATE, the user skips down the "Received" column
to the range that their target messages fall in.  And, voila!  The
messages are there.

After sorting on SUBJECT, the user skips through the results to the set
of messages with the correct normalized subject.  This works because the
user mentally normalizes the Subject header in the same way that the
SORT extension does.  (The transform's easy to do, and years of mail
client use have trained them to do this; they're not looking under 'R'
for "re: Your Brains".)  And, voila!  They find the messages.

But sorting on FROM?  That's where things break down for the draft.

I took a look at 7 different mail clients: Gmail, mutt, Outlook Express,
PC-Pine, Thunderbird, Yahoo! Mail, and Zimbra.  In every single one of
these clients, there is a list of mail messages.  And in that list,
every client has a "From" column.  And every one of those "From" columns
contains the RFC2822 display-name of the From header of the relevant
message, or the RFC2822 addr-spec if there's no display-name.  Every
single one.

Gmail ("search, not sort") doesn't allow you to sort on "From".  But
every other client offers a "From" sort.  And every client *but* PC-Pine
does it exactly the same way: A straight text sort on the data displayed
in the "From" column.  And this makes sense, because a user knows the
name of the author of the message they're trying to find, and they can
skip to the right part of the list.  Things work because (a) the user
knows the sort key of the message, (b) the sort key is displayed in the
user interface, and (c) the list is ordered on that displayed sort key.
It's just as intuitive as the received-date search.

PC-Pine is different.  Looking at the first page of results when I point
it at the INBOX of my test account, I see the following "FROM sorted"
list (names but not sort order have been changed to protect the
innocent):

  N  21 Jun 21 Betsy Avertain                  (787) ...
  N  22 Sep 24 Hurricane Electric, Inc.       (3465) ...
  N  23 Sep 19 Waverly Bindle                  (11K) ...
  N  24 Dec  7 [email-masked]         (12K) ...
  N  25 Feb 23 Brian Pearson                  (5878) ...
  N  26 Jan 24 patrick walton                  (66K) ...
  N  27 Nov  9 build                          (2022) ...
  N  28 Nov 18 Calvin Poutine                 (6275) ...
     29 Feb  8 Leonardo Merman                (2535) ...

Those are the messages that PC-Pine, using the FROM sort rules as
defined in the draft, has decided to sort in the "B-C" range.  Note that
the sort key itself (the local-part of the RFC2822 addr-spec) isn't
displayed at all.  And it's not just PC-Pine: NO CLIENT that I know of
displays that information by default in the message listing; most if not
all can't even be configured to show it.

So if I'm looking for messages from Pat Walton, I'm going to first look
under "P" and not find anything, then wonder if I haven't actually filed
his messages into a different mailbox, then rack my brain until if I'm
lucky I remember that his username is actually "bpw", then skip to
somewhere in the "B"s and start trying to remember if Brian Pearson's
username was "brianp" (in which case I need to scroll up to find the
block of messages from Pat) or "bpearson" (in which case I need to
scroll down).

In order to find a message in this "sorted" list, you have to first
remember a piece of information that the client will not display.  If
you can't remember this piece of information -- and, in the world of
display-name-based autocomplete and correspondents you rarely send
mail to, that's a large and growing probability -- you're completely
hosed.  You then have to guess roughly the right position in the list to
jump to, and then in order to scroll/page you have to again remember a
non-displayed piece of information for each sender in the page you're
looking at.

This is not a joke case or an outlier.  This is reality.  I've
personally had usernames that started with "y.", "li", "da", "dk", "t-",
and "ka", most of them not by choice.

There is a reason that your cell phone's address book sorts by name,
not by phone number.

> I see no reason to remove/deprecate the FROM, TO, and CC sorts.
> 
> You have presented no reason other than your dislike of their
> definition.  The provision of FROM, TO, and CC sorts in no way blocks
> future extensions that offer other forms of sorting these (and other
> fields).

Once the draft makes it to Proposed Standard, getting a second draft out
with a variant version of these sorts becomes exponentially harder.
Getting server and client uptake for that revision is even less likely.
It is better for "the ecosystem" (I cannot believe I just typed that) to
solve the problem just once and to do it as correctly as possible.

There is a clear consensus in the mail user agent marketplace for how
address sorts should behave.  Intentionally specifying a different
algorithm, especially one whose results can be as counterintuitive as
the one in the draft, seems unwise.

> SORT is implemented by at least two clients and at least four servers.
> 
> All of these would have to change to support the proposed crippled
> SORT.  Worse, my client would require substantial new, special case
> code to restore the functionality lost due to crippled SORT; and after
> this work will have worse performance.

Legacy servers such as yours would advertise "SORT".  Legacy clients
looking for "SORT" would find it and use it.  Nothing would break.

New revisions of the legacy servers would advertise both the legacy
"SORT" and the standard "SORT=BASE".  Legacy clients would see "SORT"
and behave exactly as before.  New clients would see both "SORT" and
"SORT=BASE" and act accordingly.

I believe that your client already does client-side FROM sorting for
servers that don't advertise SORT.  Determining whether to do client- or
server-side FROM sorting based on whether the server advertised "SORT"
or just "SORT=BASE" is probably on the order of 10 lines of code.