Subject: Re: CONDSTORE document is in AUTH48
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: Pete Resnick <[email-masked]>, Lisa Dusseault <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Tue, 30 May 2006 08:25:50 -0700 (PDT)
List-Id: <ietf-imapext.imc.org>

On Sun, 28 May 2006, Alexey Melnikov wrote:
> P.S. If still not convinced, ask what Mark thinks about this.

Mark thinks that the proposed change to use 0 to indicate "no MODSEQ" is 
reasonable.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Sun, 28 May 2006 15:02:37 +0100
From: Alexey Melnikov <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: Lisa Dusseault <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Pete Resnick wrote:

> On 5/26/06 at 12:10 PM -0700, Lisa Dusseault wrote:
>
>> This is on the edge of things you can change in an AUTH 48 for an 
>> approved document.  It is more than simply clarifying, it verges on 
>> defining on-the-wire behavior that was undefined
>
Correct.

>> (and therefore presumably unrestricted) before.  I'd like to make 
>> very sure this is WG consensus (Pete?  others chime in please...).
>
> Yeah, I'm increasingly getting the feeling that these changes are 
> *not* AUTH 48 material. Alexey: Can you justify for me why these 
> things *must* go into the document this go-round?

Because there are two related issues which might cause interoperability 
problems in the future if no advice is given in the document.
I would rather fix the document now and not touch it for 3 more years, 
rather than doing *another* revision in 6 months.
(Keep in mind that this document can't be moved to Draft standard due to 
dependencies.)

I think that the proposed changes are in spirit of IMAP, however one of 
them (STATUS response change) is not entirely trivial, so it doesn't 
hurt to spell out the solution in the document.*
*I know of only 2 server implementations of CONDSTORE, one of them is in 
Isode M-Box and another one is Cyrus. This change doesn't affect M-Box, 
as it never sends NOMODSEQ response. And I think Ken agreed to update 
Cyrus to match the new text.

Alexey

P.S. If still not convinced, ask what Mark thinks about this.

Date: Fri, 26 May 2006 14:38:20 -0700
To: Lisa Dusseault <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Cc: Ken Murchison <[email-masked]>, Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 5/26/06 at 12:10 PM -0700, Lisa Dusseault wrote:

>This is on the edge of things you can change in an AUTH 48 for an 
>approved document.  It is more than simply clarifying, it verges on 
>defining on-the-wire behavior that was undefined (and therefore 
>presumably unrestricted) before.  I'd like to make very sure this is 
>WG consensus (Pete?  others chime in please...).

Yeah, I'm increasingly getting the feeling that these changes are 
*not* AUTH 48 material. Alexey: Can you justify for me why these 
things *must* go into the document this go-round?

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated

Cc: Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
From: Lisa Dusseault <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Date: Fri, 26 May 2006 12:10:00 -0700
To: Ken Murchison <[email-masked]>
Sender: [email-masked]

This is on the edge of things you can change in an AUTH 48 for an  
approved document.  It is more than simply clarifying, it verges on  
defining on-the-wire behavior that was undefined (and therefore  
presumably unrestricted) before.  I'd like to make very sure this is  
WG consensus (Pete?  others chime in please...).

Lisa

On May 26, 2006, at 8:38 AM, Ken Murchison wrote:

>
> Alexey Melnikov wrote:
>> Ken Murchison wrote:
>>> Alexey Melnikov wrote:
>>>
>>>> Timo Sirainen wrote:
>>>>
>>>>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>>>>
>>>>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ  
>>>>>> Status Data Items):
>>>>>>      If the server doesn't support the persistent storage of  
>>>>>> mod-sequences
>>>>>>      for the mailbox (see Section 3.1.2), the server MUST omit  
>>>>>> the HIGHESTMODSEQ
>>>>>>      status data item from the untagged STATUS response. If  
>>>>>> the HIGHESTMODSEQ
>>>>>>      is the only requested status data item, the server MUST  
>>>>>> omit the untagged STATUS
>>>>>>      response, as its grammar requires at least one status  
>>>>>> data item to be returned.
>>>>>
>>>>> I don't think omitting is the right thing to do, rather return  
>>>>> NIL. This
>>>>> way clients don't break (shouldn't break..) if server starts  
>>>>> sending
>>>>> STATUS events to clients unilaterally (which I'm planning to do  
>>>>> at some
>>>>> point).
>>>>
>>>> This is a valid comment.
>>>> How about using 0 instead of NIL (0 is not a valid modsequence)?
>>>
>>> So, we're going to issue a HIGHESTMODSEQ response even if the  
>>> mailbox doesn't support CONDSTORE.  This seems counter- 
>>> intuitive.  I'm not sure we need to worry about breaking clients,  
>>> since if they ask for HIGHESTMODSEQ on a non-CONDSTORE capable  
>>> mailbox, the client is already broken.
>> The client might not know that the mailbox is NOMODSEQ, if it does  
>> STATUS before SELECT.
>
> Good point.  I can live with 0 (zero).
>
> -- 
> Kenneth Murchison
> Systems Programmer
> Project Cyrus Developer/Maintainer
> Carnegie Mellon University
>

Date: Fri, 26 May 2006 11:38:56 -0400
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Alexey Melnikov <[email-masked]>
CC: Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:
> Ken Murchison wrote:
> 
>> Alexey Melnikov wrote:
>>
>>> Timo Sirainen wrote:
>>>
>>>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>>>
>>>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>>>> Data Items):
>>>>>      If the server doesn't support the persistent storage of 
>>>>> mod-sequences
>>>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>>>> HIGHESTMODSEQ
>>>>>      status data item from the untagged STATUS response. If the 
>>>>> HIGHESTMODSEQ
>>>>>      is the only requested status data item, the server MUST omit 
>>>>> the untagged STATUS
>>>>>      response, as its grammar requires at least one status data 
>>>>> item to be returned.   
>>>>
>>>> I don't think omitting is the right thing to do, rather return NIL. 
>>>> This
>>>> way clients don't break (shouldn't break..) if server starts sending
>>>> STATUS events to clients unilaterally (which I'm planning to do at some
>>>> point).
>>>
>>> This is a valid comment.
>>> How about using 0 instead of NIL (0 is not a valid modsequence)?
>>
>> So, we're going to issue a HIGHESTMODSEQ response even if the mailbox 
>> doesn't support CONDSTORE.  This seems counter-intuitive.  I'm not 
>> sure we need to worry about breaking clients, since if they ask for 
>> HIGHESTMODSEQ on a non-CONDSTORE capable mailbox, the client is 
>> already broken.
> 
> The client might not know that the mailbox is NOMODSEQ, if it does 
> STATUS before SELECT.

Good point.  I can live with 0 (zero).

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Fri, 26 May 2006 16:20:38 +0100
From: Alexey Melnikov <[email-masked]>
To: Ken Murchison <[email-masked]>
CC: Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Ken Murchison wrote:

> Alexey Melnikov wrote:
>
>> Timo Sirainen wrote:
>>
>>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>>
>>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>>> Data Items):
>>>>      If the server doesn't support the persistent storage of 
>>>> mod-sequences
>>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>>> HIGHESTMODSEQ
>>>>      status data item from the untagged STATUS response. If the 
>>>> HIGHESTMODSEQ
>>>>      is the only requested status data item, the server MUST omit 
>>>> the untagged STATUS
>>>>      response, as its grammar requires at least one status data 
>>>> item to be returned.   
>>>
>>> I don't think omitting is the right thing to do, rather return NIL. 
>>> This
>>> way clients don't break (shouldn't break..) if server starts sending
>>> STATUS events to clients unilaterally (which I'm planning to do at some
>>> point).
>>
>> This is a valid comment.
>> How about using 0 instead of NIL (0 is not a valid modsequence)?
>
> So, we're going to issue a HIGHESTMODSEQ response even if the mailbox 
> doesn't support CONDSTORE.  This seems counter-intuitive.  I'm not 
> sure we need to worry about breaking clients, since if they ask for 
> HIGHESTMODSEQ on a non-CONDSTORE capable mailbox, the client is 
> already broken.

The client might not know that the mailbox is NOMODSEQ, if it does 
STATUS before SELECT.

Date: Fri, 26 May 2006 11:16:10 -0400
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Alexey Melnikov <[email-masked]>
CC: Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:
> 
> Timo Sirainen wrote:
> 
>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>  
>>
>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>> Data Items):
>>>      If the server doesn't support the persistent storage of 
>>> mod-sequences
>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>> HIGHESTMODSEQ
>>>      status data item from the untagged STATUS response. If the 
>>> HIGHESTMODSEQ
>>>      is the only requested status data item, the server MUST omit the 
>>> untagged STATUS
>>>      response, as its grammar requires at least one status data item 
>>> to be returned.
>>>   
>> I don't think omitting is the right thing to do, rather return NIL. This
>> way clients don't break (shouldn't break..) if server starts sending
>> STATUS events to clients unilaterally (which I'm planning to do at some
>> point).
>>  
>>
> This is a valid comment.
> How about using 0 instead of NIL (0 is not a valid modsequence)?

So, we're going to issue a HIGHESTMODSEQ response even if the mailbox 
doesn't support CONDSTORE.  This seems counter-intuitive.  I'm not sure 
we need to worry about breaking clients, since if they ask for 
HIGHESTMODSEQ on a non-CONDSTORE capable mailbox, the client is already 
broken.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Fri, 26 May 2006 14:10:37 +0100
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:

> Timo Sirainen wrote:
>
>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>
>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>> Data Items):
>>>      If the server doesn't support the persistent storage of 
>>> mod-sequences
>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>> HIGHESTMODSEQ
>>>      status data item from the untagged STATUS response. If the 
>>> HIGHESTMODSEQ
>>>      is the only requested status data item, the server MUST omit 
>>> the untagged STATUS
>>>      response, as its grammar requires at least one status data item 
>>> to be returned.   
>>
>> I don't think omitting is the right thing to do, rather return NIL. This
>> way clients don't break (shouldn't break..) if server starts sending
>> STATUS events to clients unilaterally (which I'm planning to do at some
>> point).
>
> This is a valid comment.
> How about using 0 instead of NIL (0 is not a valid modsequence)?

Ok, here is the new suggested text:

     If the server doesn't support the persistent storage of mod-sequences
     for the mailbox (see Section 3.1.2), the server MUST return 0
     as the value of HIGHESTMODSEQ status data item.

And I will update the ABNF section accordingly, i.e. it will contain:

   status-att-val      =/ "HIGHESTMODSEQ" SP mod-sequence-valzer
                          ;; extends non-terminal defined in [IMAPABNF].
                          ;; Value 0 denotes that the mailbox doesn't 
support
                          ;; persistent mod-sequences as described in
                          ;; Section 3.1.2

instead of the current:
   status-att-val      =/ "HIGHESTMODSEQ" SP mod-sequence-value
                          ;; extends non-terminal defined in [IMAPABNF].

Subject: Re: CONDSTORE document is in AUTH48
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Fri, 26 May 2006 16:09:18 +0300
Sender: [email-masked]

--=-BATVUZRdHNvKZUx1cII5
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Fri, 2006-05-26 at 13:55 +0100, Alexey Melnikov wrote:
> Timo Sirainen wrote:
>=20
> >On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
> > =20
> >
> >>Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data=20
> >>Items):
> >>      If the server doesn't support the persistent storage of mod-seque=
nces
> >>      for the mailbox (see Section 3.1.2), the server MUST omit the=20
> >>HIGHESTMODSEQ
> >>      status data item from the untagged STATUS response. If the=20
> >>HIGHESTMODSEQ
> >>      is the only requested status data item, the server MUST omit the=20
> >>untagged STATUS
> >>      response, as its grammar requires at least one status data item t=
o=20
> >>be returned.
> >>   =20
> >>
> >I don't think omitting is the right thing to do, rather return NIL. This
> >way clients don't break (shouldn't break..) if server starts sending
> >STATUS events to clients unilaterally (which I'm planning to do at some
> >point).
> > =20
> >
> This is a valid comment.
> How about using 0 instead of NIL (0 is not a valid modsequence)?

Sure, if it's not valid then it's better for clients. Just remember to
update ABNF if it's currently invalid in there too :)

--=-BATVUZRdHNvKZUx1cII5
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-BATVUZRdHNvKZUx1cII5--

Date: Fri, 26 May 2006 13:55:49 +0100
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Timo Sirainen wrote:

>On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>  
>
>>Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data 
>>Items):
>>      If the server doesn't support the persistent storage of mod-sequences
>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>HIGHESTMODSEQ
>>      status data item from the untagged STATUS response. If the 
>>HIGHESTMODSEQ
>>      is the only requested status data item, the server MUST omit the 
>>untagged STATUS
>>      response, as its grammar requires at least one status data item to 
>>be returned.
>>    
>>
>I don't think omitting is the right thing to do, rather return NIL. This
>way clients don't break (shouldn't break..) if server starts sending
>STATUS events to clients unilaterally (which I'm planning to do at some
>point).
>  
>
This is a valid comment.
How about using 0 instead of NIL (0 is not a valid modsequence)?

Subject: Re: CONDSTORE document is in AUTH48
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Fri, 26 May 2006 15:50:36 +0300
Sender: [email-masked]

--=-BRACR337ycUT8diwp/gb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data=20
> Items):*
> *      If the server doesn't support the persistent storage of mod-sequen=
ces
>       for the mailbox (see Section 3.1.2), the server MUST omit the=20
> HIGHESTMODSEQ
>       status data item from the untagged STATUS response. If the=20
> HIGHESTMODSEQ
>       is the only requested status data item, the server MUST omit the=20
> untagged STATUS
>       response, as its grammar requires at least one status data item to=20
> be returned.

I don't think omitting is the right thing to do, rather return NIL. This
way clients don't break (shouldn't break..) if server starts sending
STATUS events to clients unilaterally (which I'm planning to do at some
point).

--=-BRACR337ycUT8diwp/gb
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-BRACR337ycUT8diwp/gb--

Date: Fri, 26 May 2006 13:49:00 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:

> Arnt Gulbrandsen wrote:
>
>> Alexey Melnikov writes:
>>
>>> Anything else?
>>
>> Reference 4466 instead of the draft.
>
> This was already taken care of by the response RFC editor.

s/response/responsible

Date: Fri, 26 May 2006 13:43:43 +0100
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked]
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Arnt Gulbrandsen wrote:

>Alexey Melnikov writes:
>  
>
>>Anything else?
>>    
>>
>Reference 4466 instead of the draft.
>  
>
This was already taken care of by the response RFC editor.

Date: Fri, 26 May 2006 14:42:49 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: CONDSTORE document is in AUTH48
Cc: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> Anything else?

Reference 4466 instead of the draft.

Arnt

Date: Fri, 26 May 2006 13:40:43 +0100
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:

> Any last minute changes people want to do?
>
> I remember the following comments:
> 1). Comment from Ken regarding server behavior when the client sends a 
> CONDSTORE-extended command after the server returned NOMODSEQ on SELECT.

To be specific, I am suggesting to add to Section 3.1.2 (NOMODSEQ 
Response Code):
*
*A server that returned NOMODSEQ response code for a mailbox, which 
subsequently receives one of the following command while the mailbox is 
selected:
      -  a FETCH command with the CHANGEDSINCE modifier,
      -  a FETCH or SEARCH command that includes the MODSEQ message data
         item, or
      -  a STORE command with the UNCHANGEDSINCE modifier
MUST reject any such command with the tagged BAD response.

Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data 
Items):*
*      If the server doesn't support the persistent storage of mod-sequences
      for the mailbox (see Section 3.1.2), the server MUST omit the 
HIGHESTMODSEQ
      status data item from the untagged STATUS response. If the 
HIGHESTMODSEQ
      is the only requested status data item, the server MUST omit the 
untagged STATUS
      response, as its grammar requires at least one status data item to 
be returned.

> 2). Arnt's comments about unique IMAP tag in examples
>
> Anything else?
>
> Alexey
>

Date: Fri, 26 May 2006 13:20:08 +0100
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: CONDSTORE document is in AUTH48
Sender: [email-masked]

Any last minute changes people want to do?

I remember the following comments:
1). Comment from Ken regarding server behavior when the client sends a 
CONDSTORE-extended command after the server returned NOMODSEQ on SELECT.
2). Arnt's comments about unique IMAP tag in examples

Anything else?

Alexey

Date: Tue, 16 May 2006 12:37:54 -0700
From: "Mark Davis" <[email-masked]>
To: "Arnt Gulbrandsen" <[email-masked]>
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

------=_Part_41077_30668366.1147808274668
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64
Content-Disposition: inline

------=_Part_41077_30668366.1147808274668
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: base64
Content-Disposition: inline

------=_Part_41077_30668366.1147808274668--

Date: Tue, 16 May 2006 20:25:19 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: Mark Davis <[email-masked]>, [email-masked]
Sender: [email-masked]

Arnt Gulbrandsen writes:
> Mark Davis writes:
>> ...
>> At a quick glance, it appears that a number of comments have been 
>> incorporated.
>
> It is possible that some of my changes don't satisfy you. I had 
> conflicting requests for many things. Feel free to repeat, rephrase 
> or add arguments.

In -10 (which I'll send off once I finish work this evening) I've made 
another few changes.

>>>       > 2.4 Sort Keys
>>>
>>> The use of the term "collation canonicalization" to refer to sort 
>>> keys is very misleading. ...
>
> Changed; the text now speaks of sort keys. I'm afraid there still are 
> instances of the old term around, I found one today.

In -10, all should be dead.

>>> The term 'error' is also problematic, since what is really at issue 
>>> is a question of domain. For all those strings in the domain, 
>>> either 'equal' or 'not_equal' should be returned from the equality 
>>> function. For any string not in the domain, 'undefined' should be 
>>> returned.
>
> Not changed. Back in February, I agreed that "error" was not ideal, 
> but did not see "undefined" as better, and could not find a really 
> apt term. The collations were a little too well-defined in the 
> "undefined" cases then.
>
> However, in -10, I think they really will be undefined outside their 
> domain, so I'll change to using "undefined" instead of "error". (I'm 
> removing the bits that fall back to i;octet.)

Changed. The fallback to i;octet is now in the server, if the protocol 
requires it.

This means that if a server can escape implementing i;octet, it can keep 
all its strings in UCS-2 or UCS-4 internally, even as it implements 
collations which are defined in terms of octets.

Arnt

Date: Tue, 16 May 2006 18:21:37 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: John Cowan <[email-masked]>, Mark Davis <[email-masked]>, [email-masked]
Sender: [email-masked]

John Cowan writes:
> Instead of the requestor sending the request to IANA, who sends it to 
> the Collation Reviewer for discussion on the list, and then the 
> Collation Reviewer sends it back to IANA for registration (or 
> doesn't), remove the first pass through IANA.

Fine. Done.

Arnt

Date: Thu, 11 May 2006 09:38:01 -0400
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked], Mark Davis <[email-masked]>, [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
From: John Cowan <[email-masked]>
Sender: [email-masked]

I propose that the procedure specified in draft-newman-i18n-collation-09
for getting new collations approved should be changed to the procedure
used for new language tags.  Instead of the requestor sending the
request to IANA, who sends it to the Collation Reviewer for discussion
on the list, and then the Collation Reviewer sends it back to IANA for
registration (or doesn't), remove the first pass through IANA.

Have people post directly to the list and work out the details.  When the
requestor thinks it's ready, the Collation Reviewer wakes up and then
either sends the latest draft of the request to IANA or else sends a
rejection (with reasons) to the list.  This lowers the load on IANA.

This scheme has worked very well for [email-masked] for the
past 11 years.

-- 
John Cowan  [email-masked]   http://ccil.org/~cowan
"The exception proves the rule."  Dimbulbs think: "Your counterexample proves
my theory."  Latin students think "'Probat' means 'tests': the exception puts
the rule to the proof."  But legal historians know it means "Evidence for an
exception is evidence of the existence of a rule in cases not excepted from."

Date: Wed, 10 May 2006 15:08:53 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: Mark Davis <[email-masked]>, [email-masked]
Sender: [email-masked]

Mark Davis writes:
> The release of this is timely (we didn't get notified of a 07 or 08 
> draft), since the Unicode Technical Committee is meeting next week, 
> and can discuss it.
>
> Could you indicate which of the items raised in the email of 
> 2006-02-21 from the Unicode Technical Committee have been addressed 
> in this release (and if not accepted then why)? That would help 
> greatly with the review. (I couldn't find any archive for discussion 
> of draft-newman-i18n-comparator where that email could be publicly 
> linked from, so I am appending it at the end of this message.) At a 
> quick glance, it appears that a number of comments have been 
> incorporated.

Lots. Some not. See below.

It is possible that some of my changes don't satisfy you. I had 
conflicting requests for many things. Feel free to repeat, rephrase or 
add arguments.

> Mark
>
> BTW, despite the subject of the message, the document is at 
> http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt. 
> It helps to send out a link, especially if the name (comparator vs 
> collation) is wrong ;-)

Mea culpa. My apologies.

...
>> To:   Network Working Group
>> Re:   draft-newman-i18n-comparator
>> Date:         2006-02-21
>> From:         Unicode Technical Committee
>>
>> The Unicode Technical Committee has reviewed the document 
>> http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-06.txt. 
>> While UTC is in favor of the goal, there are a number of problems 
>> with the document. The main problems are outlined below. Once these 
>> are addressed, then further review can continue.
>>
>>     Details
>>
>>       > 2.1 Definitions
>>
>>         Content
>>
>> The document needs to include the definitions of the technical terms 
>> used in the document,  including all those that may not be familiar 
>> to implementers, such as "trichotomous" and "collation identifiers". 
>> In particular, the notion of a substring is /prima facie/ quite 
>> simple, but there are complications that require a clear definition. 
>> The text in the document does not make clear that there may be more 
>> than one match for a substring in a string, and that the matches can 
>> overlap. It says "the starting offset", for example, when there may 
>> be multiple ones.

Changed.

>> Moreover, language sensitive matches have additional complications 
>> which need to be called out. For more information, see 
>> http://www.unicode.org/reports/tr10/#Searching

Not really changed. As I recall, I added a little bit of text.

>>         Format
>>
>> If there is a "Definitions" section, readers have a reasonable 
>> expectation that that section should contain all the required 
>> definitions. However, a number of definitions are scattered within 
>> the text. One of two approaches should be taken
>>
>>    1. Move all the definitions into this section.
>>    2. Remove the definitions section, but clearly call out in the text
>>       the definitions of  each terms on its own line.
>>
>> Mixing these two styles is needlessly confusing for readers.

Not changed; I'm going by what confuses reviewers.

>>       > 2.4 Sort Keys
>>
>> The use of the term "collation canonicalization" to refer to sort 
>> keys is very misleading. ...

Changed; the text now speaks of sort keys. I'm afraid there still are 
instances of the old term around, I found one today.

>> > 3.2
>>
>> This specifies that clients that support disconnected operation 
>> should not use wildcards while clients that provide collation 
>> operations only when connected to the server may use wildcards.

This restrinction has been lifted.

>> The EBNF syntax shown in section 3.2 says that the collation-wild 
>> must not exceed 255 characters total while the section 3.1 specifies 
>> that the collation name must not exceed 254 characters.

Brought into sync.

>> It seems having the same maximum possible length for both collation 
>> name and wildcard string would be desirable for actual 
>> implementations.

I picked 254, not 255, but I confess I cannot remember why.

>>       > 4.2.1 Equality
>>
>> It needs to be made clear that the return values are not physically 
>> the strings "match", etc. but enumerated values such as /equal/ and  
>> /not_equal/.

Changed. Also other similar changes.

>> One extremely important point is that for a given comparator, the 
>> equality function must be synchronized with the ordering function.

I've done this and all the other equivalences/connections/implications I 
could see.

>> The term 'error' is also problematic, since what is really at issue 
>> is a question of domain. For all those strings in the domain, either 
>> 'equal' or 'not_equal' should be returned from the equality 
>> function. For any string not in the domain, 'undefined' should be 
>> returned.

Not changed. Back in February, I agreed that "error" was not ideal, but 
did not see "undefined" as better, and could not find a really apt 
term. The collations were a little too well-defined in the "undefined" 
cases then.

However, in -10, I think they really will be undefined outside their 
domain, so I'll change to using "undefined" instead of "error". (I'm 
removing the bits that fall back to i;octet.)

>> There is a typo at the 4'th line of the second paragraph of the 
>> section 4.2 saying "... For example, an collation" which should be 
>> changed to "... For example, a collation" instead.

Fixed.

>>       > 4.2.2 Substring
>>
>> Prefix and suffix matching are not fully spelled out.

I think they are now.

>> The operations and their results must be clarified. And as noted 
>> before, it is very important to precisely define the substring 
>> operations, especially the starting offset and ending offset. It 
>> also must be clarified whether what is being asked for is the first 
>> possible matching location in the string, the last, or the nth one.

Partly changed. I didn't do the bits you ask for in the last sentence. I 
can add an open issue.

>>       > 4.3.3 Ordering
>>
>> > It MUST be transitive and trichotomous.
>>
>> As above, these should be defined.

I did not, since I think this document is the wrong place to define 
these terms.

>> The exposition in this section would be simpler if you also defined 
>> "reversible", whereby f(a,b) = less iff f(b,a) = greater.

The exposition changed enough as a result of other commens that I 
isregarded this comment.

>> An 'undefined' value can be allowed if, as per equality above, it 
>> means that at least one of the operands is outside of the domain. 
>> The function then imposes a total order on all strings in the 
>> domain; moreover, a wrapper can easily convert the function to a 
>> total order over all strings by putting all items outside the domain 
>> either below or above the ones in the domain -- or even excluding 
>> them,/ at its choice./

I'm doing something like this in -10. (Removing the fallback to i;octet.)

>> [Note: it is very important to avoid the confusion between 
>> "identical" and "equal". According to a caseless compare, "Mark" and 
>> "mark" are equal; however, the strings are not identical.]

Changed all over the place.

>> [Either 'ordering function' or 'comparison function' should be used 
>> consistently, not sometimes 'collations'].

Changed.

>>       > 4.3.  Internal Canonicalization Algorithm
>>
>> This section is difficult to understand.

Changed; I hope the new text is better.

>>       > 4.4.  Use of Lookup Tables
>>
>> It is not at all clear what is meant by "customizable lookup tables".

Clarified and partly removed.

>>       > 4.5.  Multi-Value Attributes
>>
>> This is very unclear.

Deleted.

>> This is a very important feature that needs to be spelled out in 
>> detail, and clearly reflected in the template for registration. In 
>> particular, the template should have provision for multiple 
>> attributes, with the ability to specify the acceptable operands for 
>> that attribute. (See below). The specification of the operands could 
>> be either a list of values, or a regular expression (with the former 
>> preferred). Suggested regular expression syntax would be Perl or XML 
>> Schema.

I asked Martin D�rst and you to provide a new DTD. Martin said okay, I 
don't remember whether you answered. I think the DTD should come before 
this.

>>       > 5.1Character Encoding
>>
>>    The protocol specification has to make sure that it is clear on which
>>    characters (rather than just octets) the collations are used.  This
>>    can be done by specifying the protocol itself in terms of characters
>>    (e.g. in the case of a query language), by specifying a single
>>    character encoding for the protocol (e.g.  UTF-8 [3]), or by
>>    carefully describing the relevant issues of character encoding
>>    labeling and conversion.  In the later case, details to consider
>>    include how to handle unknown charsets, any charsets which are
>>    mandatory-to-implement, any issues with byte-order that might apply,
>>    and any transfer encodings which need to be supported.
>>
>> If a collation is able to advertise itself as being able to handle, 
>> say, SJIS and UTF-8, then there should a required description of a 
>> protocol for indicating that and for communicating which encodings 
>> are handled, and how it handles error conditions (such as a charset 
>> outside of those it can handle. Otherwise, it is difficult to 
>> understand how this paragraph would be applied in practice.
>>
>>       > 5.3
>>
>> The section 5.3 specifies:
>>
>>     The protocol MUST specify how comparisons behave in the absence of
>>     explicit collation negotiation or when a collation of "*" is
>>     requested. The protocol MAY specify that the default collation
>>     used in such circumstances is sensitive to server configuration.
>>
>> and the section 3.2 specifies:
>>
>>     ... If the wildcard string matches multiple collations, the server
>>     SHOULD select the collation with the broadest scope (preferably
>>     international scope), the most recent table versions and the
>>     greatest number of supported operations. A single wildcard
>>     character ("*") refers to the application protocol collation
>>     behavior that would occur if no explicit negotiation were used.
>>
>> These appear inconsistent.

Changed.

>>       7.5.  Example Initial Registry Summary
>>
>> The sample registry would suffer a combinatorial explosion if 
>> parameters are not handled differently.
...

This is the DTD issue.

>> > 11.  Security Considerations
>>
>> This is insufficient. It should at least point to the problems 
>> related in UCA and in 
>> http://www.unicode.org/reports/tr36/tr36-4.html (note that that 
>> document has been approved by the UTC and will be posted as an 
>> approved version soon.)

It now refers.

>>     General
>>
>> One of the real problems with the IANA character registry is that the 
>> entries are underspecified. It quite often occurs that two vendors 
>> implement the same IANA charset conversion different ways, leading 
>> to significant interoperability problems and text corruption. See, 
>> for example, 
>> http://www.w3.org/Submission/japanese-xml/#ambiguity_of_yen.
>>
>> We have the real concern that this registry could lead down the same path.

Noted.

>> > collation, it has to say so
>>
>> There are places where the text should be clarified, as to whether a 
>> MUST or SHOULD is implied; this is just an example.
>>
>> > "comparator" vs "collator"
>>
>> Either one term or the other should be used consistently.

Collator, now.

>> > Unicode 3.2
>>
>> Unicode 3.2 is obsolete; the the reference versions for the Collation 
>> Registry should be Unicode 5.0 and UCA 5.0, since those will be 
>> approved and published by the time the Internet Application Protocol 
>> Collation Registry has completed its review and been approved.

I'll update to the then-current versions immediately before submitting 
the final draft as an RFC.

>> Because of the use of NamePrep, it is probably the case that Unicode 
>> 3.2 also needs to be included, but strongly recommended for usage 
>> only by protocols or systems dependent on NamePrep. Note that as of 
>> UCA 4.0 and beyond, the version number of UCA is guaranteed to be 
>> identical with the version number of Unicode that it is defined for.
>>
>> > Versioning
>>
>> This is tricky, and should be clarified. In many instances, it is 
>> sufficient to use an unversioned collator, such as simply "UCA". In 
>> other cases, there are requirements to use a specific version, or a 
>> version of at least X. This needs to be described.

IETF documents should have only immutable references. Thus, I can 
reference "UCAv14", but not "UCA", because the latter moves to v15, v16 
and onwards.

Arnt

Date: Wed, 10 May 2006 00:47:22 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: John Cowan <[email-masked]>, [email-masked]
Sender: [email-masked]

John Cowan writes:
> Arnt Gulbrandsen scripsit:
>>  As far as I know, this addresses, ignores or adds open issues for 
>>  all requests. If something is ignored, that's because other people 
>>  wanted the opposite, or because I overlooked it when I went over 
>>  all the mail this week. I'm sorry about it in either case.
>>
>>  Review, please.
>
> Posted where? Neither rfc-editor.org nor ietf.org seems to have it.

http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt 
has it. Sorry. I posted that only an hour or two after the i-d editor 
told me it was posted (at that URL), which must have been too quickly 
for most mirrors.

Arnt

Date: Tue, 09 May 2006 15:43:23 -0700
From: Mark Davis <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Sender: [email-masked]

The release of this is timely (we didn't get notified of a 07 or 08 
draft), since the Unicode Technical Committee is meeting next week, and 
can discuss it.

Could you indicate which of the items raised in the email of 2006-02-21 
from the Unicode Technical Committee have been addressed in this release 
(and if not accepted then why)? That would help greatly with the review. 
(I couldn't find any archive for discussion of 
draft-newman-i18n-comparator where that email could be publicly linked 
from, so I am appending it at the end of this message.) At a quick 
glance, it appears that a number of comments have been incorporated.

Mark

BTW, despite the subject of the message, the document is at 
http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt. 
It helps to send out a link, especially if the name (comparator vs 
collation) is wrong ;-)

BTW, it was pointed out to us that the original email shouldn't have 
been sent to "Network Working Group", even though that is the name at 
the top of 
http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt

Arnt Gulbrandsen wrote:
>
> As far as I know, this addresses, ignores or adds open issues for all 
> requests. If something is ignored, that's because other people wanted 
> the opposite, or because I overlooked it when I went over all the mail 
> this week. I'm sorry about it in either case.
>
> Review, please.
>
> Arnt

=================

Mark Davis wrote:
> To: 	Network Working Group
> Re: 	draft-newman-i18n-comparator
> Date: 	2006-02-21
> From: 	Unicode Technical Committee
>
> The Unicode Technical Committee has reviewed the document 
> http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-06.txt. 
> While UTC is in favor of the goal, there are a number of problems with 
> the document. The main problems are outlined below. Once these are 
> addressed, then further review can continue.
>
>
>     Details
>
>
>       > 2.1 Definitions
>
>
>         Content
>
> The document needs to include the definitions of the technical terms 
> used in the document,  including all those that may not be familiar to 
> implementers, such as "trichotomous" and "collation identifiers". In 
> particular, the notion of a substring is /prima facie/ quite simple, 
> but there are complications that require a clear definition. The text 
> in the document does not make clear that there may be more than one 
> match for a substring in a string, and that the matches can overlap. 
> It says "the starting offset", for example, when there may be multiple 
> ones.
>
> Moreover, language sensitive matches have additional complications 
> which need to be called out. For more information, see 
> http://www.unicode.org/reports/tr10/#Searching
>
>
>         Format
>
> If there is a "Definitions" section, readers have a reasonable 
> expectation that that section should contain all the required 
> definitions. However, a number of definitions are scattered within the 
> text. One of two approaches should be taken
>
>    1. Move all the definitions into this section.
>    2. Remove the definitions section, but clearly call out in the text
>       the definitions of  each terms on its own line.
>
> Mixing these two styles is needlessly confusing for readers.
>
>
>       > 2.4 Sort Keys
>
> The use of the term "collation canonicalization" to refer to sort keys 
> is very misleading. The term "canonicalization" implies that the 
> results are still text in some fashion, whereas a sortkey is simply a 
> string of octets generated from a given string by a specific 
> comparator, whereby the binary comparison (ordering) of two sort keys 
> is guaranteed to match *that* comparator's compare function for the 
> original strings. The octets may have no readily discernable relation 
> to the original text. For example, the ICU sort keys generated for the 
> following strings are:
>
> cote 	2c 44 4e 30 01 08 01 08 00
> côté 	2c 44 4e 30 01 85 93 85 8d 01 0a 00
> Αραβικά 	5c 20 52 20 22 36 3a 20 01 80 8d 01 8f 0b 00
>
> See 
> http://www-950.ibm.com/software/globalization/icu/demo/locales/en/?_=el&d_=en&x=col 
> <http://www-950.ibm.com/software/globalization/icu/demo/locales/en/?_=el&d_=en&x=col> 
> for other examples.
>
> > 3.2
>
> This specifies that clients that support disconnected operation should 
> not use wildcards while clients that provide collation operations only 
> when connected to the server may use wildcards.
>
> It appears the restrictions are may not be really needed and the 
> restrictions may need to be deleted from the draft. Otherwise, it 
> would really helpful if the rationale behind the restrictions are 
> provided at the draft.
>
> The EBNF syntax shown in section 3.2 says that the collation-wild must 
> not exceed 255 characters total while the section 3.1 specifies that 
> the collation name must not exceed 254 characters.
>
> It seems having the same maximum possible length for both collation 
> name and wildcard string would be desirable for actual implementations.
>
>
>       > 4.2.1 Equality
>
> It needs to be made clear that the return values are not physically 
> the strings "match", etc. but enumerated values such as /equal/ and  
> /not_equal/. The document could describe a notation used for them, 
> such as single quotes, since italic is not available in RFCs. 
> Similarly, the results of the ordering function should be specified as 
> an enumeration with three values: /less/, /equal/, /greater./ The 
> mapping actual API return values in implementations to these 
> enumerated values can be outside of the scope of this document. For 
> example, the mapping might take -1 onto /less/ in one implementation, 
> or anything negative onto /less/ in another implementation.
>
> One extremely important point is that for a given comparator, the 
> equality function must be synchronized with the ordering function. 
> That is, it must return 'equal' if and only if the ordering function 
> returns 'equal'. Otherwise any coordinated usage of the functions will 
> fail. This also implies that either 'error' is allowed for both 
> functions or for neither.
>
> The term 'error' is also problematic, since what is really at issue is 
> a question of domain. For all those strings in the domain, either 
> 'equal' or 'not_equal' should be returned from the equality function. 
> For any string not in the domain, 'undefined' should be returned. That 
> avoids coherency problems. Then the requirements are clear:
>
>     * if A and B are in the domain, then the result of an equality
>       test is either /equal/ or /not_equal/
>     * if A or B (or both) are not in the domain, then the result of an
>       equality test is /undefined/.
>
> There is a typo at the 4'th line of the second paragraph of the 
> section 4.2 saying "... For example, an collation" which should be 
> changed to "... For example, a collation" instead.
>
>
>       > 4.2.2 Substring
>
> Prefix and suffix matching are not fully spelled out. The operations 
> and their results must be clarified. And as noted before, it is very 
> important to precisely define the substring operations, especially the 
> starting offset and ending offset. It also must be clarified whether 
> what is being asked for is the first possible matching location in the 
> string, the last, or the nth one.
>
>
>       > 4.3.3 Ordering
>
> > It MUST be transitive and trichotomous.
>
> As above, these should be defined. The exposition in this section 
> would be simpler if you also defined "reversible", whereby f(a,b) = 
> less iff f(b,a) = greater. Then the statement would be:
>
>     It MUST be transitive, trichotomous, and reversible.
>
> >When the collation is used with a
>    "-" prefix, the result of the ordering function of the collation MUST
>    be reversed.
>
> => When the collation is used with a
>    "-" prefix, the result of the ordering function of the collation 
> when applied to two strings A and B  MUST
>    be the same as the result with a "+" prefix applied to B and A.
>
> An 'undefined' value can be allowed if, as per equality above, it 
> means that at least one of the operands is outside of the domain. The 
> function then imposes a total order on all strings in the domain; 
> moreover, a wrapper can easily convert the function to a total order 
> over all strings by putting all items outside the domain either below 
> or above the ones in the domain -- or even excluding them,/ at its 
> choice./
>
>  > In general, collations SHOULD NOT return "0" unless the two strings 
> are identical.
>
> => The ordering function MUST return 'equal' if and only if the equality function returns 'equal'
>
> [Note: it is very important to avoid the confusion between "identical" 
> and "equal". According to a caseless compare, "Mark" and "mark" are 
> equal; however, the strings are not identical.]
>
> [Either 'ordering function' or 'comparison function' should be used 
> consistently, not sometimes 'collations'].
>
>
>       > 4.3.  Internal Canonicalization Algorithm
>
> This section is difficult to understand. It appears that goal is that 
> any registration must specify sufficient detail, both data and 
> algorithm, so as to enable someone to reproduce the results. But it is 
> not at all clear that that is the goal. And that would make the 
> registration require, in some cases, a huge accompanying document. To 
> duplicate the results of CLDR collators, for example, would require 
> the UCA specification, plus the LDML specification, plus all the 
> relevant data in the CLDR repository.
>
>
>       > 4.4.  Use of Lookup Tables
>
> It is not at all clear what is meant by "customizable lookup tables".
>
>
>       > 4.5.  Multi-Value Attributes
>
> This is very unclear. It describes attributes as applying to only 
> equality (since it only refers to "match" vs "no-match" (and 
> forgetting "error")).
>
> This is a very important feature that needs to be spelled out in 
> detail, and clearly reflected in the template for registration. In 
> particular, the template should have provision for multiple 
> attributes, with the ability to specify the acceptable operands for 
> that attribute. (See below). The specification of the operands could 
> be either a list of values, or a regular expression (with the former 
> preferred). Suggested regular expression syntax would be Perl or XML 
> Schema.
>
>
>       > 5.1Character Encoding
>
>    The protocol specification has to make sure that it is clear on which
>    characters (rather than just octets) the collations are used.  This
>    can be done by specifying the protocol itself in terms of characters
>    (e.g. in the case of a query language), by specifying a single
>    character encoding for the protocol (e.g.  UTF-8 [3]), or by
>    carefully describing the relevant issues of character encoding
>    labeling and conversion.  In the later case, details to consider
>    include how to handle unknown charsets, any charsets which are
>    mandatory-to-implement, any issues with byte-order that might apply,
>    and any transfer encodings which need to be supported.
>
> If a collation is able to advertise itself as being able to handle, 
> say, SJIS and UTF-8, then there should a required description of a 
> protocol for indicating that and for communicating which encodings are 
> handled, and how it handles error conditions (such as a charset 
> outside of those it can handle. Otherwise, it is difficult to 
> understand how this paragraph would be applied in practice.
>
>
>       > 5.3
>
> The section 5.3 specifies:
>
>     The protocol MUST specify how comparisons behave in the absence of
>     explicit collation negotiation or when a collation of "*" is
>     requested. The protocol MAY specify that the default collation
>     used in such circumstances is sensitive to server configuration.
>
> and the section 3.2 specifies:
>
>     ... If the wildcard string matches multiple collations, the server
>     SHOULD select the collation with the broadest scope (preferably
>     international scope), the most recent table versions and the
>     greatest number of supported operations. A single wildcard
>     character ("*") refers to the application protocol collation
>     behavior that would occur if no explicit negotiation were used.
>
> These appear inconsistent.
>
>
>       7.5.  Example Initial Registry Summary
>
> The sample registry would suffer a combinatorial explosion if 
> parameters are not handled differently. For example, with CLDR 
> collations, there can be hundreds of locales, six different strength 
> settings; four different case-first settings; three different 
> alternate settings, backwards settings, normalization settings, case 
> level settings, hiragana settings, and numeric settings; plus a 
> variable-top setting which has a string as an operand. Registering the 
> combinations that people are allowed to use would be untenable.
>
> http://www.unicode.org/draft/reports/tr35/tr35.html#Setting_Options
>
> Instead, as remarked above, the allowable attribute values need to be 
> associated with the registered name in a machine-readable form.
>
> > 11.  Security Considerations
>
> This is insufficient. It should at least point to the problems related 
> in UCA and in http://www.unicode.org/reports/tr36/tr36-4.html (note 
> that that document has been approved by the UTC and will be posted as 
> an approved version soon.)
>
>
>     General
>
> One of the real problems with the IANA character registry is that the 
> entries are underspecified. It quite often occurs that two vendors 
> implement the same IANA charset conversion different ways, leading to 
> significant interoperability problems and text corruption. See, for 
> example, http://www.w3.org/Submission/japanese-xml/#ambiguity_of_yen.
>
> We have the real concern that this registry could lead down the same path.
>
> > collation, it has to say so
>
> There are places where the text should be clarified, as to whether a 
> MUST or SHOULD is implied; this is just an example.
>
> > "comparator" vs "collator"
>
> Either one term or the other should be used consistently.
>
> > Unicode 3.2
>
> Unicode 3.2 is obsolete; the the reference versions for the Collation 
> Registry should be Unicode 5.0 and UCA 5.0, since those will be 
> approved and published by the time the Internet Application Protocol 
> Collation Registry has completed its review and been approved.
>
> Because of the use of NamePrep, it is probably the case that Unicode 
> 3.2 also needs to be included, but strongly recommended for usage only 
> by protocols or systems dependent on NamePrep. Note that as of UCA 4.0 
> and beyond, the version number of UCA is guaranteed to be identical 
> with the version number of Unicode that it is defined for.
>
> > Versioning
>
> This is tricky, and should be clarified. In many instances, it is 
> sufficient to use an unversioned collator, such as simply "UCA". In 
> other cases, there are requirements to use a specific version, or a 
> version of at least X. This needs to be described.
>

Date: Tue, 9 May 2006 16:45:02 -0400
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
From: John Cowan <[email-masked]>
Sender: [email-masked]

Arnt Gulbrandsen scripsit:

> As far as I know, this addresses, ignores or adds open issues for all 
> requests. If something is ignored, that's because other people wanted 
> the opposite, or because I overlooked it when I went over all the mail 
> this week. I'm sorry about it in either case.
> 
> Review, please.

Posted where?  Neither rfc-editor.org nor ietf.org seems to have it.

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
The penguin geeks is happy / As under the waves they lark
The closed-source geeks ain't happy / They sad cause they in the dark
But geeks in the dark is lucky / They in for a worser treat
One day when the Borg go belly-up / Guess who wind up on the street.

Date: Tue, 9 May 2006 22:14:28 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked], [email-masked]
Subject: draft-newman-i18n-collation-09.txt just posted
Sender: [email-masked]

As far as I know, this addresses, ignores or adds open issues for all 
requests. If something is ignored, that's because other people wanted 
the opposite, or because I overlooked it when I went over all the mail 
this week. I'm sorry about it in either case.

Review, please.

Arnt

Date: Tue, 02 May 2006 20:22:56 +0100
Subject: Re: SEARCH and COMPARATOR
From: Dave Cridland <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue May  2 20:05:07 2006, Arnt Gulbrandsen wrote:
> I am resolving the SEARCH issue by fiat: "COMPARATOR collation-name 
> search-key", applying that collation to that search-key (which may 
> use ordinary IMAP and/or/blah).
> 
> 
My inclination is that this might be better dealt with in more 
drastic ways, and we should actually forget about it for now. 
Specifically, I think IMAP ought to admit that Sieve has much more 
interesting tests, for the most part, and even provides ANDs and 
exciting stuff. This is not to suggest that you need write all that 
in, it's more to suggest that it's not worth tackling that problem 
there.

Good enough for me would be a comparator option for SEARCH changing 
the default comparator for that SEARCH command only.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Tue, 2 May 2006 21:05:07 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: SEARCH and COMPARATOR
Sender: [email-masked]

Hi,

now that Timo Sirainen shot down my GIF87 example (for which I am 
grateful, even as I am feeling less than grateful), could someone dream 
up a reason to use two different collations at the same time, or at 
least nearly at the same time? Ideally a search.

I am resolving the SEARCH issue by fiat: "COMPARATOR collation-name 
search-key", applying that collation to that search-key (which may use 
ordinary IMAP and/or/blah).

Arnt

Date: Tue, 30 May 2006 08:25:50 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Pete Resnick <[email-masked]>, Lisa Dusseault <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

On Sun, 28 May 2006, Alexey Melnikov wrote:
> P.S. If still not convinced, ask what Mark thinks about this.

Mark thinks that the proposed change to use 0 to indicate "no MODSEQ" is 
reasonable.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Date: Sun, 28 May 2006 15:02:37 +0100
From: Alexey Melnikov <[email-masked]>
To: Pete Resnick <[email-masked]>
CC: Lisa Dusseault <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Pete Resnick wrote:

> On 5/26/06 at 12:10 PM -0700, Lisa Dusseault wrote:
>
>> This is on the edge of things you can change in an AUTH 48 for an 
>> approved document.  It is more than simply clarifying, it verges on 
>> defining on-the-wire behavior that was undefined
>
Correct.

>> (and therefore presumably unrestricted) before.  I'd like to make 
>> very sure this is WG consensus (Pete?  others chime in please...).
>
> Yeah, I'm increasingly getting the feeling that these changes are 
> *not* AUTH 48 material. Alexey: Can you justify for me why these 
> things *must* go into the document this go-round?

Because there are two related issues which might cause interoperability 
problems in the future if no advice is given in the document.
I would rather fix the document now and not touch it for 3 more years, 
rather than doing *another* revision in 6 months.
(Keep in mind that this document can't be moved to Draft standard due to 
dependencies.)

I think that the proposed changes are in spirit of IMAP, however one of 
them (STATUS response change) is not entirely trivial, so it doesn't 
hurt to spell out the solution in the document.*
*I know of only 2 server implementations of CONDSTORE, one of them is in 
Isode M-Box and another one is Cyrus. This change doesn't affect M-Box, 
as it never sends NOMODSEQ response. And I think Ken agreed to update 
Cyrus to match the new text.

Alexey

P.S. If still not convinced, ask what Mark thinks about this.

Date: Fri, 26 May 2006 14:38:20 -0700
To: Lisa Dusseault <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Cc: Ken Murchison <[email-masked]>, Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 5/26/06 at 12:10 PM -0700, Lisa Dusseault wrote:

>This is on the edge of things you can change in an AUTH 48 for an 
>approved document.  It is more than simply clarifying, it verges on 
>defining on-the-wire behavior that was undefined (and therefore 
>presumably unrestricted) before.  I'd like to make very sure this is 
>WG consensus (Pete?  others chime in please...).

Yeah, I'm increasingly getting the feeling that these changes are 
*not* AUTH 48 material. Alexey: Can you justify for me why these 
things *must* go into the document this go-round?

pr
-- 
Pete Resnick <http://www.qualcomm.com/~presnick/>
QUALCOMM Incorporated

Cc: Alexey Melnikov <[email-masked]>, Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
From: Lisa Dusseault <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Date: Fri, 26 May 2006 12:10:00 -0700
To: Ken Murchison <[email-masked]>
Sender: [email-masked]

This is on the edge of things you can change in an AUTH 48 for an  
approved document.  It is more than simply clarifying, it verges on  
defining on-the-wire behavior that was undefined (and therefore  
presumably unrestricted) before.  I'd like to make very sure this is  
WG consensus (Pete?  others chime in please...).

Lisa

On May 26, 2006, at 8:38 AM, Ken Murchison wrote:

>
> Alexey Melnikov wrote:
>> Ken Murchison wrote:
>>> Alexey Melnikov wrote:
>>>
>>>> Timo Sirainen wrote:
>>>>
>>>>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>>>>
>>>>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ  
>>>>>> Status Data Items):
>>>>>>      If the server doesn't support the persistent storage of  
>>>>>> mod-sequences
>>>>>>      for the mailbox (see Section 3.1.2), the server MUST omit  
>>>>>> the HIGHESTMODSEQ
>>>>>>      status data item from the untagged STATUS response. If  
>>>>>> the HIGHESTMODSEQ
>>>>>>      is the only requested status data item, the server MUST  
>>>>>> omit the untagged STATUS
>>>>>>      response, as its grammar requires at least one status  
>>>>>> data item to be returned.
>>>>>
>>>>> I don't think omitting is the right thing to do, rather return  
>>>>> NIL. This
>>>>> way clients don't break (shouldn't break..) if server starts  
>>>>> sending
>>>>> STATUS events to clients unilaterally (which I'm planning to do  
>>>>> at some
>>>>> point).
>>>>
>>>> This is a valid comment.
>>>> How about using 0 instead of NIL (0 is not a valid modsequence)?
>>>
>>> So, we're going to issue a HIGHESTMODSEQ response even if the  
>>> mailbox doesn't support CONDSTORE.  This seems counter- 
>>> intuitive.  I'm not sure we need to worry about breaking clients,  
>>> since if they ask for HIGHESTMODSEQ on a non-CONDSTORE capable  
>>> mailbox, the client is already broken.
>> The client might not know that the mailbox is NOMODSEQ, if it does  
>> STATUS before SELECT.
>
> Good point.  I can live with 0 (zero).
>
> -- 
> Kenneth Murchison
> Systems Programmer
> Project Cyrus Developer/Maintainer
> Carnegie Mellon University
>

Date: Fri, 26 May 2006 11:38:56 -0400
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Alexey Melnikov <[email-masked]>
CC: Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:
> Ken Murchison wrote:
> 
>> Alexey Melnikov wrote:
>>
>>> Timo Sirainen wrote:
>>>
>>>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>>>
>>>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>>>> Data Items):
>>>>>      If the server doesn't support the persistent storage of 
>>>>> mod-sequences
>>>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>>>> HIGHESTMODSEQ
>>>>>      status data item from the untagged STATUS response. If the 
>>>>> HIGHESTMODSEQ
>>>>>      is the only requested status data item, the server MUST omit 
>>>>> the untagged STATUS
>>>>>      response, as its grammar requires at least one status data 
>>>>> item to be returned.   
>>>>
>>>> I don't think omitting is the right thing to do, rather return NIL. 
>>>> This
>>>> way clients don't break (shouldn't break..) if server starts sending
>>>> STATUS events to clients unilaterally (which I'm planning to do at some
>>>> point).
>>>
>>> This is a valid comment.
>>> How about using 0 instead of NIL (0 is not a valid modsequence)?
>>
>> So, we're going to issue a HIGHESTMODSEQ response even if the mailbox 
>> doesn't support CONDSTORE.  This seems counter-intuitive.  I'm not 
>> sure we need to worry about breaking clients, since if they ask for 
>> HIGHESTMODSEQ on a non-CONDSTORE capable mailbox, the client is 
>> already broken.
> 
> The client might not know that the mailbox is NOMODSEQ, if it does 
> STATUS before SELECT.

Good point.  I can live with 0 (zero).

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Fri, 26 May 2006 16:20:38 +0100
From: Alexey Melnikov <[email-masked]>
To: Ken Murchison <[email-masked]>
CC: Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Ken Murchison wrote:

> Alexey Melnikov wrote:
>
>> Timo Sirainen wrote:
>>
>>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>>
>>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>>> Data Items):
>>>>      If the server doesn't support the persistent storage of 
>>>> mod-sequences
>>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>>> HIGHESTMODSEQ
>>>>      status data item from the untagged STATUS response. If the 
>>>> HIGHESTMODSEQ
>>>>      is the only requested status data item, the server MUST omit 
>>>> the untagged STATUS
>>>>      response, as its grammar requires at least one status data 
>>>> item to be returned.   
>>>
>>> I don't think omitting is the right thing to do, rather return NIL. 
>>> This
>>> way clients don't break (shouldn't break..) if server starts sending
>>> STATUS events to clients unilaterally (which I'm planning to do at some
>>> point).
>>
>> This is a valid comment.
>> How about using 0 instead of NIL (0 is not a valid modsequence)?
>
> So, we're going to issue a HIGHESTMODSEQ response even if the mailbox 
> doesn't support CONDSTORE.  This seems counter-intuitive.  I'm not 
> sure we need to worry about breaking clients, since if they ask for 
> HIGHESTMODSEQ on a non-CONDSTORE capable mailbox, the client is 
> already broken.

The client might not know that the mailbox is NOMODSEQ, if it does 
STATUS before SELECT.

Date: Fri, 26 May 2006 11:16:10 -0400
From: Ken Murchison <[email-masked]>
Organization: Carnegie Mellon University
To: Alexey Melnikov <[email-masked]>
CC: Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:
> 
> Timo Sirainen wrote:
> 
>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>  
>>
>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>> Data Items):
>>>      If the server doesn't support the persistent storage of 
>>> mod-sequences
>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>> HIGHESTMODSEQ
>>>      status data item from the untagged STATUS response. If the 
>>> HIGHESTMODSEQ
>>>      is the only requested status data item, the server MUST omit the 
>>> untagged STATUS
>>>      response, as its grammar requires at least one status data item 
>>> to be returned.
>>>   
>> I don't think omitting is the right thing to do, rather return NIL. This
>> way clients don't break (shouldn't break..) if server starts sending
>> STATUS events to clients unilaterally (which I'm planning to do at some
>> point).
>>  
>>
> This is a valid comment.
> How about using 0 instead of NIL (0 is not a valid modsequence)?

So, we're going to issue a HIGHESTMODSEQ response even if the mailbox 
doesn't support CONDSTORE.  This seems counter-intuitive.  I'm not sure 
we need to worry about breaking clients, since if they ask for 
HIGHESTMODSEQ on a non-CONDSTORE capable mailbox, the client is already 
broken.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Date: Fri, 26 May 2006 14:10:37 +0100
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:

> Timo Sirainen wrote:
>
>> On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>>
>>> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status 
>>> Data Items):
>>>      If the server doesn't support the persistent storage of 
>>> mod-sequences
>>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>> HIGHESTMODSEQ
>>>      status data item from the untagged STATUS response. If the 
>>> HIGHESTMODSEQ
>>>      is the only requested status data item, the server MUST omit 
>>> the untagged STATUS
>>>      response, as its grammar requires at least one status data item 
>>> to be returned.   
>>
>> I don't think omitting is the right thing to do, rather return NIL. This
>> way clients don't break (shouldn't break..) if server starts sending
>> STATUS events to clients unilaterally (which I'm planning to do at some
>> point).
>
> This is a valid comment.
> How about using 0 instead of NIL (0 is not a valid modsequence)?

Ok, here is the new suggested text:

     If the server doesn't support the persistent storage of mod-sequences
     for the mailbox (see Section 3.1.2), the server MUST return 0
     as the value of HIGHESTMODSEQ status data item.

And I will update the ABNF section accordingly, i.e. it will contain:

   status-att-val      =/ "HIGHESTMODSEQ" SP mod-sequence-valzer
                          ;; extends non-terminal defined in [IMAPABNF].
                          ;; Value 0 denotes that the mailbox doesn't 
support
                          ;; persistent mod-sequences as described in
                          ;; Section 3.1.2

instead of the current:
   status-att-val      =/ "HIGHESTMODSEQ" SP mod-sequence-value
                          ;; extends non-terminal defined in [IMAPABNF].

Subject: Re: CONDSTORE document is in AUTH48
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Fri, 26 May 2006 16:09:18 +0300
Sender: [email-masked]

--=-BATVUZRdHNvKZUx1cII5
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Fri, 2006-05-26 at 13:55 +0100, Alexey Melnikov wrote:
> Timo Sirainen wrote:
>=20
> >On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
> > =20
> >
> >>Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data=20
> >>Items):
> >>      If the server doesn't support the persistent storage of mod-seque=
nces
> >>      for the mailbox (see Section 3.1.2), the server MUST omit the=20
> >>HIGHESTMODSEQ
> >>      status data item from the untagged STATUS response. If the=20
> >>HIGHESTMODSEQ
> >>      is the only requested status data item, the server MUST omit the=20
> >>untagged STATUS
> >>      response, as its grammar requires at least one status data item t=
o=20
> >>be returned.
> >>   =20
> >>
> >I don't think omitting is the right thing to do, rather return NIL. This
> >way clients don't break (shouldn't break..) if server starts sending
> >STATUS events to clients unilaterally (which I'm planning to do at some
> >point).
> > =20
> >
> This is a valid comment.
> How about using 0 instead of NIL (0 is not a valid modsequence)?

Sure, if it's not valid then it's better for clients. Just remember to
update ABNF if it's currently invalid in there too :)

--=-BATVUZRdHNvKZUx1cII5
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-BATVUZRdHNvKZUx1cII5--

Date: Fri, 26 May 2006 13:55:49 +0100
From: Alexey Melnikov <[email-masked]>
To: Timo Sirainen <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Timo Sirainen wrote:

>On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
>  
>
>>Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data 
>>Items):
>>      If the server doesn't support the persistent storage of mod-sequences
>>      for the mailbox (see Section 3.1.2), the server MUST omit the 
>>HIGHESTMODSEQ
>>      status data item from the untagged STATUS response. If the 
>>HIGHESTMODSEQ
>>      is the only requested status data item, the server MUST omit the 
>>untagged STATUS
>>      response, as its grammar requires at least one status data item to 
>>be returned.
>>    
>>
>I don't think omitting is the right thing to do, rather return NIL. This
>way clients don't break (shouldn't break..) if server starts sending
>STATUS events to clients unilaterally (which I'm planning to do at some
>point).
>  
>
This is a valid comment.
How about using 0 instead of NIL (0 is not a valid modsequence)?

Subject: Re: CONDSTORE document is in AUTH48
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Date: Fri, 26 May 2006 15:50:36 +0300
Sender: [email-masked]

--=-BRACR337ycUT8diwp/gb
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Fri, 2006-05-26 at 13:40 +0100, Alexey Melnikov wrote:
> Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data=20
> Items):*
> *      If the server doesn't support the persistent storage of mod-sequen=
ces
>       for the mailbox (see Section 3.1.2), the server MUST omit the=20
> HIGHESTMODSEQ
>       status data item from the untagged STATUS response. If the=20
> HIGHESTMODSEQ
>       is the only requested status data item, the server MUST omit the=20
> untagged STATUS
>       response, as its grammar requires at least one status data item to=20
> be returned.

I don't think omitting is the right thing to do, rather return NIL. This
way clients don't break (shouldn't break..) if server starts sending
STATUS events to clients unilaterally (which I'm planning to do at some
point).

--=-BRACR337ycUT8diwp/gb
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part

--=-BRACR337ycUT8diwp/gb--

Date: Fri, 26 May 2006 13:49:00 +0100
From: Alexey Melnikov <[email-masked]>
To: [email-masked]
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:

> Arnt Gulbrandsen wrote:
>
>> Alexey Melnikov writes:
>>
>>> Anything else?
>>
>> Reference 4466 instead of the draft.
>
> This was already taken care of by the response RFC editor.

s/response/responsible

Date: Fri, 26 May 2006 13:43:43 +0100
From: Alexey Melnikov <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked]
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Arnt Gulbrandsen wrote:

>Alexey Melnikov writes:
>  
>
>>Anything else?
>>    
>>
>Reference 4466 instead of the draft.
>  
>
This was already taken care of by the response RFC editor.

Date: Fri, 26 May 2006 14:42:49 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: CONDSTORE document is in AUTH48
Cc: Alexey Melnikov <[email-masked]>
Sender: [email-masked]

Alexey Melnikov writes:
> Anything else?

Reference 4466 instead of the draft.

Arnt

Date: Fri, 26 May 2006 13:40:43 +0100
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: Re: CONDSTORE document is in AUTH48
Sender: [email-masked]

Alexey Melnikov wrote:

> Any last minute changes people want to do?
>
> I remember the following comments:
> 1). Comment from Ken regarding server behavior when the client sends a 
> CONDSTORE-extended command after the server returned NOMODSEQ on SELECT.

To be specific, I am suggesting to add to Section 3.1.2 (NOMODSEQ 
Response Code):
*
*A server that returned NOMODSEQ response code for a mailbox, which 
subsequently receives one of the following command while the mailbox is 
selected:
      -  a FETCH command with the CHANGEDSINCE modifier,
      -  a FETCH or SEARCH command that includes the MODSEQ message data
         item, or
      -  a STORE command with the UNCHANGEDSINCE modifier
MUST reject any such command with the tagged BAD response.

Also add the following text to Section 3.6. (HIGHESTMODSEQ Status Data 
Items):*
*      If the server doesn't support the persistent storage of mod-sequences
      for the mailbox (see Section 3.1.2), the server MUST omit the 
HIGHESTMODSEQ
      status data item from the untagged STATUS response. If the 
HIGHESTMODSEQ
      is the only requested status data item, the server MUST omit the 
untagged STATUS
      response, as its grammar requires at least one status data item to 
be returned.

> 2). Arnt's comments about unique IMAP tag in examples
>
> Anything else?
>
> Alexey
>

Date: Fri, 26 May 2006 13:20:08 +0100
From: Alexey Melnikov <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: CONDSTORE document is in AUTH48
Sender: [email-masked]

Any last minute changes people want to do?

I remember the following comments:
1). Comment from Ken regarding server behavior when the client sends a 
CONDSTORE-extended command after the server returned NOMODSEQ on SELECT.
2). Arnt's comments about unique IMAP tag in examples

Anything else?

Alexey

Date: Tue, 16 May 2006 12:37:54 -0700
From: "Mark Davis" <[email-masked]>
To: "Arnt Gulbrandsen" <[email-masked]>
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

------=_Part_41077_30668366.1147808274668
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: base64
Content-Disposition: inline

------=_Part_41077_30668366.1147808274668
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: base64
Content-Disposition: inline

------=_Part_41077_30668366.1147808274668--

Date: Tue, 16 May 2006 20:25:19 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: Mark Davis <[email-masked]>, [email-masked]
Sender: [email-masked]

Arnt Gulbrandsen writes:
> Mark Davis writes:
>> ...
>> At a quick glance, it appears that a number of comments have been 
>> incorporated.
>
> It is possible that some of my changes don't satisfy you. I had 
> conflicting requests for many things. Feel free to repeat, rephrase 
> or add arguments.

In -10 (which I'll send off once I finish work this evening) I've made 
another few changes.

>>>       > 2.4 Sort Keys
>>>
>>> The use of the term "collation canonicalization" to refer to sort 
>>> keys is very misleading. ...
>
> Changed; the text now speaks of sort keys. I'm afraid there still are 
> instances of the old term around, I found one today.

In -10, all should be dead.

>>> The term 'error' is also problematic, since what is really at issue 
>>> is a question of domain. For all those strings in the domain, 
>>> either 'equal' or 'not_equal' should be returned from the equality 
>>> function. For any string not in the domain, 'undefined' should be 
>>> returned.
>
> Not changed. Back in February, I agreed that "error" was not ideal, 
> but did not see "undefined" as better, and could not find a really 
> apt term. The collations were a little too well-defined in the 
> "undefined" cases then.
>
> However, in -10, I think they really will be undefined outside their 
> domain, so I'll change to using "undefined" instead of "error". (I'm 
> removing the bits that fall back to i;octet.)

Changed. The fallback to i;octet is now in the server, if the protocol 
requires it.

This means that if a server can escape implementing i;octet, it can keep 
all its strings in UCS-2 or UCS-4 internally, even as it implements 
collations which are defined in terms of octets.

Arnt

Date: Tue, 16 May 2006 18:21:37 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: John Cowan <[email-masked]>, Mark Davis <[email-masked]>, [email-masked]
Sender: [email-masked]

John Cowan writes:
> Instead of the requestor sending the request to IANA, who sends it to 
> the Collation Reviewer for discussion on the list, and then the 
> Collation Reviewer sends it back to IANA for registration (or 
> doesn't), remove the first pass through IANA.

Fine. Done.

Arnt

Date: Thu, 11 May 2006 09:38:01 -0400
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked], Mark Davis <[email-masked]>, [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
From: John Cowan <[email-masked]>
Sender: [email-masked]

I propose that the procedure specified in draft-newman-i18n-collation-09
for getting new collations approved should be changed to the procedure
used for new language tags.  Instead of the requestor sending the
request to IANA, who sends it to the Collation Reviewer for discussion
on the list, and then the Collation Reviewer sends it back to IANA for
registration (or doesn't), remove the first pass through IANA.

Have people post directly to the list and work out the details.  When the
requestor thinks it's ready, the Collation Reviewer wakes up and then
either sends the latest draft of the request to IANA or else sends a
rejection (with reasons) to the list.  This lowers the load on IANA.

This scheme has worked very well for [email-masked] for the
past 11 years.

-- 
John Cowan  [email-masked]   http://ccil.org/~cowan
"The exception proves the rule."  Dimbulbs think: "Your counterexample proves
my theory."  Latin students think "'Probat' means 'tests': the exception puts
the rule to the proof."  But legal historians know it means "Evidence for an
exception is evidence of the existence of a rule in cases not excepted from."

Date: Wed, 10 May 2006 15:08:53 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: Mark Davis <[email-masked]>, [email-masked]
Sender: [email-masked]

Mark Davis writes:
> The release of this is timely (we didn't get notified of a 07 or 08 
> draft), since the Unicode Technical Committee is meeting next week, 
> and can discuss it.
>
> Could you indicate which of the items raised in the email of 
> 2006-02-21 from the Unicode Technical Committee have been addressed 
> in this release (and if not accepted then why)? That would help 
> greatly with the review. (I couldn't find any archive for discussion 
> of draft-newman-i18n-comparator where that email could be publicly 
> linked from, so I am appending it at the end of this message.) At a 
> quick glance, it appears that a number of comments have been 
> incorporated.

Lots. Some not. See below.

It is possible that some of my changes don't satisfy you. I had 
conflicting requests for many things. Feel free to repeat, rephrase or 
add arguments.

> Mark
>
> BTW, despite the subject of the message, the document is at 
> http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt. 
> It helps to send out a link, especially if the name (comparator vs 
> collation) is wrong ;-)

Mea culpa. My apologies.

...
>> To:   Network Working Group
>> Re:   draft-newman-i18n-comparator
>> Date:         2006-02-21
>> From:         Unicode Technical Committee
>>
>> The Unicode Technical Committee has reviewed the document 
>> http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-06.txt. 
>> While UTC is in favor of the goal, there are a number of problems 
>> with the document. The main problems are outlined below. Once these 
>> are addressed, then further review can continue.
>>
>>     Details
>>
>>       > 2.1 Definitions
>>
>>         Content
>>
>> The document needs to include the definitions of the technical terms 
>> used in the document,  including all those that may not be familiar 
>> to implementers, such as "trichotomous" and "collation identifiers". 
>> In particular, the notion of a substring is /prima facie/ quite 
>> simple, but there are complications that require a clear definition. 
>> The text in the document does not make clear that there may be more 
>> than one match for a substring in a string, and that the matches can 
>> overlap. It says "the starting offset", for example, when there may 
>> be multiple ones.

Changed.

>> Moreover, language sensitive matches have additional complications 
>> which need to be called out. For more information, see 
>> http://www.unicode.org/reports/tr10/#Searching

Not really changed. As I recall, I added a little bit of text.

>>         Format
>>
>> If there is a "Definitions" section, readers have a reasonable 
>> expectation that that section should contain all the required 
>> definitions. However, a number of definitions are scattered within 
>> the text. One of two approaches should be taken
>>
>>    1. Move all the definitions into this section.
>>    2. Remove the definitions section, but clearly call out in the text
>>       the definitions of  each terms on its own line.
>>
>> Mixing these two styles is needlessly confusing for readers.

Not changed; I'm going by what confuses reviewers.

>>       > 2.4 Sort Keys
>>
>> The use of the term "collation canonicalization" to refer to sort 
>> keys is very misleading. ...

Changed; the text now speaks of sort keys. I'm afraid there still are 
instances of the old term around, I found one today.

>> > 3.2
>>
>> This specifies that clients that support disconnected operation 
>> should not use wildcards while clients that provide collation 
>> operations only when connected to the server may use wildcards.

This restrinction has been lifted.

>> The EBNF syntax shown in section 3.2 says that the collation-wild 
>> must not exceed 255 characters total while the section 3.1 specifies 
>> that the collation name must not exceed 254 characters.

Brought into sync.

>> It seems having the same maximum possible length for both collation 
>> name and wildcard string would be desirable for actual 
>> implementations.

I picked 254, not 255, but I confess I cannot remember why.

>>       > 4.2.1 Equality
>>
>> It needs to be made clear that the return values are not physically 
>> the strings "match", etc. but enumerated values such as /equal/ and  
>> /not_equal/.

Changed. Also other similar changes.

>> One extremely important point is that for a given comparator, the 
>> equality function must be synchronized with the ordering function.

I've done this and all the other equivalences/connections/implications I 
could see.

>> The term 'error' is also problematic, since what is really at issue 
>> is a question of domain. For all those strings in the domain, either 
>> 'equal' or 'not_equal' should be returned from the equality 
>> function. For any string not in the domain, 'undefined' should be 
>> returned.

Not changed. Back in February, I agreed that "error" was not ideal, but 
did not see "undefined" as better, and could not find a really apt 
term. The collations were a little too well-defined in the "undefined" 
cases then.

However, in -10, I think they really will be undefined outside their 
domain, so I'll change to using "undefined" instead of "error". (I'm 
removing the bits that fall back to i;octet.)

>> There is a typo at the 4'th line of the second paragraph of the 
>> section 4.2 saying "... For example, an collation" which should be 
>> changed to "... For example, a collation" instead.

Fixed.

>>       > 4.2.2 Substring
>>
>> Prefix and suffix matching are not fully spelled out.

I think they are now.

>> The operations and their results must be clarified. And as noted 
>> before, it is very important to precisely define the substring 
>> operations, especially the starting offset and ending offset. It 
>> also must be clarified whether what is being asked for is the first 
>> possible matching location in the string, the last, or the nth one.

Partly changed. I didn't do the bits you ask for in the last sentence. I 
can add an open issue.

>>       > 4.3.3 Ordering
>>
>> > It MUST be transitive and trichotomous.
>>
>> As above, these should be defined.

I did not, since I think this document is the wrong place to define 
these terms.

>> The exposition in this section would be simpler if you also defined 
>> "reversible", whereby f(a,b) = less iff f(b,a) = greater.

The exposition changed enough as a result of other commens that I 
isregarded this comment.

>> An 'undefined' value can be allowed if, as per equality above, it 
>> means that at least one of the operands is outside of the domain. 
>> The function then imposes a total order on all strings in the 
>> domain; moreover, a wrapper can easily convert the function to a 
>> total order over all strings by putting all items outside the domain 
>> either below or above the ones in the domain -- or even excluding 
>> them,/ at its choice./

I'm doing something like this in -10. (Removing the fallback to i;octet.)

>> [Note: it is very important to avoid the confusion between 
>> "identical" and "equal". According to a caseless compare, "Mark" and 
>> "mark" are equal; however, the strings are not identical.]

Changed all over the place.

>> [Either 'ordering function' or 'comparison function' should be used 
>> consistently, not sometimes 'collations'].

Changed.

>>       > 4.3.  Internal Canonicalization Algorithm
>>
>> This section is difficult to understand.

Changed; I hope the new text is better.

>>       > 4.4.  Use of Lookup Tables
>>
>> It is not at all clear what is meant by "customizable lookup tables".

Clarified and partly removed.

>>       > 4.5.  Multi-Value Attributes
>>
>> This is very unclear.

Deleted.

>> This is a very important feature that needs to be spelled out in 
>> detail, and clearly reflected in the template for registration. In 
>> particular, the template should have provision for multiple 
>> attributes, with the ability to specify the acceptable operands for 
>> that attribute. (See below). The specification of the operands could 
>> be either a list of values, or a regular expression (with the former 
>> preferred). Suggested regular expression syntax would be Perl or XML 
>> Schema.

I asked Martin D�rst and you to provide a new DTD. Martin said okay, I 
don't remember whether you answered. I think the DTD should come before 
this.

>>       > 5.1Character Encoding
>>
>>    The protocol specification has to make sure that it is clear on which
>>    characters (rather than just octets) the collations are used.  This
>>    can be done by specifying the protocol itself in terms of characters
>>    (e.g. in the case of a query language), by specifying a single
>>    character encoding for the protocol (e.g.  UTF-8 [3]), or by
>>    carefully describing the relevant issues of character encoding
>>    labeling and conversion.  In the later case, details to consider
>>    include how to handle unknown charsets, any charsets which are
>>    mandatory-to-implement, any issues with byte-order that might apply,
>>    and any transfer encodings which need to be supported.
>>
>> If a collation is able to advertise itself as being able to handle, 
>> say, SJIS and UTF-8, then there should a required description of a 
>> protocol for indicating that and for communicating which encodings 
>> are handled, and how it handles error conditions (such as a charset 
>> outside of those it can handle. Otherwise, it is difficult to 
>> understand how this paragraph would be applied in practice.
>>
>>       > 5.3
>>
>> The section 5.3 specifies:
>>
>>     The protocol MUST specify how comparisons behave in the absence of
>>     explicit collation negotiation or when a collation of "*" is
>>     requested. The protocol MAY specify that the default collation
>>     used in such circumstances is sensitive to server configuration.
>>
>> and the section 3.2 specifies:
>>
>>     ... If the wildcard string matches multiple collations, the server
>>     SHOULD select the collation with the broadest scope (preferably
>>     international scope), the most recent table versions and the
>>     greatest number of supported operations. A single wildcard
>>     character ("*") refers to the application protocol collation
>>     behavior that would occur if no explicit negotiation were used.
>>
>> These appear inconsistent.

Changed.

>>       7.5.  Example Initial Registry Summary
>>
>> The sample registry would suffer a combinatorial explosion if 
>> parameters are not handled differently.
...

This is the DTD issue.

>> > 11.  Security Considerations
>>
>> This is insufficient. It should at least point to the problems 
>> related in UCA and in 
>> http://www.unicode.org/reports/tr36/tr36-4.html (note that that 
>> document has been approved by the UTC and will be posted as an 
>> approved version soon.)

It now refers.

>>     General
>>
>> One of the real problems with the IANA character registry is that the 
>> entries are underspecified. It quite often occurs that two vendors 
>> implement the same IANA charset conversion different ways, leading 
>> to significant interoperability problems and text corruption. See, 
>> for example, 
>> http://www.w3.org/Submission/japanese-xml/#ambiguity_of_yen.
>>
>> We have the real concern that this registry could lead down the same path.

Noted.

>> > collation, it has to say so
>>
>> There are places where the text should be clarified, as to whether a 
>> MUST or SHOULD is implied; this is just an example.
>>
>> > "comparator" vs "collator"
>>
>> Either one term or the other should be used consistently.

Collator, now.

>> > Unicode 3.2
>>
>> Unicode 3.2 is obsolete; the the reference versions for the Collation 
>> Registry should be Unicode 5.0 and UCA 5.0, since those will be 
>> approved and published by the time the Internet Application Protocol 
>> Collation Registry has completed its review and been approved.

I'll update to the then-current versions immediately before submitting 
the final draft as an RFC.

>> Because of the use of NamePrep, it is probably the case that Unicode 
>> 3.2 also needs to be included, but strongly recommended for usage 
>> only by protocols or systems dependent on NamePrep. Note that as of 
>> UCA 4.0 and beyond, the version number of UCA is guaranteed to be 
>> identical with the version number of Unicode that it is defined for.
>>
>> > Versioning
>>
>> This is tricky, and should be clarified. In many instances, it is 
>> sufficient to use an unversioned collator, such as simply "UCA". In 
>> other cases, there are requirements to use a specific version, or a 
>> version of at least X. This needs to be described.

IETF documents should have only immutable references. Thus, I can 
reference "UCAv14", but not "UCA", because the latter moves to v15, v16 
and onwards.

Arnt

Date: Wed, 10 May 2006 00:47:22 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Cc: John Cowan <[email-masked]>, [email-masked]
Sender: [email-masked]

John Cowan writes:
> Arnt Gulbrandsen scripsit:
>>  As far as I know, this addresses, ignores or adds open issues for 
>>  all requests. If something is ignored, that's because other people 
>>  wanted the opposite, or because I overlooked it when I went over 
>>  all the mail this week. I'm sorry about it in either case.
>>
>>  Review, please.
>
> Posted where? Neither rfc-editor.org nor ietf.org seems to have it.

http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt 
has it. Sorry. I posted that only an hour or two after the i-d editor 
told me it was posted (at that URL), which must have been too quickly 
for most mirrors.

Arnt

Date: Tue, 09 May 2006 15:43:23 -0700
From: Mark Davis <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
CC: [email-masked], [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
Sender: [email-masked]

The release of this is timely (we didn't get notified of a 07 or 08 
draft), since the Unicode Technical Committee is meeting next week, and 
can discuss it.

Could you indicate which of the items raised in the email of 2006-02-21 
from the Unicode Technical Committee have been addressed in this release 
(and if not accepted then why)? That would help greatly with the review. 
(I couldn't find any archive for discussion of 
draft-newman-i18n-comparator where that email could be publicly linked 
from, so I am appending it at the end of this message.) At a quick 
glance, it appears that a number of comments have been incorporated.

Mark

BTW, despite the subject of the message, the document is at 
http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt. 
It helps to send out a link, especially if the name (comparator vs 
collation) is wrong ;-)

BTW, it was pointed out to us that the original email shouldn't have 
been sent to "Network Working Group", even though that is the name at 
the top of 
http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-09.txt

Arnt Gulbrandsen wrote:
>
> As far as I know, this addresses, ignores or adds open issues for all 
> requests. If something is ignored, that's because other people wanted 
> the opposite, or because I overlooked it when I went over all the mail 
> this week. I'm sorry about it in either case.
>
> Review, please.
>
> Arnt

=================

Mark Davis wrote:
> To: 	Network Working Group
> Re: 	draft-newman-i18n-comparator
> Date: 	2006-02-21
> From: 	Unicode Technical Committee
>
> The Unicode Technical Committee has reviewed the document 
> http://www.ietf.org/internet-drafts/draft-newman-i18n-comparator-06.txt. 
> While UTC is in favor of the goal, there are a number of problems with 
> the document. The main problems are outlined below. Once these are 
> addressed, then further review can continue.
>
>
>     Details
>
>
>       > 2.1 Definitions
>
>
>         Content
>
> The document needs to include the definitions of the technical terms 
> used in the document,  including all those that may not be familiar to 
> implementers, such as "trichotomous" and "collation identifiers". In 
> particular, the notion of a substring is /prima facie/ quite simple, 
> but there are complications that require a clear definition. The text 
> in the document does not make clear that there may be more than one 
> match for a substring in a string, and that the matches can overlap. 
> It says "the starting offset", for example, when there may be multiple 
> ones.
>
> Moreover, language sensitive matches have additional complications 
> which need to be called out. For more information, see 
> http://www.unicode.org/reports/tr10/#Searching
>
>
>         Format
>
> If there is a "Definitions" section, readers have a reasonable 
> expectation that that section should contain all the required 
> definitions. However, a number of definitions are scattered within the 
> text. One of two approaches should be taken
>
>    1. Move all the definitions into this section.
>    2. Remove the definitions section, but clearly call out in the text
>       the definitions of  each terms on its own line.
>
> Mixing these two styles is needlessly confusing for readers.
>
>
>       > 2.4 Sort Keys
>
> The use of the term "collation canonicalization" to refer to sort keys 
> is very misleading. The term "canonicalization" implies that the 
> results are still text in some fashion, whereas a sortkey is simply a 
> string of octets generated from a given string by a specific 
> comparator, whereby the binary comparison (ordering) of two sort keys 
> is guaranteed to match *that* comparator's compare function for the 
> original strings. The octets may have no readily discernable relation 
> to the original text. For example, the ICU sort keys generated for the 
> following strings are:
>
> cote 	2c 44 4e 30 01 08 01 08 00
> côté 	2c 44 4e 30 01 85 93 85 8d 01 0a 00
> Αραβικά 	5c 20 52 20 22 36 3a 20 01 80 8d 01 8f 0b 00
>
> See 
> http://www-950.ibm.com/software/globalization/icu/demo/locales/en/?_=el&d_=en&x=col 
> <http://www-950.ibm.com/software/globalization/icu/demo/locales/en/?_=el&d_=en&x=col> 
> for other examples.
>
> > 3.2
>
> This specifies that clients that support disconnected operation should 
> not use wildcards while clients that provide collation operations only 
> when connected to the server may use wildcards.
>
> It appears the restrictions are may not be really needed and the 
> restrictions may need to be deleted from the draft. Otherwise, it 
> would really helpful if the rationale behind the restrictions are 
> provided at the draft.
>
> The EBNF syntax shown in section 3.2 says that the collation-wild must 
> not exceed 255 characters total while the section 3.1 specifies that 
> the collation name must not exceed 254 characters.
>
> It seems having the same maximum possible length for both collation 
> name and wildcard string would be desirable for actual implementations.
>
>
>       > 4.2.1 Equality
>
> It needs to be made clear that the return values are not physically 
> the strings "match", etc. but enumerated values such as /equal/ and  
> /not_equal/. The document could describe a notation used for them, 
> such as single quotes, since italic is not available in RFCs. 
> Similarly, the results of the ordering function should be specified as 
> an enumeration with three values: /less/, /equal/, /greater./ The 
> mapping actual API return values in implementations to these 
> enumerated values can be outside of the scope of this document. For 
> example, the mapping might take -1 onto /less/ in one implementation, 
> or anything negative onto /less/ in another implementation.
>
> One extremely important point is that for a given comparator, the 
> equality function must be synchronized with the ordering function. 
> That is, it must return 'equal' if and only if the ordering function 
> returns 'equal'. Otherwise any coordinated usage of the functions will 
> fail. This also implies that either 'error' is allowed for both 
> functions or for neither.
>
> The term 'error' is also problematic, since what is really at issue is 
> a question of domain. For all those strings in the domain, either 
> 'equal' or 'not_equal' should be returned from the equality function. 
> For any string not in the domain, 'undefined' should be returned. That 
> avoids coherency problems. Then the requirements are clear:
>
>     * if A and B are in the domain, then the result of an equality
>       test is either /equal/ or /not_equal/
>     * if A or B (or both) are not in the domain, then the result of an
>       equality test is /undefined/.
>
> There is a typo at the 4'th line of the second paragraph of the 
> section 4.2 saying "... For example, an collation" which should be 
> changed to "... For example, a collation" instead.
>
>
>       > 4.2.2 Substring
>
> Prefix and suffix matching are not fully spelled out. The operations 
> and their results must be clarified. And as noted before, it is very 
> important to precisely define the substring operations, especially the 
> starting offset and ending offset. It also must be clarified whether 
> what is being asked for is the first possible matching location in the 
> string, the last, or the nth one.
>
>
>       > 4.3.3 Ordering
>
> > It MUST be transitive and trichotomous.
>
> As above, these should be defined. The exposition in this section 
> would be simpler if you also defined "reversible", whereby f(a,b) = 
> less iff f(b,a) = greater. Then the statement would be:
>
>     It MUST be transitive, trichotomous, and reversible.
>
> >When the collation is used with a
>    "-" prefix, the result of the ordering function of the collation MUST
>    be reversed.
>
> => When the collation is used with a
>    "-" prefix, the result of the ordering function of the collation 
> when applied to two strings A and B  MUST
>    be the same as the result with a "+" prefix applied to B and A.
>
> An 'undefined' value can be allowed if, as per equality above, it 
> means that at least one of the operands is outside of the domain. The 
> function then imposes a total order on all strings in the domain; 
> moreover, a wrapper can easily convert the function to a total order 
> over all strings by putting all items outside the domain either below 
> or above the ones in the domain -- or even excluding them,/ at its 
> choice./
>
>  > In general, collations SHOULD NOT return "0" unless the two strings 
> are identical.
>
> => The ordering function MUST return 'equal' if and only if the equality function returns 'equal'
>
> [Note: it is very important to avoid the confusion between "identical" 
> and "equal". According to a caseless compare, "Mark" and "mark" are 
> equal; however, the strings are not identical.]
>
> [Either 'ordering function' or 'comparison function' should be used 
> consistently, not sometimes 'collations'].
>
>
>       > 4.3.  Internal Canonicalization Algorithm
>
> This section is difficult to understand. It appears that goal is that 
> any registration must specify sufficient detail, both data and 
> algorithm, so as to enable someone to reproduce the results. But it is 
> not at all clear that that is the goal. And that would make the 
> registration require, in some cases, a huge accompanying document. To 
> duplicate the results of CLDR collators, for example, would require 
> the UCA specification, plus the LDML specification, plus all the 
> relevant data in the CLDR repository.
>
>
>       > 4.4.  Use of Lookup Tables
>
> It is not at all clear what is meant by "customizable lookup tables".
>
>
>       > 4.5.  Multi-Value Attributes
>
> This is very unclear. It describes attributes as applying to only 
> equality (since it only refers to "match" vs "no-match" (and 
> forgetting "error")).
>
> This is a very important feature that needs to be spelled out in 
> detail, and clearly reflected in the template for registration. In 
> particular, the template should have provision for multiple 
> attributes, with the ability to specify the acceptable operands for 
> that attribute. (See below). The specification of the operands could 
> be either a list of values, or a regular expression (with the former 
> preferred). Suggested regular expression syntax would be Perl or XML 
> Schema.
>
>
>       > 5.1Character Encoding
>
>    The protocol specification has to make sure that it is clear on which
>    characters (rather than just octets) the collations are used.  This
>    can be done by specifying the protocol itself in terms of characters
>    (e.g. in the case of a query language), by specifying a single
>    character encoding for the protocol (e.g.  UTF-8 [3]), or by
>    carefully describing the relevant issues of character encoding
>    labeling and conversion.  In the later case, details to consider
>    include how to handle unknown charsets, any charsets which are
>    mandatory-to-implement, any issues with byte-order that might apply,
>    and any transfer encodings which need to be supported.
>
> If a collation is able to advertise itself as being able to handle, 
> say, SJIS and UTF-8, then there should a required description of a 
> protocol for indicating that and for communicating which encodings are 
> handled, and how it handles error conditions (such as a charset 
> outside of those it can handle. Otherwise, it is difficult to 
> understand how this paragraph would be applied in practice.
>
>
>       > 5.3
>
> The section 5.3 specifies:
>
>     The protocol MUST specify how comparisons behave in the absence of
>     explicit collation negotiation or when a collation of "*" is
>     requested. The protocol MAY specify that the default collation
>     used in such circumstances is sensitive to server configuration.
>
> and the section 3.2 specifies:
>
>     ... If the wildcard string matches multiple collations, the server
>     SHOULD select the collation with the broadest scope (preferably
>     international scope), the most recent table versions and the
>     greatest number of supported operations. A single wildcard
>     character ("*") refers to the application protocol collation
>     behavior that would occur if no explicit negotiation were used.
>
> These appear inconsistent.
>
>
>       7.5.  Example Initial Registry Summary
>
> The sample registry would suffer a combinatorial explosion if 
> parameters are not handled differently. For example, with CLDR 
> collations, there can be hundreds of locales, six different strength 
> settings; four different case-first settings; three different 
> alternate settings, backwards settings, normalization settings, case 
> level settings, hiragana settings, and numeric settings; plus a 
> variable-top setting which has a string as an operand. Registering the 
> combinations that people are allowed to use would be untenable.
>
> http://www.unicode.org/draft/reports/tr35/tr35.html#Setting_Options
>
> Instead, as remarked above, the allowable attribute values need to be 
> associated with the registered name in a machine-readable form.
>
> > 11.  Security Considerations
>
> This is insufficient. It should at least point to the problems related 
> in UCA and in http://www.unicode.org/reports/tr36/tr36-4.html (note 
> that that document has been approved by the UTC and will be posted as 
> an approved version soon.)
>
>
>     General
>
> One of the real problems with the IANA character registry is that the 
> entries are underspecified. It quite often occurs that two vendors 
> implement the same IANA charset conversion different ways, leading to 
> significant interoperability problems and text corruption. See, for 
> example, http://www.w3.org/Submission/japanese-xml/#ambiguity_of_yen.
>
> We have the real concern that this registry could lead down the same path.
>
> > collation, it has to say so
>
> There are places where the text should be clarified, as to whether a 
> MUST or SHOULD is implied; this is just an example.
>
> > "comparator" vs "collator"
>
> Either one term or the other should be used consistently.
>
> > Unicode 3.2
>
> Unicode 3.2 is obsolete; the the reference versions for the Collation 
> Registry should be Unicode 5.0 and UCA 5.0, since those will be 
> approved and published by the time the Internet Application Protocol 
> Collation Registry has completed its review and been approved.
>
> Because of the use of NamePrep, it is probably the case that Unicode 
> 3.2 also needs to be included, but strongly recommended for usage only 
> by protocols or systems dependent on NamePrep. Note that as of UCA 4.0 
> and beyond, the version number of UCA is guaranteed to be identical 
> with the version number of Unicode that it is defined for.
>
> > Versioning
>
> This is tricky, and should be clarified. In many instances, it is 
> sufficient to use an unversioned collator, such as simply "UCA". In 
> other cases, there are requirements to use a specific version, or a 
> version of at least X. This needs to be described.
>

Date: Tue, 9 May 2006 16:45:02 -0400
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: draft-newman-i18n-collation-09.txt just posted
From: John Cowan <[email-masked]>
Sender: [email-masked]

Arnt Gulbrandsen scripsit:

> As far as I know, this addresses, ignores or adds open issues for all 
> requests. If something is ignored, that's because other people wanted 
> the opposite, or because I overlooked it when I went over all the mail 
> this week. I'm sorry about it in either case.
> 
> Review, please.

Posted where?  Neither rfc-editor.org nor ietf.org seems to have it.

-- 
John Cowan  [email-masked]  http://ccil.org/~cowan
The penguin geeks is happy / As under the waves they lark
The closed-source geeks ain't happy / They sad cause they in the dark
But geeks in the dark is lucky / They in for a worser treat
One day when the Borg go belly-up / Guess who wind up on the street.

Date: Tue, 9 May 2006 22:14:28 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked], [email-masked], [email-masked]
Subject: draft-newman-i18n-collation-09.txt just posted
Sender: [email-masked]

As far as I know, this addresses, ignores or adds open issues for all 
requests. If something is ignored, that's because other people wanted 
the opposite, or because I overlooked it when I went over all the mail 
this week. I'm sorry about it in either case.

Review, please.

Arnt

Date: Tue, 02 May 2006 20:22:56 +0100
Subject: Re: SEARCH and COMPARATOR
From: Dave Cridland <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

On Tue May  2 20:05:07 2006, Arnt Gulbrandsen wrote:
> I am resolving the SEARCH issue by fiat: "COMPARATOR collation-name 
> search-key", applying that collation to that search-key (which may 
> use ordinary IMAP and/or/blah).
> 
> 
My inclination is that this might be better dealt with in more 
drastic ways, and we should actually forget about it for now. 
Specifically, I think IMAP ought to admit that Sieve has much more 
interesting tests, for the most part, and even provides ANDs and 
exciting stuff. This is not to suggest that you need write all that 
in, it's more to suggest that it's not worth tackling that problem 
there.

Good enough for me would be a comparator option for SEARCH changing 
the default comparator for that SEARCH command only.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

Date: Tue, 2 May 2006 21:05:07 +0200
From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: SEARCH and COMPARATOR
Sender: [email-masked]

Hi,

now that Timo Sirainen shot down my GIF87 example (for which I am 
grateful, even as I am feeling less than grateful), could someone dream 
up a reason to use two different collations at the same time, or at 
least nearly at the same time? Ideally a search.

I am resolving the SEARCH issue by fiat: "COMPARATOR collation-name 
search-key", applying that collation to that search-key (which may use 
ordinary IMAP and/or/blah).

Arnt