Subject: I-D ACTION:draft-hazewinkel-imap-vhost-00.txt (fwd)
From: Cyrus Daboo <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Date: Fri, 31 Jan 2003 13:08:00 -0500
List-Id: <ietf-imapext.imc.org>

------------ Forwarded Message -----------
Date: Friday, January 31, 2003 7:51 AM -0500
From: [email-masked]
To: IETF-Announce
Subject: I-D ACTION:draft-hazewinkel-imap-vhost-00.txt

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: IMAP Virtual Hosting
	Author(s)	: H. Hazewinkel
	Filename	: draft-hazewinkel-imap-vhost-00.txt
	Pages		: 6
	Date		: 2003-1-30
	
This memo describes an extension to the IMAP protocol in order to
enable virtual hosting. The concept of virtual hosting for HTTP is
used and a new IMAP command is introduced to provide the virtual
hosting feature.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-hazewinkel-imap-vhost-00.txt

---------- End Forwarded Message ----------

Just saw the above draft appear. I think this problem has already been 
solved for the most part by having domain information expressed in 
authentication ids. I did not understand the comment in the draft about 
'domain in auth-id' creating a 'single level list' and why that is a 
problem. Can you explain in more detail the benefits that VHOST offers over 
the 'domain in auth-id' approach? Its worth noting that 'domain in auth-id' 
works right now without requiring modifications to any clients, whereas 
VHOST would require client support.

-- 
Cyrus Daboo

From: "Barry Leiba" <[email-masked]>
To: "IMAP extensions Mailing List" <[email-masked]>
Subject: Re: RENAME, once more
Date: Tue, 28 Jan 2003 10:08:27 -0500
Sender: [email-masked]

> The new-uidvalidity tag would only indicate the case were the uidvalidity was
> changed (or perhaps not changed) and the uids were preserved.  It would not say
> anything about the other cases where the uids changed.

Yes, I think that's what Timo is proposing.

> If you give the client the new uidvalidity
> and old uids, when it doesn't get that information most likely the client being
> lazy will assume that neither changed.  This could lead to some nasty suprises
> and finger pointing when the validity didn't change but the uids changed

Why?  NEW-UIDVALIDITY doesn't exist now; Timo is defining it.  As such, Timo gets
to propose the definition as HE wants it, not as the proverbial lazy implementer
might want it.  I think he's proposing this:

1. If the server can do this reasonably (for some value of "reasonably" that's up
to the server to determine), it MAY leave the UIDs of the messages unchanged and
send back a NEW-UIDVALIDITY value in the response.

2. If the server chooses not to do (1) because it's unable to guarantee that the
UIDs have not changed, or because the process is, for whatever reason, not deemed
to be "reasonable", then it MUST NOT send back NEW-UIDVALIDITY.

3. In any case, the server MUST meet the current requirements for setting the
UIDVALIDITY value of the new mailbox.

4. If the client receives NEW-UIDVALIDITY, it MAY use that information to keep
its cache valid, without having to refresh it from the newly renamed mailbox.

5. If the client does not receive NEW-UIDVALIDITY, does not understand it, or,
for whatever reason, chooses not to handle it, it MUST treat the newly renamed
mailbox as is required now.  It will see a mailbox with the new name and with
a UIDVALIDITY value that it has not yet encountered (because of (3) above).  It
MUST, therefore, treat it as a new mailbox, discarding anything it thinks it
already knows about it.

Timo, please correct me if I've misstated anything here, or missed anything.
I think this is a reasonable approach, giving the server the option of improving
the situation, and giving the client a chance to behave more efficiently when a
mailbox is renamed, but not breaking any existing implementations nor requiring
anyone to change if they don't want to.

It does not address any of Mark's other issues with RENAME, of course, and for
implementations that must deal with mailstores that expose those issues, this
is a real problem.  I suppose a server can always respond "NO" to any RENAME
command, and thus require the client to use alternatives, but I suspect that
most clients would just expose the "NO" to the user, and make the user deal
with it.

I would support a move to change the new IMAP draft to suggest that a server
that has trouble with RENAME MAY say "NO", and that a client that receives
a "NO" to RENAME SHOULD try the CREATE/move alternative before exposing the
failure to the end-user.  Maybe we should define a response code to go with
the "NO", à la "TRY-CREATE", that would give a savvy client a hint that the
server doesn't like RENAME and wants the client to try alternatives.

The reason I don't want to deprecate RENAME outright is that for many servers
it IS a reasonable thing to do (though, yes, most (all?) servers do not implement
it strictly according to the spec; it'd be nice to have that fixed).

Barry
--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

Date: Thu, 23 Jan 2003 09:40:35 +0300
From: Alexey Melnikov <[email-masked]>
To: Barry Leiba <[email-masked]>
CC: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

Barry Leiba wrote:

> Here's a draft (ahem) copy of the next LIST Extensions draft.  Please
> give it a look and give comments before I send it in.  Notably added is
> the "mbox-list-extension" item in the formal grammar, and the
> associated bits.

IMHO, you should also add some text saying that we also extend LIST responses.

ABNF:
   mbox-list-extended = mbox-list-extended-item
                        *(SP mbox-list-extended-item)
   mbox-list-extended-item = string / mbox-list-extended

So this basically says that we have a list of strings without nested lists? I was thinking that we
should tag different bits of LIST information and allow for multiple tags, e.g.

       S: * LIST (\HasChildren) "/" "Meat" ((ACL user=Fred rwipslextda)(X-MAILBOX-TYPE regular)

>  I think I should say something in the text about what
> that's for, and I'm open to suggestions about what to say and where to
> put it.  Larry, have you got some comments, since you were the one who
> suggested this addition?
>
> Cyrus, Alexey, what more need I add to support using this for the
> things we discussed in Atlanta?
>
> Barry

Alexey

Date: Tue, 21 Jan 2003 15:09:38 -0500
To: Barry Leiba <[email-masked]>, IMAP extensions Mailing List <[email-masked]>
From: Pete Maclean <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

>OK, how about this?:
>
>--------
>IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we have 
>added extensions that have require specialized lists (see [MboxRefer] for 
>an example) we have expand the number of list commands, since each 
>extension must add its function to both LIST and LSUB, and these commands 
>are not, as they are defined, extensible.  This document describes 
>extensions to the LIST command that allow these additions to be done with 
>mutually compatible options to the LIST command, avoiding the exponential 
>increase in specialized list commands.

Ah, now it is making sense to me.  Thanks.  I see that a large part of my 
puzzlement stemmed from my interpretation of "since each extension must add 
its function to both LIST and LSUB".  I was understanding that as meaning 
adding capability into LIST and LSUB which is exactly what the new proposal 
does and what the old extensions did not.  But you mean supplementing the 
functionality of LIST and LSUB, right?

May I suggest a couple of improvements along the lines of:

IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we have 
added extensions that required specialized lists (see [MboxRefer] for an 
example), we have done so by defining new list commands, for example RLIST 
and RLSUB.  Each such extension has served to supplement  LIST and LSUB 
since these commands, as currently defined, are not intrinsically 
extensible.  This document describes an extension to the LIST command that 
allows such additions to be made with mutually compatible options thereby 
avoiding exponential increase in the number of specialized list commands.

Note that I have changed "extensions" to "extension" in the last 
sentence.  This may well be debatable but I see the addition of a new 
argument as essentially a single change.

Pete

Date: Tue, 21 Jan 2003 11:48:53 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: base spec clarification of UID command responses
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 21 Jan 2003, Timo Sirainen wrote:
> How does this affect FETCH replies sent by server when it notices
> external flag changes?

Since this case is truly unsolicited, I don't think that the UID command
requirement applies in this case.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: base spec clarification of UID command responses
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 21 Jan 2003 21:41:12 +0200
Sender: [email-masked]

On Wed, 2002-11-27 at 00:39, Cyrus Daboo wrote:
> I interpreted this to mean that any UID command, be it FETCH, STORE, COPY 
> or SEARCH, has to always include UID items in any FETCH responses. However, 
> I have come across at least one server that does not return the UID item in 
> FETCH responses when doing a UID STORE. I'm concerned that the above 
> paragraph might have been interpreted as meaning that the UID item is only 
> required when UID FETCH is used, as opposed to the others.

Sorry for replying a bit late, I only now began to read this list's
archives.

How does this affect FETCH replies sent by server when it notices
external flag changes? This could happen from client's point of view:

C: x UID FETCH 1 (FLAGS)
S: * 1 FETCH (FLAGS ())
S: * 1 FETCH (FLAGS () UID 1)

If the first FETCH was sent before seeing the UID FETCH command.

Date: Tue, 21 Jan 2003 12:58:46 -0500
From: Barry Leiba <[email-masked]>
To: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

>  >> "As we add extensions that require specialized lists...we expand
> the number of  >> list commands..."
>
> This puzzles me because the draft is explicitly about reducing the
> number of list commands (e.g. obsoleting RLIST and RLSUB), not
> expanding it.  Do you mean, "As we have added extensions that
> required specialized lists we have, in the past, expanded the number
> of list commands..."?

I could change it to use a past tense, yes.  It's in the "abstract" 
section; it's explaining why we're making this extension.

>  >>"as each extension must add its function to both LIST and LSUB."
>
> However, I can say that I do not even understand in what sense the
> word "as" is being used.

>From American Heritage Dictionary:
  as ... (conj.) ... 4. For the reason that; because.

OK, how about this?:

--------
IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we 
have added extensions that have require specialized lists (see 
[MboxRefer] for an example) we have expand the number of list commands, 
since each extension must add its function to both LIST and LSUB, and 
these commands are not, as they are defined, extensible.  This document 
describes extensions to the LIST command that allow these additions to 
be done with mutually compatible options to the LIST command, avoiding 
the exponential increase in specialized list commands.
--------

Barry

--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

Date: Tue, 21 Jan 2003 12:10:59 -0500
To: "Barry Leiba" <[email-masked]>, "IMAP extensions Mailing List" <[email-masked]>
From: Pete Maclean <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

> >  >>As we add extensions that require specialized lists (see [MboxRefer] 
> for an
> >  >>example) we expand the number of list commands, as each extension
> >  >>must add its function to both LIST and LSUB.
> >
> > Please excuse my saying so, but this sentence strikes me as barely
> > comprehensible.  The sentence that follows it could be better worded as 
> well.
>
>Hm.  It seems perfectly normal English to me, so I don't know how I'd re-word
>it.  Maybe if you explain why you think it's barely comprehensible...?

I'll try but it's real hard to explain why something fails to make 
sense.  By the way, I have read the sentence over and over while writing 
this response and still cannot draw any useful meaning from it.

 >> "As we add extensions that require specialized lists...we expand the 
number of
 >> list commands..."

This puzzles me because the draft is explicitly about reducing the number 
of list commands (e.g. obsoleting RLIST and RLSUB), not expanding it.  Do 
you mean, "As we have added extensions that required specialized lists we 
have, in the past, expanded the number of list commands..."?

 >>"as each extension must add its function to both LIST and LSUB."

I cannot say much about this since, without a clear understanding of the 
first part of the sentence, I have no definite reference for it.  However, 
I can say that I do not even understand in what sense the word "as" is 
being used.

I truly apologize for making such a fuss about one little sentence.  Please 
be assured that I am not trying to take issue in any way with your use of 
English but merely trying to understand something that I find baffling.

Pete Maclean

Date: Tue, 21 Jan 2003 10:11:45 -0500 (EST)
From: Rob Siemborski <[email-masked]>
To: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

On Tue, 21 Jan 2003, Barry Leiba wrote:

> > Why is there not an attribute flag to identify remote mailboxes as such?
>
> There might be.  That oughta be up to "son of RFC2193".  Or should it?  Should
> we define a "\Remote" or "\Referral" flag here?  I have a vague memory that we
> discussed this before and said "no".

It is possible to have an implementation that will not *always* give a
referral for a mailbox (for example, it might give a referral for a
SELECT, but not for a RENAME), so I'm not sure how these flags would be
used in such an environment.

Additionally, does knowing that a mailbox will receive a referral without
knowing where that referral will be to really help clients much?  I'm
tempted to say no, but perhaps a client author would disagree.

I'd rather just delay the referral decision until something is actually
done to the mailbox.

> > And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do
> > remote mailboxes have to be top-level mailboxes?  Thanks.
>
> As Rob said, no, and the answer to your amended question is also no -- the
> referral "tree" does not have to be anchored at the top level.  I think the
> existing implementations do that, but it's not required.

Cyrus definitely does not anchor referrals at the top level.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

From: "Barry Leiba" <[email-masked]>
To: "IMAP extensions Mailing List" <[email-masked]>
Subject: Re: LIST Extensions draft
Date: Tue, 21 Jan 2003 09:00:08 -0500
Sender: [email-masked]

>  >>As we add extensions that require specialized lists (see [MboxRefer] for an
>  >>example) we expand the number of list commands, as each extension
>  >>must add its function to both LIST and LSUB.
> 
> Please excuse my saying so, but this sentence strikes me as barely 
> comprehensible.  The sentence that follows it could be better worded as well.

Hm.  It seems perfectly normal English to me, so I don't know how I'd re-word
it.  Maybe if you explain why you think it's barely comprehensible...?

> It is not made clear whether or not, when responding to a LIST (SUBSCRIBED) 
> command, a server SHOULD include all appropriate mailboxes that would have 
> the "\PlaceHolder" attribute.

I don't think this has anything to do with (SUBSCRIBED) in particular.  In
general, I think the server MUST include a mailbox with the \PlaceHolder flag
whenever (1) that mailbox does not match the selection criteria AND (2) there
is a child mailbox that (a) does match the selection criteria that is NOT listed
or (b) merits the \PlaceHolder flag by this explanation.

I guess I should expand on the explanation of \PlaceHolder, to make this
clearer.

> Why is there not an attribute flag to identify remote mailboxes as such?

There might be.  That oughta be up to "son of RFC2193".  Or should it?  Should
we define a "\Remote" or "\Referral" flag here?  I have a vague memory that we
discussed this before and said "no".

> And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do 
> remote mailboxes have to be top-level mailboxes?  Thanks.

As Rob said, no, and the answer to your amended question is also no -- the
referral "tree" does not have to be anchored at the top level.  I think the
existing implementations do that, but it's not required.

Barry
--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

Date: Mon, 20 Jan 2003 16:49:55 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Barry Leiba <[email-masked]>
CC: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

Barry Leiba wrote:
> 
> Here's a draft (ahem) copy of the next LIST Extensions draft.  Please
> give it a look and give comments before I send it in.  Notably added is

Minor nit.  The references to the original CHILDREN extension should be
changed to point to RFC 3328.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Mon, 20 Jan 2003 16:04:26 -0500 (EST)
From: Rob Siemborski <[email-masked]>
To: Pete Maclean <[email-masked]>
cc: Barry Leiba <[email-masked]>, IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

On Mon, 20 Jan 2003, Pete Maclean wrote:

> And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do
> remote mailboxes have to be top-level mailboxes?  Thanks.

No, they do not.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Mon, 20 Jan 2003 15:50:42 -0500
To: Barry Leiba <[email-masked]>, IMAP extensions Mailing  List <[email-masked]>
From: Pete Maclean <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

I applaud this proposed extension and have a few comments:

 >>As we add extensions that require specialized lists (see [MboxRefer] for an
 >>example) we expand the number of list commands, as each extension
 >>must add its function to both LIST and LSUB.

Please excuse my saying so, but this sentence strikes me as barely 
comprehensible.  The sentence that follows it could be better worded as well.

It is not made clear whether or not, when responding to a LIST (SUBSCRIBED) 
command, a server SHOULD include all appropriate mailboxes that would have 
the "\PlaceHolder" attribute.

Why is there not an attribute flag to identify remote mailboxes as such?

And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do 
remote mailboxes have to be top-level mailboxes?  Thanks.

Pete Maclean

Date: Mon, 20 Jan 2003 14:13:38 -0500
From: Barry Leiba <[email-masked]>
To: IMAP extensions Mailing  List <[email-masked]>
Subject: LIST Extensions draft
Sender: [email-masked]

--==========514551984==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Here's a draft (ahem) copy of the next LIST Extensions draft.  Please 
give it a look and give comments before I send it in.  Notably added is 
the "mbox-list-extension" item in the formal grammar, and the 
associated bits.  I think I should say something in the text about what 
that's for, and I'm open to suggestions about what to say and where to 
put it.  Larry, have you got some comments, since you were the one who 
suggested this addition?

Cyrus, Alexey, what more need I add to support using this for the 
things we discussed in Atlanta?

Barry

--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

--==========514551984==========
Content-Type: text/plain; charset=iso-8859-1;
 name="draft-ietf-imapext-list-extensions-02.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="draft-ietf-imapext-list-extensions-02.txt"; size=18739

IMAP Extensions Working Group                                   B. =
Leiba
Internet Draft                           IBM T.J. Watson Research =
Center
Document: draft-ietf-imapext-list-extensions-02.txt         January =
2003
                                                       Expires July =
2003
                     IMAP4 LIST Command Extensions
Status of this Document
   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.  Internet-Drafts are =
working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.
   Internet-Drafts are draft documents valid for a maximum of six =
months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."
     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt
     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.
   A revised version of this draft document will be submitted to the =
RFC
   editor as an Proposed Standard for the Internet Community.
   Discussion and suggestions for improvement are requested, and should
   be sent to [email-masked].  This document will expire before =
31
   July 2003.  Distribution of this memo is unlimited.
1. Abstract
   IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we
   add extensions that require specialized lists (see [MboxRefer] for =
an
   example) we expand the number of list commands, as each extension
   must add its function to both LIST and LSUB.  This document =
describes
   extensions to the LIST command that allow these additions to be done
   in mutually compatible options to the LIST command, avoiding the
   exponential increase in specialized list commands.
B. Leiba                   Expires July 2003                   [Page =
1]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
2. Conventions used in this document
   In examples, "C:" indicates lines sent by a client that is connected
   to a server.  "S:" indicates lines sent by the server to the client.
   The words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" are
   used in this document as specified in RFC 2119 [Keywords].
3. Introduction
   The extensions to the LIST command will be accomplished by amending
   the syntax to allow options to be specified.  The list of options
   will replace the several commands that are currently used to mix and
   match the information requested.  The new syntax is backward-
   compatible, with no ambiguity: if the first word after the command
   name begins with a parenthesis, the new syntax is being used; if it
   does not, it's in the original syntax.
   By adding options to the LIST command, we are announcing the intent
   to phase out and eventually to deprecate the RLIST and RLSUB =
commands
   described in [MboxRefer].  We are also defining the mechanism to
   request extended mailbox information, such as is described in the
   now-expired "Child Mailbox Extension" draft [ChildMbox].  The base
   LSUB command is not deprecated by this extension; rather, this
   extension adds a way to obtain subscription information with more
   options, with those server implementations that support it.  Clients
   that simply need a list of subscribed mailboxes, as provided by the
   LSUB command, SHOULD continue to use that command.
4. LIST Command Options
   The LIST command syntax is extended by adding a parenthesized list =
of
   command options between the command name and the reference name (see
   the formal syntax in section 6 for specific details).  Command
   options will be defined in this document and in approved extension
   documents; each option will be enabled by a capability string (one
   capability may enable multiple options), and a client MUST NOT send
   an option for which the server has not advertised support.  A server
   MAY ignore options it does not recognize.
   This extension is identified by the capability string "LISTEXT", and
   support for it is a prerequisite for any future extensions that
   require specialized forms of the LIST command.  Such extensions MUST
   refer to this document and MUST add their function through command
   options as described herein.  This document also defines the "LIST-
   SUBSCRIBED" capability string; see the "SUBSCRIBED" option below.
B. Leiba                   Expires July 2003                   [Page =
2]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   The options defined in this specification are
       SUBSCRIBED - causes the LIST command to list subscribed
          mailboxes, rather than the actual mailboxes.  This will often
          be a subset of the actual mailboxes.  It's also possible for
          this list to contain the names of mailboxes that don't exist.
          In any case, the list MUST include exactly those mailbox =
names
          that match the selection criteria and are subscribed to.  =
This
          option is intended to supplement the LSUB command, and =
support
          for it is optional -- a server that supports the SUBSCRIBED
          option indicates so through the LIST-SUBSCRIBED capability.
          Of particular note are the mailbox flags as returned by this
          option, compared with what is returned by LSUB.  With the
          latter, the flags returned may not reflect the actual flag
          status on the mailbox, and the \NoSelect flag has a special
          meaning (it indicates that this mailbox is not, itself,
          subscribed, but that it has child mailboxes that are).  With
          the SUBSCRIBED option described here, the flags are accurate
          and complete, and have no special meanings.
          "LSUB" and "LIST (SUBSCRIBED)" are, thus, not the same thing,
          and some servers must do significant extra work to respond to
          "LIST (SUBSCRIBED)".  Because of this, clients SHOULD =
continue
          to use "LSUB" unless they specifically want the additional
          information offered by "LIST (SUBSCRIBED)".  At the same =
time,
          servers SHOULD support the LIST-SUBSCRIBED capability even if
          it entails extra work, because a client that wants the
          information will still obtain it by using LSUB followed by a
          series of LIST commands, so servers might as well make it
          easier.
          This option defines a new mailbox flag, "\NonExistent", that
          indicates that a mailbox is subscribed to, but does not
          actually exist.  The "\NonExistent" flag MUST be supported =
and
          MUST be accurately computed.
       REMOTE - causes the LIST command to show remote mailboxes as
          well as local ones, as described in [MboxRefer].  This option
          is intended to replace the RLIST command and, in conjunction
          with the SUBSCRIBED option, the RLSUB command.  This option =
is
          only available on servers that also support RFC-2193.
       CHILDREN - Requests mailbox child information as originally
          proposed in [ChildMbox].  See section 5, below, for details.
          Support for this is optional, but this option MUST be =
accepted
          by all servers (though it MAY be ignored).
   The LISTEXT capability also defines a new mailbox flag,
B. Leiba                   Expires July 2003                   [Page =
3]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   "\PlaceHolder", that indicates that the designated mailbox does not
   meet the selection criteria of the given LIST command, but that it
   has one or more child mailboxes that do [EDITORIAL NOTE: "might"?].
   The LSUB command indicates this condition by using the "\NoSelect"
   flag, but the LIST (SUBSCRIBED) command MUST NOT do that, since
   "\NoSelect" retains its original meaning here.  Further, the
   "\PlaceHolder" flag is more general, in that it can be used with any
   extended set of selection criteria.
5. The CHILDREN Option
   The CHILDREN option implements the Child Mailbox Extension,
   originally proposed by Mike Gahrns and Raymond Cheng, of Microsoft
   Corporation.  Most of the information in this section is taken
   directly from their original specification [ChildMbox].  The =
CHILDREN
   option is simply an indication that the client wants this
   information; a server MAY provide it even if the option is not
   specified, or MAY ignore the option entirely.
   Many IMAP4 [IMAP4] clients present to the user a hierarchical view =
of
   the mailboxes that a user has access to.  Rather than initially
   presenting to the user the entire mailbox hierarchy, it is often
   preferable to show to the user a collapsed outline list of the
   mailbox hierarchy (particularly if there is a large number of
   mailboxes).  The user can then expand the collapsed outline =
hierarchy
   as needed.  It is common to include within the collapsed hierarchy a
   visual clue (such as a ''+'') to indicate that there are child
   mailboxes under a particular mailbox.   When the visual clue is
   clicked the hierarchy list is expanded to show the child mailboxes.
   The Child Mailbox Extension provides a mechanism for a client to
   efficiently determine if a particular mailbox has children, without
   issuing a LIST '' * or a LIST '' % for each mailbox name.
   The Child Mailbox Extension defines two new attributes that MAY be
   returned within a LIST response: \HasChildren and \HasNoChildren.
   While these attributes MAY be returned in response to any LIST
   command, the CHILDREN option is provided to indicate that the client
   particularly wants this information.  If the CHILDREN option is
   present, the server SHOULD return these attributes even if their
   computation is expensive.
\HasChildren - The presence of this attribute indicates that the =
mailbox
       has child mailboxes.
       A server SHOULD NOT set this attribute if there are child
       mailboxes, and the user does not have permissions to access any
       of them. In this case, \HasNoChildren SHOULD be used.
B. Leiba                   Expires July 2003                   [Page =
4]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
       In many cases, however, a server may not be able to efficiently
       compute whether a user has access to all child mailboxes.  As
       such a client MUST be prepared to accept the \HasChildren
       attribute as a hint.  That is, a mailbox MAY be flagged with the
       \HasChildren attribute, but no child mailboxes will appear in
       the LIST response.
\HasNoChildren - The presence of this attribute indicates that the
       mailbox has NO child mailboxes that are accessible to the
       currently authenticated user.
   In some instances a server that supports the Child Mailbox Extension
   might not be able to determine whether a mailbox has children.  For
   example it may have difficulty determining whether there are child
   mailboxes when LISTing mailboxes while operating in a particular
   namespace.
   In these cases, a server MAY exclude both the \HasChildren and
   \HasNoChildren attributes in the LIST response.  As such, a client
   can not make any assumptions about whether a mailbox has children
   based upon the absence of a single attribute.  In particular, some
   servers may not be able to combine the SUBSCRIBED and CHILDREN
   options.  Such servers MUST honour the SUBSCRIBED option, and they
   will simply ignore the CHILDREN option if both are requested.
   It is an error for the server to return both a \HasChildren and a
   \HasNoChildren attribute in a LIST response.
   Note: the \HasNoChildren attribute should not be confused with the
   IMAP4 [IMAP4] defined attribute \NoInferiors which indicates that no
   child mailboxes exist now and none can be created in the future.
6. Examples
   The first example shows the complete local hierarchy that will be
   used for the other examples.
       C: A01 LIST "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST () "/" "Fruit"
       S: * LIST () "/" "Fruit/Apple"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST () "/" "Tofu"
       S: * LIST () "/" "Vegetable"
       S: * LIST () "/" "Vegetable/Broccoli"
       S: A01 OK done
B. Leiba                   Expires July 2003                   [Page =
5]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   In the next example, we'll see the subscribed mailboxes.  This is
   similar, but not equivalent, to <LSUB "" "*">.  Note that the =
mailbox
   called "Fruit/Peach" is subscribed to, but does not actually exist
   (perhaps it was deleted while still subscribed).  And the "Fruit"
   mailbox is not subscribed to, but it has two subscribed children.
       C: A02 LIST (SUBSCRIBED) "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\PlaceHolder) "/" "Fruit"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST (\NonExistent) "/" "Fruit/Peach"
       S: A02 OK done
   The next example shows the use of the CHILDREN option.  The client,
   without having to list the second level of hierarchy, now knows =
which
   of the top-level mailboxes have sub-mailboxes (children) and which =
do
   not.  Note that it's not necessary for the server to return the
   \HasNoChildren flag for the inbox, because the \NoInferiors flag
   already implies that, and has a stronger meaning.
       C: A03 LIST (CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: A03 OK done
   In this example we see more mailboxes, which reside on another =
server
   to which we may obtain referrals.  This is similar to the command
   <RLIST "" "%">.  We also see the mixing of two options.  Note that =
in
   the case of the remote mailboxes, the server might or might not be
   able to include CHILDREN information; it includes it if it can, and
   omits it if it can't.
       C: A04 LIST (REMOTE CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: * LIST () "/" "Bread"
       S: * LIST (\HasChildren) "/" "Meat"
       S: A04 OK done
7. Formal Syntax
   The following syntax specification uses the augmented Backus-Naur
   Form (BNF) as described in [ABNF].  Terms not defined here are taken
   from [IMAP4].
B. Leiba                   Expires July 2003                   [Page =
6]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   child-mbox-flag =3D "\HasChildren" / "\HasNoChildren"
                      ; flags for Child Mailbox Extension, at most one
                      ; possible per LIST response
   list            =3D "LIST" [SP list-options] SP mailbox SP =
list-mailbox
   list-options    =3D "(" [option *(SP option)] ")"
   mailbox-list    =3D "(" [mbx-list-flags] ")" SP
                      (DQUOTE QUOTED-CHAR DQUOTE / nil) SP mailbox
                      [SP "(" [mbox-list-extended] ")"]
   mbox-list-extended =3D mbox-list-extended-item
                        *(SP mbox-list-extended-item)
   mbox-list-extended-item =3D string / mbox-list-extended
   mbox-list-oflag :=3D child-mbox-flag / "\NonExistent" / =
"\PlaceHolder"
   option           =3D "SUBSCRIBED" / "CHILDREN" / "REMOTE" /
                     option-extension
   option-extension =3D atom
8. Security Considerations
   This document describes syntactic changes to the specification of =
the
   IMAP4 commands LIST, LSUB, RLIST, and RLSUB, and the modified LIST
   command has the same security considerations as those commands.  =
They
   are described in [IMAP4] and [MboxRefer].
   The Child Mailbox Extension provides a client a more efficient means
   of determining whether a particular mailbox has children.  If a
   mailbox has children, but the currently authenticated user does not
   have access to any of them, the server SHOULD respond with a
   \HasNoChildren attribute.  In many cases, however, a server may not
   be able to efficiently compute whether a user has access to all =
child
   mailboxes.  If such a server responds with a \HasChildren attribute,
   when in fact the currently authenticated user does not have access =
to
   any child mailboxes, potentially more information is conveyed about
   the mailbox than intended.   In most situations this will not be a
   security concern, because if information regarding whether a mailbox
   has children is considered sensitive, a user would not be granted
   access to that mailbox in the first place.
B. Leiba                   Expires July 2003                   [Page =
7]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
9. References
   [IMAP4]; Crispin, M.; "Internet Message Access Protocol - Version
   4rev1"; RFC 2060; University of Washington; December 1996.
   [MboxRefer]; Gahrns, M.; "IMAP4 Mailbox Referrals"; RFC 2193;
   Microsoft Corporation; September 1997.
   [ChildMbox]; Gahrns, M. & Cheng, R.; "IMAP4 Child Mailbox =
Extension";
   draft-gahrns-imap-child-mailbox-03.txt; Microsoft Corporation;
   November 1997 -- expired, listed for historical reference.
   [Keywords]; Bradner, S.; "Key words for use in RFCs to Indicate
   Requirement Levels"; RFC 2119; Harvard University; March 1997.
   [ABNF]; Crocker, D., and Overell, P. "Augmented BNF for Syntax
   Specifications: ABNF", RFC 2234, November 1997.
10. Acknowledgements
   Mike Gahrns and Raymond Cheng of Microsoft Corporation originally
   devised the Child Mailbox Extension and proposed it in 1997; the
   idea, as well as most of the text in section 5, is theirs.
11. Author's Address
   Barry Leiba
   IBM T.J. Watson Research Center
   30 Saw Mill River Road
   Hawthorne, NY  10532
   Phone: 1-914-784-7941
   Email: [email-masked]
B. Leiba                   Expires July 2003                   [Page 8]
--==========514551984==========--

Date: Fri, 17 Jan 2003 08:35:17 -0500
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: [off charter] imap extension for moderation?
Reply-To: [email-masked]
Sender: [email-masked]

I don't see this in the charter, but perhaps the list will permit me to ask a 
simple question - has anyone ever attempted to extend IMAP for the purpose of
list moderation?

The idea I have is that messages to be moderated would end up in an IMAP
folder, and that they could be flagged by a MUA with /Approved or /Rejected or
deleted to cause them to be forwarded to the list or bounced or discarded.  It
would also be useful to be able to specify a reason for rejecting a message.

It just seems like a MUA is a much better tool for reviewing messages than,
say, a web browser.

Since this is off-topic, please send replies to me personally rather than to
the list.

thanks,

Keith

From: Arnt Gulbrandsen <[email-masked]>
To: Ken Murchison <[email-masked]>
Subject: Re: Thread extension weirdness
Cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Sun, 12 Jan 2003 17:36:13 +0100
Sender: [email-masked]

Ken Murchison writes:
> So there is no interest in revisiting adding algorithm options to THREAD?

The possible semantics/implementation complexity sounds a little scary 
to this small boy. That complexity isn't, from what I can tell, 
justified.

--Arnt

Date: Sat, 11 Jan 2003 18:04:43 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Sat, 11 Jan 2003, Lawrence Greenfield wrote:
> I'm not interested in breaking THREAD=REFERENCES. I'm merely saying that if
> the current THREAD=REFERENCES is broken, we shouldn't publish it as a
> standards-track RFC.

I claim that the current THREAD=REFERENCES is not broken, and should be
published as a standards-track RFC.  That does not preclude other
threading algorithms, as anticipated by the THREAD specification, will
happen.

All threading is going to be a compromise.  For better or worse, this is
the compromise that we have, and it generally works quite well.  An
attempt to push some other compromise in lieu of the one that is already
wide will not make the situation better.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sat, 11 Jan 2003 20:54:24 -0500
From: Lawrence Greenfield <[email-masked]>
To: Mark Crispin <[email-masked]>, Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

--On Saturday, January 11, 2003 10:41 AM -0800 Mark Crispin 
<[email-masked]> wrote:

> On Sat, 11 Jan 2003 13:39:16 -0500, Cyrus Daboo wrote:
>> [proposed improved algorithm]
>> This should probably be done as a
>> separate THREAD=XXX option.
>
> To make things unambiguous:
>
> I strongly support this.  My opposition is solely to the idea of breaking
> the existing THREAD=REFERENCES algorithm.

I'm not interested in breaking THREAD=REFERENCES. I'm merely saying that if 
the current THREAD=REFERENCES is broken, we shouldn't publish it as a 
standards-track RFC. Current servers don't have to drop support, but 
there's no reason to publish both THREAD=REFERENCES and the fixed 
THREAD=XXX.

I guess I agree with Mark that there's probably no need for a seperate spec 
that returns only threads with unseen messages. (The underlying question is 
whether THREAD or SORT's O(n) size responses is acceptable. I'd rather not 
reopen that question right now.)

Larry

Date: Sat, 11 Jan 2003 13:54:48 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Cyrus Daboo <[email-masked]>
cc: Ken Murchison <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Sat, 11 Jan 2003, Cyrus Daboo wrote:
> I think there is a demonstrable need for at least one option - namely the
> 'return only threads containing unseen messages' one. That could actually
> be generalized to take a list of flags or even further to take another set
> of full search criteria so you could select threads by From address etc.
> Beyond that I can't think of anything else one might want to do...

You can accomplish the same thing by doing a SEARCH command after the
THREAD command.  It's a little bit more work, a little bit more network
I/O.  But it's still much less costly than doing a client-based thread.

Thus, you do not "need" this option, even if it is "nice to have."

As with all "nice to have" options, the question then goes to how widely
deployed and used this would likely be.

Mind you; I am not opposed to adding such a capability.  I see no harm in
doing so other than added complexity.  I'm just insisting that it make a
better case for itself.

Most IMAP clients -- Pine and Mulberry being notable exceptions -- treat
IMAP as a glorified POP3.  It is difficult to see how adding complexity to
IMAP is going to help when so many IMAP clients fail to use what is there
now.

> PS The 'search-charset' formal syntax item in the current draft is not
> defined in the draft and does not appear in any of the base-spec documents
> either. From the base-spec it ought to be an astring.

Thanks, I will fix that.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sat, 11 Jan 2003 16:05:15 -0500
From: Cyrus Daboo <[email-masked]>
To: Ken Murchison <[email-masked]>, Mark Crispin <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Hi Ken,

--On Saturday, January 11, 2003 3:00 PM -0500 Ken Murchison 
<[email-masked]> wrote:

|> I strongly support this.  My opposition is solely to the idea of
|> breaking the existing THREAD=REFERENCES algorithm.
|
| So there is no interest in revisiting adding algorithm options to
| THREAD?

I think there is a demonstrable need for at least one option - namely the 
'return only threads containing unseen messages' one. That could actually 
be generalized to take a list of flags or even further to take another set 
of full search criteria so you could select threads by From address etc. 
Beyond that I can't think of anything else one might want to do...

If we want something like the thread selection capability then it should be 
done as an option so that it be applied to different algorithms, without 
the need to spawn even more algorithm names. How about having a 
THREAD-OPTION capability to list those, and allowing something like:

* CAPABILITY THREAD=REFERENCES THREAD=REFERENCES2 THREAD-OPTIONS=UNSEEN 
THREAD-OPTIONS=SEARCH

a THREAD REFERENCES (UNSEEN) UTF8 ALL

or

a THREAD REFERENCES2 (SEARCH FROM [email-masked]) UTF8 ALL

The THREAD-OPTIONS scheme could be added to the current draft without 
affecting the installed base. Later documents could then add the actual 
options that one may want.

PS The 'search-charset' formal syntax item in the current draft is not 
defined in the draft and does not appear in any of the base-spec documents 
either. From the base-spec it ought to be an astring.

-- 
Cyrus Daboo

Date: Sat, 11 Jan 2003 15:00:51 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Sat, 11 Jan 2003 13:39:16 -0500, Cyrus Daboo wrote:
> > [proposed improved algorithm]
> > This should probably be done as a
> > separate THREAD=XXX option.
> 
> To make things unambiguous:
> 
> I strongly support this.  My opposition is solely to the idea of breaking the
> existing THREAD=REFERENCES algorithm.

So there is no interest in revisiting adding algorithm options to
THREAD?

If not, I'm fine with a THREAD=REFERENCES2 or whatever for Cyrus'
tweak.  I too have been annoyed by what Cyrus described.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Sat, 11 Jan 2003 10:41:49 -0800 (PST)
From: Mark Crispin <[email-masked]>
Subject: Re: Thread extension weirdness
To: Cyrus Daboo <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Sender: [email-masked]

On Sat, 11 Jan 2003 13:39:16 -0500, Cyrus Daboo wrote:
> [proposed improved algorithm]
> This should probably be done as a
> separate THREAD=XXX option.

To make things unambiguous:

I strongly support this.  My opposition is solely to the idea of breaking the
existing THREAD=REFERENCES algorithm.

And, to address the original issue, I rewrote UW imapd's subject extraction
rules yesterday to strictly conform with the draft.  The new code will be in
all versions after Pine 4.52 and imap-2002b (which were released on Thursday).
I also discovered, after trying numerous mailboxes and newsgroups, that the
only way that I could get the new code and the old code to thread differently
was with a contrived example.  "re"s do not occur inside quoted-words in any
example in the wild that I could discover.

Date: Sat, 11 Jan 2003 13:39:16 -0500
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
cc: IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Hi Mark,

--On Friday, January 10, 2003 8:38 PM -0800 Mark Crispin 
<[email-masked]> wrote:

|> Are you saying that there are mailboxes that will not be correctly
|> threaded in this case?
|>
|> Or do you not want to get rid of THREAD=REFERENCES because of widespread
|> deployment?
|
| Yes to both of the above.

I think what we need here is a compromise between the two extremes that we 
have: grouping by subject and only use 2822 headers. The current algorithm 
breaks when you have two distinct threads that have the same subject. I 
often get messages from colleagues that just say 'Update' in the subject, 
or online order confirmations that just say 'Your Order Confirmation' (no 
order number). These currently all get grouped together and this is the 
root of my real annoyance, because a message I got yesterday could be 
incorrectly grouped with one sent a year ago. I've had complaints from 
users about this in the sixteen months we've had thread support deployed.

Having said that, the current algorithm does work reasonably well when 
there is a thread with a unique subject that is broken (2822-wise) because 
there are still many deployed clients that don't do 2822 references headers 
properly, and it would be nice to preserve the subject grouping in those 
cases.

What I propose is to tweak the current algorithm to fix the first case 
whilst still allowing the second case to work. This could be a simple 
matter of spotting the fact that there are two distinct 'root' messages 
(i.e. messages that have the same 'original' as opposed to 'extracted' 
subject that does not contain any Re/fwd/[...] etc artifacts). Those root 
messages should not be grouped together. Then it is a matter of deciding 
how to group replies that do not have proper 2822 references headers to 
those roots in such a way that the proper replies go with the appropriate 
root. The simplest solution is to apply a grouping of replies based on date 
sent. i.e. all replies sent before the second root message are grouped with 
the first, and all replies sent after the second root message, are grouped 
with the second root. This could mean that legitimate replies to the first 
root end up under the second one, but I don't believe this is any worse 
than the current position where the roots and all replies get grouped 
together. Note that replies that have the proper 2822 headers will always 
appear under the correct root.

This change would require significant reworking of step (5) in the current 
algorithm and would lead to more complexity there, but I think it is worth 
it to provide a reasonable compromise. This should probably be done as a 
separate THREAD=XXX option.

-- 
Cyrus Daboo

Date: Fri, 10 Jan 2003 17:01:30 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: Arnt Gulbrandsen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Interest List <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Fri, 10 Jan 2003 13:12:32 -0500, Ken Murchison wrote:
> > Well, I don't think we can change the current name, because there is
> > already some deployment.
> 
> We can not change the name.
> 
> > We _could_ add a THREAD=REFERENCES-ONLY or
> > THREAD=REFERENCES-NOSUBJ or ...
> 
> Fine.  The THREAD specification explicitly allows adding algorithms.

Right, but the question is do we want to add these variants as separate
algorithms, or do we want to revisit Cyrus' optional argument syntax?

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Fri, 10 Jan 2003 10:15:41 -0800 (PST)
From: Mark Crispin <[email-masked]>
Subject: Re: Thread extension weirdness
To: Ken Murchison <[email-masked]>
cc: Arnt Gulbrandsen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Interest List <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Fri, 10 Jan 2003 13:12:32 -0500, Ken Murchison wrote:
> Well, I don't think we can change the current name, because there is
> already some deployment.

We can not change the name.

> We _could_ add a THREAD=REFERENCES-ONLY or
> THREAD=REFERENCES-NOSUBJ or ...

Fine.  The THREAD specification explicitly allows adding algorithms.

However, as a cautionary note: keep in mind that each additional algorithm
ratches up the level of complexity.

Date: Fri, 10 Jan 2003 13:12:32 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Arnt Gulbrandsen <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, IMAP Interest List <[email-masked]>, Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> 
> Cyrus Daboo writes:
> > However, I will again voice my annoyance with the fact that the
> > THREAD=REFERENCES extension does any kind of grouping based on
> > subject.
> 
> Seconded!
> 
> THREAD=REFERENCES is its name, so why can't it THREAD based on
> REFERENCES? The current draft's algorithm would be better named e.g.
> THREAD=COMPREHENSIVE.

Well, I don't think we can change the current name, because there is
already some deployment.  We _could_ add a THREAD=REFERENCES-ONLY or
THREAD=REFERENCES-NOSUBJ or ...  I also want to see a REFERENCES-UNSEEN
which only shows threads which contain unread messages (note that this
_isn't_ the same as threading unread messages).

This goes back to a discussion that Cyrus, Mark and myself had
previously about either documenting additional REFERENCES-based
algorithms or adding parameters to THREAD, ie:

THREAD REFERENCES (UNSEEN NOSUBJ) ...

I know that Mark wasn't in favor of doing either of these in the current
draft, but maybe its time to get another consensus now (especially if we
want to add an optional param list to THREAD).

Ken
-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

From: Arnt Gulbrandsen <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Subject: Re: Thread extension weirdness
Cc: Ken Murchison <[email-masked]>, IMAP Interest List <[email-masked]>, Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Fri, 10 Jan 2003 18:59:27 +0100
Sender: [email-masked]

Cyrus Daboo writes:
> However, I will again voice my annoyance with the fact that the 
> THREAD=REFERENCES extension does any kind of grouping based on 
> subject.

Seconded!

THREAD=REFERENCES is its name, so why can't it THREAD based on 
REFERENCES? The current draft's algorithm would be better named e.g. 
THREAD=COMPREHENSIVE.

--Arnt

Date: Fri, 10 Jan 2003 12:04:03 -0500
From: Cyrus Daboo <[email-masked]>
To: Ken Murchison <[email-masked]>, Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Hi Ken,

--On Friday, January 10, 2003 8:46 AM -0500 Ken Murchison <[email-masked]> 
wrote:

|> Question for the list: should it?
|
| FWIW, Cyrus does.  This is because the header is decoded for the SEARCH
| and then passed on to SORT/THREAD.  Interestingly, Netscape Messenger
| makes separate threads, even though the algorithms are supposed to be
| similar/same.
|
| I'd vote that the draft is fine, and that UW needs to be fixed (less
| work for me  ;)

FYI My client-side implementation of the THREAD algorithm also does it as 
per the current draft 'by default' since the subject is already MIME 
decoded and 2822 unfolded by the time it gets processed. I believe that is 
the right thing to do.

However, I will again voice my annoyance with the fact that the 
THREAD=REFERENCES extension does any kind of grouping based on subject. 
This is an attempt to fix the behaviour of broken (old) clients that do not 
generate proper In-Reply-To and References headers at the expense of 
legitimate clients that generate those headers in conformance to 2822.

-- 
Cyrus Daboo

Date: Fri, 10 Jan 2003 14:59:38 +0100
From: Sebastian Hagedorn <[email-masked]>
To: Ken Murchison <[email-masked]>, Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

--==========2151805189==========
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

--On Friday, January 10, 2003 08:46:46 -0500 Ken Murchison <[email-masked]> =

wrote:

>> Question for the list: should it?
>
> FWIW, Cyrus does.  This is because the header is decoded for the SEARCH
> and then passed on to SORT/THREAD.  Interestingly, Netscape Messenger
> makes separate threads, even though the algorithms are supposed to be
> similar/same.
>
> I'd vote that the draft is fine, and that UW needs to be fixed (less
> work for me  ;)

>From a user and admin perspective I agree ... it's the behavior I'd expect.
--
Sebastian Hagedorn M.A. - RZKR-R1 (Geb=E4ude 52), Zi. 18, Robert-Koch-Str. =
10
Zentrum f=FCr angewandte Informatik - Universit=E4tsweiter Service RRZK
Universit=E4t zu K=F6ln / Cologne University - Tel. +49-221-478-5587
--==========2151805189==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========2151805189==========--

Date: Fri, 10 Jan 2003 08:46:46 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Thu, 10 Jan 2003, Timo Sirainen wrote:
> > Another thing I saw with UW, it
> > didn't group these messages together:
> > Subject: =?iso-8859-1?Q?foo?=
> > Subject: =?iso-8859-1?Q?RE=3A_foo?=
> 
> Question for the list: should it?

FWIW, Cyrus does.  This is because the header is decoded for the SEARCH
and then passed on to SORT/THREAD.  Interestingly, Netscape Messenger
makes separate threads, even though the algorithms are supposed to be
similar/same.

I'd vote that the draft is fine, and that UW needs to be fixed (less
work for me  ;)

Ken

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Thu, 9 Jan 2003 18:34:49 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
cc: IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Thu, 10 Jan 2003, Timo Sirainen wrote:
> Another thing I saw with UW, it
> didn't group these messages together:
> Subject: =?iso-8859-1?Q?foo?=
> Subject: =?iso-8859-1?Q?RE=3A_foo?=

Question for the list: should it?

It turns out that UW imapd does something different from the base subject
extraction algorithm described in draft-ietf-imapext-sort-11.txt.  I'm not
sure if this is a bug in imapd or in draft-ietf-imapext-sort-11.txt, and I
am soliciting opinions on whether UW imapd should be changed to comply
with draft-ietf-imapext-sort-11, or if a revision of the draft should be
changed to comply with UW imapd.

UW imapd's algorithm is:

(1) Convert all continuations to spaces.

(2) Remove all leading WSP or subj-refwd.

(3) If the resulting text starts with a subj-blob, examine the text after
    the subj-blob.  Disregard any subj-trailer encountered.  If there is
    any text other than subj-trailer, remove the subj-blob and repeat (2)
    and (3) until no matches remain.

(4) Convert all MIME encoded-words to UTF-8.

(5) Remove all subj-trailer.

(6) same as in draft-ietf-imapext-sort-11.txt

(7) same as in draft-ietf-imapext-sort-11.txt

The significant difference between UW imapd and the draft is that in UW
imapd leading subj-leader is *not* considered if it is encapsulated in a
MIME encoded word.

It looks like the original draft agreed with UW imapd, and it got changed
in draft -03 or -04 to do the encoded-word conversion at the very start.
I forget why.  Does anyone remember?

If UW imapd is in the wrong, it will be a while before this gets fixed.
We just released Pine 4.52 and imap-2002b.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 31 Jan 2003 13:08:00 -0500
From: Cyrus Daboo <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Subject: I-D ACTION:draft-hazewinkel-imap-vhost-00.txt (fwd)
Sender: [email-masked]

------------ Forwarded Message -----------
Date: Friday, January 31, 2003 7:51 AM -0500
From: [email-masked]
To: IETF-Announce
Subject: I-D ACTION:draft-hazewinkel-imap-vhost-00.txt

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: IMAP Virtual Hosting
	Author(s)	: H. Hazewinkel
	Filename	: draft-hazewinkel-imap-vhost-00.txt
	Pages		: 6
	Date		: 2003-1-30
	
This memo describes an extension to the IMAP protocol in order to
enable virtual hosting. The concept of virtual hosting for HTTP is
used and a new IMAP command is introduced to provide the virtual
hosting feature.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-hazewinkel-imap-vhost-00.txt

---------- End Forwarded Message ----------

Just saw the above draft appear. I think this problem has already been 
solved for the most part by having domain information expressed in 
authentication ids. I did not understand the comment in the draft about 
'domain in auth-id' creating a 'single level list' and why that is a 
problem. Can you explain in more detail the benefits that VHOST offers over 
the 'domain in auth-id' approach? Its worth noting that 'domain in auth-id' 
works right now without requiring modifications to any clients, whereas 
VHOST would require client support.

-- 
Cyrus Daboo

From: "Barry Leiba" <[email-masked]>
To: "IMAP extensions Mailing List" <[email-masked]>
Subject: Re: RENAME, once more
Date: Tue, 28 Jan 2003 10:08:27 -0500
Sender: [email-masked]

> The new-uidvalidity tag would only indicate the case were the uidvalidity was
> changed (or perhaps not changed) and the uids were preserved.  It would not say
> anything about the other cases where the uids changed.

Yes, I think that's what Timo is proposing.

> If you give the client the new uidvalidity
> and old uids, when it doesn't get that information most likely the client being
> lazy will assume that neither changed.  This could lead to some nasty suprises
> and finger pointing when the validity didn't change but the uids changed

Why?  NEW-UIDVALIDITY doesn't exist now; Timo is defining it.  As such, Timo gets
to propose the definition as HE wants it, not as the proverbial lazy implementer
might want it.  I think he's proposing this:

1. If the server can do this reasonably (for some value of "reasonably" that's up
to the server to determine), it MAY leave the UIDs of the messages unchanged and
send back a NEW-UIDVALIDITY value in the response.

2. If the server chooses not to do (1) because it's unable to guarantee that the
UIDs have not changed, or because the process is, for whatever reason, not deemed
to be "reasonable", then it MUST NOT send back NEW-UIDVALIDITY.

3. In any case, the server MUST meet the current requirements for setting the
UIDVALIDITY value of the new mailbox.

4. If the client receives NEW-UIDVALIDITY, it MAY use that information to keep
its cache valid, without having to refresh it from the newly renamed mailbox.

5. If the client does not receive NEW-UIDVALIDITY, does not understand it, or,
for whatever reason, chooses not to handle it, it MUST treat the newly renamed
mailbox as is required now.  It will see a mailbox with the new name and with
a UIDVALIDITY value that it has not yet encountered (because of (3) above).  It
MUST, therefore, treat it as a new mailbox, discarding anything it thinks it
already knows about it.

Timo, please correct me if I've misstated anything here, or missed anything.
I think this is a reasonable approach, giving the server the option of improving
the situation, and giving the client a chance to behave more efficiently when a
mailbox is renamed, but not breaking any existing implementations nor requiring
anyone to change if they don't want to.

It does not address any of Mark's other issues with RENAME, of course, and for
implementations that must deal with mailstores that expose those issues, this
is a real problem.  I suppose a server can always respond "NO" to any RENAME
command, and thus require the client to use alternatives, but I suspect that
most clients would just expose the "NO" to the user, and make the user deal
with it.

I would support a move to change the new IMAP draft to suggest that a server
that has trouble with RENAME MAY say "NO", and that a client that receives
a "NO" to RENAME SHOULD try the CREATE/move alternative before exposing the
failure to the end-user.  Maybe we should define a response code to go with
the "NO", à la "TRY-CREATE", that would give a savvy client a hint that the
server doesn't like RENAME and wants the client to try alternatives.

The reason I don't want to deprecate RENAME outright is that for many servers
it IS a reasonable thing to do (though, yes, most (all?) servers do not implement
it strictly according to the spec; it'd be nice to have that fixed).

Barry
--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

Date: Thu, 23 Jan 2003 09:40:35 +0300
From: Alexey Melnikov <[email-masked]>
To: Barry Leiba <[email-masked]>
CC: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

Barry Leiba wrote:

> Here's a draft (ahem) copy of the next LIST Extensions draft.  Please
> give it a look and give comments before I send it in.  Notably added is
> the "mbox-list-extension" item in the formal grammar, and the
> associated bits.

IMHO, you should also add some text saying that we also extend LIST responses.

ABNF:
   mbox-list-extended = mbox-list-extended-item
                        *(SP mbox-list-extended-item)
   mbox-list-extended-item = string / mbox-list-extended

So this basically says that we have a list of strings without nested lists? I was thinking that we
should tag different bits of LIST information and allow for multiple tags, e.g.

       S: * LIST (\HasChildren) "/" "Meat" ((ACL user=Fred rwipslextda)(X-MAILBOX-TYPE regular)

>  I think I should say something in the text about what
> that's for, and I'm open to suggestions about what to say and where to
> put it.  Larry, have you got some comments, since you were the one who
> suggested this addition?
>
> Cyrus, Alexey, what more need I add to support using this for the
> things we discussed in Atlanta?
>
> Barry

Alexey

Date: Tue, 21 Jan 2003 15:09:38 -0500
To: Barry Leiba <[email-masked]>, IMAP extensions Mailing List <[email-masked]>
From: Pete Maclean <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

>OK, how about this?:
>
>--------
>IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we have 
>added extensions that have require specialized lists (see [MboxRefer] for 
>an example) we have expand the number of list commands, since each 
>extension must add its function to both LIST and LSUB, and these commands 
>are not, as they are defined, extensible.  This document describes 
>extensions to the LIST command that allow these additions to be done with 
>mutually compatible options to the LIST command, avoiding the exponential 
>increase in specialized list commands.

Ah, now it is making sense to me.  Thanks.  I see that a large part of my 
puzzlement stemmed from my interpretation of "since each extension must add 
its function to both LIST and LSUB".  I was understanding that as meaning 
adding capability into LIST and LSUB which is exactly what the new proposal 
does and what the old extensions did not.  But you mean supplementing the 
functionality of LIST and LSUB, right?

May I suggest a couple of improvements along the lines of:

IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we have 
added extensions that required specialized lists (see [MboxRefer] for an 
example), we have done so by defining new list commands, for example RLIST 
and RLSUB.  Each such extension has served to supplement  LIST and LSUB 
since these commands, as currently defined, are not intrinsically 
extensible.  This document describes an extension to the LIST command that 
allows such additions to be made with mutually compatible options thereby 
avoiding exponential increase in the number of specialized list commands.

Note that I have changed "extensions" to "extension" in the last 
sentence.  This may well be debatable but I see the addition of a new 
argument as essentially a single change.

Pete

Date: Tue, 21 Jan 2003 11:48:53 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: base spec clarification of UID command responses
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 21 Jan 2003, Timo Sirainen wrote:
> How does this affect FETCH replies sent by server when it notices
> external flag changes?

Since this case is truly unsolicited, I don't think that the UID command
requirement applies in this case.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: base spec clarification of UID command responses
From: Timo Sirainen <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 21 Jan 2003 21:41:12 +0200
Sender: [email-masked]

On Wed, 2002-11-27 at 00:39, Cyrus Daboo wrote:
> I interpreted this to mean that any UID command, be it FETCH, STORE, COPY 
> or SEARCH, has to always include UID items in any FETCH responses. However, 
> I have come across at least one server that does not return the UID item in 
> FETCH responses when doing a UID STORE. I'm concerned that the above 
> paragraph might have been interpreted as meaning that the UID item is only 
> required when UID FETCH is used, as opposed to the others.

Sorry for replying a bit late, I only now began to read this list's
archives.

How does this affect FETCH replies sent by server when it notices
external flag changes? This could happen from client's point of view:

C: x UID FETCH 1 (FLAGS)
S: * 1 FETCH (FLAGS ())
S: * 1 FETCH (FLAGS () UID 1)

If the first FETCH was sent before seeing the UID FETCH command.

Date: Tue, 21 Jan 2003 12:58:46 -0500
From: Barry Leiba <[email-masked]>
To: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

>  >> "As we add extensions that require specialized lists...we expand
> the number of  >> list commands..."
>
> This puzzles me because the draft is explicitly about reducing the
> number of list commands (e.g. obsoleting RLIST and RLSUB), not
> expanding it.  Do you mean, "As we have added extensions that
> required specialized lists we have, in the past, expanded the number
> of list commands..."?

I could change it to use a past tense, yes.  It's in the "abstract" 
section; it's explaining why we're making this extension.

>  >>"as each extension must add its function to both LIST and LSUB."
>
> However, I can say that I do not even understand in what sense the
> word "as" is being used.

>From American Heritage Dictionary:
  as ... (conj.) ... 4. For the reason that; because.

OK, how about this?:

--------
IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we 
have added extensions that have require specialized lists (see 
[MboxRefer] for an example) we have expand the number of list commands, 
since each extension must add its function to both LIST and LSUB, and 
these commands are not, as they are defined, extensible.  This document 
describes extensions to the LIST command that allow these additions to 
be done with mutually compatible options to the LIST command, avoiding 
the exponential increase in specialized list commands.
--------

Barry

--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

Date: Tue, 21 Jan 2003 12:10:59 -0500
To: "Barry Leiba" <[email-masked]>, "IMAP extensions Mailing List" <[email-masked]>
From: Pete Maclean <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

> >  >>As we add extensions that require specialized lists (see [MboxRefer] 
> for an
> >  >>example) we expand the number of list commands, as each extension
> >  >>must add its function to both LIST and LSUB.
> >
> > Please excuse my saying so, but this sentence strikes me as barely
> > comprehensible.  The sentence that follows it could be better worded as 
> well.
>
>Hm.  It seems perfectly normal English to me, so I don't know how I'd re-word
>it.  Maybe if you explain why you think it's barely comprehensible...?

I'll try but it's real hard to explain why something fails to make 
sense.  By the way, I have read the sentence over and over while writing 
this response and still cannot draw any useful meaning from it.

 >> "As we add extensions that require specialized lists...we expand the 
number of
 >> list commands..."

This puzzles me because the draft is explicitly about reducing the number 
of list commands (e.g. obsoleting RLIST and RLSUB), not expanding it.  Do 
you mean, "As we have added extensions that required specialized lists we 
have, in the past, expanded the number of list commands..."?

 >>"as each extension must add its function to both LIST and LSUB."

I cannot say much about this since, without a clear understanding of the 
first part of the sentence, I have no definite reference for it.  However, 
I can say that I do not even understand in what sense the word "as" is 
being used.

I truly apologize for making such a fuss about one little sentence.  Please 
be assured that I am not trying to take issue in any way with your use of 
English but merely trying to understand something that I find baffling.

Pete Maclean

Date: Tue, 21 Jan 2003 10:11:45 -0500 (EST)
From: Rob Siemborski <[email-masked]>
To: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

On Tue, 21 Jan 2003, Barry Leiba wrote:

> > Why is there not an attribute flag to identify remote mailboxes as such?
>
> There might be.  That oughta be up to "son of RFC2193".  Or should it?  Should
> we define a "\Remote" or "\Referral" flag here?  I have a vague memory that we
> discussed this before and said "no".

It is possible to have an implementation that will not *always* give a
referral for a mailbox (for example, it might give a referral for a
SELECT, but not for a RENAME), so I'm not sure how these flags would be
used in such an environment.

Additionally, does knowing that a mailbox will receive a referral without
knowing where that referral will be to really help clients much?  I'm
tempted to say no, but perhaps a client author would disagree.

I'd rather just delay the referral decision until something is actually
done to the mailbox.

> > And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do
> > remote mailboxes have to be top-level mailboxes?  Thanks.
>
> As Rob said, no, and the answer to your amended question is also no -- the
> referral "tree" does not have to be anchored at the top level.  I think the
> existing implementations do that, but it's not required.

Cyrus definitely does not anchor referrals at the top level.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

From: "Barry Leiba" <[email-masked]>
To: "IMAP extensions Mailing List" <[email-masked]>
Subject: Re: LIST Extensions draft
Date: Tue, 21 Jan 2003 09:00:08 -0500
Sender: [email-masked]

>  >>As we add extensions that require specialized lists (see [MboxRefer] for an
>  >>example) we expand the number of list commands, as each extension
>  >>must add its function to both LIST and LSUB.
> 
> Please excuse my saying so, but this sentence strikes me as barely 
> comprehensible.  The sentence that follows it could be better worded as well.

Hm.  It seems perfectly normal English to me, so I don't know how I'd re-word
it.  Maybe if you explain why you think it's barely comprehensible...?

> It is not made clear whether or not, when responding to a LIST (SUBSCRIBED) 
> command, a server SHOULD include all appropriate mailboxes that would have 
> the "\PlaceHolder" attribute.

I don't think this has anything to do with (SUBSCRIBED) in particular.  In
general, I think the server MUST include a mailbox with the \PlaceHolder flag
whenever (1) that mailbox does not match the selection criteria AND (2) there
is a child mailbox that (a) does match the selection criteria that is NOT listed
or (b) merits the \PlaceHolder flag by this explanation.

I guess I should expand on the explanation of \PlaceHolder, to make this
clearer.

> Why is there not an attribute flag to identify remote mailboxes as such?

There might be.  That oughta be up to "son of RFC2193".  Or should it?  Should
we define a "\Remote" or "\Referral" flag here?  I have a vague memory that we
discussed this before and said "no".

> And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do 
> remote mailboxes have to be top-level mailboxes?  Thanks.

As Rob said, no, and the answer to your amended question is also no -- the
referral "tree" does not have to be anchored at the top level.  I think the
existing implementations do that, but it's not required.

Barry
--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

Date: Mon, 20 Jan 2003 16:49:55 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Barry Leiba <[email-masked]>
CC: IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

Barry Leiba wrote:
> 
> Here's a draft (ahem) copy of the next LIST Extensions draft.  Please
> give it a look and give comments before I send it in.  Notably added is

Minor nit.  The references to the original CHILDREN extension should be
changed to point to RFC 3328.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Mon, 20 Jan 2003 16:04:26 -0500 (EST)
From: Rob Siemborski <[email-masked]>
To: Pete Maclean <[email-masked]>
cc: Barry Leiba <[email-masked]>, IMAP extensions Mailing List <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

On Mon, 20 Jan 2003, Pete Maclean wrote:

> And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do
> remote mailboxes have to be top-level mailboxes?  Thanks.

No, they do not.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Mon, 20 Jan 2003 15:50:42 -0500
To: Barry Leiba <[email-masked]>, IMAP extensions Mailing  List <[email-masked]>
From: Pete Maclean <[email-masked]>
Subject: Re: LIST Extensions draft
Sender: [email-masked]

I applaud this proposed extension and have a few comments:

 >>As we add extensions that require specialized lists (see [MboxRefer] for an
 >>example) we expand the number of list commands, as each extension
 >>must add its function to both LIST and LSUB.

Please excuse my saying so, but this sentence strikes me as barely 
comprehensible.  The sentence that follows it could be better worded as well.

It is not made clear whether or not, when responding to a LIST (SUBSCRIBED) 
command, a server SHOULD include all appropriate mailboxes that would have 
the "\PlaceHolder" attribute.

Why is there not an attribute flag to identify remote mailboxes as such?

And lastly, a naive question (since I am unfamiliar with RFC-2193):  Do 
remote mailboxes have to be top-level mailboxes?  Thanks.

Pete Maclean

Date: Mon, 20 Jan 2003 14:13:38 -0500
From: Barry Leiba <[email-masked]>
To: IMAP extensions Mailing  List <[email-masked]>
Subject: LIST Extensions draft
Sender: [email-masked]

--==========514551984==========
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Here's a draft (ahem) copy of the next LIST Extensions draft.  Please 
give it a look and give comments before I send it in.  Notably added is 
the "mbox-list-extension" item in the formal grammar, and the 
associated bits.  I think I should say something in the text about what 
that's for, and I'm open to suggestions about what to say and where to 
put it.  Larry, have you got some comments, since you were the one who 
suggested this addition?

Cyrus, Alexey, what more need I add to support using this for the 
things we discussed in Atlanta?

Barry

--
Barry Leiba, Internet Messaging Technology  ([email-masked])
http://www.research.ibm.com/people/l/leiba

--==========514551984==========
Content-Type: text/plain; charset=iso-8859-1;
 name="draft-ietf-imapext-list-extensions-02.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="draft-ietf-imapext-list-extensions-02.txt"; size=18739

IMAP Extensions Working Group                                   B. =
Leiba
Internet Draft                           IBM T.J. Watson Research =
Center
Document: draft-ietf-imapext-list-extensions-02.txt         January =
2003
                                                       Expires July =
2003
                     IMAP4 LIST Command Extensions
Status of this Document
   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026.  Internet-Drafts are =
working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.
   Internet-Drafts are draft documents valid for a maximum of six =
months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."
     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt
     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.
   A revised version of this draft document will be submitted to the =
RFC
   editor as an Proposed Standard for the Internet Community.
   Discussion and suggestions for improvement are requested, and should
   be sent to [email-masked].  This document will expire before =
31
   July 2003.  Distribution of this memo is unlimited.
1. Abstract
   IMAP4 has two commands for listing mailboxes: LIST and LSUB.  As we
   add extensions that require specialized lists (see [MboxRefer] for =
an
   example) we expand the number of list commands, as each extension
   must add its function to both LIST and LSUB.  This document =
describes
   extensions to the LIST command that allow these additions to be done
   in mutually compatible options to the LIST command, avoiding the
   exponential increase in specialized list commands.
B. Leiba                   Expires July 2003                   [Page =
1]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
2. Conventions used in this document
   In examples, "C:" indicates lines sent by a client that is connected
   to a server.  "S:" indicates lines sent by the server to the client.
   The words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" are
   used in this document as specified in RFC 2119 [Keywords].
3. Introduction
   The extensions to the LIST command will be accomplished by amending
   the syntax to allow options to be specified.  The list of options
   will replace the several commands that are currently used to mix and
   match the information requested.  The new syntax is backward-
   compatible, with no ambiguity: if the first word after the command
   name begins with a parenthesis, the new syntax is being used; if it
   does not, it's in the original syntax.
   By adding options to the LIST command, we are announcing the intent
   to phase out and eventually to deprecate the RLIST and RLSUB =
commands
   described in [MboxRefer].  We are also defining the mechanism to
   request extended mailbox information, such as is described in the
   now-expired "Child Mailbox Extension" draft [ChildMbox].  The base
   LSUB command is not deprecated by this extension; rather, this
   extension adds a way to obtain subscription information with more
   options, with those server implementations that support it.  Clients
   that simply need a list of subscribed mailboxes, as provided by the
   LSUB command, SHOULD continue to use that command.
4. LIST Command Options
   The LIST command syntax is extended by adding a parenthesized list =
of
   command options between the command name and the reference name (see
   the formal syntax in section 6 for specific details).  Command
   options will be defined in this document and in approved extension
   documents; each option will be enabled by a capability string (one
   capability may enable multiple options), and a client MUST NOT send
   an option for which the server has not advertised support.  A server
   MAY ignore options it does not recognize.
   This extension is identified by the capability string "LISTEXT", and
   support for it is a prerequisite for any future extensions that
   require specialized forms of the LIST command.  Such extensions MUST
   refer to this document and MUST add their function through command
   options as described herein.  This document also defines the "LIST-
   SUBSCRIBED" capability string; see the "SUBSCRIBED" option below.
B. Leiba                   Expires July 2003                   [Page =
2]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   The options defined in this specification are
       SUBSCRIBED - causes the LIST command to list subscribed
          mailboxes, rather than the actual mailboxes.  This will often
          be a subset of the actual mailboxes.  It's also possible for
          this list to contain the names of mailboxes that don't exist.
          In any case, the list MUST include exactly those mailbox =
names
          that match the selection criteria and are subscribed to.  =
This
          option is intended to supplement the LSUB command, and =
support
          for it is optional -- a server that supports the SUBSCRIBED
          option indicates so through the LIST-SUBSCRIBED capability.
          Of particular note are the mailbox flags as returned by this
          option, compared with what is returned by LSUB.  With the
          latter, the flags returned may not reflect the actual flag
          status on the mailbox, and the \NoSelect flag has a special
          meaning (it indicates that this mailbox is not, itself,
          subscribed, but that it has child mailboxes that are).  With
          the SUBSCRIBED option described here, the flags are accurate
          and complete, and have no special meanings.
          "LSUB" and "LIST (SUBSCRIBED)" are, thus, not the same thing,
          and some servers must do significant extra work to respond to
          "LIST (SUBSCRIBED)".  Because of this, clients SHOULD =
continue
          to use "LSUB" unless they specifically want the additional
          information offered by "LIST (SUBSCRIBED)".  At the same =
time,
          servers SHOULD support the LIST-SUBSCRIBED capability even if
          it entails extra work, because a client that wants the
          information will still obtain it by using LSUB followed by a
          series of LIST commands, so servers might as well make it
          easier.
          This option defines a new mailbox flag, "\NonExistent", that
          indicates that a mailbox is subscribed to, but does not
          actually exist.  The "\NonExistent" flag MUST be supported =
and
          MUST be accurately computed.
       REMOTE - causes the LIST command to show remote mailboxes as
          well as local ones, as described in [MboxRefer].  This option
          is intended to replace the RLIST command and, in conjunction
          with the SUBSCRIBED option, the RLSUB command.  This option =
is
          only available on servers that also support RFC-2193.
       CHILDREN - Requests mailbox child information as originally
          proposed in [ChildMbox].  See section 5, below, for details.
          Support for this is optional, but this option MUST be =
accepted
          by all servers (though it MAY be ignored).
   The LISTEXT capability also defines a new mailbox flag,
B. Leiba                   Expires July 2003                   [Page =
3]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   "\PlaceHolder", that indicates that the designated mailbox does not
   meet the selection criteria of the given LIST command, but that it
   has one or more child mailboxes that do [EDITORIAL NOTE: "might"?].
   The LSUB command indicates this condition by using the "\NoSelect"
   flag, but the LIST (SUBSCRIBED) command MUST NOT do that, since
   "\NoSelect" retains its original meaning here.  Further, the
   "\PlaceHolder" flag is more general, in that it can be used with any
   extended set of selection criteria.
5. The CHILDREN Option
   The CHILDREN option implements the Child Mailbox Extension,
   originally proposed by Mike Gahrns and Raymond Cheng, of Microsoft
   Corporation.  Most of the information in this section is taken
   directly from their original specification [ChildMbox].  The =
CHILDREN
   option is simply an indication that the client wants this
   information; a server MAY provide it even if the option is not
   specified, or MAY ignore the option entirely.
   Many IMAP4 [IMAP4] clients present to the user a hierarchical view =
of
   the mailboxes that a user has access to.  Rather than initially
   presenting to the user the entire mailbox hierarchy, it is often
   preferable to show to the user a collapsed outline list of the
   mailbox hierarchy (particularly if there is a large number of
   mailboxes).  The user can then expand the collapsed outline =
hierarchy
   as needed.  It is common to include within the collapsed hierarchy a
   visual clue (such as a ''+'') to indicate that there are child
   mailboxes under a particular mailbox.   When the visual clue is
   clicked the hierarchy list is expanded to show the child mailboxes.
   The Child Mailbox Extension provides a mechanism for a client to
   efficiently determine if a particular mailbox has children, without
   issuing a LIST '' * or a LIST '' % for each mailbox name.
   The Child Mailbox Extension defines two new attributes that MAY be
   returned within a LIST response: \HasChildren and \HasNoChildren.
   While these attributes MAY be returned in response to any LIST
   command, the CHILDREN option is provided to indicate that the client
   particularly wants this information.  If the CHILDREN option is
   present, the server SHOULD return these attributes even if their
   computation is expensive.
\HasChildren - The presence of this attribute indicates that the =
mailbox
       has child mailboxes.
       A server SHOULD NOT set this attribute if there are child
       mailboxes, and the user does not have permissions to access any
       of them. In this case, \HasNoChildren SHOULD be used.
B. Leiba                   Expires July 2003                   [Page =
4]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
       In many cases, however, a server may not be able to efficiently
       compute whether a user has access to all child mailboxes.  As
       such a client MUST be prepared to accept the \HasChildren
       attribute as a hint.  That is, a mailbox MAY be flagged with the
       \HasChildren attribute, but no child mailboxes will appear in
       the LIST response.
\HasNoChildren - The presence of this attribute indicates that the
       mailbox has NO child mailboxes that are accessible to the
       currently authenticated user.
   In some instances a server that supports the Child Mailbox Extension
   might not be able to determine whether a mailbox has children.  For
   example it may have difficulty determining whether there are child
   mailboxes when LISTing mailboxes while operating in a particular
   namespace.
   In these cases, a server MAY exclude both the \HasChildren and
   \HasNoChildren attributes in the LIST response.  As such, a client
   can not make any assumptions about whether a mailbox has children
   based upon the absence of a single attribute.  In particular, some
   servers may not be able to combine the SUBSCRIBED and CHILDREN
   options.  Such servers MUST honour the SUBSCRIBED option, and they
   will simply ignore the CHILDREN option if both are requested.
   It is an error for the server to return both a \HasChildren and a
   \HasNoChildren attribute in a LIST response.
   Note: the \HasNoChildren attribute should not be confused with the
   IMAP4 [IMAP4] defined attribute \NoInferiors which indicates that no
   child mailboxes exist now and none can be created in the future.
6. Examples
   The first example shows the complete local hierarchy that will be
   used for the other examples.
       C: A01 LIST "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST () "/" "Fruit"
       S: * LIST () "/" "Fruit/Apple"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST () "/" "Tofu"
       S: * LIST () "/" "Vegetable"
       S: * LIST () "/" "Vegetable/Broccoli"
       S: A01 OK done
B. Leiba                   Expires July 2003                   [Page =
5]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   In the next example, we'll see the subscribed mailboxes.  This is
   similar, but not equivalent, to <LSUB "" "*">.  Note that the =
mailbox
   called "Fruit/Peach" is subscribed to, but does not actually exist
   (perhaps it was deleted while still subscribed).  And the "Fruit"
   mailbox is not subscribed to, but it has two subscribed children.
       C: A02 LIST (SUBSCRIBED) "" "*"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\PlaceHolder) "/" "Fruit"
       S: * LIST () "/" "Fruit/Banana"
       S: * LIST (\NonExistent) "/" "Fruit/Peach"
       S: A02 OK done
   The next example shows the use of the CHILDREN option.  The client,
   without having to list the second level of hierarchy, now knows =
which
   of the top-level mailboxes have sub-mailboxes (children) and which =
do
   not.  Note that it's not necessary for the server to return the
   \HasNoChildren flag for the inbox, because the \NoInferiors flag
   already implies that, and has a stronger meaning.
       C: A03 LIST (CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: A03 OK done
   In this example we see more mailboxes, which reside on another =
server
   to which we may obtain referrals.  This is similar to the command
   <RLIST "" "%">.  We also see the mixing of two options.  Note that =
in
   the case of the remote mailboxes, the server might or might not be
   able to include CHILDREN information; it includes it if it can, and
   omits it if it can't.
       C: A04 LIST (REMOTE CHILDREN) "" "%"
       S: * LIST (\Marked \NoInferiors) "/" "inbox"
       S: * LIST (\HasChildren) "/" "Fruit"
       S: * LIST (\HasNoChildren) "/" "Tofu"
       S: * LIST (\HasChildren) "/" "Vegetable"
       S: * LIST () "/" "Bread"
       S: * LIST (\HasChildren) "/" "Meat"
       S: A04 OK done
7. Formal Syntax
   The following syntax specification uses the augmented Backus-Naur
   Form (BNF) as described in [ABNF].  Terms not defined here are taken
   from [IMAP4].
B. Leiba                   Expires July 2003                   [Page =
6]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
   child-mbox-flag =3D "\HasChildren" / "\HasNoChildren"
                      ; flags for Child Mailbox Extension, at most one
                      ; possible per LIST response
   list            =3D "LIST" [SP list-options] SP mailbox SP =
list-mailbox
   list-options    =3D "(" [option *(SP option)] ")"
   mailbox-list    =3D "(" [mbx-list-flags] ")" SP
                      (DQUOTE QUOTED-CHAR DQUOTE / nil) SP mailbox
                      [SP "(" [mbox-list-extended] ")"]
   mbox-list-extended =3D mbox-list-extended-item
                        *(SP mbox-list-extended-item)
   mbox-list-extended-item =3D string / mbox-list-extended
   mbox-list-oflag :=3D child-mbox-flag / "\NonExistent" / =
"\PlaceHolder"
   option           =3D "SUBSCRIBED" / "CHILDREN" / "REMOTE" /
                     option-extension
   option-extension =3D atom
8. Security Considerations
   This document describes syntactic changes to the specification of =
the
   IMAP4 commands LIST, LSUB, RLIST, and RLSUB, and the modified LIST
   command has the same security considerations as those commands.  =
They
   are described in [IMAP4] and [MboxRefer].
   The Child Mailbox Extension provides a client a more efficient means
   of determining whether a particular mailbox has children.  If a
   mailbox has children, but the currently authenticated user does not
   have access to any of them, the server SHOULD respond with a
   \HasNoChildren attribute.  In many cases, however, a server may not
   be able to efficiently compute whether a user has access to all =
child
   mailboxes.  If such a server responds with a \HasChildren attribute,
   when in fact the currently authenticated user does not have access =
to
   any child mailboxes, potentially more information is conveyed about
   the mailbox than intended.   In most situations this will not be a
   security concern, because if information regarding whether a mailbox
   has children is considered sensitive, a user would not be granted
   access to that mailbox in the first place.
B. Leiba                   Expires July 2003                   [Page =
7]=0C
Internet DRAFT       IMAP4 LIST Command Extensions         January 2003
9. References
   [IMAP4]; Crispin, M.; "Internet Message Access Protocol - Version
   4rev1"; RFC 2060; University of Washington; December 1996.
   [MboxRefer]; Gahrns, M.; "IMAP4 Mailbox Referrals"; RFC 2193;
   Microsoft Corporation; September 1997.
   [ChildMbox]; Gahrns, M. & Cheng, R.; "IMAP4 Child Mailbox =
Extension";
   draft-gahrns-imap-child-mailbox-03.txt; Microsoft Corporation;
   November 1997 -- expired, listed for historical reference.
   [Keywords]; Bradner, S.; "Key words for use in RFCs to Indicate
   Requirement Levels"; RFC 2119; Harvard University; March 1997.
   [ABNF]; Crocker, D., and Overell, P. "Augmented BNF for Syntax
   Specifications: ABNF", RFC 2234, November 1997.
10. Acknowledgements
   Mike Gahrns and Raymond Cheng of Microsoft Corporation originally
   devised the Child Mailbox Extension and proposed it in 1997; the
   idea, as well as most of the text in section 5, is theirs.
11. Author's Address
   Barry Leiba
   IBM T.J. Watson Research Center
   30 Saw Mill River Road
   Hawthorne, NY  10532
   Phone: 1-914-784-7941
   Email: [email-masked]
B. Leiba                   Expires July 2003                   [Page 8]
--==========514551984==========--

Date: Fri, 17 Jan 2003 08:35:17 -0500
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: [off charter] imap extension for moderation?
Reply-To: [email-masked]
Sender: [email-masked]

I don't see this in the charter, but perhaps the list will permit me to ask a 
simple question - has anyone ever attempted to extend IMAP for the purpose of
list moderation?

The idea I have is that messages to be moderated would end up in an IMAP
folder, and that they could be flagged by a MUA with /Approved or /Rejected or
deleted to cause them to be forwarded to the list or bounced or discarded.  It
would also be useful to be able to specify a reason for rejecting a message.

It just seems like a MUA is a much better tool for reviewing messages than,
say, a web browser.

Since this is off-topic, please send replies to me personally rather than to
the list.

thanks,

Keith

From: Arnt Gulbrandsen <[email-masked]>
To: Ken Murchison <[email-masked]>
Subject: Re: Thread extension weirdness
Cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Sun, 12 Jan 2003 17:36:13 +0100
Sender: [email-masked]

Ken Murchison writes:
> So there is no interest in revisiting adding algorithm options to THREAD?

The possible semantics/implementation complexity sounds a little scary 
to this small boy. That complexity isn't, from what I can tell, 
justified.

--Arnt

Date: Sat, 11 Jan 2003 18:04:43 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Sat, 11 Jan 2003, Lawrence Greenfield wrote:
> I'm not interested in breaking THREAD=REFERENCES. I'm merely saying that if
> the current THREAD=REFERENCES is broken, we shouldn't publish it as a
> standards-track RFC.

I claim that the current THREAD=REFERENCES is not broken, and should be
published as a standards-track RFC.  That does not preclude other
threading algorithms, as anticipated by the THREAD specification, will
happen.

All threading is going to be a compromise.  For better or worse, this is
the compromise that we have, and it generally works quite well.  An
attempt to push some other compromise in lieu of the one that is already
wide will not make the situation better.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sat, 11 Jan 2003 20:54:24 -0500
From: Lawrence Greenfield <[email-masked]>
To: Mark Crispin <[email-masked]>, Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

--On Saturday, January 11, 2003 10:41 AM -0800 Mark Crispin 
<[email-masked]> wrote:

> On Sat, 11 Jan 2003 13:39:16 -0500, Cyrus Daboo wrote:
>> [proposed improved algorithm]
>> This should probably be done as a
>> separate THREAD=XXX option.
>
> To make things unambiguous:
>
> I strongly support this.  My opposition is solely to the idea of breaking
> the existing THREAD=REFERENCES algorithm.

I'm not interested in breaking THREAD=REFERENCES. I'm merely saying that if 
the current THREAD=REFERENCES is broken, we shouldn't publish it as a 
standards-track RFC. Current servers don't have to drop support, but 
there's no reason to publish both THREAD=REFERENCES and the fixed 
THREAD=XXX.

I guess I agree with Mark that there's probably no need for a seperate spec 
that returns only threads with unseen messages. (The underlying question is 
whether THREAD or SORT's O(n) size responses is acceptable. I'd rather not 
reopen that question right now.)

Larry

Date: Sat, 11 Jan 2003 13:54:48 -0800 (PST)
From: Mark Crispin <[email-masked]>
To: Cyrus Daboo <[email-masked]>
cc: Ken Murchison <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Sat, 11 Jan 2003, Cyrus Daboo wrote:
> I think there is a demonstrable need for at least one option - namely the
> 'return only threads containing unseen messages' one. That could actually
> be generalized to take a list of flags or even further to take another set
> of full search criteria so you could select threads by From address etc.
> Beyond that I can't think of anything else one might want to do...

You can accomplish the same thing by doing a SEARCH command after the
THREAD command.  It's a little bit more work, a little bit more network
I/O.  But it's still much less costly than doing a client-based thread.

Thus, you do not "need" this option, even if it is "nice to have."

As with all "nice to have" options, the question then goes to how widely
deployed and used this would likely be.

Mind you; I am not opposed to adding such a capability.  I see no harm in
doing so other than added complexity.  I'm just insisting that it make a
better case for itself.

Most IMAP clients -- Pine and Mulberry being notable exceptions -- treat
IMAP as a glorified POP3.  It is difficult to see how adding complexity to
IMAP is going to help when so many IMAP clients fail to use what is there
now.

> PS The 'search-charset' formal syntax item in the current draft is not
> defined in the draft and does not appear in any of the base-spec documents
> either. From the base-spec it ought to be an astring.

Thanks, I will fix that.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sat, 11 Jan 2003 16:05:15 -0500
From: Cyrus Daboo <[email-masked]>
To: Ken Murchison <[email-masked]>, Mark Crispin <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Hi Ken,

--On Saturday, January 11, 2003 3:00 PM -0500 Ken Murchison 
<[email-masked]> wrote:

|> I strongly support this.  My opposition is solely to the idea of
|> breaking the existing THREAD=REFERENCES algorithm.
|
| So there is no interest in revisiting adding algorithm options to
| THREAD?

I think there is a demonstrable need for at least one option - namely the 
'return only threads containing unseen messages' one. That could actually 
be generalized to take a list of flags or even further to take another set 
of full search criteria so you could select threads by From address etc. 
Beyond that I can't think of anything else one might want to do...

If we want something like the thread selection capability then it should be 
done as an option so that it be applied to different algorithms, without 
the need to spawn even more algorithm names. How about having a 
THREAD-OPTION capability to list those, and allowing something like:

* CAPABILITY THREAD=REFERENCES THREAD=REFERENCES2 THREAD-OPTIONS=UNSEEN 
THREAD-OPTIONS=SEARCH

a THREAD REFERENCES (UNSEEN) UTF8 ALL

or

a THREAD REFERENCES2 (SEARCH FROM [email-masked]) UTF8 ALL

The THREAD-OPTIONS scheme could be added to the current draft without 
affecting the installed base. Later documents could then add the actual 
options that one may want.

PS The 'search-charset' formal syntax item in the current draft is not 
defined in the draft and does not appear in any of the base-spec documents 
either. From the base-spec it ought to be an astring.

-- 
Cyrus Daboo

Date: Sat, 11 Jan 2003 15:00:51 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Sat, 11 Jan 2003 13:39:16 -0500, Cyrus Daboo wrote:
> > [proposed improved algorithm]
> > This should probably be done as a
> > separate THREAD=XXX option.
> 
> To make things unambiguous:
> 
> I strongly support this.  My opposition is solely to the idea of breaking the
> existing THREAD=REFERENCES algorithm.

So there is no interest in revisiting adding algorithm options to
THREAD?

If not, I'm fine with a THREAD=REFERENCES2 or whatever for Cyrus'
tweak.  I too have been annoyed by what Cyrus described.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Sat, 11 Jan 2003 10:41:49 -0800 (PST)
From: Mark Crispin <[email-masked]>
Subject: Re: Thread extension weirdness
To: Cyrus Daboo <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Sender: [email-masked]

On Sat, 11 Jan 2003 13:39:16 -0500, Cyrus Daboo wrote:
> [proposed improved algorithm]
> This should probably be done as a
> separate THREAD=XXX option.

To make things unambiguous:

I strongly support this.  My opposition is solely to the idea of breaking the
existing THREAD=REFERENCES algorithm.

And, to address the original issue, I rewrote UW imapd's subject extraction
rules yesterday to strictly conform with the draft.  The new code will be in
all versions after Pine 4.52 and imap-2002b (which were released on Thursday).
I also discovered, after trying numerous mailboxes and newsgroups, that the
only way that I could get the new code and the old code to thread differently
was with a contrived example.  "re"s do not occur inside quoted-words in any
example in the wild that I could discover.

Date: Sat, 11 Jan 2003 13:39:16 -0500
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
cc: IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Hi Mark,

--On Friday, January 10, 2003 8:38 PM -0800 Mark Crispin 
<[email-masked]> wrote:

|> Are you saying that there are mailboxes that will not be correctly
|> threaded in this case?
|>
|> Or do you not want to get rid of THREAD=REFERENCES because of widespread
|> deployment?
|
| Yes to both of the above.

I think what we need here is a compromise between the two extremes that we 
have: grouping by subject and only use 2822 headers. The current algorithm 
breaks when you have two distinct threads that have the same subject. I 
often get messages from colleagues that just say 'Update' in the subject, 
or online order confirmations that just say 'Your Order Confirmation' (no 
order number). These currently all get grouped together and this is the 
root of my real annoyance, because a message I got yesterday could be 
incorrectly grouped with one sent a year ago. I've had complaints from 
users about this in the sixteen months we've had thread support deployed.

Having said that, the current algorithm does work reasonably well when 
there is a thread with a unique subject that is broken (2822-wise) because 
there are still many deployed clients that don't do 2822 references headers 
properly, and it would be nice to preserve the subject grouping in those 
cases.

What I propose is to tweak the current algorithm to fix the first case 
whilst still allowing the second case to work. This could be a simple 
matter of spotting the fact that there are two distinct 'root' messages 
(i.e. messages that have the same 'original' as opposed to 'extracted' 
subject that does not contain any Re/fwd/[...] etc artifacts). Those root 
messages should not be grouped together. Then it is a matter of deciding 
how to group replies that do not have proper 2822 references headers to 
those roots in such a way that the proper replies go with the appropriate 
root. The simplest solution is to apply a grouping of replies based on date 
sent. i.e. all replies sent before the second root message are grouped with 
the first, and all replies sent after the second root message, are grouped 
with the second root. This could mean that legitimate replies to the first 
root end up under the second one, but I don't believe this is any worse 
than the current position where the roots and all replies get grouped 
together. Note that replies that have the proper 2822 headers will always 
appear under the correct root.

This change would require significant reworking of step (5) in the current 
algorithm and would lead to more complexity there, but I think it is worth 
it to provide a reasonable compromise. This should probably be done as a 
separate THREAD=XXX option.

-- 
Cyrus Daboo

Date: Fri, 10 Jan 2003 17:01:30 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: Arnt Gulbrandsen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Interest List <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Fri, 10 Jan 2003 13:12:32 -0500, Ken Murchison wrote:
> > Well, I don't think we can change the current name, because there is
> > already some deployment.
> 
> We can not change the name.
> 
> > We _could_ add a THREAD=REFERENCES-ONLY or
> > THREAD=REFERENCES-NOSUBJ or ...
> 
> Fine.  The THREAD specification explicitly allows adding algorithms.

Right, but the question is do we want to add these variants as separate
algorithms, or do we want to revisit Cyrus' optional argument syntax?

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Fri, 10 Jan 2003 10:15:41 -0800 (PST)
From: Mark Crispin <[email-masked]>
Subject: Re: Thread extension weirdness
To: Ken Murchison <[email-masked]>
cc: Arnt Gulbrandsen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Interest List <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Fri, 10 Jan 2003 13:12:32 -0500, Ken Murchison wrote:
> Well, I don't think we can change the current name, because there is
> already some deployment.

We can not change the name.

> We _could_ add a THREAD=REFERENCES-ONLY or
> THREAD=REFERENCES-NOSUBJ or ...

Fine.  The THREAD specification explicitly allows adding algorithms.

However, as a cautionary note: keep in mind that each additional algorithm
ratches up the level of complexity.

Date: Fri, 10 Jan 2003 13:12:32 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Arnt Gulbrandsen <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, IMAP Interest List <[email-masked]>, Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Arnt Gulbrandsen wrote:
> 
> Cyrus Daboo writes:
> > However, I will again voice my annoyance with the fact that the
> > THREAD=REFERENCES extension does any kind of grouping based on
> > subject.
> 
> Seconded!
> 
> THREAD=REFERENCES is its name, so why can't it THREAD based on
> REFERENCES? The current draft's algorithm would be better named e.g.
> THREAD=COMPREHENSIVE.

Well, I don't think we can change the current name, because there is
already some deployment.  We _could_ add a THREAD=REFERENCES-ONLY or
THREAD=REFERENCES-NOSUBJ or ...  I also want to see a REFERENCES-UNSEEN
which only shows threads which contain unread messages (note that this
_isn't_ the same as threading unread messages).

This goes back to a discussion that Cyrus, Mark and myself had
previously about either documenting additional REFERENCES-based
algorithms or adding parameters to THREAD, ie:

THREAD REFERENCES (UNSEEN NOSUBJ) ...

I know that Mark wasn't in favor of doing either of these in the current
draft, but maybe its time to get another consensus now (especially if we
want to add an optional param list to THREAD).

Ken
-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

From: Arnt Gulbrandsen <[email-masked]>
To: Cyrus Daboo <[email-masked]>
Subject: Re: Thread extension weirdness
Cc: Ken Murchison <[email-masked]>, IMAP Interest List <[email-masked]>, Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Fri, 10 Jan 2003 18:59:27 +0100
Sender: [email-masked]

Cyrus Daboo writes:
> However, I will again voice my annoyance with the fact that the 
> THREAD=REFERENCES extension does any kind of grouping based on 
> subject.

Seconded!

THREAD=REFERENCES is its name, so why can't it THREAD based on 
REFERENCES? The current draft's algorithm would be better named e.g. 
THREAD=COMPREHENSIVE.

--Arnt

Date: Fri, 10 Jan 2003 12:04:03 -0500
From: Cyrus Daboo <[email-masked]>
To: Ken Murchison <[email-masked]>, Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Hi Ken,

--On Friday, January 10, 2003 8:46 AM -0500 Ken Murchison <[email-masked]> 
wrote:

|> Question for the list: should it?
|
| FWIW, Cyrus does.  This is because the header is decoded for the SEARCH
| and then passed on to SORT/THREAD.  Interestingly, Netscape Messenger
| makes separate threads, even though the algorithms are supposed to be
| similar/same.
|
| I'd vote that the draft is fine, and that UW needs to be fixed (less
| work for me  ;)

FYI My client-side implementation of the THREAD algorithm also does it as 
per the current draft 'by default' since the subject is already MIME 
decoded and 2822 unfolded by the time it gets processed. I believe that is 
the right thing to do.

However, I will again voice my annoyance with the fact that the 
THREAD=REFERENCES extension does any kind of grouping based on subject. 
This is an attempt to fix the behaviour of broken (old) clients that do not 
generate proper In-Reply-To and References headers at the expense of 
legitimate clients that generate those headers in conformance to 2822.

-- 
Cyrus Daboo

Date: Fri, 10 Jan 2003 14:59:38 +0100
From: Sebastian Hagedorn <[email-masked]>
To: Ken Murchison <[email-masked]>, Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

--==========2151805189==========
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

--On Friday, January 10, 2003 08:46:46 -0500 Ken Murchison <[email-masked]> =

wrote:

>> Question for the list: should it?
>
> FWIW, Cyrus does.  This is because the header is decoded for the SEARCH
> and then passed on to SORT/THREAD.  Interestingly, Netscape Messenger
> makes separate threads, even though the algorithms are supposed to be
> similar/same.
>
> I'd vote that the draft is fine, and that UW needs to be fixed (less
> work for me  ;)

>From a user and admin perspective I agree ... it's the behavior I'd expect.
--
Sebastian Hagedorn M.A. - RZKR-R1 (Geb=E4ude 52), Zi. 18, Robert-Koch-Str. =
10
Zentrum f=FCr angewandte Informatik - Universit=E4tsweiter Service RRZK
Universit=E4t zu K=F6ln / Cologne University - Tel. +49-221-478-5587
--==========2151805189==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========2151805189==========--

Date: Fri, 10 Jan 2003 08:46:46 -0500
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>, IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Thu, 10 Jan 2003, Timo Sirainen wrote:
> > Another thing I saw with UW, it
> > didn't group these messages together:
> > Subject: =?iso-8859-1?Q?foo?=
> > Subject: =?iso-8859-1?Q?RE=3A_foo?=
> 
> Question for the list: should it?

FWIW, Cyrus does.  This is because the header is decoded for the SEARCH
and then passed on to SORT/THREAD.  Interestingly, Netscape Messenger
makes separate threads, even though the algorithms are supposed to be
similar/same.

I'd vote that the draft is fine, and that UW needs to be fixed (less
work for me  ;)

Ken

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Thu, 9 Jan 2003 18:34:49 -0800 (Pacific Standard Time)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
cc: IMAP Interest List <[email-masked]>
Subject: Re: Thread extension weirdness
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Thu, 10 Jan 2003, Timo Sirainen wrote:
> Another thing I saw with UW, it
> didn't group these messages together:
> Subject: =?iso-8859-1?Q?foo?=
> Subject: =?iso-8859-1?Q?RE=3A_foo?=

Question for the list: should it?

It turns out that UW imapd does something different from the base subject
extraction algorithm described in draft-ietf-imapext-sort-11.txt.  I'm not
sure if this is a bug in imapd or in draft-ietf-imapext-sort-11.txt, and I
am soliciting opinions on whether UW imapd should be changed to comply
with draft-ietf-imapext-sort-11, or if a revision of the draft should be
changed to comply with UW imapd.

UW imapd's algorithm is:

(1) Convert all continuations to spaces.

(2) Remove all leading WSP or subj-refwd.

(3) If the resulting text starts with a subj-blob, examine the text after
    the subj-blob.  Disregard any subj-trailer encountered.  If there is
    any text other than subj-trailer, remove the subj-blob and repeat (2)
    and (3) until no matches remain.

(4) Convert all MIME encoded-words to UTF-8.

(5) Remove all subj-trailer.

(6) same as in draft-ietf-imapext-sort-11.txt

(7) same as in draft-ietf-imapext-sort-11.txt

The significant difference between UW imapd and the draft is that in UW
imapd leading subj-leader is *not* considered if it is encapsulated in a
MIME encoded word.

It looks like the original draft agreed with UW imapd, and it got changed
in draft -03 or -04 to do the encoded-word conversion at the very start.
I forget why.  Does anyone remember?

If UW imapd is in the wrong, it will be a while before this gets fixed.
We just released Pine 4.52 and imap-2002b.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.