Subject: Re: IMAP Capability and Virtual Domains
From: Simon Josefsson <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>, Harrie Hazewinkel <[email-masked]>, Michael Fair <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Wed, 30 Apr 2003 15:47:11 +0200
List-Id: <ietf-imapext.imc.org>

Arnt Gulbrandsen <[email-masked]> writes:

> Rob Siemborski writes:
>> On Tue, 29 Apr 2003, Michael Fair wrote:
>>
>>>  I'm still assuming that people on this list are still somewhat
>>> dubious about whether or not virtual hosting multiple domains on a
>>> single IP address is even useful.
>>
>> I agree that virtual hosting on a single IP is useful. I'm
>> personally unconvinced of the necessity of a VHOST command (or other
>> additional argument to CAPABILITY) to do this, its almost entirely
>> possible to do this using realms on user ids.
>
> The realms usage I see in RFC 2831 seems a little different. Realms
> there aren't hostnames, and they're used for a multiple-choice
> question. Basically, the server tells the client "pick one of these
> realms" and the client does, perhaps using a listbox or menu.
>
> I would not want to give every IMAP client my customer list,
> particularly not before authentication. And I would not want to tell
> random IMAP clients "here, make a thousand-item pop-up menu and ask
> the user to pick one".

DIGEST-MD5 allows the realm field in step 1 to be empty, and in this
case the client is allowed to chose a realm on their own.  DIGEST-MD5
doesn't have to be used as "pick one of these realms".  However, if
the server sends some realms, the server SHOULD pick one of them.  I
hoped that 2831bis would allow the client to pick any realm it wants,
regardless of what the server sent, thus further moving away from the
problem you describe, but this doesn't seem to have happened.

Date: Wed, 30 Apr 2003 09:23:11 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: Philip Guenther <[email-masked]>, Harrie Hazewinkel <[email-masked]>, Michael Fair <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

On Wed, 30 Apr 2003, Arnt Gulbrandsen wrote:

> The realms usage I see in RFC 2831 seems a little different. Realms
> there aren't hostnames, and they're used for a multiple-choice
> question. Basically, the server tells the client "pick one of these
> realms" and the client does, perhaps using a listbox or menu.

Realms in DIGEST-MD5 are a little strange.

But you *can* just append a @ or a % to an authentication identifier,
followed by a domain, and call it a realm.  This works with a much larger
variety of SASL mechanisms.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

From: Arnt Gulbrandsen <[email-masked]>
To: Rob Siemborski <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: Philip Guenther <[email-masked]>, Harrie Hazewinkel <[email-masked]>, Michael Fair <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date:   Wed, 30 Apr 2003 11:03:03 +0200
Sender: [email-masked]

Rob Siemborski writes:
> On Tue, 29 Apr 2003, Michael Fair wrote:
>
>>  I'm still assuming that people on this list are still somewhat 
>>  dubious about whether or not virtual hosting multiple domains on a 
>>  single IP address is even useful.
>
> I agree that virtual hosting on a single IP is useful. I'm personally 
> unconvinced of the necessity of a VHOST command (or other additional 
> argument to CAPABILITY) to do this, its almost entirely possible to 
> do this using realms on user ids.

The realms usage I see in RFC 2831 seems a little different. Realms 
there aren't hostnames, and they're used for a multiple-choice 
question. Basically, the server tells the client "pick one of these 
realms" and the client does, perhaps using a listbox or menu.

I would not want to give every IMAP client my customer list, 
particularly not before authentication. And I would not want to tell 
random IMAP clients "here, make a thousand-item pop-up menu and ask the 
user to pick one".

> The only problem using realms with user IDs doesn't solve is the fact 
> that you don't know which certificate to present during a STARTTLS. I 
> suppose this can be solved with an argument to STARTTLS. You'd need 
> to advertise a new STARTTLS_WITH_DOMAIN capability too.

As Simon Josefsson said, the TLS people are dealing with that.

--Arnt

Date: Wed, 30 Apr 2003 00:26:31 -0400
From: Lawrence Greenfield <[email-masked]>
To: Timo Sirainen <[email-masked]>, [email-masked]
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Review requested: draft-siemborski-mupdate-03.txt to be	published as an experimental RFC
Sender: [email-masked]

--On Wednesday, April 30, 2003 7:15 AM +0300 Timo Sirainen <[email-masked]> 
wrote:

> Just one question: Is there a reason why this should be RFC? I can see
> that specific server implementations might want to use this internally,
> but I just can't see why it should be interoperable with anything else
> but itself. Would someone actually want to create an IMAP server farm
> consisting of completely different IMAP server implementations?

The purpose of documenting this isn't necessarily to get interoperable 
implementations, but to document the protocol and the approach for the 
future so that the community has a stable reference to an open approach 
that's been done before.

Larry

Subject: Re: Review requested: draft-siemborski-mupdate-03.txt to be published as an experimental RFC
From: Timo Sirainen <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Organization: 
Date: 30 Apr 2003 07:15:32 +0300
Sender: [email-masked]

On Tue, 2003-04-29 at 22:06, [email-masked] wrote:
> I've received a request from the author to publish
> 
>     The MUPDATE Distributed Mailbox Database Protocol
>     draft-siemborski-mupdate-03.txt
> 
> as an experimental RFC. It is customary for area directors to ask for
> such documents to be reviewed by one or more working groups working in the
> same or related area. Accordingly, I'm asking folks involved in IMAPEXT
> to review this document and comment on its suitability for publication as
> an experimental protocol.

Just one question: Is there a reason why this should be RFC? I can see
that specific server implementations might want to use this internally,
but I just can't see why it should be interoperable with anything else
but itself. Would someone actually want to create an IMAP server farm
consisting of completely different IMAP server implementations?

Or maybe there's some other uses that aren't obvious to me? I had to
read Cyrus Aggregator web page before really even understanding how this
protocol was supposed to be used.

From: "Michael Fair" <[email-masked]>
To: "Simon Josefsson" <[email-masked]>
Cc: "Abhijit Menon-Sen" <[email-masked]>, <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 19:42:32 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Some environments will chose to not use STARTTLS, so this will never
> hold.  But some environments would chose to not use VHOST too, so I
> don't see the difference.  You can't ask for more than that people are
> required to implement a feature -- whether it is used or not depends
> on other things -- and people are already required to implement
> STARTTLS.

The difference is that with this proposed solution whether
or not someone can use VHOST is completely predicated on
whether or not they simultaneously use TLS.

With this proposal if you want to use VHOST, you MUST use
TLS - no choice - no option - no way around it - either
use TLS or don't have per domain specific configs.

I find unrealisticly limiting.  Whether or not someone uses 
TLS is a totally separate decision from whether or not the 
they use virtualized hosting.

Especially for domains specific configs that might not
want to use TLS for whatever reason.

-- Michael --

To: "Michael Fair" <[email-masked]>
Cc: "Abhijit Menon-Sen" <[email-masked]>, <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
From: Simon Josefsson <[email-masked]>
Date: Wed, 30 Apr 2003 03:59:37 +0200
Sender: [email-masked]

"Michael Fair" <[email-masked]> writes:

>> At 2003-04-30 00:38:48 +0200, [email-masked] wrote:
>> >
>> > Support draft-ietf-tls-extensions-06.txt in your STARTTLS
>> > implementation.
>> > 
>> > This is not a IMAP specific problem, so a IMAP specific solution is
>> > not the best solution, engineering wise.
>> 
>> I strongly prefer this approach over an optional argument to CAPABILITY,
>> or a separate VHOST command, particularly since STARTTLS support is now
>> mandatory for both servers and clients.
>
> I'm afraid I missed the part where the use of STARTTLS became
> mandatory.

See RFC 3501:

      Client and server implementations MUST implement the STARTTLS,
      LOGINDISABLED, and AUTH=PLAIN (described in [IMAP-TLS])
      capabilities.

> Fortunately it's generally accepted that STARTTLS
> requires a new CAPABILITY announcement anyway, thus if it can 
> be shown that all implementations of both clients and servers 
> not only support but MUST also use STARTTLS whenever it's 
> available then I'd probably accept this as a solution that 
> will work.

Some environments will chose to not use STARTTLS, so this will never
hold.  But some environments would chose to not use VHOST too, so I
don't see the difference.  You can't ask for more than that people are
required to implement a feature -- whether it is used or not depends
on other things -- and people are already required to implement
STARTTLS.

This doesn't mean they are required to implement the TLS extension
though...

> It'd be harder to debug since you will now need to get something 
> more intelligent than telnet to test against, but I'm sure 
> there'll be tools to help with that (perhaps even an updated telnet).

The tools exists.  The problem isn't specific to TLS, some SASL
mechanisms also encrypts the connection.

From: "Michael Fair" <[email-masked]>
To: "Abhijit Menon-Sen" <[email-masked]>, <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 18:16:10 -0700
Organization: DaClubhouse
Sender: [email-masked]

> At 2003-04-30 00:38:48 +0200, [email-masked] wrote:
> >
> > Support draft-ietf-tls-extensions-06.txt in your STARTTLS
> > implementation.
> > 
> > This is not a IMAP specific problem, so a IMAP specific solution is
> > not the best solution, engineering wise.
> 
> I strongly prefer this approach over an optional argument to CAPABILITY,
> or a separate VHOST command, particularly since STARTTLS support is now
> mandatory for both servers and clients.

I'm afraid I missed the part where the use of STARTTLS became
mandatory.  Fortunately it's generally accepted that STARTTLS
requires a new CAPABILITY announcement anyway, thus if it can 
be shown that all implementations of both clients and servers 
not only support but MUST also use STARTTLS whenever it's 
available then I'd probably accept this as a solution that 
will work.

It'd be harder to debug since you will now need to get something 
more intelligent than telnet to test against, but I'm sure 
there'll be tools to help with that (perhaps even an updated telnet).

Currently what's on the table is a Client Capability string.
While I disagree that being able to select a VHOST context
isn't a capability I'm not really opposed to anything that
solves the whole problem.

-- Michael --

Date: Wed, 30 Apr 2003 05:37:47 +0530
From: Abhijit Menon-Sen <[email-masked]>
To: [email-masked]
Cc: Michael Fair <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

At 2003-04-30 00:38:48 +0200, [email-masked] wrote:
>
> Support draft-ietf-tls-extensions-06.txt in your STARTTLS
> implementation.
> 
> This is not a IMAP specific problem, so a IMAP specific solution is
> not the best solution, engineering wise.

I strongly prefer this approach over an optional argument to CAPABILITY,
or a separate VHOST command, particularly since STARTTLS support is now
mandatory for both servers and clients.

While a client capability string may be a good idea for other reasons, I
don't think a virtual host selection mechanism should be shoehorned into
it. It may be quite convenient, but it isn't a capability, and shouldn't
be treated as one.

-- ams

Subject: Re: IMAP Capability and Virtual Domains
From: Timo Sirainen <[email-masked]>
To: Michael Fair <[email-masked]>
Cc: Philip Guenther <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Organization: 
Date: 30 Apr 2003 02:49:51 +0300
Sender: [email-masked]

On Wed, 2003-04-30 at 01:59, Michael Fair wrote:
> I actually happen to like Cyrus Daboo's latest proposal
> of doing something like:
> CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

Fine with me, except that should be more IMAP way to avoid creating new
parsers for server:

* OK [CLIENTCAPABILITY] (or something)
1 CAPABILITY VHOST (fqdn.example.com) RLIST LANGUAGE (en_US en_GB)

Server could just ignore capabilities it isn't interested in.

From: "Michael Fair" <[email-masked]>
To: "Cyrus Daboo" <[email-masked]>, <[email-masked]>
Cc: "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 16:11:50 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Hi Michael,
> 
> --On Tuesday, April 29, 2003 3:01 PM -0700 Michael Fair 
> <[email-masked]> wrote:
> 
> | The one reservation I have about using a separate VHOST command
> | that returns the same response as CAPABILITY is that you've now
> | created to commands that do essentailly the exact same thing.
> 
> Fine, but if we are going to extend capability lets do it in a way that 
> will allow future parameters to be easily added, e.g. use the capability 
> response syntax for the client command:
> 
> a CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

That would be great!
Servers that don't care can still swallow it, and other perceived
usefulness results.  Great call!

So what do we put in the HELLO response to give the client
foreknowledge for accepting the extra info?

Should we follow the SMTP example and prepend hello with
[ECAPABILITY] and then create an ECAPABILITY command?
Or are people comfortable with keeping the same named
CAPABILITY command?

I'm personally rather partial to not change the command
name.  But then again, I'm also the one who doesn't mind 
a client trying and failing as a method for determining
whether or not this server supports what will be the norm.

-- Michael --

From: "Michael Fair" <[email-masked]>
To: "Timo Sirainen" <[email-masked]>
Cc: "Philip Guenther" <[email-masked]>, <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 15:59:17 -0700
Organization: DaClubhouse
Sender: [email-masked]

> On Tue, 2003-04-29 at 23:51, Michael Fair wrote:
> > But you also mentioned PREAUTH the and intitial OK.  Let's talk 
> > about the initial OK for a second.  This is acutally pretty cool 
> > and something I hadn't thought about.  If we leave VHOST as part 
> > of the CAPABILITY announcement we can preinform a client about
> > whether or not the server supports domain specific configs.
> 
> This sounds good to me. To save some roundtrips, it could include
> capability in it's reply as well:
> 
> * OK [VHOST] IMAP server ready.
> 1 VHOST foo.bar.org
> 1 OK [CAPABILITY IMAP4rev1 ...]
> 2 STARTTLS
> 2 OK
> ..etc..

If you're going to bother with a VHOST command that returns
a CAPABILITY response why not just add it to CAPABILITY and
make the argument valid SOLELY in the presence of a preannounced
VHOST capability.  It keeps the command set down and reduces
the number of different commands that essentially do the exact
same things.
 
For instance, what's the difference between the above and:

* OK [VHOST] IMAP server ready.
1 CAPABILITY foo.bar.org
1 OK [CAPABILITY IMAP4rev1 ...]
2 STARTTLS
2 OK
..etc..

I actually happen to like Cyrus Daboo's latest proposal
of doing something like:
CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

-- Michael --

Date: Wed, 30 Apr 2003 00:43:25 +0200
From: Sebastian Hagedorn <[email-masked]>
To: Michael Fair <[email-masked]>
cc: [email-masked]
Subject: Re: IMAP Capability and Virtual Domains 
Message-Context: text-message
Sender: [email-masked]

--==========2147493761==========
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

-- Michael Fair <[email-masked]> is rumored to have mumbled on=20
Dienstag, 29. April 2003 13:51 Uhr -0700 regarding Re: IMAP Capability and=20
Virtual Domains :

> For instance Outlook[ Express] only supports LOGIN.  I wouldn't
> want to enable that except for domains that required it.

Just FYI: the most recent versions of Cyrus SASL and Cyrus IMAP offer NTLM=20
as an option, which is working pretty well with Outlook (Express) in my=20
experience. I know, it's not a standard, but if I have to choose between=20
LOGIN and that I prefer the latter ...

Cheers, Sebastian
--
Sebastian Hagedorn M.A. - Postmaster - RZKR-R1 (Flachbau), Zimmer 18
Zentrum f=FCr angewandte Informatik - Universit=E4tsweiter Service RRZK
Universit=E4t zu K=F6ln / Cologne University - Tel. +49-221-478-5587
--==========2147493761==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========2147493761==========--

To: "Michael Fair" <[email-masked]>
Cc: <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
From: Simon Josefsson <[email-masked]>
Date: Wed, 30 Apr 2003 00:38:48 +0200
Sender: [email-masked]

"Michael Fair" <[email-masked]> writes:

> The problem with this solution is that there are several
> things that happen before LOGIN or AUTH where knowing the
> domain would be useful.  Namely STARTTLS and CAPABILITY.

Support draft-ietf-tls-extensions-06.txt in your STARTTLS
implementation.

This is not a IMAP specific problem, so a IMAP specific solution is
not the best solution, engineering wise.

A good authentication mechanism should have a means to allow clients
to chose a "realm" they belong to.  DIGEST-MD5 have this, as does the
experimental KERBEROS_V5.

Date: Tue, 29 Apr 2003 18:32:24 -0400
From: Cyrus Daboo <[email-masked]>
To: Michael Fair <[email-masked]>, [email-masked]
cc: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

Hi Michael,

--On Tuesday, April 29, 2003 3:01 PM -0700 Michael Fair 
<[email-masked]> wrote:

| The one reservation I have about using a separate VHOST command
| that returns the same response as CAPABILITY is that you've now
| created to commands that do essentailly the exact same thing.

Fine, but if we are going to extend capability lets do it in a way that 
will allow future parameters to be easily added, e.g. use the capability 
response syntax for the client command:

a CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

-- 
Cyrus Daboo

From: "Michael Fair" <[email-masked]>
To: "Rob Siemborski" <[email-masked]>
Cc: "Philip Guenther" <[email-masked]>, <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 15:15:03 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Well, I've sort of been interested in a client-side capability string (how
> does a server know that a client supports referrals if it doesn't execute
> an RLIST first, for example?).  In the case of referrals though, I suppose
> an [L01 RLIST "" ""] is just as good as a client-side capability string,
> since clients don't currently support it anyway.

If a client side capability string would be something that would
be generally useful, I would certainly be one of its supporters.

It's one of those things that seems like it would be generally
useful to me and would become more useful over time as people
found ways to do things they previously couldn't, but that is
more theory than fact at this point.

If there are convincing enough arguments to sway the protocol
designers to include, then I would propose the VHOST functionality
be moved as part of that functionality.

-- Michael --

Subject: Re: IMAP Capability and Virtual Domains
From: Timo Sirainen <[email-masked]>
To: Michael Fair <[email-masked]>
Cc: Philip Guenther <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Organization: 
Date: 30 Apr 2003 01:04:35 +0300
Sender: [email-masked]

On Tue, 2003-04-29 at 23:51, Michael Fair wrote:
> But you also mentioned PREAUTH the and intitial OK.  Let's talk 
> about the initial OK for a second.  This is acutally pretty cool 
> and something I hadn't thought about.  If we leave VHOST as part 
> of the CAPABILITY announcement we can preinform a client about
> whether or not the server supports domain specific configs.

This sounds good to me. To save some roundtrips, it could include
capability in it's reply as well:

* OK [VHOST] IMAP server ready.
1 VHOST foo.bar.org
1 OK [CAPABILITY IMAP4rev1 ...]
2 STARTTLS
2 OK
..etc..

Non-vhost capable client simply works as before.

Date: Tue, 29 Apr 2003 18:03:34 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Sender: [email-masked]

On Tue, 29 Apr 2003, Mark Crispin wrote:

> On Tue, 29 Apr 2003, Rob Siemborski wrote:
> > In the case of referrals though, I suppose
> > an [L01 RLIST "" ""] is just as good as a client-side capability string,
> > since clients don't currently support it anyway.
>
> Clients don't currently support referrals?
>
> That's news to me; Pine has supported referrals for years.

Sorry, I meant to say that the protocol doesn't currently support a
client-side capability string, so for the case of mailbox referrals, a
client-side capability string is just as good as sending RLIST "" "".

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

From: "Michael Fair" <[email-masked]>
To: "Cyrus Daboo" <[email-masked]>, <[email-masked]>
Cc: "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 15:01:32 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Here is what I propose as an alternative, assuming VHOST is going to be 
> implemented:
> 
> 1) A VHOST aware server returns VHOST as the only capability in the 
> response code for the greeting string. This avoids the server having to 
> advertise any capabilites before its got the VHOST.

But this also kills the servers ability to be backwards compatible
and offer a "default domain" which would be the equivalent of 
whatever its pre-VHOST mail store mailboxes were.

You can still put VHOST in the response code of the greeting string
and it will still be useful, but I would augment this to say that it
would be included as one of the capabilities listed (but still 
possibly the only one for a VHOST only environment).  

> 2) A VHOST aware client that sees VHOST in the greeting capability MUST 
> immediately issue a VHOST command with the server fqdn as a parameter.
>
> 3) The server responds to the command with a full capability response 
> listing the capabilites now available for that virtual domain.

This simple creates an alternate CAPABILITY command.

How about we keep (1) as augmented above, but for 2 use:
2) A VHOST aware client that sees CHOST in the greeting capability
MUST immediately issue a CAPABILITY command with the server fqdn
as a parameter.  A client MUST NOT issue an argument to CAPABILITY
if it has not previously seen a VHOST capability.

The one reservation I have about using a separate VHOST command
that returns the same response as CAPABILITY is that you've now
created to commands that do essentailly the exact same thing.

I don't mind requiring servers to announce VHOST as part of the
greeting response at all.  I like the feature and if that is 
what it takes to satisfy a "client always has foreknowledge"
semantic.  Then I can live with that.

-- Michael --

Date: Tue, 29 Apr 2003 14:52:45 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Rob Siemborski <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 29 Apr 2003, Rob Siemborski wrote:
> In the case of referrals though, I suppose
> an [L01 RLIST "" ""] is just as good as a client-side capability string,
> since clients don't currently support it anyway.

Clients don't currently support referrals?

That's news to me; Pine has supported referrals for years.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Tue, 29 Apr 2003 14:45:03 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 29 Apr 2003, Cyrus Daboo wrote:
> Other issues not withstanding, I do not like the idea of an optional
> argument to capability, or using a VHOST command before issuing capability.

Agreed, for the reasons you state.

> 1) A VHOST aware server returns VHOST as the only capability in the
> response code for the greeting string. This avoids the server having to
> advertise any capabilites before its got the VHOST.

No good.  A VHOST-unaware client will issue a plaintext LOGIN command
unless the client itself was configured not to do such a thing.

That gets into including LOGINDISABLED, which in turn results in other
implications.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Tue, 29 Apr 2003 17:32:53 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Michael Fair <[email-masked]>
cc: Philip Guenther <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Sender: [email-masked]

On Tue, 29 Apr 2003, Michael Fair wrote:

> I'm still assuming that people on this list are still
> somewhat dubious about whether or not virtual hosting
> multiple domains on a single IP address is even useful.

I agree that virtual hosting on a single IP is useful.  I'm personally
unconvinced of the necessity of a VHOST command (or other additional
argument to CAPABILITY) to do this, its almost entirely possible to do
this using realms on user ids.

The only problem using realms with user IDs doesn't solve is the fact that
you don't know which certificate to present during a STARTTLS.  I suppose
this can be solved with an argument to STARTTLS.  You'd need to advertise
a new STARTTLS_WITH_DOMAIN capability too.

[As an aside, when I worked for a web hosting firm back in '97-99, I know
we were using an IP-per-domain for HTTPS.  Is this still the case?  (I
don't immediately know of any new solutions but I could be wrong).]

Of course, you now have an issue where some of your customers may not be
able to advertise STARTTLS (they don't have certs, and don't want junk
certs).  Maybe its good enough just to respond NO when these clients try
to do STARTTLS "badcustomer.com"

I agree with Cyrus's comments that placing a requirement that the client
assume a server supports something before issuing a CAPABILITY command is
bad.

> I'll tell you what would be even more useful is just a
> general command where the client can pass a set of
> generic key value pairs to inform the server of generic
> session parameters where clients and servers can be
> modified to look for and send arbitrary info.

Well, I've sort of been interested in a client-side capability string (how
does a server know that a client supports referrals if it doesn't execute
an RLIST first, for example?).  In the case of referrals though, I suppose
an [L01 RLIST "" ""] is just as good as a client-side capability string,
since clients don't currently support it anyway.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Tue, 29 Apr 2003 17:27:34 -0400
From: Cyrus Daboo <[email-masked]>
To: Michael Fair <[email-masked]>, [email-masked]
cc: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

Hi Michael,

--On Tuesday, April 29, 2003 12:30 AM -0700 Michael Fair 
<[email-masked]> wrote:

| I had an offline discussion with Mark Crispin and proposed
| adding the domain name as an optional parameter to CAPABILITY.
|
| My assumption was that the very first thing an IMAP client
| is going to do (even before STARTTLS) is CAPABILITY to find
| out if the server can even do STARTTLS and to figure out
| what methods of authentication the server makes available.

Other issues not withstanding, I do not like the idea of an optional 
argument to capability, or using a VHOST command before issuing capability. 
If you do that, how is a client to know that that is a valid thing to do 
with that server? The user would have to provide some configuration switch 
to enable it and that requires one extra step for the user that should be 
avoided.

Here is what I propose as an alternative, assuming VHOST is going to be 
implemented:

1) A VHOST aware server returns VHOST as the only capability in the 
response code for the greeting string. This avoids the server having to 
advertise any capabilites before its got the VHOST.
2) A VHOST aware client that sees VHOST in the greeting capability MUST 
immediately issue a VHOST command with the server fqdn as a parameter.
3) The server responds to the command with a full capability response 
listing the capabilites now available for that virtual domain.

This goes like:

s: * OK [CAPABILITY imap4rev1 VHOST]
c: a VHOST fqdn.example.com
s: * CAPABILITY imap4rev1 starttls logindisabled auth=gssapi
s: a OK

I believe this approach minimises round trips and doesn't require the 
client to guess as to whether an extended capability command is present or 
not.

-- 
Cyrus Daboo

From: "Michael Fair" <[email-masked]>
To: "Philip Guenther" <[email-masked]>
Cc: <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 13:51:22 -0700
Organization: DaClubhouse
Sender: [email-masked]

> There are three 'issues' that the VHOST capability claims to address:
> 1) the ad-hoc interpretation of authentication IDs as specifying user
>    and domain 
> 2) the ability to have domain-specific capability lists
> 3) the choice of STARTTLS certificate by the server
> 
> In my opinion, the first of these is a non-issue in practice.  Indeed,
> the VHOST extension leaves the client author in a sticky situation: if
> it accepts the authentication ID as a single value ready for presenting
> to LOGIN (or AUTHENTICATE PLAIN/CRAM-MD5/etc), then it must decide and
> document how it splits that into a domain for VHOST and a username for
> LOGIN.  If it accepts the authentication ID as separate domain and user
> values, then it must guess how to join them for non-VHOST-enabled
> servers, or have another configuration parameter for _that_.  The VHOST
> extension would also need to specify how it affects and/or operates with
> SASL mechanisms that convey the user's domain separate from their
> username.

The first of these is not a non-issue, but I grant is more of
a minor annoyance than an issue.  However, it forces administrators
to dictate certain parts of customer usernames, where our preference
would be to allow the customers to have complete control over that
namespace.  It save us and them time (and money).

As for the client detecting the domain name, the client wouldn't 
split anything.  Whatever was specified in the "IMAP server" 
field would be presented wholesale without modification as the 
FQDN for the VHOST or CAPABILITY argument to specify the 
targetted domain.  It's a server and admin issue to make sure 
that whatever the client has as its server address is appropriately 
mapped internally.

Think HTTP 1.1 and Apache Name Based Virtual Hosts if you have any
experience with that.  The servername is presented unmodified as
part of the header to the request.

> The second issue appears minor to me.  Is it more than a matter of
> differing domains being allowed different SASL mechanisms?  Is that a
> practical problem?

This question is equivalent to "Why should any server ever offer 
a different CAPABILITY line?"

Think ISP, think a 1,000 unique domains all on one IP, think 600 
or more different companies all being hosted on the same machine.  
Each of those 1,000 domains deserves to be treated as if it had 
its own dedicated server.  That currently requires 1,000 IP 
addresses.  The goal here is simply to give the IMAP server the
name of the server the client was trying to reach as early as
possible so it can look up a config for that server and present 
it.  Each domain should be as independantly configurable as a
standalone server would be (with the exception of anything that
happens on the wire before the client has a chance to speak).

However, to address your direct question about the second issue,
most of it surrounds SASL mechanisms, but tuning CAPABILITY based
on the preferred clients in each domain is also an issue.  

Oftentimes the only way to get the desired behavior is to add or
remove things from the CAPABILITY line.  Other times, you don't want
to offer certain CAPABILITY options unless you absolutely have to.
For instance Outlook[ Express] only supports LOGIN.  I wouldn't
want to enable that except for domains that required it.

> I agree that the third issue is a real problem.  IMHO, STARTTLS would
> have been better defined to take an "fqdn" argument that the server
> could use to decide which certificate to use.  An extension that did
> only that---add an optional domain argument to STARTTLS---would appear
> to provide most of the benefits of VHOST.

I agree that STARTTLS would be better if the only issue were 
choosing a certificate.  Perhaps the proposal should be expanded
to add an argument to both CAPABILITY and STARTTLS.  I chose 
CAPABILITY because the domain is is needed for all three issues 
listed, not just (3).  I personally figure if you're going to
bother fixing, you should look into fixing all of it not just
propping up the most important piece.  The cost of adding an 
argument is the same whether it is to STARTTLS or to CAPABILITY 
or to adding a new command.  Things break, or you get more round 
trips.  My goal was to get the server the info as early as possible.

Not all clients issue a STARTTLS, nor should all be required to
issue a STARTTLS just to get per domain information.  If all
clients issued a STARTTLS then I agree adding a domain argument
to STARTTLS would solve the problem.  However, not all clients
do that.  But, all clients do issue a CAPABILITY.  Adding a domain
argument to that would solve the problem neatly.  I will address
the "two round trips for failure" below.

> It seems to me that the CAPABILITY command is ill-suited for such
> treatment.  If the client already has the server's list of capabilities
> and therefore knows that the server supports the extension, then
> overloading the CAPABILITY command doesn't save you anything.  

Right, adding a domain argument to CAPABILITY is intended to 
completely replace VHOST.  A client would issue the extended
version and fall back in the event of BAD.  If this was just
to figure out if the server supported the VHOST command then
addidng a domain argument to CAPABILITY would be senseless.
My goal with adding the domain argument to CAPABILITY is to
reduce the number of round trips a client has to make.

> If the
> client doesn't already have the server's list of capabilities, then the
> client will have to either try the extended version and fallback on BAD,
> or send an unextended request with a followup if the capability is
> advertised, either way requiring two roundtrips against some set of
> servers.  Leaving the VHOST command separate from CAPABILITY while
> strengthing the recommendation that the server include a CAPABILITY
> response code in the initial PREAUTH or OK response and that clients
> recognize such would appear to be a simpler and cleaner approach.

You seem to think that making clients issue a separate VHOST 
coammdn "two round trips for some set of servers".  You've just
inverted the set of servers that's all.  I think it's fair to
say that clients and servers of any proposal accepted would
gravitate towards supporting it as long as it cost them very 
little to add (which I have explicity designed the proposal for).

First, if we added a domain argument to CAPABILITY, then as my
proposal suggested, servers could be modified to simply swallow
it and go about their business.  So supporting the argument
is really trivial.  It does require a code change, and it does
require a release of a new server version which will take some
time.  So between now and then we have all the existing code 
that responds with "BAD" which is, and will remain, a correct
response for servers that don't support the VHOST capability.

In general, the clients that support this feature aren't going 
to be used against the servers that don't unless most future
versions of clients decide they should support this (which then
makes a compelling case for servers to at least swallow the 
argument), however there will be plenty of exceptions and for
a time these clients will get a BAD response from those servers.
Those exceptions will need to make two round trips, but over time
those exceptions will become fewer.

If you're interested in optimizing bandwidth, it quickly becomes
very clear that, if accepted, the general trend of servers and
clients will be to support the added domain argument to CAPABILITY
(or some other proposal that provides equivalent functionality).
It costs servers nothing but a little "ignore the argument" code.
It costs clients some "tack the server name on" code, and if the
response is "BAD" or "NO" then give up.  Granted those are the 
bare minimums for compliance, but they are fairly trivial.

But you also mentioned PREAUTH the and intitial OK.  Let's talk 
about the initial OK for a second.  This is acutally pretty cool 
and something I hadn't thought about.  If we leave VHOST as part 
of the CAPABILITY announcement we can preinform a client about
whether or not the server supports domain specific configs.
Very cool!  I'll be suggesting that in to my proposal.  Originally
I had intended on not doing anything with the CAPABILITY line
because I wasn't thinking that a server could just announce
its CAPABILITY without the client requesting it.

But now we need to decide between a separate VHOST command or 
an argument to CAPABILITY?

Here's a test case:
Client shows up at the server, the server dumps a CAPABILITY 
line down its throat the client sees a VHOST option, or the
Client shows up at the server, the server gives some kind of
hello and the Client is unaware of whether or not the server
supports the VHOST capability.

It then .....
A) [issues a CAPABILITY and looks for VHOST (if unknown)]
   issues a VHOST command 
   then issues another CAPABILITY
or
B) isssues a CAPABILITY with a domain argument
   [issues a CAPABILITY without a domain argument (if BAD)]

The only other proposal that would obviate the need for
a VHOST command to require a separate round trip would
be if VHOST returned the same response as CAPABILITY.
But now you've created two commands that do the exact same
thing and for what?  To protect existing servers from having
to trip over a malformed CAPABILITY line that the servers
are already prepared to ignore and that over time will learn
to accept (even if they just ignore the argument which they 
are perfectly allowed to do).

> ...
> >As an added bonus, this also lays the groundwork for 
> >some future planning on replicating and scaling IMAP 
> >mail stores to span multiple mail stores to provide 
> >geograhic redundancy.  Without this feature the "IP burn 
> >rate" jumps to 1+N addresses per domain where N is the 
> >number of geographically redundant mail stores.
> 
> I don't understand how geographically redundant mail stores are helped
> out any more than any set of N mail stores that offer STARTTLS.  If the
> problem is the requirement for separate IPs for STARTTLS, then giving
> STARTTLS an optional argument would "lower the burn rate" just as much
> as VHOST or "CAPABILITY domain", no?

Again you are somehow assuming that every conversation with 
every IMAP server issues a STARTTLS.  An IMAP server doesn't
even have to offer STARTTLS, most do because it's a good idea,
but then you've got the same optional aspect on the client side.

The real point is being able to connect to an IMAP server and
identify what domain context you want the conversation to
occur in.  If STARTTLS were as ubiquitous as CAPABILITY
and EVERY SESSION issued one, then I'd agree.  But it's not.

I also really don't understand why you would favor STARTTLS 
over CAPABILITY?  It makes no sense to me.  STARTTLS is only
issued in the context of creating a secure connection,
CAPABILITY is always offered.  STARTTLS is discovered
post CAPABILITY, so why not do it at the first opportunity?
Further, whether or not STARTTLS is even offered may depend
completely on what domain the client was trying to reach.
(For instance if the customer hasn't provided a security
certificate to be used, then why should the server offer
that domain STARTTLS functionality?)

The point is to let the server switch configurations based
on the targetted domain.  STARTTLS doesn't give us that because
it isn't gauranteed to be issued.  CAPABILITY and VHOST are
both gauranteed to be issued if the client supports it.

An argument to STARTTLS, or an argument to CAPABILITY sounds
like six of one half a dozen of the other, it's still an
extra argument that gets added to something that currently
doesn't take one.

You've dismissed the first two benefits of the three as not
needed and then based on that came up with "oh only STARTTLS
is needed" where for no extra cost (since you gotta add 
something somewhere), if we add it to CAPABILITY we get all 
three.  Those of us who have actually had to deal with the 
issue, which by your dismissal of points 1 and 2 makes it 
obvious you haven't seriously had to deal with it, believe 1 
and 2 are important and have suggested a solution which comes 
out to costing the same as an argument to STARTTLS, and 
provides the same benefit as an argument to STARTTLS, but 
also provides extra benefits that an argument to STARTTLS 
doesn't offer us (otherwise we probably would have proposed 
it that way to begin with).

While I will certainly agree that STARTTLS and SASL mechanisms
are the primary purposes, there are other secondary gripes
that get handled by doing it via a VHOST or argument to
CAPABILITY.  I personally think that even VHOST is a waste.
It gives us nothing that an argument to CAPABILITY doesn't
give us, and it mandates extra round trips.  An argument
to CAPABILITY only causes _one_ extra round trip during the
transitional period where new clients try and access servers
that haven't been updated to ignore the argument yet.

If you haven't had to deal with hosting lots of domains
on a limit number of IPs then please don't assume you
know what the issues are and what's important and what's
not.  I've tried to make the proposal as simple as possible
_but no simpler_ to get the information we need.

I'm more than willing to explain and educate, and there
is a very large list of other things that you can start
doing once you have this type of functionality that I'm
refraining from mentioning because I believe that giving 
a huge laundry list of minor nice-to-haves distracts from
the core issues.

For instance one "nice-to-have" is to aggregate multiple 
domains to act as one single domain.  Why? you ask..  It
doesn't matter.  If you've been there you'd get it, and 
you haven't then you don't really need to know.  All you
need to know is that being able to aggregate multiple
domain names together to act as a single group is useful.
Not as useful as being able to make every domain its own
unique entity, and certainly not something all servers
are going to support, but for those people who need the
functionality, they're grateful that the protocol built 
in a space for a few extra bytes of information where
the client can announce the server it was trying to reach.

I'm still assuming that people on this list are still
somewhat dubious about whether or not virtual hosting
multiple domains on a single IP address is even useful.

If that's still the case, then I'll be more than happy
to take arguments and suggestions to find some other way
to solve the problem until we can all agree.  Believe
me when I say I'll be more than happy to listen because 
if there is a way around it then hopefully it can be 
applied to lots of protocols that lack this support, not
just IMAP and the host providers of the world won't have 
to wait for new versions of clients and servers before 
they can start rolling this out.  

However, I'll warn you, I've been working on the problem 
for six years, I've worked with lots of really smart 
networking engineers and system administrators who have 
also been working on it for several years (some longer 
than I have), and so far, no one in the entire hosting 
industry has been unable to find a generally useful 
solution that isn't based on the protocol providing a 
special place where the intended server name is provided.

I'll tell you what would be even more useful is just a
general command where the client can pass a set of 
generic key value pairs to inform the server of generic
session parameters where clients and servers can be 
modified to look for and send arbitrary info.  Kind of
like the "X-" headers in SMTP.  That way the server name
could be stuffed into one of those, and it would not
require a protocol change... A simple formal registry
could maintain the most useful and widely accepted 
options that clients and server normally adopt.
However, it's not clear to me the overall usefulness
of the feature, whether it should be unidriectional or
bidirectional, and how complicated the command should be.
So for now, I'd be more than happy with just a server 
name submitted by the client to the server as early as
possible during the conversation.

-- Michael --

Date: Tue, 29 Apr 2003 12:06:17 -0700 (PDT)
From: [email-masked]
Subject: Review requested: draft-siemborski-mupdate-03.txt to be published as an experimental RFC
To: [email-masked]
Cc: [email-masked], [email-masked]
Sender: [email-masked]

I've received a request from the author to publish

    The MUPDATE Distributed Mailbox Database Protocol
    draft-siemborski-mupdate-03.txt

as an experimental RFC. It is customary for area directors to ask for
such documents to be reviewed by one or more working groups working in the
same or related area. Accordingly, I'm asking folks involved in IMAPEXT
to review this document and comment on its suitability for publication as
an experimental protocol.

TIA.

				Ned

P.S. I have already sent the author some feedback on missing boilerplate and
other formalities.

Date: Tue, 29 Apr 2003 13:09:24 -0400
From: Cyrus Daboo <[email-masked]>
To: Timo Sirainen <[email-masked]>, Philip Guenther <[email-masked]>
cc: Michael Fair <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

Hi Timo,

--On Tuesday, April 29, 2003 7:39 PM +0300 Timo Sirainen <[email-masked]> wrote:

| Related to 2, I've thought about giving different capability lists per
| user. Couldn't both of these be solved by letting server reply with
| untagged CAPABILITY reply (or resp-text-code) after login? All clients
| might not update their lists, but that's better than having
| extra/missing capabilities listed.

rfc3501 already states that servers may send a capability response code in 
the tagged OK to the AUTHENTICATE command. Servers can also send the 
capability response code in the greeting response too. If both of those are 
implemented, then a client would only need to use the CAPABILITY command 
after STARTTLS.

-- 
Cyrus Daboo

Subject: Re: IMAP Capability and Virtual Domains
From: Timo Sirainen <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: Michael Fair <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Organization: 
Date: 29 Apr 2003 19:39:03 +0300
Sender: [email-masked]

On Tue, 2003-04-29 at 14:36, Philip Guenther wrote:
> There are three 'issues' that the VHOST capability claims to address:
> 1) the ad-hoc interpretation of authentication IDs as specifying user
>    and domain 
> 2) the ability to have domain-specific capability lists
> 3) the choice of STARTTLS certificate by the server

Related to 2, I've thought about giving different capability lists per
user. Couldn't both of these be solved by letting server reply with
untagged CAPABILITY reply (or resp-text-code) after login? All clients
might not update their lists, but that's better than having
extra/missing capabilities listed.

From: Arnt Gulbrandsen <[email-masked]>
To: Philip Guenther <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date:   Tue, 29 Apr 2003 13:58:50 +0200
Sender: [email-masked]

Philip Guenther writes:
> Arnt Gulbrandsen <[email-masked]> writes:
>> As long as VHOST does not block, it can be added almost anywere. I'd 
>> add it in step E: "send VHOST and STARTTLS".
>
> Imap servers are allowed to reorder pipelined commands, so that won't work.

I'd quibble... but you're effectively right. Pity.

> (The tagged OK of the STARTTLS command is sent _before_ TLS 
> negotiation and is therefore not protected.)

(The server can send an unsolicited untagged OK afterwards. ISTR having 
seen one do. UW, I think... but it's at least a year ago.)

OK. VHOST must add another round-trip. Then I dislike it even more.

How about using something like 'STARTTLS VHOST=fqdn'?

--Arnt

To: Arnt Gulbrandsen <[email-masked]>
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
From: Philip Guenther <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 04:40:49 -0700
Sender: [email-masked]

Arnt Gulbrandsen <[email-masked]> writes:
>
>Harrie Hazewinkel writes:
>> In such a case I believe we could 'convert' CAPABILITY by ading 
>> 'VHOST' semantics when the server is before the STARTTLS and/or 
>> AUTH/LOGIN.
>
>As long as VHOST does not block, it can be added almost anywere. I'd add 
>it in step E: "send VHOST and STARTTLS".

Imap servers are allowed to reorder pipelined commands, so that won't
work.

>> What I neither understand why was the response of a CAPABILITY not 
>> added to the response of STARTTLS for example. Allowing that would 
>> have reduced the extra CAPABILITY commands and the associated 
>> waiting.
>
>It is. RFC 3501 resp-code-text includes capability-data, and section 
>7.2.1 describes how to do it. Recent versions of UW imapd use that.

rfc3501, section 6.2.1:
      Once [TLS] has been started, the client MUST discard cached
      information about server capabilities and SHOULD re-issue the
      CAPABILITY command.  This is necessary to protect against man-in-
      the-middle attacks which alter the capabilities list prior to
      STARTTLS.  The server MAY advertise different capabilities after
      STARTTLS.

(The tagged OK of the STARTTLS command is sent _before_ TLS negotiation
and is therefore not protected.)

Philip Guenther
[email-masked]

To: "Michael Fair" <[email-masked]>
Cc: [email-masked], "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 04:36:50 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

"Michael Fair" <[email-masked]> writes:
...
>Current single IP implementations pack the domain info in
>the usernames and then do some magic after authentication to 
>create the domain specific view.  How they pack the domain
>info varies from site to site, but the username is the
>only client submitted information that can reliably be
>used to identify the targetted domain.
>
>The problem with this solution is that there are several
>things that happen before LOGIN or AUTH where knowing the
>domain would be useful.  Namely STARTTLS and CAPABILITY.

There are three 'issues' that the VHOST capability claims to address:
1) the ad-hoc interpretation of authentication IDs as specifying user
   and domain 
2) the ability to have domain-specific capability lists
3) the choice of STARTTLS certificate by the server

In my opinion, the first of these is a non-issue in practice.  Indeed,
the VHOST extension leaves the client author in a sticky situation: if
it accepts the authentication ID as a single value ready for presenting
to LOGIN (or AUTHENTICATE PLAIN/CRAM-MD5/etc), then it must decide and
document how it splits that into a domain for VHOST and a username for
LOGIN.  If it accepts the authentication ID as separate domain and user
values, then it must guess how to join them for non-VHOST-enabled
servers, or have another configuration parameter for _that_.  The VHOST
extension would also need to specify how it affects and/or operates with
SASL mechanisms that convey the user's domain separate from their
username.

The second issue appears minor to me.  Is it more than a matter of
differing domains being allowed different SASL mechanisms?  Is that a
practical problem?

I agree that the third issue is a real problem.  IMHO, STARTTLS would
have been better defined to take an "fqdn" argument that the server
could use to decide which certificate to use.  An extension that did
only that---add an optional domain argument to STARTTLS---would appear
to provide most of the benefits of VHOST.

>The current IMAP extension draft adds a "VHOST" command
>to specify which virtual host the client was trying to reach.
>Overall I think it's a good proposal, but I believe the
>same result can be accomplished without the need for adding
>a new command.
>
>I had an offline discussion with Mark Crispin and proposed
>adding the domain name as an optional parameter to CAPABILITY.

It seems to me that the CAPABILITY command is ill-suited for such
treatment.  If the client already has the server's list of capabilities
and therefore knows that the server supports the extension, then
overloading the CAPABILITY command doesn't save you anything.  If the
client doesn't already have the server's list of capabilities, then the
client will have to either try the extended version and fallback on BAD,
or send an unextended request with a followup if the capability is
advertised, either way requiring two roundtrips against some set of
servers.  Leaving the VHOST command separate from CAPABILITY while
strengthing the recommendation that the server include a CAPABILITY
response code in the initial PREAUTH or OK response and that clients
recognize such would appear to be a simpler and cleaner approach.

...
>As an added bonus, this also lays the groundwork for 
>some future planning on replicating and scaling IMAP 
>mail stores to span multiple mail stores to provide 
>geograhic redundancy.  Without this feature the "IP burn 
>rate" jumps to 1+N addresses per domain where N is the 
>number of geographically redundant mail stores.

I don't understand how geographically redundant mail stores are helped
out any more than any set of N mail stores that offer STARTTLS.  If the
problem is the requirement for separate IPs for STARTTLS, then giving
STARTTLS an optional argument would "lower the burn rate" just as much
as VHOST or "CAPABILITY domain", no?

Philip Guenther
[email-masked]

From: Arnt Gulbrandsen <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: IMAP Extensions WG <[email-masked]>
Date:   Tue, 29 Apr 2003 12:02:21 +0200
Sender: [email-masked]

Harrie Hazewinkel writes:
> In such a case I believe we could 'convert' CAPABILITY by ading 
> 'VHOST' semantics when the server is before the STARTTLS and/or 
> AUTH/LOGIN.

As long as VHOST does not block, it can be added almost anywere. I'd add 
it in step E: "send VHOST and STARTTLS".

> What I neither understand why was the response of a CAPABILITY not 
> added to the response of STARTTLS for example. Allowing that would 
> have reduced the extra CAPABILITY commands and the associated 
> waiting.

It is. RFC 3501 resp-code-text includes capability-data, and section 
7.2.1 describes how to do it. Recent versions of UW imapd use that.

> Adding a fifth wait is a trade-off compared to the extra functionality on
> can get.

Explain.

--Arnt

Date: Tue, 29 Apr 2003 11:46:49 +0200
Subject: Re: IMAP Capability and Virtual Domains
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
From: Harrie Hazewinkel <[email-masked]>
Sender: [email-masked]

On Tuesday, April 29, 2003, at 11:26 AM, Arnt Gulbrandsen wrote:

> Harrie Hazewinkel writes:
>> Correct me if I am wrong, but "do clients not have to wait for 
>> STARTTLS to finish also before a login procedure can be given??"
>
> A client must wait for STARTTLS, yes. Regrettable, but given the 
> nature of the command it's unavoidable.
>
> Today, with most servers, I have to
>
> a) connect
> b) wait until I see the IMAP banner
> c) send CAPABILITY
> d) wait for the answer
> e) send STARTTLS (or switch to port 993 and repeat from step a)
> f) wait for the answer
> g) do TLS negotiation
> h) send CAPABILITY
> i) wait for the answer
> j) send AUTH or LOGIN
> k) wait for the answer
>
> Steps d, f, i and k add up to more than enough waiting. Adding a fifth 
> wait is not a good idea.

In such a case I believe we could 'convert' CAPABILITY by ading 'VHOST' 
semantics
when the server is before the STARTTLS and/or AUTH/LOGIN.
What I neither understand why was the response of a CAPABILITY not added
to the response of STARTTLS for example. Allowing that would have 
reduced the
extra CAPABILITY commands and the associated waiting.

Adding a fifth wait is a trade-off compared to the extra functionality 
on
can get. In this case it is easier server administration that does not
directly benefit a user. :-((

Harrie

From: Arnt Gulbrandsen <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: IMAP Extensions WG <[email-masked]>
Date:   Tue, 29 Apr 2003 11:26:24 +0200
Sender: [email-masked]

Harrie Hazewinkel writes:
> Correct me if I am wrong, but "do clients not have to wait for 
> STARTTLS to finish also before a login procedure can be given??"

A client must wait for STARTTLS, yes. Regrettable, but given the nature 
of the command it's unavoidable.

Today, with most servers, I have to

a) connect
b) wait until I see the IMAP banner
c) send CAPABILITY
d) wait for the answer
e) send STARTTLS (or switch to port 993 and repeat from step a)
f) wait for the answer
g) do TLS negotiation
h) send CAPABILITY
i) wait for the answer
j) send AUTH or LOGIN
k) wait for the answer

Steps d, f, i and k add up to more than enough waiting. Adding a fifth 
wait is not a good idea.

--Arnt

Date: Tue, 29 Apr 2003 11:10:10 +0200
Subject: Re: IMAP Capability and Virtual Domains
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
From: Harrie Hazewinkel <[email-masked]>
Sender: [email-masked]

On Tuesday, April 29, 2003, at 10:51 AM, Arnt Gulbrandsen wrote:

> I don't much like VHOST. Its benefit over the current user@host hacks 
> seems limited to providing the right TLS certificate. But I could live 
> with it, provided it were simpler.
>
> a) Clients using VHOST should always send the IMAP server's FQDN as 
> argument. Nothing else.
>
> b) BAD should be sent only for syntax errors or command unknown.
>
> c) Clients should explicitly not have to wait for the answer before 
> proceeding with the login procedure.

Correct me if I am wrong, but "do clients not have to wait for STARTTLS 
to finish also
before a login procedure can be given??"

Harrie

From: Arnt Gulbrandsen <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date:   Tue, 29 Apr 2003 10:51:09 +0200
Sender: [email-masked]

I don't much like VHOST. Its benefit over the current user@host hacks 
seems limited to providing the right TLS certificate. But I could live 
with it, provided it were simpler.

a) Clients using VHOST should always send the IMAP server's FQDN as 
argument. Nothing else.

b) BAD should be sent only for syntax errors or command unknown.

c) Clients should explicitly not have to wait for the answer before 
proceeding with the login procedure.

--Arnt

From: "Michael Fair" <[email-masked]>
To: <[email-masked]>
Cc: "Harrie Hazewinkel" <[email-masked]>
Subject: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 00:30:46 -0700
Organization: DaClubhouse
Sender: [email-masked]

Hi all,

I understand that in general this list is not very actively
interested in scaling the IMAP protocol to support lots of 
domains on a single IP address.  Myself and I know a number
of others (mainly hosting companies) are interested in this 
feature.  Currently only HTTP 1.1 has any kind of support 
for this kind of scaling.

I'll be brief, a quick education on the issues for those that 
don't understand the problem or haven't read the VHOST extension 
draft and then a suggestion that I believe would quite neatly 
solve all the issues plus a couple that none of the existing 
solutions offer while still maintaing compatibility with 
existing clients/servers and allow for a mostly gradual 
transition between non-virtualized and virtualized deployment.

The issues:
At the core of it, in most hosting for hire environments
you typically get a lot of domains that only have a few
users (less than 30).  In a modest ISP, having to support
1,000 domains is very reasonable.  The vast majority of 
those domains typically have less than 5 accounts, and
the number with more than 30 accounts is usually countable
on one or two hands.  This creates a situation where you
have about 10,000-20,000 accounts spread over 1,000 domains.

This is easily hosted on any decent sized server and hosting
this on a single machine is a very reasonable thing to do.
The problem is that this ISP typically has a /21 giving
it a total of 1,275 IP addresses.  If this ISP was to
try and give the highest quality of service at 1 IP
address per domain, which the existing protocol calls 
for, this ISP would have 1,000 IP addresses all pointed
at the same server and have 273 left over for other uses.
In the case of the medium size ISP it is actually impossible 
to even consider offering such a service.  This is especially 
compounded if you start to include other services like FTP.  

As another consideration this doesn't even touch trying to 
provide geographically redundant services where the data is 
accessible from at least two (preferably three) redundant 
sites that are located on completely separate IP networks.
(This means at least two preferably three IPs per domain)

So the short of it is that for certain very common 
environments there is a clear and definite need for being 
able to multiplex multiple domains onto a single IP address.

A brief overview on the current state of trying to 
accomplish this:

When a client shows up at the server's door all it knows 
is the IP address the client was trying to reach.  This 
is completely insufficient to determine the targetted domain 
unless you have 1 IP per domain.

In short, the server must be told directly what domain
the client was trying to reach before it can access any
domain specific parameters (these include shared folders,
security certificates and any other parameters a server
might want to modify on a per domain basis (like AUTH
services)).  The other need for per domain identification
is to allow for [email-masked] to be differentiated 
from [email-masked] when the client only presents the 
username "bob".

Current single IP implementations pack the domain info in
the usernames and then do some magic after authentication to 
create the domain specific view.  How they pack the domain
info varies from site to site, but the username is the
only client submitted information that can reliably be
used to identify the targetted domain.

The problem with this solution is that there are several
things that happen before LOGIN or AUTH where knowing the
domain would be useful.  Namely STARTTLS and CAPABILITY.

The current IMAP extension draft adds a "VHOST" command
to specify which virtual host the client was trying to reach.
Overall I think it's a good proposal, but I believe the
same result can be accomplished without the need for adding
a new command.

I had an offline discussion with Mark Crispin and proposed
adding the domain name as an optional parameter to CAPABILITY.

My assumption was that the very first thing an IMAP client
is going to do (even before STARTTLS) is CAPABILITY to find
out if the server can even do STARTTLS and to figure out
what methods of authentication the server makes available.
By identifying the domain prior to any other commands,
the server has the opportunity to get into a domain
specific mode very early.  For instance, STARTTLS could
now provide a per domain certificate instead of being
forced to provide a single (usually ISP specific) cert,
or even worse, not offering IMAPS at all.

If that assumption is wrong, then I hope that someone would
correct me as it makes this suggestion rather worthless and
makes the VHOST addition a more viable solution.

Once a client had issued a CAPABILITY with a domain parameter,
the server would go into a domain specific mode where all 
responses would be local to that domain's config.  The main 
restrictions for a client would be that during a TLS session
and/or while in the AUTHENTICATED state the client MUST NOT
issue a CAPABILITY where that domain parameter is different.
If a client were allowed to jump domains in the middle of
those contexts it would be ambiguous at best, and create
security holes at worst.  The server is of course encouraged
to take whatever steps are appropriate to ensure that the
client doesn't try and jump domains in these contexts.  

Currently CAPABILITY accepts no parameters and it's an
error to present one.  Clients could try first with the
domain, and then upon receiving BAD try without the domain.

If a server supported the parameter, but not the requested
domain, the server could either respond with NO saying that
the domain isn't supported, or it could respond with YES 
if the server just swallows the parameter and gives the 
same behavior regardless of the domain.  This makes the 
existing servers behavior remain correct, allows for servers
to be compatible by simply swallowing the parameter and 
not doing anything with it, and future servers can implement
a true virtualized hosted environment.  If a client tries to
jump domain contexts, the server should answer "NO" if it 
would have answered "NO" to an unavailable or not supported 
domain, or it should simply swallow the parameter or respond 
with "BAD" if it would have swallowed the parameter or 
reposnded with "BAD" before.

Clients that support this need only be prepared to 
receive a BAD or NO when issuing a CAPABILITY and 
conform to not trying to jump domains when in a TLS 
session or AUTHENTICATED state.

As an added bonus, this also lays the groundwork for 
some future planning on replicating and scaling IMAP 
mail stores to span multiple mail stores to provide 
geograhic redundancy.  Without this feature the "IP burn 
rate" jumps to 1+N addresses per domain where N is the 
number of geographically redundant mail stores.  It's 
true that this might solveable with a distributed file 
system instead of an IMAP extension protocol, but I 
believe it would be harmful to ignore the idea that 
there are those in the world who care enough about 
reliability to use three or four distributed machines 
to maintain replicas of the mail store for performance 
and redundancy.

Thank you very much for reading and I look forward to
comments and feedback.  Again, I do understand that 
this is not a need that most people on this list consider
very important so taking the time out to lend some
grey matter to it is greatly appreciated.

-- Michael --

Subject: Re: Full text search and SEARCHM IMAP extention
From: Timo Sirainen <[email-masked]>
To: Takeshi Morishima <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 23 Apr 2003 15:09:05 +0300
Sender: [email-masked]

On Wed, 2003-04-23 at 06:28, Takeshi Morishima wrote:
> Will try to update the proposal modification based on the comments.  I
> need study LISTEXT more - don't fully understand how it's related or
> can be merged.

Actually, forget it. I was just thinking that since they added STATUS
support there, it could be used as some generic "for each listed
mailbox, do <stuff>". But it'd require replying with LIST replies, which
is probably not so useful with SEARCHM. And if result has to be sorted,
it's not so simple.

Date: Tue, 22 Apr 2003 22:28:57 -0500
From: [email-masked] (Takeshi Morishima)
To: [email-masked]
CC: [email-masked]
Subject: Re: Full text search and SEARCHM IMAP extention
Sender: [email-masked]

In message "Re: Full text search and SEARCHM IMAP extention" on
    2003/04/22, Timo Sirainen writes:
  > One thing that I've thought that could be useful is to return the text
  > that matched so client could display google-like search results directly
  > without fetching all the messages.

I'd agree that this may be useful.  This would be new data type
though, with mime content headers re-constructed for summary text.

  > I'm not sure if msgmax / msgstart is such a good idea. You
  > probably want it for prev/next page-like feature, but if new
  > matches appear between searches, you might miss some messages.

My initial motivation of these options was to allow client to specify
max number of messages to be returned.  Client could prepare only
resource that would be needed that way, thus more effcient use of
resources.  It also saves a bit of bandwidth of returning only 200 of
top score out of total 1000 hits.  Then suddenly I thought that if we
limit the amount of search result by msgmax, I would need msgstart in
case user is not satisfactory in the search result and want to get
next block of result, since otherwise the client may end up receiving
the same block of data repeatedly as it increases msgmax, which did
not sound very efficient.

For the issue of result changes between searches, I also thought about
it, and had one idea of using the search name to identify previous
search context so that the server can remember it and provide
consistent output.  This would require the client to use differnt
search name everytime new search is performed, the server side to
provide more resource, and will complicate things, which did not sound
like a good idea.  Since this modified searchm is intended for best
effort search, I thought that dup or missing messages may not be an
issue for typical case, resulted in this form.  (Probably no need of
msgend, just use msgstart+msgmax.)

Will try to update the proposal modification based on the comments.  I
need study LISTEXT more - don't fully understand how it's related or
can be merged.

Regards,
Takeshi

From: Arnt Gulbrandsen <[email-masked]>
To: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: IMAP Extensions WG <[email-masked]>
Date: Tue, 22 Apr 2003 23:28:29 +0200
Sender: [email-masked]

Mark Crispin writes:
> This is in effect an IMAP interface to an ultra-lightweight 
> notification protocol and facility that I am working on. The IMAP 
> server won't do any actual "watching"; instead, it will just be a 
> client to the notification facility.

Isn't that a natural implementation choice?

> I am interested in feedback as to whether this proposed extension is 
> interesting or not.

Yes. But as internet-draft, please.

> Also, would it be better to build an extension which has extensibility 
> but with restraints ("ultra-lightweight") on how it can be extended, 
> or if it should be strictly a new mail watching facility
...

If the latter, perhaps it can avoid the fate of so many similar proposals?

--Arnt

From: Arnt Gulbrandsen <[email-masked]>
To: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Tue, 22 Apr 2003 20:50:56 +0200
Sender: [email-masked]

Mark Crispin writes:
> On Tue, 22 Apr 2003, Arnt Gulbrandsen wrote:
>>  > LIST can never be more expensive than STATUS.
>>  I can disprove that by counterexample.
>>  C: b list "" *
>
> You are comparing apples and oranges. There is no such thing as a 
> wildcard STATUS.

Perhaps I misunderstood. I was comparing LIST to a for loop sending 
STATUS for e.g. 1800 mailboxes. Seemed a fair enough comparison.

--Arnt

Date: Tue, 22 Apr 2003 10:37:15 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 22 Apr 2003, Arnt Gulbrandsen wrote:
> > LIST can never be more expensive than STATUS.
> I can disprove that by counterexample.
> C: b list "" *

You are comparing apples and oranges.  There is no such thing as a
wildcard STATUS.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: Pending changes to CONDSTORE
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 22 Apr 2003 19:19:55 +0300
Sender: [email-masked]

On Tue, 2003-04-22 at 08:00, Alexey Melnikov wrote:
> I haven't resolved an open issue regarding atomicity of CONDSTORE for a
> message set. I am still waiting for more feedback.

Yes, it could be difficult with maildir at least since you can't prevent
flag changes by other clients. Is there even a reason to want atomicity?

Is there a reason to return MODSEQ separately for each message with
SILENT STORE? And why not return the modseq and flags directly for
modified messages, wouldn't client want to ask them anyway next?
Something like:

a STORE 100:150 (UNCHANGEDSINCE 1) +FLAGS.SILENT ($Processed)
* 101 FETCH (MODSEQ (2) FLAGS ($Processed))
* 105 FETCH (MODSEQ (2) FLAGS ($Processed))
a OK [MODSEQ 3 MODIFIED 101,105] Some flags weren't changed

b STORE 100 (UNCHANGEDSINCE 1) +FLAGS.SILENT ($Processed)
* 100 FETCH (MODSEQ (3) FLAGS ($Processed))
b OK [MODIFIED 100] No flags were changed

c STORE 100,151 (UNCHANGEDSINCE 1) +FLAGS ($Processed)
* 100 FETCH (MODSEQ (3) FLAGS ($Processed))
* 151 FETCH (MODSEQ (4) FLAGS ($Processed))
c OK [MODSEQ 4 MODIFIED 100] Some flags weren't changed

From: Arnt Gulbrandsen <[email-masked]>
To: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Tue, 22 Apr 2003 17:22:08 +0200
Sender: [email-masked]

Mark Crispin writes:
> LIST can never be more expensive than STATUS.

I can disprove that by counterexample.

I have a crappy IMAP client (came preinstalled on a machine I bought) 
whose login sequence is like this when it talks to wu-imapd:

C: a login arnt tnra
S: a ok
C: b list "" *
S: ...
C: (run out of memory and crash)

The server's resource use is limited only by bandwidth and the client's 
available RAM, which is potentially far more than it would spend 
processing STATUS.

--Arnt

Subject: Re: Full text search and SEARCHM IMAP extention
From: Timo Sirainen <[email-masked]>
To: Takeshi Morishima <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 22 Apr 2003 16:25:25 +0300
Sender: [email-masked]

On Tue, 2003-04-22 at 15:17, Takeshi Morishima wrote:
> My comments and proposed modification to the current SEARCHM draft are
> described in the following file:
> 
> http://www.ml-search.com/tmpdocs/draft-leiba-imap-search-multiple-01-comment.pdf
> 
> It proposes to allow server to ignore unsupported criteria/mailbox,
> provide sorted result fragmentation, more options to control search
> result, and a new HITS response to return search statistics numbers.

One thing that I've thought that could be useful is to return the text
that matched so client could display google-like search results directly
without fetching all the messages. This would be especially useful if
server supported searching inside non-text file attachments like ms word
docs, pdfs, postscripts, etc.

> Also, because the search will be performed outside of selected state,
> I thought that returning message sequence numbers may have a problem,
> and for other reasons, I'm proposing to return UIDs and UIDVALIDITY
> always in the SEARCHM response.

Right. Maybe remove the whole "UID SEARCHM".

Some other thoughts:

Sorting conflicts a bit with SORT extension. At least change "sentdate"
to "date" so they're compatible.

I'm not sure if msgmax / msgstart is such a good idea. You probably want
it for prev/next page-like feature, but if new matches appear between
searches, you might miss some messages.

I think "Mailbox URL" is wrong word to describe LIST-like mailbox
wildcarded names. And should this maybe be merged with LISTEXT?

Returning UIDVALIDITY before UIDs would be nicer. And it doesn't have to
be in paranthesis.

Anyway I'd like to have this kind of non-strict text searching. Current
SEARCH can be useful for client internally, but for normal searching by
user it's unnecessarily slow because it's strictness really limits the
optimizations that server can do.

Date: Tue, 22 Apr 2003 07:17:42 -0500
From: [email-masked] (Takeshi Morishima)
To: [email-masked]
Subject: Full text search and SEARCHM IMAP extention
Sender: [email-masked]

Hello there,

I've been looking into implementing a full text search on a client, to
provide a multiple mailbox search with search performance similar to
what we can find on many www based search engines.  After implementing
a repeat of select and search over multiple mailboxes, I found it slow
and in most cases the search result does not appear very optimal.  No
framework to handle something like scoring based search result
sorting, for example.

Looking through existing RFCs and imap extention drafts, the closest
to what I can use was the SEARCHM extention draft.  The current
SEARCHM draft, however, does not seem to be enough for the purpose of
the above type of search.  I would like this to be loosely coupled,
best effort and best practice search capable, instead of requiring a
strict listing for house keeping of client that SEARCHM is mainly
intended to.  A full text search would not need to deal with something
like "unseen" type of search, and can omit messages that unlikely of
user interest in a search result.

My comments and proposed modification to the current SEARCHM draft are
described in the following file:

http://www.ml-search.com/tmpdocs/draft-leiba-imap-search-multiple-01-comment.pdf

It proposes to allow server to ignore unsupported criteria/mailbox,
provide sorted result fragmentation, more options to control search
result, and a new HITS response to return search statistics numbers.
Also, because the search will be performed outside of selected state,
I thought that returning message sequence numbers may have a problem,
and for other reasons, I'm proposing to return UIDs and UIDVALIDITY
always in the SEARCHM response.

Since the purposes of SEARCHM and the modified version above slightly
differ, should it be different command?  Or is this something that can
be incorporated into the current searchm draft.

If this is not worth discussing in this WG list nor standardization,
please advice where I should take this to - any comments and
suggestions would be appreciated.

Regards,
Takeshi

To: IETF-Announce: ;
Cc: [email-masked]
From: [email-masked]
Reply-to: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-condstore-01.txt
Date: Tue, 22 Apr 2003 07:40:02 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: IMAP Extension for Conditional STORE operation
	Author(s)	: A. Melnikov, S. Hole
	Filename	: draft-ietf-imapext-condstore-01.txt
	Pages		: 16
	Date		: 2003-4-21
	
Often, multiple IMAP clients need to coordinate changes to a common
IMAP mailbox.  Examples include different clients for the same user,
and multiple users accessing shared mailboxes. These clients
need a mechanism to synchronize state changes for messages within the
mailbox. They must be able to guarantee that only one client can change
message state (e.g., message flags or annotations) at any time.  An
example of such an application is use of an IMAP mailbox as a message
queue with multiple dequeueing clients.
The Conditional Store facility provides a protected update mechanism for
message state information that can detect and resolve conflicts between
multiple writing mail clients.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-01.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-imapext-condstore-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-imapext-condstore-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-imapext-condstore-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-imapext-condstore-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Mon, 21 Apr 2003 23:00:06 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: IMAP Extensions WG <[email-masked]>
Subject: Pending changes to CONDSTORE
Sender: [email-masked]

I haven't resolved an open issue regarding atomicity of CONDSTORE for a
message set. I am still waiting for more feedback.

Also, I would like to propose to add a new capability if MODSEQ sort
criteria is implemented with SORT. Any objections to this?

Regards,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel
IETF standard
related pages: http://orthanc.ab.ca/mel/devel/Links.html

I speak for myself only, not for my employer.
__________________________________________

Date: Sat, 19 Apr 2003 23:49:27 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: IMAP Extensions WG <[email-masked]>
CC: IMAP Discusson List <[email-masked]>
Subject: Updated draft-ietf-imapext-condstore document
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------6D49FEEA8037F438115381F7
Content-Type: text/plain; charset=koi8-r
Content-Transfer-Encoding: 7bit

I've updated the document as per comments from Mark and Cyrus.
New version (draft-ietf-imapext-condstore-01) should appear shortly. In
the meantime, I've attached the diff for the changes.

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel
IETF standard
related pages: http://orthanc.ab.ca/mel/devel/Links.html

I speak for myself only, not for my employer.
__________________________________________

--------------6D49FEEA8037F438115381F7
Content-Type: text/plain; charset=koi8-r;
 name="00-01.diff.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="00-01.diff.txt"

--- draft-ietf-imapext-condstore-00.txt	Thu Mar 27 00:27:06 2003
+++ draft-ietf-imapext-condstore-01.txt	Fri Apr 11 23:29:42 2003
@@ -1,7 +1,7 @@
 Internet Draft: IMAP Extension for Conditional STORE          A. Melnikov
-Document: draft-ietf-imapext-condstore-00.txt                     S. Hole
-Expires: September 2003                     ACI WorldWide/MessagingDirect
-                                                               March 2003
+Document: draft-ietf-imapext-condstore-01.txt                     S. Hole
+Expires: October 2003                       ACI WorldWide/MessagingDirect
+                                                               April 2003
 
                IMAP Extension for Conditional STORE operation
 
@@ -43,6 +43,13 @@
 
 0.2. Change History
 
+     Changes from draft-ietf-imapext-condstore-00
+      1.  Dropped "/message" prefix in entry names as per decision in San Francisco.
+      2.  Fixed ABNF for SEARCH and SORT untagged responses.
+      3.  Changed "private" to "priv" to be consistent with ANNOTATE.
+      4.  MODIFIED response code is now returned in OK response, not NO.
+      5.  Added NOMODSEQ response code.
+
      Changes from draft-melnikov-imap-condstore-09:
       1.  Some text clarifications based on suggestions by Harrie Hazewinkel
       2.  Added paragraph about mailbox locking and DOS when conditional STORE
@@ -139,6 +146,7 @@
    4  IMAP Protocol Changes ..................................... X
    4.1 New OK untagged responses for SELECT and EXAMINE ......... X
    4.1.1 HIGHESTMODSEQ response code ............................ X
+   4.1.2 NOMODSEQ response code ................................. X
    4.2 STORE and UID STORE Commands ............................. X
    4.3 MODSEQ message data item in FETCH Command ................ X
    4.4 MODSEQ search criterion in SEARCH ........................ X
@@ -240,13 +248,18 @@
     different metadata items. If the server does so, per message mod-sequence is
     the highest mod-sequence of all metadata items for the specified message.
 
+    The server that supports this extention is not required to be able to store
+    mod-sequences for every available mailbox. Section 4.1.2 describes how
+    the server may act if a particular mailbox doesn't support the persistent
+    storage of mod-sequences.
+
     This extension makes the following changes to the IMAP4 protocol:
 
         a) extends the syntax of the STORE command to allow STORE
            modifiers
 
         b) adds the MODIFIED response code which should be used with
-           a NO response to the STORE command
+           an OK response to the STORE command
 
         c) adds a new MODSEQ message data item for use with the FETCH command
 
@@ -269,13 +282,26 @@
 
 4.1. New OK untagged responses for SELECT and EXAMINE
 
+    This document adds two new response codes HIGHESTMODSEQ and NOMODSEQ.
+    One of those response codes MUST be returned in the OK untagged
+    response for a successful SELECT and EXAMINE commands.
+
+    When opening a mailbox the server must check if the mailbox supports
+    the persistent storage of mod-sequences. If the mailbox supports
+    the persistent storage of mod-sequences and mailbox open operation succeeds,
+    the server MUST send the OK untagged response including HIGHESTMODSEQ
+    response code. If the persistent storage for the mailbox is not supported,
+    the server MUST send the OK untagged response including NOMODSEQ response
+    code instead.
+
+
 4.1.1. HIGHESTMODSEQ response code
 
     This document adds a new response code that is returned in the OK
     untagged response for the SELECT and EXAMINE commands. A server
-    supporting the CONDSTORE extension MUST send the OK untagged
-    response including HIGHESTMODSEQ response code with every successful
-    SELECT or EXAMINE command:
+    supporting the persistent storage of mod-sequences for the mailbox MUST
+    send the OK untagged response including HIGHESTMODSEQ response code with
+    every successful SELECT or EXAMINE command:
 
       OK [HIGHESTMODSEQ <mod-sequence-value>]
 
@@ -307,6 +333,24 @@
                 S: A142 OK [READ-WRITE] SELECT completed

+4.1.2 NOMODSEQ response code
+
+    A server that doesn't support the persistent storage of mod-sequences for
+    the mailbox MUST send the OK untagged response including NOMODSEQ response
+    code with every successful SELECT or EXAMINE command.
+
+    Example:    C: A142 SELECT INBOX
+                S: * 172 EXISTS
+                S: * 1 RECENT
+                S: * OK [UNSEEN 12] Message 12 is first unseen
+                S: * OK [UIDVALIDITY 3857529045] UIDs valid
+                S: * OK [UIDNEXT 4392] Predicted next UID
+                S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
+                S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
+                S: * OK [NOMODSEQ] Sorry, this mailbox format doesn't support modsequences
+                S: A142 OK [READ-WRITE] SELECT completed
+
+
 4.2. STORE and UID STORE Commands
 
     Arguments:  message set
@@ -328,8 +372,9 @@
          UNCHANGEDSINCE
             If the mod-sequence of any metadata item specified in the STORE
             operation for any message in the message set is greater than the
-            specified unchangedsince value, then the command fails.
-            On failure, a MODIFIED response code is returned which includes
+            specified unchangedsince value, then the command MUST NOT change
+            any flags. Instead, the server replies with the OK tagged response that
+            includes a MODIFIED response code. The MODIFIED response code includes
             the message set (for STORE) or set of UIDs (for UID STORE)
             of all messages that failed the UNCHANGESINCE test.
 
@@ -337,7 +382,7 @@
 
               C: a101 STORE 7,5,9 (UNCHANGEDSINCE 20000320162338)
                   +FLAGS.SILENT (\Deleted)
-              S: a101 NO [MODIFIED 7,9] Conditional STORE failed
+              S: a101 OK [MODIFIED 7,9] Conditional STORE failed
 
 		   In spite of the failure of the conditional STORE operation
                for message 7, the server continues to process the conditional
@@ -351,7 +396,7 @@
 
               C: a102 STORE 12 (UNCHANGEDSINCE 0)
                   +FLAGS.SILENT ($MDNSent)
-              S: a102 NO [MODIFIED 12] Conditional STORE failed
+              S: a102 OK [MODIFIED 12] Conditional STORE failed
 
             Note: A client trying to atomically change the state of a particular
              flag (or a set of flags) should be prepared to deal with the case
@@ -370,7 +415,7 @@
             Example:
               C: a106 STORE 100:150 (UNCHANGEDSINCE 200212030000000)
                   +FLAGS.SILENT ($Processed)
-              S: a106 NO [MODIFIED 101] Conditional STORE failed
+              S: a106 OK [MODIFIED 101] Conditional STORE failed
              
              the flag $Processed was set on the message 101 ...
               C: a107 NOOP
@@ -560,15 +605,15 @@
              cache. The client may also specify <entry-name> (name of metadata
              item) and <entry-type-req> (type of metadata item) before
              <mod-sequence-valzer>. <entry-type-req> can be one of "shared",
-             "private" or "all". The latter means that the server should use
-             the biggest value among "private" and "shared" modseqs for the
+             "priv" (private) or "all". The latter means that the server should use
+             the biggest value among "priv" and "shared" mod-sequences for the
              metadata item. If the server doesn't store internally separate
              mod-sequences for different flags and annotations, it MUST ignore
              <entry-name> and <entry-type-req>. Otherwise the server should
              use them to narrow down the search.
 
              For a flag <flagname> the corresponding <entry-name> has a form
-             "/message/flags/<flagname>" as defined in [ANNOTATE]. Note, that
+             "/flags/<flagname>" as defined in [ANNOTATE]. Note, that
              the leading "\" character that denotes a system flag has to be
              escaped as per Section 4.3 of [IMAP4], as the <entry-name> uses
              syntax for quoted strings.
@@ -580,14 +625,14 @@
     See also section 4.6.
 
     Example:
-         C: a SEARCH MODSEQ "/message/flags/draft" all 20010320162338
-                     ANNOTATION "/message/comment" "value" "IMAP4"
+         C: a SEARCH MODSEQ "/flags/draft" all 20010320162338
+                     ANNOTATION "/comment" "value" "IMAP4"
          S: * SEARCH 2 5 6 7 11 12 18 19 20 23 (MODSEQ 20010917162500)
          S: a OK Search complete
 
             In the above example, the message numbers of any messages
             containing the string "IMAP4" in the "value" attribute of the
-            "/message/comment" entry and having a mod-sequence equal to or
+            "/comment" entry and having a mod-sequence equal to or
             greater than 20010320162338 for the "\Draft" flag are returned in
             the search results.
 
@@ -684,9 +729,6 @@
 
     status-att-req      = status-att / "HIGHESTMODSEQ"
 
-    mailbox-data        =/ "STATUS" SP mailbox SP "("
-                           [status-rsp-info *(SP status-rsp-info)] ")"
-
     status-rsp-info     = status-att SP number /
                           "HIGHESTMODSEQ" SP mod-sequence-value
 
@@ -713,21 +755,22 @@
     search-modseq-ext   = SP entry-name SP entry-type-req
 
     resp-text-code      =/ "HIGHESTMODSEQ" SP mod-sequence-value /
+                           "NOMODSEQ" /
                            "MODIFIED" SP set
 
-    entry-name          = '"' "/message/flags/" attr-flag '"'
+    entry-name          = '"' "/flags/" attr-flag '"'
                           ;; each system or user defined flag <flag>
-                          ;; is mapped to "/message/flags/<flag>".
+                          ;; is mapped to "/flags/<flag>".
                           ;;
                           ;; <entry-name> follows the escape rules used
                           ;; by "quoted" string as described in Section
                           ;; 4.3 of [IMAP4], e.g. for the flag \Seen
                           ;; the corresponding <entry-name> is
-                          ;; "/message/flags/\\seen", and for the flag
+                          ;; "/flags/\\seen", and for the flag
                           ;; $MDNSent, the corresponding <entry-name>
-                          ;; is "/message/flags/$mdnsent".
+                          ;; is "/flags/$mdnsent".
 
-    entry-type-resp     = "private" | "shared"
+    entry-type-resp     = "priv" | "shared"
                           ;; metadata item type
 
     entry-type-req      = entry-type-resp | "all"
@@ -749,8 +792,10 @@
 
 ;;Borrowed from IMAP4rev1 and modified accordingly:
 
-    mailbox-data        =/ "SEARCH" [SP nz-number *(SP nz-number) search_sort_mod_seq] /
-                           "SORT" [SP nz-number *(SP nz-number) search_sort_mod_seq]
+    mailbox-data        =/ "STATUS" SP mailbox SP "("
+                           [status-rsp-info *(SP status-rsp-info)] ")" /
+                           "SEARCH" [1*(SP nz-number) SP search_sort_mod_seq] /
+                           "SORT" [1*(SP nz-number) SP search_sort_mod_seq]
 
     attr-flag           = "\\Answered" / "\\Flagged" / "\\Deleted" /
                           "\\Seen" / "\\Draft" / attr-flag-keyword /
@@ -828,8 +873,8 @@
     interact with ANNOTATE extension and for thorough review of the document.
 
     Authors also acknowledge the feedback provided by Cyrus Daboo, Larry
-    Greenfield, Chris Newman, Harrie Hazewinkel, Arnt Gulbrandsen and Timo
-    Sirainen.
+    Greenfield, Chris Newman, Harrie Hazewinkel, Arnt Gulbrandsen Timo
+    Sirainen and Mark Crispin.

 9. Author's Addresses

--------------6D49FEEA8037F438115381F7--

Date: Sat, 19 Apr 2003 23:27:48 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> Cyrus Daboo wrote:
>
> > The formal syntax for 'entry-name' should be simply defined as:
> >
> >     entry-name = string
> >
> > to fall in line with ANNOTATE.
>
> BTW, there is no ABNF element for entry-name in ANNOTATE, there are "entry" and
> "entry-match".

Also, ANNOTATE SEARCH criteria uses entry-match, which allows for wildcards.
CONDSTORE doesn't use wildcards.
Any opinions whether the document should/should not allow for wildcards?

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel
IETF standard
related pages: http://orthanc.ab.ca/mel/devel/Links.html

I speak for myself only, not for my employer.
__________________________________________

Date: Sat, 19 Apr 2003 23:20:49 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> The formal syntax for 'entry-name' should be simply defined as:
>
>     entry-name = string
>
> to fall in line with ANNOTATE.

BTW, there is no ABNF element for entry-name in ANNOTATE, there are "entry" and
"entry-match".

> There should be a comment/reference to the
> annotate spec at that point.

Sure.

> That will also allow you to remove the comment
> about escape rules at that point.

There is nothing wrong with repeating obvious ;-).

> Also, could be use "priv" instead of "private" for entry-type-resp just to
> fall in line with the suffixes that ANNOTATE uses?

So you want to use "priv" and "shared"? I've just updated the document.

Alexey Melnikov
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Sat, 19 Apr 2003 22:48:40 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Mark Crispin <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Mark Crispin wrote:

> On Sat, 12 Apr 2003, Cyrus Daboo wrote:
> > | Why would one want to use MODSEQ with THREAD?
> > For the same reason one would use it in the search criteria on sort: to
> > have the command return results only for those messages that changed since
> > the MODSEQ. I doubt this is really a useful behaviour but it has to be
> > present for consistency if you are adding MODSEQ to search unless its
> > explicitly stated MODSEQ can't be used in sort/thread search criteria.
>
> I'm hard-pressed to think of a use for MODSEQ with THREAD; and while SORT
> presents some possible use I doubt that there would be much of that in
> real life.

I agree with Mark here.
I can see a utility in MODSEQ criterion in SEARCH and MODSEQ sort order in SORT.

I prefer to live it unspecified how MODSEQ search criteria can be used in
THREAD. Any objections?

> If you declare that MODSEQ is an additional argument to SEARCH (much as
> CHARSET is) instead of a SEARCH criterion, then you avoid the issue.  But
> that's no good if you decide to have MODSEQ in SORT.

Regards,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 18 Apr 2003 20:12:06 -0400
From: Cyrus Daboo <[email-masked]>
To: Ken Murchison <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Ken,

--On Friday, April 18, 2003 3:19 PM -0400 Ken Murchison <[email-masked]> 
wrote:

|> This shows up the problem that the client still has to poll via NOOP to
|> get results. Obviously, if a mailbox was selected, the client could use
|> IDLE and WATCH would come back during IDLE without the need to poll. But
|> how would we deal with a connection that does not have a selected
|> mailbox? The client could select a 'dummy' mailbox and then use idle,
|> but wouldn't it be better to allow IDLE outside of selected state if
|> WATCH is present?
|
| IIRC, IDLE is already allowed in both the authenticated and selected
| states.

You're right - it is. The comment on what state its valid for is in the 
formal syntax and I missed it there.

-- 
Cyrus Daboo

To: Rob Siemborski <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 15:03:01 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Rob Siemborski <[email-masked]> writes:
>On Fri, 18 Apr 2003, Philip Guenther wrote:
>
>> It's all a matter to explaining to the execs the costs of the tradeoffs
>> that they're requesting.
>
>Sure, but that just moves around who is looking at the cost, the fact is
>that server costs aren't the ONLY ones to consider.

Right; that's why I used the word 'tradeoffs' to indicate that there are
costs and benefits to every choice.  Anyone who hides that fact only has
themselves to blame when all the costs end up on their shoulders.

Writing a protocol that does the same by transfering all the costs and
complexity to the server is all too easy.  There are reasons why IMAP
and LDAP are widely implemented and ACAP isn't.

...
>Saying that a proposal can't be improved because it is already better than
>what we currently have doesn't make a lot of sense.

Of course, faster is always better, but
1) going from "poll-interval" to "round-trip time" is a big saving
   compared to going from "round-trip time" to "instantly".  For
   updating 'background' information in a human interface (that's the
   use people have been talking about), I suspect the benefit to the
   user is all in that first big step.
2) clients *do* poll instead of opening N connections, one per-mailbox,
   even though the latter offers "as soon as possible" updates, so
   clients are willing to make a tradeoff when faced with it

If WATCH BIFF is implementable on most servers with what appears to be a
very short poll interval ("I always see BIFFs within a second or so")
but WATCH STATUS varies server to server, sometimes appearing to be as
responsive as BIFFs but other times appearing to have, say, a 30 second
poll interval, how will client authors react?  Always use WATCH BIFF?
Always use WATCH STATUS?  Always use WATCH STATUS and tell your users to
yell at the authors of the slow servers?  Something else?

Philip Guenther

Date: Fri, 18 Apr 2003 13:34:35 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP 
To: Rob Siemborski <[email-masked]>
cc: John Milan <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 16:30:55 -0400 (EDT), Rob Siemborski wrote:
> Sorry, perhaps I should have phrased it as "several people on the list
> have expressed interest in following up on this", (both "WATCH=BIFF" and
> "WATCH=STATUS") but certainly a WATCH extention of some type.

OK.  I agree that there is some concensus with that wording.

Date: Fri, 18 Apr 2003 16:30:55 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: John Milan <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Sender: [email-masked]

On Fri, 18 Apr 2003, Mark Crispin wrote:

> On Fri, 18 Apr 2003 15:52:59 -0400 (EDT), Rob Siemborski wrote:
> > So, I think its pretty clear that this is a desired feature to have.
>
> I don't see how you came up with that conclusion, unless you mean "desire" in
> terms of "asking for the moon".

Sorry, perhaps I should have phrased it as "several people on the list
have expressed interest in following up on this", (both "WATCH=BIFF" and
"WATCH=STATUS") but certainly a WATCH extention of some type.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Fri, 18 Apr 2003 12:55:03 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP 
To: Rob Siemborski <[email-masked]>
cc: John Milan <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 15:52:59 -0400 (EDT), Rob Siemborski wrote:
> So, I think its pretty clear that this is a desired feature to have.

I don't see how you came up with that conclusion, unless you mean "desire" in
terms of "asking for the moon".

> Is there anything more formal written than what's been in this thread on
> the list?

There is working code.  It only needs final assembly.

Date: Fri, 18 Apr 2003 15:52:59 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: John Milan <[email-masked]>
cc: "'Mark Crispin'" <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Sender: [email-masked]

On Fri, 18 Apr 2003, John Milan wrote:

> Why put so much effort into half measures? If everyone acknowledges
> WATCH to be an issue worth solving why not *completely* solve it in the
> protocol and wait for implementations to catch up.

So, I think its pretty clear that this is a desired feature to have.

However, it is decidedly difficult to discuss a hypothetical protocol
extention.

Is there anything more formal written than what's been in this thread on
the list?

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Reply-To: <[email-masked]>
From: "John Milan" <[email-masked]>
To: "'Mark Crispin'" <[email-masked]>
Cc: "'Rob Siemborski'" <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 12:25:03 -0700
Organization: TeamDirection.com
Importance: Normal
Sender: [email-masked]

>> Why put so much effort into half measures? If everyone acknowledges 
>> WATCH to be an issue worth solving why not *completely* solve it in 
>> the protocol and wait for implementations to catch up.

> Sometimes, implementations never catch up.  FTP is an excellent
example.

I think this just means that some pieces are more important than others.
As a developer for commercial software I'm often surprised at how the
market responds to features regardless of my level of effort. It would
be hard to argue that FTP is somehow wrong simply because the
marketplace hasn't clamored for full implementation. In fact, the
ubiquity of FTP speaks precisely to the fact that the designers
thoroughly solved a problem, and that the world at large prioritized the
bare minimum needed.

> On the other hand, if you do a partial solution that solves the 98%
case and is available now, it will be good enough *and* > get useful
work accomplished.

Well, 98% is more of a nearly complete solution than a partial solution
my book :) If you think this is a nearly complete solution, then you are
doing the right thing by pushing it. 

However, I would like to see expressions be watchable as well. Maybe
that falls in the missing 2%, but other people are bringing up issues as
well, so there may be a few more percentage points missing.

My thesis is, regardless of whether it technically feasible or not, why
not create a richer implementation of WATCH that addresses as many
features everyone is proposing as possible. Yeah things get more
complicated, but take a look at the XML spec. Talk about complicated!
But if it solves a problem, then complication is a necessary tradeoff.
And as the adoption of XML shows, implementors can navigate the
complexity. 

I just think its very useful to address as many issues in a protocol as
possible, not because you want to make life hard, but because its an
excellent record of what people believe is an excellent and
well-thought-out solution to a difficult problem. Or, for the executive
summary, the less you have to assume about a protocol the better.

A bit of a soap box moment, but thems my thoughts.

JM

Date: Fri, 18 Apr 2003 15:19:51 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Cyrus Daboo <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Cyrus Daboo wrote:
> 
> Hi Rob,
> 
> --On Friday, April 18, 2003 1:34 PM -0400 Rob Siemborski
> <[email-masked]> wrote:
> 
> | That said, I was imagining WATCH=STATUS to work something like:
> |
> | a WATCH public.folder (UNSEEN)
> | a OK Watching public.folder
> | b NOOP
> | b OK NOOP Done
> | c NOOP
> | * WATCH EXISTS "public.folder"
> | * WATCH EXISTS "public.folder"
> | * STATUS "public.folder" (UNSEEN 3)
> | c OK NOOP done
> 
> This shows up the problem that the client still has to poll via NOOP to get
> results. Obviously, if a mailbox was selected, the client could use IDLE
> and WATCH would come back during IDLE without the need to poll. But how
> would we deal with a connection that does not have a selected mailbox? The
> client could select a 'dummy' mailbox and then use idle, but wouldn't it be
> better to allow IDLE outside of selected state if WATCH is present?

IIRC, IDLE is already allowed in both the authenticated and selected
states.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Fri, 18 Apr 2003 11:52:06 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP 
To: [email-masked]
cc: "'Rob Siemborski'" <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 11:49:31 -0700, John Milan wrote:
> Why put so much effort into half measures? If everyone acknowledges
> WATCH to be an issue worth solving why not *completely* solve it in the
> protocol and wait for implementations to catch up.

Sometimes, implementations never catch up.  FTP is an excellent example.

> On the other hand, if you do a partial solution, then when it becomes
> technically feasible or economically viable, the protocol will not be
> correct and will splinter.

On the other hand, if you do a partial solution that solves the 98% case and
is available now, it will be good enough *and* get useful work accomplished.

We'd still be arguing IMP-IMP protocol if we refused to do anything before we
solved the last 2%.

Notifications are not a new idea.  The idea been bandied about for years.
Nothing has gotten done because people have refused to make the necessary
compromises necessary to accomplish results.

Reply-To: <[email-masked]>
From: "John Milan" <[email-masked]>
To: "'Mark Crispin'" <[email-masked]>, "'Rob Siemborski'" <[email-masked]>
Cc: "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 11:49:31 -0700
Organization: TeamDirection.com
Importance: Normal
Sender: [email-masked]

> After many years, the Cyrus server still does not implement the RENAME
command correctly because it is technically infeasible > for it to do it
right.

>> Saying that a proposal can't be improved because it is already better

>> than what we currently have doesn't make a lot of sense.

>It makes perfect sense if the "improvement" precludes the entire thing
from being implemented.  The history of Internet > protocols is littered
with specifications that weren't implemented because someone would not
take "no" for an answer.

> Sometimes if you refuse an offer as inadequate, you end up with
nothing.

Why put so much effort into half measures? If everyone acknowledges
WATCH to be an issue worth solving why not *completely* solve it in the
protocol and wait for implementations to catch up. For instance, not
implementing the RENAME command correctly hasn't stopped the progress of
Cyrus' IMAP server. And if the RENAME command is correct, then when it
becomes technically feasible to do it right, the protocol will survive
because it was correct.

On the other hand, if you do a partial solution, then when it becomes
technically feasible or economically viable, the protocol will not be
correct and will splinter.

I see no reason why the IMAP protocol can't fully define a WATCH
extension and leave server vendors to figure out what pieces they can
and can't support today.

John Milan

Date: Fri, 18 Apr 2003 11:39:14 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: re: WATCH: changes to mailboxes being watched
To: Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 14:35:27 -0400, Cyrus Daboo wrote:
> How should WATCH deal with changes to mailboxes being watched?

The ultra-lightweight facility that I am offering has no provisions to deal
with this.  All it will do is deliver an event which matches.

Date: Fri, 18 Apr 2003 14:35:27 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: WATCH: changes to mailboxes being watched
Sender: [email-masked]

How should WATCH deal with changes to mailboxes being watched? e.g.: what 
happens if a watched mailbox is deleted by some other client? Should the 
watched connection send back a response to indicate that the watched 
mailbox is now gone and there will be no more watch responses for that 
mailbox? What about renaming a mailbox? Does the watch state 'follow' a 
mailbox as it is renamed? Does the watched connection send a response 
indicating a name change by some other client?

-- 
Cyrus Daboo

Date: Fri, 18 Apr 2003 14:30:40 -0400
From: Cyrus Daboo <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Pete,

--On Friday, April 18, 2003 1:06 PM -0500 Pete Resnick 
<[email-masked]> wrote:

|> This shows up the problem that the client still has to poll via NOOP
|> to get results.
|
| Why? Isn't the WATCH response going to come in as unilateral server data?

If WATCH is written to require idle-like behaviour ithout a command in 
progress that would be fine. But the implications of that for clients needs 
to be spelled out.

-- 
Cyrus Daboo

Date: Fri, 18 Apr 2003 10:51:54 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP 
To: Rob Siemborski <[email-masked]>
cc: Philip Guenther <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 13:34:19 -0400 (EDT), Rob Siemborski wrote:
> Sure, but that just moves around who is looking at the cost, the fact is
> that server costs aren't the ONLY ones to consider.

Nevertheless, server costs can not be disregarded.

After many years, the Cyrus server still does not implement the RENAME command
correctly because it is technically infeasible for it to do it right.

Real implementation concerns can not be disregarded just because it may be
less of a concern in a particular implementation.  There has to be compromise,
and respect for the compromise.  That is what interoperability is all about.

> Saying that a proposal can't be improved because it is already better than
> what we currently have doesn't make a lot of sense.

It makes perfect sense if the "improvement" precludes the entire thing from
being implemented.  The history of Internet protocols is littered with
specifications that weren't implemented because someone would not take "no"
for an answer.

Sometimes if you refuse an offer as inadequate, you end up with nothing.

Date: Fri, 18 Apr 2003 13:06:24 -0500
To: Cyrus Daboo <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 4/18/03 at 1:49 PM -0400, Cyrus Daboo wrote:

>This shows up the problem that the client still has to poll via NOOP 
>to get results.

Why? Isn't the WATCH response going to come in as unilateral server data?

pr
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 18 Apr 2003 13:49:12 -0400
From: Cyrus Daboo <[email-masked]>
To: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Sender: [email-masked]

Hi Rob,

--On Friday, April 18, 2003 1:34 PM -0400 Rob Siemborski 
<[email-masked]> wrote:

| That said, I was imagining WATCH=STATUS to work something like:
|
| a WATCH public.folder (UNSEEN)
| a OK Watching public.folder
| b NOOP
| b OK NOOP Done
| c NOOP
| * WATCH EXISTS "public.folder"
| * WATCH EXISTS "public.folder"
| * STATUS "public.folder" (UNSEEN 3)
| c OK NOOP done

This shows up the problem that the client still has to poll via NOOP to get 
results. Obviously, if a mailbox was selected, the client could use IDLE 
and WATCH would come back during IDLE without the need to poll. But how 
would we deal with a connection that does not have a selected mailbox? The 
client could select a 'dummy' mailbox and then use idle, but wouldn't it be 
better to allow IDLE outside of selected state if WATCH is present?

-- 
Cyrus Daboo

Date: Fri, 18 Apr 2003 13:34:19 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Philip Guenther <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Sender: [email-masked]

On Fri, 18 Apr 2003, Philip Guenther wrote:

> It's all a matter to explaining to the execs the costs of the tradeoffs
> that they're requesting.

Sure, but that just moves around who is looking at the cost, the fact is
that server costs aren't the ONLY ones to consider.

> Also, complaining about round-trips seems unfair, given that *right now*
> the client has to poll.  Unless your round-trip is on the same order as
> your poll interval, Mark's lightweight proposal has a lower latency than
> polling, period.  Client that want STATUS information can throw a normal
> STATUS command into their command pipe and still end up ahead of those
> that just poll.  Indeed, the client can sanely implement the batching
> you proposed: "after receiving a notification, wait until N seconds have
> passed *without* receiving a notification for that same mailbox, but no
> more than M seconds total, before sending a STATUS command for the
> mailbox".

Saying that a proposal can't be improved because it is already better than
what we currently have doesn't make a lot of sense.

That said, I was imagining WATCH=STATUS to work something like:

a WATCH public.folder (UNSEEN)
a OK Watching public.folder
b NOOP
b OK NOOP Done
c NOOP
* WATCH EXISTS "public.folder"
* WATCH EXISTS "public.folder"
* STATUS "public.folder" (UNSEEN 3)
c OK NOOP done

(you could, in this case, just drop the "WATCH" responses, I have them
there just to indicate more than one message arrived).
But the idea was that you wouldn't queue them up based on a time interval,
you'd send the status as soon as there was new mail, within the same
response that would cover just the first message.

Maybe the looser requirements of the POLL command are good enough, I'd
like to hear from a client author though.  Certainly, it is an
improvement of having the client POLL explicitly though.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

To: Rob Siemborski <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 01:22:45 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Rob Siemborski <[email-masked]> writes:
>On Thu, 17 Apr 2003, Mark Crispin wrote:
>>> Or how about this: Suppose any time
>>> your users had to wait for the server cost you $5.
>>
>> This is also nonsense.  Customers pay their service providers, not the
>> other way around.
>
>Sure, but ISPs aren't the only ones providing IMAP service.  IT
>departments in corporations likely can't easily say to their VP,
>Presidents, or CEOs "screw you, our server resources are worth more than
>your time".

Sure they can: "enabling that feature will result in increased server
costs on the order of N thousand dollars per-year.  In the best
judgement of our staff, the ROI of such an expenditure is worse than
that of the following other items.  If you tell us to, we can, of
course, move it up on our priority list, but please be aware that
something else will have to give."  When I was sysadmin at a college in
Minnesota, we pushed back on "orders from on high" all the time.
Sometimes our VP agreed with our judgement/reasoning and relaxed the
mandate, other times we were told that the benefits were more than we
thought (publicity, inter-department deals, prestige of the VP,
whatever) and so we were the ones to give.  It's all a matter to
explaining to the execs the costs of the tradeoffs that they're
requesting.

...
<disagreement about the usefulness of \Recent elided>
...

>All that said, there is NO REASON, that a WATCH extention can not be
>extensible enough serve both the "ultra-lightweight" people and the "I
>want to STATUS every folder on every message arrival" people, without
>requiring additional round trips.

Using the same name for different actions isn't a step forward.  You've
already suggested that a WATCH STATUS should batch responses, while Mark
has mentioned dropping notifications if the queue overflows.  Those are
significantly different behaviors and I would expect some servers to
implement them with completely different backend mechanisms with
completely different efficiencies and latencies.  Calling them the same
thing encourages clients to treat them the same, which skews the
tradeoffs.

Also, complaining about round-trips seems unfair, given that *right now*
the client has to poll.  Unless your round-trip is on the same order as
your poll interval, Mark's lightweight proposal has a lower latency than
polling, period.  Client that want STATUS information can throw a normal
STATUS command into their command pipe and still end up ahead of those
that just poll.  Indeed, the client can sanely implement the batching
you proposed: "after receiving a notification, wait until N seconds have
passed *without* receiving a notification for that same mailbox, but no
more than M seconds total, before sending a STATUS command for the
mailbox".

Is that a 90% solution?  If not and what client authors really want is a
batched STATUS update, then perhaps we should be discussing a 'POLL'
extension, say, something like:

POLL Command
   Arguments:	max-period
		STATUS
		command arguments

   Responses:	untagged STATUS

   Result:	OK - initial data returned, command will be performed as
				requested 
		NO - too many POLLs requested or max-period too small
		BAD - command unknown or arguments invalid

	The POLL command requests that the IMAP server immediately
	perform the specified STATUS command and return its response.
	After the tagged OK, the server will continue to monitor the
	mailbox and unilaterally send updated STATUS responses for that
	mailbox.  The server SHOULD NOT send an update unless the
	mailbox has actually been changed, though it MAY send an update
	for a mailbox that has change in ways not reflected in the
	STATUS response.  [Alternatively, we could simply require it to
	track what it last sent and never send the exact same response
	twice in a row, even when the value changed but returned to
	their previous value before it composed the updated response.]

	If an update is to be sent, the server MAY delay sending it
	(e.g., so that the server can consolidate multiple changes into
	a single STATUS response), but MUST send it no later than if the
	server had received the specified STATUS command directly from
	the client 'max-period' seconds after the actual update
	occured.  [Blech.  How to phrase it that the server can't have
	any greater latency than a client that polled every 'max-period'
	seconds?]

	The server may limit the number of POLL commands that it will
	support on a single IMAP connection and may require the
	max-period to be no smaller than a give value.  Those limits
	will be advertised in the capability string, blah blah blah.
	[What are the minimum requirements to comply?  20 concurrent
	POLLs, supporting a max-period of 20 seconds or more?]

	If a client issues a SELECT or EXAMINE command for a mailbox for
	which it previously issued a POLL command on the same
	connection, the server SHOULD suppress all STATUS updates for
	that mailbox for as long as the client has it selected.

   Example:	C: A042 POLL 30 STATUS "Lists/imapext" (UNSEEN)
		S: * STATUS "Lists/imapext" (UNSEEN 3)
		S: A042 OK POLL started
		...time passes; new mail arrives
		S: * STATUS "Lists/imapext" (UNSEEN 10)
		...

I tried to write the above to allow both implementations that would
simply poll at the requested rate and those that would be truly
notification triggered, while hopefully making the result compelling for
clients either way.  

Philip Guenther
<[email-masked]>

Date: Thu, 17 Apr 2003 17:56:58 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> > Or how about this: Suppose any time
> > your users had to wait for the server cost you $5.
>
> This is also nonsense.  Customers pay their service providers, not the
> other way around.

Sure, but ISPs aren't the only ones providing IMAP service.  IT
departments in corporations likely can't easily say to their VP,
Presidents, or CEOs "screw you, our server resources are worth more than
your time".

> Wait a second.  You are implying that it is "frequent" that people will be
> reading mail with more than one computer simultaneously.
>
> Once again, how frequent?

Well, for me (using pine), all the time, actually.  In fact, I'll often
find myself with several pine sessions on the same machine.

> How frequent is it that a user is distraught that one of his simultaneous
> running clients isn't showing a new mail messages that he has already read
> on his other simultaneously running client?

If I accidently leave pine connected to a folder at work (happens almost
every day, to various folders), then go home, pine at home won't notice
that new mail has arrived in that folder when I hit "tab" to scan through
my incoming folders, because the \Recent flags are being unset by the
session at work.  I haven't actually read (or even been notified of) the
new messages.

Sure "doctor, it hurts when I do this!", but it's an example of a very
real problem with \Recent semantics.

> The minimum definition of "interesting" in this context is defined in the
> specification.

RFC3501>>>
      \Marked
         The mailbox has been marked "interesting" by the server; the
         mailbox probably contains messages that have been added since
         the last time the mailbox was selected.
<<<

I don't see a minimum requirement.  I see, at best, a suggestion of what
"interesting" might mean.

> > With this sort of specification, it is not surprising that clients
> > choose not to use \Marked for anything.
>
> I claim that it's because of lazy programmers who only use fast
> connections .

Nevermind the fact that servers are allowed to not return \Marked or
\Unmarked at all, if they feel like it, making the feature basically
useless in the general case anyway.

All that said, there is NO REASON, that a WATCH extention can not be
extensible enough serve both the "ultra-lightweight" people and the "I
want to STATUS every folder on every message arrival" people, without
requiring additional round trips.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 14:33:54 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Lawrence Greenfield wrote:
>    Suppose your ISP had a charging system such that LIST was free, but each
>    STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
>    differently?
> Suppose your ISP had a charging system such that LIST responses were
> 0.05 cents and STATUS was free.

That's nonsense.  LIST can never be more expensive than STATUS.

I remind you; ISPs are disabling SEARCH in IMAP because they think it is
too expensive.

> Or how about this: Suppose any time
> your users had to wait for the server cost you $5.

This is also nonsense.  Customers pay their service providers, not the
other way around.

> And this is one of the reasons that we're making LISTEXT be able to
> return STATUS responses.

Which I feel to be a serious mistake.

Tell me, when is Cyrus going to implement RENAME according to the
specification and remove its bogus NO_ATOMIC_RENAME capability?

>    True, the multiple clients connected to the same mailbox case is a problem.
>    But is that the 99% case?
> 99% case? Of course not. A very frequent thing? Yes.

How much is "very frequent"?  1% of the time?  .001% of the time?

> Many people read
> e-mail with more than one computer. Will it become more frequent?  I
> hope so---and many many people have cell phones.

Wait a second.  You are implying that it is "frequent" that people will be
reading mail with more than one computer simultaneously.

Once again, how frequent?

How frequent is it that a user is distraught that one of his simultaneous
running clients isn't showing a new mail messages that he has already read
on his other simultaneously running client?

The day that email spam hits cell phones en masse will be the day that we
see an end to that particular silliness, particularly in the US with its
"cell phone user always pays" pricing model and carriers which can't even
provide reliable service in the middle of a city.

> Since LIST \Marked isn't required and isn't even well specified. (My
> server could consider any message containing the word "penguin"
> interesting---not very useful to anyone but me, but _I_ find those
> messages "interesting".)

The minimum definition of "interesting" in this context is defined in the
specification.

It is not implausible that there might be an IMAP extension to add
"containing the word penguin" to the definition of "interesting".

> With this sort of specification, it is not surprising that clients
> choose not to use \Marked for anything.

I claim that it's because of lazy programmers who only use fast
connections .

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 23:17:26 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:54, Mark Crispin wrote:
> On 17 Apr 2003 21:46:18 +0300, Timo Sirainen wrote:
> > I haven't actually thought of a single good usage of \Marked flag. It
> > pretty much means that there's probably \Recent messages in the mailbox.
> 
> Consider this thought exercise.
> 
> Suppose your ISP had a charging system such that LIST was free, but each
> STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
> differently?

Maybe. But since it currently doesn't cost more than server resources,
good user interface is more important to me as a user (and as admin for
server with only a couple of users).

> Wouldn't you do a LIST to determine whether or not you should do a STATUS?
> After all, you can do a wildcard with the LIST.
> 
> True, the multiple clients connected to the same mailbox case is a problem.
> But is that the 99% case?

I think that's one of the best features in IMAP. I use it, so checking
only \Marked mailboxes wouldn't be enough for me. If the choice is
between using more server resources and not necessarily seeing new
message for hours or even days after it's received, I choose the
resource consumption any time.

Of course, client could do LIST often and ask STATUS for \Marked folders
immediately, but still ask STATUS for non-marked folders at least once
every n minutes.

Date: Thu, 17 Apr 2003 15:58:04 -0400
From: Lawrence Greenfield <[email-masked]>
To: Cyrus Daboo <[email-masked]>, Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>, Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

   Date: Thu, 17 Apr 2003 11:54:55 -0700 (PDT)
   From: Mark Crispin <[email-masked]>
[...]
   Suppose your ISP had a charging system such that LIST was free, but each
   STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
   differently?

Suppose your ISP had a charging system such that LIST responses were
0.05 cents and STATUS was free. Or how about this: Suppose any time
your users had to wait for the server cost you $5. Wouldn't you
prefetch as much data as possible?

   Wouldn't you do a LIST to determine whether or not you should do a STATUS?
   After all, you can do a wildcard with the LIST.

And this is one of the reasons that we're making LISTEXT be able to
return STATUS responses.

   True, the multiple clients connected to the same mailbox case is a problem.
   But is that the 99% case?

99% case? Of course not. A very frequent thing? Yes. Many people read
e-mail with more than one computer. Will it become more frequent?  I
hope so---and many many people have cell phones.

   On Thu, 17 Apr 2003 21:01:25 +0200, Simon Josefsson wrote:
   > Some clients like to only display the mailbox that have unread
   > messages on the screen.  So to find out which of the 1800 mailboxes to
   > display, the client sends STATUS for all of them.

   Such a client should use the LIST \Marked flag and not STATUS.

Since LIST \Marked isn't required and isn't even well specified. (My
server could consider any message containing the word "penguin"
interesting---not very useful to anyone but me, but _I_ find those
messages "interesting".)

With this sort of specification, it is not surprising that clients
choose not to use \Marked for anything.

It is important that any notification mechanism ("light weight" or
not) has well defined semantics that actually match what clients are
interested in.

Larry

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 21:36:49 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> On Thu, 17 Apr 2003 21:01:25 +0200, Simon Josefsson wrote:
>> Some clients like to only display the mailbox that have unread
>> messages on the screen.  So to find out which of the 1800 mailboxes to
>> display, the client sends STATUS for all of them.
>
> Such a client should use the LIST \Marked flag and not STATUS.

No, LIST \Marked means the mailbox contain "interesting" messages.
"Interesting" (recent) and unread are not the same thing, and some
users prefer to display only mailboxes with unread messages, so the
client must send STATUS for all of them.

See RFC 3501:

      The LIST command SHOULD return its data quickly, without undue
      delay.  For example, it SHOULD NOT go to excess trouble to
      calculate the \Marked or \Unmarked status or perform other
      processing; if each name requires 1 second of processing, then a
      list of 1200 names would take 20 minutes!
...
      \Marked
         The mailbox has been marked "interesting" by the server; the
         mailbox probably contains messages that have been added since
         the last time the mailbox was selected.
...
      If it is not feasible for the server to determine whether or not
      the mailbox is "interesting", or if the name is a \Noselect name,
      the server SHOULD NOT send either \Marked or \Unmarked.

Date: Thu, 17 Apr 2003 11:54:55 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Cyrus Daboo <[email-masked]>
cc: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:44:04 -0400, Cyrus Daboo wrote:
> \Marked is useful for finding out which mailboxes have new mail at the
> start of a session (i.e. when LIST is done) but to keep that information up
> to date, the client would have to do some form of LIST polling, at which
> point you might just as well use status.

On 17 Apr 2003 21:46:18 +0300, Timo Sirainen wrote:
> I haven't actually thought of a single good usage of \Marked flag. It
> pretty much means that there's probably \Recent messages in the mailbox.

Consider this thought exercise.

Suppose your ISP had a charging system such that LIST was free, but each
STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
differently?

Wouldn't you do a LIST to determine whether or not you should do a STATUS?
After all, you can do a wildcard with the LIST.

True, the multiple clients connected to the same mailbox case is a problem.
But is that the 99% case?

I suspect that the 99.99% case of multiple clients connected to the same
mailbox is the *same* client with multiple sessions to the same mailbox.  In
that case, the client is perfectly capable of resolving the confusion itself.

On Thu, 17 Apr 2003 21:01:25 +0200, Simon Josefsson wrote:
> Some clients like to only display the mailbox that have unread
> messages on the screen.  So to find out which of the 1800 mailboxes to
> display, the client sends STATUS for all of them.

Such a client should use the LIST \Marked flag and not STATUS.

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 21:05:08 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> The question is:
>
> If there was an ultra-lightweight notification mechanism available very
> soon, would you use it?
> 	OR
> Do you say, "No, I can not make use of an ultra-lightweight notification
> facility.  I'd rather wait until LEMONADE finishes its work, and then wait
> for that work to be implemented.  Until then, I'd rather have nothing at
> all."

The charter for LEMONADE does not make it clear (to me, at least) that
the message notification protocol they will develop is an IMAP
extension.  A notification protocol is not mentioned in the milestones
of the group either.  And no drafts.

> I'm trying to ascertain if it's worth the effort.  Personally, I believe
> that while it isn't ideal, it's good enough.

Since it appears (at best) unclear (to me) that LEMONADE will ever
specify a IMAP notification facility with the bells and whistles I'd
want, I don't think an effort in this direction would hurt.  If if you
changed your proposal into also providing (at least) the uidvalidity
of the mailbox and uid of the new message, I'm positively for it.  It
doesn't appear like it would cost much to implement this change.

If LEMONADE is on track towards creating a feature rich notification
IMAP extension, I think it is a mistake to push your proposal now as
it would result in two only slightly different solutions to the same
problem.

Just my .2 SEK.

To: Mark Crispin <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, Timo Sirainen <[email-masked]>, Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 21:01:25 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> On Thu, 17 Apr 2003 14:15:10 -0400, Cyrus Daboo wrote:
>> STATUS is often used to show a count of the total number of messages and
>> number of unseen messages in a 'closed' mailbox.
>
> I understand why people want/like that.  But why do STATUS of 1800 mailboxes
> when only 20 are currently visible on the screen?

Some clients like to only display the mailbox that have unread
messages on the screen.  So to find out which of the 1800 mailboxes to
display, the client sends STATUS for all of them.

The client could also be a batch client, that slurps in all
information it could possibly need, in order to be fast once it gotten
past the slow startup, instead of being slightly slow (depending on
the server) all the time.

Date: Thu, 17 Apr 2003 11:42:41 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Cyrus Daboo <[email-masked]>
cc: Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:40:21 -0400, Cyrus Daboo wrote:
> You need to think about GUI clients that may have multiple views on screen
> at the same time, all being updated and maintained as server state changes.

I mentioned that possibility.  On the other hand, there's only so much screen
real estate.  Most users do not have a screen large enough to display the
message counts for 1800 mailboxes, and display an index of messages, a message
being read, and a message being composed.

Unfortunately, I see clients behave as if this is the user's display
environment, the client has unlimited network bandwidth to the server, and the
server has unlimited resources to respond to client demands.

I see clients behave this way when the user is on a dialup IP connection from
a laptop with a 14" screen.  Much of the "IMAP is chatty" and "IMAP is slow"
and "IMAP is resource intensive" folklore comes out of such experience.

I'm not saying that Mulberry does this.  But there are clients which do so.

One of my ways of judging a "quality IMAP client" is the user experience and
how much it costs to read your mail over roaming CDPD (at $.05/kilobyte!).

LEMONADE should be an eye-opener.  A lot about LEMONADE frightens me, but
overall I think that it is a good thing that LEMONADE has happened.

Date: Thu, 17 Apr 2003 14:53:07 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: Rob Siemborski <[email-masked]>, Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Thu, 17 Apr 2003 14:01:32 -0400 (EDT), Rob Siemborski wrote:
> > Of course, if the user has recieved new mail in 1800 separate mailboxes,
> > in a notification period, I'd say the server is under a pretty high load
> > already.
> 
> I wasn't talking about.  I was talking about client startup.
> 
> Does the client do 1800 STATUS commands when the user first goes into the list
> of mailboxes?  If so, I think that client has a problem.
> 
> > I'm not sure if its better for the client to say on per session basis
> > "tell me about mailboxes x, y, z, a, b, c" or just let the client say
> > "tell me about my subscribed mailboxes".  I think I like the former,
> > actually.
> 
> So do we!
> 
> Originally, that was what subscribed mailboxes were supposed to do, but the
> abuse of the subscription feature by certain other clients put an end to that
> idea.
> 
> In effect, it is a type of subscription; it's just kept in client
> configuration instead of by the server.  That keeps it safe from the
> aforementioned certain other clients, but on the other hand other clients
> won't inherit that information either.

This might be a good mailbox annotation, eg:
/mailbox/{<mailbox-name>}/watch  (or whatever the syntax ends up being)

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 21:50:32 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:36, Mark Crispin wrote:
> What I see is neither of the above.
> 
> I see clients doing LIST "" * followed by STATUS of each returned name, at
> startup.  I see this repeated at periodic (obviously polling) intervals.
> 
> The resulting experience is not good for the server or the user.

Yeah, most IMAP clients suck.

BTW. I wrote this some weeks ago:
http://dovecot.procontrol.fi/imap-client-coding-howto.html

From: "John Milan" <[email-masked]>
To: "Mark Crispin" <[email-masked]>
Cc: "IMAP Extensions WG" <[email-masked]>
Subject: RE: WATCH extension for IMAP
Date: Thu, 17 Apr 2003 11:57:37 -0700
Importance: Normal
Sender: [email-masked]

> That is, in your "WATCH new mail where from = [email-masked]" example,
you
> would get all new mail events, and locally filter out the ones that are
not
> from "[email-masked]".

Hmmm... But I thought one of the features of the IMAP protocol was 'smart'
remote access. For instance, if I'm on a slow connection, I don't
necessarily have to get all my mail messages. It seems to me that if I am in
a connection speed challeged environment, I'd rather have the server filter
out the messages I'm watching for than have to download all new messages and
filter locally. Especially if I have a hotmail account with lots of spam.

Such an environment might have a powerful server, but many remote access
users, perhaps each using a dialup connection-- like the travelling salesman
in his hotel room. They might be very interested in what their boss has to
say, but not the funny movies their friends sent to them.

I suppose just filtering off the message headers takes care of the above
example, but then you can never really do content evaluation-- to costly to
send everything across the wire. This just seems like building a problem
into the protocol, especially when mail clients 'expect' this behavior in
the future.

John Milan

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 21:46:18 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:26, Mark Crispin wrote:
> > Just indicating which have
> > new mail is not enough - after all that is what \Marked does in LIST and
> > very few implementations have bothered to use it.
> 
> I never understood why this is the case.

I haven't actually thought of a single good usage of \Marked flag. It
pretty much means that there's probably \Recent messages in the mailbox.
Those are another thing which I haven't found very useful. In single
client case I guess it works well enough, but as soon as you have
multiple clients connected to same mailbox, the behaviour gets pretty
much random. Just because the client at work office noticed the mail
before the client at home, doesn't mean that I wouldn't want to get
notification of it when I'm at home.

\Marked flag tells only that there's a folder where you very likely find
new mail, but a client can't make any assumption if non-marked folders
have new mail. So it ends up sending STATUS for all of them anyway.

Date: Thu, 17 Apr 2003 14:44:04 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Thursday, April 17, 2003 11:26 AM -0700 Mark Crispin 
<[email-masked]> wrote:

|> Just indicating which have
|> new mail is not enough - after all that is what \Marked does in LIST and
|> very few implementations have bothered to use it.
|
| I never understood why this is the case.

\Marked is useful for finding out which mailboxes have new mail at the 
start of a session (i.e. when LIST is done) but to keep that information up 
to date, the client would have to do some form of LIST polling, at which 
point you might just as well use status.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 11:36:58 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Rob Siemborski <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:31:10 -0400 (EDT), Rob Siemborski wrote:
> I suppose that depends on your goal.  If you assume STATUS is going to
> take a long time, then from a user experience perspective it makes sense
> to do them when the network connection is otherwise idle.
>
> If, on the other hand, you are solely concerned with overall server
> performance, then you probably want this to be as lazy as possible.

What I see is neither of the above.

I see clients doing LIST "" * followed by STATUS of each returned name, at
startup.  I see this repeated at periodic (obviously polling) intervals.

The resulting experience is not good for the server or the user.

Date: Thu, 17 Apr 2003 14:40:21 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Rob Siemborski <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Thursday, April 17, 2003 11:03 AM -0700 Mark Crispin 
<[email-masked]> wrote:

|> Of course, if the user has recieved new mail in 1800 separate mailboxes,
|> in a notification period, I'd say the server is under a pretty high load
|> already.
|
| I wasn't talking about.  I was talking about client startup.
|
| Does the client do 1800 STATUS commands when the user first goes into the
| list of mailboxes?  If so, I think that client has a problem.

That depends on how the client is implemented. We do all STATUS work via a 
background thread so it won't interfere with the user. And yes, the user 
could easily set things up so that they do 1800 STATUS commands. We have 
many users that like to have message counts on their entire personal 
hierarchy appear on startup - so those can all get checked once, and then a 
different (smaller) set checked at varying intervals after that.

|> I'm not sure if its better for the client to say on per session basis
|> "tell me about mailboxes x, y, z, a, b, c" or just let the client say
|> "tell me about my subscribed mailboxes".  I think I like the former,
|> actually.
|
| So do we!
|
| Originally, that was what subscribed mailboxes were supposed to do, but
| the abuse of the subscription feature by certain other clients put an end
| to that idea.
|
| In effect, it is a type of subscription; it's just kept in client
| configuration instead of by the server.  That keeps it safe from the
| aforementioned certain other clients, but on the other hand other clients
| won't inherit that information either.

Yes - subscribe has been abused :-( Note that ANNOTATEMORE does have an 
annotation that's meant to be used to indicate which mailboxes should be 
checked for new mail, so hopefully co-operating clients can use that once 
its implemented.

|> But if the client author knows they are going to do a status for every
|> notification anyway (or every set of notifications on a mailbox), then
|> why cost them the additional round trip?
|
| Perhaps to discourage clients from doing that?  Perhaps to encourage
| innovative thinking that would avoid unnecessary STATUS commands?
|
| One very obvious thing would be to defer doing a STATUS command until the
| user is in the list of mailboxes screen.  It's rather silly to do 50
| STATUS commands on a mailbox while the user's screen is busy in composing
| a message.
|
| This wouldn't hold for a client that always shows a list of mailboxes,
| but the general idea still applies.  If the data isn't visible on the
| user's screen (the client is in a different mode, or that particular data
| is scrolled out of the visible view, or even if another window overlaps),
| then there's no need to update it until it becomes visible.
|

You need to think about GUI clients that may have multiple views on screen 
at the same time, all being updated and maintained as server state changes. 
All that can be done in the background without interfering with the user's 
current focus. Plus, whether the mailbox is on screen or not is not 
relevant as the new mail check might bring it on screen if it has new mail 
or require some other form of notification that does not depend on the 
visibility of the mailbox itself (e.g. an audible text-to-speech alert that 
reads the name of the mailbox and the number of new messages).

All of this is very much a client implementation issue. Providing a way to 
get status automatically will benefit a number of clients that are already 
doing expensive (polled) status calls. For low volume status changes, 
WATCH=STATUS will be a big win. For the high volume case, we need to have 
recommendations or limits on what should be done.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 14:36:22 -0400
From: Cyrus Daboo <[email-masked]>
To: Rob Siemborski <[email-masked]>, John Milan <[email-masked]>
cc: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: RE: WATCH extension for IMAP
Sender: [email-masked]

Hi Rob,

--On Thursday, April 17, 2003 2:22 PM -0400 Rob Siemborski 
<[email-masked]> wrote:

|> I'm thinking of a WATCH much like mail filtering.
|
| I suspect it would be possible to use the sieve notify extention to feed
| this in some way.

Indeed we could define WATCH=SIEVENOTIFY and have * WATCH responses that 
return the sieve data. However, this would require a WATCH command without 
a mailbox argument since sieve notifications are not tied to mailboxes. 
Using the empty string as an argument to WATCH in that case would make 
sense (and would mirror the syntax of server vs mailbox annotations in the 
next ANNOTATEMORE draft).

| I think I'd rather see the IMAP side of this stay simple (but
| extensible, if more features are desired later).

Agreed - WATCH=XXX allows us to do that.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 11:26:02 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Cyrus Daboo <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:15:10 -0400, Cyrus Daboo wrote:
> STATUS is often used to show a count of the total number of messages and
> number of unseen messages in a 'closed' mailbox.

I understand why people want/like that.  But why do STATUS of 1800 mailboxes
when only 20 are currently visible on the screen?

> Just indicating which have
> new mail is not enough - after all that is what \Marked does in LIST and
> very few implementations have bothered to use it.

I never understood why this is the case.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 21:35:27 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:03, Mark Crispin wrote:
> On Thu, 17 Apr 2003 14:01:32 -0400 (EDT), Rob Siemborski wrote:
> > Of course, if the user has recieved new mail in 1800 separate mailboxes,
> > in a notification period, I'd say the server is under a pretty high load
> > already.
> 
> I wasn't talking about.  I was talking about client startup.
> 
> Does the client do 1800 STATUS commands when the user first goes into the list
> of mailboxes?  If so, I think that client has a problem.

Optimally it would send STATUS only for those mailboxes that are visible
in screen. Good enough solution would be to check only the mailboxes
that can become visible by scrolling (similiar to LIST "" % usage).

Currently Evolution seems to do it for all subscribed folders if "check
mail in all folders" flag is set. I'm not sure if it does it for all
folders if I uncheck "show only subscribed folders" flag.

Date: Thu, 17 Apr 2003 14:31:10 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> One very obvious thing would be to defer doing a STATUS command until
> the user is in the list of mailboxes screen.  It's rather silly to do 50
> STATUS commands on a mailbox while the user's screen is busy in
> composing a message.

I suppose that depends on your goal.  If you assume STATUS is going to
take a long time, then from a user experience perspective it makes sense
to do them when the network connection is otherwise idle.

If, on the other hand, you are solely concerned with overall server
performance, then you probably want this to be as lazy as possible.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 14:22:22 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: John Milan <[email-masked]>
cc: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: RE: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, John Milan wrote:

> Taking advantage of at least the RFC headers as watchable data. That way, no
> matter how many folders I might have, I'll get that important message from
> the boss. More aggressive implementations could probably extend the
> expression matching to the message body.
>
> I'm thinking of a WATCH much like mail filtering.

I suspect it would be possible to use the sieve notify extention to feed
this in some way.

I think I'd rather see the IMAP side of this stay simple (but
extensible, if more features are desired later).

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 14:15:10 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Timo Sirainen <[email-masked]>
cc: Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Thursday, April 17, 2003 9:47 AM -0700 Mark Crispin 
<[email-masked]> wrote:

| After all, what's the point of STATUS, other than to see if there is new
| messages in the mailbox?  In this case, we *know* that that are new
| messages!

STATUS is often used to show a count of the total number of messages and 
number of unseen messages in a 'closed' mailbox. Just indicating which have 
new mail is not enough - after all that is what \Marked does in LIST and 
very few implementations have bothered to use it.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 11:03:29 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Rob Siemborski <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:01:32 -0400 (EDT), Rob Siemborski wrote:
> Of course, if the user has recieved new mail in 1800 separate mailboxes,
> in a notification period, I'd say the server is under a pretty high load
> already.

I wasn't talking about.  I was talking about client startup.

Does the client do 1800 STATUS commands when the user first goes into the list
of mailboxes?  If so, I think that client has a problem.

> I'm not sure if its better for the client to say on per session basis
> "tell me about mailboxes x, y, z, a, b, c" or just let the client say
> "tell me about my subscribed mailboxes".  I think I like the former,
> actually.

So do we!

Originally, that was what subscribed mailboxes were supposed to do, but the
abuse of the subscription feature by certain other clients put an end to that
idea.

In effect, it is a type of subscription; it's just kept in client
configuration instead of by the server.  That keeps it safe from the
aforementioned certain other clients, but on the other hand other clients
won't inherit that information either.

> But if the client author knows they are going to do a status for every
> notification anyway (or every set of notifications on a mailbox), then why
> cost them the additional round trip?

Perhaps to discourage clients from doing that?  Perhaps to encourage
innovative thinking that would avoid unnecessary STATUS commands?

One very obvious thing would be to defer doing a STATUS command until the user
is in the list of mailboxes screen.  It's rather silly to do 50 STATUS
commands on a mailbox while the user's screen is busy in composing a message.

This wouldn't hold for a client that always shows a list of mailboxes, but the
general idea still applies.  If the data isn't visible on the user's screen
(the client is in a different mode, or that particular data is scrolled out of
the visible view, or even if another window overlaps), then there's no need to
update it until it becomes visible.

> (though I
> could see WATCH=BIFF as just being WATCH=STATUS with no status fields
> being returned).

I agree with this observation.

Date: Thu, 17 Apr 2003 10:52:57 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP
To: John Milan <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

The underlying protocol has the potential for such a capability.  It deals in
terms of "object identifiers", so anything can be an object identifier.

However, in the ultra-lightweight version, there's only exact match of object
identifiers.  So, in the case of a new mail event, you would have to receive
events for everything that matches the object, and use data in the event
payload to determine whether to do anything with the event or discard it.

That is, in your "WATCH new mail where from = [email-masked]" example, you
would get all new mail events, and locally filter out the ones that are not
from "[email-masked]".

The ultra-lightweight version also has a further restriction; what the payload
can carry is limited.  Only the SMTP return-path (which may not necessarily be
the from) is available from mail delivery, and mail delivered via IMAP COPY or
APPEND wouldn't even have that.

It is possible to envision future versions in which the event generator (mail
delivery or IMAP server) would parse the RFC 2822 headers and deliver such
information as From, Subject, etc.  In fact, that is absolutely part of the
design; we're going to need that!  But it isn't going to be in version 1.

-- Mark --

Date: Thu, 17 Apr 2003 14:01:32 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> On 17 Apr 2003 20:05:17 +0300, Timo Sirainen wrote:
> > Well, my client has a list of around 20 mailboxes which show the number
> > of unread messages after the mailbox name. I like this behaviour, I can
> > leave some messages unread if they require some action that I didn't
> > want to do immediately.
>
> I understand why you like this behavior.
>
> On the other hand, suppose the user has 1800 mailboxes.  Do you do the same
> thing?  What if STATUS takes a second or so of real time on that server?  It's
> going to take a while to do 1800 STATUS commands even if the mail store has
> fast STATUS metadata.

Of course, if the user has recieved new mail in 1800 separate mailboxes,
in a notification period, I'd say the server is under a pretty high load
already.

> Do you require that the user identify 20 mailboxes of particular interest, and
> only do it for those 20 mailboxes?

I'm not sure if its better for the client to say on per session basis
"tell me about mailboxes x, y, z, a, b, c" or just let the client say
"tell me about my subscribed mailboxes".  I think I like the former,
actually.

> > I don't think it would be considered very good behaviour to select the
> > mailbox immediately and fetch the new messages. I don't necessarily want
> > to read them, and I may delete them using another client.
>
> I agree with that.  I personally would like the marker, then a command that I
> could give to get the current count.  That way, a STATUS isn't done unless I
> do it.

But if the client author knows they are going to do a status for every
notification anyway (or every set of notifications on a mailbox), then why
cost them the additional round trip?

I do like Cyrus's idea of WATCH=BIFF and WATCH=STATUS, etc (though I
could see WATCH=BIFF as just being WATCH=STATUS with no status fields
being returned).

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 10:25:44 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Timo Sirainen <[email-masked]>
cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 17 Apr 2003 20:05:17 +0300, Timo Sirainen wrote:
> Well, my client has a list of around 20 mailboxes which show the number
> of unread messages after the mailbox name. I like this behaviour, I can
> leave some messages unread if they require some action that I didn't
> want to do immediately.

I understand why you like this behavior.

On the other hand, suppose the user has 1800 mailboxes.  Do you do the same
thing?  What if STATUS takes a second or so of real time on that server?  It's
going to take a while to do 1800 STATUS commands even if the mail store has
fast STATUS metadata.

Do you require that the user identify 20 mailboxes of particular interest, and
only do it for those 20 mailboxes?

In my lightweight notification system, the only cost of watching 1800
mailboxes instead of 20 is that the notification server will have to keep some
number of bytes per watch request.

> Maybe it would be enough for some people to just see some kind of marker
> that there's new mail in the mailbox.

I hope that this is enough for many people.

> I don't think it would be considered very good behaviour to select the
> mailbox immediately and fetch the new messages. I don't necessarily want
> to read them, and I may delete them using another client.

I agree with that.  I personally would like the marker, then a command that I
could give to get the current count.  That way, a STATUS isn't done unless I
do it.

I use CDPD a lot.  I don't like it when my client decides to do something
under the covers automatically.  I recognize that sometimes it has no choice.

The behavior of "just spawn another connection" makes things worse on CDPD.
When your CDPD modem gets into channel scanning, the more that such a client
does that, the longer the recovery time.

From: "John Milan" <[email-masked]>
To: "Mark Crispin" <[email-masked]>, "IMAP Extensions WG" <[email-masked]>
Subject: RE: WATCH extension for IMAP
Date: Thu, 17 Apr 2003 10:49:56 -0700
Importance: Normal
Sender: [email-masked]

Are the event types predefined? Such as you might have the following event
types:

new mail
deleted mail
flag changes

Or can you WATCH an expression?

For instance, could say (pardon the SQLness)

WATCH new mail where from = [email-masked]

Taking advantage of at least the RFC headers as watchable data. That way, no
matter how many folders I might have, I'll get that important message from
the boss. More aggressive implementations could probably extend the
expression matching to the message body.

I'm thinking of a WATCH much like mail filtering.

John Milan

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Mark Crispin
Sent: Wednesday, April 16, 2003 6:37 PM
To: IMAP Extensions WG
Subject: WATCH extension for IMAP

I've been thinking about defining a WATCH extension to IMAP which would
allow an IMAP client to watch for certain events happening to a
non-selected mailbox.  Once a WATCH is set, it remains in effect until
that IMAP session is logged out, and untagged WATCH events for the watched
mailbox(es) can be returned at any time.

The command will be something like:
	tag WATCH mailbox event-list
and the responses will be something like
	* WATCH mailbox event-pair-list
where an event-pair contains an event type and event data.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 20:05:17 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 19:47, Mark Crispin wrote:
> On Thu, 17 Apr 2003, Timo Sirainen wrote:
> > I think it's more likely that clients will be abusive with "new mail in
> > mailbox xyz" notification. They'd probably just request STATUS every
> > time they receive that.
> 
> A client that does that will quickly have its notification queue filled
> and notifications will get dropped.

Wouldn't the same thing happen if server did it automatically?

> I question why a client would want to issue a STATUS if notified about new
> mail in such-and-such mailbox, as opposed to noting that that mailbox has
> new mail and/or opening it.
> 
> After all, what's the point of STATUS, other than to see if there is new
> messages in the mailbox?  In this case, we *know* that that are new
> messages!

Well, my client has a list of around 20 mailboxes which show the number
of unread messages after the mailbox name. I like this behaviour, I can
leave some messages unread if they require some action that I didn't
want to do immediately.

So, once in a while I look at the list and see if there's anything
interesting in there. Not just new mail, but also those mails that I
left unread. I may have marked some of them as read elsewhere, that
pretty much requires STATUS (UNSEEN).

Maybe it would be enough for some people to just see some kind of marker
that there's new mail in the mailbox.

I don't think it would be considered very good behaviour to select the
mailbox immediately and fetch the new messages. I don't necessarily want
to read them, and I may delete them using another client.

Date: Thu, 17 Apr 2003 09:47:47 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Timo Sirainen wrote:
> I think it's more likely that clients will be abusive with "new mail in
> mailbox xyz" notification. They'd probably just request STATUS every
> time they receive that.

A client that does that will quickly have its notification queue filled
and notifications will get dropped.

I question why a client would want to issue a STATUS if notified about new
mail in such-and-such mailbox, as opposed to noting that that mailbox has
new mail and/or opening it.

After all, what's the point of STATUS, other than to see if there is new
messages in the mailbox?  In this case, we *know* that that are new
messages!

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 19:43:33 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 19:20, Mark Crispin wrote:
> It's technically easy (trivial even) to do an automatic STATUS, and
> perhaps a subscription type mechanism will suffice to avoid the problem.
> However, the architecture has to be carefully designed to minimize the
> likelihood of abusive client implementations.

I think it's more likely that clients will be abusive with "new mail in
mailbox xyz" notification. They'd probably just request STATUS every
time they receive that. If server sends the STATUS replies directly, it
can at least force some "max. n seconds between STATUS notifications"
limit.

Date: Thu, 17 Apr 2003 09:39:53 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Simon Josefsson <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Simon Josefsson wrote:
> If the extension isn't IMAP aware, my initial reaction is that it
> should be a vendor specific capability.  I.e., your server can
> announce a WATCH capability that enables the command you propose.

Vendor-specific is not useful.  To be useful, it has to be something that
is generally adopted.

This does NOT preclude a separate, more heavyweight notification facility.
It certainly does not compete with a heavyweight facility.

The question is:

If there was an ultra-lightweight notification mechanism available very
soon, would you use it?
	OR
Do you say, "No, I can not make use of an ultra-lightweight notification
facility.  I'd rather wait until LEMONADE finishes its work, and then wait
for that work to be implemented.  Until then, I'd rather have nothing at
all."

LEMONADE is working on the specification for a heavyweight notification
facility.  I am not competing with that effort.  The LEMONADE charter says
that they will have that effort done by summer.  You can decide for
yourself whether you believe that will happen.

I know that everybody would like to have a heavyweight notification
facility, with lots of bells and whistles.  This is not such a facility.
I have a prototype ultra-lightweight notification system which I wrote and
demonstrated a year ago.  All that needs to be done is to stitch it in.
It is not likely to provoke server managers into turning it off either.

I'm trying to ascertain if it's worth the effort.  Personally, I believe
that while it isn't ideal, it's good enough.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Thu, 17 Apr 2003 12:08:42 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> What if there is a mail bomb, and multiple messages/second are arriving?
> STATUS is not necessarily an inexpensive operation (and the specification
> specifically says as much).

True, but perhaps multiple notifications for a folder could be aggregated
into a single STATUS reply as well?

> Nor is it certain that every client would want to do a STATUS.  Perhaps it
> just wants to note that the mailbox has new messages, and let the user
> decide to SELECT it.

This is why I'd want to see clients able to subscribe to the particular
information that comes back with the notifications.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 09:08:43 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Timo Sirainen wrote:
> How are you going to send the notification to IMAP server? Some signal?
> Socket?

Currently, it would be a TCP connection from the IMAP server to the
notification system.  There's a good effort being made to avoid dropping
notifications (of course, if a drop has to happen, it will happen).  The
notification system itself does not require that a client receiving
notifications be on the same system as the notification system.

Generating notifications is currently also done by TCP, but it is likely
to move to a datagram type mechanism.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Thu, 17 Apr 2003 10:33:21 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Wednesday, April 16, 2003 7:43 PM -0700 Mark Crispin 
<[email-masked]> wrote:

|> I think most of the clients would be just interested in getting
|> automatic updates for "STATUS (MESSAGES UNSEEN)", maybe UIDVALIDITY and
|> UIDNEXT too. For those it'd be easier if untagged STATUS replies were
|> received directly. Maybe "WATCH mailbox (STATUS (<status attributes>))"
|
| I agree with your observation; this would be nice.  Unfortunately, this
| functionality is outside of the scope of ultra-lightweight notification.
| The notification protocol can support this; however, the current set of
| event generators (the mail delivery agents) can not.
|
| The mail delivery agents merely append to the mailbox file; they have no
| concept of "number of messages", "number of unseen messages",
| "UIDVALIDITY", "UIDNEXT", or any other STATUS-type metadata.  The only
| thing they know is "appended a string purported to be a new mail message
| to such-and-such mailbox name, with return-path of such-and such".

Well, the server will know which mailbox is being watched and when the 
event for that is seen, why can't the server do an implicit STATUS (with 
client provided parameters) and return that? i.e. the server does what the 
client would otherwise do - issue a status for that mailbox.

I think for this to be useful within IMAP it has to return STATUS 
information - that's what clients will want - but I don't see that as being 
a problem or as interfering with the way the outside notification works.

I think it does also have to handle COPY/APPEND changes to the mailbox as 
well.

-- 
Cyrus Daboo

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 16:50:58 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 05:43, Mark Crispin wrote:
> I agree with your observation; this would be nice.  Unfortunately, this
> functionality is outside of the scope of ultra-lightweight notification.
> The notification protocol can support this; however, the current set of
> event generators (the mail delivery agents) can not.

How are you going to send the notification to IMAP server? Some signal?
Socket?

I've been thinking about using Linux's dnotify and BSD's kqueue() for
this. They should be quite lightweight as well, and they notice all
changes in mailbox file/directory.

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 12:28:10 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> The single most important event type will be new mail, with the event data
> being the SMTP return-path of the message.  So, in effect, this will be
> "biff for IMAP".

Yay!  This would be very useful, at least for my client.

I concur with Timo's observation, it is much more useful if the WATCH
extension responded with requested information (i.e., one of the
STATUS parameters, or possibly even a FETCH parameter -- imagine
receiving BODY and immediately being able to display information about
the new message) instead of just a maybe-there-is-new-mail-over-here
type of indication.  The latter kind of notification is sufficient in
a non-IMAP notification protocol, like zephyr (which the client can
support already), but if the feature is built into the IMAP protocol I
think it should be IMAP-aware instead of simply a external
notification protocol tunneled through IMAP.

If the extension isn't IMAP aware, my initial reaction is that it
should be a vendor specific capability.  I.e., your server can
announce a WATCH capability that enables the command you propose.

A real IMAP-aware NOTIFICATION extension would be of separate utility,
and could be defined in parallel with the WATCH capability.  Then a
server supporting WATCH can decide if it wants to do a STATUS
internally in order to be able to support NOTIFICATION too, and
clients can decide which kind of notification they are most helped by
and decide accordingly.

Thoughts?

Date: Wed, 16 Apr 2003 21:23:59 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP 
To: Philip Guenther <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Wed, 16 Apr 2003 21:23:04 -0700, Philip Guenther wrote:
> Does the IMAP server generate "new mail" notifications when it performs
> APPEND and COPY operations?

This could be done within the "ultra-lightweight" framework, although perhaps
it should be a different type of event than new mail.  Is this something that
people would like to have/find useful?

In the case of multiple-message COPY and multiple APPEND, it would most likely
generate a single event and not an event per-message.

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Date: Wed, 16 Apr 2003 21:23:04 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:
...
>I am interested in feedback as to whether this proposed extension is
>interesting or not.  Also, would it be better to build an extension which
>has extensibility but with restraints ("ultra-lightweight") on how it can
>be extended, or if it should be strictly a new mail watching facility and
>leave the possibility of other types of notifications to a different
>facilty (such as what is being discussed in LEMONADE).

Does the IMAP server generate "new mail" notifications when it performs
APPEND and COPY operations?

Philip Guenther
<[email-masked]>

Date: Wed, 16 Apr 2003 19:43:03 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Wed, 17 Apr 2003, Timo Sirainen wrote:
> I think most of the clients would be just interested in getting
> automatic updates for "STATUS (MESSAGES UNSEEN)", maybe UIDVALIDITY and
> UIDNEXT too. For those it'd be easier if untagged STATUS replies were
> received directly. Maybe "WATCH mailbox (STATUS (<status attributes>))"

I agree with your observation; this would be nice.  Unfortunately, this
functionality is outside of the scope of ultra-lightweight notification.
The notification protocol can support this; however, the current set of
event generators (the mail delivery agents) can not.

The mail delivery agents merely append to the mailbox file; they have no
concept of "number of messages", "number of unseen messages",
"UIDVALIDITY", "UIDNEXT", or any other STATUS-type metadata.  The only
thing they know is "appended a string purported to be a new mail message
to such-and-such mailbox name, with return-path of such-and such".

So, to rephase my original question.  Would it be useful to receive, from
the IMAP server, a notification that a mail delivery agent appended what
purports to be new mail to such-and-such mailbox name with return-path of
such-and-such?

You could, in reaction to this notification, then do a STATUS.

What this buys you is that you could react much faster if a mailbox gets
new mail.  Also, you won't have to issue STATUS (at least not as often) to
poll for new messages.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 05:12:50 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 04:36, Mark Crispin wrote:
> I've been thinking about defining a WATCH extension to IMAP which would
> allow an IMAP client to watch for certain events happening to a
> non-selected mailbox.  Once a WATCH is set, it remains in effect until
> that IMAP session is logged out, and untagged WATCH events for the watched
> mailbox(es) can be returned at any time.
> 
> The command will be something like:
> 	tag WATCH mailbox event-list
> and the responses will be something like
> 	* WATCH mailbox event-pair-list
> where an event-pair contains an event type and event data.

I think most of the clients would be just interested in getting
automatic updates for "STATUS (MESSAGES UNSEEN)", maybe UIDVALIDITY and
UIDNEXT too. For those it'd be easier if untagged STATUS replies were
received directly. Maybe "WATCH mailbox (STATUS (<status attributes>))"

Date: Wed, 16 Apr 2003 18:36:56 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

I've been thinking about defining a WATCH extension to IMAP which would
allow an IMAP client to watch for certain events happening to a
non-selected mailbox.  Once a WATCH is set, it remains in effect until
that IMAP session is logged out, and untagged WATCH events for the watched
mailbox(es) can be returned at any time.

The command will be something like:
	tag WATCH mailbox event-list
and the responses will be something like
	* WATCH mailbox event-pair-list
where an event-pair contains an event type and event data.

The single most important event type will be new mail, with the event data
being the SMTP return-path of the message.  So, in effect, this will be
"biff for IMAP".

We have talked about other event types, such as "message flags changed" or
"mailbox expunged", but it isn't clear how well this will scale (it's one
thing to generate an event when delivering a new mail message, it's quite
another to do so every time some flag gets altered!).

This is in effect an IMAP interface to an ultra-lightweight notification
protocol and facility that I am working on.  The IMAP server won't do any
actual "watching"; instead, it will just be a client to the notification
facility.

I need to emphasize the "ultra-lightweight" nature of this facility.
Although the underlying notification protocol is completely extensible, a
foremost consideration has been lightweight and low-cost (hence the
tenative nature of the flags/expunge event types).  That is also why the
SMTP return-path is in the new-mail notification, because many mail
delivery agents only know that.

Last but not least, this is something that can be made available Real
Soon.  Much of the code already exists.  I know that LEMONADE has a fairly
aggressive schedule for its notification facility, but I suspect that
schedule will slip.

I am interested in feedback as to whether this proposed extension is
interesting or not.  Also, would it be better to build an extension which
has extensibility but with restraints ("ultra-lightweight") on how it can
be extended, or if it should be strictly a new mail watching facility and
leave the possibility of other types of notifications to a different
facilty (such as what is being discussed in LEMONADE).

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sun, 13 Apr 2003 16:39:09 -0700 (PDT)
From: Paul Hoffman <[email-masked]>
To: [email-masked]
Subject: ADMINISTRIVIA: fighting with the new mail server
Sender: [email-masked]

I just changed the OS for the mail server, and am not convinced that I got
all the bugs worked out. So I'm inflicting this on you. :-) Hopefully,
you'll only see this once.

--Paul Hoffman

Date: Sun, 13 Apr 2003 16:35:20 -0700 (PDT)
From: Paul Hoffman <[email-masked]>
To: [email-masked]
Subject: ADMINISTRIVIA: testing the mail server
Sender: [email-masked]

I just changed the OS for the mail server, and am not convinced that I got
all the bugs worked out. So I'm inflicting this on you. :-) Hopefully,
you'll only see this once.

--Paul Hoffman

Date: Sun, 13 Apr 2003 16:33:27 -0700 (PDT)
From: Paul Hoffman <[email-masked]>
To: [email-masked]
Subject: ADMINISTRIVIA: testing out the new mail server setup
Sender: [email-masked]

I just changed the OS for the mail server, and am not convinced that I got
all the bugs worked out. So I'm inflicting this on you. :-) Hopefully,
you'll only see this once.

--Paul Hoffman

Date: Sat, 12 Apr 2003 10:16:07 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Cyrus Daboo <[email-masked]>
cc: Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Sat, 12 Apr 2003, Cyrus Daboo wrote:
> | Why would one want to use MODSEQ with THREAD?
> For the same reason one would use it in the search criteria on sort: to
> have the command return results only for those messages that changed since
> the MODSEQ. I doubt this is really a useful behaviour but it has to be
> present for consistency if you are adding MODSEQ to search unless its
> explicitly stated MODSEQ can't be used in sort/thread search criteria.

I'm hard-pressed to think of a use for MODSEQ with THREAD; and while SORT
presents some possible use I doubt that there would be much of that in
real life.

If you declare that MODSEQ is an additional argument to SEARCH (much as
CHARSET is) instead of a SEARCH criterion, then you avoid the issue.  But
that's no good if you decide to have MODSEQ in SORT.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sat, 12 Apr 2003 11:46:39 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Alexey,

--On Friday, April 11, 2003 10:32 PM -0600 Alexey Melnikov 
<[email-masked]> wrote:

|> You need to change the annotation entry names to exclude the "/message"
|> prefix. We agreed to eliminate the /message and /body entry name prefixes
|> at the last meeting. I'm hoping to get an updated annotation document out
|> in the next week with that and other changes.
|
| Is the \Seen flag now "/flags/\\seen" or "flags/\\seen"?

"/flags/\\seen" - i.e. / prefix is still present.

-- 
Cyrus Daboo

Date: Sat, 12 Apr 2003 11:36:28 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Alexey,

--On Friday, April 11, 2003 10:20 PM -0600 Alexey Melnikov 
<[email-masked]> wrote:

|> Interaction with SORT and THREAD - note that both SORT and THREAD take
|> search criteria as arguments. Right now the draft only discusses SORT but
|> it needs to include THREAD as well (an example for a thread response with
|> MODSEQ would be good too).
|
| Why would one want to use MODSEQ with THREAD?

For the same reason one would use it in the search criteria on sort: to 
have the command return results only for those messages that changed since 
the MODSEQ. I doubt this is really a useful behaviour but it has to be 
present for consistency if you are adding MODSEQ to search unless its 
explicitly stated MODSEQ can't be used in sort/thread search criteria.

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 23:31:29 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> |> I have no problems with the last suggested change (i.e. HIGHESTMODSEQ
> |> with 0).
> |> Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
> |> then CONDSTORE is not supported for the mailbox? This is probably better.
> |
> | Either way works for me.  I think though that an affirmative statement is
> | better than the absence of one; so HIGHESTMODSEQ 0 or NOMODSEQ seems
> | better than a missing HIGHESTMODSEQ on SELECT.
>
> Agreed, and I would favour NOMODSEQ for clarity as well.

Ok, I've added NOMODSEQ to the document.

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 22:49:47 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Mark Crispin <[email-masked]>, Chris Newman <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> Mark Crispin wrote:

>

> >  . STORE should NEVER reply NO; instead, the untagged FETCH response
> >     should be relied upon to indicate what happened
> >
> > I disapprove of the idea of NO with the MODIFIED status code.  This
> > introduces, for the first time, an instance where the server replied NO
> > but something happened anyway.

Actually, nothing happened. This is not much different from [removed] "NEWNAME"
response code.

But I don't really care whether MODIFIED is returned with OK or NO.

Does anybody else have any preferences?

>  The command succeeded and did what was
> > requested; therefore OK should be returned (with the MODIFIED status
> > code).
>
> This is fine with me.

Regards,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 22:32:40 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> You need to change the annotation entry names to exclude the "/message"
> prefix. We agreed to eliminate the /message and /body entry name prefixes
> at the last meeting. I'm hoping to get an updated annotation document out
> in the next week with that and other changes.

Is the \Seen flag now "/flags/\\seen" or "flags/\\seen"?

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 22:20:00 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> Interaction with SORT and THREAD - note that both SORT and THREAD take
> search criteria as arguments. Right now the draft only discusses SORT but
> it needs to include THREAD as well (an example for a thread response with
> MODSEQ would be good too).

Why would one want to use MODSEQ with THREAD?

> Also the formal syntax for the search and sort responses is slightly wrong
> (missing SP)

Thank you.

> and should have thread added:
>
> mailbox-data =/
>         "SEARCH" [1*(SP nz-number) SP search_sort_mod_seq] /
>         "SORT" [1*(SP nz-number) SP search_sort_mod_seq] /
>         "THREAD" [SP 1*thread-list SP search_sort_mod_seq]

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 18:06:27 -0400
From: Cyrus Daboo <[email-masked]>
To: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Interaction with SORT and THREAD - note that both SORT and THREAD take 
search criteria as arguments. Right now the draft only discusses SORT but 
it needs to include THREAD as well (an example for a thread response with 
MODSEQ would be good too).

Also the formal syntax for the search and sort responses is slightly wrong 
(missing SP) and should have thread added:

mailbox-data =/
        "SEARCH" [1*(SP nz-number) SP search_sort_mod_seq] /
        "SORT" [1*(SP nz-number) SP search_sort_mod_seq] /
        "THREAD" [SP 1*thread-list SP search_sort_mod_seq]

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 17:52:21 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Mark,

--On Friday, April 11, 2003 2:45 PM -0700 Mark Crispin 
<[email-masked]> wrote:

|> I have no problems with the last suggested change (i.e. HIGHESTMODSEQ
|> with 0).
|> Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
|> then CONDSTORE is not supported for the mailbox? This is probably better.
|
| Either way works for me.  I think though that an affirmative statement is
| better than the absence of one; so HIGHESTMODSEQ 0 or NOMODSEQ seems
| better than a missing HIGHESTMODSEQ on SELECT.

Agreed, and I would favour NOMODSEQ for clarity as well.

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 15:50:51 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Mark Crispin <[email-masked]>
CC: Chris Newman <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Mark Crispin wrote:

> Comments:
>
> A document in Last Call should not have open issues.  Resolve them one way
> or another.  I recommend that:
>  . it NOT be required that conditional STORE be atomic across the message
>     set

Actually I was in favor of this (i.e. not atomic across the message set) for a
long time. It might be just to complex for some servers to add transaction
support.

Chris?

>  . STORE should NEVER reply NO; instead, the untagged FETCH response
>     should be relied upon to indicate what happened
>
> I disapprove of the idea of NO with the MODIFIED status code.  This
> introduces, for the first time, an instance where the server replied NO
> but something happened anyway.  The command succeeded and did what was
> requested; therefore OK should be returned (with the MODIFIED status
> code).

This is fine with me.

> Otherwise, the document seems technically fine to me.  I think that we
> should be kind to the RFC Editor and edit it into a shape that's somewhat
> closer to publication status.

Yep.

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 14:45:06 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Fri, 11 Apr 2003, Alexey Melnikov wrote:
> I have no problems with the last suggested change (i.e. HIGHESTMODSEQ with
> 0).
> Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
> then CONDSTORE is not supported for the mailbox? This is probably better.

Either way works for me.  I think though that an affirmative statement is
better than the absence of one; so HIGHESTMODSEQ 0 or NOMODSEQ seems
better than a missing HIGHESTMODSEQ on SELECT.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 11 Apr 2003 15:44:44 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Mark,
>
> --On Friday, April 11, 2003 1:58 PM -0700 Mark Crispin
> <[email-masked]> wrote:
>
> | I know that this is rather late, but is there a recommended way for a
> | server which has CONDSTORE to operate with a mail store that does not
> | support it (e.g. always return 0)?
> |
> | I could offer CONDSTORE much faster in my server if I don't have to
> | support in in all possible mailbox formats.
>
> I guess that could be done by either allowing 0 or having an untagged
> response as part of SELECT to indicate CONDSTORE is not supported on the
> mailbox being selected, even though the capability is present. The later
> would be better because clients could then not use the extended commands
> for that mailbox. Perhaps the HIGHESTMODSEQ response with value '0' could
> be used as an indication of this?

I have no problems with the last suggested change (i.e. HIGHESTMODSEQ with
0).
Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
then CONDSTORE is not supported for the mailbox? This is probably better.

Anybody else?

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 17:40:41 -0400
From: Cyrus Daboo <[email-masked]>
To: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

The formal syntax for 'entry-name' should be simply defined as:

    entry-name = string

to fall in line with ANNOTATE. There should be a comment/reference to the 
annotate spec at that point. That will also allow you to remove the comment 
about escape rules at that point.

Also, could be use "priv" instead of "private" for entry-type-resp just to 
fall in line with the suffixes that ANNOTATE uses?

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 17:26:58 -0400
From: Cyrus Daboo <[email-masked]>
To: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

You need to change the annotation entry names to exclude the "/message" 
prefix. We agreed to eliminate the /message and /body entry name prefixes 
at the last meeting. I'm hoping to get an updated annotation document out 
in the next week with that and other changes.

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 17:13:41 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Pete Resnick <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Mark,

--On Friday, April 11, 2003 1:58 PM -0700 Mark Crispin 
<[email-masked]> wrote:

| I know that this is rather late, but is there a recommended way for a
| server which has CONDSTORE to operate with a mail store that does not
| support it (e.g. always return 0)?
|
| I could offer CONDSTORE much faster in my server if I don't have to
| support in in all possible mailbox formats.

I guess that could be done by either allowing 0 or having an untagged 
response as part of SELECT to indicate CONDSTORE is not supported on the 
mailbox being selected, even though the capability is present. The later 
would be better because clients could then not use the extended commands 
for that mailbox. Perhaps the HIGHESTMODSEQ response with value '0' could 
be used as an indication of this?

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 13:58:43 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks & Distributed Computing
Sender: [email-masked]

One last thing...

I know that this is rather late, but is there a recommended way for a
server which has CONDSTORE to operate with a mail store that does not
support it (e.g. always return 0)?

I could offer CONDSTORE much faster in my server if I don't have to
support in in all possible mailbox formats.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 11 Apr 2003 13:47:38 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks & Distributed Computing
Sender: [email-masked]

Comments:

A document in Last Call should not have open issues.  Resolve them one way
or another.  I recommend that:
 . it NOT be required that conditional STORE be atomic across the message
    set
 . STORE should NEVER reply NO; instead, the untagged FETCH response
    should be relied upon to indicate what happened

I disapprove of the idea of NO with the MODIFIED status code.  This
introduces, for the first time, an instance where the server replied NO
but something happened anyway.  The command succeeded and did what was
requested; therefore OK should be returned (with the MODIFIED status
code).

Otherwise, the document seems technically fine to me.  I think that we
should be kind to the RFC Editor and edit it into a shape that's somewhat
closer to publication status.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 11 Apr 2003 12:34:24 -0500
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Alexey has submitted the latest version of conditional store as a WG 
draft. We've all had plenty of time with it, so this is a WG last 
call on it. Please see:

http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-00.txt

Please post affirmative comments as well as any problems you might 
have. I'd like to submit this for IETF last call ASAP.

pr
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 06 Apr 2003 22:40:37 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Subject: Re: 'Label' flags
Sender: [email-masked]

Mark Crispin wrote:

> On Sun, 06 Apr 2003 23:02:24 -0400, Cyrus Daboo wrote:
> > I've seen 'in the wild' keywords such as $Label1, $Label2 etc Since I'm in
> > the process of adding keyword support and want to have a fixed set of
> > 'labels' that a user can define
>                   ^^^^ client
>
> I think that my correction is important.
>
> If you want "labels" that a user can defined, that's keywords.
>
> What you are asking about is a set of fixed names which has a different
> meaning on a per-client basis.

I agree.

Regarding the original question: I think we should standardize used client
labels, but $Label1/$Label2/... should not be standardized, because they have no
associated meaning.

I've only seen $Label1/... being implemented in Mozilla. Mozilla allows a user
to set on a message one of 5 mutually exclusive labels and a user can modify the
meaning of these labels (i.e. their text description is editable). This is
wrong, as there is no way to share what $Label1 means for different clients, and
it is possible to have unpredictable results as two Mozillas can be configured
differently.

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Sun, 6 Apr 2003 20:31:27 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: re: 'Label' flags
To: Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Sender: [email-masked]

On Sun, 06 Apr 2003 23:02:24 -0400, Cyrus Daboo wrote:
> I've seen 'in the wild' keywords such as $Label1, $Label2 etc Since I'm in
> the process of adding keyword support and want to have a fixed set of
> 'labels' that a user can define
                  ^^^^ client

I think that my correction is important.

If you want "labels" that a user can defined, that's keywords.

What you are asking about is a set of fixed names which has a different
meaning on a per-client basis.

> I'm interested in knowing whether this is
> something that should be 'standardised' behaviour to promote interop
> between clients.

I certainly hope not.  But I'm not going to oppose it either.

I am somewhat perplexed as to how this would be useful; but I don't see
particular harm either.

Date: Sun, 06 Apr 2003 23:02:24 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Subject: 'Label' flags
Sender: [email-masked]

I've seen 'in the wild' keywords such as $Label1, $Label2 etc Since I'm in 
the process of adding keyword support and want to have a fixed set of 
'labels' that a user can define I'm interested in knowing whether this is 
something that should be 'standardised' behaviour to promote interop 
between clients.

-- 
Cyrus Daboo

To: Arnt Gulbrandsen <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>, Harrie Hazewinkel <[email-masked]>, Michael Fair <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
From: Simon Josefsson <[email-masked]>
Date: Wed, 30 Apr 2003 15:47:11 +0200
Sender: [email-masked]

Arnt Gulbrandsen <[email-masked]> writes:

> Rob Siemborski writes:
>> On Tue, 29 Apr 2003, Michael Fair wrote:
>>
>>>  I'm still assuming that people on this list are still somewhat
>>> dubious about whether or not virtual hosting multiple domains on a
>>> single IP address is even useful.
>>
>> I agree that virtual hosting on a single IP is useful. I'm
>> personally unconvinced of the necessity of a VHOST command (or other
>> additional argument to CAPABILITY) to do this, its almost entirely
>> possible to do this using realms on user ids.
>
> The realms usage I see in RFC 2831 seems a little different. Realms
> there aren't hostnames, and they're used for a multiple-choice
> question. Basically, the server tells the client "pick one of these
> realms" and the client does, perhaps using a listbox or menu.
>
> I would not want to give every IMAP client my customer list,
> particularly not before authentication. And I would not want to tell
> random IMAP clients "here, make a thousand-item pop-up menu and ask
> the user to pick one".

DIGEST-MD5 allows the realm field in step 1 to be empty, and in this
case the client is allowed to chose a realm on their own.  DIGEST-MD5
doesn't have to be used as "pick one of these realms".  However, if
the server sends some realms, the server SHOULD pick one of them.  I
hoped that 2831bis would allow the client to pick any realm it wants,
regardless of what the server sent, thus further moving away from the
problem you describe, but this doesn't seem to have happened.

Date: Wed, 30 Apr 2003 09:23:11 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: Philip Guenther <[email-masked]>, Harrie Hazewinkel <[email-masked]>, Michael Fair <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

On Wed, 30 Apr 2003, Arnt Gulbrandsen wrote:

> The realms usage I see in RFC 2831 seems a little different. Realms
> there aren't hostnames, and they're used for a multiple-choice
> question. Basically, the server tells the client "pick one of these
> realms" and the client does, perhaps using a listbox or menu.

Realms in DIGEST-MD5 are a little strange.

But you *can* just append a @ or a % to an authentication identifier,
followed by a domain, and call it a realm.  This works with a much larger
variety of SASL mechanisms.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

From: Arnt Gulbrandsen <[email-masked]>
To: Rob Siemborski <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: Philip Guenther <[email-masked]>, Harrie Hazewinkel <[email-masked]>, Michael Fair <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date:   Wed, 30 Apr 2003 11:03:03 +0200
Sender: [email-masked]

Rob Siemborski writes:
> On Tue, 29 Apr 2003, Michael Fair wrote:
>
>>  I'm still assuming that people on this list are still somewhat 
>>  dubious about whether or not virtual hosting multiple domains on a 
>>  single IP address is even useful.
>
> I agree that virtual hosting on a single IP is useful. I'm personally 
> unconvinced of the necessity of a VHOST command (or other additional 
> argument to CAPABILITY) to do this, its almost entirely possible to 
> do this using realms on user ids.

The realms usage I see in RFC 2831 seems a little different. Realms 
there aren't hostnames, and they're used for a multiple-choice 
question. Basically, the server tells the client "pick one of these 
realms" and the client does, perhaps using a listbox or menu.

I would not want to give every IMAP client my customer list, 
particularly not before authentication. And I would not want to tell 
random IMAP clients "here, make a thousand-item pop-up menu and ask the 
user to pick one".

> The only problem using realms with user IDs doesn't solve is the fact 
> that you don't know which certificate to present during a STARTTLS. I 
> suppose this can be solved with an argument to STARTTLS. You'd need 
> to advertise a new STARTTLS_WITH_DOMAIN capability too.

As Simon Josefsson said, the TLS people are dealing with that.

--Arnt

Date: Wed, 30 Apr 2003 00:26:31 -0400
From: Lawrence Greenfield <[email-masked]>
To: Timo Sirainen <[email-masked]>, [email-masked]
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Review requested: draft-siemborski-mupdate-03.txt to be	published as an experimental RFC
Sender: [email-masked]

--On Wednesday, April 30, 2003 7:15 AM +0300 Timo Sirainen <[email-masked]> 
wrote:

> Just one question: Is there a reason why this should be RFC? I can see
> that specific server implementations might want to use this internally,
> but I just can't see why it should be interoperable with anything else
> but itself. Would someone actually want to create an IMAP server farm
> consisting of completely different IMAP server implementations?

The purpose of documenting this isn't necessarily to get interoperable 
implementations, but to document the protocol and the approach for the 
future so that the community has a stable reference to an open approach 
that's been done before.

Larry

Subject: Re: Review requested: draft-siemborski-mupdate-03.txt to be published as an experimental RFC
From: Timo Sirainen <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Organization: 
Date: 30 Apr 2003 07:15:32 +0300
Sender: [email-masked]

On Tue, 2003-04-29 at 22:06, [email-masked] wrote:
> I've received a request from the author to publish
> 
>     The MUPDATE Distributed Mailbox Database Protocol
>     draft-siemborski-mupdate-03.txt
> 
> as an experimental RFC. It is customary for area directors to ask for
> such documents to be reviewed by one or more working groups working in the
> same or related area. Accordingly, I'm asking folks involved in IMAPEXT
> to review this document and comment on its suitability for publication as
> an experimental protocol.

Just one question: Is there a reason why this should be RFC? I can see
that specific server implementations might want to use this internally,
but I just can't see why it should be interoperable with anything else
but itself. Would someone actually want to create an IMAP server farm
consisting of completely different IMAP server implementations?

Or maybe there's some other uses that aren't obvious to me? I had to
read Cyrus Aggregator web page before really even understanding how this
protocol was supposed to be used.

From: "Michael Fair" <[email-masked]>
To: "Simon Josefsson" <[email-masked]>
Cc: "Abhijit Menon-Sen" <[email-masked]>, <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 19:42:32 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Some environments will chose to not use STARTTLS, so this will never
> hold.  But some environments would chose to not use VHOST too, so I
> don't see the difference.  You can't ask for more than that people are
> required to implement a feature -- whether it is used or not depends
> on other things -- and people are already required to implement
> STARTTLS.

The difference is that with this proposed solution whether
or not someone can use VHOST is completely predicated on
whether or not they simultaneously use TLS.

With this proposal if you want to use VHOST, you MUST use
TLS - no choice - no option - no way around it - either
use TLS or don't have per domain specific configs.

I find unrealisticly limiting.  Whether or not someone uses 
TLS is a totally separate decision from whether or not the 
they use virtualized hosting.

Especially for domains specific configs that might not
want to use TLS for whatever reason.

-- Michael --

To: "Michael Fair" <[email-masked]>
Cc: "Abhijit Menon-Sen" <[email-masked]>, <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
From: Simon Josefsson <[email-masked]>
Date: Wed, 30 Apr 2003 03:59:37 +0200
Sender: [email-masked]

"Michael Fair" <[email-masked]> writes:

>> At 2003-04-30 00:38:48 +0200, [email-masked] wrote:
>> >
>> > Support draft-ietf-tls-extensions-06.txt in your STARTTLS
>> > implementation.
>> > 
>> > This is not a IMAP specific problem, so a IMAP specific solution is
>> > not the best solution, engineering wise.
>> 
>> I strongly prefer this approach over an optional argument to CAPABILITY,
>> or a separate VHOST command, particularly since STARTTLS support is now
>> mandatory for both servers and clients.
>
> I'm afraid I missed the part where the use of STARTTLS became
> mandatory.

See RFC 3501:

      Client and server implementations MUST implement the STARTTLS,
      LOGINDISABLED, and AUTH=PLAIN (described in [IMAP-TLS])
      capabilities.

> Fortunately it's generally accepted that STARTTLS
> requires a new CAPABILITY announcement anyway, thus if it can 
> be shown that all implementations of both clients and servers 
> not only support but MUST also use STARTTLS whenever it's 
> available then I'd probably accept this as a solution that 
> will work.

Some environments will chose to not use STARTTLS, so this will never
hold.  But some environments would chose to not use VHOST too, so I
don't see the difference.  You can't ask for more than that people are
required to implement a feature -- whether it is used or not depends
on other things -- and people are already required to implement
STARTTLS.

This doesn't mean they are required to implement the TLS extension
though...

> It'd be harder to debug since you will now need to get something 
> more intelligent than telnet to test against, but I'm sure 
> there'll be tools to help with that (perhaps even an updated telnet).

The tools exists.  The problem isn't specific to TLS, some SASL
mechanisms also encrypts the connection.

From: "Michael Fair" <[email-masked]>
To: "Abhijit Menon-Sen" <[email-masked]>, <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 18:16:10 -0700
Organization: DaClubhouse
Sender: [email-masked]

> At 2003-04-30 00:38:48 +0200, [email-masked] wrote:
> >
> > Support draft-ietf-tls-extensions-06.txt in your STARTTLS
> > implementation.
> > 
> > This is not a IMAP specific problem, so a IMAP specific solution is
> > not the best solution, engineering wise.
> 
> I strongly prefer this approach over an optional argument to CAPABILITY,
> or a separate VHOST command, particularly since STARTTLS support is now
> mandatory for both servers and clients.

I'm afraid I missed the part where the use of STARTTLS became
mandatory.  Fortunately it's generally accepted that STARTTLS
requires a new CAPABILITY announcement anyway, thus if it can 
be shown that all implementations of both clients and servers 
not only support but MUST also use STARTTLS whenever it's 
available then I'd probably accept this as a solution that 
will work.

It'd be harder to debug since you will now need to get something 
more intelligent than telnet to test against, but I'm sure 
there'll be tools to help with that (perhaps even an updated telnet).

Currently what's on the table is a Client Capability string.
While I disagree that being able to select a VHOST context
isn't a capability I'm not really opposed to anything that
solves the whole problem.

-- Michael --

Date: Wed, 30 Apr 2003 05:37:47 +0530
From: Abhijit Menon-Sen <[email-masked]>
To: [email-masked]
Cc: Michael Fair <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

At 2003-04-30 00:38:48 +0200, [email-masked] wrote:
>
> Support draft-ietf-tls-extensions-06.txt in your STARTTLS
> implementation.
> 
> This is not a IMAP specific problem, so a IMAP specific solution is
> not the best solution, engineering wise.

I strongly prefer this approach over an optional argument to CAPABILITY,
or a separate VHOST command, particularly since STARTTLS support is now
mandatory for both servers and clients.

While a client capability string may be a good idea for other reasons, I
don't think a virtual host selection mechanism should be shoehorned into
it. It may be quite convenient, but it isn't a capability, and shouldn't
be treated as one.

-- ams

Subject: Re: IMAP Capability and Virtual Domains
From: Timo Sirainen <[email-masked]>
To: Michael Fair <[email-masked]>
Cc: Philip Guenther <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Organization: 
Date: 30 Apr 2003 02:49:51 +0300
Sender: [email-masked]

On Wed, 2003-04-30 at 01:59, Michael Fair wrote:
> I actually happen to like Cyrus Daboo's latest proposal
> of doing something like:
> CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

Fine with me, except that should be more IMAP way to avoid creating new
parsers for server:

* OK [CLIENTCAPABILITY] (or something)
1 CAPABILITY VHOST (fqdn.example.com) RLIST LANGUAGE (en_US en_GB)

Server could just ignore capabilities it isn't interested in.

From: "Michael Fair" <[email-masked]>
To: "Cyrus Daboo" <[email-masked]>, <[email-masked]>
Cc: "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 16:11:50 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Hi Michael,
> 
> --On Tuesday, April 29, 2003 3:01 PM -0700 Michael Fair 
> <[email-masked]> wrote:
> 
> | The one reservation I have about using a separate VHOST command
> | that returns the same response as CAPABILITY is that you've now
> | created to commands that do essentailly the exact same thing.
> 
> Fine, but if we are going to extend capability lets do it in a way that 
> will allow future parameters to be easily added, e.g. use the capability 
> response syntax for the client command:
> 
> a CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

That would be great!
Servers that don't care can still swallow it, and other perceived
usefulness results.  Great call!

So what do we put in the HELLO response to give the client
foreknowledge for accepting the extra info?

Should we follow the SMTP example and prepend hello with
[ECAPABILITY] and then create an ECAPABILITY command?
Or are people comfortable with keeping the same named
CAPABILITY command?

I'm personally rather partial to not change the command
name.  But then again, I'm also the one who doesn't mind 
a client trying and failing as a method for determining
whether or not this server supports what will be the norm.

-- Michael --

From: "Michael Fair" <[email-masked]>
To: "Timo Sirainen" <[email-masked]>
Cc: "Philip Guenther" <[email-masked]>, <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 15:59:17 -0700
Organization: DaClubhouse
Sender: [email-masked]

> On Tue, 2003-04-29 at 23:51, Michael Fair wrote:
> > But you also mentioned PREAUTH the and intitial OK.  Let's talk 
> > about the initial OK for a second.  This is acutally pretty cool 
> > and something I hadn't thought about.  If we leave VHOST as part 
> > of the CAPABILITY announcement we can preinform a client about
> > whether or not the server supports domain specific configs.
> 
> This sounds good to me. To save some roundtrips, it could include
> capability in it's reply as well:
> 
> * OK [VHOST] IMAP server ready.
> 1 VHOST foo.bar.org
> 1 OK [CAPABILITY IMAP4rev1 ...]
> 2 STARTTLS
> 2 OK
> ..etc..

If you're going to bother with a VHOST command that returns
a CAPABILITY response why not just add it to CAPABILITY and
make the argument valid SOLELY in the presence of a preannounced
VHOST capability.  It keeps the command set down and reduces
the number of different commands that essentially do the exact
same things.
 
For instance, what's the difference between the above and:

* OK [VHOST] IMAP server ready.
1 CAPABILITY foo.bar.org
1 OK [CAPABILITY IMAP4rev1 ...]
2 STARTTLS
2 OK
..etc..

I actually happen to like Cyrus Daboo's latest proposal
of doing something like:
CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

-- Michael --

Date: Wed, 30 Apr 2003 00:43:25 +0200
From: Sebastian Hagedorn <[email-masked]>
To: Michael Fair <[email-masked]>
cc: [email-masked]
Subject: Re: IMAP Capability and Virtual Domains 
Message-Context: text-message
Sender: [email-masked]

--==========2147493761==========
Content-Type: text/plain; charset=iso-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

-- Michael Fair <[email-masked]> is rumored to have mumbled on=20
Dienstag, 29. April 2003 13:51 Uhr -0700 regarding Re: IMAP Capability and=20
Virtual Domains :

> For instance Outlook[ Express] only supports LOGIN.  I wouldn't
> want to enable that except for domains that required it.

Just FYI: the most recent versions of Cyrus SASL and Cyrus IMAP offer NTLM=20
as an option, which is working pretty well with Outlook (Express) in my=20
experience. I know, it's not a standard, but if I have to choose between=20
LOGIN and that I prefer the latter ...

Cheers, Sebastian
--
Sebastian Hagedorn M.A. - Postmaster - RZKR-R1 (Flachbau), Zimmer 18
Zentrum f=FCr angewandte Informatik - Universit=E4tsweiter Service RRZK
Universit=E4t zu K=F6ln / Cologne University - Tel. +49-221-478-5587
--==========2147493761==========
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit

--==========2147493761==========--

To: "Michael Fair" <[email-masked]>
Cc: <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
From: Simon Josefsson <[email-masked]>
Date: Wed, 30 Apr 2003 00:38:48 +0200
Sender: [email-masked]

"Michael Fair" <[email-masked]> writes:

> The problem with this solution is that there are several
> things that happen before LOGIN or AUTH where knowing the
> domain would be useful.  Namely STARTTLS and CAPABILITY.

Support draft-ietf-tls-extensions-06.txt in your STARTTLS
implementation.

This is not a IMAP specific problem, so a IMAP specific solution is
not the best solution, engineering wise.

A good authentication mechanism should have a means to allow clients
to chose a "realm" they belong to.  DIGEST-MD5 have this, as does the
experimental KERBEROS_V5.

Date: Tue, 29 Apr 2003 18:32:24 -0400
From: Cyrus Daboo <[email-masked]>
To: Michael Fair <[email-masked]>, [email-masked]
cc: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

Hi Michael,

--On Tuesday, April 29, 2003 3:01 PM -0700 Michael Fair 
<[email-masked]> wrote:

| The one reservation I have about using a separate VHOST command
| that returns the same response as CAPABILITY is that you've now
| created to commands that do essentailly the exact same thing.

Fine, but if we are going to extend capability lets do it in a way that 
will allow future parameters to be easily added, e.g. use the capability 
response syntax for the client command:

a CAPABILITY VHOST=fqdn.example.com RLIST LANGUAGE=en_US,en_GB

-- 
Cyrus Daboo

From: "Michael Fair" <[email-masked]>
To: "Rob Siemborski" <[email-masked]>
Cc: "Philip Guenther" <[email-masked]>, <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 15:15:03 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Well, I've sort of been interested in a client-side capability string (how
> does a server know that a client supports referrals if it doesn't execute
> an RLIST first, for example?).  In the case of referrals though, I suppose
> an [L01 RLIST "" ""] is just as good as a client-side capability string,
> since clients don't currently support it anyway.

If a client side capability string would be something that would
be generally useful, I would certainly be one of its supporters.

It's one of those things that seems like it would be generally
useful to me and would become more useful over time as people
found ways to do things they previously couldn't, but that is
more theory than fact at this point.

If there are convincing enough arguments to sway the protocol
designers to include, then I would propose the VHOST functionality
be moved as part of that functionality.

-- Michael --

Subject: Re: IMAP Capability and Virtual Domains
From: Timo Sirainen <[email-masked]>
To: Michael Fair <[email-masked]>
Cc: Philip Guenther <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Organization: 
Date: 30 Apr 2003 01:04:35 +0300
Sender: [email-masked]

On Tue, 2003-04-29 at 23:51, Michael Fair wrote:
> But you also mentioned PREAUTH the and intitial OK.  Let's talk 
> about the initial OK for a second.  This is acutally pretty cool 
> and something I hadn't thought about.  If we leave VHOST as part 
> of the CAPABILITY announcement we can preinform a client about
> whether or not the server supports domain specific configs.

This sounds good to me. To save some roundtrips, it could include
capability in it's reply as well:

* OK [VHOST] IMAP server ready.
1 VHOST foo.bar.org
1 OK [CAPABILITY IMAP4rev1 ...]
2 STARTTLS
2 OK
..etc..

Non-vhost capable client simply works as before.

Date: Tue, 29 Apr 2003 18:03:34 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Sender: [email-masked]

On Tue, 29 Apr 2003, Mark Crispin wrote:

> On Tue, 29 Apr 2003, Rob Siemborski wrote:
> > In the case of referrals though, I suppose
> > an [L01 RLIST "" ""] is just as good as a client-side capability string,
> > since clients don't currently support it anyway.
>
> Clients don't currently support referrals?
>
> That's news to me; Pine has supported referrals for years.

Sorry, I meant to say that the protocol doesn't currently support a
client-side capability string, so for the case of mailbox referrals, a
client-side capability string is just as good as sending RLIST "" "".

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

From: "Michael Fair" <[email-masked]>
To: "Cyrus Daboo" <[email-masked]>, <[email-masked]>
Cc: "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 15:01:32 -0700
Organization: DaClubhouse
Sender: [email-masked]

> Here is what I propose as an alternative, assuming VHOST is going to be 
> implemented:
> 
> 1) A VHOST aware server returns VHOST as the only capability in the 
> response code for the greeting string. This avoids the server having to 
> advertise any capabilites before its got the VHOST.

But this also kills the servers ability to be backwards compatible
and offer a "default domain" which would be the equivalent of 
whatever its pre-VHOST mail store mailboxes were.

You can still put VHOST in the response code of the greeting string
and it will still be useful, but I would augment this to say that it
would be included as one of the capabilities listed (but still 
possibly the only one for a VHOST only environment).  

> 2) A VHOST aware client that sees VHOST in the greeting capability MUST 
> immediately issue a VHOST command with the server fqdn as a parameter.
>
> 3) The server responds to the command with a full capability response 
> listing the capabilites now available for that virtual domain.

This simple creates an alternate CAPABILITY command.

How about we keep (1) as augmented above, but for 2 use:
2) A VHOST aware client that sees CHOST in the greeting capability
MUST immediately issue a CAPABILITY command with the server fqdn
as a parameter.  A client MUST NOT issue an argument to CAPABILITY
if it has not previously seen a VHOST capability.

The one reservation I have about using a separate VHOST command
that returns the same response as CAPABILITY is that you've now
created to commands that do essentailly the exact same thing.

I don't mind requiring servers to announce VHOST as part of the
greeting response at all.  I like the feature and if that is 
what it takes to satisfy a "client always has foreknowledge"
semantic.  Then I can live with that.

-- Michael --

Date: Tue, 29 Apr 2003 14:52:45 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Rob Siemborski <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 29 Apr 2003, Rob Siemborski wrote:
> In the case of referrals though, I suppose
> an [L01 RLIST "" ""] is just as good as a client-side capability string,
> since clients don't currently support it anyway.

Clients don't currently support referrals?

That's news to me; Pine has supported referrals for years.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Tue, 29 Apr 2003 14:45:03 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 29 Apr 2003, Cyrus Daboo wrote:
> Other issues not withstanding, I do not like the idea of an optional
> argument to capability, or using a VHOST command before issuing capability.

Agreed, for the reasons you state.

> 1) A VHOST aware server returns VHOST as the only capability in the
> response code for the greeting string. This avoids the server having to
> advertise any capabilites before its got the VHOST.

No good.  A VHOST-unaware client will issue a plaintext LOGIN command
unless the client itself was configured not to do such a thing.

That gets into including LOGINDISABLED, which in turn results in other
implications.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Tue, 29 Apr 2003 17:32:53 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Michael Fair <[email-masked]>
cc: Philip Guenther <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Sender: [email-masked]

On Tue, 29 Apr 2003, Michael Fair wrote:

> I'm still assuming that people on this list are still
> somewhat dubious about whether or not virtual hosting
> multiple domains on a single IP address is even useful.

I agree that virtual hosting on a single IP is useful.  I'm personally
unconvinced of the necessity of a VHOST command (or other additional
argument to CAPABILITY) to do this, its almost entirely possible to do
this using realms on user ids.

The only problem using realms with user IDs doesn't solve is the fact that
you don't know which certificate to present during a STARTTLS.  I suppose
this can be solved with an argument to STARTTLS.  You'd need to advertise
a new STARTTLS_WITH_DOMAIN capability too.

[As an aside, when I worked for a web hosting firm back in '97-99, I know
we were using an IP-per-domain for HTTPS.  Is this still the case?  (I
don't immediately know of any new solutions but I could be wrong).]

Of course, you now have an issue where some of your customers may not be
able to advertise STARTTLS (they don't have certs, and don't want junk
certs).  Maybe its good enough just to respond NO when these clients try
to do STARTTLS "badcustomer.com"

I agree with Cyrus's comments that placing a requirement that the client
assume a server supports something before issuing a CAPABILITY command is
bad.

> I'll tell you what would be even more useful is just a
> general command where the client can pass a set of
> generic key value pairs to inform the server of generic
> session parameters where clients and servers can be
> modified to look for and send arbitrary info.

Well, I've sort of been interested in a client-side capability string (how
does a server know that a client supports referrals if it doesn't execute
an RLIST first, for example?).  In the case of referrals though, I suppose
an [L01 RLIST "" ""] is just as good as a client-side capability string,
since clients don't currently support it anyway.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Tue, 29 Apr 2003 17:27:34 -0400
From: Cyrus Daboo <[email-masked]>
To: Michael Fair <[email-masked]>, [email-masked]
cc: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

Hi Michael,

--On Tuesday, April 29, 2003 12:30 AM -0700 Michael Fair 
<[email-masked]> wrote:

| I had an offline discussion with Mark Crispin and proposed
| adding the domain name as an optional parameter to CAPABILITY.
|
| My assumption was that the very first thing an IMAP client
| is going to do (even before STARTTLS) is CAPABILITY to find
| out if the server can even do STARTTLS and to figure out
| what methods of authentication the server makes available.

Other issues not withstanding, I do not like the idea of an optional 
argument to capability, or using a VHOST command before issuing capability. 
If you do that, how is a client to know that that is a valid thing to do 
with that server? The user would have to provide some configuration switch 
to enable it and that requires one extra step for the user that should be 
avoided.

Here is what I propose as an alternative, assuming VHOST is going to be 
implemented:

1) A VHOST aware server returns VHOST as the only capability in the 
response code for the greeting string. This avoids the server having to 
advertise any capabilites before its got the VHOST.
2) A VHOST aware client that sees VHOST in the greeting capability MUST 
immediately issue a VHOST command with the server fqdn as a parameter.
3) The server responds to the command with a full capability response 
listing the capabilites now available for that virtual domain.

This goes like:

s: * OK [CAPABILITY imap4rev1 VHOST]
c: a VHOST fqdn.example.com
s: * CAPABILITY imap4rev1 starttls logindisabled auth=gssapi
s: a OK

I believe this approach minimises round trips and doesn't require the 
client to guess as to whether an extended capability command is present or 
not.

-- 
Cyrus Daboo

From: "Michael Fair" <[email-masked]>
To: "Philip Guenther" <[email-masked]>
Cc: <[email-masked]>, "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 13:51:22 -0700
Organization: DaClubhouse
Sender: [email-masked]

> There are three 'issues' that the VHOST capability claims to address:
> 1) the ad-hoc interpretation of authentication IDs as specifying user
>    and domain 
> 2) the ability to have domain-specific capability lists
> 3) the choice of STARTTLS certificate by the server
> 
> In my opinion, the first of these is a non-issue in practice.  Indeed,
> the VHOST extension leaves the client author in a sticky situation: if
> it accepts the authentication ID as a single value ready for presenting
> to LOGIN (or AUTHENTICATE PLAIN/CRAM-MD5/etc), then it must decide and
> document how it splits that into a domain for VHOST and a username for
> LOGIN.  If it accepts the authentication ID as separate domain and user
> values, then it must guess how to join them for non-VHOST-enabled
> servers, or have another configuration parameter for _that_.  The VHOST
> extension would also need to specify how it affects and/or operates with
> SASL mechanisms that convey the user's domain separate from their
> username.

The first of these is not a non-issue, but I grant is more of
a minor annoyance than an issue.  However, it forces administrators
to dictate certain parts of customer usernames, where our preference
would be to allow the customers to have complete control over that
namespace.  It save us and them time (and money).

As for the client detecting the domain name, the client wouldn't 
split anything.  Whatever was specified in the "IMAP server" 
field would be presented wholesale without modification as the 
FQDN for the VHOST or CAPABILITY argument to specify the 
targetted domain.  It's a server and admin issue to make sure 
that whatever the client has as its server address is appropriately 
mapped internally.

Think HTTP 1.1 and Apache Name Based Virtual Hosts if you have any
experience with that.  The servername is presented unmodified as
part of the header to the request.

> The second issue appears minor to me.  Is it more than a matter of
> differing domains being allowed different SASL mechanisms?  Is that a
> practical problem?

This question is equivalent to "Why should any server ever offer 
a different CAPABILITY line?"

Think ISP, think a 1,000 unique domains all on one IP, think 600 
or more different companies all being hosted on the same machine.  
Each of those 1,000 domains deserves to be treated as if it had 
its own dedicated server.  That currently requires 1,000 IP 
addresses.  The goal here is simply to give the IMAP server the
name of the server the client was trying to reach as early as
possible so it can look up a config for that server and present 
it.  Each domain should be as independantly configurable as a
standalone server would be (with the exception of anything that
happens on the wire before the client has a chance to speak).

However, to address your direct question about the second issue,
most of it surrounds SASL mechanisms, but tuning CAPABILITY based
on the preferred clients in each domain is also an issue.  

Oftentimes the only way to get the desired behavior is to add or
remove things from the CAPABILITY line.  Other times, you don't want
to offer certain CAPABILITY options unless you absolutely have to.
For instance Outlook[ Express] only supports LOGIN.  I wouldn't
want to enable that except for domains that required it.

> I agree that the third issue is a real problem.  IMHO, STARTTLS would
> have been better defined to take an "fqdn" argument that the server
> could use to decide which certificate to use.  An extension that did
> only that---add an optional domain argument to STARTTLS---would appear
> to provide most of the benefits of VHOST.

I agree that STARTTLS would be better if the only issue were 
choosing a certificate.  Perhaps the proposal should be expanded
to add an argument to both CAPABILITY and STARTTLS.  I chose 
CAPABILITY because the domain is is needed for all three issues 
listed, not just (3).  I personally figure if you're going to
bother fixing, you should look into fixing all of it not just
propping up the most important piece.  The cost of adding an 
argument is the same whether it is to STARTTLS or to CAPABILITY 
or to adding a new command.  Things break, or you get more round 
trips.  My goal was to get the server the info as early as possible.

Not all clients issue a STARTTLS, nor should all be required to
issue a STARTTLS just to get per domain information.  If all
clients issued a STARTTLS then I agree adding a domain argument
to STARTTLS would solve the problem.  However, not all clients
do that.  But, all clients do issue a CAPABILITY.  Adding a domain
argument to that would solve the problem neatly.  I will address
the "two round trips for failure" below.

> It seems to me that the CAPABILITY command is ill-suited for such
> treatment.  If the client already has the server's list of capabilities
> and therefore knows that the server supports the extension, then
> overloading the CAPABILITY command doesn't save you anything.  

Right, adding a domain argument to CAPABILITY is intended to 
completely replace VHOST.  A client would issue the extended
version and fall back in the event of BAD.  If this was just
to figure out if the server supported the VHOST command then
addidng a domain argument to CAPABILITY would be senseless.
My goal with adding the domain argument to CAPABILITY is to
reduce the number of round trips a client has to make.

> If the
> client doesn't already have the server's list of capabilities, then the
> client will have to either try the extended version and fallback on BAD,
> or send an unextended request with a followup if the capability is
> advertised, either way requiring two roundtrips against some set of
> servers.  Leaving the VHOST command separate from CAPABILITY while
> strengthing the recommendation that the server include a CAPABILITY
> response code in the initial PREAUTH or OK response and that clients
> recognize such would appear to be a simpler and cleaner approach.

You seem to think that making clients issue a separate VHOST 
coammdn "two round trips for some set of servers".  You've just
inverted the set of servers that's all.  I think it's fair to
say that clients and servers of any proposal accepted would
gravitate towards supporting it as long as it cost them very 
little to add (which I have explicity designed the proposal for).

First, if we added a domain argument to CAPABILITY, then as my
proposal suggested, servers could be modified to simply swallow
it and go about their business.  So supporting the argument
is really trivial.  It does require a code change, and it does
require a release of a new server version which will take some
time.  So between now and then we have all the existing code 
that responds with "BAD" which is, and will remain, a correct
response for servers that don't support the VHOST capability.

In general, the clients that support this feature aren't going 
to be used against the servers that don't unless most future
versions of clients decide they should support this (which then
makes a compelling case for servers to at least swallow the 
argument), however there will be plenty of exceptions and for
a time these clients will get a BAD response from those servers.
Those exceptions will need to make two round trips, but over time
those exceptions will become fewer.

If you're interested in optimizing bandwidth, it quickly becomes
very clear that, if accepted, the general trend of servers and
clients will be to support the added domain argument to CAPABILITY
(or some other proposal that provides equivalent functionality).
It costs servers nothing but a little "ignore the argument" code.
It costs clients some "tack the server name on" code, and if the
response is "BAD" or "NO" then give up.  Granted those are the 
bare minimums for compliance, but they are fairly trivial.

But you also mentioned PREAUTH the and intitial OK.  Let's talk 
about the initial OK for a second.  This is acutally pretty cool 
and something I hadn't thought about.  If we leave VHOST as part 
of the CAPABILITY announcement we can preinform a client about
whether or not the server supports domain specific configs.
Very cool!  I'll be suggesting that in to my proposal.  Originally
I had intended on not doing anything with the CAPABILITY line
because I wasn't thinking that a server could just announce
its CAPABILITY without the client requesting it.

But now we need to decide between a separate VHOST command or 
an argument to CAPABILITY?

Here's a test case:
Client shows up at the server, the server dumps a CAPABILITY 
line down its throat the client sees a VHOST option, or the
Client shows up at the server, the server gives some kind of
hello and the Client is unaware of whether or not the server
supports the VHOST capability.

It then .....
A) [issues a CAPABILITY and looks for VHOST (if unknown)]
   issues a VHOST command 
   then issues another CAPABILITY
or
B) isssues a CAPABILITY with a domain argument
   [issues a CAPABILITY without a domain argument (if BAD)]

The only other proposal that would obviate the need for
a VHOST command to require a separate round trip would
be if VHOST returned the same response as CAPABILITY.
But now you've created two commands that do the exact same
thing and for what?  To protect existing servers from having
to trip over a malformed CAPABILITY line that the servers
are already prepared to ignore and that over time will learn
to accept (even if they just ignore the argument which they 
are perfectly allowed to do).

> ...
> >As an added bonus, this also lays the groundwork for 
> >some future planning on replicating and scaling IMAP 
> >mail stores to span multiple mail stores to provide 
> >geograhic redundancy.  Without this feature the "IP burn 
> >rate" jumps to 1+N addresses per domain where N is the 
> >number of geographically redundant mail stores.
> 
> I don't understand how geographically redundant mail stores are helped
> out any more than any set of N mail stores that offer STARTTLS.  If the
> problem is the requirement for separate IPs for STARTTLS, then giving
> STARTTLS an optional argument would "lower the burn rate" just as much
> as VHOST or "CAPABILITY domain", no?

Again you are somehow assuming that every conversation with 
every IMAP server issues a STARTTLS.  An IMAP server doesn't
even have to offer STARTTLS, most do because it's a good idea,
but then you've got the same optional aspect on the client side.

The real point is being able to connect to an IMAP server and
identify what domain context you want the conversation to
occur in.  If STARTTLS were as ubiquitous as CAPABILITY
and EVERY SESSION issued one, then I'd agree.  But it's not.

I also really don't understand why you would favor STARTTLS 
over CAPABILITY?  It makes no sense to me.  STARTTLS is only
issued in the context of creating a secure connection,
CAPABILITY is always offered.  STARTTLS is discovered
post CAPABILITY, so why not do it at the first opportunity?
Further, whether or not STARTTLS is even offered may depend
completely on what domain the client was trying to reach.
(For instance if the customer hasn't provided a security
certificate to be used, then why should the server offer
that domain STARTTLS functionality?)

The point is to let the server switch configurations based
on the targetted domain.  STARTTLS doesn't give us that because
it isn't gauranteed to be issued.  CAPABILITY and VHOST are
both gauranteed to be issued if the client supports it.

An argument to STARTTLS, or an argument to CAPABILITY sounds
like six of one half a dozen of the other, it's still an
extra argument that gets added to something that currently
doesn't take one.

You've dismissed the first two benefits of the three as not
needed and then based on that came up with "oh only STARTTLS
is needed" where for no extra cost (since you gotta add 
something somewhere), if we add it to CAPABILITY we get all 
three.  Those of us who have actually had to deal with the 
issue, which by your dismissal of points 1 and 2 makes it 
obvious you haven't seriously had to deal with it, believe 1 
and 2 are important and have suggested a solution which comes 
out to costing the same as an argument to STARTTLS, and 
provides the same benefit as an argument to STARTTLS, but 
also provides extra benefits that an argument to STARTTLS 
doesn't offer us (otherwise we probably would have proposed 
it that way to begin with).

While I will certainly agree that STARTTLS and SASL mechanisms
are the primary purposes, there are other secondary gripes
that get handled by doing it via a VHOST or argument to
CAPABILITY.  I personally think that even VHOST is a waste.
It gives us nothing that an argument to CAPABILITY doesn't
give us, and it mandates extra round trips.  An argument
to CAPABILITY only causes _one_ extra round trip during the
transitional period where new clients try and access servers
that haven't been updated to ignore the argument yet.

If you haven't had to deal with hosting lots of domains
on a limit number of IPs then please don't assume you
know what the issues are and what's important and what's
not.  I've tried to make the proposal as simple as possible
_but no simpler_ to get the information we need.

I'm more than willing to explain and educate, and there
is a very large list of other things that you can start
doing once you have this type of functionality that I'm
refraining from mentioning because I believe that giving 
a huge laundry list of minor nice-to-haves distracts from
the core issues.

For instance one "nice-to-have" is to aggregate multiple 
domains to act as one single domain.  Why? you ask..  It
doesn't matter.  If you've been there you'd get it, and 
you haven't then you don't really need to know.  All you
need to know is that being able to aggregate multiple
domain names together to act as a single group is useful.
Not as useful as being able to make every domain its own
unique entity, and certainly not something all servers
are going to support, but for those people who need the
functionality, they're grateful that the protocol built 
in a space for a few extra bytes of information where
the client can announce the server it was trying to reach.

I'm still assuming that people on this list are still
somewhat dubious about whether or not virtual hosting
multiple domains on a single IP address is even useful.

If that's still the case, then I'll be more than happy
to take arguments and suggestions to find some other way
to solve the problem until we can all agree.  Believe
me when I say I'll be more than happy to listen because 
if there is a way around it then hopefully it can be 
applied to lots of protocols that lack this support, not
just IMAP and the host providers of the world won't have 
to wait for new versions of clients and servers before 
they can start rolling this out.  

However, I'll warn you, I've been working on the problem 
for six years, I've worked with lots of really smart 
networking engineers and system administrators who have 
also been working on it for several years (some longer 
than I have), and so far, no one in the entire hosting 
industry has been unable to find a generally useful 
solution that isn't based on the protocol providing a 
special place where the intended server name is provided.

I'll tell you what would be even more useful is just a
general command where the client can pass a set of 
generic key value pairs to inform the server of generic
session parameters where clients and servers can be 
modified to look for and send arbitrary info.  Kind of
like the "X-" headers in SMTP.  That way the server name
could be stuffed into one of those, and it would not
require a protocol change... A simple formal registry
could maintain the most useful and widely accepted 
options that clients and server normally adopt.
However, it's not clear to me the overall usefulness
of the feature, whether it should be unidriectional or
bidirectional, and how complicated the command should be.
So for now, I'd be more than happy with just a server 
name submitted by the client to the server as early as
possible during the conversation.

-- Michael --

Date: Tue, 29 Apr 2003 12:06:17 -0700 (PDT)
From: [email-masked]
Subject: Review requested: draft-siemborski-mupdate-03.txt to be published as an experimental RFC
To: [email-masked]
Cc: [email-masked], [email-masked]
Sender: [email-masked]

I've received a request from the author to publish

    The MUPDATE Distributed Mailbox Database Protocol
    draft-siemborski-mupdate-03.txt

as an experimental RFC. It is customary for area directors to ask for
such documents to be reviewed by one or more working groups working in the
same or related area. Accordingly, I'm asking folks involved in IMAPEXT
to review this document and comment on its suitability for publication as
an experimental protocol.

TIA.

				Ned

P.S. I have already sent the author some feedback on missing boilerplate and
other formalities.

Date: Tue, 29 Apr 2003 13:09:24 -0400
From: Cyrus Daboo <[email-masked]>
To: Timo Sirainen <[email-masked]>, Philip Guenther <[email-masked]>
cc: Michael Fair <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Sender: [email-masked]

Hi Timo,

--On Tuesday, April 29, 2003 7:39 PM +0300 Timo Sirainen <[email-masked]> wrote:

| Related to 2, I've thought about giving different capability lists per
| user. Couldn't both of these be solved by letting server reply with
| untagged CAPABILITY reply (or resp-text-code) after login? All clients
| might not update their lists, but that's better than having
| extra/missing capabilities listed.

rfc3501 already states that servers may send a capability response code in 
the tagged OK to the AUTHENTICATE command. Servers can also send the 
capability response code in the greeting response too. If both of those are 
implemented, then a client would only need to use the CAPABILITY command 
after STARTTLS.

-- 
Cyrus Daboo

Subject: Re: IMAP Capability and Virtual Domains
From: Timo Sirainen <[email-masked]>
To: Philip Guenther <[email-masked]>
Cc: Michael Fair <[email-masked]>, [email-masked], Harrie Hazewinkel <[email-masked]>
Organization: 
Date: 29 Apr 2003 19:39:03 +0300
Sender: [email-masked]

On Tue, 2003-04-29 at 14:36, Philip Guenther wrote:
> There are three 'issues' that the VHOST capability claims to address:
> 1) the ad-hoc interpretation of authentication IDs as specifying user
>    and domain 
> 2) the ability to have domain-specific capability lists
> 3) the choice of STARTTLS certificate by the server

Related to 2, I've thought about giving different capability lists per
user. Couldn't both of these be solved by letting server reply with
untagged CAPABILITY reply (or resp-text-code) after login? All clients
might not update their lists, but that's better than having
extra/missing capabilities listed.

From: Arnt Gulbrandsen <[email-masked]>
To: Philip Guenther <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date:   Tue, 29 Apr 2003 13:58:50 +0200
Sender: [email-masked]

Philip Guenther writes:
> Arnt Gulbrandsen <[email-masked]> writes:
>> As long as VHOST does not block, it can be added almost anywere. I'd 
>> add it in step E: "send VHOST and STARTTLS".
>
> Imap servers are allowed to reorder pipelined commands, so that won't work.

I'd quibble... but you're effectively right. Pity.

> (The tagged OK of the STARTTLS command is sent _before_ TLS 
> negotiation and is therefore not protected.)

(The server can send an unsolicited untagged OK afterwards. ISTR having 
seen one do. UW, I think... but it's at least a year ago.)

OK. VHOST must add another round-trip. Then I dislike it even more.

How about using something like 'STARTTLS VHOST=fqdn'?

--Arnt

To: Arnt Gulbrandsen <[email-masked]>
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
From: Philip Guenther <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 04:40:49 -0700
Sender: [email-masked]

Arnt Gulbrandsen <[email-masked]> writes:
>
>Harrie Hazewinkel writes:
>> In such a case I believe we could 'convert' CAPABILITY by ading 
>> 'VHOST' semantics when the server is before the STARTTLS and/or 
>> AUTH/LOGIN.
>
>As long as VHOST does not block, it can be added almost anywere. I'd add 
>it in step E: "send VHOST and STARTTLS".

Imap servers are allowed to reorder pipelined commands, so that won't
work.

>> What I neither understand why was the response of a CAPABILITY not 
>> added to the response of STARTTLS for example. Allowing that would 
>> have reduced the extra CAPABILITY commands and the associated 
>> waiting.
>
>It is. RFC 3501 resp-code-text includes capability-data, and section 
>7.2.1 describes how to do it. Recent versions of UW imapd use that.

rfc3501, section 6.2.1:
      Once [TLS] has been started, the client MUST discard cached
      information about server capabilities and SHOULD re-issue the
      CAPABILITY command.  This is necessary to protect against man-in-
      the-middle attacks which alter the capabilities list prior to
      STARTTLS.  The server MAY advertise different capabilities after
      STARTTLS.

(The tagged OK of the STARTTLS command is sent _before_ TLS negotiation
and is therefore not protected.)

Philip Guenther
[email-masked]

To: "Michael Fair" <[email-masked]>
Cc: [email-masked], "Harrie Hazewinkel" <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains 
Date: Tue, 29 Apr 2003 04:36:50 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

"Michael Fair" <[email-masked]> writes:
...
>Current single IP implementations pack the domain info in
>the usernames and then do some magic after authentication to 
>create the domain specific view.  How they pack the domain
>info varies from site to site, but the username is the
>only client submitted information that can reliably be
>used to identify the targetted domain.
>
>The problem with this solution is that there are several
>things that happen before LOGIN or AUTH where knowing the
>domain would be useful.  Namely STARTTLS and CAPABILITY.

There are three 'issues' that the VHOST capability claims to address:
1) the ad-hoc interpretation of authentication IDs as specifying user
   and domain 
2) the ability to have domain-specific capability lists
3) the choice of STARTTLS certificate by the server

In my opinion, the first of these is a non-issue in practice.  Indeed,
the VHOST extension leaves the client author in a sticky situation: if
it accepts the authentication ID as a single value ready for presenting
to LOGIN (or AUTHENTICATE PLAIN/CRAM-MD5/etc), then it must decide and
document how it splits that into a domain for VHOST and a username for
LOGIN.  If it accepts the authentication ID as separate domain and user
values, then it must guess how to join them for non-VHOST-enabled
servers, or have another configuration parameter for _that_.  The VHOST
extension would also need to specify how it affects and/or operates with
SASL mechanisms that convey the user's domain separate from their
username.

The second issue appears minor to me.  Is it more than a matter of
differing domains being allowed different SASL mechanisms?  Is that a
practical problem?

I agree that the third issue is a real problem.  IMHO, STARTTLS would
have been better defined to take an "fqdn" argument that the server
could use to decide which certificate to use.  An extension that did
only that---add an optional domain argument to STARTTLS---would appear
to provide most of the benefits of VHOST.

>The current IMAP extension draft adds a "VHOST" command
>to specify which virtual host the client was trying to reach.
>Overall I think it's a good proposal, but I believe the
>same result can be accomplished without the need for adding
>a new command.
>
>I had an offline discussion with Mark Crispin and proposed
>adding the domain name as an optional parameter to CAPABILITY.

It seems to me that the CAPABILITY command is ill-suited for such
treatment.  If the client already has the server's list of capabilities
and therefore knows that the server supports the extension, then
overloading the CAPABILITY command doesn't save you anything.  If the
client doesn't already have the server's list of capabilities, then the
client will have to either try the extended version and fallback on BAD,
or send an unextended request with a followup if the capability is
advertised, either way requiring two roundtrips against some set of
servers.  Leaving the VHOST command separate from CAPABILITY while
strengthing the recommendation that the server include a CAPABILITY
response code in the initial PREAUTH or OK response and that clients
recognize such would appear to be a simpler and cleaner approach.

...
>As an added bonus, this also lays the groundwork for 
>some future planning on replicating and scaling IMAP 
>mail stores to span multiple mail stores to provide 
>geograhic redundancy.  Without this feature the "IP burn 
>rate" jumps to 1+N addresses per domain where N is the 
>number of geographically redundant mail stores.

I don't understand how geographically redundant mail stores are helped
out any more than any set of N mail stores that offer STARTTLS.  If the
problem is the requirement for separate IPs for STARTTLS, then giving
STARTTLS an optional argument would "lower the burn rate" just as much
as VHOST or "CAPABILITY domain", no?

Philip Guenther
[email-masked]

From: Arnt Gulbrandsen <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: IMAP Extensions WG <[email-masked]>
Date:   Tue, 29 Apr 2003 12:02:21 +0200
Sender: [email-masked]

Harrie Hazewinkel writes:
> In such a case I believe we could 'convert' CAPABILITY by ading 
> 'VHOST' semantics when the server is before the STARTTLS and/or 
> AUTH/LOGIN.

As long as VHOST does not block, it can be added almost anywere. I'd add 
it in step E: "send VHOST and STARTTLS".

> What I neither understand why was the response of a CAPABILITY not 
> added to the response of STARTTLS for example. Allowing that would 
> have reduced the extra CAPABILITY commands and the associated 
> waiting.

It is. RFC 3501 resp-code-text includes capability-data, and section 
7.2.1 describes how to do it. Recent versions of UW imapd use that.

> Adding a fifth wait is a trade-off compared to the extra functionality on
> can get.

Explain.

--Arnt

Date: Tue, 29 Apr 2003 11:46:49 +0200
Subject: Re: IMAP Capability and Virtual Domains
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
From: Harrie Hazewinkel <[email-masked]>
Sender: [email-masked]

On Tuesday, April 29, 2003, at 11:26 AM, Arnt Gulbrandsen wrote:

> Harrie Hazewinkel writes:
>> Correct me if I am wrong, but "do clients not have to wait for 
>> STARTTLS to finish also before a login procedure can be given??"
>
> A client must wait for STARTTLS, yes. Regrettable, but given the 
> nature of the command it's unavoidable.
>
> Today, with most servers, I have to
>
> a) connect
> b) wait until I see the IMAP banner
> c) send CAPABILITY
> d) wait for the answer
> e) send STARTTLS (or switch to port 993 and repeat from step a)
> f) wait for the answer
> g) do TLS negotiation
> h) send CAPABILITY
> i) wait for the answer
> j) send AUTH or LOGIN
> k) wait for the answer
>
> Steps d, f, i and k add up to more than enough waiting. Adding a fifth 
> wait is not a good idea.

In such a case I believe we could 'convert' CAPABILITY by ading 'VHOST' 
semantics
when the server is before the STARTTLS and/or AUTH/LOGIN.
What I neither understand why was the response of a CAPABILITY not added
to the response of STARTTLS for example. Allowing that would have 
reduced the
extra CAPABILITY commands and the associated waiting.

Adding a fifth wait is a trade-off compared to the extra functionality 
on
can get. In this case it is easier server administration that does not
directly benefit a user. :-((

Harrie

From: Arnt Gulbrandsen <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Cc: IMAP Extensions WG <[email-masked]>
Date:   Tue, 29 Apr 2003 11:26:24 +0200
Sender: [email-masked]

Harrie Hazewinkel writes:
> Correct me if I am wrong, but "do clients not have to wait for 
> STARTTLS to finish also before a login procedure can be given??"

A client must wait for STARTTLS, yes. Regrettable, but given the nature 
of the command it's unavoidable.

Today, with most servers, I have to

a) connect
b) wait until I see the IMAP banner
c) send CAPABILITY
d) wait for the answer
e) send STARTTLS (or switch to port 993 and repeat from step a)
f) wait for the answer
g) do TLS negotiation
h) send CAPABILITY
i) wait for the answer
j) send AUTH or LOGIN
k) wait for the answer

Steps d, f, i and k add up to more than enough waiting. Adding a fifth 
wait is not a good idea.

--Arnt

Date: Tue, 29 Apr 2003 11:10:10 +0200
Subject: Re: IMAP Capability and Virtual Domains
Cc: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
From: Harrie Hazewinkel <[email-masked]>
Sender: [email-masked]

On Tuesday, April 29, 2003, at 10:51 AM, Arnt Gulbrandsen wrote:

> I don't much like VHOST. Its benefit over the current user@host hacks 
> seems limited to providing the right TLS certificate. But I could live 
> with it, provided it were simpler.
>
> a) Clients using VHOST should always send the IMAP server's FQDN as 
> argument. Nothing else.
>
> b) BAD should be sent only for syntax errors or command unknown.
>
> c) Clients should explicitly not have to wait for the answer before 
> proceeding with the login procedure.

Correct me if I am wrong, but "do clients not have to wait for STARTTLS 
to finish also
before a login procedure can be given??"

Harrie

From: Arnt Gulbrandsen <[email-masked]>
To: Harrie Hazewinkel <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: IMAP Capability and Virtual Domains
Date:   Tue, 29 Apr 2003 10:51:09 +0200
Sender: [email-masked]

I don't much like VHOST. Its benefit over the current user@host hacks 
seems limited to providing the right TLS certificate. But I could live 
with it, provided it were simpler.

a) Clients using VHOST should always send the IMAP server's FQDN as 
argument. Nothing else.

b) BAD should be sent only for syntax errors or command unknown.

c) Clients should explicitly not have to wait for the answer before 
proceeding with the login procedure.

--Arnt

From: "Michael Fair" <[email-masked]>
To: <[email-masked]>
Cc: "Harrie Hazewinkel" <[email-masked]>
Subject: IMAP Capability and Virtual Domains
Date: Tue, 29 Apr 2003 00:30:46 -0700
Organization: DaClubhouse
Sender: [email-masked]

Hi all,

I understand that in general this list is not very actively
interested in scaling the IMAP protocol to support lots of 
domains on a single IP address.  Myself and I know a number
of others (mainly hosting companies) are interested in this 
feature.  Currently only HTTP 1.1 has any kind of support 
for this kind of scaling.

I'll be brief, a quick education on the issues for those that 
don't understand the problem or haven't read the VHOST extension 
draft and then a suggestion that I believe would quite neatly 
solve all the issues plus a couple that none of the existing 
solutions offer while still maintaing compatibility with 
existing clients/servers and allow for a mostly gradual 
transition between non-virtualized and virtualized deployment.

The issues:
At the core of it, in most hosting for hire environments
you typically get a lot of domains that only have a few
users (less than 30).  In a modest ISP, having to support
1,000 domains is very reasonable.  The vast majority of 
those domains typically have less than 5 accounts, and
the number with more than 30 accounts is usually countable
on one or two hands.  This creates a situation where you
have about 10,000-20,000 accounts spread over 1,000 domains.

This is easily hosted on any decent sized server and hosting
this on a single machine is a very reasonable thing to do.
The problem is that this ISP typically has a /21 giving
it a total of 1,275 IP addresses.  If this ISP was to
try and give the highest quality of service at 1 IP
address per domain, which the existing protocol calls 
for, this ISP would have 1,000 IP addresses all pointed
at the same server and have 273 left over for other uses.
In the case of the medium size ISP it is actually impossible 
to even consider offering such a service.  This is especially 
compounded if you start to include other services like FTP.  

As another consideration this doesn't even touch trying to 
provide geographically redundant services where the data is 
accessible from at least two (preferably three) redundant 
sites that are located on completely separate IP networks.
(This means at least two preferably three IPs per domain)

So the short of it is that for certain very common 
environments there is a clear and definite need for being 
able to multiplex multiple domains onto a single IP address.

A brief overview on the current state of trying to 
accomplish this:

When a client shows up at the server's door all it knows 
is the IP address the client was trying to reach.  This 
is completely insufficient to determine the targetted domain 
unless you have 1 IP per domain.

In short, the server must be told directly what domain
the client was trying to reach before it can access any
domain specific parameters (these include shared folders,
security certificates and any other parameters a server
might want to modify on a per domain basis (like AUTH
services)).  The other need for per domain identification
is to allow for [email-masked] to be differentiated 
from [email-masked] when the client only presents the 
username "bob".

Current single IP implementations pack the domain info in
the usernames and then do some magic after authentication to 
create the domain specific view.  How they pack the domain
info varies from site to site, but the username is the
only client submitted information that can reliably be
used to identify the targetted domain.

The problem with this solution is that there are several
things that happen before LOGIN or AUTH where knowing the
domain would be useful.  Namely STARTTLS and CAPABILITY.

The current IMAP extension draft adds a "VHOST" command
to specify which virtual host the client was trying to reach.
Overall I think it's a good proposal, but I believe the
same result can be accomplished without the need for adding
a new command.

I had an offline discussion with Mark Crispin and proposed
adding the domain name as an optional parameter to CAPABILITY.

My assumption was that the very first thing an IMAP client
is going to do (even before STARTTLS) is CAPABILITY to find
out if the server can even do STARTTLS and to figure out
what methods of authentication the server makes available.
By identifying the domain prior to any other commands,
the server has the opportunity to get into a domain
specific mode very early.  For instance, STARTTLS could
now provide a per domain certificate instead of being
forced to provide a single (usually ISP specific) cert,
or even worse, not offering IMAPS at all.

If that assumption is wrong, then I hope that someone would
correct me as it makes this suggestion rather worthless and
makes the VHOST addition a more viable solution.

Once a client had issued a CAPABILITY with a domain parameter,
the server would go into a domain specific mode where all 
responses would be local to that domain's config.  The main 
restrictions for a client would be that during a TLS session
and/or while in the AUTHENTICATED state the client MUST NOT
issue a CAPABILITY where that domain parameter is different.
If a client were allowed to jump domains in the middle of
those contexts it would be ambiguous at best, and create
security holes at worst.  The server is of course encouraged
to take whatever steps are appropriate to ensure that the
client doesn't try and jump domains in these contexts.  

Currently CAPABILITY accepts no parameters and it's an
error to present one.  Clients could try first with the
domain, and then upon receiving BAD try without the domain.

If a server supported the parameter, but not the requested
domain, the server could either respond with NO saying that
the domain isn't supported, or it could respond with YES 
if the server just swallows the parameter and gives the 
same behavior regardless of the domain.  This makes the 
existing servers behavior remain correct, allows for servers
to be compatible by simply swallowing the parameter and 
not doing anything with it, and future servers can implement
a true virtualized hosted environment.  If a client tries to
jump domain contexts, the server should answer "NO" if it 
would have answered "NO" to an unavailable or not supported 
domain, or it should simply swallow the parameter or respond 
with "BAD" if it would have swallowed the parameter or 
reposnded with "BAD" before.

Clients that support this need only be prepared to 
receive a BAD or NO when issuing a CAPABILITY and 
conform to not trying to jump domains when in a TLS 
session or AUTHENTICATED state.

As an added bonus, this also lays the groundwork for 
some future planning on replicating and scaling IMAP 
mail stores to span multiple mail stores to provide 
geograhic redundancy.  Without this feature the "IP burn 
rate" jumps to 1+N addresses per domain where N is the 
number of geographically redundant mail stores.  It's 
true that this might solveable with a distributed file 
system instead of an IMAP extension protocol, but I 
believe it would be harmful to ignore the idea that 
there are those in the world who care enough about 
reliability to use three or four distributed machines 
to maintain replicas of the mail store for performance 
and redundancy.

Thank you very much for reading and I look forward to
comments and feedback.  Again, I do understand that 
this is not a need that most people on this list consider
very important so taking the time out to lend some
grey matter to it is greatly appreciated.

-- Michael --

Subject: Re: Full text search and SEARCHM IMAP extention
From: Timo Sirainen <[email-masked]>
To: Takeshi Morishima <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 23 Apr 2003 15:09:05 +0300
Sender: [email-masked]

On Wed, 2003-04-23 at 06:28, Takeshi Morishima wrote:
> Will try to update the proposal modification based on the comments.  I
> need study LISTEXT more - don't fully understand how it's related or
> can be merged.

Actually, forget it. I was just thinking that since they added STATUS
support there, it could be used as some generic "for each listed
mailbox, do <stuff>". But it'd require replying with LIST replies, which
is probably not so useful with SEARCHM. And if result has to be sorted,
it's not so simple.

Date: Tue, 22 Apr 2003 22:28:57 -0500
From: [email-masked] (Takeshi Morishima)
To: [email-masked]
CC: [email-masked]
Subject: Re: Full text search and SEARCHM IMAP extention
Sender: [email-masked]

In message "Re: Full text search and SEARCHM IMAP extention" on
    2003/04/22, Timo Sirainen writes:
  > One thing that I've thought that could be useful is to return the text
  > that matched so client could display google-like search results directly
  > without fetching all the messages.

I'd agree that this may be useful.  This would be new data type
though, with mime content headers re-constructed for summary text.

  > I'm not sure if msgmax / msgstart is such a good idea. You
  > probably want it for prev/next page-like feature, but if new
  > matches appear between searches, you might miss some messages.

My initial motivation of these options was to allow client to specify
max number of messages to be returned.  Client could prepare only
resource that would be needed that way, thus more effcient use of
resources.  It also saves a bit of bandwidth of returning only 200 of
top score out of total 1000 hits.  Then suddenly I thought that if we
limit the amount of search result by msgmax, I would need msgstart in
case user is not satisfactory in the search result and want to get
next block of result, since otherwise the client may end up receiving
the same block of data repeatedly as it increases msgmax, which did
not sound very efficient.

For the issue of result changes between searches, I also thought about
it, and had one idea of using the search name to identify previous
search context so that the server can remember it and provide
consistent output.  This would require the client to use differnt
search name everytime new search is performed, the server side to
provide more resource, and will complicate things, which did not sound
like a good idea.  Since this modified searchm is intended for best
effort search, I thought that dup or missing messages may not be an
issue for typical case, resulted in this form.  (Probably no need of
msgend, just use msgstart+msgmax.)

Will try to update the proposal modification based on the comments.  I
need study LISTEXT more - don't fully understand how it's related or
can be merged.

Regards,
Takeshi

From: Arnt Gulbrandsen <[email-masked]>
To: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: IMAP Extensions WG <[email-masked]>
Date: Tue, 22 Apr 2003 23:28:29 +0200
Sender: [email-masked]

Mark Crispin writes:
> This is in effect an IMAP interface to an ultra-lightweight 
> notification protocol and facility that I am working on. The IMAP 
> server won't do any actual "watching"; instead, it will just be a 
> client to the notification facility.

Isn't that a natural implementation choice?

> I am interested in feedback as to whether this proposed extension is 
> interesting or not.

Yes. But as internet-draft, please.

> Also, would it be better to build an extension which has extensibility 
> but with restraints ("ultra-lightweight") on how it can be extended, 
> or if it should be strictly a new mail watching facility
...

If the latter, perhaps it can avoid the fate of so many similar proposals?

--Arnt

From: Arnt Gulbrandsen <[email-masked]>
To: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Tue, 22 Apr 2003 20:50:56 +0200
Sender: [email-masked]

Mark Crispin writes:
> On Tue, 22 Apr 2003, Arnt Gulbrandsen wrote:
>>  > LIST can never be more expensive than STATUS.
>>  I can disprove that by counterexample.
>>  C: b list "" *
>
> You are comparing apples and oranges. There is no such thing as a 
> wildcard STATUS.

Perhaps I misunderstood. I was comparing LIST to a for loop sending 
STATUS for e.g. 1800 mailboxes. Seemed a fair enough comparison.

--Arnt

Date: Tue, 22 Apr 2003 10:37:15 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 22 Apr 2003, Arnt Gulbrandsen wrote:
> > LIST can never be more expensive than STATUS.
> I can disprove that by counterexample.
> C: b list "" *

You are comparing apples and oranges.  There is no such thing as a
wildcard STATUS.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: Pending changes to CONDSTORE
From: Timo Sirainen <[email-masked]>
To: Alexey Melnikov <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 22 Apr 2003 19:19:55 +0300
Sender: [email-masked]

On Tue, 2003-04-22 at 08:00, Alexey Melnikov wrote:
> I haven't resolved an open issue regarding atomicity of CONDSTORE for a
> message set. I am still waiting for more feedback.

Yes, it could be difficult with maildir at least since you can't prevent
flag changes by other clients. Is there even a reason to want atomicity?

Is there a reason to return MODSEQ separately for each message with
SILENT STORE? And why not return the modseq and flags directly for
modified messages, wouldn't client want to ask them anyway next?
Something like:

a STORE 100:150 (UNCHANGEDSINCE 1) +FLAGS.SILENT ($Processed)
* 101 FETCH (MODSEQ (2) FLAGS ($Processed))
* 105 FETCH (MODSEQ (2) FLAGS ($Processed))
a OK [MODSEQ 3 MODIFIED 101,105] Some flags weren't changed

b STORE 100 (UNCHANGEDSINCE 1) +FLAGS.SILENT ($Processed)
* 100 FETCH (MODSEQ (3) FLAGS ($Processed))
b OK [MODIFIED 100] No flags were changed

c STORE 100,151 (UNCHANGEDSINCE 1) +FLAGS ($Processed)
* 100 FETCH (MODSEQ (3) FLAGS ($Processed))
* 151 FETCH (MODSEQ (4) FLAGS ($Processed))
c OK [MODSEQ 4 MODIFIED 100] Some flags weren't changed

From: Arnt Gulbrandsen <[email-masked]>
To: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: Cyrus Daboo <[email-masked]>, Lawrence Greenfield <[email-masked]>, IMAP Extensions WG <[email-masked]>
Date: Tue, 22 Apr 2003 17:22:08 +0200
Sender: [email-masked]

Mark Crispin writes:
> LIST can never be more expensive than STATUS.

I can disprove that by counterexample.

I have a crappy IMAP client (came preinstalled on a machine I bought) 
whose login sequence is like this when it talks to wu-imapd:

C: a login arnt tnra
S: a ok
C: b list "" *
S: ...
C: (run out of memory and crash)

The server's resource use is limited only by bandwidth and the client's 
available RAM, which is potentially far more than it would spend 
processing STATUS.

--Arnt

Subject: Re: Full text search and SEARCHM IMAP extention
From: Timo Sirainen <[email-masked]>
To: Takeshi Morishima <[email-masked]>
Cc: [email-masked]
Organization: 
Date: 22 Apr 2003 16:25:25 +0300
Sender: [email-masked]

On Tue, 2003-04-22 at 15:17, Takeshi Morishima wrote:
> My comments and proposed modification to the current SEARCHM draft are
> described in the following file:
> 
> http://www.ml-search.com/tmpdocs/draft-leiba-imap-search-multiple-01-comment.pdf
> 
> It proposes to allow server to ignore unsupported criteria/mailbox,
> provide sorted result fragmentation, more options to control search
> result, and a new HITS response to return search statistics numbers.

One thing that I've thought that could be useful is to return the text
that matched so client could display google-like search results directly
without fetching all the messages. This would be especially useful if
server supported searching inside non-text file attachments like ms word
docs, pdfs, postscripts, etc.

> Also, because the search will be performed outside of selected state,
> I thought that returning message sequence numbers may have a problem,
> and for other reasons, I'm proposing to return UIDs and UIDVALIDITY
> always in the SEARCHM response.

Right. Maybe remove the whole "UID SEARCHM".

Some other thoughts:

Sorting conflicts a bit with SORT extension. At least change "sentdate"
to "date" so they're compatible.

I'm not sure if msgmax / msgstart is such a good idea. You probably want
it for prev/next page-like feature, but if new matches appear between
searches, you might miss some messages.

I think "Mailbox URL" is wrong word to describe LIST-like mailbox
wildcarded names. And should this maybe be merged with LISTEXT?

Returning UIDVALIDITY before UIDs would be nicer. And it doesn't have to
be in paranthesis.

Anyway I'd like to have this kind of non-strict text searching. Current
SEARCH can be useful for client internally, but for normal searching by
user it's unnecessarily slow because it's strictness really limits the
optimizations that server can do.

Date: Tue, 22 Apr 2003 07:17:42 -0500
From: [email-masked] (Takeshi Morishima)
To: [email-masked]
Subject: Full text search and SEARCHM IMAP extention
Sender: [email-masked]

Hello there,

I've been looking into implementing a full text search on a client, to
provide a multiple mailbox search with search performance similar to
what we can find on many www based search engines.  After implementing
a repeat of select and search over multiple mailboxes, I found it slow
and in most cases the search result does not appear very optimal.  No
framework to handle something like scoring based search result
sorting, for example.

Looking through existing RFCs and imap extention drafts, the closest
to what I can use was the SEARCHM extention draft.  The current
SEARCHM draft, however, does not seem to be enough for the purpose of
the above type of search.  I would like this to be loosely coupled,
best effort and best practice search capable, instead of requiring a
strict listing for house keeping of client that SEARCHM is mainly
intended to.  A full text search would not need to deal with something
like "unseen" type of search, and can omit messages that unlikely of
user interest in a search result.

My comments and proposed modification to the current SEARCHM draft are
described in the following file:

http://www.ml-search.com/tmpdocs/draft-leiba-imap-search-multiple-01-comment.pdf

It proposes to allow server to ignore unsupported criteria/mailbox,
provide sorted result fragmentation, more options to control search
result, and a new HITS response to return search statistics numbers.
Also, because the search will be performed outside of selected state,
I thought that returning message sequence numbers may have a problem,
and for other reasons, I'm proposing to return UIDs and UIDVALIDITY
always in the SEARCHM response.

Since the purposes of SEARCHM and the modified version above slightly
differ, should it be different command?  Or is this something that can
be incorporated into the current searchm draft.

If this is not worth discussing in this WG list nor standardization,
please advice where I should take this to - any comments and
suggestions would be appreciated.

Regards,
Takeshi

To: IETF-Announce: ;
Cc: [email-masked]
From: [email-masked]
Reply-to: [email-masked]
Subject: I-D ACTION:draft-ietf-imapext-condstore-01.txt
Date: Tue, 22 Apr 2003 07:40:02 -0400
Sender: [email-masked]

--NextPart

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Internet Message Access Protocol Extension Working Group of the IETF.

	Title		: IMAP Extension for Conditional STORE operation
	Author(s)	: A. Melnikov, S. Hole
	Filename	: draft-ietf-imapext-condstore-01.txt
	Pages		: 16
	Date		: 2003-4-21
	
Often, multiple IMAP clients need to coordinate changes to a common
IMAP mailbox.  Examples include different clients for the same user,
and multiple users accessing shared mailboxes. These clients
need a mechanism to synchronize state changes for messages within the
mailbox. They must be able to guarantee that only one client can change
message state (e.g., message flags or annotations) at any time.  An
example of such an application is use of an IMAP mailbox as a message
queue with multiple dequeueing clients.
The Conditional Store facility provides a protected update mechanism for
message state information that can detect and resolve conflicts between
multiple writing mail clients.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-01.txt

To remove yourself from the IETF Announcement list, send a message to 
ietf-announce-request with the word unsubscribe in the body of the message.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-imapext-condstore-01.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-imapext-condstore-01.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

--NextPart
Content-Type: Multipart/Alternative; Boundary="OtherAccess"

--OtherAccess
Content-Type: Message/External-body;
	access-type="mail-server";
	server="[email-masked]"

Content-Type: text/plain
Content-ID:	<[email-masked]>

ENCODING mime
FILE /internet-drafts/draft-ietf-imapext-condstore-01.txt

--OtherAccess
Content-Type: Message/External-body;
	name="draft-ietf-imapext-condstore-01.txt";
	site="ftp.ietf.org";
	access-type="anon-ftp";
	directory="internet-drafts"

Content-Type: text/plain
Content-ID:	<[email-masked]>

--OtherAccess--

--NextPart--

Date: Mon, 21 Apr 2003 23:00:06 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: IMAP Extensions WG <[email-masked]>
Subject: Pending changes to CONDSTORE
Sender: [email-masked]

I haven't resolved an open issue regarding atomicity of CONDSTORE for a
message set. I am still waiting for more feedback.

Also, I would like to propose to add a new capability if MODSEQ sort
criteria is implemented with SORT. Any objections to this?

Regards,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel
IETF standard
related pages: http://orthanc.ab.ca/mel/devel/Links.html

I speak for myself only, not for my employer.
__________________________________________

Date: Sat, 19 Apr 2003 23:49:27 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: IMAP Extensions WG <[email-masked]>
CC: IMAP Discusson List <[email-masked]>
Subject: Updated draft-ietf-imapext-condstore document
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------6D49FEEA8037F438115381F7
Content-Type: text/plain; charset=koi8-r
Content-Transfer-Encoding: 7bit

I've updated the document as per comments from Mark and Cyrus.
New version (draft-ietf-imapext-condstore-01) should appear shortly. In
the meantime, I've attached the diff for the changes.

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel
IETF standard
related pages: http://orthanc.ab.ca/mel/devel/Links.html

I speak for myself only, not for my employer.
__________________________________________

--------------6D49FEEA8037F438115381F7
Content-Type: text/plain; charset=koi8-r;
 name="00-01.diff.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="00-01.diff.txt"

--- draft-ietf-imapext-condstore-00.txt	Thu Mar 27 00:27:06 2003
+++ draft-ietf-imapext-condstore-01.txt	Fri Apr 11 23:29:42 2003
@@ -1,7 +1,7 @@
 Internet Draft: IMAP Extension for Conditional STORE          A. Melnikov
-Document: draft-ietf-imapext-condstore-00.txt                     S. Hole
-Expires: September 2003                     ACI WorldWide/MessagingDirect
-                                                               March 2003
+Document: draft-ietf-imapext-condstore-01.txt                     S. Hole
+Expires: October 2003                       ACI WorldWide/MessagingDirect
+                                                               April 2003
 
                IMAP Extension for Conditional STORE operation
 
@@ -43,6 +43,13 @@
 
 0.2. Change History
 
+     Changes from draft-ietf-imapext-condstore-00
+      1.  Dropped "/message" prefix in entry names as per decision in San Francisco.
+      2.  Fixed ABNF for SEARCH and SORT untagged responses.
+      3.  Changed "private" to "priv" to be consistent with ANNOTATE.
+      4.  MODIFIED response code is now returned in OK response, not NO.
+      5.  Added NOMODSEQ response code.
+
      Changes from draft-melnikov-imap-condstore-09:
       1.  Some text clarifications based on suggestions by Harrie Hazewinkel
       2.  Added paragraph about mailbox locking and DOS when conditional STORE
@@ -139,6 +146,7 @@
    4  IMAP Protocol Changes ..................................... X
    4.1 New OK untagged responses for SELECT and EXAMINE ......... X
    4.1.1 HIGHESTMODSEQ response code ............................ X
+   4.1.2 NOMODSEQ response code ................................. X
    4.2 STORE and UID STORE Commands ............................. X
    4.3 MODSEQ message data item in FETCH Command ................ X
    4.4 MODSEQ search criterion in SEARCH ........................ X
@@ -240,13 +248,18 @@
     different metadata items. If the server does so, per message mod-sequence is
     the highest mod-sequence of all metadata items for the specified message.
 
+    The server that supports this extention is not required to be able to store
+    mod-sequences for every available mailbox. Section 4.1.2 describes how
+    the server may act if a particular mailbox doesn't support the persistent
+    storage of mod-sequences.
+
     This extension makes the following changes to the IMAP4 protocol:
 
         a) extends the syntax of the STORE command to allow STORE
            modifiers
 
         b) adds the MODIFIED response code which should be used with
-           a NO response to the STORE command
+           an OK response to the STORE command
 
         c) adds a new MODSEQ message data item for use with the FETCH command
 
@@ -269,13 +282,26 @@
 
 4.1. New OK untagged responses for SELECT and EXAMINE
 
+    This document adds two new response codes HIGHESTMODSEQ and NOMODSEQ.
+    One of those response codes MUST be returned in the OK untagged
+    response for a successful SELECT and EXAMINE commands.
+
+    When opening a mailbox the server must check if the mailbox supports
+    the persistent storage of mod-sequences. If the mailbox supports
+    the persistent storage of mod-sequences and mailbox open operation succeeds,
+    the server MUST send the OK untagged response including HIGHESTMODSEQ
+    response code. If the persistent storage for the mailbox is not supported,
+    the server MUST send the OK untagged response including NOMODSEQ response
+    code instead.
+
+
 4.1.1. HIGHESTMODSEQ response code
 
     This document adds a new response code that is returned in the OK
     untagged response for the SELECT and EXAMINE commands. A server
-    supporting the CONDSTORE extension MUST send the OK untagged
-    response including HIGHESTMODSEQ response code with every successful
-    SELECT or EXAMINE command:
+    supporting the persistent storage of mod-sequences for the mailbox MUST
+    send the OK untagged response including HIGHESTMODSEQ response code with
+    every successful SELECT or EXAMINE command:
 
       OK [HIGHESTMODSEQ <mod-sequence-value>]
 
@@ -307,6 +333,24 @@
                 S: A142 OK [READ-WRITE] SELECT completed

+4.1.2 NOMODSEQ response code
+
+    A server that doesn't support the persistent storage of mod-sequences for
+    the mailbox MUST send the OK untagged response including NOMODSEQ response
+    code with every successful SELECT or EXAMINE command.
+
+    Example:    C: A142 SELECT INBOX
+                S: * 172 EXISTS
+                S: * 1 RECENT
+                S: * OK [UNSEEN 12] Message 12 is first unseen
+                S: * OK [UIDVALIDITY 3857529045] UIDs valid
+                S: * OK [UIDNEXT 4392] Predicted next UID
+                S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
+                S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited
+                S: * OK [NOMODSEQ] Sorry, this mailbox format doesn't support modsequences
+                S: A142 OK [READ-WRITE] SELECT completed
+
+
 4.2. STORE and UID STORE Commands
 
     Arguments:  message set
@@ -328,8 +372,9 @@
          UNCHANGEDSINCE
             If the mod-sequence of any metadata item specified in the STORE
             operation for any message in the message set is greater than the
-            specified unchangedsince value, then the command fails.
-            On failure, a MODIFIED response code is returned which includes
+            specified unchangedsince value, then the command MUST NOT change
+            any flags. Instead, the server replies with the OK tagged response that
+            includes a MODIFIED response code. The MODIFIED response code includes
             the message set (for STORE) or set of UIDs (for UID STORE)
             of all messages that failed the UNCHANGESINCE test.
 
@@ -337,7 +382,7 @@
 
               C: a101 STORE 7,5,9 (UNCHANGEDSINCE 20000320162338)
                   +FLAGS.SILENT (\Deleted)
-              S: a101 NO [MODIFIED 7,9] Conditional STORE failed
+              S: a101 OK [MODIFIED 7,9] Conditional STORE failed
 
 		   In spite of the failure of the conditional STORE operation
                for message 7, the server continues to process the conditional
@@ -351,7 +396,7 @@
 
               C: a102 STORE 12 (UNCHANGEDSINCE 0)
                   +FLAGS.SILENT ($MDNSent)
-              S: a102 NO [MODIFIED 12] Conditional STORE failed
+              S: a102 OK [MODIFIED 12] Conditional STORE failed
 
             Note: A client trying to atomically change the state of a particular
              flag (or a set of flags) should be prepared to deal with the case
@@ -370,7 +415,7 @@
             Example:
               C: a106 STORE 100:150 (UNCHANGEDSINCE 200212030000000)
                   +FLAGS.SILENT ($Processed)
-              S: a106 NO [MODIFIED 101] Conditional STORE failed
+              S: a106 OK [MODIFIED 101] Conditional STORE failed
              
              the flag $Processed was set on the message 101 ...
               C: a107 NOOP
@@ -560,15 +605,15 @@
              cache. The client may also specify <entry-name> (name of metadata
              item) and <entry-type-req> (type of metadata item) before
              <mod-sequence-valzer>. <entry-type-req> can be one of "shared",
-             "private" or "all". The latter means that the server should use
-             the biggest value among "private" and "shared" modseqs for the
+             "priv" (private) or "all". The latter means that the server should use
+             the biggest value among "priv" and "shared" mod-sequences for the
              metadata item. If the server doesn't store internally separate
              mod-sequences for different flags and annotations, it MUST ignore
              <entry-name> and <entry-type-req>. Otherwise the server should
              use them to narrow down the search.
 
              For a flag <flagname> the corresponding <entry-name> has a form
-             "/message/flags/<flagname>" as defined in [ANNOTATE]. Note, that
+             "/flags/<flagname>" as defined in [ANNOTATE]. Note, that
              the leading "\" character that denotes a system flag has to be
              escaped as per Section 4.3 of [IMAP4], as the <entry-name> uses
              syntax for quoted strings.
@@ -580,14 +625,14 @@
     See also section 4.6.
 
     Example:
-         C: a SEARCH MODSEQ "/message/flags/draft" all 20010320162338
-                     ANNOTATION "/message/comment" "value" "IMAP4"
+         C: a SEARCH MODSEQ "/flags/draft" all 20010320162338
+                     ANNOTATION "/comment" "value" "IMAP4"
          S: * SEARCH 2 5 6 7 11 12 18 19 20 23 (MODSEQ 20010917162500)
          S: a OK Search complete
 
             In the above example, the message numbers of any messages
             containing the string "IMAP4" in the "value" attribute of the
-            "/message/comment" entry and having a mod-sequence equal to or
+            "/comment" entry and having a mod-sequence equal to or
             greater than 20010320162338 for the "\Draft" flag are returned in
             the search results.
 
@@ -684,9 +729,6 @@
 
     status-att-req      = status-att / "HIGHESTMODSEQ"
 
-    mailbox-data        =/ "STATUS" SP mailbox SP "("
-                           [status-rsp-info *(SP status-rsp-info)] ")"
-
     status-rsp-info     = status-att SP number /
                           "HIGHESTMODSEQ" SP mod-sequence-value
 
@@ -713,21 +755,22 @@
     search-modseq-ext   = SP entry-name SP entry-type-req
 
     resp-text-code      =/ "HIGHESTMODSEQ" SP mod-sequence-value /
+                           "NOMODSEQ" /
                            "MODIFIED" SP set
 
-    entry-name          = '"' "/message/flags/" attr-flag '"'
+    entry-name          = '"' "/flags/" attr-flag '"'
                           ;; each system or user defined flag <flag>
-                          ;; is mapped to "/message/flags/<flag>".
+                          ;; is mapped to "/flags/<flag>".
                           ;;
                           ;; <entry-name> follows the escape rules used
                           ;; by "quoted" string as described in Section
                           ;; 4.3 of [IMAP4], e.g. for the flag \Seen
                           ;; the corresponding <entry-name> is
-                          ;; "/message/flags/\\seen", and for the flag
+                          ;; "/flags/\\seen", and for the flag
                           ;; $MDNSent, the corresponding <entry-name>
-                          ;; is "/message/flags/$mdnsent".
+                          ;; is "/flags/$mdnsent".
 
-    entry-type-resp     = "private" | "shared"
+    entry-type-resp     = "priv" | "shared"
                           ;; metadata item type
 
     entry-type-req      = entry-type-resp | "all"
@@ -749,8 +792,10 @@
 
 ;;Borrowed from IMAP4rev1 and modified accordingly:
 
-    mailbox-data        =/ "SEARCH" [SP nz-number *(SP nz-number) search_sort_mod_seq] /
-                           "SORT" [SP nz-number *(SP nz-number) search_sort_mod_seq]
+    mailbox-data        =/ "STATUS" SP mailbox SP "("
+                           [status-rsp-info *(SP status-rsp-info)] ")" /
+                           "SEARCH" [1*(SP nz-number) SP search_sort_mod_seq] /
+                           "SORT" [1*(SP nz-number) SP search_sort_mod_seq]
 
     attr-flag           = "\\Answered" / "\\Flagged" / "\\Deleted" /
                           "\\Seen" / "\\Draft" / attr-flag-keyword /
@@ -828,8 +873,8 @@
     interact with ANNOTATE extension and for thorough review of the document.
 
     Authors also acknowledge the feedback provided by Cyrus Daboo, Larry
-    Greenfield, Chris Newman, Harrie Hazewinkel, Arnt Gulbrandsen and Timo
-    Sirainen.
+    Greenfield, Chris Newman, Harrie Hazewinkel, Arnt Gulbrandsen Timo
+    Sirainen and Mark Crispin.

 9. Author's Addresses

--------------6D49FEEA8037F438115381F7--

Date: Sat, 19 Apr 2003 23:27:48 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> Cyrus Daboo wrote:
>
> > The formal syntax for 'entry-name' should be simply defined as:
> >
> >     entry-name = string
> >
> > to fall in line with ANNOTATE.
>
> BTW, there is no ABNF element for entry-name in ANNOTATE, there are "entry" and
> "entry-match".

Also, ANNOTATE SEARCH criteria uses entry-match, which allows for wildcards.
CONDSTORE doesn't use wildcards.
Any opinions whether the document should/should not allow for wildcards?

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel
IETF standard
related pages: http://orthanc.ab.ca/mel/devel/Links.html

I speak for myself only, not for my employer.
__________________________________________

Date: Sat, 19 Apr 2003 23:20:49 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> The formal syntax for 'entry-name' should be simply defined as:
>
>     entry-name = string
>
> to fall in line with ANNOTATE.

BTW, there is no ABNF element for entry-name in ANNOTATE, there are "entry" and
"entry-match".

> There should be a comment/reference to the
> annotate spec at that point.

Sure.

> That will also allow you to remove the comment
> about escape rules at that point.

There is nothing wrong with repeating obvious ;-).

> Also, could be use "priv" instead of "private" for entry-type-resp just to
> fall in line with the suffixes that ANNOTATE uses?

So you want to use "priv" and "shared"? I've just updated the document.

Alexey Melnikov
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Sat, 19 Apr 2003 22:48:40 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Mark Crispin <[email-masked]>
CC: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Mark Crispin wrote:

> On Sat, 12 Apr 2003, Cyrus Daboo wrote:
> > | Why would one want to use MODSEQ with THREAD?
> > For the same reason one would use it in the search criteria on sort: to
> > have the command return results only for those messages that changed since
> > the MODSEQ. I doubt this is really a useful behaviour but it has to be
> > present for consistency if you are adding MODSEQ to search unless its
> > explicitly stated MODSEQ can't be used in sort/thread search criteria.
>
> I'm hard-pressed to think of a use for MODSEQ with THREAD; and while SORT
> presents some possible use I doubt that there would be much of that in
> real life.

I agree with Mark here.
I can see a utility in MODSEQ criterion in SEARCH and MODSEQ sort order in SORT.

I prefer to live it unspecified how MODSEQ search criteria can be used in
THREAD. Any objections?

> If you declare that MODSEQ is an additional argument to SEARCH (much as
> CHARSET is) instead of a SEARCH criterion, then you avoid the issue.  But
> that's no good if you decide to have MODSEQ in SORT.

Regards,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 18 Apr 2003 20:12:06 -0400
From: Cyrus Daboo <[email-masked]>
To: Ken Murchison <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Ken,

--On Friday, April 18, 2003 3:19 PM -0400 Ken Murchison <[email-masked]> 
wrote:

|> This shows up the problem that the client still has to poll via NOOP to
|> get results. Obviously, if a mailbox was selected, the client could use
|> IDLE and WATCH would come back during IDLE without the need to poll. But
|> how would we deal with a connection that does not have a selected
|> mailbox? The client could select a 'dummy' mailbox and then use idle,
|> but wouldn't it be better to allow IDLE outside of selected state if
|> WATCH is present?
|
| IIRC, IDLE is already allowed in both the authenticated and selected
| states.

You're right - it is. The comment on what state its valid for is in the 
formal syntax and I missed it there.

-- 
Cyrus Daboo

To: Rob Siemborski <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 15:03:01 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Rob Siemborski <[email-masked]> writes:
>On Fri, 18 Apr 2003, Philip Guenther wrote:
>
>> It's all a matter to explaining to the execs the costs of the tradeoffs
>> that they're requesting.
>
>Sure, but that just moves around who is looking at the cost, the fact is
>that server costs aren't the ONLY ones to consider.

Right; that's why I used the word 'tradeoffs' to indicate that there are
costs and benefits to every choice.  Anyone who hides that fact only has
themselves to blame when all the costs end up on their shoulders.

Writing a protocol that does the same by transfering all the costs and
complexity to the server is all too easy.  There are reasons why IMAP
and LDAP are widely implemented and ACAP isn't.

...
>Saying that a proposal can't be improved because it is already better than
>what we currently have doesn't make a lot of sense.

Of course, faster is always better, but
1) going from "poll-interval" to "round-trip time" is a big saving
   compared to going from "round-trip time" to "instantly".  For
   updating 'background' information in a human interface (that's the
   use people have been talking about), I suspect the benefit to the
   user is all in that first big step.
2) clients *do* poll instead of opening N connections, one per-mailbox,
   even though the latter offers "as soon as possible" updates, so
   clients are willing to make a tradeoff when faced with it

If WATCH BIFF is implementable on most servers with what appears to be a
very short poll interval ("I always see BIFFs within a second or so")
but WATCH STATUS varies server to server, sometimes appearing to be as
responsive as BIFFs but other times appearing to have, say, a 30 second
poll interval, how will client authors react?  Always use WATCH BIFF?
Always use WATCH STATUS?  Always use WATCH STATUS and tell your users to
yell at the authors of the slow servers?  Something else?

Philip Guenther

Date: Fri, 18 Apr 2003 13:34:35 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP 
To: Rob Siemborski <[email-masked]>
cc: John Milan <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 16:30:55 -0400 (EDT), Rob Siemborski wrote:
> Sorry, perhaps I should have phrased it as "several people on the list
> have expressed interest in following up on this", (both "WATCH=BIFF" and
> "WATCH=STATUS") but certainly a WATCH extention of some type.

OK.  I agree that there is some concensus with that wording.

Date: Fri, 18 Apr 2003 16:30:55 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: John Milan <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Sender: [email-masked]

On Fri, 18 Apr 2003, Mark Crispin wrote:

> On Fri, 18 Apr 2003 15:52:59 -0400 (EDT), Rob Siemborski wrote:
> > So, I think its pretty clear that this is a desired feature to have.
>
> I don't see how you came up with that conclusion, unless you mean "desire" in
> terms of "asking for the moon".

Sorry, perhaps I should have phrased it as "several people on the list
have expressed interest in following up on this", (both "WATCH=BIFF" and
"WATCH=STATUS") but certainly a WATCH extention of some type.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Fri, 18 Apr 2003 12:55:03 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP 
To: Rob Siemborski <[email-masked]>
cc: John Milan <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 15:52:59 -0400 (EDT), Rob Siemborski wrote:
> So, I think its pretty clear that this is a desired feature to have.

I don't see how you came up with that conclusion, unless you mean "desire" in
terms of "asking for the moon".

> Is there anything more formal written than what's been in this thread on
> the list?

There is working code.  It only needs final assembly.

Date: Fri, 18 Apr 2003 15:52:59 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: John Milan <[email-masked]>
cc: "'Mark Crispin'" <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Sender: [email-masked]

On Fri, 18 Apr 2003, John Milan wrote:

> Why put so much effort into half measures? If everyone acknowledges
> WATCH to be an issue worth solving why not *completely* solve it in the
> protocol and wait for implementations to catch up.

So, I think its pretty clear that this is a desired feature to have.

However, it is decidedly difficult to discuss a hypothetical protocol
extention.

Is there anything more formal written than what's been in this thread on
the list?

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Reply-To: <[email-masked]>
From: "John Milan" <[email-masked]>
To: "'Mark Crispin'" <[email-masked]>
Cc: "'Rob Siemborski'" <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 12:25:03 -0700
Organization: TeamDirection.com
Importance: Normal
Sender: [email-masked]

>> Why put so much effort into half measures? If everyone acknowledges 
>> WATCH to be an issue worth solving why not *completely* solve it in 
>> the protocol and wait for implementations to catch up.

> Sometimes, implementations never catch up.  FTP is an excellent
example.

I think this just means that some pieces are more important than others.
As a developer for commercial software I'm often surprised at how the
market responds to features regardless of my level of effort. It would
be hard to argue that FTP is somehow wrong simply because the
marketplace hasn't clamored for full implementation. In fact, the
ubiquity of FTP speaks precisely to the fact that the designers
thoroughly solved a problem, and that the world at large prioritized the
bare minimum needed.

> On the other hand, if you do a partial solution that solves the 98%
case and is available now, it will be good enough *and* > get useful
work accomplished.

Well, 98% is more of a nearly complete solution than a partial solution
my book :) If you think this is a nearly complete solution, then you are
doing the right thing by pushing it. 

However, I would like to see expressions be watchable as well. Maybe
that falls in the missing 2%, but other people are bringing up issues as
well, so there may be a few more percentage points missing.

My thesis is, regardless of whether it technically feasible or not, why
not create a richer implementation of WATCH that addresses as many
features everyone is proposing as possible. Yeah things get more
complicated, but take a look at the XML spec. Talk about complicated!
But if it solves a problem, then complication is a necessary tradeoff.
And as the adoption of XML shows, implementors can navigate the
complexity. 

I just think its very useful to address as many issues in a protocol as
possible, not because you want to make life hard, but because its an
excellent record of what people believe is an excellent and
well-thought-out solution to a difficult problem. Or, for the executive
summary, the less you have to assume about a protocol the better.

A bit of a soap box moment, but thems my thoughts.

JM

Date: Fri, 18 Apr 2003 15:19:51 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Cyrus Daboo <[email-masked]>
CC: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Cyrus Daboo wrote:
> 
> Hi Rob,
> 
> --On Friday, April 18, 2003 1:34 PM -0400 Rob Siemborski
> <[email-masked]> wrote:
> 
> | That said, I was imagining WATCH=STATUS to work something like:
> |
> | a WATCH public.folder (UNSEEN)
> | a OK Watching public.folder
> | b NOOP
> | b OK NOOP Done
> | c NOOP
> | * WATCH EXISTS "public.folder"
> | * WATCH EXISTS "public.folder"
> | * STATUS "public.folder" (UNSEEN 3)
> | c OK NOOP done
> 
> This shows up the problem that the client still has to poll via NOOP to get
> results. Obviously, if a mailbox was selected, the client could use IDLE
> and WATCH would come back during IDLE without the need to poll. But how
> would we deal with a connection that does not have a selected mailbox? The
> client could select a 'dummy' mailbox and then use idle, but wouldn't it be
> better to allow IDLE outside of selected state if WATCH is present?

IIRC, IDLE is already allowed in both the authenticated and selected
states.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Date: Fri, 18 Apr 2003 11:52:06 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP 
To: [email-masked]
cc: "'Rob Siemborski'" <[email-masked]>, "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 11:49:31 -0700, John Milan wrote:
> Why put so much effort into half measures? If everyone acknowledges
> WATCH to be an issue worth solving why not *completely* solve it in the
> protocol and wait for implementations to catch up.

Sometimes, implementations never catch up.  FTP is an excellent example.

> On the other hand, if you do a partial solution, then when it becomes
> technically feasible or economically viable, the protocol will not be
> correct and will splinter.

On the other hand, if you do a partial solution that solves the 98% case and
is available now, it will be good enough *and* get useful work accomplished.

We'd still be arguing IMP-IMP protocol if we refused to do anything before we
solved the last 2%.

Notifications are not a new idea.  The idea been bandied about for years.
Nothing has gotten done because people have refused to make the necessary
compromises necessary to accomplish results.

Reply-To: <[email-masked]>
From: "John Milan" <[email-masked]>
To: "'Mark Crispin'" <[email-masked]>, "'Rob Siemborski'" <[email-masked]>
Cc: "'Philip Guenther'" <[email-masked]>, "'IMAP Extensions WG'" <[email-masked]>
Subject: RE: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 11:49:31 -0700
Organization: TeamDirection.com
Importance: Normal
Sender: [email-masked]

> After many years, the Cyrus server still does not implement the RENAME
command correctly because it is technically infeasible > for it to do it
right.

>> Saying that a proposal can't be improved because it is already better

>> than what we currently have doesn't make a lot of sense.

>It makes perfect sense if the "improvement" precludes the entire thing
from being implemented.  The history of Internet > protocols is littered
with specifications that weren't implemented because someone would not
take "no" for an answer.

> Sometimes if you refuse an offer as inadequate, you end up with
nothing.

Why put so much effort into half measures? If everyone acknowledges
WATCH to be an issue worth solving why not *completely* solve it in the
protocol and wait for implementations to catch up. For instance, not
implementing the RENAME command correctly hasn't stopped the progress of
Cyrus' IMAP server. And if the RENAME command is correct, then when it
becomes technically feasible to do it right, the protocol will survive
because it was correct.

On the other hand, if you do a partial solution, then when it becomes
technically feasible or economically viable, the protocol will not be
correct and will splinter.

I see no reason why the IMAP protocol can't fully define a WATCH
extension and leave server vendors to figure out what pieces they can
and can't support today.

John Milan

Date: Fri, 18 Apr 2003 11:39:14 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: re: WATCH: changes to mailboxes being watched
To: Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 14:35:27 -0400, Cyrus Daboo wrote:
> How should WATCH deal with changes to mailboxes being watched?

The ultra-lightweight facility that I am offering has no provisions to deal
with this.  All it will do is deliver an event which matches.

Date: Fri, 18 Apr 2003 14:35:27 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: WATCH: changes to mailboxes being watched
Sender: [email-masked]

How should WATCH deal with changes to mailboxes being watched? e.g.: what 
happens if a watched mailbox is deleted by some other client? Should the 
watched connection send back a response to indicate that the watched 
mailbox is now gone and there will be no more watch responses for that 
mailbox? What about renaming a mailbox? Does the watch state 'follow' a 
mailbox as it is renamed? Does the watched connection send a response 
indicating a name change by some other client?

-- 
Cyrus Daboo

Date: Fri, 18 Apr 2003 14:30:40 -0400
From: Cyrus Daboo <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Pete,

--On Friday, April 18, 2003 1:06 PM -0500 Pete Resnick 
<[email-masked]> wrote:

|> This shows up the problem that the client still has to poll via NOOP
|> to get results.
|
| Why? Isn't the WATCH response going to come in as unilateral server data?

If WATCH is written to require idle-like behaviour ithout a command in 
progress that would be fine. But the implications of that for clients needs 
to be spelled out.

-- 
Cyrus Daboo

Date: Fri, 18 Apr 2003 10:51:54 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP 
To: Rob Siemborski <[email-masked]>
cc: Philip Guenther <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Fri, 18 Apr 2003 13:34:19 -0400 (EDT), Rob Siemborski wrote:
> Sure, but that just moves around who is looking at the cost, the fact is
> that server costs aren't the ONLY ones to consider.

Nevertheless, server costs can not be disregarded.

After many years, the Cyrus server still does not implement the RENAME command
correctly because it is technically infeasible for it to do it right.

Real implementation concerns can not be disregarded just because it may be
less of a concern in a particular implementation.  There has to be compromise,
and respect for the compromise.  That is what interoperability is all about.

> Saying that a proposal can't be improved because it is already better than
> what we currently have doesn't make a lot of sense.

It makes perfect sense if the "improvement" precludes the entire thing from
being implemented.  The history of Internet protocols is littered with
specifications that weren't implemented because someone would not take "no"
for an answer.

Sometimes if you refuse an offer as inadequate, you end up with nothing.

Date: Fri, 18 Apr 2003 13:06:24 -0500
To: Cyrus Daboo <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: WATCH extension for IMAP
Cc: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 4/18/03 at 1:49 PM -0400, Cyrus Daboo wrote:

>This shows up the problem that the client still has to poll via NOOP 
>to get results.

Why? Isn't the WATCH response going to come in as unilateral server data?

pr
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Fri, 18 Apr 2003 13:49:12 -0400
From: Cyrus Daboo <[email-masked]>
To: Rob Siemborski <[email-masked]>, Philip Guenther <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Sender: [email-masked]

Hi Rob,

--On Friday, April 18, 2003 1:34 PM -0400 Rob Siemborski 
<[email-masked]> wrote:

| That said, I was imagining WATCH=STATUS to work something like:
|
| a WATCH public.folder (UNSEEN)
| a OK Watching public.folder
| b NOOP
| b OK NOOP Done
| c NOOP
| * WATCH EXISTS "public.folder"
| * WATCH EXISTS "public.folder"
| * STATUS "public.folder" (UNSEEN 3)
| c OK NOOP done

This shows up the problem that the client still has to poll via NOOP to get 
results. Obviously, if a mailbox was selected, the client could use IDLE 
and WATCH would come back during IDLE without the need to poll. But how 
would we deal with a connection that does not have a selected mailbox? The 
client could select a 'dummy' mailbox and then use idle, but wouldn't it be 
better to allow IDLE outside of selected state if WATCH is present?

-- 
Cyrus Daboo

Date: Fri, 18 Apr 2003 13:34:19 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Philip Guenther <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Sender: [email-masked]

On Fri, 18 Apr 2003, Philip Guenther wrote:

> It's all a matter to explaining to the execs the costs of the tradeoffs
> that they're requesting.

Sure, but that just moves around who is looking at the cost, the fact is
that server costs aren't the ONLY ones to consider.

> Also, complaining about round-trips seems unfair, given that *right now*
> the client has to poll.  Unless your round-trip is on the same order as
> your poll interval, Mark's lightweight proposal has a lower latency than
> polling, period.  Client that want STATUS information can throw a normal
> STATUS command into their command pipe and still end up ahead of those
> that just poll.  Indeed, the client can sanely implement the batching
> you proposed: "after receiving a notification, wait until N seconds have
> passed *without* receiving a notification for that same mailbox, but no
> more than M seconds total, before sending a STATUS command for the
> mailbox".

Saying that a proposal can't be improved because it is already better than
what we currently have doesn't make a lot of sense.

That said, I was imagining WATCH=STATUS to work something like:

a WATCH public.folder (UNSEEN)
a OK Watching public.folder
b NOOP
b OK NOOP Done
c NOOP
* WATCH EXISTS "public.folder"
* WATCH EXISTS "public.folder"
* STATUS "public.folder" (UNSEEN 3)
c OK NOOP done

(you could, in this case, just drop the "WATCH" responses, I have them
there just to indicate more than one message arrived).
But the idea was that you wouldn't queue them up based on a time interval,
you'd send the status as soon as there was new mail, within the same
response that would cover just the first message.

Maybe the looser requirements of the POLL command are good enough, I'd
like to hear from a client author though.  Certainly, it is an
improvement of having the client POLL explicitly though.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

To: Rob Siemborski <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Date: Fri, 18 Apr 2003 01:22:45 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Rob Siemborski <[email-masked]> writes:
>On Thu, 17 Apr 2003, Mark Crispin wrote:
>>> Or how about this: Suppose any time
>>> your users had to wait for the server cost you $5.
>>
>> This is also nonsense.  Customers pay their service providers, not the
>> other way around.
>
>Sure, but ISPs aren't the only ones providing IMAP service.  IT
>departments in corporations likely can't easily say to their VP,
>Presidents, or CEOs "screw you, our server resources are worth more than
>your time".

Sure they can: "enabling that feature will result in increased server
costs on the order of N thousand dollars per-year.  In the best
judgement of our staff, the ROI of such an expenditure is worse than
that of the following other items.  If you tell us to, we can, of
course, move it up on our priority list, but please be aware that
something else will have to give."  When I was sysadmin at a college in
Minnesota, we pushed back on "orders from on high" all the time.
Sometimes our VP agreed with our judgement/reasoning and relaxed the
mandate, other times we were told that the benefits were more than we
thought (publicity, inter-department deals, prestige of the VP,
whatever) and so we were the ones to give.  It's all a matter to
explaining to the execs the costs of the tradeoffs that they're
requesting.

...
<disagreement about the usefulness of \Recent elided>
...

>All that said, there is NO REASON, that a WATCH extention can not be
>extensible enough serve both the "ultra-lightweight" people and the "I
>want to STATUS every folder on every message arrival" people, without
>requiring additional round trips.

Using the same name for different actions isn't a step forward.  You've
already suggested that a WATCH STATUS should batch responses, while Mark
has mentioned dropping notifications if the queue overflows.  Those are
significantly different behaviors and I would expect some servers to
implement them with completely different backend mechanisms with
completely different efficiencies and latencies.  Calling them the same
thing encourages clients to treat them the same, which skews the
tradeoffs.

Also, complaining about round-trips seems unfair, given that *right now*
the client has to poll.  Unless your round-trip is on the same order as
your poll interval, Mark's lightweight proposal has a lower latency than
polling, period.  Client that want STATUS information can throw a normal
STATUS command into their command pipe and still end up ahead of those
that just poll.  Indeed, the client can sanely implement the batching
you proposed: "after receiving a notification, wait until N seconds have
passed *without* receiving a notification for that same mailbox, but no
more than M seconds total, before sending a STATUS command for the
mailbox".

Is that a 90% solution?  If not and what client authors really want is a
batched STATUS update, then perhaps we should be discussing a 'POLL'
extension, say, something like:

POLL Command
   Arguments:	max-period
		STATUS
		command arguments

   Responses:	untagged STATUS

   Result:	OK - initial data returned, command will be performed as
				requested 
		NO - too many POLLs requested or max-period too small
		BAD - command unknown or arguments invalid

	The POLL command requests that the IMAP server immediately
	perform the specified STATUS command and return its response.
	After the tagged OK, the server will continue to monitor the
	mailbox and unilaterally send updated STATUS responses for that
	mailbox.  The server SHOULD NOT send an update unless the
	mailbox has actually been changed, though it MAY send an update
	for a mailbox that has change in ways not reflected in the
	STATUS response.  [Alternatively, we could simply require it to
	track what it last sent and never send the exact same response
	twice in a row, even when the value changed but returned to
	their previous value before it composed the updated response.]

	If an update is to be sent, the server MAY delay sending it
	(e.g., so that the server can consolidate multiple changes into
	a single STATUS response), but MUST send it no later than if the
	server had received the specified STATUS command directly from
	the client 'max-period' seconds after the actual update
	occured.  [Blech.  How to phrase it that the server can't have
	any greater latency than a client that polled every 'max-period'
	seconds?]

	The server may limit the number of POLL commands that it will
	support on a single IMAP connection and may require the
	max-period to be no smaller than a give value.  Those limits
	will be advertised in the capability string, blah blah blah.
	[What are the minimum requirements to comply?  20 concurrent
	POLLs, supporting a max-period of 20 seconds or more?]

	If a client issues a SELECT or EXAMINE command for a mailbox for
	which it previously issued a POLL command on the same
	connection, the server SHOULD suppress all STATUS updates for
	that mailbox for as long as the client has it selected.

   Example:	C: A042 POLL 30 STATUS "Lists/imapext" (UNSEEN)
		S: * STATUS "Lists/imapext" (UNSEEN 3)
		S: A042 OK POLL started
		...time passes; new mail arrives
		S: * STATUS "Lists/imapext" (UNSEEN 10)
		...

I tried to write the above to allow both implementations that would
simply poll at the requested rate and those that would be truly
notification triggered, while hopefully making the result compelling for
clients either way.  

Philip Guenther
<[email-masked]>

Date: Thu, 17 Apr 2003 17:56:58 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Lawrence Greenfield <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> > Or how about this: Suppose any time
> > your users had to wait for the server cost you $5.
>
> This is also nonsense.  Customers pay their service providers, not the
> other way around.

Sure, but ISPs aren't the only ones providing IMAP service.  IT
departments in corporations likely can't easily say to their VP,
Presidents, or CEOs "screw you, our server resources are worth more than
your time".

> Wait a second.  You are implying that it is "frequent" that people will be
> reading mail with more than one computer simultaneously.
>
> Once again, how frequent?

Well, for me (using pine), all the time, actually.  In fact, I'll often
find myself with several pine sessions on the same machine.

> How frequent is it that a user is distraught that one of his simultaneous
> running clients isn't showing a new mail messages that he has already read
> on his other simultaneously running client?

If I accidently leave pine connected to a folder at work (happens almost
every day, to various folders), then go home, pine at home won't notice
that new mail has arrived in that folder when I hit "tab" to scan through
my incoming folders, because the \Recent flags are being unset by the
session at work.  I haven't actually read (or even been notified of) the
new messages.

Sure "doctor, it hurts when I do this!", but it's an example of a very
real problem with \Recent semantics.

> The minimum definition of "interesting" in this context is defined in the
> specification.

RFC3501>>>
      \Marked
         The mailbox has been marked "interesting" by the server; the
         mailbox probably contains messages that have been added since
         the last time the mailbox was selected.
<<<

I don't see a minimum requirement.  I see, at best, a suggestion of what
"interesting" might mean.

> > With this sort of specification, it is not surprising that clients
> > choose not to use \Marked for anything.
>
> I claim that it's because of lazy programmers who only use fast
> connections .

Nevermind the fact that servers are allowed to not return \Marked or
\Unmarked at all, if they feel like it, making the feature basically
useless in the general case anyway.

All that said, there is NO REASON, that a WATCH extention can not be
extensible enough serve both the "ultra-lightweight" people and the "I
want to STATUS every folder on every message arrival" people, without
requiring additional round trips.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 14:33:54 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Lawrence Greenfield <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Lawrence Greenfield wrote:
>    Suppose your ISP had a charging system such that LIST was free, but each
>    STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
>    differently?
> Suppose your ISP had a charging system such that LIST responses were
> 0.05 cents and STATUS was free.

That's nonsense.  LIST can never be more expensive than STATUS.

I remind you; ISPs are disabling SEARCH in IMAP because they think it is
too expensive.

> Or how about this: Suppose any time
> your users had to wait for the server cost you $5.

This is also nonsense.  Customers pay their service providers, not the
other way around.

> And this is one of the reasons that we're making LISTEXT be able to
> return STATUS responses.

Which I feel to be a serious mistake.

Tell me, when is Cyrus going to implement RENAME according to the
specification and remove its bogus NO_ATOMIC_RENAME capability?

>    True, the multiple clients connected to the same mailbox case is a problem.
>    But is that the 99% case?
> 99% case? Of course not. A very frequent thing? Yes.

How much is "very frequent"?  1% of the time?  .001% of the time?

> Many people read
> e-mail with more than one computer. Will it become more frequent?  I
> hope so---and many many people have cell phones.

Wait a second.  You are implying that it is "frequent" that people will be
reading mail with more than one computer simultaneously.

Once again, how frequent?

How frequent is it that a user is distraught that one of his simultaneous
running clients isn't showing a new mail messages that he has already read
on his other simultaneously running client?

The day that email spam hits cell phones en masse will be the day that we
see an end to that particular silliness, particularly in the US with its
"cell phone user always pays" pricing model and carriers which can't even
provide reliable service in the middle of a city.

> Since LIST \Marked isn't required and isn't even well specified. (My
> server could consider any message containing the word "penguin"
> interesting---not very useful to anyone but me, but _I_ find those
> messages "interesting".)

The minimum definition of "interesting" in this context is defined in the
specification.

It is not implausible that there might be an IMAP extension to add
"containing the word penguin" to the definition of "interesting".

> With this sort of specification, it is not surprising that clients
> choose not to use \Marked for anything.

I claim that it's because of lazy programmers who only use fast
connections .

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 23:17:26 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:54, Mark Crispin wrote:
> On 17 Apr 2003 21:46:18 +0300, Timo Sirainen wrote:
> > I haven't actually thought of a single good usage of \Marked flag. It
> > pretty much means that there's probably \Recent messages in the mailbox.
> 
> Consider this thought exercise.
> 
> Suppose your ISP had a charging system such that LIST was free, but each
> STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
> differently?

Maybe. But since it currently doesn't cost more than server resources,
good user interface is more important to me as a user (and as admin for
server with only a couple of users).

> Wouldn't you do a LIST to determine whether or not you should do a STATUS?
> After all, you can do a wildcard with the LIST.
> 
> True, the multiple clients connected to the same mailbox case is a problem.
> But is that the 99% case?

I think that's one of the best features in IMAP. I use it, so checking
only \Marked mailboxes wouldn't be enough for me. If the choice is
between using more server resources and not necessarily seeing new
message for hours or even days after it's received, I choose the
resource consumption any time.

Of course, client could do LIST often and ask STATUS for \Marked folders
immediately, but still ask STATUS for non-marked folders at least once
every n minutes.

Date: Thu, 17 Apr 2003 15:58:04 -0400
From: Lawrence Greenfield <[email-masked]>
To: Cyrus Daboo <[email-masked]>, Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>, Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

   Date: Thu, 17 Apr 2003 11:54:55 -0700 (PDT)
   From: Mark Crispin <[email-masked]>
[...]
   Suppose your ISP had a charging system such that LIST was free, but each
   STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
   differently?

Suppose your ISP had a charging system such that LIST responses were
0.05 cents and STATUS was free. Or how about this: Suppose any time
your users had to wait for the server cost you $5. Wouldn't you
prefetch as much data as possible?

   Wouldn't you do a LIST to determine whether or not you should do a STATUS?
   After all, you can do a wildcard with the LIST.

And this is one of the reasons that we're making LISTEXT be able to
return STATUS responses.

   True, the multiple clients connected to the same mailbox case is a problem.
   But is that the 99% case?

99% case? Of course not. A very frequent thing? Yes. Many people read
e-mail with more than one computer. Will it become more frequent?  I
hope so---and many many people have cell phones.

   On Thu, 17 Apr 2003 21:01:25 +0200, Simon Josefsson wrote:
   > Some clients like to only display the mailbox that have unread
   > messages on the screen.  So to find out which of the 1800 mailboxes to
   > display, the client sends STATUS for all of them.

   Such a client should use the LIST \Marked flag and not STATUS.

Since LIST \Marked isn't required and isn't even well specified. (My
server could consider any message containing the word "penguin"
interesting---not very useful to anyone but me, but _I_ find those
messages "interesting".)

With this sort of specification, it is not surprising that clients
choose not to use \Marked for anything.

It is important that any notification mechanism ("light weight" or
not) has well defined semantics that actually match what clients are
interested in.

Larry

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 21:36:49 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> On Thu, 17 Apr 2003 21:01:25 +0200, Simon Josefsson wrote:
>> Some clients like to only display the mailbox that have unread
>> messages on the screen.  So to find out which of the 1800 mailboxes to
>> display, the client sends STATUS for all of them.
>
> Such a client should use the LIST \Marked flag and not STATUS.

No, LIST \Marked means the mailbox contain "interesting" messages.
"Interesting" (recent) and unread are not the same thing, and some
users prefer to display only mailboxes with unread messages, so the
client must send STATUS for all of them.

See RFC 3501:

      The LIST command SHOULD return its data quickly, without undue
      delay.  For example, it SHOULD NOT go to excess trouble to
      calculate the \Marked or \Unmarked status or perform other
      processing; if each name requires 1 second of processing, then a
      list of 1200 names would take 20 minutes!
...
      \Marked
         The mailbox has been marked "interesting" by the server; the
         mailbox probably contains messages that have been added since
         the last time the mailbox was selected.
...
      If it is not feasible for the server to determine whether or not
      the mailbox is "interesting", or if the name is a \Noselect name,
      the server SHOULD NOT send either \Marked or \Unmarked.

Date: Thu, 17 Apr 2003 11:54:55 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Cyrus Daboo <[email-masked]>
cc: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:44:04 -0400, Cyrus Daboo wrote:
> \Marked is useful for finding out which mailboxes have new mail at the
> start of a session (i.e. when LIST is done) but to keep that information up
> to date, the client would have to do some form of LIST polling, at which
> point you might just as well use status.

On 17 Apr 2003 21:46:18 +0300, Timo Sirainen wrote:
> I haven't actually thought of a single good usage of \Marked flag. It
> pretty much means that there's probably \Recent messages in the mailbox.

Consider this thought exercise.

Suppose your ISP had a charging system such that LIST was free, but each
STATUS cost $.05 (.05 Euros, 5 yen, take your pick...).  Would you do things
differently?

Wouldn't you do a LIST to determine whether or not you should do a STATUS?
After all, you can do a wildcard with the LIST.

True, the multiple clients connected to the same mailbox case is a problem.
But is that the 99% case?

I suspect that the 99.99% case of multiple clients connected to the same
mailbox is the *same* client with multiple sessions to the same mailbox.  In
that case, the client is perfectly capable of resolving the confusion itself.

On Thu, 17 Apr 2003 21:01:25 +0200, Simon Josefsson wrote:
> Some clients like to only display the mailbox that have unread
> messages on the screen.  So to find out which of the 1800 mailboxes to
> display, the client sends STATUS for all of them.

Such a client should use the LIST \Marked flag and not STATUS.

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 21:05:08 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> The question is:
>
> If there was an ultra-lightweight notification mechanism available very
> soon, would you use it?
> 	OR
> Do you say, "No, I can not make use of an ultra-lightweight notification
> facility.  I'd rather wait until LEMONADE finishes its work, and then wait
> for that work to be implemented.  Until then, I'd rather have nothing at
> all."

The charter for LEMONADE does not make it clear (to me, at least) that
the message notification protocol they will develop is an IMAP
extension.  A notification protocol is not mentioned in the milestones
of the group either.  And no drafts.

> I'm trying to ascertain if it's worth the effort.  Personally, I believe
> that while it isn't ideal, it's good enough.

Since it appears (at best) unclear (to me) that LEMONADE will ever
specify a IMAP notification facility with the bells and whistles I'd
want, I don't think an effort in this direction would hurt.  If if you
changed your proposal into also providing (at least) the uidvalidity
of the mailbox and uid of the new message, I'm positively for it.  It
doesn't appear like it would cost much to implement this change.

If LEMONADE is on track towards creating a feature rich notification
IMAP extension, I think it is a mistake to push your proposal now as
it would result in two only slightly different solutions to the same
problem.

Just my .2 SEK.

To: Mark Crispin <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, Timo Sirainen <[email-masked]>, Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 21:01:25 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> On Thu, 17 Apr 2003 14:15:10 -0400, Cyrus Daboo wrote:
>> STATUS is often used to show a count of the total number of messages and
>> number of unseen messages in a 'closed' mailbox.
>
> I understand why people want/like that.  But why do STATUS of 1800 mailboxes
> when only 20 are currently visible on the screen?

Some clients like to only display the mailbox that have unread
messages on the screen.  So to find out which of the 1800 mailboxes to
display, the client sends STATUS for all of them.

The client could also be a batch client, that slurps in all
information it could possibly need, in order to be fast once it gotten
past the slow startup, instead of being slightly slow (depending on
the server) all the time.

Date: Thu, 17 Apr 2003 11:42:41 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Cyrus Daboo <[email-masked]>
cc: Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:40:21 -0400, Cyrus Daboo wrote:
> You need to think about GUI clients that may have multiple views on screen
> at the same time, all being updated and maintained as server state changes.

I mentioned that possibility.  On the other hand, there's only so much screen
real estate.  Most users do not have a screen large enough to display the
message counts for 1800 mailboxes, and display an index of messages, a message
being read, and a message being composed.

Unfortunately, I see clients behave as if this is the user's display
environment, the client has unlimited network bandwidth to the server, and the
server has unlimited resources to respond to client demands.

I see clients behave this way when the user is on a dialup IP connection from
a laptop with a 14" screen.  Much of the "IMAP is chatty" and "IMAP is slow"
and "IMAP is resource intensive" folklore comes out of such experience.

I'm not saying that Mulberry does this.  But there are clients which do so.

One of my ways of judging a "quality IMAP client" is the user experience and
how much it costs to read your mail over roaming CDPD (at $.05/kilobyte!).

LEMONADE should be an eye-opener.  A lot about LEMONADE frightens me, but
overall I think that it is a good thing that LEMONADE has happened.

Date: Thu, 17 Apr 2003 14:53:07 -0400
From: Ken Murchison <[email-masked]>
Organization: Oceana Matrix Ltd.
To: Mark Crispin <[email-masked]>
CC: Rob Siemborski <[email-masked]>, Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Mark Crispin wrote:
> 
> On Thu, 17 Apr 2003 14:01:32 -0400 (EDT), Rob Siemborski wrote:
> > Of course, if the user has recieved new mail in 1800 separate mailboxes,
> > in a notification period, I'd say the server is under a pretty high load
> > already.
> 
> I wasn't talking about.  I was talking about client startup.
> 
> Does the client do 1800 STATUS commands when the user first goes into the list
> of mailboxes?  If so, I think that client has a problem.
> 
> > I'm not sure if its better for the client to say on per session basis
> > "tell me about mailboxes x, y, z, a, b, c" or just let the client say
> > "tell me about my subscribed mailboxes".  I think I like the former,
> > actually.
> 
> So do we!
> 
> Originally, that was what subscribed mailboxes were supposed to do, but the
> abuse of the subscription feature by certain other clients put an end to that
> idea.
> 
> In effect, it is a type of subscription; it's just kept in client
> configuration instead of by the server.  That keeps it safe from the
> aforementioned certain other clients, but on the other hand other clients
> won't inherit that information either.

This might be a good mailbox annotation, eg:
/mailbox/{<mailbox-name>}/watch  (or whatever the syntax ends up being)

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 21:50:32 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:36, Mark Crispin wrote:
> What I see is neither of the above.
> 
> I see clients doing LIST "" * followed by STATUS of each returned name, at
> startup.  I see this repeated at periodic (obviously polling) intervals.
> 
> The resulting experience is not good for the server or the user.

Yeah, most IMAP clients suck.

BTW. I wrote this some weeks ago:
http://dovecot.procontrol.fi/imap-client-coding-howto.html

From: "John Milan" <[email-masked]>
To: "Mark Crispin" <[email-masked]>
Cc: "IMAP Extensions WG" <[email-masked]>
Subject: RE: WATCH extension for IMAP
Date: Thu, 17 Apr 2003 11:57:37 -0700
Importance: Normal
Sender: [email-masked]

> That is, in your "WATCH new mail where from = [email-masked]" example,
you
> would get all new mail events, and locally filter out the ones that are
not
> from "[email-masked]".

Hmmm... But I thought one of the features of the IMAP protocol was 'smart'
remote access. For instance, if I'm on a slow connection, I don't
necessarily have to get all my mail messages. It seems to me that if I am in
a connection speed challeged environment, I'd rather have the server filter
out the messages I'm watching for than have to download all new messages and
filter locally. Especially if I have a hotmail account with lots of spam.

Such an environment might have a powerful server, but many remote access
users, perhaps each using a dialup connection-- like the travelling salesman
in his hotel room. They might be very interested in what their boss has to
say, but not the funny movies their friends sent to them.

I suppose just filtering off the message headers takes care of the above
example, but then you can never really do content evaluation-- to costly to
send everything across the wire. This just seems like building a problem
into the protocol, especially when mail clients 'expect' this behavior in
the future.

John Milan

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Cyrus Daboo <[email-masked]>, Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 21:46:18 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:26, Mark Crispin wrote:
> > Just indicating which have
> > new mail is not enough - after all that is what \Marked does in LIST and
> > very few implementations have bothered to use it.
> 
> I never understood why this is the case.

I haven't actually thought of a single good usage of \Marked flag. It
pretty much means that there's probably \Recent messages in the mailbox.
Those are another thing which I haven't found very useful. In single
client case I guess it works well enough, but as soon as you have
multiple clients connected to same mailbox, the behaviour gets pretty
much random. Just because the client at work office noticed the mail
before the client at home, doesn't mean that I wouldn't want to get
notification of it when I'm at home.

\Marked flag tells only that there's a folder where you very likely find
new mail, but a client can't make any assumption if non-marked folders
have new mail. So it ends up sending STATUS for all of them anyway.

Date: Thu, 17 Apr 2003 14:44:04 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Thursday, April 17, 2003 11:26 AM -0700 Mark Crispin 
<[email-masked]> wrote:

|> Just indicating which have
|> new mail is not enough - after all that is what \Marked does in LIST and
|> very few implementations have bothered to use it.
|
| I never understood why this is the case.

\Marked is useful for finding out which mailboxes have new mail at the 
start of a session (i.e. when LIST is done) but to keep that information up 
to date, the client would have to do some form of LIST polling, at which 
point you might just as well use status.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 11:36:58 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Rob Siemborski <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:31:10 -0400 (EDT), Rob Siemborski wrote:
> I suppose that depends on your goal.  If you assume STATUS is going to
> take a long time, then from a user experience perspective it makes sense
> to do them when the network connection is otherwise idle.
>
> If, on the other hand, you are solely concerned with overall server
> performance, then you probably want this to be as lazy as possible.

What I see is neither of the above.

I see clients doing LIST "" * followed by STATUS of each returned name, at
startup.  I see this repeated at periodic (obviously polling) intervals.

The resulting experience is not good for the server or the user.

Date: Thu, 17 Apr 2003 14:40:21 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Rob Siemborski <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Thursday, April 17, 2003 11:03 AM -0700 Mark Crispin 
<[email-masked]> wrote:

|> Of course, if the user has recieved new mail in 1800 separate mailboxes,
|> in a notification period, I'd say the server is under a pretty high load
|> already.
|
| I wasn't talking about.  I was talking about client startup.
|
| Does the client do 1800 STATUS commands when the user first goes into the
| list of mailboxes?  If so, I think that client has a problem.

That depends on how the client is implemented. We do all STATUS work via a 
background thread so it won't interfere with the user. And yes, the user 
could easily set things up so that they do 1800 STATUS commands. We have 
many users that like to have message counts on their entire personal 
hierarchy appear on startup - so those can all get checked once, and then a 
different (smaller) set checked at varying intervals after that.

|> I'm not sure if its better for the client to say on per session basis
|> "tell me about mailboxes x, y, z, a, b, c" or just let the client say
|> "tell me about my subscribed mailboxes".  I think I like the former,
|> actually.
|
| So do we!
|
| Originally, that was what subscribed mailboxes were supposed to do, but
| the abuse of the subscription feature by certain other clients put an end
| to that idea.
|
| In effect, it is a type of subscription; it's just kept in client
| configuration instead of by the server.  That keeps it safe from the
| aforementioned certain other clients, but on the other hand other clients
| won't inherit that information either.

Yes - subscribe has been abused :-( Note that ANNOTATEMORE does have an 
annotation that's meant to be used to indicate which mailboxes should be 
checked for new mail, so hopefully co-operating clients can use that once 
its implemented.

|> But if the client author knows they are going to do a status for every
|> notification anyway (or every set of notifications on a mailbox), then
|> why cost them the additional round trip?
|
| Perhaps to discourage clients from doing that?  Perhaps to encourage
| innovative thinking that would avoid unnecessary STATUS commands?
|
| One very obvious thing would be to defer doing a STATUS command until the
| user is in the list of mailboxes screen.  It's rather silly to do 50
| STATUS commands on a mailbox while the user's screen is busy in composing
| a message.
|
| This wouldn't hold for a client that always shows a list of mailboxes,
| but the general idea still applies.  If the data isn't visible on the
| user's screen (the client is in a different mode, or that particular data
| is scrolled out of the visible view, or even if another window overlaps),
| then there's no need to update it until it becomes visible.
|

You need to think about GUI clients that may have multiple views on screen 
at the same time, all being updated and maintained as server state changes. 
All that can be done in the background without interfering with the user's 
current focus. Plus, whether the mailbox is on screen or not is not 
relevant as the new mail check might bring it on screen if it has new mail 
or require some other form of notification that does not depend on the 
visibility of the mailbox itself (e.g. an audible text-to-speech alert that 
reads the name of the mailbox and the number of new messages).

All of this is very much a client implementation issue. Providing a way to 
get status automatically will benefit a number of clients that are already 
doing expensive (polled) status calls. For low volume status changes, 
WATCH=STATUS will be a big win. For the high volume case, we need to have 
recommendations or limits on what should be done.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 14:36:22 -0400
From: Cyrus Daboo <[email-masked]>
To: Rob Siemborski <[email-masked]>, John Milan <[email-masked]>
cc: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: RE: WATCH extension for IMAP
Sender: [email-masked]

Hi Rob,

--On Thursday, April 17, 2003 2:22 PM -0400 Rob Siemborski 
<[email-masked]> wrote:

|> I'm thinking of a WATCH much like mail filtering.
|
| I suspect it would be possible to use the sieve notify extention to feed
| this in some way.

Indeed we could define WATCH=SIEVENOTIFY and have * WATCH responses that 
return the sieve data. However, this would require a WATCH command without 
a mailbox argument since sieve notifications are not tied to mailboxes. 
Using the empty string as an argument to WATCH in that case would make 
sense (and would mirror the syntax of server vs mailbox annotations in the 
next ANNOTATEMORE draft).

| I think I'd rather see the IMAP side of this stay simple (but
| extensible, if more features are desired later).

Agreed - WATCH=XXX allows us to do that.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 11:26:02 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Cyrus Daboo <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:15:10 -0400, Cyrus Daboo wrote:
> STATUS is often used to show a count of the total number of messages and
> number of unseen messages in a 'closed' mailbox.

I understand why people want/like that.  But why do STATUS of 1800 mailboxes
when only 20 are currently visible on the screen?

> Just indicating which have
> new mail is not enough - after all that is what \Marked does in LIST and
> very few implementations have bothered to use it.

I never understood why this is the case.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 21:35:27 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 21:03, Mark Crispin wrote:
> On Thu, 17 Apr 2003 14:01:32 -0400 (EDT), Rob Siemborski wrote:
> > Of course, if the user has recieved new mail in 1800 separate mailboxes,
> > in a notification period, I'd say the server is under a pretty high load
> > already.
> 
> I wasn't talking about.  I was talking about client startup.
> 
> Does the client do 1800 STATUS commands when the user first goes into the list
> of mailboxes?  If so, I think that client has a problem.

Optimally it would send STATUS only for those mailboxes that are visible
in screen. Good enough solution would be to check only the mailboxes
that can become visible by scrolling (similiar to LIST "" % usage).

Currently Evolution seems to do it for all subscribed folders if "check
mail in all folders" flag is set. I'm not sure if it does it for all
folders if I uncheck "show only subscribed folders" flag.

Date: Thu, 17 Apr 2003 14:31:10 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> One very obvious thing would be to defer doing a STATUS command until
> the user is in the list of mailboxes screen.  It's rather silly to do 50
> STATUS commands on a mailbox while the user's screen is busy in
> composing a message.

I suppose that depends on your goal.  If you assume STATUS is going to
take a long time, then from a user experience perspective it makes sense
to do them when the network connection is otherwise idle.

If, on the other hand, you are solely concerned with overall server
performance, then you probably want this to be as lazy as possible.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 14:22:22 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: John Milan <[email-masked]>
cc: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: RE: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, John Milan wrote:

> Taking advantage of at least the RFC headers as watchable data. That way, no
> matter how many folders I might have, I'll get that important message from
> the boss. More aggressive implementations could probably extend the
> expression matching to the message body.
>
> I'm thinking of a WATCH much like mail filtering.

I suspect it would be possible to use the sieve notify extention to feed
this in some way.

I think I'd rather see the IMAP side of this stay simple (but
extensible, if more features are desired later).

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 14:15:10 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Timo Sirainen <[email-masked]>
cc: Rob Siemborski <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Thursday, April 17, 2003 9:47 AM -0700 Mark Crispin 
<[email-masked]> wrote:

| After all, what's the point of STATUS, other than to see if there is new
| messages in the mailbox?  In this case, we *know* that that are new
| messages!

STATUS is often used to show a count of the total number of messages and 
number of unseen messages in a 'closed' mailbox. Just indicating which have 
new mail is not enough - after all that is what \Marked does in LIST and 
very few implementations have bothered to use it.

-- 
Cyrus Daboo

Date: Thu, 17 Apr 2003 11:03:29 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Rob Siemborski <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Thu, 17 Apr 2003 14:01:32 -0400 (EDT), Rob Siemborski wrote:
> Of course, if the user has recieved new mail in 1800 separate mailboxes,
> in a notification period, I'd say the server is under a pretty high load
> already.

I wasn't talking about.  I was talking about client startup.

Does the client do 1800 STATUS commands when the user first goes into the list
of mailboxes?  If so, I think that client has a problem.

> I'm not sure if its better for the client to say on per session basis
> "tell me about mailboxes x, y, z, a, b, c" or just let the client say
> "tell me about my subscribed mailboxes".  I think I like the former,
> actually.

So do we!

Originally, that was what subscribed mailboxes were supposed to do, but the
abuse of the subscription feature by certain other clients put an end to that
idea.

In effect, it is a type of subscription; it's just kept in client
configuration instead of by the server.  That keeps it safe from the
aforementioned certain other clients, but on the other hand other clients
won't inherit that information either.

> But if the client author knows they are going to do a status for every
> notification anyway (or every set of notifications on a mailbox), then why
> cost them the additional round trip?

Perhaps to discourage clients from doing that?  Perhaps to encourage
innovative thinking that would avoid unnecessary STATUS commands?

One very obvious thing would be to defer doing a STATUS command until the user
is in the list of mailboxes screen.  It's rather silly to do 50 STATUS
commands on a mailbox while the user's screen is busy in composing a message.

This wouldn't hold for a client that always shows a list of mailboxes, but the
general idea still applies.  If the data isn't visible on the user's screen
(the client is in a different mode, or that particular data is scrolled out of
the visible view, or even if another window overlaps), then there's no need to
update it until it becomes visible.

> (though I
> could see WATCH=BIFF as just being WATCH=STATUS with no status fields
> being returned).

I agree with this observation.

Date: Thu, 17 Apr 2003 10:52:57 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: RE: WATCH extension for IMAP
To: John Milan <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

The underlying protocol has the potential for such a capability.  It deals in
terms of "object identifiers", so anything can be an object identifier.

However, in the ultra-lightweight version, there's only exact match of object
identifiers.  So, in the case of a new mail event, you would have to receive
events for everything that matches the object, and use data in the event
payload to determine whether to do anything with the event or discard it.

That is, in your "WATCH new mail where from = [email-masked]" example, you
would get all new mail events, and locally filter out the ones that are not
from "[email-masked]".

The ultra-lightweight version also has a further restriction; what the payload
can carry is limited.  Only the SMTP return-path (which may not necessarily be
the from) is available from mail delivery, and mail delivered via IMAP COPY or
APPEND wouldn't even have that.

It is possible to envision future versions in which the event generator (mail
delivery or IMAP server) would parse the RFC 2822 headers and deliver such
information as From, Subject, etc.  In fact, that is absolutely part of the
design; we're going to need that!  But it isn't going to be in version 1.

-- Mark --

Date: Thu, 17 Apr 2003 14:01:32 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Timo Sirainen <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> On 17 Apr 2003 20:05:17 +0300, Timo Sirainen wrote:
> > Well, my client has a list of around 20 mailboxes which show the number
> > of unread messages after the mailbox name. I like this behaviour, I can
> > leave some messages unread if they require some action that I didn't
> > want to do immediately.
>
> I understand why you like this behavior.
>
> On the other hand, suppose the user has 1800 mailboxes.  Do you do the same
> thing?  What if STATUS takes a second or so of real time on that server?  It's
> going to take a while to do 1800 STATUS commands even if the mail store has
> fast STATUS metadata.

Of course, if the user has recieved new mail in 1800 separate mailboxes,
in a notification period, I'd say the server is under a pretty high load
already.

> Do you require that the user identify 20 mailboxes of particular interest, and
> only do it for those 20 mailboxes?

I'm not sure if its better for the client to say on per session basis
"tell me about mailboxes x, y, z, a, b, c" or just let the client say
"tell me about my subscribed mailboxes".  I think I like the former,
actually.

> > I don't think it would be considered very good behaviour to select the
> > mailbox immediately and fetch the new messages. I don't necessarily want
> > to read them, and I may delete them using another client.
>
> I agree with that.  I personally would like the marker, then a command that I
> could give to get the current count.  That way, a STATUS isn't done unless I
> do it.

But if the client author knows they are going to do a status for every
notification anyway (or every set of notifications on a mailbox), then why
cost them the additional round trip?

I do like Cyrus's idea of WATCH=BIFF and WATCH=STATUS, etc (though I
could see WATCH=BIFF as just being WATCH=STATUS with no status fields
being returned).

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 10:25:44 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP
To: Timo Sirainen <[email-masked]>
cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On 17 Apr 2003 20:05:17 +0300, Timo Sirainen wrote:
> Well, my client has a list of around 20 mailboxes which show the number
> of unread messages after the mailbox name. I like this behaviour, I can
> leave some messages unread if they require some action that I didn't
> want to do immediately.

I understand why you like this behavior.

On the other hand, suppose the user has 1800 mailboxes.  Do you do the same
thing?  What if STATUS takes a second or so of real time on that server?  It's
going to take a while to do 1800 STATUS commands even if the mail store has
fast STATUS metadata.

Do you require that the user identify 20 mailboxes of particular interest, and
only do it for those 20 mailboxes?

In my lightweight notification system, the only cost of watching 1800
mailboxes instead of 20 is that the notification server will have to keep some
number of bytes per watch request.

> Maybe it would be enough for some people to just see some kind of marker
> that there's new mail in the mailbox.

I hope that this is enough for many people.

> I don't think it would be considered very good behaviour to select the
> mailbox immediately and fetch the new messages. I don't necessarily want
> to read them, and I may delete them using another client.

I agree with that.  I personally would like the marker, then a command that I
could give to get the current count.  That way, a STATUS isn't done unless I
do it.

I use CDPD a lot.  I don't like it when my client decides to do something
under the covers automatically.  I recognize that sometimes it has no choice.

The behavior of "just spawn another connection" makes things worse on CDPD.
When your CDPD modem gets into channel scanning, the more that such a client
does that, the longer the recovery time.

From: "John Milan" <[email-masked]>
To: "Mark Crispin" <[email-masked]>, "IMAP Extensions WG" <[email-masked]>
Subject: RE: WATCH extension for IMAP
Date: Thu, 17 Apr 2003 10:49:56 -0700
Importance: Normal
Sender: [email-masked]

Are the event types predefined? Such as you might have the following event
types:

new mail
deleted mail
flag changes

Or can you WATCH an expression?

For instance, could say (pardon the SQLness)

WATCH new mail where from = [email-masked]

Taking advantage of at least the RFC headers as watchable data. That way, no
matter how many folders I might have, I'll get that important message from
the boss. More aggressive implementations could probably extend the
expression matching to the message body.

I'm thinking of a WATCH much like mail filtering.

John Milan

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Mark Crispin
Sent: Wednesday, April 16, 2003 6:37 PM
To: IMAP Extensions WG
Subject: WATCH extension for IMAP

I've been thinking about defining a WATCH extension to IMAP which would
allow an IMAP client to watch for certain events happening to a
non-selected mailbox.  Once a WATCH is set, it remains in effect until
that IMAP session is logged out, and untagged WATCH events for the watched
mailbox(es) can be returned at any time.

The command will be something like:
	tag WATCH mailbox event-list
and the responses will be something like
	* WATCH mailbox event-pair-list
where an event-pair contains an event type and event data.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 20:05:17 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 19:47, Mark Crispin wrote:
> On Thu, 17 Apr 2003, Timo Sirainen wrote:
> > I think it's more likely that clients will be abusive with "new mail in
> > mailbox xyz" notification. They'd probably just request STATUS every
> > time they receive that.
> 
> A client that does that will quickly have its notification queue filled
> and notifications will get dropped.

Wouldn't the same thing happen if server did it automatically?

> I question why a client would want to issue a STATUS if notified about new
> mail in such-and-such mailbox, as opposed to noting that that mailbox has
> new mail and/or opening it.
> 
> After all, what's the point of STATUS, other than to see if there is new
> messages in the mailbox?  In this case, we *know* that that are new
> messages!

Well, my client has a list of around 20 mailboxes which show the number
of unread messages after the mailbox name. I like this behaviour, I can
leave some messages unread if they require some action that I didn't
want to do immediately.

So, once in a while I look at the list and see if there's anything
interesting in there. Not just new mail, but also those mails that I
left unread. I may have marked some of them as read elsewhere, that
pretty much requires STATUS (UNSEEN).

Maybe it would be enough for some people to just see some kind of marker
that there's new mail in the mailbox.

I don't think it would be considered very good behaviour to select the
mailbox immediately and fetch the new messages. I don't necessarily want
to read them, and I may delete them using another client.

Date: Thu, 17 Apr 2003 09:47:47 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Timo Sirainen wrote:
> I think it's more likely that clients will be abusive with "new mail in
> mailbox xyz" notification. They'd probably just request STATUS every
> time they receive that.

A client that does that will quickly have its notification queue filled
and notifications will get dropped.

I question why a client would want to issue a STATUS if notified about new
mail in such-and-such mailbox, as opposed to noting that that mailbox has
new mail and/or opening it.

After all, what's the point of STATUS, other than to see if there is new
messages in the mailbox?  In this case, we *know* that that are new
messages!

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: Rob Siemborski <[email-masked]>, Cyrus Daboo <[email-masked]>, IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 19:43:33 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 19:20, Mark Crispin wrote:
> It's technically easy (trivial even) to do an automatic STATUS, and
> perhaps a subscription type mechanism will suffice to avoid the problem.
> However, the architecture has to be carefully designed to minimize the
> likelihood of abusive client implementations.

I think it's more likely that clients will be abusive with "new mail in
mailbox xyz" notification. They'd probably just request STATUS every
time they receive that. If server sends the STATUS replies directly, it
can at least force some "max. n seconds between STATUS notifications"
limit.

Date: Thu, 17 Apr 2003 09:39:53 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Simon Josefsson <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Simon Josefsson wrote:
> If the extension isn't IMAP aware, my initial reaction is that it
> should be a vendor specific capability.  I.e., your server can
> announce a WATCH capability that enables the command you propose.

Vendor-specific is not useful.  To be useful, it has to be something that
is generally adopted.

This does NOT preclude a separate, more heavyweight notification facility.
It certainly does not compete with a heavyweight facility.

The question is:

If there was an ultra-lightweight notification mechanism available very
soon, would you use it?
	OR
Do you say, "No, I can not make use of an ultra-lightweight notification
facility.  I'd rather wait until LEMONADE finishes its work, and then wait
for that work to be implemented.  Until then, I'd rather have nothing at
all."

LEMONADE is working on the specification for a heavyweight notification
facility.  I am not competing with that effort.  The LEMONADE charter says
that they will have that effort done by summer.  You can decide for
yourself whether you believe that will happen.

I know that everybody would like to have a heavyweight notification
facility, with lots of bells and whistles.  This is not such a facility.
I have a prototype ultra-lightweight notification system which I wrote and
demonstrated a year ago.  All that needs to be done is to stitch it in.
It is not likely to provoke server managers into turning it off either.

I'm trying to ascertain if it's worth the effort.  Personally, I believe
that while it isn't ideal, it's good enough.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Thu, 17 Apr 2003 12:08:42 -0400 (EDT)
From: Rob Siemborski <[email-masked]>
To: Mark Crispin <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, Timo Sirainen <[email-masked]>, IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

On Thu, 17 Apr 2003, Mark Crispin wrote:

> What if there is a mail bomb, and multiple messages/second are arriving?
> STATUS is not necessarily an inexpensive operation (and the specification
> specifically says as much).

True, but perhaps multiple notifications for a folder could be aggregated
into a single STATUS reply as well?

> Nor is it certain that every client would want to do a STATUS.  Perhaps it
> just wants to note that the mailbox has new messages, and let the user
> decide to SELECT it.

This is why I'd want to see clients able to subscribe to the particular
information that comes back with the notifications.

-Rob

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Rob Siemborski | Andrew Systems Group * Research Systems Programmer
PGP:0x5CE32FCC | Cyert Hall 207 * [email-masked] * 412.268.7456
-----BEGIN GEEK CODE BLOCK----
Version: 3.12
GCS/IT/CM/PA d- s+: a-- C++++$ ULS++++$ P+++$ L+++(++++) E W+ N o? K-
w O- M-- V-- PS+ PE++ Y+ PGP+ t+@ 5+++ R@ tv-@ b+ DI+++ G e h r- y?
------END GEEK CODE BLOCK-----

Date: Thu, 17 Apr 2003 09:08:43 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Thu, 17 Apr 2003, Timo Sirainen wrote:
> How are you going to send the notification to IMAP server? Some signal?
> Socket?

Currently, it would be a TCP connection from the IMAP server to the
notification system.  There's a good effort being made to avoid dropping
notifications (of course, if a drop has to happen, it will happen).  The
notification system itself does not require that a client receiving
notifications be on the same system as the notification system.

Generating notifications is currently also done by TCP, but it is likely
to move to a datagram type mechanism.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Thu, 17 Apr 2003 10:33:21 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Sender: [email-masked]

Hi Mark,

--On Wednesday, April 16, 2003 7:43 PM -0700 Mark Crispin 
<[email-masked]> wrote:

|> I think most of the clients would be just interested in getting
|> automatic updates for "STATUS (MESSAGES UNSEEN)", maybe UIDVALIDITY and
|> UIDNEXT too. For those it'd be easier if untagged STATUS replies were
|> received directly. Maybe "WATCH mailbox (STATUS (<status attributes>))"
|
| I agree with your observation; this would be nice.  Unfortunately, this
| functionality is outside of the scope of ultra-lightweight notification.
| The notification protocol can support this; however, the current set of
| event generators (the mail delivery agents) can not.
|
| The mail delivery agents merely append to the mailbox file; they have no
| concept of "number of messages", "number of unseen messages",
| "UIDVALIDITY", "UIDNEXT", or any other STATUS-type metadata.  The only
| thing they know is "appended a string purported to be a new mail message
| to such-and-such mailbox name, with return-path of such-and such".

Well, the server will know which mailbox is being watched and when the 
event for that is seen, why can't the server do an implicit STATUS (with 
client provided parameters) and return that? i.e. the server does what the 
client would otherwise do - issue a status for that mailbox.

I think for this to be useful within IMAP it has to return STATUS 
information - that's what clients will want - but I don't see that as being 
a problem or as interfering with the way the outside notification works.

I think it does also have to handle COPY/APPEND changes to the mailbox as 
well.

-- 
Cyrus Daboo

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 16:50:58 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 05:43, Mark Crispin wrote:
> I agree with your observation; this would be nice.  Unfortunately, this
> functionality is outside of the scope of ultra-lightweight notification.
> The notification protocol can support this; however, the current set of
> event generators (the mail delivery agents) can not.

How are you going to send the notification to IMAP server? Some signal?
Socket?

I've been thinking about using Linux's dnotify and BSD's kqueue() for
this. They should be quite lightweight as well, and they notice all
changes in mailbox file/directory.

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
From: Simon Josefsson <[email-masked]>
Date: Thu, 17 Apr 2003 12:28:10 +0200
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:

> The single most important event type will be new mail, with the event data
> being the SMTP return-path of the message.  So, in effect, this will be
> "biff for IMAP".

Yay!  This would be very useful, at least for my client.

I concur with Timo's observation, it is much more useful if the WATCH
extension responded with requested information (i.e., one of the
STATUS parameters, or possibly even a FETCH parameter -- imagine
receiving BODY and immediately being able to display information about
the new message) instead of just a maybe-there-is-new-mail-over-here
type of indication.  The latter kind of notification is sufficient in
a non-IMAP notification protocol, like zephyr (which the client can
support already), but if the feature is built into the IMAP protocol I
think it should be IMAP-aware instead of simply a external
notification protocol tunneled through IMAP.

If the extension isn't IMAP aware, my initial reaction is that it
should be a vendor specific capability.  I.e., your server can
announce a WATCH capability that enables the command you propose.

A real IMAP-aware NOTIFICATION extension would be of separate utility,
and could be defined in parallel with the WATCH capability.  Then a
server supporting WATCH can decide if it wants to do a STATUS
internally in order to be able to support NOTIFICATION too, and
clients can decide which kind of notification they are most helped by
and decide accordingly.

Thoughts?

Date: Wed, 16 Apr 2003 21:23:59 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: Re: WATCH extension for IMAP 
To: Philip Guenther <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Sender: [email-masked]

On Wed, 16 Apr 2003 21:23:04 -0700, Philip Guenther wrote:
> Does the IMAP server generate "new mail" notifications when it performs
> APPEND and COPY operations?

This could be done within the "ultra-lightweight" framework, although perhaps
it should be a different type of event than new mail.  Is this something that
people would like to have/find useful?

In the case of multiple-message COPY and multiple APPEND, it would most likely
generate a single event and not an event per-message.

To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP 
Date: Wed, 16 Apr 2003 21:23:04 -0700
From: Philip Guenther <[email-masked]>
Sender: [email-masked]

Mark Crispin <[email-masked]> writes:
...
>I am interested in feedback as to whether this proposed extension is
>interesting or not.  Also, would it be better to build an extension which
>has extensibility but with restraints ("ultra-lightweight") on how it can
>be extended, or if it should be strictly a new mail watching facility and
>leave the possibility of other types of notifications to a different
>facilty (such as what is being discussed in LEMONADE).

Does the IMAP server generate "new mail" notifications when it performs
APPEND and COPY operations?

Philip Guenther
<[email-masked]>

Date: Wed, 16 Apr 2003 19:43:03 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Timo Sirainen <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>
Subject: Re: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Wed, 17 Apr 2003, Timo Sirainen wrote:
> I think most of the clients would be just interested in getting
> automatic updates for "STATUS (MESSAGES UNSEEN)", maybe UIDVALIDITY and
> UIDNEXT too. For those it'd be easier if untagged STATUS replies were
> received directly. Maybe "WATCH mailbox (STATUS (<status attributes>))"

I agree with your observation; this would be nice.  Unfortunately, this
functionality is outside of the scope of ultra-lightweight notification.
The notification protocol can support this; however, the current set of
event generators (the mail delivery agents) can not.

The mail delivery agents merely append to the mailbox file; they have no
concept of "number of messages", "number of unseen messages",
"UIDVALIDITY", "UIDNEXT", or any other STATUS-type metadata.  The only
thing they know is "appended a string purported to be a new mail message
to such-and-such mailbox name, with return-path of such-and such".

So, to rephase my original question.  Would it be useful to receive, from
the IMAP server, a notification that a mail delivery agent appended what
purports to be new mail to such-and-such mailbox name with return-path of
such-and-such?

You could, in reaction to this notification, then do a STATUS.

What this buys you is that you could react much faster if a mailbox gets
new mail.  Also, you won't have to issue STATUS (at least not as often) to
poll for new messages.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Subject: Re: WATCH extension for IMAP
From: Timo Sirainen <[email-masked]>
To: Mark Crispin <[email-masked]>
Cc: IMAP Extensions WG <[email-masked]>
Organization: 
Date: 17 Apr 2003 05:12:50 +0300
Sender: [email-masked]

On Thu, 2003-04-17 at 04:36, Mark Crispin wrote:
> I've been thinking about defining a WATCH extension to IMAP which would
> allow an IMAP client to watch for certain events happening to a
> non-selected mailbox.  Once a WATCH is set, it remains in effect until
> that IMAP session is logged out, and untagged WATCH events for the watched
> mailbox(es) can be returned at any time.
> 
> The command will be something like:
> 	tag WATCH mailbox event-list
> and the responses will be something like
> 	* WATCH mailbox event-pair-list
> where an event-pair contains an event type and event data.

I think most of the clients would be just interested in getting
automatic updates for "STATUS (MESSAGES UNSEEN)", maybe UIDVALIDITY and
UIDNEXT too. For those it'd be easier if untagged STATUS replies were
received directly. Maybe "WATCH mailbox (STATUS (<status attributes>))"

Date: Wed, 16 Apr 2003 18:36:56 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: IMAP Extensions WG <[email-masked]>
Subject: WATCH extension for IMAP
Organization: Networks and Distributed Computing
Sender: [email-masked]

I've been thinking about defining a WATCH extension to IMAP which would
allow an IMAP client to watch for certain events happening to a
non-selected mailbox.  Once a WATCH is set, it remains in effect until
that IMAP session is logged out, and untagged WATCH events for the watched
mailbox(es) can be returned at any time.

The command will be something like:
	tag WATCH mailbox event-list
and the responses will be something like
	* WATCH mailbox event-pair-list
where an event-pair contains an event type and event data.

The single most important event type will be new mail, with the event data
being the SMTP return-path of the message.  So, in effect, this will be
"biff for IMAP".

We have talked about other event types, such as "message flags changed" or
"mailbox expunged", but it isn't clear how well this will scale (it's one
thing to generate an event when delivering a new mail message, it's quite
another to do so every time some flag gets altered!).

This is in effect an IMAP interface to an ultra-lightweight notification
protocol and facility that I am working on.  The IMAP server won't do any
actual "watching"; instead, it will just be a client to the notification
facility.

I need to emphasize the "ultra-lightweight" nature of this facility.
Although the underlying notification protocol is completely extensible, a
foremost consideration has been lightweight and low-cost (hence the
tenative nature of the flags/expunge event types).  That is also why the
SMTP return-path is in the new-mail notification, because many mail
delivery agents only know that.

Last but not least, this is something that can be made available Real
Soon.  Much of the code already exists.  I know that LEMONADE has a fairly
aggressive schedule for its notification facility, but I suspect that
schedule will slip.

I am interested in feedback as to whether this proposed extension is
interesting or not.  Also, would it be better to build an extension which
has extensibility but with restraints ("ultra-lightweight") on how it can
be extended, or if it should be strictly a new mail watching facility and
leave the possibility of other types of notifications to a different
facilty (such as what is being discussed in LEMONADE).

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sun, 13 Apr 2003 16:39:09 -0700 (PDT)
From: Paul Hoffman <[email-masked]>
To: [email-masked]
Subject: ADMINISTRIVIA: fighting with the new mail server
Sender: [email-masked]

I just changed the OS for the mail server, and am not convinced that I got
all the bugs worked out. So I'm inflicting this on you. :-) Hopefully,
you'll only see this once.

--Paul Hoffman

Date: Sun, 13 Apr 2003 16:35:20 -0700 (PDT)
From: Paul Hoffman <[email-masked]>
To: [email-masked]
Subject: ADMINISTRIVIA: testing the mail server
Sender: [email-masked]

I just changed the OS for the mail server, and am not convinced that I got
all the bugs worked out. So I'm inflicting this on you. :-) Hopefully,
you'll only see this once.

--Paul Hoffman

Date: Sun, 13 Apr 2003 16:33:27 -0700 (PDT)
From: Paul Hoffman <[email-masked]>
To: [email-masked]
Subject: ADMINISTRIVIA: testing out the new mail server setup
Sender: [email-masked]

I just changed the OS for the mail server, and am not convinced that I got
all the bugs worked out. So I'm inflicting this on you. :-) Hopefully,
you'll only see this once.

--Paul Hoffman

Date: Sat, 12 Apr 2003 10:16:07 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: Cyrus Daboo <[email-masked]>
cc: Alexey Melnikov <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks and Distributed Computing
Sender: [email-masked]

On Sat, 12 Apr 2003, Cyrus Daboo wrote:
> | Why would one want to use MODSEQ with THREAD?
> For the same reason one would use it in the search criteria on sort: to
> have the command return results only for those messages that changed since
> the MODSEQ. I doubt this is really a useful behaviour but it has to be
> present for consistency if you are adding MODSEQ to search unless its
> explicitly stated MODSEQ can't be used in sort/thread search criteria.

I'm hard-pressed to think of a use for MODSEQ with THREAD; and while SORT
presents some possible use I doubt that there would be much of that in
real life.

If you declare that MODSEQ is an additional argument to SEARCH (much as
CHARSET is) instead of a SEARCH criterion, then you avoid the issue.  But
that's no good if you decide to have MODSEQ in SORT.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Sat, 12 Apr 2003 11:46:39 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Alexey,

--On Friday, April 11, 2003 10:32 PM -0600 Alexey Melnikov 
<[email-masked]> wrote:

|> You need to change the annotation entry names to exclude the "/message"
|> prefix. We agreed to eliminate the /message and /body entry name prefixes
|> at the last meeting. I'm hoping to get an updated annotation document out
|> in the next week with that and other changes.
|
| Is the \Seen flag now "/flags/\\seen" or "flags/\\seen"?

"/flags/\\seen" - i.e. / prefix is still present.

-- 
Cyrus Daboo

Date: Sat, 12 Apr 2003 11:36:28 -0400
From: Cyrus Daboo <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Alexey,

--On Friday, April 11, 2003 10:20 PM -0600 Alexey Melnikov 
<[email-masked]> wrote:

|> Interaction with SORT and THREAD - note that both SORT and THREAD take
|> search criteria as arguments. Right now the draft only discusses SORT but
|> it needs to include THREAD as well (an example for a thread response with
|> MODSEQ would be good too).
|
| Why would one want to use MODSEQ with THREAD?

For the same reason one would use it in the search criteria on sort: to 
have the command return results only for those messages that changed since 
the MODSEQ. I doubt this is really a useful behaviour but it has to be 
present for consistency if you are adding MODSEQ to search unless its 
explicitly stated MODSEQ can't be used in sort/thread search criteria.

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 23:31:29 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> |> I have no problems with the last suggested change (i.e. HIGHESTMODSEQ
> |> with 0).
> |> Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
> |> then CONDSTORE is not supported for the mailbox? This is probably better.
> |
> | Either way works for me.  I think though that an affirmative statement is
> | better than the absence of one; so HIGHESTMODSEQ 0 or NOMODSEQ seems
> | better than a missing HIGHESTMODSEQ on SELECT.
>
> Agreed, and I would favour NOMODSEQ for clarity as well.

Ok, I've added NOMODSEQ to the document.

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 22:49:47 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Mark Crispin <[email-masked]>, Chris Newman <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Alexey Melnikov wrote:

> Mark Crispin wrote:

>

> >  . STORE should NEVER reply NO; instead, the untagged FETCH response
> >     should be relied upon to indicate what happened
> >
> > I disapprove of the idea of NO with the MODIFIED status code.  This
> > introduces, for the first time, an instance where the server replied NO
> > but something happened anyway.

Actually, nothing happened. This is not much different from [removed] "NEWNAME"
response code.

But I don't really care whether MODIFIED is returned with OK or NO.

Does anybody else have any preferences?

>  The command succeeded and did what was
> > requested; therefore OK should be returned (with the MODIFIED status
> > code).
>
> This is fine with me.

Regards,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 22:32:40 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> You need to change the annotation entry names to exclude the "/message"
> prefix. We agreed to eliminate the /message and /body entry name prefixes
> at the last meeting. I'm hoping to get an updated annotation document out
> in the next week with that and other changes.

Is the \Seen flag now "/flags/\\seen" or "flags/\\seen"?

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 22:20:00 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> Interaction with SORT and THREAD - note that both SORT and THREAD take
> search criteria as arguments. Right now the draft only discusses SORT but
> it needs to include THREAD as well (an example for a thread response with
> MODSEQ would be good too).

Why would one want to use MODSEQ with THREAD?

> Also the formal syntax for the search and sort responses is slightly wrong
> (missing SP)

Thank you.

> and should have thread added:
>
> mailbox-data =/
>         "SEARCH" [1*(SP nz-number) SP search_sort_mod_seq] /
>         "SORT" [1*(SP nz-number) SP search_sort_mod_seq] /
>         "THREAD" [SP 1*thread-list SP search_sort_mod_seq]

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 18:06:27 -0400
From: Cyrus Daboo <[email-masked]>
To: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Interaction with SORT and THREAD - note that both SORT and THREAD take 
search criteria as arguments. Right now the draft only discusses SORT but 
it needs to include THREAD as well (an example for a thread response with 
MODSEQ would be good too).

Also the formal syntax for the search and sort responses is slightly wrong 
(missing SP) and should have thread added:

mailbox-data =/
        "SEARCH" [1*(SP nz-number) SP search_sort_mod_seq] /
        "SORT" [1*(SP nz-number) SP search_sort_mod_seq] /
        "THREAD" [SP 1*thread-list SP search_sort_mod_seq]

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 17:52:21 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Alexey Melnikov <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Mark,

--On Friday, April 11, 2003 2:45 PM -0700 Mark Crispin 
<[email-masked]> wrote:

|> I have no problems with the last suggested change (i.e. HIGHESTMODSEQ
|> with 0).
|> Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
|> then CONDSTORE is not supported for the mailbox? This is probably better.
|
| Either way works for me.  I think though that an affirmative statement is
| better than the absence of one; so HIGHESTMODSEQ 0 or NOMODSEQ seems
| better than a missing HIGHESTMODSEQ on SELECT.

Agreed, and I would favour NOMODSEQ for clarity as well.

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 15:50:51 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Mark Crispin <[email-masked]>
CC: Chris Newman <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Mark Crispin wrote:

> Comments:
>
> A document in Last Call should not have open issues.  Resolve them one way
> or another.  I recommend that:
>  . it NOT be required that conditional STORE be atomic across the message
>     set

Actually I was in favor of this (i.e. not atomic across the message set) for a
long time. It might be just to complex for some servers to add transaction
support.

Chris?

>  . STORE should NEVER reply NO; instead, the untagged FETCH response
>     should be relied upon to indicate what happened
>
> I disapprove of the idea of NO with the MODIFIED status code.  This
> introduces, for the first time, an instance where the server replied NO
> but something happened anyway.  The command succeeded and did what was
> requested; therefore OK should be returned (with the MODIFIED status
> code).

This is fine with me.

> Otherwise, the document seems technically fine to me.  I think that we
> should be kind to the RFC Editor and edit it into a shape that's somewhat
> closer to publication status.

Yep.

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 14:45:06 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Alexey Melnikov <[email-masked]>
cc: Cyrus Daboo <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Fri, 11 Apr 2003, Alexey Melnikov wrote:
> I have no problems with the last suggested change (i.e. HIGHESTMODSEQ with
> 0).
> Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
> then CONDSTORE is not supported for the mailbox? This is probably better.

Either way works for me.  I think though that an affirmative statement is
better than the absence of one; so HIGHESTMODSEQ 0 or NOMODSEQ seems
better than a missing HIGHESTMODSEQ on SELECT.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 11 Apr 2003 15:44:44 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: Mark Crispin <[email-masked]>, [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Cyrus Daboo wrote:

> Hi Mark,
>
> --On Friday, April 11, 2003 1:58 PM -0700 Mark Crispin
> <[email-masked]> wrote:
>
> | I know that this is rather late, but is there a recommended way for a
> | server which has CONDSTORE to operate with a mail store that does not
> | support it (e.g. always return 0)?
> |
> | I could offer CONDSTORE much faster in my server if I don't have to
> | support in in all possible mailbox formats.
>
> I guess that could be done by either allowing 0 or having an untagged
> response as part of SELECT to indicate CONDSTORE is not supported on the
> mailbox being selected, even though the capability is present. The later
> would be better because clients could then not use the extended commands
> for that mailbox. Perhaps the HIGHESTMODSEQ response with value '0' could
> be used as an indication of this?

I have no problems with the last suggested change (i.e. HIGHESTMODSEQ with
0).
Alternatively, we can say that if no HIGHESTMODSEQ is returned on SELECT,
then CONDSTORE is not supported for the mailbox? This is probably better.

Anybody else?

Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Fri, 11 Apr 2003 17:40:41 -0400
From: Cyrus Daboo <[email-masked]>
To: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

The formal syntax for 'entry-name' should be simply defined as:

    entry-name = string

to fall in line with ANNOTATE. There should be a comment/reference to the 
annotate spec at that point. That will also allow you to remove the comment 
about escape rules at that point.

Also, could be use "priv" instead of "private" for entry-type-resp just to 
fall in line with the suffixes that ANNOTATE uses?

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 17:26:58 -0400
From: Cyrus Daboo <[email-masked]>
To: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

You need to change the annotation entry names to exclude the "/message" 
prefix. We agreed to eliminate the /message and /body entry name prefixes 
at the last meeting. I'm hoping to get an updated annotation document out 
in the next week with that and other changes.

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 17:13:41 -0400
From: Cyrus Daboo <[email-masked]>
To: Mark Crispin <[email-masked]>, Pete Resnick <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Hi Mark,

--On Friday, April 11, 2003 1:58 PM -0700 Mark Crispin 
<[email-masked]> wrote:

| I know that this is rather late, but is there a recommended way for a
| server which has CONDSTORE to operate with a mail store that does not
| support it (e.g. always return 0)?
|
| I could offer CONDSTORE much faster in my server if I don't have to
| support in in all possible mailbox formats.

I guess that could be done by either allowing 0 or having an untagged 
response as part of SELECT to indicate CONDSTORE is not supported on the 
mailbox being selected, even though the capability is present. The later 
would be better because clients could then not use the extended commands 
for that mailbox. Perhaps the HIGHESTMODSEQ response with value '0' could 
be used as an indication of this?

-- 
Cyrus Daboo

Date: Fri, 11 Apr 2003 13:58:43 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks & Distributed Computing
Sender: [email-masked]

One last thing...

I know that this is rather late, but is there a recommended way for a
server which has CONDSTORE to operate with a mail store that does not
support it (e.g. always return 0)?

I could offer CONDSTORE much faster in my server if I don't have to
support in in all possible mailbox formats.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 11 Apr 2003 13:47:38 -0700 (Pacific Daylight Time)
From: Mark Crispin <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: [email-masked]
Subject: Re: WG last call on draft-ietf-imapext-condstore-00.txt
Organization: Networks & Distributed Computing
Sender: [email-masked]

Comments:

A document in Last Call should not have open issues.  Resolve them one way
or another.  I recommend that:
 . it NOT be required that conditional STORE be atomic across the message
    set
 . STORE should NEVER reply NO; instead, the untagged FETCH response
    should be relied upon to indicate what happened

I disapprove of the idea of NO with the MODIFIED status code.  This
introduces, for the first time, an instance where the server replied NO
but something happened anyway.  The command succeeded and did what was
requested; therefore OK should be returned (with the MODIFIED status
code).

Otherwise, the document seems technically fine to me.  I think that we
should be kind to the RFC Editor and edit it into a shape that's somewhat
closer to publication status.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.

Date: Fri, 11 Apr 2003 12:34:24 -0500
To: [email-masked]
From: Pete Resnick <[email-masked]>
Subject: WG last call on draft-ietf-imapext-condstore-00.txt
Sender: [email-masked]

Alexey has submitted the latest version of conditional store as a WG 
draft. We've all had plenty of time with it, so this is a WG last 
call on it. Please see:

http://www.ietf.org/internet-drafts/draft-ietf-imapext-condstore-00.txt

Please post affirmative comments as well as any problems you might 
have. I'd like to submit this for IETF last call ASAP.

pr
-- 
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

Date: Sun, 06 Apr 2003 22:40:37 -0600
From: Alexey Melnikov <[email-masked]>
Organization: ACI WorldWide / MessagingDirect
To: Cyrus Daboo <[email-masked]>
CC: Mark Crispin <[email-masked]>, IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Subject: Re: 'Label' flags
Sender: [email-masked]

Mark Crispin wrote:

> On Sun, 06 Apr 2003 23:02:24 -0400, Cyrus Daboo wrote:
> > I've seen 'in the wild' keywords such as $Label1, $Label2 etc Since I'm in
> > the process of adding keyword support and want to have a fixed set of
> > 'labels' that a user can define
>                   ^^^^ client
>
> I think that my correction is important.
>
> If you want "labels" that a user can defined, that's keywords.
>
> What you are asking about is a set of fixed names which has a different
> meaning on a per-client basis.

I agree.

Regarding the original question: I think we should standardize used client
labels, but $Label1/$Label2/... should not be standardized, because they have no
associated meaning.

I've only seen $Label1/... being implemented in Mozilla. Mozilla allows a user
to set on a message one of 5 mutually exclusive labels and a user can modify the
meaning of these labels (i.e. their text description is editable). This is
wrong, as there is no way to share what $Label1 means for different clients, and
it is possible to have unpredictable results as two Mozillas can be configured
differently.

Cheers,
Alexey
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Watford, UK

Work Phone: +44 1923 81 2877
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

Date: Sun, 6 Apr 2003 20:31:27 -0700 (PDT)
From: Mark Crispin <[email-masked]>
Subject: re: 'Label' flags
To: Cyrus Daboo <[email-masked]>
cc: IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Sender: [email-masked]

On Sun, 06 Apr 2003 23:02:24 -0400, Cyrus Daboo wrote:
> I've seen 'in the wild' keywords such as $Label1, $Label2 etc Since I'm in
> the process of adding keyword support and want to have a fixed set of
> 'labels' that a user can define
                  ^^^^ client

I think that my correction is important.

If you want "labels" that a user can defined, that's keywords.

What you are asking about is a set of fixed names which has a different
meaning on a per-client basis.

> I'm interested in knowing whether this is
> something that should be 'standardised' behaviour to promote interop
> between clients.

I certainly hope not.  But I'm not going to oppose it either.

I am somewhat perplexed as to how this would be useful; but I don't see
particular harm either.

Date: Sun, 06 Apr 2003 23:02:24 -0400
From: Cyrus Daboo <[email-masked]>
To: IMAP Extensions WG <[email-masked]>, IMAP Discusson List <[email-masked]>
Subject: 'Label' flags
Sender: [email-masked]

I've seen 'in the wild' keywords such as $Label1, $Label2 etc Since I'm in 
the process of adding keyword support and want to have a fixed set of 
'labels' that a user can define I'm interested in knowing whether this is 
something that should be 'standardised' behaviour to promote interop 
between clients.

-- 
Cyrus Daboo