Subject: [MORG] Plan of Action

From: Dave Cridland <[email-masked]>
Date: Thu, 03 May 2007 14:19:12 +0100

Hi folks,

A quick recap on what we're doing here. I know we've some people 
who're not familiar with the IETF here, and some people who've not 
seen the thread on IMAPEXT which kicked this list into life.

This list has been created to look carefully at how existing mail 
clients search through, sort, and generally organize email. We're 
also going to - hopefully - have client developers talk about what 
they'd like to be able to do. There's another group that might be 
interested in this, too - the "desktop search" people might well be 
interested in the ability to search effectively through remote 
message stores.

Kjetil commented (on IMAPEXT) that this area might be very 
interesting to IMAP webmail people, who can't simply download 
everything and work locally, so if there are any IMAP webmail 
developers about it'd be interesting to hear from you, and if anyone 
knows any, please invite them to join.

Meanwhile, we're also going to look at what features we have on the 
store - the IMAP server - that we can leverage for this. There's a 
number of RFCs and Drafts in the area of searching, organization, 
etc, and we'll need to ensure we're all aware of the current state of 
play. So, IMAP specialists need to start to enumerate these - any 
volunteers?

It occurs to me that both the above might benefit from a Wiki page or 
two - Lisa, is there an Apps Wiki we can use for this, or should I 
just set one up? [For anyone new to the IETF, Lisa and Chris are the 
Applications Area Directors]

Finally, we'll look at what new work we can do to satisfy client 
requirements, write up a draft charter, and move to BOF (the first 
stage in forming a new IETF working group), aiming to form a new 
working group to do the work, and produce RFCs as we decide is 
appropriate. I'd like to take a decision on going down this road 
after Chicago in the summer, hopefully ready for a BOF at the IETF in 
December.

I personally think our ultimate output is likely to be some Standards 
Track and some Experimental specifications for extensions, and maybe 
a couple of Informational documents on how this stuff ties together.

I'd also note that searching is something that, like a lot of IETF 
work, has patent minefields in places - please do read that Note Well 
stuff, and let us know of any IPR you're aware of as soon as you can.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

_______________________________________________
MORG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/morg

---

From: Philip Van Hoof <[email-masked]>
Date: Thu, 03 May 2007 16:36:38 +0200

On Thu, 2007-05-03 at 14:19 +0100, Dave Cridland wrote:

> they'd like to be able to do. 

One thing that might be important with searching is to have IDLE
integrated with it. I'm hoping to see NOTIFY and CONTEXT become adopted
for this.

Second to have a search that spans more than one folder, and having an
IDLE that works on that too.

Third a lot people don't really ask for folders, they ask for virtual
folders based on tags that they'll mark their messages with. So A search
that can efficiently work with this tagging. You could see a tag as the
equivalent of a folder, only that a message can have n tags in stead of
only one folder.

-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 03 May 2007 09:16:02 -0600

Dave Cridland wrote:

> A quick recap on what we're doing here. I know we've some people who're 
> not familiar with the IETF here, and some people who've not seen the 
> thread on IMAPEXT which kicked this list into life.
> 
> clients search through, sort, and generally organize email. We're also 
> going to - hopefully - have client developers talk about what they'd 
> like to be able to do. There's another group that might be interested in 
> this, too - the "desktop search" people might well be interested in the 
> ability to search effectively through remote message stores.

Messaging is more than email. We might want to rope in some folks from 
the IM world might as well. I can try to represent their interests, but 
I'm not a client developer.

> Kjetil commented (on IMAPEXT) that this area might be very interesting 
> to IMAP webmail people, who can't simply download everything and work 
> locally, so if there are any IMAP webmail developers about it'd be 
> interesting to hear from you, and if anyone knows any, please invite 
> them to join.

FWIW, in the XMPP community we have been working on ways to archive 
messages server-side:

http://www.xmpp.org/extensions/xep-0136.html

So searching those archives is a logical extension of that work.

> number of RFCs and Drafts in the area of searching, organization, etc, 
> and we'll need to ensure we're all aware of the current state of play. 

RFC 4417 (Report of the 2004 IAB Messaging Workshop) may be of interest, 
too:

http://www.ietf.org/rfc/rfc4417.txt

Peter

-- 
Peter Saint-Andre
XMPP Standards Foundation
http://www.xmpp.org/xsf/people/stpeter.shtml

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 04 May 2007 18:29:04 +0100

Dave Cridland wrote:

> So, IMAP specialists need to start to enumerate these - any volunteers?

I will do that.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 10 May 2007 15:25:20 +0100

(So I meant to reply to this in less than a week).

On Thu May  3 15:36:38 2007, Philip Van Hoof wrote:
> > also going to - hopefully - have client developers talk about 
> what > they'd like to be able to do. 
Yes, CONTEXT should give dynamic, "push-updated" searches, and NOTIFY 
can be used to explicitly allow these to be sent without a command in 
progress - not that CONTEXT requires use of IDLE at all. I'm 
regarding this much as an essentially solved problem, and CONTEXT was 
designed to handle the SORT case too - this was just stripped out of 
the document, but the syntax is all there.

The remaining item in this area is dynamically updating THREAD 
results, which is rather more tricky.

The whole other area is in considering whether we need new search 
keys and sort criteria.

> IDLE that works on that too.
Right - I think Alexey Melnikov has a multi-mailbox search draft 
somewhere?

Ah-ha: 
http://www.tools.ietf.org/html/draft-melnikov-imapext-multimailbox-search-01

> equivalent of a folder, only that a message can have n tags in 

Chris Newman suggested something similar in a private mail discussing 
the formation of this list - get rid of mailboxes entirely - or at 
least only have one - and have the client instead select messages via 
keywords. So you're not alone, and in good company too.

IMAP has, of course, had "tagging" long before it became popular, as 
the user-defined keyword functionality. Another of Alexey's drafts 
proposes some new keywords: 
http://www.melnikov.ca/mel/Drafts/draft-melnikov-imap-keywords-03.txt

We also have ANNOTATE, which allows arbitrary labelled metadata to be 
attached to messages - you can consider this the "next step" beyond 
tagging, but it's seen little uptake as yet - it's still draft, and 
it'll be Experimental rather than Standards Track.

Arnt Gulbrandsen has VFOLDER which can be used to explicitly 
represent search results as an IMAP mailbox, too, which'd be useful 
to handle legacy clients, although it works best (IMO) with rarely 
changing search results.

However, even given this support, and the legacy support offered by 
VFOLDER, I'd still have to argue against.

I suspect that the major problem with the "folder-less" mail store is 
that it would become quite complex when considering what are 
currently shared mailboxes, and extremely painful when we try to deal 
with ACLs (which are essentially forced to be per-message).

Currently, IMAP ACLs are per-mailbox, which is nicely broad - making 
them per-message would mean dramatically harder resynchronization, 
especially with QRESYNC. For example, since a message which has its 
ACL changed has to be seen to have been EXPUNGEd, when trying to 
decide which messages have been EXPUNGEd, you need to figure out 
whether any messages not visible to the user were previously visible 
to the user, and if so, when. That's sufficiently painful that I 
essentially side-stepped the corresponding issue in my ACAP server 
(to do with per-entry ACLs and DELETEDSINCE) - it's almost insoluble.

In general, for efficiency reasons, ACLs should apply on a 
per-synchronization scope, which in turn implies there's still a use 
for mailboxes.

Dave.

_______________________________________________
MORG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/morg
Note Well: http://www.ietf.org/maillist.html

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 11 May 2007 10:52:34 +0200

Dave Cridland writes:
> Arnt Gulbrandsen has VFOLDER which can be used to explicitly represent 
> search results as an IMAP mailbox, too, which'd be useful to handle 
> legacy clients, although it works best (IMO) with rarely changing 
> search results.

Actually, I'm dropping VFOLDER as an IMAP extension. I keep the code, 
but don't bother with an IMAP extension document. (The searches can 
then be accessed but not manipulated by IMAP clients, which is okay 
IMO.)

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 11 May 2007 21:02:31 +0100

>> play. So, IMAP specialists need to start to enumerate these - any 

Ok, here is a draft list:

New search criteria:
draft-gulbrandsen-imap-inthread-01.txt
draft-ietf-lemonade-search-within-04.txt

Views/virtual mailboxes:
draft-cridland-imap-context-01.txt
draft-ietf-lemonade-vfolder-01.txt (expired)

Per-message/per-mailbox attributes (annotations):
draft-ietf-imapext-annotate-16.txt
draft-daboo-imap-annotatemore-11.txt

Searches accross multiple mailboxes:
draft-melnikov-imapext-multimailbox-search-01.txt (expired)

Sorting and threading:
draft-ietf-imapext-sort-18.txt

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 11 May 2007 21:58:04 +0100

Alexey Melnikov wrote:

>>> etc, and we'll need to ensure we're all aware of the current state 
>>> of play. So, IMAP specialists need to start to enumerate these - any 
>>> volunteers?
>>
>> I will do that.
>

And also:
draft-melnikov-imapext-filters-00.txt (expired) - used to store search 
criteria as server annotations.

---

From: Anil SRIVASTAVA <[email-masked]>
Date: Fri, 11 May 2007 18:37:59 -0700

> draft-ietf-lemonade-vfolder-01.txt (expired)

If Stephane and/or Arnt are not revising this, I can work on getting an 
updated draft out.

-- 
______________
Anil SRIVASTAVA  |  [email-masked]  |  Sun Microsystems, Inc
_______________________________________________
MORG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/morg
Note Well: http://www.ietf.org/maillist.html

---

From: Alexey Melnikov <[email-masked]>
Date: Sat, 12 May 2007 13:15:18 +0100

Alexey Melnikov wrote:

>>>> of play. So, IMAP specialists need to start to enumerate these - 
>>>> any volunteers?
>>>
>>>
>>> I will do that.
>>
>>
This might be relevant as well:

draft-neystadt-imap-status-counters (expired) - allows to get separate 
unseen/total counters per different message types, such as fax, voice, 
video, text, etc.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 12 May 2007 19:04:29 +0200

Anil SRIVASTAVA writes:
> If Stephane and/or Arnt are not revising this, I can work on getting 
> an updated draft out.

Feel free.

Arnt