Subject: [MORG] [Technical Errata Reported] RFC6154 (4422)

From: RFC Errata System <[email-masked]>
Date: Mon, 20 Jul 2015 00:25:30 -0700 (PDT)

The following errata report has been submitted for RFC6154,
"IMAP LIST Extension for Special-Use Mailboxes".

--------------------------------------
You may review the report below and at:
http://www.rfc-editor.org/errata_search.php?rfc=6154&eid=4422

--------------------------------------
Type: Technical
Reported by: Chris Newman <[email-masked]>

Section: GLOBAL

Original Text
-------------
N/A

Corrected Text
--------------
N/A

Notes
-----
There is missing information in this RFC that resulted in interoperability problems when we performed an informal interop test of multiple implementations on 2015-07-19. This is assumed to be a "hold for document update" errata.

There is no guidance in the document related to how special-use attributes are bootstrapped. Servers may pre-create them but there's no requirement to do so and clients may use the CREATE-SPECIAL-USE or METADATA extensions to bootstrap but there is no requirement to use those mechanisms if the server offers them. This resulted in a server implementation assuming the client would bootstrap these attributes and a client assuming the server would do so, thus interoperability failed.

Given this experience, the fastest way to resolve this interop problem over time is for both client and server to take responsibility for bootstrapping special use mailboxes. This means if clients create a mailbox intended for special use (e.g., drafts, sent, junk, trash, etc), and the CREATE-SPECIAL-USE extension is offered, then clients need to provide the USE extension to improve interoperability.  Servers that are provisioned with an end-user's locale or language need to pre-create appropriate special-use mailboxes to improve interoperability of this extension. For existing accounts where the server is upgraded to support this feature, the server can look for mailboxes with names that imply special use and add the attribute to those mailboxes (especially if the server has locale/language awareness provisioned for users). Clients that implement a name-based search if special use attributes are not present and decide to use a mailbox for a special use based on its name can use the 
 METADATA extension, if offered, to label that mailbox appropriately.

Instructions:
-------------
This erratum is currently posted as "Reported". If necessary, please
use "Reply All" to discuss whether it should be verified or
rejected. When a decision is reached, the verifying party (IESG)
can log in to change the status and edit the report, if necessary. 

--------------------------------------
RFC6154 (draft-ietf-morg-list-specialuse-06)
--------------------------------------
Title               : IMAP LIST Extension for Special-Use Mailboxes
Publication Date    : March 2011
Author(s)           : B. Leiba, J. Nicolson
Category            : PROPOSED STANDARD
Source              : Message Organization
Area                : Applications
Stream              : IETF
Verifying Party     : IESG

---

From: Jamie Nicolson <[email-masked]>
Date: Mon, 20 Jul 2015 13:46:50 -0700

[I'm not sure of the procedure for discussing errata, but here's my
thinking.]

Thanks for implementing the spec and providing feedback.

Certainly the spec is vague on the point of who creates special-use
folders. If the server supports CREATE-SPECIAL-USE (presumably that was the
case in your example), then either the server or client can create them. In
this case the client software could have seen that 1) the special-use
folders were missing but 2) the server supported CREATE-SPECIAL-USE, and
then decided to create the folders itself. There's nothing in the spec to
counterindicate this behavior--i.e. nothing that requires the server to
create the special use folders. What happened next--did the client create
new folders but not add the special-use attributes?

It seems to me that, even without this spec, someone (client or server)
would be creating these folders. With the spec, the same entity should
create the folders with the appropriate special-use tag, but first check to
see if a folder already exists with that tag and, if so, use it instead.
So, a client that needs to put Drafts somewhere should 1) look for an
existing folder with \Drafts and 2) if that doesn't exist, create one and
annotate it with \Drafts if the server supports CREATE-SPECIAL-USE.
Similarly, if the server does automatic junk mail classification and needs
a folder for junk mail, it should 1) look for an existing \Junk folder and
2) if it doesn't exist create one. (As a corollary, the server can create
any needed special-use folders at account initialization, since it knows
there will not yet be any existing folders.)

Anyway, this seems like a good topic for discussion, but is not (yet) a
clear error in the specification. Perhaps you should raise it on
[email-masked]?

On Mon, Jul 20, 2015 at 12:25 AM, RFC Errata System <
[email-masked]> wrote:

> There is missing information in this RFC that resulted in interoperability
> problems when we performed an informal interop test of multiple
> implementations on 2015-07-19. This is assumed to be a "hold for document
> update" errata.
>
> There is no guidance in the document related to how special-use attributes
> are bootstrapped. Servers may pre-create them but there's no requirement to
> do so and clients may use the CREATE-SPECIAL-USE or METADATA extensions to
> bootstrap but there is no requirement to use those mechanisms if the server
> offers them. This resulted in a server implementation assuming the client
> would bootstrap these attributes and a client assuming the server would do
> so, thus interoperability failed.
>
> Given this experience, the fastest way to resolve this interop problem
> over time is for both client and server to take responsibility for
> bootstrapping special use mailboxes. This means if clients create a mailbox
> intended for special use (e.g., drafts, sent, junk, trash, etc), and the
> CREATE-SPECIAL-USE extension is offered, then clients need to provide the
> USE extension to improve interoperability.  Servers that are provisioned
> with an end-user's locale or language need to pre-create appropriate
> special-use mailboxes to improve interoperability of this extension. For
> existing accounts where the server is upgraded to support this feature, the
> server can look for mailboxes with names that imply special use and add the
> attribute to those mailboxes (especially if the server has locale/language
> awareness provisioned for users). Clients that implement a name-based
> search if special use attributes are not present and decide to use a
> mailbox for a special use based on its name can use the

---

From: Barry Leiba <[email-masked]>
Date: Mon, 20 Jul 2015 18:17:41 -0400

> [I'm not sure of the procedure for discussing errata, but here's my

Thanks for that, Jamie -- responding with your thoughts is exactly
what you should do.

Normally, errata should not be used for issue tracking, which is what
Chris did here.  He said up front that he expected this to be "held
for document update", which means that the "error" he raised is too
involved for a simple fix that might be "verified", and that it needs
to be discussed further if the document is ever revised.

In fact, you're right that this wasn't an error in the document: we
never intended to specify how special-use folders are bootstrapped.
The reason I agreed with Chris and marked it HFDU is that I think any
revision of the spec *should* address that issue, and that, while it
wasn't, strictly speaking, an error in the spec, it was an error on
our part not to consider it.  And I think we should consider it if we
ever revise it.

Unfortunately, I think it's likely that we won't ever revise it, so
the point may be moot.  Maybe the best way to handle this is for
someone (Chris?) to write a separate document that updates this one,
which specifies how the special-use mailboxes should be set up in the
first place.

Barry

> Certainly the spec is vague on the point of who creates special-use folders.
> If the server supports CREATE-SPECIAL-USE (presumably that was the case in
> your example), then either the server or client can create them. In this
> case the client software could have seen that 1) the special-use folders
> were missing but 2) the server supported CREATE-SPECIAL-USE, and then
> decided to create the folders itself. There's nothing in the spec to
> see if a folder already exists with that tag and, if so, use it instead. So,
> a client that needs to put Drafts somewhere should 1) look for an existing
> folder with \Drafts and 2) if that doesn't exist, create one and annotate it
> with \Drafts if the server supports CREATE-SPECIAL-USE. Similarly, if the
> server does automatic junk mail classification and needs a folder for junk
> mail, it should 1) look for an existing \Junk folder and 2) if it doesn't
> exist create one. (As a corollary, the server can create any needed
> special-use folders at account initialization, since it knows there will not
> yet be any existing folders.)
>
> On Mon, Jul 20, 2015 at 12:25 AM, RFC Errata System
> <[email-masked]> wrote:
>>
>> awareness provisioned for users). Clients that implement a name-based search
>> if special use attributes are not present and decide to use a mailbox for a
>> special use based on its name can use the

---

From: Jamie Nicolson <[email-masked]>
Date: Mon, 20 Jul 2015 15:20:30 -0700

Perfect, thanks for the explanation.

On Mon, Jul 20, 2015 at 3:17 PM, Barry Leiba <[email-masked]>
wrote:

> folders.
> So,
> search

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 21 Jul 2015 07:25:39 +0100

Hi Jamie,

On 20/07/2015 21:46, Jamie Nicolson wrote:
> folders. If the server supports CREATE-SPECIAL-USE (presumably that was
> the case in your example), then either the server or client can create
> them. In this case the client software could have seen that 1) the
> special-use folders were missing but 2) the server supported
> CREATE-SPECIAL-USE, and then decided to create the folders itself.
> There's nothing in the spec to counterindicate this behavior--i.e.
> nothing that requires the server to create the special use folders. What
> happened next--did the client create new folders but not add the
> special-use attributes?

Yes, because CREATE-SPECIAL-USE capability is optional to implement.

Additionally, if a client that doesn't support SPECIAL-USE logs in
first, it will create Drafts, Sent, etc folders without special use
attributes. When a SPECIAL-USE aware client logs in, it can't create
these folders with special use attributes, as they are already created.
So it is either forced to create new folders with special use attributes
(which is annoying) or it has to use yet another extension to add these
attributes to existing folders.

> create the folders with the appropriate special-use tag, but first check
> to see if a folder already exists with that tag and, if so, use it
> instead. So, a client that needs to put Drafts somewhere should 1) look
> for an existing folder with \Drafts and 2) if that doesn't exist, create
> one and annotate it with \Drafts if the server supports
> CREATE-SPECIAL-USE. Similarly, if the server does automatic junk mail
> classification and needs a folder for junk mail, it should 1) look for
> an existing \Junk folder and 2) if it doesn't exist create one. (As a
> corollary, the server can create any needed special-use folders at
> account initialization, since it knows there will not yet be any
> existing folders.)
> 
> [email-masked] <mailto:[email-masked]>?
> 
> 
> 
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     Reported by: Chris Newman <[email-masked]
>     <mailto:[email-masked]>>
> 
>     Section: GLOBAL
> 
>     Original Text
>     -------------
>     N/A
> 
>     Corrected Text
>     --------------
>     N/A
> 
>     Notes
>     -----
>     There is missing information in this RFC that resulted in
>     interoperability problems when we performed an informal interop test
>     of multiple implementations on 2015-07-19. This is assumed to be a
>     "hold for document update" errata.
> 
>     There is no guidance in the document related to how special-use
>     attributes are bootstrapped. Servers may pre-create them but there's
>     no requirement to do so and clients may use the CREATE-SPECIAL-USE
>     or METADATA extensions to bootstrap but there is no requirement to
>     use those mechanisms if the server offers them. This resulted in a
>     server implementation assuming the client would bootstrap these
>     attributes and a client assuming the server would do so, thus
>     interoperability failed.
> 
>     Given this experience, the fastest way to resolve this interop
>     problem over time is for both client and server to take
>     responsibility for bootstrapping special use mailboxes. This means
>     if clients create a mailbox intended for special use (e.g., drafts,
>     sent, junk, trash, etc), and the CREATE-SPECIAL-USE extension is
>     offered, then clients need to provide the USE extension to improve
>     interoperability.  Servers that are provisioned with an end-user's
>     locale or language need to pre-create appropriate special-use
>     mailboxes to improve interoperability of this extension. For
>     existing accounts where the server is upgraded to support this
>     feature, the server can look for mailboxes with names that imply
>     special use and add the attribute to those mailboxes (especially if
>     the server has locale/language awareness provisioned for users).
>     Clients that implement a name-based search if special use attributes
>     are not present and decide to use a mailbox for a special use based
>     on its name can use the
> _______________________________________________
> MORG mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/morg
> Note Well: http://www.ietf.org/maillist.html
>

---

From: Jamie Nicolson <[email-masked]>
Date: Tue, 21 Jul 2015 10:29:28 -0700

On Mon, Jul 20, 2015 at 11:25 PM, Alexey Melnikov <[email-masked]

> Yes, because CREATE-SPECIAL-USE capability is optional to implement.

Sure, but in this case I assume the server supported CREATE-SPECIAL-USE
because it expected the client to create the folders.

> attributes to existing folders.

OK, agreed that is a bit of a mess. With the current spec, if a server
doesn't prepopulate the special-use folders, it should support
CREATE-SPECIAL-USE and METADATA in order to be useful to clients. (Gmail
doesn't support either one, but it prepopulates the folders.)