Subject: Fwd: Re: [Jmap] Fwd: Re: WG Review: JSON Mail Access Protocol (jmap) - reducing configuration complexity

From: Alexey Melnikov <[email-masked]>
Date: Wed, 8 Feb 2017 13:52:00 +0000

Forwarding here, because Arnt only replied to the JMAP mailing list and 
my reply to him went to the same place.

-------- Forwarded Message --------
Subject: 	Re: [Jmap] Fwd: Re: WG Review: JSON Mail Access Protocol 
(jmap) - reducing configuration complexity
Date: 	Wed, 8 Feb 2017 13:48:54 +0000
From: 	Alexey Melnikov <[email-masked]>
To: 	Arnt Gulbrandsen <[email-masked]>, [email-masked]

On 08/02/2017 12:56, Arnt Gulbrandsen wrote:

> Randy Bush writes:
>> yet another protocol that requires a flag day.  that has worked out
>> so well for ipv6.
>
> JMAP does not, AFAICT, require a flag day, any more than IMAP required
> a flag day over POP. If Fastmail's deployment of JMAP-in-spe was
> noticed by its customers, the noise certainly hasn't been loud enough
> that I've heard about it.

Agreed. JMAP and IMAP are likely to co-exist for long time.

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 8 Feb 2017 09:22:41 -0500

> Agreed. JMAP and IMAP are likely to co-exist for long time.

Isn't that exactly the complaint?  Something like, "Long periods of
co-existence need to have lots of benefits or an existential one, or
else they're not a good idea."

Best regards,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 8 Feb 2017 14:27:50 +0000

Hi Andrew,

> else they're not a good idea."
They have slightly different audiences. People who want to do something 
like JMAP are already doing something like JMAP. They might or might not 
care about IMAP. People who are doing IMAP and not webmail might not 
care about JMAP.
It is not yet clear to me that JMAP will replace IMAP. But I don't think 
this is a reason not to let JMAP progress.

Best Regards,
Alexey

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 8 Feb 2017 08:51:09 -0600

FWIW ...

> else they're not a good idea."

Andrew didn't point this out, but since TSVAREA has been paying attention
to it for the past couple of IETF cycles, the IAB does have
https://datatracker.ietf.org/doc/draft-iab-protocol-transitions/ in active
development.

We're finding it helpful. Your mileage may vary, of course.

Spencer

---

From: John C Klensin <[email-masked]>
Date: Wed, 08 Feb 2017 10:03:49 -0500

>> Isn't that exactly the complaint?  Something like, "Long
>> periods of co-existence need to have lots of benefits or an
>> existential one, or else they're not a good idea."

It is certainly my main complaint.

> They have slightly different audiences. People who want to do
> something like JMAP are already doing something like JMAP.
> They might or might not care about IMAP. People who are doing
> IMAP and not webmail might not care about JMAP.
> It is not yet clear to me that JMAP will replace IMAP. But I
> don't think this is a reason not to let JMAP progress.

Alexey, I think you are missing the point.  Except for the small
minority of users who switch MUAs back and forth (see below), I
would expect any given user to use either an IMAP approach, a
JSON or JSON-like one, or neither.    No long transition as far
as they are concerned.    However, from the perspective of
someone trying to maintain servers or a mailstore, the fact that
there will be both types of users (for a long time if not
forever), it implies the need to maintain (and configure,
support, etc.) both IMAP/SMTP and JMAP facilities in parallel
and to support, also for a long time, the ability to convert
between the two formats.  Also, if that conversion is not
absolutely lossless, there will be a large collection of ongoing
problems, for an equally long time.

Those _are_  reasons to not let JMAP progress because it could
easily make the mail system work worse.  Not, as Andrew (and
several others) have suggested, not a risk we should encourage
unless the benefits and improvements are significant.

     best,
      john

---

From: Dave Crocker <[email-masked]>
Date: Wed, 8 Feb 2017 07:33:53 -0800

On 2/8/2017 7:03 AM, John C Klensin wrote:
> However, from the perspective of someone trying to maintain servers
> or a mailstore, the fact that there will be both types of users (for
> a long time if not forever), it implies the need to maintain (and
> configure, support, etc.) both IMAP/SMTP and JMAP facilities in
> parallel and to support, also for a long time, the ability to convert
> between the two formats.  Also, if that conversion is not absolutely
> lossless, there will be a large collection of ongoing problems, for
> an equally long time.

+1

This creates a dual-stack operational environment.  They always have
real -- and often problematic -- operational effects.

Given the long-standing complaints about IMAP's complexity, it still
might be worth doing.  But as has already been noted, what is required
here is a very compelling value statement that justifies the second
'stack'.

The posted list of folk who are expected to implement this helps quite a
bit, IMO.  However we seem to be missing the compelling
/technical-operations/ value proposition.  Will the resulting protocol 
be simpler, more efficient, faster, ... and what is the basis for 
asserting such improvements?

On 2/6/2017 7:40 AM, Alexey Melnikov wrote:
> The current proposal is "advancing on the old". It is improving on
> some IMAP use cases

Where is this explained in the charter -- with enough detail to make it
possible to assess it?  This seems to me foundational for the effort's
value proposition.

On 2/8/2017 5:34 AM, Alexey Melnikov wrote:
>> Drafts?  What drafts?  Perhaps you mean:
>>
>> https://tools.ietf.org/html/draft-jenkins-jmap
>>
>> Except that no draft is cited in the charter.
>
> I realized that I accidentally deleted an important sentence from
> the Charter while doing editing. One of the paragraphs should have
> read:
>
> The work of this group is limited to developing a protocol for a
> client synchronising data with a server. The work will be based on
> draft-jenkins-jmap and draft-jenkins-jmapmail. Any server-to-server
> issues are out of scope for this working group. New end-to-end
> encryption mechanisms are out of scope, but the work should consider
> how to integrate with existing standards such as S/MIME and OpenPGP.

Ahh.  Yes, very helpful.  Thanks!

This then raises a point of nuance about a charter's reference to the 
documents that are foundational to a working group effort:  indicating 
constraints on their role.  That is, what is the working group 
authorized to do with the document. This can be extremely helpful with 
working group management and helping participants to focus.

Roughly the choices are a continuum between having no constraints -- the 
doc is simply used as input and advice -- all the way to only being 
permitted to fix bugs and improve editorial writing.

The text you've provided here says 'based on'.  In a more benign world, 
clarifying what that means wouldn't be necessary, but in the current 
reality, I urge adding text that indicates why degree of modification 
the wg is allowed to make on the drafts.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 8 Feb 2017 16:36:45 +0000

John,

On 08/02/2017 15:03, John C Klensin wrote:
> Alexey, I think you are missing the point.
That is entirely possible :-).
> Except for the small
As far as deployments are concerned, if there is a case of 
co-existentence, then it would fall into one of two categories:

1). JMAP is just a a proxy and uses IMAP on the backend. Configuring 
JMAP server is the same as configuring a MUA.

2). Both are supported by the same software (i.e. Cyrus and Dovecot use 
case). I doubt that incremental cost of configuring both is much higher 
than just configuring one of them.

In either case configuring JMAP clients is a simple(r) proposition: just 
distribute HTTP URI for a JMAP instance.

> between the two formats.
There are no 2 formats, both IMAP and JMAP operate on RFC 5322 objects, 
so the rest of your argument is invalid.
> Also, if that conversion is not
There is a single message format, so no need to convert.

Best Regards,
Alexey

---

From: Ted Hardie <[email-masked]>
Date: Wed, 8 Feb 2017 10:33:22 -0800

On Wed, Feb 8, 2017 at 7:03 AM, John C Klensin <[email-masked]> wrote:

>
>
> Except for the small minority of users who switch MUAs back and forth (see
> below), I
> JSON or JSON-like one, or neither.

I am confused by this claim.  There are substantial numbers of users who
access their mail from mobile and from laptop/desktop systems.  Those,
generally speaking, are different MUAs.

> No long transition as far

> easily make the mail system work worse.

I think you and the proponents fundamentally disagree about what the
current mail system is.  They see a welter of proprietary systems based on
JSON and HTTP, and they want to clean it up by standardizing on one
approach.  You see a standardized system of IMAP/SMTP and want to avoid
standardizing another system.  From the proponents perspective Andrew's
question of "long periods of coexistence" has to be seen in the context of
"long periods of coexistence" _of proprietary systems_ .

If you don't agree on what the mail system is, it's not particularly
surprising that you don't agree on the method for optimizing it.

Not, as Andrew (and
> unless the benefits and improvements are significant.
The proponents seem to have demonstrated that there are proprietary systems
in place.  So I believe the calculus of benefits and improvements must be
done over a different set of data points.

regards,

Ted

---

From: "John Levine" <[email-masked]>
Date: 8 Feb 2017 19:05:46 -0000

>real -- and often problematic -- operational effects.

Well, we've survived with POP and IMAP for two decades.  

We might want to put something in the charter noting that the
underlying mail has to be RFC822/5322, but the drafts all seem to
assume that.

Re separating mail pickup from submission, I understand why we decided
to keep it separate when we did RFC4409, but that was a decade ago and
the mail world has changed.  In particular, I don't think anyone in
2006 expected that the majority of mail users would be sending and
receiving mail using apps on the phones in their pockets.  

At this point, if someone says that their POP/IMAP/JMAP and SUBMIT
servers need different credentials, I'd suggest that is a cruel joke
to play on your users, you should fix it, and while you're at it,
you should accept submission through JMAP, too.  If you can't fix
it, there will still be some IMAP and SUBMIT clients for a long time.

R's,
John

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 8 Feb 2017 21:08:52 +0000

Dave and John,

> asserting such improvements?

Neil Jenkins did a very good summary of existing problems and how JMAP
is trying to improve over IMAP:
<https://www.ietf.org/mail-archive/web/imapext/current/msg05820.html>
> possible to assess it?  This seems to me foundational for the effort's
See pointer to Neil's email above. The list of improvements is rather
long, so I didn't want to include it in the Charter. Instead, the
Charter already says:

The working group will deliver the following:

- A problem statement detailing the deployment environment and
  situations that motivate work on a new protocol for client to
  server email synchronisation.  The working group may choose
  not to publish this as an RFC.

This item was intended to cover exactly that.
> Roughly the choices are a continuum between having no constraints --
> the doc is simply used as input and advice -- all the way to only
> being permitted to fix bugs and improve editorial writing.
>
> The text you've provided here says 'based on'.  In a more benign
> world, clarifying what that means wouldn't be necessary, but in the
> current reality, I urge adding text that indicates why degree of
> modification the wg is allowed to make on the drafts.

I think this is an excellent idea and I will add more text to the
charter about constraints/expected changes.

Best Regards,
Alexey

---

From: Ted Lemon <[email-masked]>
Date: Wed, 8 Feb 2017 16:16:45 -0500

On Feb 8, 2017, at 4:08 PM, Alexey Melnikov <[email-masked]> wrote:
> <https://www.ietf.org/mail-archive/web/imapext/current/msg05820.html <https://www.ietf.org/mail-archive/web/imapext/current/msg05820.html>>

Wow, this is _really_ useful.   Thank you for pointing it out.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 8 Feb 2017 14:16:01 -0800

On 2/8/2017 1:16 PM, Ted Lemon wrote:
> On Feb 8, 2017, at 4:08 PM, Alexey Melnikov
> <[email-masked] <mailto:[email-masked]>>
> wrote:
>> Neil Jenkins did a very good summary of existing problems and how
>> JMAP is trying to improve over IMAP:

Sure is.

This, and Pete's note, have finally given me enough to be able to
suggest specific changes to the charter.

The underlying concern I've been pressing is that a working group
charter needs to make its case in pragmatic terms, with enough detail to
aid someone who is not already onboard to assess the proposed effort.

So I suggest replacing:

> A number of JSON-based representations of email have been developed
> that are proprietary, non-standard, and incompatible with each other.
...
> The JMAP working group will specify a mechanism to allow clients to
> both view and send email from a server over a single HTTPS
> channel with minimal round trips. A single protocol for receipt and
...

with:

> JMAP specifies the interactions between email clients and mail
> stores, providing an alternative to IMAP and SMTP submission. JMAP
> allows clients to both view and send email via a server, over a
> single HTTPS channel, with significantly reduced computational and
> communication resource consumption.
 >
> Currently it is difficult to write a good MUA using standards; it is
> especially problematic for embedded web clients. IMAP suffers the
> combination of semantic bottleneck and leakage: It constrains the
> semantics that can be communicated across the pipe in some ways
> (hence the myriad of extensions required to have a workable IMAP
> implementation), but leaks various server-specific semantics (e.g.,
> the storage model, the constraints on network resources) through to
> the client instead of having a clean protocol. Further, the protocols
> perform poorly over constrained channels.

d/

-- 

   Dave Crocker

-- 

   Dave Crocker

---

From: [email-masked]
Date: Wed, 08 Feb 2017 12:39:30 -0800 (PST)

> > minority of users who switch MUAs back and forth (see below),

FWIW, this is not a small minority any more. A fair number of people own and
use multiple mobile devices and access their email from all of them.

> > I would expect any given user to use either an IMAP approach, a

> JMAP server is the same as configuring a MUA.

This is a key point. A lot of people have criticized this proposal based on the
assumption that new clients will have to support both IMAP/SUBMIT and JMAP.

I think this is very unlikely to be the case. Once JMAP is out there I suspect
there will be a significant number of clients - perhaps the majority - that
will only support JMAP.

These clients will depend on a combination of (1) Large MSPs quickly moving to
deploy JMAP in addition to IMAP, (2) The availability of JMAP->IMAP/SUBMIT
proxies, and (3) The addition of JMAP support to at least some of the open
source message store implemenations.

(2) in particular creates additional requirements that need to be explicitly
called out in the charter. In particular, it's imperative that (a) It be
reasonable to implement JMAP->IMAP/SUBMIT proxies, even if that constrains JMAP
in ways some folks do not like, (b) The list of IMAP extensions needed to
properly implement JMAP be called out, and (c) The security considerations
involved in operating such a proxy need to be described in detail.

> so the rest of your argument is invalid.

Agreed. I have to say I really don't understand the confusion surrounding this
point. IMAP has bodystructure plus a couple of other formats for returning
message data, and a means of creating new messages from pieces of other
messages, none of which look remotely like MIME/RFC 5322, and nobody cares.

This is effectively the same thing; why should anyone care here?

				Ned

---

From: "John Levine" <[email-masked]>
Date: 9 Feb 2017 03:51:49 -0000

>use multiple mobile devices and access their email from all of them.

I can't emphasize this enough.  Everyone in my family from my 20 year
old daugher to my 95 year old father has both a computer (usually a
laptop) and a phone, and reads and sends mail from both.  I'd say that
multiple devices have become the norm, so if want to use your mail in
a sane way, you need to use IMAP or something else that manages a
server mail store shared by all the clients.

R's,
John

---

From: John C Klensin <[email-masked]>
Date: Sun, 12 Feb 2017 15:15:31 -0500

--On Wednesday, February 8, 2017 12:39 -0800 Ned Freed
<[email-masked]> wrote:

>> 1). JMAP is just a a proxy and uses IMAP on the backend.
>> Configuring JMAP server is the same as configuring a MUA.
> 
> This is a key point. A lot of people have criticized this
> proposal based on the assumption that new clients will have to
> support both IMAP/SUBMIT and JMAP.

To be clear about what I'm concerned about, it isn't that.  My
assumption is that, although there might be exceptions, any
given client will support either one or the other, not both.  I
am concerned about what the servers and message stores need to
support and I think that would likely be "both" for a long time
or those who depend on IMAP clients would suffer gradual but
significant degradation of service.   See below for the latter.
Even if every client created after today supports JMAP only, I
would predict a long transition period.   That isn't really an
objection to the WG, just a request that the charter explicitly
requiring addressing the issue.

Parts of that are irrelevant if the message store supports pure
5322 objects with no metadata, but IMAP requires a certain
amount of metadata, so that, AFAICT, either makes the above
relevant or requires that JMAP support a strict superset on
those metadata in compatible formats.

> I think this is very unlikely to be the case. Once JMAP is out
> there I suspect there will be a significant number of clients
> - perhaps the majority - that will only support JMAP.

Yes.  See above.  But note that "JMAP replaces IMAP" requires
that it support identical functionality, be a superset of IMAP
function, or drop only the functions that no one is using or
cares about.  Note that I'm talking about functionality, not
syntax -- there are clearly a number of clumsy constructions in
IMAP that work that way only for historical compatibility
reasons and some cases of multiple ways to do the same thing.
If a new syntax and protocol model were adopted, no one sensible
would continue to defend those IMAP characteristics.

> These clients will depend on a combination of (1) Large MSPs
> quickly moving to deploy JMAP in addition to IMAP, (2) The
> availability of JMAP->IMAP/SUBMIT proxies, and (3) The
> addition of JMAP support to at least some of the open source
> message store implemenations.
> 
> (2) in particular creates additional requirements that need to
> be explicitly called out in the charter. In particular, it's
> imperative that (a) It be reasonable to implement
> JMAP->IMAP/SUBMIT proxies, even if that constrains JMAP in
> ways some folks do not like, (b) The list of IMAP extensions
> needed to properly implement JMAP be called out, and (c) The
> security considerations involved in operating such a proxy
> need to be described in detail.

Exactly.  If one expects JMAP to be wildly successful, and
unless there is either a plausible plan to make all of those
IMAP clients go away (even if there are no new ones), I think
that list should include either its being reasonable to
implement and support IMAP/SUBMIT-> JMAP proxies or other
overlays or a charter requirement to discuss residual use cases
for IMAP.    I'm particularly concerned about supporting
IMAP-native clients with a JMAP-native mailstore.

>> 2). Both are supported by the same software (i.e. Cyrus and
>> Dovecot use case). I doubt that incremental cost of
>> configuring both is much higher than just configuring one of
>> them.
> 
>> In either case configuring JMAP clients is a simple(r)
>> proposition: just distribute HTTP URI for a JMAP instance.
> 
>> There are no 2 formats, both IMAP and JMAP operate on RFC
>> 5322 objects, so the rest of your argument is invalid.
> 
> Agreed. I have to say I really don't understand the confusion
> surrounding this point. IMAP has bodystructure plus a couple
> of other formats for returning message data, and a means of
> creating new messages from pieces of other messages, none of
> which look remotely like MIME/RFC 5322, and nobody cares.
> 

I can't speak for anyone else, but I'm concerned about what it
takes to create and maintain servers and mailstores that are
compatible with both IMAP and JMAP.  If that means proxies, we
either need to address the question of IMAP proxies over JMAP
and JMAP mailstores as well as the JMAP over IMAP-compatible
ones or I'd like the charter is require a good explanation of
why that isn't necessary.  That might be "IMAP-based systems
with JMAP overlays forever", but, the more people argue that
JMAP will take over (rapidly or otherwise), the less plausible
that position feels.

best,
    john

---

From: Dave Cridland <[email-masked]>
Date: Mon, 13 Feb 2017 17:17:17 +0000

On 12 February 2017 at 20:15, John C Klensin <[email-masked]> wrote:
> syntax

I don't think this is true.

If JMAP is to supplant IMAP - and I think that's a worthy goal even if
its likelihood remains a matter for debate - then JMAP has to support
the same model.

The model of IMAP is that:

* Each message resides in a single mailbox,
* Each message has a set of independent flags,
* Each message is immutable.

(One could argue that other metadata exists, given ANNOTATE, but
nobody [to a reasonable approximation] uses ANNOTATE).

Gmail is an example of a case where the model of IMAP doesn't fit the
underlying data model - this is a shame, as IMAP's original design
took considerable effort to fit the wide, existing models of the time.
JMAP, on the other hand, can cope with both gmail style labels and
IMAP-style mailboxes, by stated design.

So I'm confident that, while your statement seems incorrect to me, the
corrected statement would be satisfied.

Dave.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 13 Feb 2017 09:30:09 -0800

...
> IMAP-style mailboxes, by stated design.

The constraints/flexibility of an effort depend on the goals it asserts 
and the support for those goals.  I think it's dandy to target an access 
protocol that is substantially different from the 
features/style/whatever of IMAP.  The requirements are to be very clear 
about the features/style/whatever of the new effort, very clear about 
the expected benefits, and very clear about the support for that.

Given the long history of IMAP, the proposed JMAP effort should offer 
clear assertions of the relative differences and benefits of the new 
effort, to aid in assessing likely appeal of the effort.

So, for example, the above comment about JMAP's handling of different 
reference models is substantive and appealing.  However the various 
notes that have been posted about JMAP leave me thinking that there is 
less clarity about the effort than one would like, among JMAP's supporters.

The embodiment of the clarity needs to be the charter text and I don't 
think it's made its case well enough.  For example, I don't see a 
reference to the ability to support different referencing models.

d/
-- 

   Dave Crocker

---

From: Ted Lemon <[email-masked]>
Date: Mon, 13 Feb 2017 13:48:37 -0500

> * Each message is immutable.

This would be unfortunate.  The "mailbox" paradigm really doesn't work well—as someone mentioned earlier in this thread, it makes synchronization needlessly slow and painful, and it also restricts the end user's ability to tag messages with more than one meaning.

---

From: John C Klensin <[email-masked]>
Date: Mon, 13 Feb 2017 14:04:29 -0500

(Top Post)
Exactly.  Could not have said it better myself.

There is one additional issue here.  JMAP (or some hypothetical
xMAP) can either aspire to be a cleaner, more modern, less
LISP-like interface to (more or less, see below) the same
functionality as IMAP _or_ it can aspire to be a better idea,
with a different underlying architecture and/or reference model,
functionality, etc.  

Can't have it both ways, at least without moving from
discussions about proxies and overlays and into discussions
about translators that are a lot more like gateways, gateways
that, like most of their predecessors, mostly work and mostly
preserve functionality and semantics.

My confusion about this effort -- and it has been considerable--
is that I've heard different statements claiming goals in both
categories.  That isn't acceptable, if one because a WG proposal
for one approach should be evaluated differently from one
proposing the there one.   The choice really must be made now
and, whatever it is, should be absolutely crystal-clear in the
charter... in addition to the things Dave describes below, which
should also be spelled out.

    john

--On Monday, February 13, 2017 09:30 -0800 Dave Crocker
<[email-masked]> wrote:

>> JMAP, on the other hand, can cope with both gmail style
>> labels and IMAP-style mailboxes, by stated design.
> 
> 
> The constraints/flexibility of an effort depend on the goals
> it asserts and the support for those goals.  I think it's
> dandy to target an access protocol that is substantially
> different from the features/style/whatever of IMAP.  The
> requirements are to be very clear about the
> features/style/whatever of the new effort, very clear about
> Given the long history of IMAP, the proposed JMAP effort
> should offer clear assertions of the relative differences and
> benefits of the new effort, to aid in assessing likely appeal
> of the effort.
> 
> So, for example, the above comment about JMAP's handling of
> different reference models is substantive and appealing.
> However the various notes that have been posted about JMAP
> leave me thinking that there is less clarity about the effort
> than one would like, among JMAP's supporters.
> 
> The embodiment of the clarity needs to be the charter text and
> I don't think it's made its case well enough.  For example, I
> don't see a reference to the ability to support different
> referencing models.
> 
> d/

---

From: John C Klensin <[email-masked]>
Date: Mon, 13 Feb 2017 14:20:36 -0500

--On Monday, February 13, 2017 13:48 -0500 Ted Lemon
<[email-masked]> wrote:

> This would be unfortunate.  The "mailbox" paradigm really
> doesn't work well—as someone mentioned earlier in this
> thread, it makes synchronization needlessly slow and painful,
> and it also restricts the end user's ability to tag messages
> with more than one meaning.

Just a clarification in the hope of understanding where we
disagree...

I would have described the IMAP model as assuming that each
message is associated with a single mailbox.  Most of the issues
you mention are associated with "resides".  Up to a point, it is
possible to use a Google-like arrangement to support all of the
IMAP functionality I can think of simply by attaching a label or
attribute with the mailbox name and constraining retrievals,
searches, and displays accordingly.   That is exactly what at
least some of the systems that use a Relational DBMS as a
mailstore do, whether they then have "relation per user" or
treat the UserID as yet another attribute.

That may not be the best thing to do and may or may not provide
equivalent functionality and efficiency for various operations.
In particular, it still implies some advantages for having all
of the messages associated with a given mailbox on the same host
or closely-related set of hosts and I have no idea whether gmail
does that or not.    But, if one goes much past that, my earlier
comments about a switch in reference models and fundamental
architecture apply.

   john

---

From: Ted Lemon <[email-masked]>
Date: Mon, 13 Feb 2017 14:25:23 -0500

> treat the UserID as yet another attribute.

Indeed, but then what you are saying is that replicating the IMAP data model is _not_ necessary, which is my point.   If you think of the data store of email messages as a single relation, with something like a message ID as the key, then it's easy to synchronize with the server: you just make sure that your copy of the relation has the intended subset of all messages in the relation, and it's very easy to compare to see whether this is the case.   If, on the other hand, you have N relations, each of which is a mailbox, then synchronizing becomes a process of iterating across the set of all relations, synchronizing each one.

When you describe it using the relational model, I think it's pretty obvious why you want the single-relation model, and not the multiple-relation model.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 13 Feb 2017 19:52:19 +0000

On 13 February 2017 at 18:48, Ted Lemon <[email-masked]> wrote:
> This would be unfortunate.  The "mailbox" paradigm really doesn't work
> well—as someone mentioned earlier in this thread, it makes synchronization
> needlessly slow and painful, and it also restricts the end user's ability to
> tag messages with more than one meaning.
>

I'm not claiming it's a good model, or even (see John's messages) that
this precludes any other model being made to fit. But I *am* asserting
this is the model around which IMAP was designed. In fact, I could go
further and suggest that at least some portions of the IMAP design
assume that a mailbox is embodied by a single file, but workarounds
for that issue have been long-implemented in the community. And other
models are possible to expose with greater or lesser degrees of
fidelity.

When gmail, for example, exposes its mailstore over IMAP, it does so
by playing fast and loose (and often loose) with the IMAP model. So
messages are visible in multiple mailboxes, but there's no method
exposed within IMAP that allows a client to understand that - to the
client, they're distinct messages with the same content. Flags (or
labels, which are exposed as mailbox names) change on all the copies
in step, but the copies remain copies, from an IMAP perspective.

Within JMAP, the semantics available are a proper superset of IMAP, so
while the IMAP model can be exposed, so can a more gmail-like concept
wherein the same message can appear in multiple mailboxes, for
example. This means, as a casual observation, that the JMAP model (and
therefore protocol) can be used to expose nearly every mailstore
available. Curiously, this was the driver behind some of IMAP's design
choices at the time.

I'd recommend, if you're interested in the differences between the
JMAP and IMAP models, to go look at the https://jmap.io site, which
explains them very thoroughly.

I'd note in passing that I was deeply skeptical of JMAP initially, and
I've been persuaded by solid technical argument as well as market
pressures. I'd be curious about concrete issues rather than
hypothetical issues.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Mon, 13 Feb 2017 19:58:43 +0000

On 13 February 2017 at 19:04, John C Klensin <[email-masked]> wrote:
> Can't have it both ways

I feel sure I'm misunderstanding something here, because clearly you can.

If the underlying model of JMAP is a proper superset of that of IMAP,
then it can both provide the option of different underlying
architectures and/or reference models, and yet also provide a cleaner
interface to the existing IMAP model.

It means that some mailservices, which wish to remain perfectly
compatible with IMAP, can be equally exposed over JMAP (for example,
Cyrus) or exposed via a JMAP/IMAP gateway services. Yet others with a
model that is different to some degree (gmail) will be exposed with
higher fidelity and less awful hacks.

How is this not achieving both goals here?

I understand that these goals *could* be in conflict, but I think it
is a false dichotomy, and one in this case that does not exist.

Dave.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 13 Feb 2017 15:04:02 -0500

> available. Curiously, this was the driver behind some of IMAP's design

Okay, thanks for clarifying.   The jmap.io <http://jmap.io/> site unfortunately has an invalid certificate, but had some useful info.   For some reason I thought you were proposing going back to the straight IMAP model instead of the new JMAP model, which would sadden me.   :)

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 16 Feb 2017 15:20:12 +0000

John and Ned,

> <[email-masked]>  wrote:
  [snip]
>> need to be described in detail.
I updated Charter text to mention these.
> JMAP will take over (rapidly or otherwise), the less plausible
I've done some prototyping of existing JMAP proposal and I am quite 
confident that underlying models are close enough that implementing one 
on top of another is doable. IMAP+QRESYNC extension require IMAP 
mailstores to store change sequence numbers, which is required feature 
in JMAP.

Best Regards,
Alexey

---

From: [email-masked]
Date: Thu, 16 Feb 2017 17:15:55 -0800 (PST)

> I updated Charter text to mention these.

Good.

> mailstores to store change sequence numbers, which is required feature

I agree with the assessment, but what if the propsoal changes substantially?
That's the case I'm worried about.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Thu, 16 Feb 2017 22:34:13 -0500

+1 to both points.  With regard to changed proposals, the
charter improvements will help, but there have been several
comments that imply people have ideas about how to "do things
better" that would invalidate that compatibility assessment.
The combination of the charter, your choices about WG
leadership, and fairly aggressive monitoring need to prevent
such an outcome.

best,
   john

--On Thursday, February 16, 2017 17:15 -0800
[email-masked] wrote:

>> >> (2) in particular creates additional requirements that
>> >> need to be explicitly called out in the charter. In
>> >> particular, it's imperative that (a) It be reasonable to
>> >> implement JMAP->IMAP/SUBMIT proxies, even if that
>> >> constrains JMAP in ways some folks do not like, (b) The
>> >> list of IMAP extensions needed to properly implement JMAP
>> >> be called out, and (c) The security considerations
>> > IMAP clients go away (even if there are no new ones), I
>> > think that list should include either its being reasonable
>> > to implement and support IMAP/SUBMIT-> JMAP proxies or other
>> > overlays or a charter requirement to discuss residual use
>> > cases for IMAP.    I'm particularly concerned about
>> > supporting IMAP-native clients with a JMAP-native mailstore.
>> > 
>> >>> 2). Both are supported by the same software (i.e. Cyrus
>> >>> and Dovecot use case). I doubt that incremental cost of
>> >>> configuring both is much higher than just configuring one
>> >>> of them.
>> >>>> and to support, also for a long time, the ability to
>> >>>> convert between the two formats.
>> >> Agreed. I have to say I really don't understand the
>> >> confusion surrounding this point. IMAP has bodystructure
>> >> plus a couple of other formats for returning message data,
>> >> and a means of creating new messages from pieces of other
>> > I can't speak for anyone else, but I'm concerned about what
>> > it takes to create and maintain servers and mailstores that
>> > are compatible with both IMAP and JMAP.  If that means
>> > proxies, we either need to address the question of IMAP
>> > proxies over JMAP and JMAP mailstores as well as the JMAP
>> > over IMAP-compatible ones or I'd like the charter is
>> > require a good explanation of why that isn't necessary.
>> > That might be "IMAP-based systems with JMAP overlays
>> > forever", but, the more people argue that JMAP will take
>> > over (rapidly or otherwise), the less plausible that
>> > position feels.
> 
>> I've done some prototyping of existing JMAP proposal and I am
>> quite confident that underlying models are close enough that
>> implementing one on top of another is doable. IMAP+QRESYNC
>> extension require IMAP mailstores to store change sequence
>> numbers, which is required feature in JMAP.
>