Subject: Re: [imapext] Mail submission over IMAP

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 00:30:35 +0000

------ Original Message ------
From: "Ned Freed" <[email-masked]>
>
>P.S. And doing this stuff "right", for some value of "right", doesn't 
>make 
>submission facilities in IMAP a good idea. 
I'm one of those who believes submission in IMAP is a great idea, long 
overdue.

But obviously others think it's a bad idea.

I'm keen to make sure I didn't overlook something that would alter my 
position on this.

So, my questions are:

1. Why is submission in IMAP such a bad idea?

2. Why should the benefits of:

* simplified client configuration (one set of credentials and 1 
protocol to configure for mail rather than 2)
* simplified server administration (one firewall port to open)
* reduced support burden for mail operators (no more "I can receive 
mail fine, but I can't send" support queries)
* reduced problems for clients, e.g.
     - ISP blocks port 25 (common)
     - debugging issues related to errors in any configuration item 
(since these are halved).

be abandoned?

Maybe there's a use-case where the badness doesn't apply, and maybe 
that use-case is enough to justify submission in IMAP, and in other 
cases it can be disabled - e.g. shouldn't it then just be a policy 
issue for the operator of the server.

Regards

Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 2 Aug 2012 03:42:57 +0300

On 2.8.2012, at 3.30, Adrien W. de Croy wrote:

> * simplified client configuration (one set of credentials and 1 protocol to configure for mail rather than 2)
> * reduced support burden for mail operators (no more "I can receive mail fine, but I can't send" support queries)
>    - debugging issues related to errors in any configuration item (since these are halved).
> 
> be abandoned?

All of these can be reduced to one question:

 * Why can't the client ask how to submit a message via SMTP protocol (via whatever host:port)?

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 1 Aug 2012 17:44:19 -0700

> I'm keen to make sure I didn't overlook something that would alter my position on this.

Ned, which WG is the right one for the 'mail submission via mail reading protocols considered harmful' draft?  That document needs to be written. Soon. There is a lot of history that must be codified and condensed to prevent future futility ...

--lyndon

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 2 Aug 2012 03:48:42 +0300

> Ned, which WG is the right one for the 'mail submission via mail reading protocols considered harmful' draft?  That document needs to be written. Soon. There is a lot of history that must be codifi…

Even though I don't want to implement it (for a server), and I think it's probably not a good idea .. I'm still not sure if it actually a really bad idea. I can understand why client people want it at least.

---

From: Lyndon Nerenberg <[email-masked]>
Date: Wed, 1 Aug 2012 17:51:06 -0700

On 2012-08-01, at 17:42 PM, Timo Sirainen wrote:

> (via whatever host:port)?

Timo nails it.

All that is missing is resource discovery to find the SMTP submission server.

Given that we don't even have discovery for the mail reading part, it shouldn't be that hard to come up with something that encapsulates *both* reading and sending in a single simple service discovery mechanism.  No?

--lyndon

---

From: Derek Diget <[email-masked]>
Date: Wed, 01 Aug 2012 20:57:40 -0400 (EDT)

On Aug 2, 2012 at 03:42 +0300, Timo Sirainen wrote:
=>On 2.8.2012, at 3.30, Adrien W. de Croy wrote:
=>
=>> 2. Why should the benefits of:
=>> 
=>> * simplified client configuration (one set of credentials and 1 protocol to configure for mail rather than 2)
=>> * simplified server administration (one firewall port to open)

Lets just make the Internet work on TCP port 80. :,)  If we have to use 
another port because this one is full there is always 443. :)

=>> * reduced support burden for mail operators (no more "I can receive mail fine, but I can't send" support queries)
=>> * reduced problems for clients, e.g.
=>>    - ISP blocks port 25 (common)

Clients should be using SUBMISSION (TCP 587 - RFC 6409) and operators 
should be looking at RFC 5068.

=>>    - debugging issues related to errors in any configuration item (since these are halved).
=>> 
=>> be abandoned?
=>
=>All of these can be reduced to one question:
=>
=> * Why can't the client ask how to submit a message via SMTP protocol (via whatever host:port)?

RFC 6186 - Use of SRV Records for Locating Email Submission/Access Services
(Not sure if there are any clients supporting this yet. :(

-- 
***********************************************************************
Derek Diget                            Office of Information Technology
Western Michigan University - Kalamazoo  Michigan  USA - www.wmich.edu/
***********************************************************************

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 01:00:36 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
>Even though I don't want to implement it (for a server), and I think it's probably not a good idea .. I'm still not sure if it actually a really bad idea. I can understand why client people want it…

Actually we're just server implementors as well.

We do get a heap of support issues about inability to do one or other 
of SMTP or IMAP.  I'm sick of debugging those issues with clients, and 
I'm sure customers are constantly bewildered that mail is still so 
"difficult" in the 21st century when we've been doing it for so long.  
There's a perceived lack of progress on basic usability.

The causes are normally due to issues such as inaccessibility of one 
port vs another through a firewall or ISP, or issues relating to 
different realms of credentials, or quite often simple human error 
entering server names and passwords twice.

Anyway, that's why I'm keen to see it.

Ironically, webmail has it.  And where does the future of mail seem to 
be going?

The contortions that mail client vendors go through to try and ease 
this process is obvious when you try to create a user account in 
Thunderbird or many other mail clients.

Any time you use another protocol for something you have issues 
associated with accessibility of that protocol, realms of credentials 
etc.  Each of SMTP, POP3, IMAP have their own methods to establish 
credentials.  

In the context of this discussion, there are 2 functions of mail 
clients.  Retrieval, and submission.

There is a cost associated with maintaining and supporting users of any 
1 protocol.  Requiring 2 protocols for the function of a mail client 
has a double cost compared to a single protocol that could do both.

If we abstracted, back to some sort of discovery protocol where you 
queried for configuration of SMTP or IMAP, we'd be not necessarily any 
better off.

If a client can connect to the discovery service, it knows it has that 
connection, it doesn't know it can connect on other ports.  Any port 
you require to be used has a non-zero chance of being unavailable due 
to whatever reason, so the more ports you require for mail function, 
the higher the chance that you won't get full function.
Adrien
>
>
>_______________________________________________
>imapext mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/imapext
>
>

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 01:03:17 +0000

------ Original Message ------
From: "Lyndon Nerenberg" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 2/08/2012 12:51:06 p.m.
Subject: Re: [imapext] Mail submission over IMAP
>All that is missing is resource discovery to find the SMTP submission server.

Discovery is independent of accessibility.  So whilst I think discovery 
could be a great thing, and simplify a lot of client configuration, you 
still don't address basic accessibility to a port.

Furthermore, the discovery server would probably also require some 
credentials if it is handing out potentially sensitive information 
(which server to connect to for such-and-such a mailbox).

So this would mean we'd either have to hobble discovery to only public 
information, or we'd actually exacerbate the problem.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 01:21:40 +0000

------ Original Message ------
From: "Derek Diget" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 2/08/2012 12:57:40 p.m.
Subject: Re: [imapext] Mail submission over IMAP
>another port because this one is full there is always 443. :)

mail is already going there (along with seemingly everything else).

Webmail, GMail, Live mail, Yahoo, OWA etc etc etc.  All provide 
submission and retrieval in the 1 protocol, with 1 set of credentials.

Personally I prefer to use a dedicated email client, so I'm keen to see 
a dedicated protocol rather than using HTTP (esp in-house).

And we also have to implement policy on HTTP, so actually I'd prefer to 
see port 80/443 used for fewer things rather than more.

>should be looking at RFC 5068.
Sure.  

Is there any way to advertise that IMAP server A and SUBMISSION server 
B use the same realm of credentials (e.g. "you should use same 
user/pass")

Otherwise you still need to enter 2 usernames and passwords.  You'd be 
amazed how wrong people can get this.

>(Not sure if there are any clients supporting this yet. :(

DNS is a difficult answer to that question.  Many organisations would 
love to run a mail server without having to run a DNS server.  
Education here is a cost as well, as people need to be educated to set 
up SRV records etc.

Providing an integrated solution for those that do run both (e.g. 
publish necessary records to DNS from mail server) is also difficult, 
and adds links in the chain which can cause support issues.

Adrien

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 01:32:01 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Adrien W. de Croy" <[email-masked]>
Cc: "Arnt Gulbrandsen" <[email-masked]>;"Ned Freed" 
<[email-masked]>;"[email-masked]" <[email-masked]>
Sent: 2/08/2012 12:42:57 p.m.
Subject: Re: [imapext] Mail submission over IMAP
>* Why can't the client ask how to submit a message via SMTP protocol (via whatever host:port)?

there's one more benefit I forgot.

The simple issue of bandwidth etc (having to transmit the content using 
SMTP, and also APPEND to outbox), especially when dealing with large 
attachments, obviously was considered enough of an issue to propose 
BURL.

Sorry but BURL is hideous IMO.  Requiring SMTP/submission server 
vendors to write IMAP clients, solve inter-realm trust issues etc, when 
compared to writing an SMTP submission agent (very simple) into an IMAP 
server I guess that's why it remains enticing to simply allow 
submission in IMAP itself... you have already established credentials, 
you have the message, you can cheaply copy it to sent items, all you 
need is cheap submission.

The sorts of privacy / trust issues involved with mail submission are 
entirely different to those involved in mail retrieval (which is what 
an SMTP server needs to do to process BURL).  This had all sorts of 
knock-on effects relating to security / privacy of the SMTP transaction 
to protected embedded IMAP credentials, or required some token-based 
approach to avert this.

Seems to me the solutions involved around the sanctioned approach to 
solving this problem are worse than SUBMIT in IMAP.

Adrien

---

From: Ned Freed <[email-masked]>
Date: Wed, 01 Aug 2012 22:49:14 -0700 (PDT)

> protocol to configure for mail rather than 2)

Actually, it's exactly the opposite - since for the forseeble future
a significant number of servers won't suport such an extension, even
assuming it deploys, clients still have to support SUBMIT, so this
complicates configuration rather than simplifying it.

Additionally, unless the extension is properly defined to support future SMTP
extensions - and almost all of the proposals, including the present one, fail
on this, you grot up the client with having to use different protocols in
different cases. Extensive past experience argues strongly that addingc
complexity of this sort is rarely if ever a good idea.

> * simplified server administration (one firewall port to open)

Wrong again, for the the opposite reason - clients suporting this
are even less likely to deploy in a timely way, so you get rid of nothing.

> mail fine, but I can't send" support queries)

This assumes the dominant reason mail can't be sent is because of a submission
problem. My experience says otherwise.

>      - ISP blocks port 25 (common)

Problem already solved by SUBMIT.

> (since these are halved).

More like multiplied.

> be abandoned?

See above.

> issue for the operator of the server.

The only advantage I can see here is for mobile clients, where the use
of a single port and shared authentication has some advantages. But
all of these advantages accrue from a much simpler approach of tunneling
SUBMIT inside the IMAP session. This solves almsot all of the problems
noted above and requires essentially no new protocol work.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 2 Aug 2012 10:50:21 +0200

On 08/02/2012 02:51 AM, Lyndon Nerenberg wrote:
> Given that we don't even have discovery for the mail reading part, it shouldn't be that hard to come up with something that encapsulates*both*  reading and sending in a single simple service discovery mechanism.  No?

We do have an SRV-based mechanism to locate IMAP and Submit servers now. 
Hardly anyone uses it, AFAIK because of the chicken and egg problem.

We might add a way to ask the IMAP server about the Submit server, but I 
don't see any advantage in that over the SRV approach.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 2 Aug 2012 11:03:40 +0200

> But obviously others think it's a bad idea.

It depends what the idea is.

I think providing an exact duplicate of SMTP in IMAP is design and 
duplication, mandating code duplication, and not likely to have a good fate.

Providing something else, with a stated set of use cases and no scope 
extension, might be a good idea. If sending this mail is _the_ use case 
I think it might be a good idea.

Arnt

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 12:28:33 +0000

------ Original Message ------
>complicates configuration rather than simplifying it. 

I agree it complicates the task for client authors, but I'm not certain 
that this would necessarily affect end users.

In the discovery phase of account creation, if the client started with 
IMAP, and established that the server supported SUBMIT via IMAP, then 
it could skip the SMTP / SUBMISSION setup phase, and hide the 
associated configuration UI.

This would therefore result in a simplification for the user 
configuration.

As for the task for the client author, in the scheme of things it's a 
pretty simple spec and set of requirements.  Compared to something like 
QRESYNC / CONDSTORE it's a walk in the park.
>complexity of this sort is rarely if ever a good idea. 

I'm struggling to think of a single ESMTP extension (apart from DSN) 
that would be required to be used by the client for submission.

Most ESMTP extensions I'm familiar with relate to things that are either

a) already covered in the IMAP connection (e.g. authentication, TLS 
negotiation etc)
b) not applicable since transfer of the message is done by 
APPEND/COPY/CATENATE (e.g. transfer encodings, chunking, resume etc)

The one that stands out is DSN control, which was already envisaged by 
the proposal.

Or are you thinking of things like SRS etc?  I wouldn't think those 
were applicable for submission, but only transfer.

>are even less likely to deploy in a timely way, so you get rid of 

I don't know about that either.

Corporates tend to standardise on software for various reasons.  It 
would only take someone to write an Outlook connector to do this.

Obviously there would be a certain amount of time required for uptake, 
during which both systems would be required, but the interim would be 
no worse for general usage.  There's one more dimension added for 
troubleshooting (e.g. how was the mail submitted), but I don't see that 
as a big issue.
>problem. My experience says otherwise. 
OK, I'm interested in your experience here, what are the main causes 
you see?

In fact our biggest cause is user error.

>Problem already solved by SUBMIT. 

just another port.
>>(since these are halved). 

Sure there's more debugging for submission problems, since you inserted 
an option for submission, however in a well-designed system, it should 
be simple to determine how a message was submitted.

I'm mainly thinking corporate use here, rather than say ISP.  ISP use 
would be an entirely different scenario.  Maybe the ISP would then 
choose not to advertise SUBMIT in IMAP.  They can judge the impact on 
their own support-desk either way.

>of a single port and shared authentication has some advantages. 

I think also corporate email use would benefit a lot from this.  That's 
mainly who our customers are.  I don't know what it would mean for 
ISPs, although I understand they do generally field a bunch of support 
about submission.  Maybe that's improved.

I'm sure others here have some interesting comments to make about that.

>But 

That could be an interesting approach, but I still think way more 
complex to implement than a SUBMIT command in IMAP.

Adrien

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 12:30:02 +0000

------ Original Message ------
From: "Arnt Gulbrandsen" <[email-masked]>
>>Given that we don't even have discovery for the mail reading part, it 
>>shouldn't be that hard to come up with something that 
>>encapsulates*both* reading and sending in a single simple service 
>>discovery mechanism. No? 
>
>We do have an SRV-based mechanism to locate IMAP and Submit servers 
>now. Hardly anyone uses it, AFAIK because of the chicken and egg 
>problem. 
>
>We might add a way to ask the IMAP server about the Submit server, but 
>I don't see any advantage in that over the SRV approach. 

It would have 1 advantage I can think of and that's the administrator 
can configure it in the IMAP server, regardless of whether they run a 
DNS server or have ready access to publish SRV records or not.

Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 2 Aug 2012 15:55:14 +0300

On 2.8.2012, at 15.28, Adrien de Croy wrote:

>> Additionally, unless the extension is properly defined to support future SMTP extensions - and almost all of the proposals, including the present one, fail on this, you grot up the client with having to use different protocols in different cases. Extensive past experience argues strongly that addingc complexity of this sort is rarely if ever a good idea. 
> 
> I'm struggling to think of a single ESMTP extension (apart from DSN) that would be required to be used by the client for submission.

EAI in future maybe.

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 13:21:01 +0000

------ Original Message ------
From: "Timo Sirainen" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 3/08/2012 12:55:14 a.m.
Subject: Re: [imapext] Mail submission over IMAP
>>I'm struggling to think of a single ESMTP extension (apart from DSN) that would be required to be used by the client for submission.

I presume you mean draft-ietf-eai-smtpext?

I just skimmed it, I can see how that sort of extension would create 
problems that's for sure.

that I-D already has a bunch of problems, but who knows what likelihood 
it has of being adopted.  If you need to also provide an 
ASCII-compatible email address alternative, the question "what's the 
point" comes to mind.  I also wonder why they allow options for 
encoding of domain part.

if SUBMIT over IMAP mandated support for internationalisable forward 
and return path, the problem could be just pushed to the IMAP server.

But that's only 1 I-D, I guess there may be others.  The rot sets in.

In any extensible system though, we have the choice how to move 
forward.  If someone defines a new SMTP extension, then a client needs 
a server to implement it.  

So we do come back to extending SUBMIT over IMAP to support those 
submission extensions that require it.  

It's hard to gauge how much real impact this would have.  That EAI 
draft is fairly special case in that it requires additional data from 
the submitter.  But any ESMTP extension is optional.

Adrien

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 2 Aug 2012 15:28:38 +0200

On 08/02/2012 03:21 PM, Adrien de Croy wrote:
> it has of being adopted.

Close to 100% according to what I've heard. CJKV users who damned well 
want the features and will live with the problems. Then the rest of us 
will have to choose whether we'll interoperate or not.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 02 Aug 2012 10:32:02 -0400

Hi Adrien,

--On August 2, 2012 1:21:40 AM +0000 "Adrien W. de Croy" <[email-masked]> 
wrote:

>> RFC 6186 - Use of SRV Records for Locating Email Submission/Access
>> Services (Not sure if there are any clients supporting this yet. :(
>>
>
> love to run a mail server without having to run a DNS server.  Education
> here is a cost as well, as people need to be educated to set up SRV
> records etc.
>
> Providing an integrated solution for those that do run both (e.g. publish
> necessary records to DNS from mail server) is also difficult, and adds
> links in the chain which can cause support issues.

There was some recent discussion about developing a service discovery 
protocol that would encapsulate not only email related services, but 
calendaring, contacts, im, directory, security etc - basically anything 
that a user needs to setup in the "Accounts" panel of their device. Folks 
at the calendaring and scheduling consortium (CalConnect) are very 
interested in this and we will have a draft with the basic idea out soon.

The basic premise is an SRV record that points to an http server where a 
service discovery document can be retrieved and that document will contain 
all the necessary setup parameters for a device (or pointers/URLs to other 
config options). Whilst SRV will be present, it will not be mandatory in 
that the discovery document could reside at a .well-known URI at the 
top-level of the domain where accounts are being configured. There are lots 
of nitty-gritty details that would need to be worked out and we can leave 
that discussion for when the draft is out - soon.

-- 
Cyrus Daboo

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 07:27:46 -0700 (PDT)

> All that is missing is resource discovery to find the SMTP submission server.

RFC 6186?

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 07:31:40 -0700 (PDT)

> Ned, which WG is the right one for the 'mail submission via mail reading
> protocols considered harmful' draft?

Probably best to write a draft and take it to appaawg. These days most
documents that don't really warrant a WG of their own are processed there.
OTOH, if a WG is deemed necessary, that's the place to start the process
of chartering it.

> That document needs to be written. Soon. There is a lot of history that must
> be codified and condensed to prevent future futility ...

Agreed. IMO the only proposal that even comes close to making the cut would
be a pure tunneling scheme, and even then I remain to be convinced it is worth
it. 

				Ned

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 02 Aug 2012 10:36:46 -0400

Hi Ned,

> noted above and requires essentially no new protocol work.

I think that is absolutely the right approach if only one port can be used 
(in particular on the basis that SMTP is going to exist no matter what and 
clients are still going to have to do SMTP-over-587 for various services no 
matter what). It would need to be somewhat smarter than just doing plain 
SMTP - e.g. there ought to be a way to pass a reference to an existing IMAP 
message to be used as the DATA or from which the basic FROM/RCPT TO can be 
extracted. But beyond that, it ought to look and feel just like SMTP.

-- 
Cyrus Daboo

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 10:54:20 -0700 (PDT)

> that this would necessarily affect end users.

You have a lot of faith in client authors. Faith which appears to me to
be supported by evidence of past behavior.

> associated configuration UI.

Discovery phases are inherently problematic because they assume the
server is available at the time of account creation.

Moreover, the setup phase you seem so concerned about can simply default to
same set of credentials, which is certainly the norm.

> This would therefore result in a simplification for the user

I'm far from convinced of that.

> that would be required to be used by the client for submission.

8BITMIME. BINARY. SIZE. ENHANCEDSTATUSCODES. FUTURERELEASE. DELIVERBY.
MT-PRIORITY. CHUNKING. PIPELINING. MTRK.

None of these are *required*, but for that matter neither is DSN. But all of
can be quite useful to clients, and with the exception of MT-PRIORITY (too new)
are used by various clients when they are available.

And perhaps more to the point, who can say what extensions will be defined in
the future, and for what purpose? MT-PRIORITY is about to be published as an
RFC.

> a) already covered in the IMAP connection (e.g. authentication, TLS

AFAICT the only ones you "cover" are 8BITMIME, BINARY, CHUNKING, PIPELINING,
and perhaps ENHANCEDSTATUSCODES.

> APPEND/COPY/CATENATE (e.g. transfer encodings, chunking, resume etc)

Your experience appears to be extremely narrow and limited then.

> The one that stands out is DSN control, which was already envisaged by

On the contrary, it doesn't stand out at all in SUBMIT. (Relay is another
matter.)

> were applicable for submission, but only transfer.

If you're talking about post-SRS, that's not a standardized, nor reasonable,
SMTP extension.

> I don't know about that either.

You might want to look at statistics for typical mixes of clients seen
by servers then, with an eye to how many old versions are in use.

> would only take someone to write an Outlook connector to do this.

Even if that were true - and I am highly skeptical it is - it in no way makes a
case for standarizing something.

> OK, I'm interested in your experience here, what are the main causes

Main cause is probably credential issues of some sort. But of the issues that
have to be esclated and which therefore cost the most to address, broken
filtering and bad MIME top the list. In other words, the top complaint after
they're able to connect to their mailbox isn't, "Why can't I send  mail?", it's
"Why didn't the recipient get what I sent?" and "Why was what they got a pile
of garbage?"

> In fact our biggest cause is user error.

Yes, but that doesn't mean the error is problems filling in the name of the
SUBMIT server.

> just another port.

The point is it is a *different* port. ISPs specifically block port 25, and for
good reason. They don't block the SUBMIT port, again for good reason.

> be simple to determine how a message was submitted.

Not when the problem only turns up after it was received. (Please tell me
you're not counting on being able to see Received: fields.)

> their own support-desk either way.

The corporations you appear to be envisioning are a lot more homogenous
than I think it is is reasonable to assume.

> complex to implement than a SUBMIT command in IMAP.

Actually, it's far simpler and provides much more flexibility. I rather suspect
I could implement it in a couple of hours, whereas a properly designed submit
extension would take far longer and require far more testing and QA. The only
part that's in any way tricky is the handling of disconnects.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 2 Aug 2012 22:29:35 +0200

> be supported by evidence of past behavior.

At least we can have faith that the proposal will reappear.

Imap move is progressing nicely. I think we can last call it in a week 
or two. That draft exists because people keep asking for it, and at some 
point it's better to just do a reasonable job of an extension than to 
keep repeating no no no and you should be doing that not this. It's 
quite clear that clients will have to continue supporting 
copy/store/expunge, but still, move is wanted.

Imap submit hasn't had quite the level of requests that imap move has 
had. But the same argument does apply, correspondingly weaker. The 
argument about broken packet filters does, too. And clients _have_ been 
good about supporting some simple extensions. 4978 quite shocked me.

Personally I like 6186 very much. Personally.

Arnt

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 21:03:07 +0000

------ Original Message ------
>At least we can have faith that the proposal will reappear. 
I think so

>or two. That draft exists because people keep asking for it, and at 
>some point it's better to just do a reasonable job of an extension 
>than to keep repeating no no no and you should be doing that not this. 
>It's quite clear that clients will have to continue supporting 
and used.

>argument about broken packet filters does, too. And clients _have_ 
>been good about supporting some simple extensions. 4978 quite shocked 
>me. 
4978 is fantastic for us, especially for mobile or other remote 
clients.  IMAP esp S->C is _highly_ redundant/compressible.  We get 
over 80% reduction in most connections.  On a capped paid $/MB 3G 
connection it's a life saver, and also speeds things up for the client 
quite a bit.

>Personally I like 6186 very much. Personally. 

I am a big fan of discovery, but we need to learn from things like the 
WPAD fiasco (DNS or DHCP to resolve a configuration URL).

Any time you introduce another step to intialisation, it's a point of 
failure.  With WPAD, you get issues like being unable to retrieve the 
WPAD.dat file, or it's syntactically incorrect, or DHCP option 252 
wasn't there, or specified correctly, or the name in the specified URL 
wasn't resolvable etc etc etc etc etc.  There are a lot of things to 
get right in order to get WPAD working.

Adrien

---

From: "Adrien de Croy" <[email-masked]>
Date: Thu, 02 Aug 2012 21:26:59 +0000

------ Original Message ------
>be supported by evidence of past behavior. 

I guess you mean unsupported. I don't disagree, but remain optimistic.

>server is available at the time of account creation. 
definitely, and I've had big arguments with TB devs about this.

As long as the discovery process allows not-too-painful fallback to 
manual configuration, and the number of configuration data points can 
be minimised, that's about the best we can do.

>are used by various clients when they are available. 

Many extensions would become the responsibility of the IMAP server 
acting as a submission client on behalf.  It would need to do the same 
thing its MTA needs to do anyway, connect, evaluate capabilities etc 
etc.

I see things like deliverby, and futurerelease being something that 
should be under client control though.

Anything that requires decisions / data from the client on a 
per-message basis would need to be supplied at the time that submission 
is initiated (either by SUBMIT within IMAP, or something else).

>the future, and for what purpose? MT-PRIORITY is about to be published 

Sure, hence my comment about rot setting in in my later email.

>Your experience appears to be extremely narrow and limited then. 
Anything is possible.  

I'm still keen to solve double-transmission.  in order to do that, the 
message that has already been composed (e.g. in Drafts or Outbox or 
whatever, by way of CATENATE, COPY whatever) needs to be able to be 
submitted for delivery on behalf of the client, without requiring the 
client to retrieve it, and retransmit it to a submission server.

>If you're talking about post-SRS, that's not a standardized, nor 
sure, I agree with you, it's a nightmare.

>by servers then, with an eye to how many old versions are in use. 
most of our customers (and admittedly I'm talking a fairly restricted 
use case) are using 1 or more of 

outlook or outlook express or Thunderbird
and/or IOS mail client

We really don't see significant use of other clients.  But this is not 
rigorous or general by any means.

>case for standarizing something. 
IMO a case for standardisation of something is made in order to enable 
interoperability when a significant (whatever that is) number of people 
are affected.

So if MS decided to put such a thing into Exchange, and Outlook, and 
published a spec for it, I think you'd find it got adopted.

>"Why didn't the recipient get what I sent?" and "Why was what they got 

that's not generally something we can address by anything IMAP-related.

>Yes, but that doesn't mean the error is problems filling in the name 
often entering of credentials is the problem.  But there definitely are 
problems discovering usable submission services.

>good reason. They don't block the SUBMIT port, again for good reason. 
I don't disagree.  I guess my point was that any time you introduce a 
requirement to make another TCP connection you have potential issues 
relating to name resolution, local (client-side) firewalls, routing (if 
it's a different IP), credential establishment etc etc etc.
>you're not counting on being able to see Received: fields.) 
hell no.  Mostly server logs.  It depends on the realm of authority of 
the problem location (e.g. whether you can even get server logs).

>than I think it is is reasonable to assume. 
We mainly deal with SMBs, so maybe they are skewed that way.

>part that's in any way tricky is the handling of disconnects. 
I did think a bit about this before I replied.

Any time the submission service is a different physical process to the 
IMAP service, you have a problem of transmission of credentials.  This 
would require the client to tunnel auth through the IMAP server to the 
submission server, and therefore require configuration for submission 
credentials.  So that problem remains.  You could mandate the same 
creds be used, but that would probably cause problems, or at best place 
limitations on how submission and IMAP are deployed (must be in same 
auth realm).  Maybe that's not so bad.

I don't know how you'd avoid re-transmission of the message from the 
client though, unless the IMAP server proxied the tunneled SMTP, and 
inspected the protocol and you had a BURL-like extension that could 
access the message content.  At least in this case, you wouldn't have 
inter-domain trust issues with a submit server trying to retrieve an 
IMAP message for submission.

The only thing this really would solve is the TCP port access.

Proxying SMTP over IMAP begs the question "why?", and "why not proxy 
everything through some simpler protocol that does the auth/security 
leg-work?".  E.g. CONNECT over HTTP.  I think there was an RFC for 
something that abstracted establishment of creds, security etc before 
passing over to a service.  Does SSH do this?

Adrien

---

From: Brandon Long <[email-masked]>
Date: Thu, 2 Aug 2012 15:07:42 -0700

On Thu, Aug 2, 2012 at 2:26 PM, Adrien de Croy <[email-masked]> wrote:

>> Actually, it's far simpler and provides much more flexibility. I rather
>> suspect I could implement it in a couple of hours, whereas a properly
>> designed submit extension would take far longer and require far more
>> testing and QA. The only part that's in any way tricky is the handling of
>> disconnects.
>>
> credentials.  So that problem remains.  You could mandate the same creds be
> used, but that would probably cause problems, or at best place limitations
> on how submission and IMAP are deployed (must be in same auth realm).
>  Maybe that's not so bad.
>

I'm confused, are there really that many systems which would have separate
trust for IMAP and SMTP?

The servers should be in the same trust domain.  I would imagine the
default for most open source servers on Unix would be something like
calling /usr/lib/sendmail with -f of the user.  For us, we have "send
message" on the Gmail app server, and that server is set to trust us to
choose who to send as (or we can just pass the internal credentials, but we
have to do that for many other requests already).  There's already a mail
server in the same trust domain to delivery mail to the IMAP server, after
all.

If your server really makes this hard, then don't implement SUBMIT.  As
with almost every IMAP extension ever made, the clients have to support the
server not offering it.

> inspected the protocol and you had a BURL-like extension that could access
> the message content.  At least in this case, you wouldn't have inter-domain
> trust issues with a submit server trying to retrieve an IMAP message for
> submission.
>

It removes the re-transmission from the client to the server, which is
often over much slower/expensive links.  Piping the message to
/usr/lib/sendmail hardly qualifies as the same as the client sending the
message twice over a 2G mobile connection.

> The only thing this really would solve is the TCP port access.

It solves the client configuration problem, complete with all the possible
error cases that are associated with every service that needs to be
configured.  In fact, I imagine for the average linux user
self-installation, it also makes it easier to install, as they need to make
passwords work with only a single service (the IMAP server) and not two
services (admittedly its been a while, but smtp-auth for MSA is annoying to
configure on a bunch of the popular mail servers).

Brandon

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 14:45:42 -0700 (PDT)

> client to retrieve it, and retransmit it to a submission server.

RFC 4468.

> outlook or outlook express or Thunderbird

And what versions of Outlook are people using?

> rigorous or general by any means.

Irrelevant to my point.

> published a spec for it, I think you'd find it got adopted.

I'm still failing to see how this is relevant to the current
proposal.

> that's not generally something we can address by anything IMAP-related.

That's exactly my point.

> it's a different IP), credential establishment etc etc etc.

Yes, and every time you introduce a major new extension, you have issues with
bugs, client adoption, server adoption, stateful firewalls (which alone should
be cause for grave concern), credential binding, etc. etc.

It's always a tradeoff, but it is one that is heavily canted towards existing
mechanism, especially when the new mechanism offers at best offers a small
measure of increased operational utility and no new functionality.

I'm really not seeing anything that's even close to making the case here.

> the problem location (e.g. whether you can even get server logs).

Not much of an improvement. Try finding something like this in a server
log in a setup with, oh, say, 10 servers and an aggregate transaction
rate of 10,000 messages a second.

That's not an especially large setup in my world. And given the complexity of
message composition, part of the problem here is that you'll almost certainly
need the actual message in order to be able to eliminate client composition
problems from the mix.

And yes, this is also a problem with BURL.

> credentials.  So that problem remains.

There is nothing that says the SMTP connection that appears through the tunnel
is in the same state as one you get from the start on the SUBMIT port. What
you'd want to have happen is for the IMAP server to proxy over the user's
credentials. How this is done doesn't need to be standardized, although I
suppose you could if you wanted to.

> You could mandate the same

Huh? The current proposal has exactly the same set of problems: You're
depending on the credentials the user has already established with the IMAP
server to be sufficient for the submission operation.

As it happens we solved this problem 10+ years ago in order to support various
sorts of proxies. I won't say it is trivial - there are a couple of details
that aren't obvious - but it's hardly rocket science.

And I doubt very, very much we're the only vendor who has done it. In fact I
know we're not.

> IMAP message for submission.

Exactly. The main problem with BURL is that it tries to address the
inter-domain trust situation, and that makes the protocol overly complex.

> The only thing this really would solve is the TCP port access.

Not true.

> Proxying SMTP over IMAP begs the question "why?",

So does the current proposal, only more so. This is having your work translated
into French because you're hoping against all odds that the translation will be
better than the original. (Apologies to Thurber.)

> and "why not proxy

That's more or less what BEEP tried to do. The lack of interest was
overwhelming.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 15:37:47 -0700 (PDT)

> At least we can have faith that the proposal will reappear.

That doesn't mean it's a good idea.

> had.

Not even close.

And besides, it's an apples to oranges comparison. Wanting to be able to submit
over the same port while resuing credentials doesn't necessarily translate into
doing it the way that's being proposed. Whereas wanting to move messages around
in an atomic fashion in IMAP necessarily translates into something that isn't
going to deviate that much from the current proposal.

> But the same argument does apply, correspondingly weaker.

No, not really. IMAP MOVE provides semantics that are a PITA to implement
otherwise. It makes sense for clients even when they have to support servers
that don't offer it - the marginal utility is that great.

IMAP SUBMIT as proposed, OTOH, provides no new semantics. In fact it provides a
subset, and IMO an unreasonably small subset. And then it locks you in to that
subset. Very different thing.

And if you're assuming that client authors are going to jump on stuff
that doesn't offer them direct and obvious benefit as implementers, then I have
to ask why that hasn't happened in the case of RFC 6186?

> The

> Personally I like 6186 very much. Personally.

I don't. It has various problems, largely due to the wierd way DNS is accessed
and/or provisioned in some environments. I'm not overly fond of the available
DNS libraries or servers either.

But here's the thing. When it comes to service discovery, you have to make a
fundamental choice: Piggyback on an existing implemented prototol or build
something new. Building something new has *huge* associated cost: Development,
implementation, and most of all, deployment. And it's not like it hasn't been
tried. Remember all those SRVLOC servers? (Yeah, me neither.)

Reusing breaks down further into two cases: Reusing the protocol versus reusing
the service. This is where HTTP falls short - you're still talking about
building and deploying a new service if you use it. DNS, OTOH, is deployed and
even has something in it (SRV records) with the correct basic semantics.

That's why RFC 6186 was and is the right choice. Given my druthers I'd prefer
something purpose-built to solve the problem, but that's not going to happen,
and I'm not foolish enough to think it will.

I'll also note that even if the SUBMIT problem were to magically vanish you
still have the which of the four mailbox access protocols should you use. A
problem which, by the way, interacts with the SUBMIT-in-IMAP situation in some
interesting ways.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 16:03:18 -0700 (PDT)

> I'm confused, are there really that many systems which would have separate

You mean IMAP and SUBMIT (even if it's done over port 25, the semantics are
different from those of SMTP). In my experience the answer is no, which is why
I think it makes sense to pass the credentials out of band in a tunneling
scheme.

And there certainly are plenty of clients that force you to enter the
credentials separately instead of making that an option you have to enable. I'm
fairly confident that's just obsolete, or perhaps just poor, client design.

> calling /usr/lib/sendmail with -f of the user.

That may work for some extremely simplistic low end cases, but there's almost
always more to it than that.

But it's not like this is uncharted territory. A wide variety of service
proxies have been commonplace for a long time.

> For us, we have "send

> with almost every IMAP extension ever made, the clients have to support the

This isn't why the current proposal is a bad idea.

> message twice over a 2G mobile connection.

Um, you appear to be in agreement with the text you're arguing with.

My point here (which the text you're quoting was responding to) isn't not that
the general *idea* of IMAP SUBMIT is bad, but rather than the current proposal
is a poor one.

> configure on a bunch of the popular mail servers).

I doubt if any of these proposals will alter that significantly. Even if
you have a means of proxying the credentials over, that necessarily means
the services have to be operating in the same authentication realm, and making
that happen is a big part of the annoyance.

				Ned

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 03 Aug 2012 00:05:38 +0000

------ Original Message ------
>I think it makes sense to pass the credentials out of band in a tunneling

To pass credentials requires either

a) like you say out of band (e.g. kerberos) + AUTH=EXTERNAL or similar
b) pre-arranged trust between the IMAP server and the submit server so 
that the token identifying the client can be assumed trustworthy - this 
would require an SMTP extension probably.

neither of these is a problem if the submit server and IMAP server are 
from the same vendor, but otherwise we'd want to standardise on that 
aspect as well.

>fairly confident that's just obsolete, or perhaps just poor, client design.
still see this a fair bit - esp Outlook.  The checkbox for using same 
creds is still a decision point for the user, and source of errors.

I think it just reflects the fact that it's a lot less likely you'll 
need creds for submission (< 100%) than for retrieval (100%).

>the general *idea* of IMAP SUBMIT is bad, but rather than the current proposal
so how would we make it a good one?  I'm open to anything that 
basically 

a) solves port access / creds
b) solves multiple transmission
c) isn't over-complex
d) doesn't create more/worse issues than it solves (esp 
security-related / cred leakage)

BURL fails on c and d IMO.
Does anyone have any stats on support for BURL in SUBMIT servers in the 
wild?

In the end, there's really only 3 options.

1) PUSH (SUBMIT some way via IMAP connection, either tunneled, or 
proxied or something like the current proposal)
2) PULL (BURL from SMTP/SUBMIT)
3) do nothing (leave it in the too hard / not worth it basket, and wait 
for the next time we have this discussion :-) ).

In terms of constraints, in our use-cases (again restricted), requiring 
same auth domain for submission and IMAP is no real problem, since it's 
most often the case anyway.

Adrien

---

From: Timo Sirainen <[email-masked]>
Date: Fri, 3 Aug 2012 03:53:59 +0300

On 3.8.2012, at 3.05, Adrien W. de Croy wrote:

> Does anyone have any stats on support for BURL in SUBMIT servers in the wild?

BURL was implemented by Apple as a patch for Postfix, but it never got included.

Synchronica Mobile Gateway also advertises BURL.

I've seen no other servers advertise this in port 25, but I don't know about submission port (where it obviously is more useful/likely).

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Fri, 03 Aug 2012 01:29:18 +0000

so what's the general feeling about the success / worthiness of BURL 
then?

A considerable effort went into the standardisation of it.  It was 
obviously felt that the retransmission issue was worth solving.

As soon as the decision was made that SMTP should pull the data from 
another source, where it ended up was pretty much a foregone conclusion 
dictated by the issues that need resolving.

We've avoided adding it to our SMTP server for several reasons, maybe 
my reasons are shared by other authors.  

These are mainly:

a) complexity:

For a SMTP/SUBMIT server to retrieve the message, it needs to
* decode the URL
* check it
* connect to the server
* establish credentials
* select a mailbox
* FETCH the body/part whatever

and deal with the myriad of error conditions that can arise at each of 
these steps

b) investment:

I don't have any IMAP client code.  I have IMAP server code, SMTP 
server code, and SMTP client code.  IMAP client code is non-trivial, 
esp when dealing with the plethora of optional IMAP extensions that may 
come into play.  Cost to develop this is on the high side it seems for 
this function.

c) deployability

It's hard for a client to pass credentials to an SMTP/SUBMIT server to 
enable it to auth to an IMAP server in a way that doesn't offer creds 
to passive snoopers.  So, you have to secure the transport with TLS.  
This has a bunch of deployment / customer education issues.

I guess the idea was to allow BURL to fetch message content from any 
sort of source, but for every different one, the SMTP server needs to 
implement a protocol binding (client) of some sort.  This isn't that 
scalable, and we may not have that many current real (non-hypothetical) 
use-cases other than IMAP.

So, if we were to conclude (I don't know if we're there yet) that BURL 
failed, and that the problem is still worth solving, that only leaves 
some form of push via IMAP somehow.

But that's a few ifs.

Adrien

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 18:02:51 -0700 (PDT)

> would require an SMTP extension probably.

A simple shared secret should suffice.

> from the same vendor, but otherwise we'd want to standardise on that

Seems reasonable.

> need creds for submission (< 100%) than for retrieval (100%).

Not that it matters, but...

The <100% is certainly true for SMTP used for submission, and of course there's
a lot of that, albeit less and less all the time due to spam. For SUBMIT,
given the main use-case for deploying it is authentication, I rather suspect
that number is pretty close to 100%. 

There are also some odd cases where unauthenticated IMAP is useful. But again,
not really relevant.

> so how would we make it a good one?

First and foremonst, a good proposal is necessarily future-proof. And the
evidence supporting the need for this is overwhelming: Just compare what's
involved with mail submission as little as 10 years ago with what is done now.

Future-proofing means supporting future SMTP extensions regardless of form. The
simplest way to do that is with a tunnel. An alternative would be something
modelled on batch SMTP with some form of extension discovery, but that's a lot
more work to implement.

> I'm open to anything that basically

> BURL fails on c and d IMO.

If you believe BURL fails on D, I think there's some misunderstanding of the
security model involved. It's actually quite strong and done properly leaks
nothing. As for C, the problem, as pointed out previously, is that BURL is
designed for a very general case. I don't really see a difficulty with
profiling BURL down to a special case where essentially regular IMAP URLs could
be used with none of the BURL add-ons.

> Does anyone have any stats on support for BURL in SUBMIT servers in the

We support it. Don't have any more general data.

> proxied or something like the current proposal)

Right. Nothing says you can't use push in a tunneling scenario, although it
does make it more complex since it can no longer be a simple modal thing.

> most often the case anyway.

As I've said previously, I don't think that's an unreasonable restriction. In
fact I don't really see it as a restriction at all, because the minute you put
separate authentication into any scheme you can come up with all you're really
solving is the multiple port problem. And I'm sorry, but I don't buy the whole,
"Ohh, opening one more port on the firewall is soo HHHAAAARRRDDD! Waah!"
argument for even a femtosecond, especially when the very same firewall that's
so bloody hard to configure is the one that's does broken deep inspection and
spazzes big-time on any extension being discussed here.

(Did I forget to mention that crap firewalls are a *major* support headache?)

The case for single port in in the mobile space, where if anything the
liklihood of a common domain is higher and the need for simple configuration
much greater.

But I'll warn you that while a tunneling approach removes many objections,
including a lot of my own, it's still going to be a major uphill slog to get it
through the process. I recommend full provisions and a very tough skin.

				Ned

P.S. I'll again repeat that while I'm a lot happier with something that's
future-prooffed and which can be supported without a lot of additional code,
I'm still not convinced the benefits really outweigh the costs.

---

From: Ned Freed <[email-masked]>
Date: Thu, 02 Aug 2012 19:02:45 -0700 (PDT)

> I've seen no other servers advertise this in port 25, but I don't know about submission port (where it obviously is more useful/likely).

Given that BURL is only supposed to be enabled on SUBMIT, and it should show up
only when authenticated, it's next to impossible to scan for.

				Ned

---

From: Brandon Long <[email-masked]>
Date: Thu, 2 Aug 2012 23:21:16 -0700

> involved with mail submission as little as 10 years ago with what is done

Like what?  I ask because smtp.gmail.com seems to be getting along fine for
MSA with nothing particularly special, and nothing that isn't protocol
specific.  For all the complaints about our IMAP implementation, I haven't
heard anyone clamoring for any smtp extensions we should implement.

> modelled on batch SMTP with some form of extension discovery, but that's a

I think this also focuses too much on SMTP.  I don't want to embed an smtp
server into my IMAP server, I don't want my IMAP server to act as a network
proxy to an SMTP server.  In some cases, SMTP is already a protocol
conversion and its probably easier to implement this using something else
anyways (ie, shell out to /usr/lib/sendmail in the most basic case, I'm
sure Exchange can just call the appropriate MAPI function, we'd just call
the gmail app servers send mail method).

Sending mail is "send this message from me to these addresses".  IMAP
already has a model for what messages look like.  No need for 8BITMIME or
extensions like that, no need for SMTPUTF8.

No need for any of the auth extensions, auth sharing would be up to the
imap server to implement if necessary.

DSN, sure.  It would need to be optional since not all systems support it
anyways.

Which is basically what this proposal was.  APPEND the message to the
server, then tell the server to send it from foo to bar.

I feel like a lot of the hand-wringing about this has been very meta
without very specific use cases or problems that will occur.

Might be implemented incorrectly, might not be implemented, might hit a bad
stateful firewall... these are all true for each of the existing IMAP
extensions... hasn't seemed to keep us from having a lot of IMAP extensions.

As for future proof, assuming any new extension will have wide-spread
adoption in either smtp or imap seems pretty unlikely without some huge
benefits.  If some great new SMTP extension comes along that just has to be
used and is being adopted wildly, I can't imagine there would be that much
impediment to also adopting a related extension for IMAP.  Worst case, we'd
be right back where we are now, with clients using two protocols.

Brandon

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 3 Aug 2012 10:41:43 +0200

On 08/03/2012 12:37 AM, Ned Freed wrote:
> that doesn't offer them direct and obvious benefit as implementers,

It's a bit of a mystery to me what gains wide usage and what not. 
Simplicity seems to help, visible benefits too.

> then I have

I think that hasn't happened for four reasons.

1. 6186 is late. It's also recent, so maybe a conclusion is premature.

2. Few resolver APIs offer SRV queries, so it's bothersome to implement.

3. There's a competitor which offers better answers in many cases (I 
forget the name, it's a combination of an algorithm and a database from 
mozilla.org).

4. It doesn't fit well into a popular client design: Do lookups at 
install time (the "discovery phase") and expect that to never ever change.

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 3 Aug 2012 07:28:23 -0700

On 3 Aug 2012, at 01:41, Arnt Gulbrandsen <[email-masked]> wrote:

> It's a bit of a mystery to me what gains wide usage and what not. Simplicity seems to help, visible benefits too.
> 
>> then I have

I hope so.

And of course this is not a good argument in favour of Mail Submission over IMAP.
> 2. Few resolver APIs offer SRV queries, so it's bothersome to implement.

Stupid OSes, although this is changing.
> 
> 3. There's a competitor which offers better answers in many cases (I forget the name, it's a combination of an algorithm and a database from mozilla.org).

A static database is not going to work long term.

I am also curious to know what is missing in 6186.
> 
> 4. It doesn't fit well into a popular client design: Do lookups at install time (the "discovery phase") and expect that to never ever change.

I don't agree with this. RFC 6186 allows for lookups at later times.

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 3 Aug 2012 07:38:21 -0700

> I've seen no other servers advertise this in port 25, but I don't know about submission port (where it obviously is more useful/likely).

Isode M-Switch does it on the Submission Port.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 3 Aug 2012 16:48:17 +0200

> A static database is not going to work long term.

I think it's a web thing. You query via GET to a specified URL, and the 
server tells you what it knows about the domain that interests you.

> I am also curious to know what is missing in 6186.

The ability to override when the domain owner doesn't do the right thing.

Overriding doesn't really scale, but if you try mail.<domain>, 
imap.<domain> and the web database thingy has a list of a few hundred 
big well-known exceptions, your results will be good right away. You 
don't need to wait for site admins to add SRV records before your 
results will be good.

> I don't agree with this. RFC 6186 allows for lookups at later times.

Exactly. If a client's design doesn't, 6186 is at odds with the client.

Yes, you can just look up and store, but once you start doing that 
someone will tell you "fix the bug properly, look it up later" and then 
it turns into a more complicated fix, and before you know it the simple 
fix doesn't get implemented and shipped. Been there, done that, and once 
lost a customer because various people shouted incompatibly at me and in 
the end I didn't satisfy any of them.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Fri, 03 Aug 2012 05:36:27 -0700 (PDT)

> Like what?

10 years ago authenticated SUBMIT, first standardized in 1998, was finally
starting to roll out. The authentication situation was finally starting settle
down to TLS+PLAIN, with the realization that DIGEST-MD5 wasn't working.

10 years ago it was finally becoming clear that DSNs were the way to go.
Prior to that there were any number of alternate schemes in use.

As a result a lot of products released around that time (and therefore designed
some time before that) shipped with the wrong defaults and sometimes even the
wrong things implemented.

> I ask because smtp.gmail.com seems to be getting along fine for
> specific.

And your point is? Gmail was two years away from beta 10 years ago. Standards
are supposed to be written to last a lot longer than Gmail has even existed.

And not to be snarky or anything, but "getting along" is not what I'd call a
ringing endorsement.

> For all the complaints about our IMAP implementation, I haven't

Perhaps because people who want those things don't look to Gmail to provide
them.

There's also another factor. SMTP degrades a lot more smoothly than IMAP when
an extension the client wasn't to use isn't present. But it does degrade. And
people notice. But since things work, for some value of "work", they're less
likely to complain.

> the gmail app servers send mail method).

Well, on this one we simply disagree, because what I don't want to do is
reimplement SMTP somewhere else and then have to keep two implementations
up to date. Google may have the time and money to throw at this sort of thing,
a lot of others don't.

> extensions like that, no need for SMTPUTF8.

The fact that your personal model for the service message transfer offers is 
extremely limited is not a reason to codify those limitations into standards.

And with this I'm done. If you cannot see the essential point that none
of us has a crystal ball and cannot therefore predict what will happen
to email as a service in the future, then there's really no sense in continuing
this discussion.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 4 Aug 2012 00:15:02 +0200

> Prior to that there were any number of alternate schemes in use.

All this sounds irrelevant to a submit command in IMAP.

Arnt

---

From: Ned Freed <[email-masked]>
Date: Fri, 03 Aug 2012 15:35:11 -0700 (PDT)

> All this sounds irrelevant to a submit command in IMAP.

Sigh. One last time...

Of course it's irrelevant to IMAP SUBMIT *now*. But it would have been highly
relevant had IMAP SUBMIT been designed ten years ago. Had that been done, I
rather suspect that DSN support would not have been included. (In fact there
were proposals for an IMAP SUBMIT around that time, and guess what? No DSN
support.) Which would have kinda sucked for clients that want to use it now,
don't you think?

For the Nth time, the point is that we cannot predict the future. That's why we
make protocols extensible, and why we spend so much time worrying over the
extensibility model we've selected. We've designed plenty of non-extensible
protocols in the past. And often as not we've subsequently had to go to a lot
of trouble to fix them later, often at significant cost.

And now we have a proposal that's basically saying, "We know for sure that the
evolution of message transfer is over and we therefore need not provide for
extensibility." And that's quite simply crap.

And spare me your arguments about limited use cases and similar nonsense. We're
nothing short of terrible at predicting how things will actualy be used once
they are standardized. As such, our designs need to as robust and flexible as
we can reasonably make them. And in this particular doing so is a real win,
since it actually simplifies things.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 03 Aug 2012 18:21:52 -0700 (PDT)

> 2. Few resolver APIs offer SRV queries, so it's bothersome to implement.

FWIW, it's not too hard to hack one out of a MX lookup routine. Only one
more field, as I recall.

> install time (the "discovery phase") and expect that to never ever change.

Yeah, and that works swimmingly well when things do change. I've had a number
of friends with the "can't access their mail" issue that boiled down to this.

That's said, I'm interested in an item that wasn't on your list - the problem
some enterprise-level name lookup services had (or maybe used to have) with
doing SRV lookups, or for that matter anything but A and TXT records.

If this is no longer an issue, that's a (small) win.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Fri, 03 Aug 2012 18:57:22 -0700 (PDT)

> come into play.  Cost to develop this is on the high side it seems for

I wrote our BURL IMAP client from scratch using no external library. I just
checked: 520 lines of fairly linear C, including comments. Both imap: and
imaps: are supported. It can probably done in 2/3 of that if  you don't have as
much debugging code as we tend to put in.

I would not call that trivial, exactly, but my rough estimate for the SMTP
client code needed for IMAP SUBMIT is at least 3X that, the main difference
being the error handling is much more complicated.

I also don't see how IMAP extensions are a factor. The only extension
you need is URLFETCH; if you don't have that, you can't implement BURL.

				Ned

---

From: Alexey Melnikov <[email-masked]>
Date: Fri, 3 Aug 2012 21:46:19 -0700

> Yes, you can just look up and store, but once you start doing that someone will tell you "fix the bug properly, look it up later" and then it turns into a more complicated fix, and before you know it the simple fix doesn't get implemented and shipped. Been there, done that, and once lost a customer because various people shouted incompatibly at me and in the end I didn't satisfy any of them.

I fail to see how is this relevant to "6186 versa something else" discussion.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 4 Aug 2012 11:16:22 +0200

On 08/04/2012 12:35 AM, Ned Freed wrote:
> relevant had IMAP SUBMIT been designed ten years ago.

I read your other mail and see what you mean. But it's not correct in my 
case.

The submit code would 50-100 lines in my case. It would call code 
already written for e.g. append, sieve vacation, blah. But I can see 
that if you have nothing already, imap submit is hard.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 4 Aug 2012 15:30:35 +0200

This thread troubles me now and isn't going to lead to anything good. 
I'll stop paying attention.

Jan, I'll implement it if you publish a specification and it's not too 
tricky.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Sat, 04 Aug 2012 09:39:37 -0400

Hi Ned,

> FWIW, it's not too hard to hack one out of a MX lookup routine. Only one

Interestingly enough support for SRV service discovery on OS X has been 
driven by calendar and contacts clients that use it with CalDAV and 
CardDAV. At present the mail client still uses the traditional "try and 
guess a suitable host name at a domain" approach. I am doing my best to 
persuade them to adopt SRV.

That said, the focus here is too narrow. As I mentioned before, there is a 
need for "account-wide" service discovery - i.e. setting up all services 
for a domain in one go. It is not sufficient to make setup for email easy, 
it needs to apply across the board for calendaring, contacts, directory, 
etc.

One of the driving forces behind this is the fact that several vendors at 
CalConnect, those who offer both an IETF-standards based suite of 
protocols, and one other proprietary, popular suite, have found that, even 
though their servers offer both suites, that site admins typically favor 
the proprietary solution because it has a single step setup. Now one can 
argue this is the fault of the clients - clients could take a single 
identifier from the user and do the separate SRV, guess-the-hostname tricks 
for each service and configure those. But that is a pain on the client 
side, and a pain to setup on the server side (with lots of different SRVs). 
So we want a way to deliver a single service discovery document that the 
overall "account setup" UI in a client can use to populate all services.

> of friends with the "can't access their mail" issue that boiled down to

One thing we have been doing in CalDAV is to support in-protocol service 
redirection. What happens is that clients typically on startup, or at 
regular intervals, check a user's calendar-home property. Typically that is 
a absolute URI without the host portion, but when the server wants to 
"re-base" a user's account, that will change to include the new host. 
Clients are then expected to follow that URI and reconfigure their 
settings. For a while, the server will proxy requests from the original to 
the new servers, so that the whole process is transparent to the end user.

Now IMAP does have something like that in the form of RLOGIN, but I guess 
client support is limited. If we do develop a broad service discovery 
mechanism, it would make sense to come up with an IMAP response code of 
some kind to allow for in-protocol service redirection akin to what we do 
in CalDAV. I am not sure whether that is sufficient to cover the use cases 
we have complaints about, but it may be a good start.

-- 
Cyrus Daboo

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 4 Aug 2012 17:05:36 +0300

On 4.8.2012, at 16.39, Cyrus Daboo wrote:

> Now IMAP does have something like that in the form of RLOGIN, but I guess client support is limited. If we do develop a broad service discovery mechanism, it would make sense to come up with an IMAP response code of some kind to allow for in-protocol service redirection akin to what we do in CalDAV. I am not sure whether that is sufficient to cover the use cases we have complaints about, but it may be a good start.

Isn't that login referrals (rfc 2221)?

a login user pass
a OK [REFERRAL imap://[email-masked]/] Please move to another server, but I'll proxy you for now anyway.

Maybe the same reply could also be sent untagged after login.

---

From: Cyrus Daboo <[email-masked]>
Date: Sat, 04 Aug 2012 10:37:36 -0400

Hi Timo,

--On August 4, 2012 5:05:36 PM +0300 Timo Sirainen <[email-masked]> wrote:

>> Now IMAP does have something like that in the form of RLOGIN, but I
>> guess client support is limited. If we do develop a broad service
>> discovery mechanism, it would make sense to come up with an IMAP
>> response code of some kind to allow for in-protocol service redirection
>> akin to what we do in CalDAV. I am not sure whether that is sufficient
>> to cover the use cases we have complaints about, but it may be a good
>> start.
>

All that proves is that client developers are very good at ignoring that 
spec!

Actually, 2221 does not specifically mention the possibility of proxy on 
the old server. Instead it talks about "home server referral" where an OK 
with a [REFERRAL ...] indicates that the user's personal mailboxes have 
moved, but pubic mailboxes may still be available. So perhaps some 
clarification is needed. I am hoping that a broader service discovery 
solution can suggest use of this - in fact I will add something like that 
to the document I am working on.

-- 
Cyrus Daboo

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 4 Aug 2012 17:43:09 +0300

On 4.8.2012, at 17.37, Cyrus Daboo wrote:

> All that proves is that client developers are very good at ignoring that spec!
> 
> Actually, 2221 does not specifically mention the possibility of proxy on the old server. Instead it talks about "home server referral" where an OK with a [REFERRAL ...] indicates that the user's personal mailboxes have moved, but pubic mailboxes may still be available. So perhaps some clarification is needed. I am hoping that a broader service discovery solution can suggest use of this - in fact I will add something like that to the document I am working on.

Yeah. Somehow a long time ago when I read that spec I immediately thought of it as alternative to proxying the connection. Dovecot's default login referral string is even "Logged in, but you should use this server instead." Of course I'm sure there are about 0 installations which have actually enabled this. :) Client-transparent proxying is much more commonly used everywhere.

---

From: Ned Freed <[email-masked]>
Date: Sat, 04 Aug 2012 07:28:04 -0700 (PDT)

> it needs to apply across the board for calendaring, contacts, directory,

Yes, you're absolutely right - our focus is too narrow. OTOH, one of the main
reasons other proposals in this area have failed is because they attempted to
do too much.

One boundary that IMO must not be crossed is attempting to describe service
semantics or relationships in the discovery mechanism. The notion that a client
could make use of a generic sort of service never made any sense to me, but
that's what a lot of past attempts tried to cover.

> overall "account setup" UI in a client can use to populate all services.

Another problem here is that you're back to having to deploy a new discovery
service, with all the headaches that implies. And yes, the proprietary stuff
has similar issues (and the hack of using port 80 only solves some of them),
but in many cases it has the advantage of already being deployed.

But I think the biggest problem if you go down this path is going to be getting
it scoped correctly and keeping it focuses within those bounds.

> the new servers, so that the whole process is transparent to the end user.

On a side note, I was forced to abandon iCal because of serious bugs in this
area. With depressing regularly it would lose its ability to connect to the
CalDAV server. Resetting the configuration would fix it, but who wants to do
that every couple of days?

> we have complaints about, but it may be a good start.

I wish I could be more optimistic about this approach, but I'm not. There's a
long history of attempts to do this sort of redirection in protocols, going
back at least as far as the 251 response in RFC 821. I can't think of a single
case where this sort of thing has deployed. And I don't think the failure has
been a result of security concerns either.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sat, 04 Aug 2012 11:26:21 -0700 (PDT)

> that if you have nothing already, imap submit is hard.

If you see this as a relevant response,  I guess I'm not being clear and need
to try again.

This was about standardization and deployment. Not implementation details. The
current specification contains support for exactly one SMTP extension - DSN -
despite the fact that there are quite a few that require input from the client
and therefore cannot be done on behalf of the client by the IMAP server. (I
note in passing that the DSN support as specified is broken: There needs to be
a way of specifying an envelope id, there needs to be a way to specify return
of content, and ORCPT is per-recipient, not per-message.)

The argument has been made that other extensions aren't used very often and
that this proposal is therefore justified in ignoring them. In fact it doesn't
even specify a means of addiing additional what it call "submission options".

I believe this is exceptionally poor design choice, but let's suppose for a
moment that's not the case.

But what happens when an extension appears in the future that has widespread
appeal and which requires client input? As it is written now, the specification
would have to be revised, yet another IMAP extension would be needed, yadda
yadda yadda. You could get rid of the need to revise the specification by
including some means of adding new submission options. But even then you have
significantly complicated the upgrade scenario for submitting messages.

This is what I mean when I say this proposal isn't future-proof. I'm talking
about standardization and deployment, not how easy it is to implement the
current IMAP SUBMIT proposal.

Contrast this with the approach I have been, well, not advocating, but at least
suggesting as an alternative: An SUBMIT tunneling approach. This removes the
need for the IMAP server to be aware of these sorts of extensions, effectively
removing the IMAP server as an inpediment to SUBMIT upgrades. Done correctly
this approach retains the ability to forward without download, eliminates the
need for a SUBMIT port, and eliminates the additional authentication step.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Sat, 04 Aug 2012 18:53:17 -0700 (PDT)

> of content, and ORCPT is per-recipient, not per-message.)

Sorry, that should have been NOTIFY, not ORCPT.

				Ned

---

From: Barry Leiba <[email-masked]>
Date: Sun, 5 Aug 2012 11:43:29 -0400

I've been reading this now-70-message thread with some interest:
This means that MOVE is done, now?

Barry, kinda-sorta poking in as AD

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 5 Aug 2012 21:15:45 +0200

I perceive implementer happiness and do not expect more substantive changes. Of course i may be wrong.

I suggest WGLC when the lull after the meeting has passed. Say in a week from now.

Arnt

---

From: Barry Leiba <[email-masked]>
Date: Sun, 5 Aug 2012 23:15:46 -0400

> I perceive implementer happiness and do not expect more substantive changes.

OK.  Then we should make sure we push on that conversation, to confirm this.

> Of course i may be wrong.

It's been known to happen.

> I suggest WGLC when the lull after the meeting has passed. Say in a week from now.

I suggest that we've been discussing other stuff too much, and that
before the chairs step in for a call on this, we make sure we pull the
conversation back in to MOVE, and stop talking about SUBMIT for a
while.  I know this was one of the dangers of re-using an old mailing
list, and so I'm not being very picky about it.  Still....

Barry, just suggesting a re-focus, as questionably responsible AD

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 6 Aug 2012 10:20:09 +0200

On 08/06/2012 05:15 AM, Barry Leiba wrote:
> while.

Well. The recent change was: MSN-based move is now in the draft instead 
of being an open issue.

Noone has a problem (in the first person) with MSN-based move AFAICT. 
Problems have been mentioned in the third person or past tense umpty 
times in the past decade, but when I tried to chase them down they've 
evaporated.

Personally I think MSN-based move is just so much foo, but most similar 
commands have MSN-based variants and there's value on following the 
overall design. IMAP has too many odd rules already, omitting MSN-based 
move merely because I don't like and/or see no use for it would be wrong.

 > I know this was one of the dangers of re-using an old mailing

It's happened often enough on WG lists too.

Arnt

---

From: Brandon Long <[email-masked]>
Date: Mon, 6 Aug 2012 14:00:08 -0700

Does anyone want to write up a set of tests for the new functionality?

Either based on the Timo's existing test suite or something else.

Brandon

On Mon, Aug 6, 2012 at 1:20 AM, Arnt Gulbrandsen
<[email-masked]>wrote:

> Well. The recent change was: MSN-based move is now in the draft instead of
> being an open issue.
>
> Problems have been mentioned in the third person or past tense umpty times
> in the past decade, but when I tried to chase them down they've evaporated.
>
> commands have MSN-based variants and there's value on following the overall
> design. IMAP has too many odd rules already, omitting MSN-based move merely
> because I don't like and/or see no use for it would be wrong.
>
>
> ______________________________**_________________
> imapext mailing list
> [email-masked]
> https://www.ietf.org/mailman/**listinfo/imapext<https://www.ietf.org/mailman/listinfo/imapext>
>

---

From: Timo Sirainen <[email-masked]>
Date: Sun, 12 Aug 2012 01:55:27 +0300

> Either based on the Timo's existing test suite or something else.

I added a move test now to latest imaptest hg (requires new imaptest because it triggered a couple of imaptest bugs):
http://hg.dovecot.org/imaptest/file/tip/src/tests/move

The test works the way I think we decided how the SHOULDs would be in the next draft.

I also found a bug from my implementation with it: MOVE command didn't always send EXPUNGE before OK reply, because the code assumed that this command didn't change the source mailbox (which was true with COPY, but not with MOVE).

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 12 Aug 2012 09:11:49 +0200

> The test works the way I think we decided how the SHOULDs would be in the next draft.

It's posted.

Arnt

---

From: Jan Kundrát <[email-masked]>
Date: Thu, 16 Aug 2012 19:38:16 +0200

> MT-PRIORITY. CHUNKING. PIPELINING. MTRK.

Dear Ned,
thanks for your participation in this thread, and please accept my 
apologies for a late reply (it was vacation time for me). I'm happy to 
see someone who is signed under so many SMTP-related RFCs reviewing my 
proposal.

First of all, your biggest complaint is about the lack of extensibility 
that my proposal brings. I suspect this is due to a misunderstanding -- 
even though the ABNF does not contain any explicit notion of future 
extensions at this point, the command was designed from the very start 
with extensibility in mind -- see the discussion about further 
SENDMAIL=* capabilities which serve as a way to inform the client about 
presence of these extensions, and the submission-option grammar 
non-terminal which is intended to be easily extended by future proposals.

In basic IMAP, the protocol reserves the X<atom> commands for 
experimental use, but all of the extensions in the end ended up defining 
their own commands anyway. I could certainly have added a similar thing 
to the grammar, i.e. something like this:

submission-option /= sub-option-future-use

sub-option-future-use = identifier SP sub-option-any-data

identifier = atom
	;; MUST start with X or be registered in ... IANA registry

sub-option-any-data = atom / number /
                       ( "(" sub-option-any-data
                         1*(SP sub-option-any-data) ")" )

Quite frankly, this is not something which I find terribly clear, and 
I'm not a huge fan of adding grammar extensions without defining their 
meaning. At the same time, it illustrates the design idea pretty well, 
so I might add that in if that's the consensus.

The goal of the proposal from the very start was to create a command 
which accepts the following data as input:

* the identification of an existing IMAP message to send
* the identification of recipients, if desired
* a list of optional submission options

Where the list of "submission options" is designed to be extensible by 
future extensions, as they appear in ESMTP. The DSN example shows how 
such an "extension" will look like, and it was my intention to allow for 
similar additions when they prove to be worth the effort.

Now the second complaint is that each ESMTP extension will have to find 
its equivalent in this IMAP-SUBMIT protocol -- and that's right, but 
it's also a tautology. Any proposal which works in a different way than 
proxying an actual ESMTP session to an SMTP server *will* necessarily 
result in the need to re-create an extension for each relevant ESMTP 
addon. That's an inherent disadvantage of my proposal, yes.

The third complaint was that I do not address many issues which prompted 
creation of several existing ESMTP extensions. Let's review each of them 
separately, as they appear in [1], and let's see how they are relevant 
to the submission over IMAP. I suspect that I might be missing some 
obvious things, as SMTP is not my cup of tea -- which is why I'm open to 
feedback and would like to learn from my mistakes. I'll be grateful if 
you (or anyone else, for that matter) corrects my misunderstanding or 
point out flaws in my ideas.

- 8BITMIME tells the ESMTP clients that they can submit messages 
containing 8bit octets. You raise a good point in that the clients of 
the UID SUBMIT command SHOULD know whether the message body they're 
creating is actually allowed to contain 8bit data; that's something I 
missed in my proposal. Would it be enough if we mandate that the UID 
SUBMIT command MUST always accept such messages? Are there any MTAs 
(which could be expected to be installed along an IMAP server, i.e. no 
ancient versions of already deployed software) and which cannot support 
the 8bit MIME messages at the same time? Or shall we mandate another 
capability like SENDMAIL=8BITMIME?

- SIZE --- the IMAP server knows the message size, and if the "real 
submission" takes place over the ESMTP, it can use the actual size and 
check the ESMTP server's constraints prior to the message delivery. Do 
you want me to add an explicit machine-readable code 
MESSAGETOOBIGFORSUBMISSION to make it possible to report back this 
failure in a machine-readable way? Apart from that, I don't see any 
further action to be necessary here.

- ONEX -- I don't believe this one is relevant here.

- CHUNKING, BINARYMIME -- not applicable in the context of 
submission-over-IMAP; they're already handled by CATENATE anyway.

- CHECKPOINT -- the submission is "atomic" from the IMAP client's point 
of view already, and the amount of data being sent between the IMAP 
client and the server is minimal, so there's no point in being able to 
"restart" this command, IMHO.

- DELIVERBY -- if MUAs want this feature, it can be added in a future 
extension. The grammar change would be trivial. Or is it worth defining 
that already in the basic proposal?

- PIPELINING -- optimization of the ESMTP protocol. IMAP doesn't need 
that, pipelining is already supported. Just to make it absolutely clear, 
nothing prevents the IMAP servers from utilizing this extension *when 
delivering over ESMTP*, but at the same time, this is their own 
implementation detail which does not require any support in the 
submit-over-IMAP proposal.

- You're right in that the DSN extension is per-recipient and not 
per-message. This will require further work; I'll address that in the 
next version of the proposal. Thanks for letting me know! However, I 
believe that the ORCPT is not required for e-mail submission -- please 
correct me if I'm wrong. The ENVID and RET look like they should be 
supported -- opinions about this are welcome.

- ETRN, ENHANCEDSTATUSCODES, STARTTLS, NO-SOLICITING don't apply for 
mail submission over IMAP, AFAIK.

- MTRK can be added in future if there's interest.

- SUBMITTER -- I've skimmed over that experimental RFC and it appears to 
me that this might not necessarily apply in the context of outgoing 
e-mail submission. Comments welcome.

- ATRN, AUTH -- not applicable, AFAIK.

- BURL -- already handled by CATENATE, hence not needed.

- FUTURERELEASE -- can be supported by future extension if needed. Is 
that worth being defined in the basic protocol?

- UTF8SMTP -- would require extending the allowed domain of the 
addresses passing through this command. That's a material for future 
extension, IMHO, unless the consensus is that it shall be there from the 
very beginning. Shall it? I don't have an opinion here.

- CONPERM, CONNEG -- based on the complete non-existence of any IMAP 
server supporting the CONVERT extension, I seriously doubt this one will 
be ever needed. If there's a need for it, though, it should be easy to 
support it using the usual submit-option syntax in a future extension. 
As usual, comments welcome.

- MT-PRIORITY -- can be covered by a submit extension, if there's a need 
for it. Is it? I don't know, I'm honestly asking.

So all in all, it seems to me that the UID SUBMIT proposal is capable of 
accommodating all ESMTP extensions introduced so far. Yes, there's a 
certain amount of work required by "porting" the extension to this IMAP 
command, but the amount of required work seems to be negligible. There's 
also a question whether these options are really needed for 
client-originated mail submission -- I can easily write a document for 
it, and IMAP servers which use ESMTP can easily support that, but is it 
worth the effort? Would clients actually make use of, say, MTRK or 
MT-PRIORITY?

One of the goals I had in mind was to be able to implement this on the 
server-side using just the `sendmail`-compatible interface, like what 
Postfix offers. In its current version, it doesn't support anything 
besides DSN (and that on a per-message manner only), so that's what the 
current draft allows. I'll be happy to add more stuff to the list if 
that's the direction that people prefer, but I also don't prefer to 
design extensions which nobody would use anyway. In this mail, I just 
wanted to show that the proposed command in fact *can* accommodate the 
requirements of all of the existing ESMTP extensions.

Have I managed to change your mind about the value of this proposal?

With kind regards,
Jan

[1] http://www.iana.org/assignments/mail-parameters

-- 
Trojita, a fast e-mail client -- http://trojita.flaska.net/

---

From: Barry Leiba <[email-masked]>
Date: Sat, 25 Aug 2012 14:35:39 -0400

>> The test works the way I think we decided how the SHOULDs would be in the

The "it", here, being http://tools.ietf.org/html/draft-ietf-imapmove-command-01
Arnt posted that almost a month ago, and there have been no comments
since.  Does that mean it's ready to go?

Barry, doing a little AD poke here

---

From: Timo Sirainen <[email-masked]>
Date: Sat, 25 Aug 2012 21:38:30 +0300

> since.  Does that mean it's ready to go?

I think there were some comments after that that Arnt said he changed already? But basically yes, I'd say it's done.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 25 Aug 2012 21:41:20 +0200

Yes, I think it's done too.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 25 Aug 2012 21:44:03 +0200

I will be back at my desk in a week and able to attend to mail properly then. Feel free to issue a wglc any time and an lc after sep 3.

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 28 Aug 2012 14:08:43 +0100

> I think there were some comments after that that Arnt said he changed already? But basically yes, I'd say it's done.
As a chair I would like to see the final editor's version before the 
document exits the WG (i.e. before IETF LC). But I would be personally 
Ok with starting WGLC on the current version (I haven't consulted with 
Ned, so he might disagree.)

---

From: Ned Freed <[email-masked]>
Date: Tue, 28 Aug 2012 07:16:58 -0700 (PDT)

> Ned, so he might disagree.)

Wfm.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Aug 2012 09:09:39 +0200

I didn't have any diff when I left for my vacation, a far as I can remember. Maybe I confused my two drafts?

Start the wglc.

My vacation ends this week. Naturally there's a business trip next week (why does this have to happen to me?) but I will read mail.

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Wed, 29 Aug 2012 21:18:28 +0300

I think I was mainly thinking about the text for MOVE where source and destination mailbox are the same, but I see there is already some text in -01 for that:

>    Note that moving messages to the current mailbox is well-defined, so
>    that MOVE is defined for all the cases where the COPY/STORE/EXPUNGE
>    sequence is.

Although I don't think it's quite enough text for a reader to understand what this sentence is really talking about. At the very least "current" should be replaced with "selected" or maybe "selected mailbox itself". Even after that I'm not sure if the reader will realize what should happen in such situation..

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Aug 2012 23:28:28 +0200

That was one rotten paragraph. How about this?

"Note that as a consequence of the definition above, moving a message to 
the currently selected mailbox is legal, and has the effect of assigning 
a new UID for the message and setting its \recent pseudo-flag."

Arnt

---

From: Timo Sirainen <[email-masked]>
Date: Thu, 30 Aug 2012 00:33:09 +0300

On 30.8.2012, at 0.28, Arnt Gulbrandsen wrote:

> "Note that as a consequence of the definition above, moving a message to the currently selected mailbox is legal, and has the effect of assigning a new UID for the message and setting its \recent pseudo-flag."

I like it.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 29 Aug 2012 23:43:47 +0200

I replaced the upgefucked nonsense with that sentence. Will do the 
idnits chores and post when I come home, which should happen next week, 
but who knows which day.

Arnt

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 30 Aug 2012 22:18:34 +0100

On behalf of Ned and myself I would like to start 2 weeks WGLC (which will end on September 14th) draft-ietf-imapmove-command-01.txt.

Please send your comments to the mailing list or directly to Ned and myself. Statement of support (e.g. "this document is well written and ready for publication") or objections (with a reason) are also welcomed.

Alexey, as a co-chair.

---

From: Dan Karp <[email-masked]>
Date: Thu, 30 Aug 2012 14:50:31 -0700 (PDT)

> Please send your comments to the mailing list or directly to Ned and
> myself. Statement of support (e.g. "this document is well written
> and ready for publication") or objections (with a reason) are also
> welcomed.

The document defines UID MOVE in terms of three other commands
(UID COPY, UID STORE, UID EXPUNGE).  As for MOVE, it states

   The MOVE command performs the same actions UID MOVE.

Besides the missing "as", I'm not sure if this is exactly clear.
There's no sequence-set-parameterized EXPUNGE that corresponds to
UID MOVE.  I mean, I get what the document is saying, but do we
need to specify non-UID MOVE a little more concretely?

There are two section 4.3s.  Both the first one (UIDPLUS) and the
second one (QRESYNC) describes the extensions' interaction with
this draft, but they reference only UID MOVE.  Same with section 4.2
(ACL).  We should clarify all of these to state that MOVE commands
interact similarly.  Also, the UIDPLUS section has a typo ("Servers
authors").

The ABNF should probably be phrased similarly to that of RFC 3501.
Thus, instead of

       command-select =/ ["UID "] "MOVE" SP set SP mailbox

it should probably read

    command-select =/ move
    move            = "MOVE" SP sequence-set SP mailbox
    uid             = "UID" SP (copy / fetch / search / store / move)

(Note that even if you don't make this change, "set" needs to be
changed to "sequence-set".  Also, use SP instead of including the
space in the "UID " literal.)

- Dan