Subject: [art] Auto-configuring Email Clients via WebFinger

From: "Paul E. Jones" <[email-masked]>
Date: Mon, 15 Jul 2019 19:30:53 +0000

ART folks,

Several years ago when I was working on WebFinger, one of the use cases 
I presented was using WebFinger to facilitate auto-configuring email 
clients.  It was and still is a problem I deal with today.

For my own family, I have to manually configure several different 
clients on several different platforms for each member of the family.  
It's time consuming and really needs to be made simpler.

My wife also has to deal with this issue where she works, because her 
company, while just 100 or so employees, has offices in two different 
countries and the mail server settings an employee uses depends on his 
or her geographic location.  To use standard IETF protocols, it means a 
lot of manual provisioning.

I see the same sort of challenges with service providers. If one wants 
to have his or her own domain, but isn't technically savvy, they're in 
for a lot of "fun" trying to figure out the various settings. Seriously, 
no normal person should have to understand what SMTP or IMAP means, and 
definitely what port numbers or security settings to fill in.

While there has been a generic DNS-based method for email provision for 
a while, it doesn't work for me. It doesn't work for my wife's company, 
either. It also doesn't define everything one might need to define 
(e.g., required security settings or policies).

So we put together a very simple example to show how this might be done 
with WebFinger.  See the draft here:
https://tools.ietf.org/html/draft-jones-webfinger-email-autoconfig-00

The example in that document should make it clear how we intend this to 
work, though the detailed procedures and syntax are missing.  We first 
want to see what interest exists and if this general approach will work 
for everyone before getting into too much detail.

We'd love to have some dialog on this to see how we can address this 
problem of auto-configuring email clients.

Paul

---

From: "John Levine" <[email-masked]>
Date: 15 Jul 2019 22:34:05 -0400

>problem of auto-configuring email clients.

The Thunderbird MUA has has this kind of autoconfiguration for many
years, with the MUA fetching an XML config file.  I wouldn't say it's
perfect but I would say it's widely deployed.

See

https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration
https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 16 Jul 2019 04:13:03 +0000

John,

I use primarily two different email clients, and neither support that 
method.  One does support Microsoft's method:
https://docs.microsoft.com/en-us/Exchange/architecture/client-access/autodiscover?view=exchserver-2019

I have no strong preference for how we solve the problem, but I think we 
need a standard for this.  I also think we should take into 
consideration the privacy concerns that were raised when I first talked 
in context of WebFinger several years ago.  Specifically, people wanted 
a way to ensure access to the config file was protected.  (I do not 
recall now why people wanted to keep that private, but they did and were 
insistent on it.)

Paul

---

From: "John Levine" <[email-masked]>
Date: 16 Jul 2019 01:05:06 -0400

>recall now why people wanted to keep that private, but they did and were 

Probably to make it harder to enumerate e-mail addresses.  I don't have much
sympathy for that argument, since bad guys already have more addresses to
spam than they know what to do with, but it's not a big deal.

More concretely, it seems to me that it would be better to make this a
profile of the existing autoconfigure scheme, which some MUAs
implement, rather than webfinger, which no MUAs implement.

The Mozilla design (over-design, really) looks up the configuration
for [email-masked] at 
https://example.com/.well-known/autoconfig/mail/config-v1.1.xml or
https://autoconfig.example.com/mail/config-v1.1.xml?emailaddress=[email-masked]

On the theory that web servers generally ignore GET parameters that they don't
understand, we can look up

https://example.com/.well-known/autoconfig/mail/config-v1.1.xml?emailaddress=[email-masked]&password=swordfish

and it returns Alice's mail setup, possibly ignoring the address and password and returning a common
setup for all the mailboxes at example.com.  The password makes it harder for people who aren't
Alice to snoop on her mail setup.

This doesn't give you the extra Personal/Business level of
indirection, but I have no idea what it would mean to have two
different mail configurations for the same e-mail address.  I have
personal and business mail addresses, but they're different addresses
which I presumably know and configure either or both as needed.

R's,
John

---

From: "Bron Gondwana" <[email-masked]>
Date: Tue, 16 Jul 2019 15:22:56 +1000

On Tue, Jul 16, 2019, at 05:31, Paul E. Jones wrote:
> ART folks,
> 
> Several years ago when I was working on WebFinger, one of the use cases I presented was using WebFinger to facilitate auto-configuring email clients. It was and still is a problem I deal with today.
> 
> For my own family, I have to manually configure several different clients on several different platforms for each member of the family. It's time consuming and really needs to be made simpler.
> 
> My wife also has to deal with this issue where she works, because her company, while just 100 or so employees, has offices in two different countries and the mail server settings an employee uses depends on his or her geographic location. To use standard IETF protocols, it means a lot of manual provisioning.
> 
> I see the same sort of challenges with service providers. If one wants to have his or her own domain, but isn't technically savvy, they're in for a lot of "fun" trying to figure out the various settings. Seriously, no normal person should have to understand what SMTP or IMAP means, and definitely what port numbers or security settings to fill in.
> 
> While there has been a generic DNS-based method for email provision for a while, it doesn't work for me. It doesn't work for my wife's company, either. It also doesn't define everything one might need to define (e.g., required security settings or policies).
> 
> So we put together a very simple example to show how this might be done with WebFinger. See the draft here:

There's also been discussion about doing the same thing for caldav and carddav in CalConnect, which was led by Marten. It would be good to combine this work!

Cheers,

Bron.

-- 
 Bron Gondwana
 [email-masked]

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 16 Jul 2019 17:06:29 +0000

John,

The scheme you show is very similar to the WebFinger proposal.  
Essentially, the client queries a well-known address and is given a JSON 
document inside of which is the URL of the config file.  That config 
could easily be put into the initial query, but folks did not want to do 
that back then.  I do have an old example of how to do that, though, but 
it's not in this draft (since security concerns were expressed back 
then).

Another thing to consider is migration.  When a service provider needs 
to move a bunch of users from one server cluster to another, would it 
make more sense to write configs into the customer's /autoconfig/ 
directories or update config files they maintain separately?

Given the push-back I got about XML back then, too, it's probably best 
this is in JSON format (though, again, I have no strong preference).

I think a standard needs to exist so that client developers can adhere 
to something and I think it's unfortunate all we have a vendor-specific 
solutions.

Do others agree?  Or am I one of the few who has this frustration?  I 
suspect I'm not. :)

Paul

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 16 Jul 2019 17:31:57 +0000

Agreed.  These things should all be automated.

------ Original Message ------
From: "Bron Gondwana" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Cc: "Marten Gajda" <[email-masked]>
Sent: 7/16/2019 1:22:56 AM
Subject: Re: [art] Auto-configuring Email Clients via WebFinger

>>Several years ago when I was working on WebFinger, one of the use 
>>cases I presented was using WebFinger to facilitate auto-configuring 
>>email clients.  It was and still is a problem I deal with today.
>>
>>or her geographic location.  To use standard IETF protocols, it means 
>>a lot of manual provisioning.
>>
>>for a lot of "fun" trying to figure out the various settings. 
>>Seriously, no normal person should have to understand what SMTP or 
>>IMAP means, and definitely what port numbers or security settings to 
>>fill in.
>>
>>While there has been a generic DNS-based method for email provision 
>>for a while, it doesn't work for me. It doesn't work for my wife's 
>>company, either. It also doesn't define everything one might need to 
>>define (e.g., required security settings or policies).
>>
>There's also been discussion about doing the same thing for caldav and 
>carddav in CalConnect, which was led by Marten. It would be good to 
>combine this work!
>
>Cheers,
>
>Bron.
>
>--
>   Bron Gondwana
>   [email-masked]
>
>

---

From: Larry Masinter <[email-masked]>
Date: Tue, 16 Jul 2019 10:45:56 -0700

> Putting passwords in cleartext in the request uri is ill advised because many servers keep logs of requests, caches, etc

---

From: "John R Levine" <[email-masked]>
Date: 16 Jul 2019 15:31:34 -0400

> The scheme you show is very similar to the WebFinger proposal.

Yes, I realize that.  But my point is that the scheme I show actually 
exists, whereas webfinger in MUAs does not.

> Another thing to consider is migration.  When a service provider needs to 
> move a bunch of users from one server cluster to another, would it make more 
> sense to write configs into the customer's /autoconfig/ directories or update 
> config files they maintain separately?

Neither, I'd say.  Any provider big enough for that to be a problem should 
be able to generate the config files on the fly from their main database 
as requested.  I presume you know that slashes in a URL need not have any 
connection to a file system on a server.

> Given the push-back I got about XML back then, too, it's probably best this 
> is in JSON format (though, again, I have no strong preference).

I don't care either, except that the MUA code that exists now uses XML.

R's,
John

>> On the theory that web servers generally ignore GET parameters that they 
>> don't understand, we can look up
>>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 16 Jul 2019 21:46:42 +0200

John Levine:
> Yes, I realize that.  But my point is that the scheme I show 
> actually exists, whereas webfinger in MUAs does not.

And it has a little over ten years of deployment history, so it works for 
quite a lot of users now. Anything that's functionally similar but not 
compatible (from the viewpoint of domain owners) will have to work hard to 
gain that deployment.

Arnt

---

From: "Paul E. Jones" <[email-masked]>
Date: Tue, 16 Jul 2019 19:46:50 +0000

John,

Other than Thunderbird, what clients support Mozilla's procedures?

I really don't care about any existing proprietary solutions out there, 
as they're clearly doing me no good at all.

I'd prefer we define something simple that people will adopt.  WebFinger 
is just an HTTP query, but if we wanted to have a different HTTP query, 
that's fine.  I just think we need to document something in an RFC, else 
nothing will ever get universally adopted.

Paul

------ Original Message ------
From: "John R Levine" <[email-masked]>
To: "Paul E. Jones" <[email-masked]>
Cc: "John Levine" <[email-masked]>; [email-masked]
Sent: 7/16/2019 3:31:34 PM
Subject: Re: [art] Auto-configuring Email Clients via WebFinger

>Yes, I realize that.  But my point is that the scheme I show actually exists, whereas webfinger in MUAs does not.
>
>>Another thing to consider is migration.  When a service provider needs to move a bunch of users from one server cluster to another, would it make more sense to write configs into the customer's /autoconfig/ directories or update config files they maintain separately?
>
>Neither, I'd say.  Any provider big enough for that to be a problem should be able to generate the config files on the fly from their main database as requested.  I presume you know that slashes in a URL need not have any connection to a file system on a server.
>
>>Given the push-back I got about XML back then, too, it's probably best this is in JSON format (though, again, I have no strong preference).
>
>>>On the theory that web servers generally ignore GET parameters that they don't understand, we can look up
>>>

---

From: "John R Levine" <[email-masked]>
Date: 16 Jul 2019 15:56:35 -0400

> Other than Thunderbird, what clients support Mozilla's procedures?

I don't know.  But it exists, it works, and it is available to the public.

> I really don't care about any existing proprietary solutions out there, as 
> they're clearly doing me no good at all.

For one thing, Thunderbird is MPL licensed open source, so the design and 
the code that implements it are available to anyone who wants to use them. 
This is an odd definition of "proprietary."

Also, "It doesn't work in my MUA" is not much of an argument.  It's 
unlikely that my MUA (alpine) will implement whatever webfinger thing you 
propose so by that arguement we're all wasting our time.

> I'd prefer we define something simple that people will adopt.

Not to repeat myself, but the Mozilla thing exists, is implemented in at 
least one fairly popular MUA, and has code available.

I suppose I could talk to people at Microsoft and Google and see if they'd 
be interested if we came up with something.  What's your plan?

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "John Levine" <[email-masked]>
Date: 16 Jul 2019 18:05:19 -0400

In article <[email-masked]> you write:
>-=-=-=-=-=-

Good point.

Plan A: if the server needs the user name and password, it sends back
a 401 response and the MUA sends them via RFC 7617 basic
authentication.  That seems largely backward compatible and invents a
minimum of new stuff.

Plan B: do it as a POST like most web password entries.  I realize
this has backward compatibility issues with the current GET.

---

From: Austin Wright <[email-masked]>
Date: Tue, 16 Jul 2019 20:00:25 -0700

> On Jul 16, 2019, at 15:05, John Levine <[email-masked]> wrote:
> 

What are the risks if the mail server and HTTP server are maintained by different authorities?

Cheers,

Austin Wright.

---

From: "John Levine" <[email-masked]>
Date: 17 Jul 2019 11:41:33 -0400

>What are the risks if the mail server and HTTP server are maintained by different authorities?

I suppose there might be administrative issues giving the web people
access to the mail passwords but that's hardly a new issue.  I see
that Apache has a module that does authentication via LDAP, which
would often be enough to solve that particular problem.

One of the reasons Mozilla checks https://autoconfig.<domain>/ is exactly this:
it lets the mail department run a small special purpose web server separate from
the main web servers.

---

From: Steffen Nurpmeso <[email-masked]>
Date: Thu, 18 Jul 2019 00:31:14 +0200

John Levine wrote in <[email-masked]>:
 |In article <[email-masked]> you write:
 |>>>> Putting passwords in cleartext in the request uri is ill advised \
 |>>>> because many servers keep logs of requests, caches, etc
 |>> 
 |>> Good point.
 |>> 
 |>> Plan A: if the server needs the user name and password, it sends back
 |>> a 401 response and the MUA sends them via RFC 7617 basic
 |>> authentication.  That seems largely backward compatible and invents a
 |>> minimum of new stuff.

Just yesterday i (currently implementing OAUTHBEARER for a BSD
Mail clone) thought how easy it would be if the OAUTHBEARER
request would have additional fields, like refresh-token etc.
A lot of things, including password changes etc., could be done in
there, protected via TLS, and managable by the mail protocol
itself (SMTP, POP3, IMAP), simply by putting value in data
following (on) the response code (line), instead of using a web
browser.

 |>> Plan B: do it as a POST like most web password entries.  I realize
 |>> this has backward compatibility issues with the current GET.
 |>
 |>What are the risks if the mail server and HTTP server are maintained \
 |>by different authorities?
 |
 |I suppose there might be administrative issues giving the web people
 |access to the mail passwords but that's hardly a new issue.  I see
 |that Apache has a module that does authentication via LDAP, which
 |would often be enough to solve that particular problem.
 |
 |One of the reasons Mozilla checks https://autoconfig.<domain>/ is exactly \
 |this:
 |it lets the mail department run a small special purpose web server \
 |separate from
 |the main web servers.

I will never understand where the benefit of turning mail clients
into fully blown browsers actually is.  I know in the Microsoft
world they have sophisticated integration of everything and a XML
protocol for that (several years ago i had contact with a person
from Microsoft who pointed me to this infrastructure, i was
impressed but since have forgotten most; it was not the one who
had a soapy bath on a stage).  This is actually hard to do with nc
/ telnet.  I mean, i am currently doing OAUTHBEARER: where is the
benefit (compared to kinit / kdestroy, which i have local control
of)?  I really have to go over that immense amount of code that
a browser is, and possibly the tab (even if i open an explicit
window) shares runtime with other tabs, which possibly have
sourced in megabytes of Javascript from who knows where, the most
weird seem to come in via reading newspapers.  I seem to know
Chromium at least optionally separates that better
(--site-per-process?).

Are you trying to enforce code reuse of anyway linked-in web kits,
otherwise i do not know.  If that is new code then it needs to be
written anyway.  Mail protocols offer plenty of possibilities to
send passwords, and could easily become extended with some plain
text commands.  It is all secured via TLS now, everywhere.
Extending usage of client certificates in favour of other
credentials is also interesting, moving away authentication from
the protocol to the transport, and reducing round trips.  And POP3
could need a SYNC for enforcing pending deletes to be fixated, and
i think a body only receive command would be nice too, also.

In the end i will have to implement actions aka "clickable" URLs
for a text-mode mail reader.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

---

From: "Paul E. Jones" <[email-masked]>
Date: Thu, 18 Jul 2019 03:14:29 +0000

Austin,

>What are the risks if the mail server and HTTP server are maintained by different authorities?

I expect they would be in some cases, such as service providers that 
sell simple web hosting services. That's another reason why people 
several years ago suggested that we separate the initial inquiry from 
the follow-up that actually returns the mail configuration data. That's 
how the example is presented in my draft.

Paul

---

From: "John R Levine" <[email-masked]>
Date: 18 Jul 2019 00:23:56 -0400

On Thu, 18 Jul 2019, Paul E. Jones wrote:
> I expect they would be in some cases, such as service providers that sell 
> simple web hosting services. That's another reason why people several years 
> ago suggested that we separate the initial inquiry from the follow-up that 
> actually returns the mail configuration data. That's how the example is 
> presented in my draft.

But in your draft, the initial response is often specific to the 
particular address, e.g., it has a personal and business version.  How 
does that help?  It still needs to know something about each address.

Regards,

---

From: "John Levine" <[email-masked]>
Date: 18 Jul 2019 01:17:55 -0400

>request would have additional fields, like refresh-token etc.

I believe you, but as I may have said once or twice before, my
proposal is intended to be a compatible extension of the autoconfig
that the most popular open source MUA already has.

> |separate from the main web servers.
>
>into fully blown browsers actually is.

I don't see how this follows.  In both of these proposals, the MUA makes
a few HTTP requests and then gets a blob of XML or JSON.  That skips the 95%
of browsers that render HTML.  (We'll ignore the detail that modern MUAs all
have that 95% anyway to render HTML mail.)

>otherwise i do not know.

No, I'm trying to make it backward compatible with existing MUAs.

>for a text-mode mail reader.

I don't understand that at all.  Why would you need to do that?

R's,
John

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 18 Jul 2019 08:33:44 +0200

> I don't understand that at all.  Why would you need to do that?

It sounds as if Steffen hasn't looked at Thunderbird autoconfiguration, at 
all, and is arguing that something else would be bad.

BTW, Thunderbird is unlikely to be the only client that uses its scheme. 
The relevant stack overflow questions overwhelmingly point to the 
Thunderbird scheme.

https://stackoverflow.com/search?q=%5Bimap%5D+autoconfiguration
https://stackoverflow.com/search?q=%5Bmail%5D+autoconfiguration

Arnt

---

From: Dave Cridland <[email-masked]>
Date: Thu, 18 Jul 2019 09:40:41 +0100

Obviously I'd prefer ACAP.

But also, please note the existence of RFC 6186. It's not widely deployed,
but if we were to open this can of worms, and not simply reuse
Thunderbird's autoconfig XML format (which, if I recall, is also supported
by various other Open Source MUAs), then I'd be keen to get SRV up and
running properly.

Dave.

On Mon, 15 Jul 2019 at 20:31, Paul E. Jones <paulej=
[email-masked]> wrote:

> ART folks,
>
> Several years ago when I was working on WebFinger, one of the use cases I
> presented was using WebFinger to facilitate auto-configuring email
> For my own family, I have to manually configure several different clients
> on several different platforms for each member of the family.  It's time
> consuming and really needs to be made simpler.
>
> countries and the mail server settings an employee uses depends on his or
> her geographic location.  To use standard IETF protocols, it means a lot of
> manual provisioning.
>
> I see the same sort of challenges with service providers. If one wants to
> have his or her own domain, but isn't technically savvy, they're in for a
> lot of "fun" trying to figure out the various settings. Seriously, no
> normal person should have to understand what SMTP or IMAP means, and
> While there has been a generic DNS-based method for email provision for a
> while, it doesn't work for me. It doesn't work for my wife's company,
> either. It also doesn't define everything one might need to define (e.g.,
> required security settings or policies).
>
> work, though the detailed procedures and syntax are missing.  We first want
> to see what interest exists and if this general approach will work for
> everyone before getting into too much detail.
>
> _______________________________________________
> art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/art
>

---

From: "Paul E. Jones" <[email-masked]>
Date: Thu, 18 Jul 2019 10:12:04 -0400

Dave,

RFC 6186 is the DNS-based one to which I referred below. Unfortunately, it doesn't allow setting various configuration options and wouldn't work at all for my wife's use case where employees use different mail servers based on the country in which they reside.

Paul

-------- Original Message --------
From: Dave Cridland <[email-masked]>
Sent: July 18, 2019 4:40:41 AM EDT
To: "Paul E. Jones" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [art] Auto-configuring Email Clients via WebFinger

Dave.

---

From: "Paul E. Jones" <[email-masked]>
Date: Thu, 18 Jul 2019 10:40:05 -0400

John,

The first query in my draft is the WebFinger one where the client would receive a list of URLs to then query to get various mail server configurations.

Using my wife's use case, the web server is maintained by a different team than the mail server. The only thing the web server team needs to know is where to direct the client for email configuration for each user, thus one static entry in WebFinger.  I'd expect the team managing mail server to deal with the actual mail configuration data and any authentication requirements.

The example in the draft with two addresses is to show that a single account identifier could have multiple named email accounts behind it. I suspect that would be unusual, but the possibility exists. If people don't want that flexibility, we could require that only one link relation be returned of the mail configuration type.

Does that help clarify my point?

Paul

-------- Original Message --------
From: John R Levine <[email-masked]>
Sent: July 18, 2019 12:23:56 AM EDT
To: "Paul E. Jones" <[email-masked]>
Cc: [email-masked]
Subject: Re: [art] Auto-configuring Email Clients via WebFinger

Regards,

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 18 Jul 2019 12:02:53 -0400

Responding to Paul's point about his wife needing a country specific
configuration, I think we need to think about this a bit differently.

This is one example of an application configuration problem and a service
discovery problem. What we need here is a scheme that is capable of
supporting the general case.

It makes no sense for any of the providers to change their existing schemes
unless it is to move to a scheme that is more powerful because it is more
general. Which means that at minimum it is supported by all the MUAs but
ideally means it is supported by multiple applications as well.

I have been thinking about this problem from the other side. I started what
is now the Mathematical Mesh trying to work out how to automatically
configure and maintain S/MIME and OpenPGP. While doing that, I suddenly
realized that I could add in the SMTP configuration as well. So once Alice
has configured her email service on one device, it is configured for them
all. She will get her SUBMIT/IMAP/POP3 settings at the same time that she
gets the S/MIME and OpenPGP keys to work on the device. I showed this to
Dave Clark who asked for SSH as well (which is a real pig to configure as
there is a bootstrap problem and a foul up may require someone to take a
plane trip to a server to fix it).

Being able to replicate the config across devices does not solve the
problem of initial acquisition of course. But the replication mechanism
should certainly use the same data structure (though translated to JSON
probably).

I have not thought about the Enterprise application of the Mesh in detail
yet because I have to get it working for individual users first. But I am
pretty sure that what we want here is single point onboarding (and
terminations).

So Alice joins example.com. She is issued a (new) company laptop and BYOB's
her mobile.

The onboarder checks the boxes for the corporate applications she is
authorized for, email, ssh, etc. and the group encryption services for the
teams she belongs to. The configulator then does all the rest.

All that remains is to connect up Alice and her devices to these
configurations. Which can be done in person via QR code or remotely via a
Base32 fingerprint. It presents a workfactor of at least 2^112 either way.

So from Alice's point of view, when she goes to the badge office to pick up
her badge, she scans a QR code with her phone and that is it - she is all
configured for email, chat, slack, slick, ssh, S/MIME and all the encrypted
data she needs from the cloud. If she is working on some really sensitive
stuff, her line manager or a VP might need to meet her in person to
authorize adding her to certain groups.

Termination should mean no more than unchecking the boxes.

The cryptography I am using was discovered by Torben Pedersen and Matt
Blaze in the mid 1990s. The reason it hasn't been applied was that it was
too hard to use and until Robert Snowden came along, people thought the
administrator of the key service in the cloud could be trusted. So I have
made it really easy to use and administrate even in the case that we have
separation of duties at the administrator level as well.

On Thu, Jul 18, 2019 at 10:12 AM Paul E. Jones <paulej=
[email-masked]> wrote:

> Dave,
>
> RFC 6186 is the DNS-based one to which I referred below. Unfortunately, it
> doesn't allow setting various configuration options and wouldn't work at
> all for my wife's use case where employees use different mail servers based
> on the country in which they reside.
>
> Paul
>
> ------------------------------
> *From:* Dave Cridland <[email-masked]>
> *Sent:* July 18, 2019 4:40:41 AM EDT
> *To:* "Paul E. Jones" <[email-masked]>
> *Cc:* "[email-masked]" <[email-masked]>, "[email-masked]" <
> [email-masked]>
> *Subject:* Re: [art] Auto-configuring Email Clients via WebFinger
>

---

From: "John R Levine" <[email-masked]>
Date: 18 Jul 2019 14:16:26 -0400

> Using my wife's use case, the web server is maintained by a different team than the mail server. The only thing the web server team needs to know is where to direct the client for email configurati…

As you say, it needs info for each user.  I'd think that would be a 
problem in a place where the web and mail departments don't like to 
talk to each other.

> The example in the draft with two addresses is to show that a single 
> account identifier could have multiple named email accounts behind it. I 
> suspect that would be unusual, but the possibility exists. If people 
> don't want that flexibility, we could require that only one link 
> relation be returned of the mail configuration type.

> Does that help clarify my point?

Not really.  What is an "account identifier" if it's not an e-mail 
address, and why would an MUA need to know about it?

R's,
John

---

From: Steffen Nurpmeso <[email-masked]>
Date: Thu, 18 Jul 2019 20:46:32 +0200

Arnt Gulbrandsen wrote in <9c6cde19-be14-4d13-9194-eff01770a61e@gulbrand\
sen.priv.no>:
 |On Thursday 18 July 2019 07:17:55 CEST, John Levine wrote:
 |>> In the end i will have to implement actions aka "clickable" URLs
 |>> for a text-mode mail reader.
 |>
 |> I don't understand that at all.  Why would you need to do that?
 |
 |It sounds as if Steffen hasn't looked at Thunderbird autoconfiguration, at 
 |all, and is arguing that something else would be bad.
 |
 |BTW, Thunderbird is unlikely to be the only client that uses its scheme. 
 |The relevant stack overflow questions overwhelmingly point to the 
 |Thunderbird scheme.
 |
 |https://stackoverflow.com/search?q=%5Bimap%5D+autoconfiguration
 |https://stackoverflow.com/search?q=%5Bmail%5D+autoconfiguration

The final pages [1] and [2] would benefit from an update, to me
direct TLS connection via POP3S/SUBMISSIONS etc. are not
"old-style", but i personally prefer them over STARTTLS/STLS etc.
Also "remove the following and leave to client/user?" and "<!--
not yet supported -->" i would like to know the final outcome of
before possibly writing code and having the need to start linking
against expat in order to get this stuff right.

I personally fail to see the value in not making that a plain
UTF-8-by-definition text file in Windows resource file format,
maybe [section.subsection] or [section/subsection] so to group the
key/value pairs, and # comments.  It has to be XML / JSON / CBOR.
(I would go for the latter two if plain text is not feasible.
That is a ~500 line C code parser at maximum, which does not count
for such a feature, especially in a codebase like Thunderbird.)

  Please support authentication with CRAM-MD5. It is simple to
  implement, and to set up, and you can still use RADIUS or
  a database that stores passwords in plaintext, so you don't need
  ...

  As an ISP, you should ideally store passwords in encrypted
  format, which removes the risk of mass password theft (and
  possibly reuse on other sites) if somebody hacks your
  servers. You can still support plaintext passwords in this case,

It is often wise to reiterate things in public!
Please do not store the passwords on a server that is exposed to
the internet, get it from an internal server via TCP/TLS or
whatever; maybe the internal service could require a client
certificate before responding, that would be nice.  (I am not an
administrator, i really lack that much of knowledge for being
a good one even, but i became a bit frightened.)

  [1] https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration/FileFormat/HowTo
  [2] https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat

So many things are moving to .well-known now.  If this here
passes, then maybe S/MIME certificates via .well-known are not
that far away, too.  (My German passport does not offer any keys
or certificates for digital credentials, which is very sad.)

Personally i do not understand why this cannot be in DNS.  I have
seen SRV being brought up, and isn't that exactly the information
that DNS is designed for?

And all the best to your wife, Mr. Cridland!

--steffen
|

---

From: Steffen Nurpmeso <[email-masked]>
Date: Thu, 18 Jul 2019 21:05:59 +0200

I did not want to be unfriendly by not responding to you!

John Levine wrote in <[email-masked]>:
 |In article <[email-masked]> you write:
 |>|>> Plan A: if the server needs the user name and password, it sends back
 |>|>> a 401 response and the MUA sends them via RFC 7617 basic
 |>|>> authentication.  That seems largely backward compatible and invents a
 |>|>> minimum of new stuff.
 |>
 |>Just yesterday i (currently implementing OAUTHBEARER for a BSD
 |>Mail clone) thought how easy it would be if the OAUTHBEARER
 |>request would have additional fields, like refresh-token etc.
 |
 |I believe you, but as I may have said once or twice before, my
 |proposal is intended to be a compatible extension of the autoconfig
 |that the most popular open source MUA already has.

The pages which describe the feature seem to have been patched
together wildly over the years.  As they are now they do not put
confidence in what should really be done (to an extend).

 |>|One of the reasons Mozilla checks https://autoconfig.<domain>/ is \
 |>|exactly \
 |>|this:
 |>|it lets the mail department run a small special purpose web server \
 |>|separate from the main web servers.
 |>
 |>I will never understand where the benefit of turning mail clients
 |>into fully blown browsers actually is.
 |
 |I don't see how this follows.  In both of these proposals, the MUA makes
 |a few HTTP requests and then gets a blob of XML or JSON.  That skips \
 |the 95%
 |of browsers that render HTML.  (We'll ignore the detail that modern \
 |MUAs all
 |have that 95% anyway to render HTML mail.)

Ok, yes, simply talking a minimal set of HTTP in order to get
automatic configuration would really be easy to do; parsing the
answers could require an additional library, for the MUA
i maintain.  I personally would prefer seeing this in the DNS, but
since everything seems to move into .well-known, this is a target
as valid as everything else.

 |>Are you trying to enforce code reuse of anyway linked-in web kits,
 |>otherwise i do not know.
 |
 |No, I'm trying to make it backward compatible with existing MUAs.
 |
 |>In the end i will have to implement actions aka "clickable" URLs
 |>for a text-mode mail reader.
 |
 |I don't understand that at all.  Why would you need to do that?

You are of course right, this does not follow at all.
And, for one i definitely need to find a solution for such anyway;
and second the ConfigFileFormat Wiki indeed mentions form input
fields, except that this is declared "not yet supported" in an
accompanying comment.  (But the latter could simply be prompts in
a console based UI, of course.)

--steffen
|

---

From: Marten Gajda <[email-masked]>
Date: Fri, 19 Jul 2019 00:16:33 +0200

As a client developer, I want to provide a smooth UX to our users. This
means I need much more information than just a server URL.

For instance, I also want to

* present the name or logo of the provider which hosts the user's
account, ideally before they enter their credentials,
* provide links to support channels, reset password pages, account
management,
* know the OAuth endpoints, if available,
* know the OpenID Connect endpoints, so I can register my OAuth client
dynamically,
* know the OAuth scope tokens I need in order to access a service,
* know which endpoints serve the same data via different protocols,
* know *all* the available services (not just the ones I know about), so
I can suggest other clients which may be useful.

We had a few more discussion items at
https://github.com/CalConnect/AUTODISCOVERY/issues

One of the initial ideas was to have the accounts managed by the
operating system and data access/synchronization handled by individual
applications. In this scenario, the operating system would provide the
UI to configure an account and perform the service discovery. Afterwards
it would delegate the service configuration along with the account
identifier to the applications which can handle the services found. If
the provider offers services for which no application is present on the
system, it could suggest suitable applications from the app store or
software repository. The operating system could rerun the service
discovery from time to time and notify the user about changes (e.g.
"Your provider XYZ now supports chatting, here are the applications you
can use: ...").

I don't see this as a use case for WebFinger though. The services
offered by a provider is not user data (although they may vary for each
users), it's primarily provider data. As such, I'd prefer to provide a
single JSON document at a well-known URL under the provider's domain,
e.g. at "/.well-known/services". This would make it trivial to host a
static document, which would do the trick in most setups. More complex
providers could still return per-user configurations (after authentication).

The attached example shows how a document might look like as per the
current status.

Cheers,

Marten

Am 16.07.19 um 07:22 schrieb Bron Gondwana:
>> countries and the mail server settings an employee uses depends on
>> his or her geographic location.  To use standard IETF protocols, it
>> means a lot of manual provisioning.
>>
>> I see the same sort of challenges with service providers. If one
>> wants to have his or her own domain, but isn't technically savvy,
>> they're in for a lot of "fun" trying to figure out the various
>> settings. Seriously, no normal person should have to understand what
>> SMTP or IMAP means, and definitely what port numbers or security
>> settings to fill in.
>>
-- 
Marten Gajda
CEO

dmfs GmbH
Schandauer Straße 34
01309 Dresden
GERMANY

phone: +49 177 4427167
email: [email-masked]

Managing Director: Marten Gajda
Registered address: Dresden
Registered No.: AG Dresden HRB 34881
VAT Reg. No.: DE303248743

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 19 Jul 2019 00:41:02 +0000

John,

>As you say, it needs info for each user.  I'd think that would be a problem in a place where the web and mail departments don't like to talk to each other.

I did not mean to suggest they will not talk, but it's always best to 
minimize touch points, as coordination can be difficult. I've found that 
particularly true with larger companies, but I suspect that's why so 
many just use Outlook and Exchange.

>Not really.  What is an "account identifier" if it's not an e-mail address, and why would an MUA need to know about it?

That's a WebFinger thing. When querying a WebFinger server, you ask it 
"tell me about acct:[email-masked]", for example. The account 
identifier looks like an email address, but doesn't fully conform to the 
syntax of an email address. It would for 99% of the users out there, I 
suspect, but somebody somewhere might still use some UUCP-style 
addresses. WebFinger would still work for that, as the account ID would 
be different from the email address. That's why I put the email address 
into the mail server config JSON document.

Paul

---

From: "Paul E. Jones" <[email-masked]>
Date: Fri, 19 Jul 2019 01:00:34 +0000

Marten,

It doesn't even require much complexity before we need user-specific 
data. I hate mentioning my wife's use-case again, but it is valid and I 
suspect her small company isn't the only one.  Some users use services 
on one SMTP and IMAP server in one country, and other users use those 
services in another country.  The data is stored in those respective 
countries, too, and never moved between countries, with the exception 
that the mail server for all incoming mail will forward to the server in 
the appropriate geography for the user.

If you're suggesting that the initial query for the "services.json" file 
may prompt for credentials and allow to server different JSON documents 
based on the user requesting, good. That would work.

Paul

------ Original Message ------
From: "Marten Gajda" <[email-masked]>
To: "[email-masked]" <[email-masked]>
Sent: 7/18/2019 6:16:33 PM
Subject: Re: [art] Auto-configuring Email Clients via WebFinger

>* know *all* the available services (not just the ones I know about), 
>so I can suggest other clients which may be useful.
>
>UI to configure an account and perform the service discovery. 
>Afterwards it would delegate the service configuration along with the 
>account identifier to the applications which can handle the services 
>found. If the provider offers services for which no application is 
>present on the system, it could suggest suitable applications from the 
>app store or software repository. The operating system could rerun the 
>service discovery from time to time and notify the user about changes 
>(e.g. "Your proviare the applications you can use: ...").
>

---

From: "John R Levine" <[email-masked]>
Date: 18 Jul 2019 21:48:29 -0400

> particularly true with larger companies, but I suspect that's why so many 
> just use Outlook and Exchange.

Yes indeed.  That's why the Mozilla design lets the mail people run their 
own server so they don't have to talk at all if they don't want to.

> email address. It would for 99% of the users out there, I suspect, but 
> somebody somewhere might still use some UUCP-style addresses.

As an old uucp user (ima!johnl) I can assure you that UUCP and every other 
non RFC 822 mail address scheme is dead dead dead.  There may be some uucp 
transport lurking but there's no bangs in the addresses.

Regards,

---

From: Marten Gajda <[email-masked]>
Date: Fri, 19 Jul 2019 09:24:48 +0200

Am 19.07.19 um 03:00 schrieb Paul E. Jones:
>
> If you're suggesting that the initial query for the "services.json"
> file may prompt for credentials and allow to server different JSON
> documents based on the user requesting, good. That would work.

Yes, that's exactly what I'm suggesting. In fact, the driver behind the
original draft was a company which had a setup similar to your wife's
company. They had offices in the US and in Europe and their employee's
user accounts where hosted in their respective locations.

We need to bootstrap authentication though (e.g. to support OAuth and
provide links to support and account management sites). My suggestion
would be to serve a redacted services.json to unauthenticated users
which doesn't contain user specific service entries and add a flag
indicating that you need to authenticate to get the entire set of data.

Marten

> From: "Marten Gajda" <[email-masked] <mailto:[email-masked]>>
> To: "[email-masked]" <[email-masked] <mailto:[email-masked]>>
>> As a client developer, I want to provide a smooth UX to our users.
>> This means I need much more information than just a server URL.
>>
>> * know the OpenID Connect endpoints, so I can register my OAuth
>> client dynamically,
>> operating system and data access/synchronization handled by
>> individual applications. In this scenario, the operating system would
>> provide the UI to configure an account and perform the service
>> discovery. Afterwards it would delegate the service configuration
>> along with the account identifier to the applications which can
>> handle the services found. If the provider offers services for which
>> no application is present on the system, it could suggest suitable
>> applications from the app store or software repository. The operating
>> system could rerun the service discovery from time to time and notify
>> the user about changes (e.g. "Your proviare the applications you can
>> use: ...").
>>
>> offered by a provider is not user data (although they may vary for
>> each users), it's primarily provider data. As such, I'd prefer to
>> provide a single JSON document at a well-known URL under the
>> provider's domain, e.g. at "/.well-known/services". This would make
>> it trivial to host a static document, which would do the trick in
>> most setups. More complex providers could still return per-user
>> configurations (after authentication).
>>
>>>> cases I presented was using WebFinger to facilitate
>>>> auto-configuring email clients.  It was and still is a problem I
>>>> deal with today.
>>>>
>>>> clients on several different platforms for each member of the
>>>> family.  It's time consuming and really needs to be made simpler.
>>>>
>>>> My wife also has to deal with this issue where she works, because
>>>> her company, while just 100 or so employees, has offices in two
>>>> different countries and the mail server settings an employee uses
>>>> depends on his or her geographic location.  To use standard IETF
>>>> protocols, it means a lot of manual provisioning.
>>>>
>>>> settings. Seriously, no normal person should have to understand
>>>> what SMTP or IMAP means, and definitely what port numbers or
>>>> security settings to fill in.
>>>>
>>>> company, either. It also doesn't define everything one might need
>>>> to define (e.g., required security settings or policies).
>>>>
>>> There's also been discussion about doing the same thing for caldav
>>> and carddav in CalConnect, which was led by Marten. It would be good
>>> to combine this work!
>>>
>>> Cheers,
>>>
>>> Bron.
>>>
>>> -- 
>>>   Bron Gondwana
>>>   [email-masked]
>>>
>>>
>> -- 
>> Marten Gajda
>> CEO
>>
>> dmfs GmbH
>> Schandauer Straße 34
>> 01309 Dresden
>> GERMANY
>>
>> phone: +49 177 4427167
>> email: [email-masked]
>>

-- 
Marten Gajda
CEO

dmfs GmbH
Schandauer Straße 34
01309 Dresden
GERMANY

phone: +49 177 4427167
email: [email-masked]

---

From: Dave Cridland <[email-masked]>
Date: Fri, 19 Jul 2019 09:13:11 +0100

> transport lurking but there's no bangs in the addresses.
Also, while most acct URIs look like traditional email addresses, the
scheme definition itself allows for the more exciting variants of 822
addresses, like MIXER.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Fri, 19 Jul 2019 09:20:50 +0100

> on the country in which they reside.

I'm not saying that additional information isn't needed, but it'd be nice
to be able to use SRV alongside.

I'd expect SRV to be used on every connection, whereas an auto
configuration file would be used less, potentially only on initial
provision.

The Thunderbird config would need to be adapted anyway; it's not easily
extensible, due to not using XML namespaces, for one thing.

Dave.

---

From: Steffen Nurpmeso <[email-masked]>
Date: Fri, 19 Jul 2019 15:33:20 +0200

Marten Gajda wrote in <[email-masked]>:
 |As a client developer, I want to provide a smooth UX to our users. \
 |This means I need much more information than just a server URL.

You want to present them a barcode that they can scan with their
smart phone and which is made available via some local registry to
any interested application (via some kind of IPC, say DBUS), then.

  (Please take into account that white-race-first-world develops
  diabetes in teenage age, overweight, a massive front of teenage
  allergies, increasing antibiotic resistencies due to misuse,
  drug misuse in general (shouldn't be buried even, only as
  special waste, that sort of), and decreased fertility, and much,
  much more.  Is this related; maybe not.)

 |For instance, I also want to 
 |
 |* present the name or logo of the provider which hosts the user's account, \
 |ideally before they enter their credentials,

You made a contract with a provider, and it has given you
credentials to use.  The provider has an IP address, and you have
submissions, pop3s and imaps access.  On the web site's welcome page
the services URLs were noted.  (Let's say imap.XY.Z.)

 |* provide links to support channels, reset password pages, account \
 |management,

Ha!  If it were that easy to find these pages on the web site of
your provider!!  Shiny happy people there will be, i bet.

 |* know the OAuth endpoints, if available,
 |* know the OpenID Connect endpoints, so I can register my OAuth client \
 |dynamically,
 |* know the OAuth scope tokens I need in order to access a service,

I personally have problems with OAuth.  Not because of the
standard group split, but mostly because you have to go over
dedicated web sites (likely), granted, today without advertising.
You can also go over onion to avoid IP-based location tracking
(maybe, still), i develop fears this may not legally be possible
anymore in a not too distant future.  (I do not use onion.)

It would be nicer to have a client certificate (with password,
possibly), that can be very easily refreshed/replaced, maybe as
part of normal TLS payload, through whatever protocol i am
currently connected to my provider, may it be SMTPS, IMAPS, POP3S,
(HTTPS).  With a master account key.  And the possibility to
revoke the old client certificate and refetch a new one simply by
establishing a normal TLS session followed by a normal login with
that key.  You can diverse that and use diverse client id, client
secret and an access token.

I just have implemented OAUTHBEARER for my MUA's SMTP, POP3 and
IMAP parts.  While testing i needed the refresh token once an
hour, and the client id, and the client secret.  So, once an hour,
i need to load/decrypt the file which stores that triple.  It does
not contain the account's master password, this is true.
When i close the lid of my notebook i am protected, because the
encrypted filesystem will then be automatically unmounted, but
different to kdestroy(1) -A the token is still valid for some
time.

 |* know which endpoints serve the same data via different protocols,
 |* know *all* the available services (not just the ones I know about), \
 |so I can suggest other clients which may be useful.

Wow.  A different scenario.  You are the employee of a company and
have been sent via airplane (which i would refuse to do, since
airplanes are killers) to a different state/country, to another
subsidiary of the company.  You are expected to login via their
local net.  (I would wonder why they do not use some kind of
company VPN, and if it is as simplicistic as via tinc.  But ok,
i have no idea for one, and second the requirement to go over
a local subdomain i could still imagine.)

This points us back to the first paragraph.  Or the second.
What is wrong with having the corresponding (A,AAAA,MX,SRV etc.)
DNS entries for state1.company.xy and state2.company.xy?
Add some more for logos (if not yet existing) or whatever.
There is also that WWW paradigm of having a favicon.ico in the
root of your web presence, it will be fetched up automatically by
browsers, why can't you?

You need to have A/AAAA for the web presence in the DNS, why can't
you have MX / SRV for the other non-HTTPS services, too?
You want personalization on a per-user level, and you do not
control your DNS zone, and you do not want to spend a little bit
of money to get your own little DNS zone?  You would spawn a DNS
server on your box to overcome this, but getaddrinfo(3) does not
allow specifying which DNS server is to be asked?  Aha!  Yes, that
is a problem.  You want to furtherly enforce decentralisation by
moving responsibility away from the controlled and otherwise
constrained DNS to anyway running "private" HTTP servers.  (A
company could still have a VPN to overcome this, me thinks.)

If it is because DNS is controlled and constrained then following
the pigeon RFC there should possibly be an albatross RFC.  So that
slow people like me understand the issue, finally.  There is
mischief!  There is mischief!  I see more robots passing by that
collect all the information stored in .well-known.  I see people
getting bombed because of information stored under .well-known,
for nothing, who could they ask instead of a lawyer in that case?
In DNS you have at least the zone above, and their
administrator(s), which you could ask to ok that content in such
a case.   Would be cool to have S/MIME certificates via that
mechanism.  And POP3 would be cool if the current state of affairs
could be SYNCed, and if only the body of a message could be loaded
(especially autocrypt blows the headers enormously, and doing
a RETR after the summaries' TOP is a shame).

  ...
 |I don't see this as a use case for WebFinger though. The services offered \
 |by a provider is not user data (although they may vary for each users), \
 |it's primarily provider data. As such, I'd prefer to 
 |provide a single JSON document at a well-known URL under the provider's \
 |domain, e.g. at "/.well-known/services". This would make it trivial \
 |to host a static document, which would do the trick in most 
 |setups. More complex providers could still return per-user configurations \
 |(after authentication).
 |
 |The attached example shows how a document might look like as per the \
 |current status.

The draft[1] contains examples.

  [1] https://tools.ietf.org/html/draft-jones-webfinger-email-autoconfig-00

--steffen
|

---

From: "John Levine" <[email-masked]>
Date: 20 Jul 2019 01:04:44 -0400

>to be able to use SRV alongside.

There are places where, say, mailboxes with local parts starting with
A-M are on one IMAP server and those starting with N-Z are on another.
It's hard to see how you could do that with SRV records tied to the
domain.

>extensible, due to not using XML namespaces, for one thing.

It definitely needs extensions.

R's,
John

PS: for anyone who was going to suggest encoding the local part in the DNS for
SRV lookups, you really don't want to go there.

---

From: Marten Gajda <[email-masked]>
Date: Tue, 23 Jul 2019 00:29:29 +0200

> any interested application (via some kind of IPC, say DBUS), then.
Sure, but that provisioning part, which also contains user specific data
(e.g. account name or maybe even credentials), should be a separate
standard. Also, you don't want to share account data with "any
interested application". I'd prefer if the setup component checks the
system (and optionally the app store or software repository) for
applications which can handle the protocols in the service configuration
document and asks the user for which ones to provision with the
configuration and access tokens
> the services URLs were noted.  (Let's say imap.XY.Z.)
So? The idea behind an auto-configuration or service-discovery is to
avoid having to type any URLs into client applications. Also I'd expect
the client to rerun the discovery every now and then (for instance when
the cache expired) and pick up configuration changes (like newly
supported services).
> your provider!!  Shiny happy people there will be, i bet.

I'm not sure how to understand this statement. If you suggest that some
service providers deliberately "hide" password reset and settings pages
to present more ads to the user and they wouldn't be interested in
providing these shortcuts, you may be right. But many providers still
value usability and user experience, not to mention all the other
non-freemail providers and use cases (like corporate services). Maybe
you just have the wrong provider.

If that's not what you suggest, please be more specific.

> different to kdestroy(1) -A the token is still valid for some
You're off-topic. Anyway, you're welcome to propose a new standard and
I'd be happy to implement it once its ratified. For now OAuth2 is one of
the better options we have in terms of privacy and security. In general,
a service configuration protocol certainly should be agnostic to how
authentication is performed by the services. However, if the
authentication method requires the knowledge of certain endpoints they
should be provided in that document.
> a local subdomain i could still imagine.)
Not sure what you're talking about.
> Add some more for logos (if not yet existing) or whatever.

For starters, not all Frameworks allow you to resolve anything but
A(AAA) records. On Java (and Android for that matter) it's not so
trivial to lookup MX or SRV records or even the current DNS server
address. A simple GET request to a well-known address, on the other
hand, is barely an issue. There may be use cases in which it may not be
possible or reasonable to set up a webserver just to provide that
document, for instance when a service only provides email. In these
cases we could offer a fallback mechanism which allows a client to
retrieve the document from another location, for instance by providing
an SRV record which points to the location of the service configuration
document. Other fallbacks may be possible as well, but .well-known would
definitely preferred.

> root of your web presence, it will be fetched up automatically by

That's exactly what I'm proposing, allowing clients to get the
configuration for all services offered by a provider with a single GET
request to a well-known address (e.g. `/.well-known/services`).
Imagine the favicon.ico location would have to be retrieved by looking
up the "favicon" SRV record.

> you have MX / SRV for the other non-HTTPS services, too?

What's your point? You also need A/AAAA records for the non-HTTPS
services? You're comparing Apples and Oranges.

DNS is not suitable to provide that kind of structured information. One
of the reasons is that you only get information that you've explicitly
asked for, so you'd have to request every single bit of information that
you can use, without even knowing whether it exists.

> collect all the information stored in .well-known.

That's one more point for .well-known, if you ask me. Unless you have
your own private DNS server, all the SRV records are publicly available.
A service configuration document under .well-known can easily be
protected with authentication. IMO you should be able to retrieve a
redacted version of the document without authentication which just gives
enough information to know who you're talking to and how to authenticate
to get more info (e.g. by providing OAuth2 endpoints). There is no need
to disclose more information to unauthenticated users than you'd
disclose by setting up SRV records.

Hosting your own DNS zone comes at a price as you've just noted above.
You'll have higher efforts to maintain the server and you'll have to
make sure your client application uses the correct DNS server. So you'll
most likely have to enter it manually somewhere. This sort of defeats
the purpose of an automated service configuration.

In many cases a simple service configuration document can be hosted
statically. In other cases your server software (e.g. your Groupware)
could handle that for you. Maintaining DNS is not so easy and while TLS
encryption is almost trivial to set up these days (e.g. using
letsencrypt), DNSSEC is not that common yet
(https://stats.labs.apnic.net/dnssec).

>   I see people
Again, you can protect sensitive, non-public information with
authentication when using a .well-known resource. In addition a simple
document wouldn't have to contain much more information than you're
proposing to publish via DNS to the world.
>   [1] https://tools.ietf.org/html/draft-jones-webfinger-email-autoconfig-00

Yes, but my example doesn't use WebFinger and addresses a few other
issues. Have you actually seen it?

Cheers,

Marten

-- 
Marten Gajda
CEO

dmfs GmbH
Schandauer Straße 34
01309 Dresden
GERMANY

phone: +49 177 4427167
email: [email-masked]

---

From: Dave Cridland <[email-masked]>
Date: Tue, 23 Jul 2019 09:55:25 +0100

On Sat, 20 Jul 2019 at 06:04, John Levine <[email-masked]> wrote:

> In article <CAKHUCzzS+k_DUwaBcvTH_68G4JS_6UHA=
> [email-masked]> you write:
Sure. I'm just thinking that anywhere you say "IMAP server" we'd put an SRV
resolution.

> It definitely needs extensions.
s/extensions/extensibility/ but yes.

Dave.

---

From: "John R Levine" <[email-masked]>
Date: 23 Jul 2019 12:27:19 -0400

> Sure. I'm just thinking that anywhere you say "IMAP server" we'd put an SRV

In general I like SRV just fine but in this case, the config already 
provides a hostname and port so what does it add?  I don't thinkml MUAs 
are likely to use a backup IMAP server if the main one is unreachable.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 23 Jul 2019 18:25:09 +0100

> are likely to use a backup IMAP server if the main one is unreachable.

Hmmm... "thinkml"? Sounds familiar.

But to the point,  I don't know if MUAs will do SRV or not, if it's
presented to them in a standard. On a quick skim, some decided RFC 6186
wasn't useful:
https://mail.gnome.org/archives/commits-list/2012-November/msg01976.html and
others (like K9, for example) decided it was.

I do know, however, that Google deploy RFC 6186 records for gmail.com, so
there's hope - it's a useful mechanism for allowing multiple endpoints with
preferences etc, and I'm quite sure that we do have cases where multiple
IMAP hosts are deployed.

We could bake that functionality into the autoconfig fomat, of course, but
it feels like refreshing the config can happen at a different cadence to a
SRV lookup, and besides, SRV works well.

Dave.

---

From: "John Levine" <[email-masked]>
Date: 23 Jul 2019 10:34:56 -0700

Some do 6186, most don't.  I publish the srv records but my users all have to config manually. The real question of course is whether mua makers care about this enough to use it.Autocorrectly,John

In general I like SRV just fine but in this case, the config already 
provides a hostname and port so what does it add?  I don't thinkml MUAs 
are likely to use a backup IMAP server if the main one is unreachable.Hmmm... "thinkml"? Sounds familiar.But to the point,  I don't know if MUAs will do SRV or not, if it's presented to them in a standard. On a quick skim, some decided RFC 6186 wasn't useful: https://mail.gnome.org/archives/commits-list/2012-November/msg01976.html and others (like K9, for example) decided it was.I do know, however, that Google deploy RFC 6186 records for gmail.com, so there's hope - it's a useful mechanism for allowing multiple endpoints with preferences etc, and I'm quite sure that we do have cases where multiple IMAP hosts are deployed.We could bake that functionality into the autoconfig fomat, of course, but it feels like refreshing the config can happen at a different cadence to a SRV lookup, and besides, SRV works well.Dave.

---

From: Steffen Nurpmeso <[email-masked]>
Date: Wed, 24 Jul 2019 16:22:32 +0200

Hello.

Please excuse the late reply, i try to get any quantum of clowd
free summer i can, not that much this year..

Marten Gajda wrote in <[email-masked]>:
 |Am 19.07.19 um 15:33 schrieb Steffen Nurpmeso:
 |> Marten Gajda wrote in <[email-masked]>:
 |>|As a client developer, I want to provide a smooth UX to our users. \
 |>|This means I need much more information than just a server URL.
 |>
 |> You want to present them a barcode that they can scan with their
 |> smart phone and which is made available via some local registry to
 |> any interested application (via some kind of IPC, say DBUS), then.

 |Sure, but that provisioning part, which also contains user specific data
 |(e.g. account name or maybe even credentials), should be a separate
 |standard. Also, you don't want to share account data with "any
 |interested application". I'd prefer if the setup component checks the
 |system (and optionally the app store or software repository) for
 |applications which can handle the protocols in the service configuration
 |document and asks the user for which ones to provision with the
 |configuration and access tokens

 |>|For instance, I also want to 
 |>|* present the name or logo of the provider which hosts the user's \
 |>|account, \
 |>|ideally before they enter their credentials,
 |>
 |> You made a contract with a provider, and it has given you
 |> credentials to use.  The provider has an IP address, and you have
 |> submissions, pop3s and imaps access.  On the web site's welcome page
 |> the services URLs were noted.  (Let's say imap.XY.Z.)

 |So? The idea behind an auto-configuration or service-discovery is to
 |avoid having to type any URLs into client applications. Also I'd expect
 |the client to rerun the discovery every now and then (for instance when
 |the cache expired) and pick up configuration changes (like newly
 |supported services).

The barcode was a little irony; You seem to have in mind a totally
free world where you move in between say wireless networks and you
get automatically configured access wherever you are.
I hope i get informed and have to "ok" who debits my money, and
please let me wonder how this should work for anything but plain
transport providers?

 |>|* provide links to support channels, reset password pages, account \
 |>|management,
 |>
 |> Ha!  If it were that easy to find these pages on the web site of
 |> your provider!!  Shiny happy people there will be, i bet.
 |
 |I'm not sure how to understand this statement. If you suggest that some
 |service providers deliberately "hide" password reset and settings pages
 |to present more ads to the user and they wouldn't be interested in
 |providing these shortcuts, you may be right. But many providers still
 |value usability and user experience, not to mention all the other
 |non-freemail providers and use cases (like corporate services). Maybe
 |you just have the wrong provider.
 |
 |If that's not what you suggest, please be more specific.

I was talking about user interfaces.
Take the city München who moved from Linux back to another vendor,
just like Wien did many years before that.  All voices i have
heard were not talking about technical problems.

 |>|* know the OAuth endpoints, if available,
 ...
 |> I personally have problems with OAuth.  Not because of the
  ...
 |You're off-topic. Anyway, you're welcome to propose a new standard and
 |I'd be happy to implement it once its ratified. For now OAuth2 is one of
 ...

There are more than enough in my opinion.  I am a supporter of the
strict separation of transport and protocol.  That is easy for me,
i came to the Unix world in 1999 and C in 2000, and by then
securing the transport was already a thing.

I am also a supporter of moving authentication to the transport's
security setup as such.  This of course already happens, and
i think as long as i can password protect all the contents of
client certificates this is ok.  Maybe that could be further
diversified (several passwords for different data; but of course
why not using multiple certificates).
Or a simple login- or plain text authentication thereafter. 

 |the better options we have in terms of privacy and security. In general,
 |a service configuration protocol certainly should be agnostic to how
 |authentication is performed by the services. However, if the
 |authentication method requires the knowledge of certain endpoints they
 |should be provided in that document.

I fail to see the need for moving away to something new.
I mean, if something lacks, you need a new resource record type,
i do not know whether there is one for corporate logos yet.

And if that is not flexible enough because you want to have IMAP
server A serving names from a-n and B the rest, and SRV does not
do that for you, then maybe two RFCs should be joined and a SRVT
record should be introduced as an indirection, and SRVT should
contain an URI template which a client application can use to get
to the real SRV that applies to it.

So that is my thought: why move away from all the established
mechanisms and invent something completely new, when all that is
needed to overcome the missing flexibility is that?

How can a JSON file be better than that?  Or is it dynamically
created when a user authenticates via client certificate, with
content exactly as desired for that user?  You surely do not want
to place a potentially infinitely sized file with contents for all
the users, so -- what could a JSON file under .well-known do
better than a SRVT with an URI template?

 |>|* know which endpoints serve the same data via different protocols,
 |>|* know *all* the available services (not just the ones I know about), \
 |>|so I can suggest other clients which may be useful.
 ...
 |> This points us back to the first paragraph.  Or the second.
 |> What is wrong with having the corresponding (A,AAAA,MX,SRV etc.)
 |> DNS entries for state1.company.xy and state2.company.xy?
 |> Add some more for logos (if not yet existing) or whatever.
 |
 |For starters, not all Frameworks allow you to resolve anything but
 |A(AAA) records. On Java (and Android for that matter) it's not so
 |trivial to lookup MX or SRV records or even the current DNS server

That is definetely a problem for those frameworks, no?  I have no
idea of those, i left JAVA in Y2K, but i still have my JAVA
Programmierhandbuch by Middendorf/Singer around.

 |address. A simple GET request to a well-known address, on the other
 |hand, is barely an issue. There may be use cases in which it may not be

Is it still that easy with HTTP/2?

 |possible or reasonable to set up a webserver just to provide that
 |document, for instance when a service only provides email. In these
 |cases we could offer a fallback mechanism which allows a client to
 |retrieve the document from another location, for instance by providing
 |an SRV record which points to the location of the service configuration
 |document. Other fallbacks may be possible as well, but .well-known would
 |definitely preferred.

I see this differently, and i tell you why.  I think it deserts
the infrastructure.  You have a thing here, and a thing there, and
all those parts move on individually and need to be tracked, and
of course administred, too.

For example, email.  From the three protocols SMTP, IMAP and POP3
the last one is deserted, no one ever cared that it requires
a really stable internet connection because if you loose
connection after doing twenty things the server will have
forgotten you did so.  And DKIM, DMARC and AUTOCRYPT were
introduced happily, massively increasing the size of message
headers, not updating POP3 to have a "body-only" command (even
though it seems trivial to add the counterpart to the "TOP"
command that anyway exists).

So i am post-office, look at the summary of old headers, and
decide that some of them are important even after hours, the
others not, what am i left with.  Granted that a more holistic
point of view is very hard, especially if so many parties are
involved.  But here it seems more about overall interest, this is
at least my impression.

 |> There is also that WWW paradigm of having a favicon.ico in the
 |> root of your web presence, it will be fetched up automatically by
 |> browsers, why can't you?
 |
 |That's exactly what I'm proposing, allowing clients to get the
 |configuration for all services offered by a provider with a single GET
 |request to a well-known address (e.g. `/.well-known/services`).
 |Imagine the favicon.ico location would have to be retrieved by looking
 |up the "favicon" SRV record.

Well, yes and no.  That was very early, i think i already had
a favicon in 1996?  So to me this is web/HTTP since ever.
It would have been better to have a '<meta name="favicon"..',
then, maybe that even exists in addition?

 |> You need to have A/AAAA for the web presence in the DNS, why can't
 |> you have MX / SRV for the other non-HTTPS services, too?
 |
 |What's your point? You also need A/AAAA records for the non-HTTPS
 |services? You're comparing Apples and Oranges.

I actually do not understand this.

 |DNS is not suitable to provide that kind of structured information. One
 |of the reasons is that you only get information that you've explicitly
 |asked for, so you'd have to request every single bit of information that
 |you can use, without even knowing whether it exists.

You ask for auto-configuration, do you?  What shall be in that
JSON file you proclaim?  If it is more than something that fits in
a RR then what?  You need a naming scheme, an iterator, ... ?

 |> You want personalization on a per-user level, and you do not
 |> control your DNS zone, and you do not want to spend a little bit
 |> of money to get your own little DNS zone?  You would spawn a DNS
 |> server on your box to overcome this, but getaddrinfo(3) does not
 |> allow specifying which DNS server is to be asked?  Aha!  Yes, that
 |> is a problem.  You want to furtherly enforce decentralisation by
 |> moving responsibility away from the controlled and otherwise
 |> constrained DNS to anyway running "private" HTTP servers.  (A
 |> company could still have a VPN to overcome this, me thinks.)
 |>
 |> If it is because DNS is controlled and constrained then following
 |> the pigeon RFC there should possibly be an albatross RFC.  So that
 |> slow people like me understand the issue, finally.  There is
 |> mischief!  There is mischief!  I see more robots passing by that
 |> collect all the information stored in .well-known.
 |
 |That's one more point for .well-known, if you ask me. Unless you have
 |your own private DNS server, all the SRV records are publicly available.
 |A service configuration document under .well-known can easily be
 |protected with authentication. IMO you should be able to retrieve a
 |redacted version of the document without authentication which just gives
 |enough information to know who you're talking to and how to authenticate
 |to get more info (e.g. by providing OAuth2 endpoints). There is no need
 |to disclose more information to unauthenticated users than you'd
 |disclose by setting up SRV records.

Again, what should be in this JSON you want to serve?
Is it personalized data then you need to have authenticated
yourself already -- for this you need to have an account?
If it is not personalized data then what is missing from SRV RRs
...  that cannot be provided by indirecting via some URI template
pattern from a hypothetic SRVT?

What is better, something completely shiny new, like a nice little
JSON document under a new .well-known URL, or a mechanism that has
seen iterations and evolved to something that addresses problems
discovered after having former iterations in use.
(I know the former is the answer, of course, as in rewrite in C++,
from plain text to XML, rewrite in rust, move from XML to JSON,
etc.)

 |Hosting your own DNS zone comes at a price as you've just noted above.
 |You'll have higher efforts to maintain the server and you'll have to
 |make sure your client application uses the correct DNS server. So you'll
 |most likely have to enter it manually somewhere. This sort of defeats
 |the purpose of an automated service configuration.
 |
 |In many cases a simple service configuration document can be hosted
 |statically. In other cases your server software (e.g. your Groupware)
 |could handle that for you. Maintaining DNS is not so easy and while TLS
 |encryption is almost trivial to set up these days (e.g. using
 |letsencrypt), DNSSEC is not that common yet
 |(https://stats.labs.apnic.net/dnssec).

Oh, i am totally opposed to DNSSEC.  I would go for TLS/DTLS, and
i do not understand why the data cannot be signed with some TLS
certificate that can simply be looked up by doing a say HTTPS
connect to the zone's main address.

 |>   I see people
 |> getting bombed because of information stored under .well-known,
 |> for nothing, who could they ask instead of a lawyer in that case?

 |Again, you can protect sensitive, non-public information with
 |authentication when using a .well-known resource. In addition a simple
 |document wouldn't have to contain much more information than you're
 |proposing to publish via DNS to the world.

Yes.  Exactly.  And i really wonder what should be the content of
that non-public version, and how it can be accessed, and even
more, interpreted.

 |>|The attached example shows how a document might look like as per the \
 |>|current status.
 |>
 |> The draft[1] contains examples.
 |>
 |>   [1] https://tools.ietf.org/html/draft-jones-webfinger-email-autoconfig-\
 |>   00
 |
 |Yes, but my example doesn't use WebFinger and addresses a few other
 |issues. Have you actually seen it?

I had seen it by then, yes.

--steffen
|

---

From: Marten Gajda <[email-masked]>
Date: Thu, 25 Jul 2019 03:13:05 +0200

> get automatically configured access wherever you are.

No, I just want to configure my devices without having to enter all the
addresses of the services I use very time. At present this only works
well for proprietary software (e.g. MS Exchange) and dedicated clients
(e.g. setting up a Google account on Android or iCloud on an Apple
device). It's still a pain with arbitrary, generic clients and open
standard protocols.

The UX I'm striving for goes like this:

Me: configure "[email-masked]"

UA: Do you want to configure the account "[email-masked]" at Example
Inc. (<provider icon>)?
(provider name and Icon discovered automatically)

Me: yep, that's my provider

UA: Please enter your password for "[email-masked]"
(or whatever the commonly supported authentication method ofclient and
provider requires)

Me: *************************

UA: Congratulations, your Email, Calendars and Contacts are now synced.
    Btw, you if you install one of these apps you can also chat    with
other users: ......
Entering more details would only be necessary if the provider I want to
set up is not "example.com" but that's still part of my account
identifier. Some providers allow you to pick a login name which is an
email address of another provider. Average users don't recognize the
difference between email address and account identifier, hence the extra
step.

> please let me wonder how this should work for anything but plain

Sorry, I don't get your irony. What does this have to do with
service-discovery? The sole purpose is to allow UAs to detect service
configuration parameters that you would have to enter manually otherwise.

Also, what's a "plain transport provider"?

> heard were not talking about technical problems.

I still don't get it. Auto-discovery of service configurations enables
UAs to build user-friendly setup routines.

> content exactly as desired for that user?

Yes, that's how it's supposed to work. An authenticated user would
essentially see his/her "personal" service configuration, dynamically
generated. The user specific configuration may vary for a lot of
reasons, some of them are:

* user accounts are stored on different hosts because they are hosted in
a data center close to the user's office
* different users may have subscribed to different services
* different users may have different SLAs

For simple use cases a statically hosted document could work well, for
more complex use cases the configuration could be generated dynamically
for each user.

> Is it still that easy with HTTP/2?

It may be difficult on older platforms if the server only supports
HTTP/2 and not HTTP/1.1. However it's easier to overcome that with a 3rd
party library than it is to overcome lack of support for DNS. With
HTTP/2 you can use a library and you're done. With DNS you still need
the address of a DNS server trusted by the user. And that's a platform
specific issue.

> command that anyway exists).

Yeah, I know. It's also getting harder and harder to buy floppy disks.
Looks like most stores have neglected them since CDs and DVDs became
hip. And no one cared about increasing their capacity anymore.

I hate to break it to you, but IMAP will probably have the same future.
The potential successor is already in the works:
https://datatracker.ietf.org/wg/jmap/about/

> JSON file you proclaim?

I thought I already made that clear. Anyway here is a (non-exhaustive)
list of potential data points

* provider information, like
  * name
  * branding information, e.g.
    * an icon
    * a color scheme
  * legal information
* contact and support information like
  * link to provider home page
  * link to provider terms of service
  * link to password reset page
  * link to account management page (e.g. where you can revoke access
tokens or set defaults)
  * link to online support
  * support hotline phone number
  * address of email support
* available services, including
  * a (ideally localized) display name of the service (e.g. "Inbox" or
"Calendars")
  * endpoint URL
  * authentication information, e.g.
    * required OAuth2 scopes
  * information about the data set
    * which services provide the same data set (e.g. a provider might
offer access to the same mailbox via IMAP, POP3, JMAP, MS Exchange,
WebMail, so a multi protocol client should pick only one of them)
  * certificate pinning information
* authentication information
  * for OAuth2 that would be the various endpoints
  * for client certificate authentication, that could be the required issuer

> yourself already -- for this you need to have an account?

Yes, why would you need service auto-discovery if you don't have an account?

> pattern from a hypothetic SRVT?

For starters, the document can be localized, either as per location
and/or language settings in the user's account or by adhering the
ACCEPT-LANGUAGE header.

The list of services can be adjusted to the user's pricing/subscription
model, e.g. premium/paying users see more services than free users,
employees may have access to different services (developers see the git
repo, sales gets to see the salesforces URL)

If the server supports a (yet to be standardized) push protocol, the
client could subscribe to changes on the service configuration.

Scalability, multi-tenant setups can simply redirect all domains to the
same document URL. With DNS you'd have to duplicate and maintain the
entries for every domain a provider supports.

Checking for updates is cheap, you can use conditional requests and get
a "Not Modified" response if nothing has changed. I don't think you can
do that with DNS.

> from plain text to XML, rewrite in rust, move from XML to JSON,

IPV4 to IPV6, HTTP/1.0 to HTTP/1.1 to HTTP/2, floppy disk to CD to DVD,
HDD to SSD, smoke signals to telephone, telnet to ssh, light bulb to LED
... that's how it works. Things get improved and eventually replaced.

"Being better" is not a matter of old vs. new. It's a matter of "does it
solve my problem?". Apparently DNS can not solve all the issues related
to auto-configuration (and it doesn't have to).

Btw, an auto-configuration document is not a new concept. MS Exchange
had it for years and one of the default mechanisms is to look up a URL
derived from the user's domain. Mozilla Thunderbird has a similar mechanism.

What's missing is an open and standardized mechanism that's not tailored
for email but very generic.

> connect to the zone's main address.

Whoa! Your refuting hosting a service-configuration document on a
webserver but you would base your DNS security on HTTPS? As per you
logic you probably should provide the certificate in another DNS RR.

If you have a better (simpler) solution than DNSSEC, why don't you write
up an RFC draft and submit it for discussion?

Anyway this is far off-topic.

> that non-public version, and how it can be accessed, and even

See above. Services and contact/support information could only be
accessible after authentication, because they may not be meant for the
public or be user specific (e.g. an employee in the European office
could get a different support hotline than an employee located in America).

Interpretation is easy if there is a standardized document schema.

-- 
Marten Gajda
CEO

dmfs GmbH
Schandauer Straße 34
01309 Dresden
GERMANY

phone: +49 177 4427167
email: [email-masked]

---

From: "Paul E. Jones" <[email-masked]>
Date: Thu, 25 Jul 2019 04:08:57 +0000

Dave,

>We could bake that functionality into the autoconfig fomat, of course, 
>but it feels like refreshing the config can happen at a different 
>cadence to a SRV lookup, and besides, SRV works well.

It works well for the simplest use case, but SRV records are really 
limited. I can name a three things SRV records cannot do today that are 
effectively render it useless for me:

1) It cannot tell the MUA the login ID to use (which may or may not be 
the email address) [or what credential mechanism to use]

2) It cannot specify (if I recall) that TLS must be used per corporate 
policy [this is important since both IMAP and mail submission generally 
offer STARTTLS as an option]

3) It doesn't allow specifying different servers for different users

Paul

---

From: "John Levine" <[email-masked]>
Date: 24 Jul 2019 21:45:55 -0700

> 2) It cannot specify (if I recall) that TLS must be used per corporate policy [this is important since
> both IMAP and mail submission generally offer STARTTLS as an option]

It can sort of do that by publishing _imaps._tcp rather than _imap._tcp 
but in general you are correct that SRV just isn't up to the job in this 
case.

R's,
John

---

From: Dave Cridland <[email-masked]>
Date: Thu, 25 Jul 2019 09:28:56 +0100

On Thu, 25 Jul 2019 at 05:09, Paul E. Jones <[email-masked]> wrote:

> 1) It cannot tell the MUA the login ID to use (which may or may not be the
> email address) [or what credential mechanism to use]
>
And to re-iterate, I'm not suggesting we should forgo any kind of
autoconfiguration format in favour of SRV; I'm suggesting that having the
autoconfiguration format specify servers in the form of SRV would be useful.

That said, (2) is irrelevant on two counts :

a) You can use `imaps` SRV records just fine. They're published for
gmail.com, go have a look.

b) You can require TLS via STARTTLS within the protocols, and you'd have to
anyway, irrespective of what any autoconfiguration stipulated. Would you
consider stipulating a policy in an autoconfiguration file but not enforce
it in the services?

I shall now reiterate the arguments in favour of using SRV in this way,
because it seems I've not be clear enough.

First, some background:

I have written both email servers and an MUA, and for the latter, I wrote a
complete ACAP server to support autoconfiguration. ACAP is effectively
dead, but the experience has left me with some practical experience of how
to write autoconfiguration support in an MUA, and - more importantly - some
idea of how one might retrofit that support into an existing MUA.

The fact is that existing MUAs often have plenty of support for an
autoconfiguration step, but it exists during setup. So you can easily
specify authentication identifiers and so on at this point, but if the
autoconfiguration source changes, you're out of luck. This is unsurprising,
since with IMAP at least, a decent MUA has to do quite substantial effort
on first connect anyway in order to make decisions on the remote server's
mailbox layout. It might change with JMAP, but all existing MUAs are
currently IMAP-based, so in order to gain market traction, we need to aim
for this "auto setup" model.

The UX we're expecting is that, during Setup and "Add New Account", the
user enters an email address, and specifies credentials, and they're done -
the MUA will then autoconfigure the new account. There's lots of useful
information that can be sourced here from an autoconfiguration service.
Does the service support subaddressing? Is the authentication for the
submission service the same as the authentication for the IMAP service (so,
can I share credentials?). All of this is useful and vital information, and
I'm not going to advocate scrapping it (instead, though, please do look at
the ACAP dataset definitions for some thoughts and ideas - Randy Gellens
put a lot of good work into those).

During connection, there is more discovery. This includes locating the
hosts via DNS, discovering what TLS support is available, discovering what
authentication mechanisms are supported, and so on. There are pages of RFCs
already in existence for how these different processes work, and much of it
we - in the app layer - don't care about, because it's taken care of by DNS
libraries, TLS libraries, and SASL.

However, this second phase is routinely done - sometimes from scratch -
every time a connection is made. It's here I'm suggesting that existing
clients could easily be made to use SRV (with A/AAA fallback), and gain
some useful resilience, since this resolution occurs on every connection,
and not just on service configuration. Thanks to a lot of time spent in the
XMPP space (where most clients fully autoconfigure off the Jid), I'm
familiar with the benefits of SRV, and how easy or hard it is to do on most
platforms (these days, very easy, and with added DNSSEC, DTLS, and/or DOH).

There's also final part of discovery for an MUA. Here, after connection and
authentication, an MUA looks at server layout, features, and so on. It will
examine the mailbox layout. Is there a specific Drafts folder? What does
NAMESPACE return? Is BURL supported on the submission server? While these
are all, in principle, subject to change, doing so will deeply confuse many
MUAs, so it's best to consider these a one-off setup. It is, I think, this
that means that the pragmatic aim for this is to assume that whatever we
choose to design, the protocol will mostly be used for "auto setup", and
not a more continuous "auto configuration". Perhaps something else is the
forcing factor though - in any event, auto configuration sources are used
as one-off procedures during setup and not on as a part of connection,
startup, etc.

I hope this is clearer.

Dave.

---

From: Steffen Nurpmeso <[email-masked]>
Date: Thu, 25 Jul 2019 16:49:07 +0200

Hello.

Sorry, but i am drawning in spam since i post in this thread, and
so i lost your response by accident.  Therefore like this.
I also make it short since our personal discurs leads to nothing
anyway, and that list is noisy as has not been for many many
months.

  |list of potential data points
  |
  |* provider information, like
  |  * name
  |  * branding information, e.g.
  |    * an icon
  |    * a color scheme
  |  * legal information
  |* contact and support information like
  |  * link to provider home page
  |  * link to provider terms of service
  |  * legal information
  |* contact and support information like
  |  * link to provider home page
  |  * link to provider terms of service
  |  * link to password reset page
  |  * link to account management page (e.g. where you can revoke access
  |tokens or set defaults)
  |  * link to online support
  |  * support hotline phone number
  |  * address of email support
  |* available services, including
  |  * a (ideally localized) display name of the service (e.g. "Inbox" or
  |"Calendars")
  |  * endpoint URL
  |  * authentication information, e.g.
  |    * required OAuth2 scopes
  |  * information about the data set
  |    * which services provide the same data set (e.g. a provider might
  |offer access to the same mailbox via IMAP, POP3, JMAP, MS Exchange,
  |WebMail, so a multi protocol client should pick only one of them)
  |  * certificate pinning information
  |* authentication information
  |  * for OAuth2 that would be the various endpoints
  |  * for client certificate authentication, that could be the required issuer

Well, this is obviously a different type of thing that i have in
mind when i think about email, which is SMTP/POP3(/IMAP) here.
(In fact ssh has to be added since it transports to postfix here.)
This reduces the above a lot.  What i need can be done by SRV,
maybe a SRVT i could think about.

I think the above is highly artificial, but having a SRV entry
which provides an URI that leads to some kind of impressum, or
imprint in english, is definetely something that is missing, when
i think about it.  Such things are often very hard to find on web
pages, even though legally required (in Germany).  The rest not,
sorry.

  |The potential successor is already in the works:
  |https://datatracker.ietf.org/wg/jmap/about/

Ah ja, that really is interesting.  Mysterious RFC 2047 examples
that should not occur in practice soil the stuff, but they of
course make the UTF-8 data that the future brings stand out even
more!  Just like always, very nice.  I see human beings happily
looking at DKIM signatures in clear text, in that bright future.
Mind you, keeping connections open more often in other protocols
would improve the results, plain DNS is an example, but as you say
correctly that now is also passed via the HTTP server (to the
browser).  If so, one could also imagine a mail superserver that
switches in between SMTP and IMAP on request.  Well, maybe not,
the asynchronousity is missing for SMTP.

Shall i live long enough to pimp the little text MUA i maintain,
and then have a nice I/O layer, then if dovecot etc. and the major
free mail services do offer JMAP already i might be going for this
thing, rather than for IMAP itself!  It really looks ok, and
i enjoyed the option that user agents no longer need to worry
about mail standards as such -- they can then simply take the JSON
part and display it directly, this is almost what UPAS of Unix v10
and even more in Plan9 had a quarter of a century before that:
using plain text processing on the individual parts of messages
simply by accessing those as files in a (virtual) file system.
The nmh people seem to locally reencode all data to UTF-8 to be
able to do something *almost* similar.

--steffen
|

---

From: "John Levine" <[email-masked]>
Date: 26 Jul 2019 10:32:04 -0400

I think we agree more than we disagree.  I totally agree that only 
one-time setup when you add a new account to your MUA makes sense, not 
every time it starts up.

But in this application I still don't see the benefit of an extra level of 
indirection via SRV.  Whether we start with ACAP or (more likely) with the 
Mozilla XML format, you've already got the indirection.  Large mail 
systems like Gmail and Hotmail/Outlook get redundancy now via multiple A 
and AAAA records.

R's,
John

On Thu, 25 Jul 2019, Dave Cridland wrote:

> It works well for the simplest use case, but SRV records are really limited. I can name
> a three things SRV records cannot do today that are effectively render it useless for
> me:
> 
> 1) It cannot tell the MUA the login ID to use (which may or may not be the email
> address) [or what credential mechanism to use]
> 
> 2) It cannot specify (if I recall) that TLS must be used per corporate policy [this is
> important since both IMAP and mail submission generally offer STARTTLS as an option]
> 
> And to re-iterate, I'm not suggesting we should forgo any kind of autoconfiguration format in
> favour of SRV; I'm suggesting that having the autoconfiguration format specify servers in the
> form of SRV would be useful.
> 
> a) You can use `imaps` SRV records just fine. They're published for gmail.com, go have a look.
> 
> b) You can require TLS via STARTTLS within the protocols, and you'd have to anyway,
> irrespective of what any autoconfiguration stipulated. Would you consider stipulating a policy
> in an autoconfiguration file but not enforce it in the services?
> 
> I shall now reiterate the arguments in favour of using SRV in this way, because it seems I've
> not be clear enough.
> 
> First, some background:
> 
> I have written both email servers and an MUA, and for the latter, I wrote a complete ACAP
> server to support autoconfiguration. ACAP is effectively dead, but the experience has left me
> with some practical experience of how to write autoconfiguration support in an MUA, and - more
> importantly - some idea of how one might retrofit that support into an existing MUA.
> 
> The fact is that existing MUAs often have plenty of support for an autoconfiguration step, but
> it exists during setup. So you can easily specify authentication identifiers and so on at this
> point, but if the autoconfiguration source changes, you're out of luck. This is unsurprising,
> since with IMAP at least, a decent MUA has to do quite substantial effort on first connect
> anyway in order to make decisions on the remote server's mailbox layout. It might change with
> JMAP, but all existing MUAs are currently IMAP-based, so in order to gain market traction, we
> need to aim for this "auto setup" model.
> 
> The UX we're expecting is that, during Setup and "Add New Account", the user enters an email
> address, and specifies credentials, and they're done - the MUA will then autoconfigure the new
> account. There's lots of useful information that can be sourced here from an autoconfiguration
> service. Does the service support subaddressing? Is the authentication for the submission
> service the same as the authentication for the IMAP service (so, can I share credentials?).
> All of this is useful and vital information, and I'm not going to advocate scrapping it
> (instead, though, please do look at the ACAP dataset definitions for some thoughts and ideas -
> Randy Gellens put a lot of good work into those).
> 
> During connection, there is more discovery. This includes locating the hosts via DNS,
> discovering what TLS support is available, discovering what authentication mechanisms are
> supported, and so on. There are pages of RFCs already in existence for how these different
> processes work, and much of it we - in the app layer - don't care about, because it's taken
> care of by DNS libraries, TLS libraries, and SASL.
> 
> However, this second phase is routinely done - sometimes from scratch - every time a
> connection is made. It's here I'm suggesting that existing clients could easily be made to use
> SRV (with A/AAA fallback), and gain some useful resilience, since this resolution occurs on
> every connection, and not just on service configuration. Thanks to a lot of time spent in the
> XMPP space (where most clients fully autoconfigure off the Jid), I'm familiar with the
> benefits of SRV, and how easy or hard it is to do on most platforms (these days, very easy,
> and with added DNSSEC, DTLS, and/or DOH).
> 
> There's also final part of discovery for an MUA. Here, after connection and authentication, an
> MUA looks at server layout, features, and so on. It will examine the mailbox layout. Is there
> a specific Drafts folder? What does NAMESPACE return? Is BURL supported on the submission
> server? While these are all, in principle, subject to change, doing so will deeply confuse
> many MUAs, so it's best to consider these a one-off setup. It is, I think, this that means
> that the pragmatic aim for this is to assume that whatever we choose to design, the protocol
> will mostly be used for "auto setup", and not a more continuous "auto configuration". Perhaps
> something else is the forcing factor though - in any event, auto configuration sources are
> used as one-off procedures during setup and not on as a part of connection, startup, etc.
> 
> I hope this is clearer.
> 
> Dave.
> 
>