Subject: [Uta] Questions about MTA STS complexity

From: Alberto Bertogli <[email-masked]>
Date: Sun, 4 Dec 2016 16:06:54 +0000

Hi!

I recently came across the MTA STS draft as part of some SMTP work I'm
doing on my spare time.

I really like the end goals of it, and would love to see them widely
deployed.

However, I couldn't help but find it a bit complex, in particular in
ways that I think will make it difficult for smaller domains to adopt.

While surely it's not outside the abilities of a strong dedicated team
or a careful individual, it requires significant domain knowledge in a
few areas, and is easier to get wrong.

In particular:

- Requiring additional DNS records make it more difficult to set up.
- Having to keep the IDs in sync between the DNS records and the HTTPS
  policy is, IMO, a significant operational burden, and makes it more
  difficult to “get it right" when making changes.
- The above also carries a complexity overhead in the RFC itself and the
  software implementation, by having to consider the interactions
  between the two (such as the caching logic, multiple level domains,
  etc.).

I was wondering if this you had any rationale about those decisions that
would help me understand why the current draft chose to go in this
direction, instead of potentially simpler approaches.

I hope this doesn't come across the wrong way, I debated whether to send
this or not, but seeing the invitation to comment on your git repository
tipped the scale :)

Thanks a lot!
                Alberto

---

From: Daniel Margolis <[email-masked]>
Date: Wed, 7 Dec 2016 20:09:28 +0100

Hi,

Thanks very much for the feedback. I think the good news is that some of
these comments are addressed in the upcoming draft.

Comments inline.

> - Requiring additional DNS records make it more difficult to set up.

DNS seems to me the best way to cheaply discover both domain participation
and new-policy-presence. In any event, given that SPF, DKIM, and DMARC all
require similar TXT records, I would assume this isn't *that* hard, is it?

>   difficult to “get it right" when making changes.

We've removed this. But you're right. :)

>   between the two (such as the caching logic, multiple level domains,

There are a few potential problems you are alluding to, I think.

1. The "don't use a new policy if it hasn't been successfully used before"
thing. We have gotten rid of this. :)

2. Deployment issues, e.g. publishing a policy on the HTTPS endpoint and
updating the TXT record may be problematic if the HTTPS is served via a CDN
with some unclear cache update time. Will people see the new TXT, update
the policy, and get the old policy? Fortunately I think this kind of issue
is fairly insignificant, since the DNS version ID is in fact unimportant.
You can publish a new policy via HTTPS and then rotate the TXT record as
many times as you like. So policy publishers who can't figure out their CDN
can either a) wait a while for the CDN to update before updating the TXT or
b) update the TXT today *and* tomorrow.

> direction, instead of potentially simpler approaches.

Well, the generic answer is that we didn't see simpler approaches that
didn't have some other downside. But there may be them--nondiscovery is not
a proof of non-existence!

Do you have a suggested change or set of changes?

> this or not, but seeing the invitation to comment on your git repository

Oh, definitely not. I appreciate the thoughtful comments!

>
> Thanks a lot!
>                 Alberto
>
> _______________________________________________
> Uta mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/uta
>

---

From: Alberto Bertogli <[email-masked]>
Date: Sat, 10 Dec 2016 12:23:58 +0000

> these comments are addressed in the upcoming draft.

Thanks for the detailed replies, and happy to hear about the upcoming
draft, look forward to reading it :)

> require similar TXT records, I would assume this isn't *that* hard, is it?

I agree, but I think it's the combination of additional subdomain +
record for it + policy ID embedded in DNS that adds up.

I have a suggestion about this below.

> We've removed this. But you're right. :)

Great!

> b) update the TXT today *and* tomorrow.

But having the policy ID on TXT means that you eventually should keep
them in sync.  It's less harmful complexity, but it's still complexity.

It also makes delegation more difficult, as you can't just set a record
once pointing to someone else, and forget about it.

What do you think of the following alternative: publish the policy
contents on https://_smtp_policy.<domain>/....

- No TXT record, no ID on DNS. Being able to resolve A/AAAA/CNAME for
  _smtp_policy.<domain> is equivalent to seeing the valid TXT record for
  _mta-sts.<domain>.

- Implementations can just use http client libraries, and avoid direct
  DNS lookups and parsing, simplifying the implementation.
  If the policy is not present, this fails just as fast as the _mta-sts
  subdomain not existing.

- Delegation continues to be reasonably easy: point _smtp_policy to the
  email provider. The email provider will need to get a valid cert for
  _smtp_policy.<domain>, just like in the current draft.

About the policy ID: remove it completely, and rely on standard HTTP
caching instead. Recommend in the RFC that an ETag+Cache-Control should
be used when replying to GET requests for the policy, and they indicate
the clients for how long they can cache it, and provide an ID for quick
checking.

This mechanism is well deployed, most CDNs support it already, libraries
to manipulate them well are readily available, and removes all the cache
management complexity from the RFC itself.

> Do you have a suggested change or set of changes?

I think the above are the main ideas I had :)

I may have a couple of suggestions for the policy contents, if it's ok
with you I can send them next week, but (to me) they are less important,
as they don't relate to deployment and just content and extensibility.

> Oh, definitely not. I appreciate the thoughtful comments!

Thanks a lot for your thoughtful response, and for working on this!

		Alberto

---

From: Daniel Margolis <[email-masked]>
Date: Sat, 10 Dec 2016 15:04:02 +0100

Thanks again.

The primary suggestion you make is very similar to some early discussions
we had. The reason we stuck with DNS version signalling is to ensure that
senders can cheaply check for new policies (e.g. in the case of a delivery
failure) without worrying too much about availability of the HTTPS policy
host; I agree that you can probably do the same with HTTP cache control,
but it still means the HTTPS endpoint has to be able to serve every
request, whereas DNS naturally provides caching at the recursive resolver.

This allows us, I believe, to more easily require that senders MUST check
for a new policy before permanently failing a delivery, which provides an
important safety hatch for long-lived policy caching. (In error conditions,
the "is there a new policy?" check is essentially as frequent as failed
delivery attempts, whereas in normal conditions it's far lower, so there's
some fear of cascading errors where a failure to resolve a new policy--and
a failing cached policy--results in a DoS that prevents fetching the new
policy.)

That said, is it worth the complexity to deployments? As you say, it means
that rolling out a new policy requires some DNS change on part of the
mail-to domain, even if they are delegating all other aspects of their
delivery. However, I believe a new policy should be required no more
frequently than new MX records themselves, since all the policy specifies
is the trusted MX records--and changing the MX records requires the
customer to make a change to their DNS already.

I do think the concern you raise is far greater if we were to move to e.g.
public key pinning in policies; in that case, policy updates could be far
more frequent than DNS changes, and the result is a real deployment
complexity. So if in a later version we wanted to make such more specific
policies, I believe we'd have to think about better methods of policy
deployment.

So in short, I believe in the current version the deployment complexity is
not that big a concern because the DNS changes should be no more frequent
than any existing DNS changes. But I do think you make a good observation,
and the complexity/risk tradeoff is a hard one to really measure.

Does it make sense?

Happy to read other comments you have on the policy contents as well.

Dan

---

From: Alberto Bertogli <[email-masked]>
Date: Wed, 14 Dec 2016 20:20:29 +0000

> and the complexity/risk tradeoff is a hard one to really measure.

It does, thanks for the detailed and thoughtful response.

In particular, I think the tradeoff you're characterizing (HTTPS serving
requirements vs. deployment complexity) is spot on; and that the
assumption that changing the policy is very infrequent carries a
significant weight in favour of the DNS approach.

However, within this context, I'd like to challenge that assumption a
little, if you don't mind :)

To me, one of the main benefits of the current proposal is
extensibility: it opens the door for using the policy definition file in
very useful ways, even if the current draft does not include them.

Key pinning is a great example that you mention in your email. It could
also be used to include SPF, DKIM and DMARC information; and who knows
what else people will come up in the future (list capabilities? signal
support for new protocols?).

Going with the assumption that the information in the policy changes
very infrequently can be quite limiting to future expansions. It won't
make them impossible to deploy, of course, but just much harder, which
will, in turn, likely hurt their adoption.

It also makes delegation more difficult, or limits the policies to a
coordinated change between the email handler and the domain owner, which
is hard to do specially at scale.

I fully agree that the HTTPS-only option puts a burden on reliably
running an HTTPS service. But I think these days that is a quite well
understood problem, and there are lots of options and general expertise
for doing it right.

Client-side caching can also serve as short-term unavailability
mitigation: HTTP Cache-Control's max-age and max-stale can be used by
policy servers to suggest clients how to cache the data and handle
expirations.  The complexity of this is not very high, and will be
included in SMTP software, so the cost is on people implementing the
standard (few) and not using it (hopefully lots :).

As I see it, going HTTPS-only now in the interest of increasing adoption
and aiming at making it easier to extend the policy in the future is a
better tradeoff than going with DNS now and having to tweak/change it
later when expansions appear.

Does this make sense to you?

In any case, thanks again for taking the time to consider all this and
replying to thoughtfully, I really appreciate it.

> Happy to read other comments you have on the policy contents as well.

I have a suggestion to split the contents in "incoming email" and
"outgoing mail", for example, by url (mta-sts-in.json and
mta-sts-out.json), to make it easier to understand by having a clearer
scope.

Reflecting on it now, I still like it but I don't think is particularly
important, as entries can still be unambiguously defined and be
reasonably readable anyway :)

Thanks a lot!
		Alberto

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 14 Dec 2016 15:38:20 -0500

> On Dec 14, 2016, at 3:20 PM, Alberto Bertogli <[email-masked]> wrote:
> 
> 

I would not support implementation in Postfix of a protocol that causes
the SMTP client to trigger unsolicited HTTPS probes for all destination
domains.  The proposed DNS probe is cheap and cached by the local resolver.
HTTPS probes are then only initiated for domains that publish a new policy
ID (or whose cached policy is sufficiently near expiration).

The proposed DNS record can also be used to expedite policy refresh without
requiring frequent HTTPS polling.  I'd like to see the DNS record retained,
and would recommend avoiding an over-engineered kitchen-sink policy mechanism.

-- 
	Viktor.

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Thu, 15 Dec 2016 00:07:08 +0100

> and would recommend avoiding an over-engineered kitchen-sink policy mechanism.

+1

/rolf

---

From: Daniel Margolis <[email-masked]>
Date: Fri, 16 Dec 2016 15:25:52 +0100

Right, so I think as Rolf and Viktor said, there are real concerns *on the
MUA side* about the difficulty of an HTTP-only indicator of policy
freshness.

I think there are some reasonable points on both sides, and you (Alberto)
definitely raise a good one about future extensibility versus our
assumptions on infrequent policy updates. I don't claim to have the right
answer, but in the interests of achieving a v1 I think we have been sort of
weighting the feedback of MTA developers like Viktor fairly highly. ;)

But to sum it up, I think in the foreseeable future the DNS version
indicator is (I hope) a means of making implementation *easier* (for MTAs,
primarily) and adoption for senders easier, not the other way around.

Thanks again for all the comments. As you may see, we just published the
second draft with many changes; I have sent an email summarizing the major
ones.

Dan

---

From: Alberto Bertogli <[email-masked]>
Date: Fri, 16 Dec 2016 23:04:34 +0000

On Wed, Dec 14, 2016 at 03:38:20PM -0500, Viktor Dukhovni wrote:
> domains.

I think this is totally reasonable, but not exactly what I'm proposing :)

> The proposed DNS probe is cheap and cached by the local resolver.

But using HTTPS only has the same polling frequency. The MTA doesn't
need to trigger the probe any more often than with the current proposal.

MTAs would do _less_ probing, as with HTTPS only there would be a window
of time without any probing at all, while in the current proposal you do
a DNS lookup to see if the policy has changed.

On domains without a published policy, it's exactly the same cost (a
single failed DNS lookup).

<longer explanation of the above>

The first time the MTA sends an email to a domain, it would do a DNS A
lookup for _mta-sts.domain. If the domain does not exist, there's no
published policy and there's nothing else to do.
This is equivalent to the current proposal (A lookup vs TXT lookup).

Then, an HTTPS request is made to fetch the policy. Alongside the
contents of the policy (unchanged from the draft), you get a max-age and
max-stale parameters which control caching.

MTA then caches the policy (unchanged from draft).

- MTA can use the policy without probing or checking anything until
  max-age.
  - Note max-age could be taken from the policy contents as in today's
    draft, or from the HTTPS headers.
- After max-age, the policy has to be fetched again.

So in both the draft and my HTTPS-only proposal you only need to do a
single HTTPS probe every max-age.

The difference is that in the current draft you need to do DNS-only
checks for every email on top of that.

</>

I hope this clarifies!

> and would recommend avoiding an over-engineered kitchen-sink policy mechanism.

I don't know what this refers to, as the policy itself would not be
changed (and if anything the RFC would be shorter). Can you elaborate?

Thanks!
		Alberto

---

From: Alberto Bertogli <[email-masked]>
Date: Fri, 16 Dec 2016 23:12:51 +0000

> weighting the feedback of MTA developers like Viktor fairly highly. ;)

FWIW, I totally understand and agree it is definitely the right thing to
do, even if MTA developers are unconvinced by my suggestions :)

Thanks very much for all your patience and thoughtful responses.

		Alberto

---

From: Daniel Margolis <[email-masked]>
Date: Sat, 17 Dec 2016 10:35:28 +0100

On Sat, Dec 17, 2016 at 12:04 AM, Alberto Bertogli <[email-masked]

> need to trigger the probe any more often than with the current proposal.

I think you're basically right about polling frequency, but I believe there
are two other reasons to keep the current design.

First, DNS (leveraging the local resolver cache) avoids the need for a
local read-write cache that remembers last-check-time. (We still need a
local policy cache.)

What I mean by that is that in the case where an MTA wants to check if a
*new* policy exists--say, when the cached policy has failed to
validate--the MTA may be doing so on every send. In a distributed
environment (i.e. multiple MTAs in the same datacenter, but *sharing the
same recursive resolver*), the check for "is there a new policy version?"
is cached by the recursive resolver.

In the HTTP-only model, however, each MTA must check-and-maybe-write the
last-fetch-time (and whatever other sort of cache control data) upon fetch.

I haven't thought deeply about this tradeoff, to be honest; given that
there has to be some cache (preferably centralized anyway) of *discovered
policies*, this may not be a big deal at all, but it's something to think
about. We chose DNS for this because it was (fairly) easy to expect certain
semantics from senders in terms of remembering when they last checked for a
new policy; you can certainly do the same thing with HTTP cache semantics,
but it may require slightly more explicit implementation on part of sending
MTAs. (A well-behaved HTTP client library may at least honor the cache
controls per-machine, so I'm not sure if the only risk here is that the
cache is not distributed, which isn't likely *that* big a deal.)

The second reason, potentially more compelling, is that the use of a
magical TXT record makes it slightly harder for domains that allow
untrusted content on user-chosen subdomains (like tumblr.com or dyndns.org)
to have a malicious user serve a phony policy. I think this reason is a
good bit more compelling (though there are, again, alternatives, like
requiring a special EKU in the SSL certificate presented over HTTPS).

Dan