Closing a critical gap in email authentication

This week the IETF took a significant step to improve authentication for the email ecosystem. RFC 8617, the Authenticated Received Chain (ARC) Protocol, describes a protocol for Internet mail handlers to preserve authentication data when forwarding email messages. ARC improves the ease, reliability, and ubiquity of authenticated email for those who run email systems, and eliminates the need for some work-arounds email users have had to put in place to overcome limitations of previous technologies.

Previous limitations

Existing authentication mechanisms have run into difficulties with complex mail flows. When authentication information is lost in transit, and failing authentication is supposed to result in message rejection, as with Domain-based Message Authentication, Reporting & Conformance (DMARC, RFC 7489), problems are compounded. Those problems are detailed exhaustively in RFC 7960, and while they represent a small percentage of overall Internet email volume, the problems are significant enough to have created frustration within the IETF and other communities that rely heavily on mailing lists, and has further prevented adoption of authentication in some quarters.

In short: Sender Policy Framework (SPF, RFC 7208) breaks under most forwarding circumstances, and DomainKeys Identified Mail (DKIM, RFC 6376) breaks when messages pass through forwarding services that modify content covered by the DKIM signature. Of these types of forwarders, the most problematic are mailing lists, email gateways, and other message-modifying filters. When a policy is supposed to be applied to messages that fail to authenticate, as with DMARC, these forwarding situations result in legitimate messages being marked as spam or outright rejected.

Addressing the gaps

ARC solves these problems by providing a means for Internet mail handlers to attest to the authentication status of a message at the time they receive it. These attestations are then signed and bundled with the message as it is forwarded (a process called “sealing”). As multiple Internet mail handlers process the email and undertake sealing, the result is an ARC Chain.  (For clarity in this post, I’ll use “ARC” to refer to the protocol and, despite the redundancy, “ARC Chain” to refer to the ARC header fields attached to a message.)

Utilizing this ARC Chain, validators further along can examine the attestations to verify that the message was properly authenticated when it originated despite any changes made to it since then. Specifically, when an ARC Chain validates, the list of domains that provide authentication assessments is guaranteed to include all the domains which modified the message. It is still possible for mail systems that do not modify the message to have handled it in transit and not added to the ARC Chain. But, it is not possible for the ARC Chain to validate if an Internet mail handler has modified the message without sealing it.

Without ARC, many receiving systems have to guess if unauthenticated mail might need policy overrides, resulting in heuristics that are inefficient, constantly changing, and abusable by malicious parties. With ARC, the Internet mail ecosystem now has a way to preserve authentication information—even across the most complex mail routing scenarios. This opens the door to more widespread adoption of authentication through DMARC, SPF, and DKIM, creating a more secure ecosystem overall by reducing the usage of heuristics and increasing the quantity of legitimate messages that are able to be authenticated.

Deploying ARC

There is a world of community support around ARC. Especially since ARC deals with chained messages and complex mail routing, interoperability testing events have been useful in making sure systems can talk to each other at the protocol level. But these interops alone haven’t been sufficient to ensure that ARC works as expected through all email routing scenarios. This is where the ARC test suite has proven useful in identifying specific issues and potential bugs to focus on at each interop. There are also numerous mature open source libraries that support ARC:

• C

• Python

• Perl

And adoption of ARC has already begun. Google and Fastmail are currently using it publicly, and other operators are testing it internally, with public rollouts to come. Sympa already has ARC support, and Mailman3 merged in ARC support in mid-June 2019.

The journey to RFC 8617 included extensive investigation and interop amongst the ecosystem over the course of several years. Early implementations made a large difference for participants in understanding their internal mail flows. 

Brandon Long of Google, co-editor of RFC 8617, says, “ARC allows authorization information to flow when internal routing is particularly complex, solving previously intractable authorization problems with multi-tenant, multi-party email routing. Further, this is even useful to preserve authentication information within a single administrative management domain (ADMD), especially when messages traverse virtual ADMDs within a single system. ARC has also proven useful in preventing messages from improperly gaining authorization by transiting through a permissively allowed forwarder.” 

We expect these benefits in turn to make an enormous difference to the ecosystem generally, and improve the amount of authenticated mail that is received overall.

Finally, in addition to improving the reliability and ubiquity of authentication, RFC 8617 will have tangible benefits for end users. Mailing lists that have inconsistent reply-to behaviors, perform “header munging,” or have home-grown behaviors, such as how the IETF lists behave, will be able to return to original behavior after ARC adoption grows. And people who have been using personal email addresses to participate on mailing lists (because their work addresses are at DMARC p=reject or p=quarantine) will find that they can now use work email addresses. Once ARC is in place and widely supported, you’ll be able to use mailing lists normally, even from addresses with DMARC enforcement, so long as the mailing list and your receiving system both support ARC. This should resolve a major headache that the IETF and other major industry lists have been having ever since DMARC was published.

Next Steps

Receiving systems and Internet mail handlers need to adopt ARC in order for this protocol to make a meaningful difference. And a world of software is needed to make it easy for these systems to participate. If you run these systems or work on software to support them, please get ARC on your roadmap and contribute your learnings back to the DMARC Working Group!

About the author

Seth Blank is Secretary of the IETF Domain-based Message Authentication, Reporting & Conformance Working Group, Co-Chair of the M3AAWG Collaboration Committee, and Director of Industry Initiatives at Valimail, where he leads initiatives to strengthen and broaden the reach of email authentication technologies, working with industry organizations to get better tools for email authentication and security into the hands of everyone to make the world a safer place.

Photo by Bryson Hammer