Subject: [dmarc-ietf] Definition of "value" in RFC8601

From: Damian Lukowski <[email-masked]>
Date: Sat, 28 Mar 2020 11:39:59 +0100

Hello,

RFC8601 section 2.2 defines

> authserv-id = value
>             ; see below for a description of this element
> [...]
> The "value" is as defined in Section 5.1 of [MIME], with
> "quoted-string" updated as specified in [RFC6532].
[MIME] (RFC2045) section 5.1

> value := token / quoted-string
> token := 1*<any (US-ASCII) CHAR except SPACE, CTLs,
>             or tspecials>

The token is not said to be updated by [RFC6532] as is quoted-string, so
by my interpretation non-ASCII cannot be used without quotes.

If so, then current implementations of OpenDKIM and OpenDMARC are wrong
or imcomplete, as they do not check for Non-ASCII AuthservID and produce
headers like

> Authentication-Results: öde; dkim=none; dkim-atps=neutral
Is my understanding correct?

Regards
 Damian

---

From: Brandon Long <[email-masked]>
Date: Mon, 30 Mar 2020 13:57:57 -0700

Hmm, we didn't include this in RFC 8616 either, I could imagine that it
should be punycoded also, though it really depends on whether in 6532 or
5322.

Brandon

On Sat, Mar 28, 2020 at 3:39 AM Damian Lukowski <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Damian Lukowski <[email-masked]>
Date: Mon, 30 Mar 2020 23:09:03 +0200

> should be punycoded also, though it really depends on whether in 6532 or
Why punycoded? Section 2.5 does not specify that it needs to be a domain
name, even though it usually is.

---

From: Brandon Long <[email-masked]>
Date: Mon, 30 Mar 2020 14:13:50 -0700

if your admd is an IDN, and you had to include that in 5322, then
specifying to do so in punycode would make sense.

Just including it in double quotes in a 5322 message would not be correct,
so then you're down to which mechanism to use, I don't think 2045-style
would be correct... or 2231.

But yes, as its not specified as a domain or domain-like, it would be odd
to specify a domain encoding.

Brandon

---

From: "John Levine" <[email-masked]>
Date: 30 Mar 2020 17:23:11 -0400

In article <[email-masked]> you write:
>-=-=-=-=-=-

This is another mistake in the ABNF in 8601.  The point of 6532 is
that with the exception of Message-IDs, any header fields that used to
be limited to ASCII can now contain UTF-8 printable characters.

If you're putting an A-R header on an EAI message, it has to allow 
UTF-8 since lots of the fields in an A-R header can contain mailboxes
and other UTF-8 strings. 

I realize that the authserv-id is usually a domain name, in which case
it would be reasonable to represent it as A-labels, but it doesn't
have to be.

The only, and I mean ONLY, strings that are "punycoded" are the
characters in IDN labels that follow xn--.  To pick a common point of
confusion, you absolutely cannot "punycode" a mailbox name.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 30 Mar 2020 15:38:51 -0700

On Mon, Mar 30, 2020 at 2:23 PM John Levine <[email-masked]> wrote:

> In article <
> [email-masked]> you
> write:
> >-=-=-=-=-=-
> >

Does someone have a fix in mind that could be submitted as an erratum?  The
intent was indeed to make the authserv-id either a plain old ASCII domain
name or an A-label which doesn't need quoting.  I missed that RFC 6532
didn't update "value", unfortunately.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 30 Mar 2020 19:21:18 -0400

> didn't update "value", unfortunately.

Unfortunately, 8601's defintion is a mess.  At the bottom of page 15 it 
says this about authserv-id, which contradicts the ABNF:

    Note that in an EAI-formatted message, this identifier may be
    expressed in UTF-8.

If you want it to be a domain, the ABNF should say:

      authserv-id = domain-name

The text below says domain-name is imported from RFC 6376, which I presume 
includes the updates from RFC 8616 that encourage U-labels in EAI 
messages.

This is not strictly backward compatible with the current text, but I 
don't think I've ever seen an authserv-id which wasn't syntactically a 
domain name.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 30 Mar 2020 22:43:34 -0700

> says this about authserv-id, which contradicts the ABNF:

OK, what does the WG think might be the best way to clean it up?

-MSK

---

From: Damian Lukowski <[email-masked]>
Date: Tue, 31 Mar 2020 08:23:04 +0200

> The intent was indeed to make the authserv-id either a plain old ASCII
> domain name or an A-label which doesn't need quoting.  I missed that RFC
> 6532 didn't update "value", unfortunately.

I don't understand. If the intent is to have an old ASCII domain, or
A-labels (still ASCII), then why is it relevant that "value" has not
been updated to include real UTF-8?

Shall authserv-id contain UTF8-non-ascii or not?

Damian

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 31 Mar 2020 11:55:02 -0400

> don't think I've ever seen an authserv-id which wasn't syntactically a

I've seen bare host names used.  I guess that's syntactically a domain name 
also, but might not be what you expect if you assume it's really a domain 
name.  It probably doesn't matter.

Scott K

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 31 Mar 2020 08:55:32 -0700

> OK, what does the WG think might be the best way to clean it up?

 I think that John's suggestion (ABNF: " authserv-id = domain-name ") makes
the most sense and the fewest contortions.

--Kurt

---

From: Seth Blank <[email-masked]>
Date: Tue, 31 Mar 2020 09:28:26 -0700

On Tue, Mar 31, 2020 at 8:56 AM Kurt Andersen (b) <[email-masked]> wrote:

>  I think that John's suggestion (ABNF: " authserv-id = domain-name ")
> makes the most sense and the fewest contortions.
>

(as an individual)

I've definitely seen non domain-name based authserv-ids, but they are rare
and tend to be from opendmarc. We can certainly fix that in opendmarc.

I do think John's suggestion is the most straightforward, and I'm for it,
but it does constitute a breaking change.

Seth

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2020 12:56:51 -0400

>name.  It probably doesn't matter.

I don't see any reason to exclude them.  We could do this:

      autbserv-id     = sub-domain *("." sub-domain)

Where sub-domain is imported from 5321 for ASCII mail and 6531 for EAI mail.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2020 12:58:10 -0400

>Shall authserv-id contain UTF8-non-ascii or not?

The intention in 8601 is pretty clear, even though the ABNF doesn't agree:

2.5.  Authentication Service Identifier Field

   Every Authentication-Results header field has an authentication
   service identifier field (authserv-id above).  Specifically, this is
   any string intended to identify the authentication service within the
   ADMD that conducted authentication checks on the message.  This
   identifier is intended to be machine-readable and not necessarily
   meaningful to users.

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2020 13:00:40 -0400

>and tend to be from opendmarc. We can certainly fix that in opendmarc.

My proposal says they're syntactically domain names, not that they
actually have to exist in the DNS.  What do those ID's look like?

The only things I can think of likely to fail would be exotica like
foo..bar that can't be parsed into DNS labels.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 31 Mar 2020 17:07:38 +0000

On March 31, 2020 4:56:51 PM UTC, John Levine <[email-masked]> wrote:
>      authserv-id     = sub-domain *("." sub-domain)
>

That does allow IP address literals.  Do we want that?

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 31 Mar 2020 19:36:50 +0200

> didn't update "value", unfortunately.

That seems to be an RFC 6532 Erratum.

By allowing UTF-8 in MIME tokens, we preserve the spirit of RFC 6532, maintain
that the definition in RFC 8601 is valid, and the headers reported by the OP
turn out to be valid, as they should be, shouldn't they?

Best
Ale
--

---

From: Damian Lukowski <[email-masked]>
Date: Tue, 31 Mar 2020 19:43:34 +0200

> The intention in 8601 is pretty clear, even though the ABNF doesn't agree:

Yes, the RFC says the authserv-id can contain UTF-8, but its author
stated that he intended it to be "either a plain old ASCII domain name
or an A-label". One does not need UTF-8 for that. If authserv-id were not

> authserv-id = value

but

> authserv-id = token ; token from RFC2045

wouldn't the author's intent have been covered as well, without any
reference to RFC6531/6532?

Damian

---

From: Damian Lukowski <[email-masked]>
Date: Tue, 31 Mar 2020 19:55:36 +0200

> Where sub-domain is imported from 5321 for ASCII mail and 6531 for EAI mail.

Does this mean that a parser would need to know if the message has been
sent with SMTPUTF8 in order to pass or refute U-labels?

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Tue, 31 Mar 2020 12:11:40 -0700

> sent with SMTPUTF8 in order to pass or refute U-labels?

You don't need SMTPUTF8 to handle IDNs.

--Kurt

---

From: Damian Lukowski <[email-masked]>
Date: Tue, 31 Mar 2020 21:30:03 +0200

> You don't need SMTPUTF8 to handle IDNs.

I'm not sure if this answers my question. A parser for an ABNF grammar
needs to know the grammar. If the grammar is not fixed but depends on
external factors, the parser also needs to know the external factors. In
EML files that external factor is unavailable.

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2020 20:00:00 -0400

>That does allow IP address literals.  Do we want that?

No, take a look at RFC 5321 and search for "address-literal".  You'll see
they're always an alternative to a domain.  Sub-domain is trivial:

   sub-domain     = Let-dig [Ldh-str]

   Ldh-str        = *( ALPHA / DIGIT / "-" ) Let-dig

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 1 Apr 2020 00:02:13 -0000

In article <[email-masked]>,
>>     sent with SMTPUTF8 in order to pass or refute U-labels?

The definition of an EAI message is separate from SMTPUTF8.

As a close approximation, any message which has an x80 bit set in a
character in the header is an EAI message.

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 01 Apr 2020 00:06:02 +0000

On April 1, 2020 12:00:00 AM UTC, John Levine <[email-masked]> wrote:
>>>Where sub-domain is imported from 5321 for ASCII mail and 6531 for
>EAI
>>>mail.
>>

Odd.  In RFC 6376 it's:

      domain-name     = sub-domain 1*("." sub-domain)
                        ; from [RFC5321] Domain,
                        ; excluding address-literal

I wonder why it excludes address-literal if they are already excluded?

Scott K

---

From: "John Levine" <[email-masked]>
Date: 31 Mar 2020 20:11:36 -0400

>reference to RFC6531/6532?

You'll have to ask Murray what he meant, although as we've seen section 2.5
specifically says that an EAI authserv-id "may be expressed in UTF-8."

In an EAI message, an A-R header has to allow UTF-8 characters in
mailboxes, so it might as well allow them in other places including
domain names and the authserv-id.

R's,
John

PS: for anyone who might suggest one could "punycode" a mailbox, don't even think about it.

---

From: Damian Lukowski <[email-masked]>
Date: Wed, 1 Apr 2020 08:21:43 +0200

>    ;  Where sub-domain is imported from 5321 for
>    ; ASCII mail and 6531 for EAI mail

As the authserv-id is part of the header, this boils down to:

"If the authserv-id is UTF-8, then it might be UTF-8"

This seems equivalent to

>    ; Where sub-domain is imported from 6531 for
>    ; any mail