Subject: [dmarc-ietf] SPFAuthResultType unbounded

From: Tomki <[email-masked]>
Date: Tue, 15 Mar 2016 15:27:15 -0700

Does it make sense that SPFAuthResultType element counts are allowed to be unbounded?
I would think that it should be a maximum of 2, and then only if the scope is indicated (helo/mfrom)

from https://tools.ietf.org/html/rfc7489

    <xs:complexType name="AuthResultType">
      <xs:sequence>
        <!-- There may be no DKIM signatures, or multiple DKIM
             signatures. -->
        <xs:element name="dkim" type="DKIMAuthResultType"
          minOccurs="0" maxOccurs="unbounded"/>
        <!-- There will always be at least one SPF result. -->
        <xs:element name="spf" type="SPFAuthResultType" minOccurs="1"
                    maxOccurs="unbounded"/>
      </xs:sequence>
    </xs:complexType>

--Tomki

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

---

From: Franck Martin <[email-masked]>
Date: Tue, 15 Mar 2016 18:10:23 -0500 (CDT)

----- Original Message -----

> From: "Tomki" <[email-masked]>
> To: "dmarc" <[email-masked]>
> Sent: Tuesday, March 15, 2016 3:27:15 PM
> Subject: [dmarc-ietf] SPFAuthResultType unbounded

> from https://tools.ietf.org/html/rfc7489 <xs:complexType
> name="AuthResultType">
> <xs:sequence>

Makes sense, does it matter?

---

From: Tomki <[email-masked]>
Date: Tue, 15 Mar 2016 23:35:02 -0700

Yes, it does matter. Because one very large DMARC reporter has read this 
item in the specification's XML, and actually *is* providing far more 
than 2 SPF results per record object. I've counted 312 <spf> blocks in 
one instance.  I'm not clear on the logic behind that, perhaps it's an 
aggregation level.  But I don't grok it, and I want to either understand 
how it makes sense, or fix the spec to be entirely clear.

On 3/15/16 16:10, Franck Martin wrote:
>
>
>
>
>
> ------------------------------------------------------------------------
>
>     *From: *"Tomki" <[email-masked]>
>     *To: *"dmarc" <[email-masked]>
>     *Sent: *Tuesday, March 15, 2016 3:27:15 PM
>     *Subject: *[dmarc-ietf] SPFAuthResultType unbounded
>
>     fromhttps://tools.ietf.org/html/rfc7489
>
> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc

---

From: Elizabeth Zwicky <[email-masked]>
Date: Wed, 16 Mar 2016 07:27:04 -0700

Rows are defined by IP; if the same IP uses multiple MAILFROM and SPF is bounded what is the reporter supposed to do? Duplicate rows? 

Limiting SPF changes the row key in bad ways. (Unless all senders are well-behaved in ways they are not required to be.)

Elizabeth

[email-masked]

> On Mar 15, 2016, at 4:10 PM, Franck Martin <[email-masked]> wrote:
> 
> 
> 
> 
> 
>

---

From: Les Barstow <[email-masked]>
Date: Wed, 16 Mar 2016 11:00:14 -0600

The question then is - how does a report receiver interpret the multiple
SPF results? A row has only a single auth message count; if there are three
SPF rows, then which SPF result goes with how many messages? It's ambiguous
(not to mention hard to render to an end user).

If an IP has multiple auth result combinations, shouldn't it have one row
per combination so that the report receiver can get an accurate idea of how
many instances of each combination are being reported?

--
Les Barstow

On Wed, Mar 16, 2016 at 8:27 AM, Elizabeth Zwicky <[email-masked]> wrote:

>
> Rows are defined by IP; if the same IP uses multiple MAILFROM and SPF is
> bounded what is the reporter supposed to do? Duplicate rows?
>
> Limiting SPF changes the row key in bad ways. (Unless all senders are
> well-behaved in ways they are not required to be.)
>
> Elizabeth
>
> [email-masked]
>
> ------------------------------
>

---

From: Franck Martin <[email-masked]>
Date: Wed, 16 Mar 2016 12:09:29 -0500 (CDT)

----- Original Message -----

> From: "Elizabeth Zwicky" <[email-masked]>
> To: "Franck Martin" <[email-masked]>
> Cc: "dmarc" <[email-masked]>, [email-masked]
> Sent: Wednesday, March 16, 2016 7:27:04 AM
> Subject: Re: [dmarc-ietf] SPFAuthResultType unbounded

> Elizabeth

Are you saying for the same (IP, RFC5322.From domain) tuple you have several RFC5321.Helo and RFC5321.MailFrom domains? 

It seems to me, if for the same tuple you have different DKIM d= with different results (pass,fail) you do create a row for each case. So same for SPF.... 

The record key is certainly multidimensional and not by IP alone.

---

From: Les Barstow <[email-masked]>
Date: Wed, 16 Mar 2016 11:38:15 -0600

To answer the first - we certainly receive reports from some providers
containing multiple SPF results within the same row. It makes things
"difficult"; without scope, there's no correct way to interpret the row.
But if you report one and only one of each scope (helo and mfrom), then the
normal SPF authentication rules apply - mfrom rules over helo when present.

With DKIM, a row could conceivably be accurate while containing multiple
DKIM results with different d= values if all of the messages were signed
with the same multiple DKIM signing keys.

--
Les Barstow

On Wed, Mar 16, 2016 at 11:09 AM, Franck Martin <[email-masked]>
wrote:

> *From: *"Elizabeth Zwicky" <[email-masked]>
> *To: *"Franck Martin" <[email-masked]>
> *Cc: *"dmarc" <[email-masked]>, [email-masked]
> *Sent: *Wednesday, March 16, 2016 7:27:04 AM
> *Subject: *Re: [dmarc-ietf] SPFAuthResultType unbounded
>
>
> Are you saying for the same (IP, RFC5322.From domain) tuple you have
> several RFC5321.Helo and RFC5321.MailFrom domains?
>
> It seems to me, if for the same tuple you have different DKIM d= with
> different results (pass,fail) you do create a row for each case. So same
> for SPF....
>

---

From: Tomki <[email-masked]>
Date: Wed, 16 Mar 2016 11:13:48 -0700

Les: right, I think you've seen the same problem I am describing.

Elizabeth: this is a problem that I have not identified at more than one 
provider, and don't worry you're not it.

For a concrete example, please see the attached XML record object.
There are 1834 messages reported in this object by 'count', but 312 SPF 
items reported.  I guess if these were all either pass or failure cases, 
and the counts lined up I could disambiguate properly, but as it stands 
I don't think it's comprehensible.

--
Tomki

On 3/16/16 10:00, Les Barstow wrote:
> The question then is - how does a report receiver interpret the 
> multiple SPF results? A row has only a single auth message count; if 
> there are three SPF rows, then which SPF result goes with how many 
> messages? It's ambiguous (not to mention hard to render to an end user).
>
> If an IP has multiple auth result combinations, shouldn't it have one 
> row per combination so that the report receiver can get an accurate 
> idea of how many instances of each combination are being reported?
>
> --
> Les Barstow
>
> On Wed, Mar 16, 2016 at 8:27 AM, Elizabeth Zwicky <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>     Rows are defined by IP; if the same IP uses multiple MAILFROM and
>     SPF is bounded what is the reporter supposed to do? Duplicate rows?
>
>     Limiting SPF changes the row key in bad ways. (Unless all senders
>     are well-behaved in ways they are not required to be.)
>
>     Elizabeth
>
>     [email-masked] <mailto:[email-masked]>
>
>     On Mar 15, 2016, at 4:10 PM, Franck Martin <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>>
>>
>>
>>
>>
>>         *From: *"Tomki" <[email-masked]
>>         <mailto:[email-masked]>>
>>         *To: *"dmarc" <[email-masked] <mailto:[email-masked]>>
>>     [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 17 Mar 2016 19:04:32 +0100

On Wed 16/Mar/2016 19:13:48 +0100 Tomki wrote: 
> There are 1834 messages reported in this object by 'count', but 312 SPF items
> reported.  I guess if these were all either pass or failure cases, and the
> counts lined up I could disambiguate properly, but as it stands I don't think
> it's comprehensible.

It looks a bug to me.  It contains results for 312 <spf> domains and 2 <dkim>
results for tomki.com.  The <spf> domains seem to be a mixture of "helo" and
"mfrom" (e.g. bounce.secureserver.net must have been "mfrom", since it passed;
the hundreds of domains like "p3plcpnl[0-9]+.prod.phx3.secureserver.net" must
have been "helo", unless there is some sort of helo-attack underway).

I agree with Les that rows ought to be split by result type.  The report
doesn't disclose how 1834 messages are distributed over 312 domains.

IMHO, *reporting the bug is better than tightening the specs*.

BTW, isn't there any monitoring service which sends a few email from a couple
of domains and verifies the aggregate feedback?  A semantic check would have to
send various messages with varying authentication methods from a few domains
with varying dmarc policies, and then verify that the reports from the target
domain are consistent with what was sent.  I checked out dmarcian and
dmarcanalyzer, but neither seems to do that.

Ale

---

From: Tomki Camp <[email-masked]>
Date: Thu, 17 Mar 2016 16:21:11 -0700

The bug was reported some time ago, and the spec was referenced as allowing
the questionable data+format.
I am of the opinion that tightening the spec is necessary in this case,
both to prevent future implementations from making the same
misinterpretation, and as reference for fixing the current problem.

--Tomki

On Thu, Mar 17, 2016 at 11:04 AM, Alessandro Vesely <[email-masked]> wrote:

> items

---

From: Franck Martin <[email-masked]>
Date: Fri, 18 Mar 2016 13:42:13 -0500 (CDT)

----- Original Message -----

> From: "Tomki Camp" <[email-masked]>
> To: "Alessandro Vesely" <[email-masked]>
> Cc: [email-masked]
> Sent: Thursday, March 17, 2016 4:21:11 PM

> I am of the opinion that tightening the spec is necessary in this case, both
> to prevent future implementations from making the same misinterpretation,
> and as reference for fixing the current problem.

I think you can submit an errata. 

https://www.rfc-editor.org/errata.php

---

From: Tomki Camp <[email-masked]>
Date: Fri, 18 Mar 2016 14:01:26 -0700

Ok, I can do that.
I'd like a further suggestion on how to specify more detail for this change:

-       <xs:element name="spf" type="SPFAuthResultType" minOccurs="1"
-                   maxOccurs="unbounded"/>
+       <xs:element name="spf" type="SPFAuthResultType" minOccurs="1"
+                   maxOccurs="2"/>

Because if there are more than just 1, both of the possible
SPFAuthResultType entries need to indicate the scope, and they must be
different. (i.e. one must be mfrom, the other must be helo)

On Fri, Mar 18, 2016 at 11:42 AM, Franck Martin <[email-masked]>
wrote:

> *From: *"Tomki Camp" <[email-masked]>
> *To: *"Alessandro Vesely" <[email-masked]>
> *Cc: [email-masked]
> *Sent: *Thursday, March 17, 2016 4:21:11 PM
> The bug was reported some time ago, and the spec was referenced as
> allowing the questionable data+format.

---

From: Franck Martin <[email-masked]>
Date: Fri, 18 Mar 2016 18:00:18 -0500 (CDT)

----- Original Message -----

> Cc: [email-masked], "Alessandro Vesely" <[email-masked]>
> Sent: Friday, March 18, 2016 2:01:26 PM

> Because if there are more than just 1, both of the possible SPFAuthResultType
> entries need to indicate the scope, and they must be different. (i.e. one
> must be mfrom, the other must be helo)

If not mistaken, in the spec, we say DMARC uses only the mform evaluation (rfc5321.mailfrom or rfc5321.helo if the first one is empty).

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 19 Mar 2016 11:55:42 +0100

>> + <xs:element name="spf" type="SPFAuthResultType" minOccurs="1"

Why not maxOccurs="1", after Franck's note below?  Since most feedback
producers supply one SPF result, I'd rather seek homogeneity than introduce
further intricacies.

> If not mistaken, in the spec, we say DMARC uses only the mform evaluation
> (rfc5321.mailfrom or rfc5321.helo if the first one is empty).

Exactly, see https://tools.ietf.org/html/rfc7489#section-4.1

The possibility to use helo if mfrom is void is described in the second
paragraph of http://tools.ietf.org/html/rfc7208#section-2.4

Ale
--