Subject: [dmarc-ietf] nits in draft-ietf-dmarc-aggregate-reporting-02

From: Martin Kealey <[email-masked]>
Date: Fri, 7 May 2021 15:23:43 +1000

I'm not quite sure how I'm supposed to submit nitpickery like this, so if
there's a better forum please let me know.

1. Filename & content-type

Section 2.6.1 among other things says that the name for the mime-part
containing the report MUST end with ".xml" or ".xml.gz", yet the example
given ends with neither of those (it ends with just ".gz").

The main use for this is as a unique report identifier; its use as a
filename is entirely secondary and only relevant to manual processing by a
human, so MUST seems quite excessive.

It seems like there are separate drivers for each part of the filename
suffix, and perhaps they should be two independent SHOULD requirements. If
we want to facilitate its use as a filename, perhaps we should just say
that the filename SHOULD be universally unique and MUST NOT contain "/" or
start with ".".

It seems strange to vary the content-type based solely on what amounts to a
transport optimization, namely gzip; this smells of working around
deficiencies in other standards. (From the perspective of an application
using email as a transport, it would seem to make more sense to allow
"content-transfer-encoding" to be a chain such as "base64+gzip", or
alternatively, for "content-type" to accept the addition of a "gzip/"
prefix, forming "gzip/text/xml". However I digress, as that's a discussion
for an entirely different standards track.)

According to rfc 7303 §9.2 <https://tools.ietf.org/html/rfc7303#section-9.2>,
the "text/xml" content-type is merely an alias for "application/xml". Other
standards such as related documents by w3c
<https://www.w3.org/2006/02/son-of-3023/draft-murata-kohn-lilley-xml-04.html#textxml>
go further in actively declaring it deprecated.

It seems to me that rfc7303 §4.2
<https://tools.ietf.org/html/rfc7303#section-4.2> and rfc6838 §4.2.5
<https://tools.ietf.org/html/rfc6838#section-4.2.5> taken together indicate
that registration of a content type such as *application/dmarc-feedback+xml*
would be appropriate.

2. Size limit

I'm concerned that specifying the maximum report size *after* compression
is possibly focussing on the wrong costs, and distorts the conceptual model:

   1. It implies that the *compressed* file is the relevant artefact being
   transported, which leads to the weirdness with filenames and content-types
   mentioned above.
   2. The size of the report is trivial compared with the size of the
   messages it's reporting on, both in terms of storage and bandwidth, and
   gzip decompression is very cheap, so compression makes negligible
   difference to those costs.
   3. The cost of processing the received report to incorporate it into the
   bulk reporting correlates more closely with its "uncompressed" size. In
   particular, the memory footprint of the receiver process is likely to be
   correlated with this limit, especially if its first step is to build an
   in-memory DOM from the XML. (I would be surprised if any real
   report-accepting system *didn't* work this way.)

3. Scheduling

Concern about processing load also brings me to section 2.4.2, which
essentially directs everyone to send their reports simultaneously. Since
the receiver needs to be able handle reports with any reporting period, it
seems likely that having most but not all reports arriving at the same time
would be the worst outcome, needing both (a) complex coding to cope with
asynchronous reporting, and (b) having to cope with high load spikes (or
suffer delays with the reports spooled for batch processing).

It also imposes a load spike on the report generators, to generate all
their reports at once (or spool and delay), but at least they can derive
some benefit from not having overlapping reporting periods.

In the scheme of things this isn't a huge load compared with the actual
processing of email, it seems like it would be preferable to allow the
receiver to specify their preference in this regard, at least to choose
between "UTC synchronized" and "randomized". Or for this document to
specify "randomized" as the default.

-Martin

---

From: "John Levine" <[email-masked]>
Date: 7 May 2021 11:10:06 -0400

It appears that Martin Kealey  <[email-masked]> said:
>-=-=-=-=-=-

The example is wrong.  It should obviously be .xml.gz

>deficiencies in other standards.

It does, but that's how we've been doing compressed MIME forever.

Early on, the DMARC drafts said to commpress with ZIP and send files
ending in .zip as application/zip. A lot of reporters still do that so
the draft needs to say to accept those even though nobody, not even
Google, should be generating them any more.

>is possibly focussing on the wrong costs, and distorts the conceptual model:

The problem is that many mail systems have a maximum message size
they will accept.  It's true that the expanded XML can be pretty big but
on modern computers, a few gigabytes of RAM are not a big deal.

>essentially directs everyone to send their reports simultaneously. ...

It does, but as far as I know it is not a problem in practice.

I could see getting rid of the 0000 UTC advice since I don't see that
it solves any real problem.

R's,
John

---

From: "Brotman, Alex" <[email-masked]>
Date: Fri, 7 May 2021 15:10:02 +0000

Martin,

  1.  I’ll see if we can get this cleaned up (I’ll create a proper ticket for tracking this)
  2.  I’d welcome other inputs here on the original idea for this option.  I would imagine modern systems would be able to deal with rather large XML files, though MTAs routinely set limits under 50M for accepting messages.
  3.  I don’t think it suggests we should all send at the same time (Unless I’m reading a different section). It suggests that the report producer should create reports on the same UTC boundaries.  For example, we do abide by the day boundary, but our reports are generated a few hours after 0000UTC (and delivered upon completion).  If you’d like, I can put a clarifying note into the document.

--
Alex Brotman
Sr. Engineer, Anti-Abuse & Messaging Policy
Comcast

From: dmarc <[email-masked]> On Behalf Of Martin Kealey
Sent: Friday, May 7, 2021 1:24 AM
To: [email-masked]
Subject: [dmarc-ietf] nits in draft-ietf-dmarc-aggregate-reporting-02

I'm not quite sure how I'm supposed to submit nitpickery like this, so if there's a better forum please let me know.

Section 2.6.1 among other things says that the name for the mime-part containing the report MUST end with ".xml" or ".xml.gz", yet the example given ends with neither of those (it ends with just ".gz").

The main use for this is as a unique report identifier; its use as a filename is entirely secondary and only relevant to manual processing by a human, so MUST seems quite excessive.

It seems like there are separate drivers for each part of the filename suffix, and perhaps they should be two independent SHOULD requirements. If we want to facilitate its use as a filename, perhaps we should just say that the filename SHOULD be universally unique and MUST NOT contain "/" or start with ".".

It seems strange to vary the content-type based solely on what amounts to a transport optimization, namely gzip; this smells of working around deficiencies in other standards. (From the perspective of an application using email as a transport, it would seem to make more sense to allow "content-transfer-encoding" to be a chain such as "base64+gzip", or alternatively, for "content-type" to accept the addition of a "gzip/" prefix, forming "gzip/text/xml". However I digress, as that's a discussion for an entirely different standards track.)

According to rfc 7303 §9.2<https://urldefense.com/v3/__https:/tools.ietf.org/html/rfc7303*section-9.2__;Iw!!CQl3mcHX2A!RFC9J_lhPahv_fk_lJ87nHfTTbdL4ck7ruqYS3RZJXmOfkOe73AOD1_nSftMp7jrqYjm$>, the "text/xml" content-type is merely an alias for "application/xml". Other standards such as related documents by w3c<https://urldefense.com/v3/__https:/www.w3.org/2006/02/son-of-3023/draft-murata-kohn-lilley-xml-04.html*textxml__;Iw!!CQl3mcHX2A!RFC9J_lhPahv_fk_lJ87nHfTTbdL4ck7ruqYS3RZJXmOfkOe73AOD1_nSftMp3d4W5dH$> go further in actively declaring it deprecated.

It seems to me that rfc7303 §4.2<https://urldefense.com/v3/__https:/tools.ietf.org/html/rfc7303*section-4.2__;Iw!!CQl3mcHX2A!RFC9J_lhPahv_fk_lJ87nHfTTbdL4ck7ruqYS3RZJXmOfkOe73AOD1_nSftMp6dQnJve$> and rfc6838 §4.2.5<https://urldefense.com/v3/__https:/tools.ietf.org/html/rfc6838*section-4.2.5__;Iw!!CQl3mcHX2A!RFC9J_lhPahv_fk_lJ87nHfTTbdL4ck7ruqYS3RZJXmOfkOe73AOD1_nSftMpzTRLTLD$> taken together indicate that registration of a content type such as application/dmarc-feedback+xml would be appropriate.

2. Size limit

I'm concerned that specifying the maximum report size after compression is possibly focussing on the wrong costs, and distorts the conceptual model:

  1.  It implies that the compressed file is the relevant artefact being transported, which leads to the weirdness with filenames and content-types mentioned above.
  2.  The size of the report is trivial compared with the size of the messages it's reporting on, both in terms of storage and bandwidth, and gzip decompression is very cheap, so compression makes negligible difference to those costs.
  3.  The cost of processing the received report to incorporate it into the bulk reporting correlates more closely with its "uncompressed" size. In particular, the memory footprint of the receiver process is likely to be correlated with this limit, especially if its first step is to build an in-memory DOM from the XML. (I would be surprised if any real report-accepting system didn't work this way.)

3. Scheduling

Concern about processing load also brings me to section 2.4.2, which essentially directs everyone to send their reports simultaneously. Since the receiver needs to be able handle reports with any reporting period, it seems likely that having most but not all reports arriving at the same time would be the worst outcome, needing both (a) complex coding to cope with asynchronous reporting, and (b) having to cope with high load spikes (or suffer delays with the reports spooled for batch processing).

It also imposes a load spike on the report generators, to generate all their reports at once (or spool and delay), but at least they can derive some benefit from not having overlapping reporting periods.

In the scheme of things this isn't a huge load compared with the actual processing of email, it seems like it would be preferable to allow the receiver to specify their preference in this regard, at least to choose between "UTC synchronized" and "randomized". Or for this document to specify "randomized" as the default.

-Martin

---

From: "John Levine" <[email-masked]>
Date: 7 May 2021 12:41:13 -0400

It appears that Brotman, Alex <[email-masked]> said:
>  2.  I’d welcome other inputs here on the original idea for this option.  I would imagine modern systems would be able to deal with rather
>large XML files, though MTAs routinely set limits under 50M for accepting messages.

I suggested an option to deliver reports by https POST or PUT, like MTA-STS does,
with precious little interest, even though it's a much more efficient way to ship
large files around since it doesn't need base64 encoding and doesn't relay.

This suggests that the size limits are not a problem, at least not one
people think is worth fixing.

>  3.  I don’t think it suggests we should all send at the same time (Unless I’m reading a different section). It suggests that the report
>producer should create reports on the same UTC boundaries.  For example, we do abide by the day boundary, but our reports are generated a few
>hours after 0000UTC (and delivered upon completion).  If you’d like, I can put a clarifying note into the document.

I read it the way he did.  I suppose a comment like "but wait a random number of hours after 0000 before you actually send
the report" would do the trick.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 7 May 2021 12:13:44 -0700

On Fri, May 7, 2021 at 9:46 AM John Levine <[email-masked]> wrote:

> >  2.  I’d welcome other inputs here on the original idea for this
> option.  I would imagine modern systems would be able to deal with rather

I agree that it's optimal.  But long ago (in DKIM, at least) we took note
of the fact that mail people and DNS people in large organizations are
often not the same teams and sometimes it's hard for one to get something
out of the other.  If that logic also holds for mail people and web people,
I imagine the lack of interest here has a similar basis; we're talking
about standing up a whole service or endpoint here, not just adding records
to a zone file.

-MSK

---

From: "John R Levine" <[email-masked]>
Date: 7 May 2021 15:22:21 -0400

On Fri, 7 May 2021, Murray S. Kucherawy wrote:
> [ mail and web departments don't talk to each other ]
> If that logic also holds for mail people and web people,

That's true but there's also the fact that a handful of analysis services 
like dmarcian and valimail collect a large fraction of DMARC reports.  If 
a few of them were interested in https reports, it'd be worth adding.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sat, 8 May 2021 11:13:43 -0700

> a few of them were interested in https reports, it'd be worth adding.

Ah, that's a good point.

I think both of them are represented here, and maybe some other report
processors.  Any feedback on this from them?

-MSK

---

From: Seth Blank <[email-masked]>
Date: Sat, 8 May 2021 11:22:07 -0700

(With Chair hat off and Valimail hat on)

Email works fine for reports, even huge ones at scale. HTTPS adds nothing
for us and adds complexity to report processing (multiple paths to
ingestion) that we’d rather avoid.

Of course, were HTTPS added to the spec, we’d implement it.

Seth

On Sat, May 8, 2021 at 11:14 Murray S. Kucherawy <[email-masked]>
wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>
-- 

*Seth Blank* | VP, Product
*e:* [email-masked]
*p:* 415.273.8818

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: "John Levine" <[email-masked]>
Date: 8 May 2021 14:59:03 -0400

Just out of nosinesss, how big are the big reports?  Do people put
size limits in the URIs and if so do report senders follow them?

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly