Subject: Re: [ietf-dkim] The problem with the DKIM design community

From: Alessandro Vesely <[email-masked]>
Date: Mon, 01 Jul 2013 18:53:24 +0200

On Sun 30/Jun/2013 15:21:29 +0200 Michael Deutschmann wrote:
> 
> "EDSP" would only pay attention to signatures where the "d=" matches
> the right hand side of the RFC821 MAIL FROM:.
> 
> This means that someone can publish the strictest possible EDSP
> without causing mailing list false positives.  Mailing lists take
> ownership of the MAIL FROM:, hence only an EDSP set by the list itself
> will apply, and the original poster's EDSP will be correctly ignored.
> Just like in SPF.
> 
> Of course, since the MAIL FROM: is usually not visible without pressing a
> "show all headers" button, this would be more about leaving a clearer
> audit trail than actually foiling phishes.

Well, not really.  MAIL FROM: is only visible after delivery, so to
avoid dangling signatures one should store its value in some other
header field or... in the i= tag.  Heck, is that the semantics that the
OP was talking about?

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Michael Deutschmann <[email-masked]>
Date: Mon, 1 Jul 2013 12:24:25 -0700 (PDT)

On Mon, 1 Jul 2013, Alessandro Vesely wrote:
> header field or... in the i= tag.

ITYM "only visible *before* delivery"

There's long been a standard header for that purpose, "Return-path:".
But EDSP checking in the MUA is not very useful -- if the Return-path is
disavowed, then bouncing is an even worse idea than normal.  The only
place to act is in the SMTP transaction at the border.

It does mean that if the mail passes through an SPF Sender Rewriting
Scheme forwarder, then it will end up with an unbroken but irrelevant
signature.  Even if that forwarder knows about EDSP, it can't strip the
signature because it can't know that it isn't there to serve a different
accessory protocol yet to be invented.  After all, most of the time MAIL
FROM: = From:, so the signature added for the sake of EDSP will
simultaneously be serving ADSP or DMARC.

But I don't think that's a problem.  The message will get through,
because the forwarder now owns the MAIL FROM and it's up to him whether an
EDSP check is needed.

Also, if any DKIM accessory protocol will false positive OR false negative
due to the presence of irrelevant (ie: not the d= you're looking for)
signatures, then that accessory protocol is really broken.

Of course, it's always correct for an accessory protocol to dismiss a
message as "unsigned" if the irrelevant signatures the only signatures
present.  Any other behavior would fall under "false negative due to
irrelevant signatures".

---- Michael Deutschmann <[email-masked]>
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 1 Jul 2013 22:59:30 -0700

On Sun, Jun 30, 2013 at 9:21 AM, John R. Levine <[email-masked]> wrote:

> What I'm asking for is nothing like SES.  I want the signature to be
>> based on the envelope MAIL FROM:, but it is still the body that gets
>> signed.  No VERPing is called for. ...
>>
>
> Where can we read the draft?
>
>
+1.  I pledge open source support if a workable draft appears.

-MSK

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 02 Jul 2013 12:13:27 +0200

(subject adjusted)

A sender using SRS would need to maintain a database of valid addresses.
However, that task can become unduly complicated if the database has to
be kept in sync across several distant hosts.  A digital signature can
substantially complement the security of the necessarily-too-short hash
added to an SRS address, making it possible to avoid the database
entirely, except for address length worries.  That's where EDSP can save
the day.

> ITYM "only visible *before* delivery"

It has to be in the message content for DKIM to be applicable.

> There's long been a standard header for that purpose, "Return-path:".

The signature the OP posted had h=content-type:mime-version:subject:
list-unsubscribe:reply-to:to:from:date:message-id, so the sender had to
stuff it in i=.

> place to act is in the SMTP transaction at the border.

It is not very useful to the SMTP receiver either.  The only value added
is the d= domain supplied by EDSP.  SPF leaves the receiver
wondering about where the cut lies, but that's a non-actionable datum
anyway, for the time being.

> present.  Any other behavior would fall under "false negative due to

Yes.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 2 Jul 2013 07:11:20 -0700

> ITYM "only visible *before* delivery"

He means "after".  There is no guarantee that the MAIL FROM address appears
anywhere in the signed content of the message; its addition to Received is
non-standard, and although RFC5321 says the addition of Return-Path at the
time of delivery is mandatory, there are some legacy systems that don't
insert it.  If the sender inserts it, it could be removed or replaced in
transit or upon delivery, invalidating the signature.

One could do what you're talking about by inventing a DKIM canonicalization
that includes the MAIL FROM address in one of the two hashes DKIM generates
to produce its signature.  That's easy enough.  I'd like to know what the
gain is, however.  As far as I can tell, by itself, that simply ensures the
same content re-injected anywhere will not produce a "valid" result unless
the MAIL FROM is unchanged.

It seems to me this renders your scheme even more sensitive to failures
than DKIM already is.  Specifically, a mailing list server that resends the
message byte-for-byte identical to the original and only changes the
envelope will cause the signature to be invalid, while DKIM will survive
such re-mailing.

> simultaneously be serving ADSP or DMARC.

There are legitimate cases where this is not true, such as mailing lists
(which was your original complaint about "accessory protocols").

> because the forwarder now owns the MAIL FROM and it's up to him whether an

The forwarder would have to be EDSP-aware and re-sign the message when
changing the envelope.  That makes a lot of assumptions about all the hosts
through which the message will pass.

-MSK

---

From: Michael Deutschmann <[email-masked]>
Date: Tue, 2 Jul 2013 08:37:20 -0700 (PDT)

On Tue, 2 Jul 2013, Alessandro Vesely wrote:
> [...] That's where EDSP can save the day.

That's off in the weeds.  EDSP would not take any notice of i=, and is
not there to enhance SRS -- rather it's something of a competitor.  (Both
try to make return path validation work in spite of forwarding.)

I'll note however that nothing done in the body or headers can bypass the
length limits on VERPing (SRS being a subset).  The whole reason VERPing
is popular is because no one can count on a bounce message being in a
standard format.  You might get no recoverable information about what was
*in* the failed message, but the MAIL FROM: is guaranteed to come back to
you verbatim as the envelope RCPT TO: of the bounce.

I just mentioned SRS as an innocent example of a message being relayed
with verbatim body and headers but an altered envelope sender.  And while
that case could result in couple hundred bytes of wasted space in the
header, it won't cause false-positives or false-negatives.  So it's not
worth losing sleep over.

> It has to be in the message content for DKIM to be applicable.

Core DKIM is only tasked with determining if a signature is genuine, not
if the signature is relevant.   Therefore it doesn't matter if part of the
information EDSP uses to determine relevancy is out of band.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 02 Jul 2013 19:48:54 +0200

> try to make return path validation work in spite of forwarding.)

The point is what any of them might be useful for.

> information EDSP uses to determine relevancy is out of band.

Yes, one just needs to maintain his own software to run it :-/

BTW, there is already a hack in how that's implemented, because they
used no l=0 tag.  So, if the bounce they get has text/rfc822-headers
only, they have to "assume" that the body hash matches.  Perhaps, they
reasoned that they can still verify the SRS hash, and that an assumed
but variable bh= is still better than the constant body hash specified
for l=0.  Depending on what library they use, implementing that could be
as simple as checking the return code.

---

From: Hector Santos <[email-masked]>
Date: Tue, 02 Jul 2013 13:38:36 -0400

On 7/2/2013 10:11 AM, Murray S. Kucherawy wrote:
> He means "after".

Actually, I believe he meant BEFORE.  It (old school UUCP/SLIP "From " 
and SMTP  "Return-Path:") can be pulled/stripped once the status of 
delivery is determined (i.e. the user exist) at the MDA and the backend 
stores the mail for user online NON-RFC based mail reading or NON-RFC 
and RFC based pickup. At that point, "From " and/or "Return-Path:" have 
no real remaining value for the transport - it has ended.

In fact, the only real reason to keep RFC headers is for MIME rendering. 
But overall, you only need four key headers for multiple mail 
network/systems display rendering, including non-RFC:

   From:
   To:
   Date:
   Subject:

For a network, a Reply-To: needs to be considered, otherwise useless for 
local communications.

I guess overall, don't depend on RFCs after the transport is completed. 
Not every system is going to use RFC formats across the aboard from end 
to end and if a RFC mail protocol is dependent on end to end RFC 
persistency, either it won't work very well or not everyone is going to 
use it or use it as much as they can within their mail framework.

Transformed with the body to a backend storage format, attachments 
extracted and stored separately, MIME/HTML rendering desires is the only 
reason RFC storage is used.  A local-only mail site certainly don't need 
all the RFC storage overhead and processing necessary.   Just consider 
the current most used "BBS" today - facebook.  No RFC required. While I 
won't be surprise it is used an RFC storage format, it doesn't need it 
from a design standpoint.  You only need the above field fields plus a 
body.   For the Facebook private email communications, of course, some 
level of RFCs is at its gateway, but I will not be surprise if its goes 
into a transformation at the FaceBook gateway so any DKIM calculations 
need to be done at the transport, in IMO.

Once received, a converter can take the key items necessary for storage 
to convey whatever ones want to convey.

Keep in mind one reason for using RFC is that it may allow a mail system 
growth HOWEVER for each new thing, it adds more complex overhead 
processing and then you have to make sure it is all integrated 
correctly.   What I mean  is that a SQL or fixed mail header block 
containing a field "DKIM-Status" is far more efficient than having to 
keep the RFC headers around to repeat the calculations.

--
HLS

---

From: Michael Deutschmann <[email-masked]>
Date: Tue, 2 Jul 2013 17:12:52 -0700 (PDT)

On Tue, 2 Jul 2013, Alessandro Vesely wrote:
> So, if the bounce they get has text/rfc822-headers only, they [...]

This is getting OT, but you can't even count on getting
text/rfc822-headers in a bounce.  I use Exim, a very popular MTA with the
latest stable release just 8 months old, and it doesn't give MIME bounces
*at all*.

But back to EDSP:

I still don't quite see how Return-path:'s special status is such a
problem.  I know that it's only generated from the envelope just before
being written to the mailbox, and never appears in the SMTP transaction
itself, and for that reason it cannot be *covered* by the signature.  But
it can still determine relevance.

If you were to change the From: field of a message signed to pass
ADSP/DMARC, you would make the signature bogus, and also make it
irrelevant if the new address is in a different domain.

If you change just the MAIL FROM: of a message signed to pass EDSP, you
would make the signature irrelevant but not bogus.

But I don't see how the above difference leads to any practical problem.

I suppose a forwarder or other MITM could change only the left-hand-side
of the MAIL FROM: and "get away with it".  But why would they be tempted
to do that?

---

From: Michael Deutschmann <[email-masked]>
Date: Sun, 14 Jul 2013 02:38:34 -0700 (PDT)

> The point is what any of them might be useful for.

It's about deployability.

Many mail filtering strategies are not deployable widely even assuming
the smartest possible mail admin and perfect MTA software.  They require
the cooperation of the end user in order to work properly.

I classify the strategies into three tiers.

On the first tier, you have things like Spamhaus ZEN, which can be
deployed across all users at a domain, without any knowledge of the
end-users habits.

On the second tier, you have things like Distributed Checksum
Clearinghouse, which require some information about the end-user's
habits in order to work properly.  For DCC and most other second-tier
methods, this is a complete whitelist of their mailing list
subscriptions.  Without the whitelist, DCC will correctly notice that
the mailing lists are bulk, and incorrectly assume that they are
unsolicited bulk.

On the third tier, you have things that require additional sacrifice (or
great arrogance...) on the part of the user.  One third-tier approach I
personally use is blocking all HTML e-mails (including
multipart/alternative).

(I get away with that because I give any corporate entities I actually
wish to hear from special canary-trap addresses that aren't so
filtered.)

Anti-forgery mail filtering methods are special in that they have
separate receiverside and senderside components.  Unless both components
are present, they will always false-negative.

(Note: below I'm counting "?all" SPF records as "not deploying SPF
senderside", and ADSP/DMARC policies that do not recommend rejecting
mail as "not deploying ADSP/DMARC senderside".  Those configurations may
be useful but don't facilitate an easy decision to reject an e-mail.)

ADSP and DMARC are tier 1 on the receiverside and tier 3 senderside.
Not many domains can make the claim that no mailbox in them participates
in any mailing list.

SPF is supposed to be tier 2 on the receiverside (needs a forwarder 
whitelist) and tier 1 on the senderside.

SRS is a failed attempt to make SPF tier 1 on the receiverside.  SRS's
backers encouraged receivers to act as if it was already fully accepted
and deployed, in order to pressure forwarders to cooperate or face false
positives.  But instead, this led many senders to avoid those same false
positives by treating SPF as tier 3 on the senderside.

A trick using VERPing (of which SES is a brand) can be used to make SPF
tier 1 senderside again despite the damage done by SRS.  That requires a
special dynamic DNS setup, however.  VERP also has a cost in that
whitelisting is made harder for the recipient.

"EDSP" would be tier 1 both senderside and receiverside.  That's its
selling point.

The "dkim=except-mlist" ADSP enhancement I suggested back in 2011 would
be tier 2 receiverside and just-slightly-below tier 1 senderside.  It
would not be obsoleted by "EDSP" or SPF, because when it can be
deployed, it can stop some messages where From: is forged but MAIL FROM
isn't.  Although slightly less than ADSP or DMARC in the rare case that
those are deployable senderside.

TPA ADSP enhancements are tier 1 receiverside and just-barely-above tier
3 senderside.  They could be as powerful as except-mlist in terms of
false-negative rate where deployed at both ends, but require much more
elaborate setup work at the sender.

---

From: Douglas Otis <[email-masked]>
Date: Mon, 22 Jul 2013 15:23:24 -0700

> elaborate setup work at the sender.

Dear Michael,

Envelope-DKIM-Signing Policy?
Isn't this the DMARC policy assertion which includes SPF results as a fallback?

Regardless of policy, DKIM signatures MUST BE ignored when the message includes invalidly prefixed header fields.
As such, invalidly prefixed header fields MUST BE checked during DKIM signature evaluations.
It seems doubtful that any large provider will reject messages due to SPF fail.

RFC6541ATPS attempted to adopt concepts found in:
http://tools.ietf.org/html/draft-otis-dkim-tpa-label-06 

IMHO, RFC6541flaws will ensure it is never adopted--
 1) Requires a special DKIM signature (should depend on the policy scheme instead)
 2) Allows optional per-domain label construction (not manageable by a community)
 3) Omitted authentication methods by assuming DKIM (not easily extensible)

Since sender side policy primarily benefits senders, it should be senders doing the work.  After all, senders should track which third-party services their users are sending messages.  The senders' work can become a community effort, since the tpa-label specification allowed redirection to various conforming zones selected by senders.

Why require receivers to guess which third-party services are being used?
How would any such guessing scheme be either fair or safe?  

Any policy that lacks the TPA label scheme can never be suitable as an extension for many of the social services that use email to bridge between the various groups for extending invitations.  Without an invalidly prefixed header check result added to the Authentication-Results header field, dkim=pass can not be safely used since rarely will a DKIM domain double list singleton header fields.

I made an effort to standardize use of an opaque tag added to the DKIM signature to facilitate abuse reporting.  It is absurd to expect large providers will review timestamps and logs to ascertain problem sources.  Lack of standardization due to an unfounded fear this reduces user privacy simply misunderstood what is meant by the term opaque.  What we have now is definitely not better at preventing abuse or protecting privacy.

Regards,
Douglas Otis

---

From: "John R. Levine" <[email-masked]>
Date: 22 Jul 2013 19:12:57 -0400

>> selling point. ...

>> 3 senderside. ...

Did I miss some I-Ds describing these?

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html