Subject: [ietf-dkim] 8bit downgrades

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 19 May 2011 10:50:18 -0700

Can anyone remember why there's a SHOULD for the downgrade to 7-bit in RFC4871 Section 5.3, rather than a MUST?  The likelihood of breakage is so high when sending 8-bit data that DKIM almost becomes pointless without the upgrade.

Not advocating for this to be changed in -bis (yet), but someone's asking me for the history behind that decision.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Wietse Venema <[email-masked]>
Date: Thu, 19 May 2011 14:16:47 -0400 (EDT)

Murray S. Kucherawy:
> Can anyone remember why there's a SHOULD for the downgrade to
> 7-bit in RFC4871 Section 5.3, rather than a MUST?  The likelihood
> of breakage is so high when sending 8-bit data that DKIM almost
> becomes pointless without the upgrade.

This would require minor surgery for MTAs that DKIM-sign on arrival,
and that downgrade only on delivery (such as Postfix).

This would require major surgery for MTAs that don't implement
MIME, or that implement MIME but have no support for downgrading
(the latter is allowed by the MIME RFCs).

> Not advocating for this to be changed in -bis (yet), but someone's
> asking me for the history behind that decision.

We could pretend that the future is 8-bit clean, and hope the
problem will go away eventually.

	Wietse

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 May 2011 11:10:02 -0700

On 05/19/2011 10:50 AM, Murray S. Kucherawy wrote:
>
> Can anyone remember why there’s a SHOULD for the downgrade to 7-bit in 
> RFC4871 Section 5.3, rather than a MUST? The likelihood of breakage is 
> so high when sending 8-bit data that DKIM almost becomes pointless 
> without the upgrade.
>
>
> Not advocating for this to be changed in –bis (yet), but someone’s 

I don't remember the original rationale, but it would be bizarre for
DKIM to hem in any future SMTP improvements/standards regarding 8 bit.

Mike

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 May 2011 15:30:17 -0400

Murray S. Kucherawy wrote:
> Can anyone remember why there's a SHOULD for the downgrade to 7-bit 
> in RFC4871 Section 5.3, rather than a MUST?  The likelihood of 
> breakage is so high when sending 8-bit data that DKIM almost 

The top two:

1) DKIM signed mail is not an email requirement.  A MUST would be too 
extreme.

2) There is a natural expectation for passthru non-tampering.

Practical:

3) If high cost changes are required to satisfy this downgrade, it is 
cheaper not to sign mail at all.

4) The MSA who is DKIM-AWARE might do the translation in preparation 
for the DKIM signing component, but this again one of those "narrow" 
things that you expect adaptation or a fix to occur.

The two top sentences are prevailing:

1st paragraph, last sentence:

    Such conversion is outside the scope of DKIM; the actual
    message SHOULD be converted to 7-bit MIME by an MUA or MSA
    prior to presentation to the DKIM

last paragraph:

    More generally, the signer MUST sign the message as it is
    expected to be received by the verifier rather than in
    some local or internal form.

That first paragraph is closer to a GateWay issue which is out of 
scope in DKIM.

You're touching base with boundary layer INPUT requirements argument 
Murry, I suggest to avoid this.   You can't enforce this with a MUST 
and if you do, you will find systems taking the easy, no cost path of 
not signing mail

-- 
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 May 2011 15:53:32 -0400

> problem will go away eventually.

+1 and it will. 8-bit is natural adaption expectation w/o DKIM.

Half the problem with the DKIM specs was that it was written with 6-7 
years ago with semantics that are less important today.

Even then, the idea of making this a MUST would be, IMV, really 
radical.  An intermediary doing such downgrades would be behaving like 
a Gateway:

    ANY FORMAT --> GATEWAY -->  RFC5322 --> DKIM SIGNER

It doesn't make sense for anyone to add DKIM in the gateway.

Murray,

Look at this way, the SHOULD is probably already ignore today.  I 
doubt a MUST will change that, and worst, force signers to write 
simple cheap signing rules:

   if mail contains 8-bit content, then skip signing.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 19 May 2011 15:16:51 -0400

> problem will go away eventually.

I have a vague recollection that this is the reason it's SHOULD vice MUST. 

Scott K

---

From: Pete Resnick <[email-masked]>
Date: Thu, 19 May 2011 16:53:03 -0500

On 5/19/11 12:50 PM, Murray S. Kucherawy wrote:
>
> Can anyone remember why there's a SHOULD for the downgrade to 7-bit in 
> RFC4871 Section 5.3, rather than a MUST?  The likelihood of breakage 
> is so high when sending 8-bit data that DKIM almost becomes pointless 
> without the upgrade.
>
>
> Not advocating for this to be changed in --bis (yet), but someone's 

I can't speak directly to the history of why the WG *thought* they were 
putting in the SHOULD, but Michael, Wietse, Scott, and Hector seemed to 
have missed the point of why the SHOULD is absolutely appropriate:

In RFC 2119 (the document that defines MUST, SHOULD, etc.), "MUST" does 
not mean "vitally important" and "SHOULD" does not mean "really really 
important, but less important than MUST". "MUST" means "you have to do 
this or you're not going to interoperate." "SHOULD" means, "there are 
ways to not do this which will still interoperate, but you had better 
know what those ways are and you better be sure to do them, and if you 
don't, then you MUST NOT do this." That is, "SHOULD" is equivalent to 
"MUST unless you know exactly what you are doing."

In this case, the spec says that you MUST downgrade prior to signing 
*unless you know that the end-to-end path is 8-bit clean and will not 
downgrade later*. That's what SHOULD downgrade means. If there is an 
implementation that doesn't downgrade and sends a message without 
knowing that the path is end-to-end 8-bit clean, then it is in violation 
of the spec. Changing it to MUST doesn't change anything for such an 
implementation; it is already in full violation.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 May 2011 16:09:04 -0700

On 05/19/2011 02:53 PM, Pete Resnick wrote:
> knowing that the path is end-to-end 8-bit clean, then it is in 
> violation of the spec. Changing it to MUST doesn't change anything for 
> such an implementation; it is already in full violation.

This is all a rather academic argument, but it doesn't really seem quite
right. We send things that get forwarded through all kinds of manglers,
8bit manglers just being one variety. In the abstract, you can never know
as a signer that a path is "clean"... it can always be forwarded. So by your
argument it should be a MUST since you can never know. But that creates
the silly-state of DKIM wagging the 8bit SMTP tail, which is a wrong
outcome.

In reality, I haven't ever seen a failure that was attributable to 8bit
mangling, and I've probably seen sample sizes as big or bigger than
Murray's. Maybe it's happened, but it seems extremely rare.

Mike

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 May 2011 19:52:46 -0400

> implementation; it is already in full violation.

Pete, it isn't that complicated:

MUST is a "throw in your face" requirement - don't follow, expect 
things to break and not match up with your peers - shame on you - a bug.

SHOULD is an optional requirement - Its a recommendation for the 
better, but things will not break things for your peers if you don't 
follow it.  You may be shamed but the person shaming you is the one 
wrong if they depended on a SHOULD operation as a MUST and his 
software broke.

SHOULD is appropriate here IMV because it approves things (for DKIM), 
but it can't be enforced for common sense historical reasons. 
Primarily, it can't enforce change.  Going to 8-bit MIME was an 
expensive and complex endeavor and now asking people they should 
downgrade and worst, tamper with passthru mail is hard to swallow.

In fact, I may even support a change to a MAY, simple because there is 
a natural expectation for non-tampering passthru mail and more and 
more systems are using 8 bit communications.

The problem isn't 8bit, the problem is DKIM and this topic is yet 
another example of the many WG contradictions.  Before we attempt to 
force change for the sake of DKIM verification, it might be a good 
idea to renew our focus on the serious repercussions for the already 
existing relaxations and marginally errors we have accepted for DKIM 
verification failures

If we change this downgrade to a MUST, then we must also fix the C14N 
problem we forgot about the extra <CRLF> possible at the top the 
message possible in IETF streams like IETF-SMTP.   Can't have it both 
ways: its important here, but not there.

---

From: Pete Resnick <[email-masked]>
Date: Thu, 19 May 2011 21:11:04 -0500

On 5/19/11 6:09 PM, Michael Thomas wrote:
> We send things that get forwarded through all kinds of manglers,
> argument it should be a MUST since you can never know.

(I'll assume here that you're using a loose definition for "forwarded" 
and are really talking about either relaying or resending. Forwarding in 
the usual sense is not something that DKIM *should* survive.)

I absolutely agree that most of the world is an environment where you 
can't know end to end and therefore *most* implementations MUST 
downgrade. However, there can be environments, normally by out-of-band 
agreement, that are 8-bit clean end to end. There, it is perfectly 
reasonable not to downgrade.

> But that creates

I'm not sure what you mean here. What is the "right" outcome?

pr

--

---

From: "John Levine" <[email-masked]>
Date: 20 May 2011 02:20:22 -0000

>so high when sending 8-bit data that DKIM almost becomes pointless

I think Pete's analysis is correct, but my advice would be to take
it out altogether.  We don't have any great insight into the warts
of what paths are likely to downcode a message and what paths aren't,
so I would prefer not to purport to offer advice about it.

R's,
John

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 May 2011 19:24:27 -0700

On 05/19/2011 07:11 PM, Pete Resnick wrote:
> and are really talking about either relaying or resending. Forwarding 
> in the usual sense is not something that DKIM *should* survive.)

Yes.

> I'm not sure what you mean here. What is the "right" outcome?

DKIM should never be in the position of dictating what is "right"
for the email world at large. If that world says that 8 bit is a
Good Thing, it would be a pretty ridiculous situation to have
DKIM being the impediment: it's a tail wagging the dog situation
which is always a wrong outcome.

Like I said, this is sort of academic... I don't think we're in any
huge danger here :)

Mike

---

From: Michael Thomas <[email-masked]>
Date: Thu, 19 May 2011 19:34:39 -0700

> so I would prefer not to purport to offer advice about it.

Miracles... my implementation never bothered to worry about
that SHOULD and I don't feel dirty about it in the least: it
never made any difference that I could determine. This
was mostly an academic piece of advice, IMO. Since dev
managers literally looks at MUST's and SHOULD and ignore
MAY's to determine what gets implemented, this is not
quite as academic.

Mike

---

From: Pete Resnick <[email-masked]>
Date: Thu, 19 May 2011 21:21:04 -0500

> wrong if they depended on a SHOULD operation as a MUST and his

This is 100% incorrect, and if a WG were to produce a document where it 
assumed the above definition of SHOULD, the chair should immediately put 
the brakes on, because it should get bounced by the IETF Last Call and 
the IESG Review. SHOULD has a singular meaning in IETF Standards Track 
documents which cite RFC 2119, and it is stated quite clearly in section 
3 and section 6 of that document. If anyone is using it differently, 
they need to go back and re-read RFC 2119 *very* carefully. This is not 
a matter of opinion. Implementations very much depend on the RFC 2119 
definitions of these terms and this document had better as well.

pr

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 May 2011 23:37:33 -0400

100%?  That is extreme.

I hope you are not stating giving two interfacing points, one end will 
read a SHOULD as a MUST?  I hope not. One end point expecting a MUST 
behavior from a SHOULD recommendation is seriously broken especially 
if it doesn't get what it expects.  That is not a matter of opinion - 
the system will break down if end points read a SHOULD as a MUST for 
one basic reason:

    	     Backward Compatibility

Any implementator who reads SHOULD as a MUST implement and expects all 
standard software, old and new, to be working as if was a MUST is in 
trouble.

I think you know that Pete.

But lets going over this:

First, an ESMTP server is not required to support 8BITMIME, therefore 
there can not be any DKIM expectation nor a MUST requirement for the 
ESMTP server to be even ready  do any sort of 8BIT downgrades.  It 
would be incorrect for DKIM to assume it can force this ESMTP 
extension on the SMTP server.

Second, even if the payload is accepted for a time-shifted isolated, 
plug and play DKIM signing component, RFC4671 section 5.3 clearly 
states it is out of scope:

So I don't even know why we are talking about this. If its out of 
scope how we can contemplate a MUST here.  I concur with Levine, take 
it out.

---

From: Hector Santos <[email-masked]>
Date: Thu, 19 May 2011 23:42:50 -0400

John Levine wrote:
> it out altogether.  

I don't believe Pete's analysis is correct, but I concur that first 
paragraph should be taken out.  This is a Gateway, Border issue and 
SMTP has plenty to see already about it.

> We don't have any great insight into the warts

We do have great insight - Don't Tamper with Passthru mail - that 
should be the advice burn into systems if we want a future DKIM life 
to be without stress.

---

From: Hector Santos <[email-masked]>
Date: Fri, 20 May 2011 00:24:06 -0400

> definitions of these terms and this document had better as well.

I was curious about how 100% incorrect I was:

 From 2119:

3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
    may exist valid reasons in particular circumstances to ignore a
    particular item, but the full implications must be understood and
    carefully weighed before choosing a different course.

hmmm, I don't think my understanding is off base at all:

      RECOMMENDED
      reasons to ignore

Of course, any good protocol implementator is not normally going to go 
against recommendations without good reason and there shouldn't be any 
shock its common place especially for things that are controversial. 
That 80% of the reason things of this nature are made SHOULD or MAY.

In this case, if this is enforced with a MUST, for a system that is 
not 8BITMIME ready but is adding DKIM signing support, to remain 
compliant it is far more feasible to add a rule to a DKIM signing 
component:

    If mail is 8bit then SKIP signing.

So unless this document is now going to revamp the mail industry by 
stating all mail MUST be signed, this 8bit downgrading can not be 
enforced.

---

From: Franck Martin <[email-masked]>
Date: Fri, 20 May 2011 04:09:29 +0000

>Murray's. Maybe it's happened, but it seems extremely rare.

I have seen:

X-MIME-Autoconverted: from quoted-printable to 8bit by xxxx

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 19 May 2011 22:18:39 -0700

> -----Original Message-----
> From: John Levine [mailto:[email-masked]]
> Sent: Thursday, May 19, 2011 7:20 PM
> To: [email-masked]
> Cc: Murray S. Kucherawy
> Subject: Re: [ietf-dkim] 8bit downgrades
> 

Actually, I kinda prefer to leave it in.  It seems to me "assume a downgrade will happen unless you're certain it won't, and plan accordingly" is good advice without having to know the details of a transport path.  And the paragraph gives discussion of the how and why.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 19 May 2011 22:33:32 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Hector Santos
> Sent: Thursday, May 19, 2011 8:38 PM
> To: Pete Resnick
> Cc: [email-masked]

...but correct.

>     prior to presentation to the DKIM

I read that (in the context of the paragraph containing it, of course) to mean the particular selection of a conversion method is out of scope for DKIM.  But that does not mean the issue of whether or not conversion is necessary or advisable is also out of scope.  If it's okay for us to discuss things like header field reordering and spacing changes that might affect validation of DKIM signatures, it certainly seems to me that this is a valid topic as well.

> So I don't even know why we are talking about this.

The presented argument, which comes from an IETF outsider involved with MTA development, is whether or not that SHOULD is worthy of a MUST because failing to do it in the vast majority of cases will result in a downgrade somewhere on the path that will invalidate the signature.  The question, then, is why we didn't do MUST in the first place.  It's a perfectly legitimate question.

The answer is twofold: One, explain what SHOULD and MUST really mean, especially in context.  Two, list the options (downgrade and sign, don't downgrade and sign, don't sign) and the effects of each.

Providing a solid and helpful answer, rather than a dismissive or hostile one, promotes understanding from people that weren't involved, who are the ones that get to implement and deploy what we've specified.  That's certainly my preference.

-MSK

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 20 May 2011 08:04:43 -0400

On Friday, May 20, 2011 01:18:39 AM Murray S. Kucherawy wrote:
> Actually, I kinda prefer to leave it in.  It seems to me "assume a
> downgrade will happen unless you're certain it won't, and plan
> accordingly" is good advice without having to know the details of a
> transport path.  And the paragraph gives discussion of the how and why.

+1.

Scott K

---

From: "John Levine" <[email-masked]>
Date: 20 May 2011 13:33:52 -0000

>transport path.  And the paragraph gives discussion of the how and why.

So long as it's clear that the advice is illustrative rather than
definitive, I'm not extremely opposed.

What I want to avoid is the impression that we think we've anticipated
all possible message mutations, so if there's one we didn't mention,
that means we need to add another hack to DKIM to work around it.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Fri, 20 May 2011 11:30:53 -0400

Murray S. Kucherawy wrote:

> I read that (in the context of the paragraph containing it, of course) 
> to mean the particular selection of a conversion method is out of scope 
> for DKIM.  

In the context of the entire section, the main requirements are
RFC5322 and RFC5321 readiness and the general requirement to make sure
the DKIM signing output is RFC5322 ready for receivers and that
includes WYSIWYG goals.

> But that does not mean the issue of whether or not conversion is 
> necessary or advisable is also out of scope.  If it's okay for us 
> to discuss things like header field reordering and spacing changes 
> that might affect validation of DKIM signatures, it certainly seems 
> to me that this is a valid topic as well.

Sure, thats a boundary layer issue and valid RFC5322 input requirement.

> The presented argument, which comes from an IETF outsider involved 
> with MTA development, is whether or not that SHOULD is worthy of a 
> MUST because failing to do it in the vast majority of cases will 
> result in a downgrade somewhere on the path that will invalidate 
> the signature.  The question, then, is why we didn't do MUST in the 
> first place.  It's a perfectly legitimate question.

As with any new software development, you always have the luxury to do 
everything that is new but it is naive to believe everyone else is new 
too, up-to-par  and there don't exist legacy or just plain old 
standard requirements operations.

> The answer is twofold: One, explain what SHOULD and MUST 
> really mean, especially in context.  Two, list the options 
> (downgrade and sign, don't downgrade and sign, don't sign) and 
> the effects of each.

The SHOULD is enough for him to handle his needs for his own MTA/MSA, 
DKIM signing and distribution.  This issue is no different than all 
the general overall concerns regarding losing mail integrity and 
increasing realization that DKIM deployment needs to include 
target/path knowledge to minimize these
issues.

> Providing a solid and helpful answer, rather than a dismissive 
> or hostile one, promotes understanding from people that weren't 
> involved, who are the ones that get to implement and deploy 
> what we've specified.  That's certainly my preference.

First, I wasn't dismissive nor hostile and take offense to the on-going
continued labeling me as such.  Please stop the hostility towards me.

What I strongly and passionately disagree with is two folds; 1) the 
perpetual contradictions with the justifications used for one thing 
and not equally applied to others, including the ignorance of DKIM 
failures for one thing and not something else; and 2) it is unfair to 
be questioning in 2011 all wisdoms gained from all the WG-MAN-YEARS 
and the entire 40+ years history known with mail systems behaviors to 
come up with that SHOULD in 2005.

You asked for a reason. It was provided by multiple posters, but then 
it turned into into something where each poster was labeled of not 
understanding RFC2119  to justify a change to a MUST. Talk about being 
hostile.

The direction is to be 8 bit compliant in mail, intermediary are 
expected not to tamper mail. There is even a greater reason today why 
it should remain a SHOULD. If you do this as a MUST, then we lose the 
payload binary transparency aspect of communications when we force 
MTAs to downgrade 8bit.

How do we know it will break if its not downgraded?  Unless there is 
full knowledge that a node on the path will do break things, you 
don't.  This is the same thing with MLMs and the IETF-SMTP extra 
<CRLF> issue.

Why don't we care about that? Thats a real live scenario today.  How 
about this idea for section 5.3? Add this paragraph before the last one:

     Mail intermediaries MUST make sure they don't an extra line
     after the header and before the beginning of the body content.

See the slippery slope?

What do we expect will happen in the future when the MLMs are more 
DKIM aware, and the 8bit streams are not altered to the extent that 
DKIM faults are negligible and this MUST is now doing needless 
alterations? Are we changing it back to a SHOULD recommendation or 
even a MAY?

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 20 May 2011 15:21:56 -0500

On May 19, 2011, at 4:53 PM, Pete Resnick wrote:

> In RFC 2119 (the document that defines MUST, SHOULD, etc.), "MUST" does not mean "vitally important" and "SHOULD" does not mean "really really important, but less important than MUST". "MUST" means "you have to do this or you're not going to interoperate." "SHOULD" means, "there are ways to not do this which will still interoperate, but you had better know what those ways are and you better be sure to do them, and if you don't, then you MUST NOT do this." That is, "SHOULD" is equivalent to "MUST unless you know exactly what you are doing."

Sounds like we SHOULD all go back and re-read RFC 2119.

--
J.D. Falk
the leading purveyor of industry counter-rhetoric solutions

---

From: Dave CROCKER <[email-masked]>
Date: Fri, 20 May 2011 21:25:48 -0700

On 5/19/2011 7:34 PM, Michael Thomas wrote:
>  Since dev

That's a rather significant assessment.  It means that all of the Internet 
specifications done for the last 20 or 30 years, that have had the 'MAY' 
qualifier were wasting their effort.

It's odd that working groups continue to use the construct, since you claim it 
is not at all useful for the software that implements the specifications.

Such a basic claim warrants understanding its basis.  Please share it.

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 21 May 2011 11:35:38 +0200

> that means we need to add another hack to DKIM to work around it.

For example, MTAs that autoconvert from quoted-printable to 8bit, a
rather common circumstance.

Wietse reminded us that MTAs with native MIME rewriting capabilities
can run filters (including DKIM filtering) either before or after MIME
conversion.  The former is good for signing while the latter is good
for verifying, but what about subsequent dot-forwards?  We can hope
MTAs will stop autoconverting.  Until then, it is useless to say
/when/ to invoke a DKIM filter, let alone whether to use MUST, SHOULD,
or MAY.

Up/down grading depends on conversion, so I concur with John's
original statement: I would prefer not to purport to offer advice
about it --which is not the same as not mentioning the problem.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Michael Deutschmann <[email-masked]>
Date: Sun, 22 May 2011 01:39:55 -0700 (PDT)

On Thu, 19 May 2011, Murray S. Kucherawy wrote:
> The presented argument, which comes from an IETF outsider involved with
> MTA development, is whether or not that SHOULD is worthy of a MUST because
> failing to do it in the vast majority of cases will result in a downgrade
> somewhere on the path that will invalidate the signature.  The question,
> then, is why we didn't do MUST in the first place.  It's a perfectly
> legitimate question.

One suggestion for compromise, from another "outsider" (myself):

Specify MUST, but clarify that this is just for now and may be revisited
at a later time -- for example, if the SMTP protcol design community ever
backs down and accepts DJB's approach to the 8-bit message problem
(<http://cr.yp.to/smtp/8bitmime.html>, essentially that it is OK to break
any remaining 7-bit enforcing servers).  They probably won't ever, but
just in case...

So then there would be also a MUST that *incoming* 8-bit DKIM-signed
messages be processed in the obvious way.  This would never be used in
the present, but allow future protocol designers some leeway.

By the way, note that DKIM is not alone in this. S/MIME (RFC 1847) says,
on page [4]:
# In addition, if the multipart/signed object is EVER to be
# transferred over the standard Internet SMTP infrastructure, the
# resulting MIME body is constrained to 7 bits -- that is, the use
# of material requiring either 8bit or binary
# content-transfer-encoding is NOT allowed.  Such 8bit or binary
# material MUST be encoded using either the quoted-printable or
# base64 encodings.

---- Michael Deutschmann <[email-masked]>
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Dave CROCKER <[email-masked]>
Date: Sun, 22 May 2011 07:42:57 -0700

On 5/19/2011 10:50 AM, Murray S. Kucherawy wrote:
> Can anyone remember why there’s a SHOULD for the downgrade to 7-bit in RFC4871
> Section 5.3, rather than a MUST?

To concur with one of the lines of explanation that has been offered:

      It is generally a good idea to refrain from mandating something that adds 
cost without benefit.  The recommendation is for a bit of additional processing 
-- the conversion -- that is not /always/ necessary, because downgrading does 
not always happen.

      If the signer has no knowledge of the path that the message will travel -- 
as, indeed, is /currently/ true for nearly all signers -- then they have to use 
the safest choice, namely performing the conversion, to increase the 
survivability of the signature, in case the path does happen to mess with the 
data in the way being discussed.

There are existing scenarios in which the signer well might know that the extra 
processing is /not/ required, such as for internal mail.  This might not be 
common, but it is legitimate.  The same applies for the emergence of 8-bit-clean 
environments, as has been mentioned.

The concept that there will at least be islands of 8-bit clean is not all that 
silly and has been driving the latest round of EAI work.  That sort of concept 
certainly does tend to be viewed more optimistically by proponents than history 
would advise, but there are reasonable indicators to suggest that it's not an 
entirely silly idea with respect to MIME and email transit.  So, again, for such 
environments, the extra protection of the processing specified as SHOULD would 
be wasteful. This is why SHOULD is the correct choice and MUST is not.

> On 5/19/2011 2:53 PM, Pete Resnick wrote:
>> In RFC 2119 (the document that defines MUST, SHOULD, etc.), "MUST" does not mean
>> "vitally important" and "SHOULD" does not mean "really really important, but
>> less important than MUST". "MUST" means "you have to do this or you're not going
>> to interoperate." "SHOULD" means, "there are ways to not do this which will
>> still interoperate, but you had better know what those ways are and you better
>> be sure to do them, and if you don't, then you MUST NOT do this." That is,
>> "SHOULD" is equivalent to "MUST unless you know exactly what you are doing."

Correct, of course, and nicely said.

This confusion about the meaning of normative language is one of the useful 
touchstones to some of the problems that have plagued the DKIM working group.

Among folks who haven't read the relevant bit of RFC 2119, the confusion isn't 
surprising.  However one would think that anyone with extended participation in 
a working group would do the homework of learning the basics of essential 
specification vocabulary, such as the meaning of normative language or at least 
the difference between relaying and forwarding.

The fact that erroneous definitions persist among among some who actually have 
read RFC 2119 suggests possible issues with reading comprehension, since the RFC 
language about this is rather simple, direct and clear.

 > so I would prefer not to purport to offer advice about it.

The SHOULD being discussed actually represents an important bit of responsible 
specification writing.  It MUST be retained.

First, it is necessary for the reason I described above:  It specifies something 
that adds cost, but there are legitimate scenarios that do not need it AND there 
is reason to believe the need will reduce over time.  If the signer knows it is 
not needed, then it is entirely wasteful to do the processing.

Second, a protocol specification SHOULD help the reader understand something of 
the usage dynamics of a protocol.  What are some of the interesting scenarios 
that can cause problems and what is the way to deal with them?  If a scenario 
has sufficiently severe effects and is plausibly going to happen, then the 
method of dealing with the situation is best specified normatively.

It is generally not a good idea to leave a protocol implementer guessing what 
situations to worry about and guessing about how to deal with them.

> On 5/22/2011 1:39 AM, Michael Deutschmann wrote:

While it's always good to see efforts at compromise, in this case, there is no 
need:  The current language fits the current situation nicely..

>> at a later time

This would miss the fact that there are existing scenarios that do not require 
the extra mechanism and there are already efforts ongoing that will make is 
/less/ useful.  To gain extremely widespread deployment, those efforts will need 
a very long time, of course, but nonetheless, they are happening.

d/
-- 

   Dave Crocker

---

From: Douglas Otis <[email-masked]>
Date: Sun, 22 May 2011 22:47:55 +0800

On 5/22/11 10:38 PM, John R. Levine wrote:
> If you were following the EAI work, you'd know that they probably will do
> that within the next couple of months, albeit with an SMTP flag so servers
> and clients can tell whether a hop is 8-bit UTF or legacy.  They
> specifically do NOT provide any downgrade mechanism -- if a path isn't EAI
> from end to end, the message can't be delivered.  (Please read the many
> years of archives of the EAI list, in which they tried every imaginable
> approach via many experimental RFCs and a lot of running code, before
> commenting on the wisdom of this approach.)
>
> I beseech this group to refrain from hypothetiecal guesses about what some
> of us might think would be a good idea to address some anticipated
> problem, even though nobody has tried it. It was a mistake in the mailing
> list so-called BCP, and it would be a mistake here.
>
> There will be DKIM signatures on EAI messages.  It is pretty obvious how
> to do it, and in the few corners where it's not obvious, we won't know the
> right answer any better than anyone else until we've tried it and seen
> what happens.
Agreed.

-Doug

---

From: "John R. Levine" <[email-masked]>
Date: 22 May 2011 10:38:00 -0400

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 22 May 2011 12:29:47 -0700

> Sent: Sunday, May 22, 2011 10:43 AM
> Please refrain from passing the buck to the WG. The document editors
> are:
> 
>     D. Crocker (editor)
>     Tony Hansen (editor)
>     M. Kucherawy (editor)
> 
> If the WG was technically incapable as you are implying, then the
> *onus* was on the editors to make sure it was writing it correctly.

Given that it's been pointed out the use of SHOULD in this case is entirely appropriate, I'm happy to accept blame on behalf of the other authors for getting it right.

---

From: Hector Santos <[email-masked]>
Date: Sun, 22 May 2011 13:43:29 -0400

> Correct, of course, and nicely said.

Quite clear from what it actually says:

> language about this is rather simple, direct and clear.

Please refrain from passing the buck to the WG. The document editors are:

    D. Crocker (editor)
    Tony Hansen (editor)
    M. Kucherawy (editor)

The problem in the WG was key cogs not following the charter, 
introducing deliberate out of scope concepts, mixing up the functional 
requirements with the technical specifications, doing global word 
replacements and not seeing if it was correct, removing/adding 
semantics and not seeing it fit the functional goals, confusing 
synergism with past or concurrent document productions, lost of WG 
interest participation with a resultant Consensus by Osmosis process, 
and frankly, all of which, all reflected a mark of poor leadership.

In this case (8BIT Downgrades), it is my view the SHOULD language is 
fine and was quite fitting the understood technical feasibility realities.

---

From: Hector Santos <[email-masked]>
Date: Sun, 22 May 2011 14:25:41 -0400

> any remaining 7-bit enforcing servers).  They probably won't ever, but

-1

The only reason for a MUST would be to enforce a change in software 
and to move it in into a positive direction.   In situations where 
this is known to create interoperability problems, you can only do 
this MUST if there is a technical fallback mechanism.

Case in point: For SMTP, HELO vs EHLO (extended HELO)

By RFC2119, EHLO should of been written as a SHOULD, but we wanted the 
market to move towards Extended SMTP operations and since SMTP was 
built with a

        "500 COMMAND NOT UNDERSTOOD"

the SMTP state machine was still capable of falling back to a standard 
SMTP operation using the HELO command.  In other words:

  -  Client MUST always been with EHLO
  -  Client MUST always fallback to HELO if EHLO is not supported by 
server.

The issue with 8bit downloads is that DKIM does not have that fallback 
client/server state machine capability.

So if you make this a MUST, then you MUST have a fallback. If you make 
this (keep it as) a SHOULD, then you SHOULD have a fallback.

At the end of the day, you have to look at the realities there exist 
non-8BITMIME ready systems, so you won't get the conversions DKIM is 
looking for.  The better DKIM spec language would of been one related 
to "DKIM Input Validation Contracts."

    http://www.google.com/search?q=Validation+Contracts

IOW, input requirements would be isolated to DKIM itself:

    DKIM signing components SHOULD|MUST have 8bit downgrading. The idea
    of whether MSA/MTA perform this conversion is OUT OF SCOPE.

SHOULD is my preference.  MUST is selected if that is what we want the 
DKIM MARKET to move to, but it MUST always have a fallback.

What will happen Michael if this is may a MUST is that systems will 
look for the fallback or skipping:

    if mail is 8bit then SKIP signing mail

but it could be done with downlink target/path knowledge:

    if mail is 8bit then
       if target path does destroy 8bit then convert
       sign mail

While that may be a functional description of a fallback, we don't 
have the automated technical capability to define it reliably.

---

From: Hector Santos <[email-masked]>
Date: Sun, 22 May 2011 17:25:21 -0400

Murray S. Kucherawy wrote:

> Hector Santos followed up Crocker'ss passing of the buck:
>>
>> *onus* was on the editors to make sure it was writing correctly.
> 
> Given that it's been pointed out the use of SHOULD in this case
> is entirely appropriate, I'm happy to accept blame on behalf of 
> the other authors for getting it right.

With all due respect, for the record, actually it was the original 
editors/authors:

      E. Allman
      J. Callas
      M. Delany
      M. Libbey
      J. Fenton
      M. Thomas

with the April 2006 draft-ietf-dkim-base-00 original text.  Updated in 
August 2006 base-05 draft update to clarify the 2nd paragraph in 
particular regarding Unix (LF) vs DOS (CRLF) translation requirements, 
and it (correctly) has remained that way since then.   This 
unfortunate issue is with new editors trying to change it and then 
labeling the WG for not understanding RFC2119 as a reason.

---

From: Hector Santos <[email-masked]>
Date: Mon, 23 May 2011 00:35:49 -0400

> rather common circumstance.

I did the following Content-Transfer-Encoding failure analysis:

     Failure rates for message top level encoding type
+--------------------------------------------------------+
| enctype               total       bodyfail     pct     |
|--------------------------------------------------------|
| 8bit                  31          25           80.6    |
|                       5461        1236         22.6    |
| binary                5           1            20.0    |
| quoted-printable      1188        179          15.0    |
| base64                32          3             9.3    |
| 7bit                  1465        106           7.2    |
+--------------------------------------------------------+

Based on my PCN, 8bit does have a high rate of failure, but they are a 
very small amount of the grand total.  The encoding types; QP, base64 
and 7bit offers the better lower failure rate.  Except for the 3 
base64 fails, but I can't tell if they originated that way or not. I 
will take a SWAG that these represent direct messages and were not 
going thru a list.

It would be interesting to see what Murray can show for his volume 
collection.

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 23 May 2011 11:56:40 +0000

>    If mail is 8bit then SKIP signing.

But why skip? Usually the message won't be downgraded. And even if they are, usually a broken signature will cause no harm.

-- 
Ian Eiloart
Postmaster, University of Sussex
+44 (0) 1273 87-3148

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 23 May 2011 11:53:47 +0000

On 19 May 2011, at 22:53, Pete Resnick wrote:

> In this case, the spec says that you MUST downgrade prior to signing *unless you know that the end-to-end path is 8-bit clean and will not downgrade later*. That's what SHOULD downgrade means. If there is an implementation that doesn't downgrade and sends a message without knowing that the path is end-to-end 8-bit clean, then it is in violation of the spec. Changing it to MUST doesn't change anything for such an implementation; it is already in full violation.

Downgrading is undesirable, especially given the increasing popularity of mobile clients. 

In theory, the signature could be added at SMTP time, with downgrading occurring dependent upon whether 8bitmime is advertised by the recipient's MTA. Of course, there's a risk that the message will then require downgrading when forwarded by the recipient's MTA, but that risk should decrease with time (for example, Exim's developers have 8bitmime development as a priority now). And, it's also an option for the recipient's MTA to add a new signature (whatever that's worth).

---

From: Hector Santos <[email-masked]>
Date: Mon, 23 May 2011 10:19:39 -0400

Ian Eiloart wrote:
> But why skip? Usually the message won't be downgraded. And even if they 
> are, usually a broken signature will cause no harm.

Thats the problem - define "usually" and also define "no harm."

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 23 May 2011 14:44:48 +0000

> Thats the problem - define "usually" and also define "no harm."

Well, harm will only be done when someone incorrectly punishes a broken signature. They should not do that, so the damage is actually done by the recipient, not by the downgrading.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 23 May 2011 16:49:08 +0200

>      Failure rates for message top level encoding type

It is not clear what part of these 8bit failures is due to messages
that had been downgraded before signing, and then upgraded before
verifying.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Hector Santos <[email-masked]>
Date: Mon, 23 May 2011 12:10:31 -0400

> Well, harm will only be done when someone incorrectly punishes a 
> broken signature. They should not do that,

Rhetorically, why not?  Put another way, why should a receiver 
tolerate failure, or better, why should DKIM itself - the technology - 
tolerate failure?  Sounds like DKIM has some inner soul turmoils - a 
devil on one shoulder and angel on the other.

> so the damage is actually done by the recipient, not by the downgrading.

Well, thats a difference in two reasonable mindsets - a receiver who 
views faults as part of the strength of securing a technology and a 
receiver who tolerates faults - accepts everything including one that 
are direct and indirectly created and passes the buck to end-users.  I 
like to believe there exist a commonality where false positive 
deterministic methods can be use to detect violations of an 
authentication and integrity technology.

Rhetorically, its all for nothing, why bother looking at how to fix 
C14H hashing, talk about content formatting downgrades when failure is 
tolerated and per specification, deliberately ignored?

---

From: "John R. Levine" <[email-masked]>
Date: 23 May 2011 12:35:02 -0400

> In the real world signature reliability matters. If a domain signs mail 
> as a rule then an absent or broken signature will be treated as 
> suspicious.

I hope you're wrong, since that violates an explicit SHOULD in RFC 4871, 
and in my experience, most broken signatures are due to innocent 
modification in transit, not malice.

Do you have numbers to show that broken signatures indicate that messages 
are malicious, or spam, or otherwise worse than otherwise?

For that matter, since we're not talking about ADSP, what do you mean by 
an absent signature?  Do you track which domains sign what mail? How do 
you tell what signature you're expecting?  From line domain? Sender? 
Message ID? Something in the Received lines?

Regards,

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 23 May 2011 11:29:06 -0400

>Ian Eiloart <[email-masked]> wrote:
>
>On 23 May 2011, at 15:19, Hector Santos wrote: > Ian Eiloart wrote: >>
>On 20 May 2011, at 05:24, Hector Santos wrote: >> >>> In this case, if
>this is enforced with a MUST, for a system that is >>> not 8BITMIME
>ready but is adding DKIM signing support, to remain >>> compliant it is
>far more feasible to add a rule to a DKIM signing >>> component: >>>
>>>> If mail is 8bit then SKIP signing. >> >> But why skip? Usually the
>message won't be downgraded. And even if they >> are, usually a broken
>signature will cause no harm. > > Thats the problem - define "usually"
>and also define "no harm." > Well, harm will only be done when someone
>incorrectly punishes a broken signature. They should not do that, so
>the damage is actually done by the recipient, not by the downgrading.

In the real world signature reliability matters. If a domain signs mail as a rule then an absent or broken signature will be treated as suspicious.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 23 May 2011 13:11:53 -0400

> modification in transit, not malice.

Which one is that?  AFAIK treating a broken signature the same as no signature 
is what the RFC wants me to do.

> are malicious, or spam, or otherwise worse than otherwise?

None that I can share unfortunately.  IME no signature is more suspicious than 
a broken one (as you suggest, I think most breakage is innocent), but putting 
broken and no signature into the same bucket is the most sensible and RFC 
compliant way to approach it.

> Message ID? Something in the Received lines?

The specific cases (which are non-ADSP) that I'm aware of use the body From as 
a key.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Mon, 23 May 2011 16:04:00 -0400

> that had been downgraded before signing, and then upgraded before

None.

Of the 31, 20 were from Keith Moore signed messages into the IETF-SMTP 
list with a 3rd party signature and Hoffman's list server (non-dkim 
aware) doing this:

   Content-Transfer-Encoding: 8bit
   X-MIME-Autoconverted: from quoted-printable to 8bit by
                         hoffman.proper.com id p4BLC7Hl032165

These 20 all failed and the hop count was 6.  It was the only list 
message,

Of the remaining 11, all spam, all direct (hops=1), 5 failed - 2 
domains - 3 pure American Spam and 2 Spanish Spam.

Moore sent me a direct email recently, and from what I can see, its 
the same 3rd party signer with:

    Content-Type: text/plain; charset=us-ascii
    Content-Transfer-Encoding: quoted-printable

---

From: Franck Martin <[email-masked]>
Date: Mon, 23 May 2011 22:10:10 +0000

There is an interesting post today on
http://chilli.nosignal.org/mailman/listinfo/mailop about exim and 8bit

It seems they will stop to downgrade.

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Tue, 24 May 2011 00:09:31 +0200

> are malicious, or spam, or otherwise worse than otherwise?

SpamAssassin assigns a score of something like 0.1 for a message 
carrying a DKIM signature and compensates that with -0.1 if the 
signature can be verified to be correct. Effectively, this means SA is 
penalizing broken signatures...

/rolf

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 23 May 2011 16:12:22 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of John R. Levine
> Sent: Monday, May 23, 2011 9:35 AM
> To: Scott Kitterman

Count of failed signatures compared to spam true/false flag:

+----------+------+
| count(*) | spam |
+----------+------+
|   120257 |    0 |
|    18945 |    1 | (13.6%)
+----------+------+

Doesn't look like there's a valid correlation there to me.

> Message ID? Something in the Received lines?

For domains that have at least once signed their own mail with a signature that passed, here's the correlation of message counts from those domains versus whether or not the mail is signed (by that same domain) and whether or not those messages are spam:

+----------+------+--------+
| count(*) | spam | signed |
+----------+------+--------+
|   230426 |    0 |      0 |
|     9925 |    1 |      0 | (4.1% of unsigned mail from domains that sometimes sign)
|  1352623 |    0 |      1 |
|    95962 |    1 |      1 | (6.5% of signed mail from domains that sometimes sign)
+----------+------+--------+

What this tells me is: Ignoring ADSP, if a domain sometimes signs its mail, then mail from it (signed or not) is usually not spam.  From this I suspect we could conclude that a missing signature doesn't tell us much of anything.

Now of course there are some domains that sign nothing but spam.  We could narrow this set down by selecting for signing domains that generally don't sign spam, but I think all that would do is shrink the "spam" rows (i.e. the second and fourth counts) without measurably changing the other two.  We could also change "sometimes" to "usually" and see if that matters, but I'm skeptical.

---

From: SM <[email-masked]>
Date: Mon, 23 May 2011 16:23:40 -0700

Hi Rolf,
>penalizing broken signatures...

Not really.  There is a score so that the rules are evaluated.  It 
doesn't weigh much in practice.

Regards,
-sm

---

From: Hector Santos <[email-masked]>
Date: Mon, 23 May 2011 20:03:54 -0400

> What this tells me is: Ignoring ADSP, if a domain sometimes signs its 
> mail, then mail from it (signed or not) is usually not spam.  From this I 
> suspect we could conclude that a missing signature doesn't tell us much 
> of anything.

And it would be an incorrect conclusion.  This shows a lack of 
understanding of policy concepts.

Look, I can clearly say right now, that 100%, not 99.99% of all DKIM 
signed mail in my PCN have untrusted SIGNERS even if I known who they 
are - they are 100% not vouched. I will venture that the majority DKIM 
receivers see a 100% or close to it.

Is that evidence to conclude that the TRUST idea is bad?  No.

Now, if I had a local table of TRUSTED signer domains, then I can make 
an assertion that an VALID signature from that signer is ok, but if I 
see it broken, its going to a classification that is lower than OK.

In the same vain if an Author Domain has a policy says THIS, but you 
see THAT, thats a clear policy violations.

Either way, Author or Signer - there is always a policy concept 
involved - when you  have neither, then we have want we have now which 
is pretty much nothing.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 24 May 2011 10:05:16 +0200

>> that had been downgraded before signing, and then upgraded before

Sorry, by "upgraded" I mean the same as "X-MIME-Autoconverted: from
/any encoding/ to 8bit".  Thus I take your answer as 20/31.

>                          hoffman.proper.com id p4BLC7Hl032165

Thanks!

It was recently mentioned that Hoffman's MLM inserts a white line on
top of the body.  Unfortunately, relaxed body canonicalization regards
this line as significant.  This autoconversion is another, unrelated
change that breaks signatures.  Now, let me say that my MTA
contravenes the "SHOULD downgrade" precept.  I hypothesize that if it
wasn't for the extra white line, that is, taking into account only
"normalization" issues, then my signatures would survive while Keith's
would not.

IOW: the 1st paragraph of Section 5.3 can be *misleading*, as in some
cases a signature's survivability gets worsened.  What encoding
minimizes the chances of conversion is a moving target whose current
state we should not purport to know.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 24 May 2011 10:55:48 +0000

> It seems they will stop to downgrade.

Exim doesn't downgrade. It doesn't advertise 8bitmime either, by default. If you switch on 8bitmime advertising, it still doesn't downgrade. I think it just tries to deliver the mail as 8bit, regardless of what the receiving MTA does. I think postfix and sendmail do the same, but I'm not sure.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 24 May 2011 10:40:52 +0000

On 23 May 2011, at 17:10, Hector Santos wrote:

>> Well, harm will only be done when someone incorrectly punishes a broken signature. They should not do that,
> 
> Rhetorically, why not?  Put another way, why should a receiver tolerate failure, or better, why should DKIM itself - the technology - tolerate failure?  Sounds like DKIM has some inner soul turmoils - a devil on one shoulder and angel on the other.

Because there are known to be paths that break DKIM signatures. And because of this: http://www.apps.ietf.org/rfc/rfc4871.html#sec-6.3

> Well, thats a difference in two reasonable mindsets - a receiver who views faults as part of the strength of securing a technology and a receiver who tolerates faults - accepts everything including one that are direct and indirectly created and passes the buck to end-users.  I like to believe there exist a commonality where false positive deterministic methods can be use to detect violations of an authentication and integrity technology.
> 
> Rhetorically, its all for nothing, why bother looking at how to fix C14H hashing, talk about content formatting downgrades when failure is tolerated and per specification, deliberately ignored?

Because success has value, if you have a good reputation as a signer.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 24 May 2011 11:30:07 +0000

> penalizing broken signatures...

Barely. That's 0.1 on a default threshold of 5.0, I think.

---

From: "John Levine" <[email-masked]>
Date: 24 May 2011 13:22:54 -0000

>> list with a 3rd party signature and Hoffman's list server (non-dkim 

Oh, it's a mailing list.  Why are we even having this discussion?  We all
know there's a million ways that lists break incoming signatures, which
is why they should sign on the way out.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 24 May 2011 16:05:02 +0200

> is why they should sign on the way out.

IMHO, that MLM is only responsible for the inserted whiteline; MIME
rewriting is done by MTA/MDA.  This rewriting is typical of a class of
messages that arrive at an MTA and then undergo a dot-forward or
similar mechanism.

Although it is a minor number of messages, I don't think that
ignore-by-design could play a winning role here, because --unlike
mailing lists-- there is no way to eventually fix this at the
forwarding MTA.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "John R. Levine" <[email-masked]>
Date: 24 May 2011 10:14:56 -0400

> mailing lists-- there is no way to eventually fix this at the

If the EAI work is any guide, in the long run everything will be 8 bit, 
and downgrades will eventually go away.

In the shorter term, if the forwarding MTA is inclined to be helpful, it 
can re-sign on the way out.

Regards,

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 24 May 2011 17:17:38 +0200

> and downgrades will eventually go away.

Well, this consideration suggests it would be smoother to remove the
recommendation to use transfer encoding now, rather than suddenly turn
it into the opposite recommendation in a post-EAI DKIM spec.

Anyway, if that is going to take decades, a more robust
canonicalization could be worth its while.

> can re-sign on the way out.

SPF experience says MTAs often don't help.  Even if they did, the
resulting SDID would be neither the author's domain nor the MLM.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Tue, 24 May 2011 17:42:20 +0200

> Barely. That's 0.1 on a default threshold of 5.0, I think.

Granted, it's a small penalty, yet it's a penalty. And also (to get back 
to John's question) it doesn't mean that [...] a broken signature 
indicate that messages are malicious, or spam [...]. It just means that 
in the real world there are systems, even widely used systems, which 
does by default treat messages with a broken signature not equal as if 
the message had no signature at all.

/rolf

---

From: "John R. Levine" <[email-masked]>
Date: 24 May 2011 11:56:46 -0400

> Granted, it's a small penalty, yet it's a penalty.

I'll ask the spamassassin developers where the number came from.  SM's 
suggestion that it has to be non-zero to exercise the code may be it.

Regards,

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 May 2011 14:21:08 -0400

FDS - Final Delivery Signer

The perfect solution is the Mail Delivery Agent (MDA) signing all 
Incoming Mail for the User!!  With a Final Delivery Signer or FDS, it 
doesn't matter who signed it, can't blame it on the mail man, this 
truck, or for delivering a torn up letter.

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 May 2011 14:42:36 -0400

>> tolerate failure, or better, why should DKIM itself - the technology 
>> - tolerate failure?  Sounds like DKIM has some inner soul turmoils - a 
> Because there are known to be paths that break DKIM signatures. And 
>> because of this: http://www.apps.ietf.org/rfc/rfc4871.html#sec-6.3

That doesn't answer the question of why should it be tolerated. Why is 
the sender who's intention is to get the receiver to "trust him" 
tolerate the sender's ignorance of taking the wrong path is not 
helping him?

This is what promotes a very serious problem of relaxation neglect or 
the "Crying Wolf" Syndrome.

In other words, at what point is your signature ok? and ifs that 
acceptable why isn't the case 100% of the time?   If thats not 
possible, when whats the difference?

> Because success has value, if you have a good reputation as a signer.

And more usually than not, success can only be determine after you 
remove all the dirt from it.

Can't talk about Pareto without having this being a very fundamental 
part of the thinking process.

The fact is everyone does it - filters information before squeeze out 
the good.  We are trying to ignore that fundamental concept for DKIM - 
doesn't work very well when its plagued with all sorts of error 
conditions.

We are assuming that receivers will endure high volume abuse and 
overhead just to look for that limited golden needle in the hay stack 
of failure.

Sorry, its an incorrect mentality that is often exhibited by mail 
blasters believing that every receiver/MDA will accept all mail with 
no questions asked.

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 May 2011 15:19:02 -0400

Ian Eiloart wrote:
> Exim doesn't downgrade. It doesn't advertise 8bitmime either, by default. 
> If you switch on 8bitmime advertising, it still doesn't downgrade. I think 
> it just tries to deliver the mail as 8bit, regardless of what the receiving 
> MTA does. I think postfix and sendmail do the same, but I'm not sure.

Look, the general rule of thumb is PASSTHRU mail is untouched (except 
for adding network control/trace header lines).  Only upon final 
delivery begins the idea of any transformations and gateways so you 
expect under normal circumstances, the payload will be delivered as it 
was created. That is why, overall, the system has worked over the last 
two scores of years and allowed us to get to this point.

Honestly, if the Internet mail network was that chaotic, I highly 
doubt I will be doing this work since 1982, nor do I think 
RFC822/RFC821 would of taking over the existing mail networks at the 
time, commercial or otherwise.  There were at least 4-6 competing mail 
frameworks existing and in development in the 80s and RFC822/RFC821 
won out for many obvious reasons - its flexibility was one key reason, 
and mainly, it (predecessor, X.400) was government funded and already 
used in government and academia.

This is no different than the early telecommunications days when 
dealing with half/full duplex issues, client/host LF/CR translations 
issues or 7bit vs 8 bits file transfer protocols and which worked 
better over the different available wires, including X.25 networks 
(i.e. PAD configurations, etc).  How do you think ZMODEM got invented? 
  As well as KERMIT?   Before then, you has ASCII (7bit), then XMODEM, 
YMODEM, then ZMODEM and KERMIT, and it *not* ironic, the Postel principle:

         Be conservative in what you send; be liberal in what you accept.

always applied in data communications:

         Receive HIGH (buffers), Send Low (Buffers)

I always used the Bucket Brigade idea to illustrate this; A fireman is 
passing water in a bigger bucket than the next fireman in the brigade 
has -  you will have "flow control" issues or spillage.  Someone has 
to slow down.

The system works because we have an expectation for an optimal 
behavior across the board.  When one or more node begins to do things 
differently with an unrealistic unknown expectation for downlinks, its 
no surprise problems develop for some or many.  The only reason we are 
seeing it now, its because of this DKIM integrity invention highlight 
the issues.

---

From: Steve Atkins <[email-masked]>
Date: Tue, 24 May 2011 11:46:26 -0700

> Exim doesn't downgrade. It doesn't advertise 8bitmime either, by default. If you switch on 8bitmime advertising, it still doesn't downgrade. I think it just tries to deliver the mail as 8bit, regar…

Exchange advertises 8bit and then bounces the mail if it tries to forward it to a server that doesn't advertise 8bit.

This (entirely RFC valid yet completely broken) behaviour has bitten me a couple of times.

I'm not sure any of this affects DKIM much, other than "Hey, look! Yet another way DKIM signatures might get removed in transit!".

Cheers,
  Steve

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 May 2011 15:47:01 -0400

Steve Atkins wrote:

>> If you switch on 8bitmime advertising, it still doesn't downgrade. I 
>> think it just tries to deliver the mail as 8bit, regardless of what the 
>> receiving MTA does. I think postfix and sendmail do the same, but I'm 
not sure.
> This (entirely RFC valid yet completely broken) behaviour has bitten me 

+1

If everyone (mail transport/mail handlers) just followed the basic 
mail networking principle of:

     Thou should not touch passthru mail (except for network traces)

We would not be having these "side" issues today.  There are legit 
reasons for transformations but mainly at the final destination (local 
routing).

The 8BITMIME thing is ideal to tell MUA/MTA (at the Interactive Level) 
what is not acceptable at the MSA.  So if the USER has 8bit enable in 
his UI, then he can switch it to 7bit compatible format.   But its 
create problems when there is a "Mind Guessing" game between 
intermediaries.

> I'm not sure any of this affects DKIM much, other than "Hey, look! Yet 
> another way DKIM signatures might get removed in transit!".

The only solution I see for DKIM is to use Target/Path dependency 
methods, but in principle, intermediaries need to refrain from 
thinking they know better for downlinks.

hmmm, perhaps "DKIM Scouts" :)

---

From: Franck Martin <[email-masked]>
Date: Tue, 24 May 2011 19:56:48 +0000

On 5/24/11 12:47 , "Hector Santos" <[email-masked]> wrote:

>> Exchange advertises 8bit and then bounces the mail if it tries to
>>forward it 
>> to a server that doesn't advertise 8bit.
>> 

Are there the same issues with PGP or S/Mime email?

---

From: "John R. Levine" <[email-masked]>
Date: 24 May 2011 18:34:06 -0400

> This (entirely RFC valid yet completely broken) behaviour has bitten me a couple of times.

Better get used to it, since that's what EAI is going to do, too.

Regards,

---

From: "John R. Levine" <[email-masked]>
Date: 24 May 2011 18:35:37 -0400

> Are there the same issues with PGP or S/Mime email?

Generally no.  They're a group of MIME parts that shouldn't need to be 
recoded, or even if they are, will decode to the same value.

Regards,

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 24 May 2011 15:41:16 -0700

> Better get used to it, since that's what EAI is going to do, too.

Maybe yes, maybe no.

That actally means definitely yes, for some scenarios.  The maybe no means that 
it might not occur for some others.

One possibility is that the two paths are distinguished by near-term vs. 
long-term, assuming one believes that 'near-term' is a useful construct when 
projecting Internet-scale transitions of infrastructure service...

d/

-- 

   Dave Crocker

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 May 2011 19:49:21 -0400

Franck Martin wrote:

>> Steve Atkins mentioned:

> Hector followed up: 

RFC3851 (S/MIME) states this under the security section:

    Modification of the ciphertext can go undetected if authentication is
    not also used, which is the case when sending EnvelopedData without
    wrapping it in SignedData or enclosing SignedData within it.

IMO, with the known issues in the wild related to using MIME parts, I 
would say yes.  Since our MSA/MDA/MTA does not tamper with passthru 
mail and since we never heard of a complaint, it will suggest it 
didn't cause problems for any customer either. My general point is 
based on painful experiences learned with multiple different mail 
networking software (old and new) and the common and basic long 
traditional rule of thumb was to refrain from screwing around with 
passthru mail and when followed, things generally worked better, there 
were less issues, less surprises and future things would basically fit 
right in.

With new needs such as EAI (internalization) and DKIM 
(authentication), it is highlighting the cases where certain methods 
in the network were not ideal.

---

From: Franck Martin <[email-masked]>
Date: Wed, 25 May 2011 00:28:55 +0000

Interestingly enough, outlook tells me this message has been tampered
with, but not sure why...

---

From: Hector Santos <[email-masked]>
Date: Tue, 24 May 2011 20:38:10 -0400

Cry Wolf!   Thunderbird says "Not Digitally signed" :)

---

From: "John R. Levine" <[email-masked]>
Date: 24 May 2011 21:13:23 -0400

> with, but not sure why...

Probably doesn't have the Comodo validation certificate.

I took the copy of my message that came back from the mailing list, ran it 
through some rather violent transformations (mail to usenet and back) and 
the S/MIME signature still is OK.

R's,
John

---

From: Franck Martin <[email-masked]>
Date: Wed, 25 May 2011 02:06:27 +0000

It tells me signing and encryption certificates are valid and even their
root certificates are valid...

---

From: "John R. Levine" <[email-masked]>
Date: 24 May 2011 23:44:33 -0400

> root certificates are valid...

Well, something's wrong with it.  I checked the signature in Alpine, 
Thunderbird, and Evolution, and they all agree it's fine.

Regards,

---

From: Hector Santos <[email-masked]>
Date: Wed, 25 May 2011 00:12:45 -0400

> Thunderbird, and Evolution, and they all agree it's fine.

Not in Thunderbird V2.0, V3.1. It knows nothings about your signature 
- Click View | Message Security Info and it says:

       Message Has No Digital Signature
       Message Not Encrypted

What version of TBird did you use?

-- 
Sincerely

Hector Santos

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 25 May 2011 01:05:29 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Scott Kitterman
> Sent: Monday, May 23, 2011 10:12 AM

Interesting.  I ran some queries on our data for ebay.com, paypal.com, chase.com and bankofamerica.com.  In all cases, messages with failed signatures were never tagged by Spamassassin, and at most 7% (usually less) of unsigned mail where the From: field contained those domains was tagged.  This seems to concur with the "most breakage is innocent" theory and also supports the notion that treating a broken signature as equal to no signature is almost always the right way to go.

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 25 May 2011 10:13:23 +0000

> Probably doesn't have the Comodo validation certificate.

Maybe, but my Mac does, and it complains. As does a Thunderbird client and an Outlook client. That's three different setups that are complaining about your signature now.

Whether they're right or wrong is largely irrelevant. The fact is that S/MIME doesn't seem to be bullet proof.

> I took the copy of my message that came back from the mailing list, ran it through some rather violent transformations (mail to usenet and back) and the S/MIME signature still is OK.
> 
> R's,
> John
> 
>>

---

From: Michael Thomas <[email-masked]>
Date: Wed, 25 May 2011 07:02:40 -0700

> Interesting.  I ran some queries on our data for ebay.com, paypal.com, chase.com and bankofamerica.com.  In all cases, messages with failed signatures were never tagged by Spamassassin, and at most…

Heuristic based systems like SA are subject to the phases of the moon
with respect to what they find valuable and for how long. If they find
it useful to educe something from DKIM or lack thereof, more power to
them. Heck, if they just used the signature header pattern to determine
spam from ham for different senders, that would be cool too. This is not
in conflict from the statement that _cryptographically_ a broken signature
is no different than a missing signature. SA and its ilk just don't operate
on the plane of mathematical provables is all; nothing wrong with that.

Mike

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 25 May 2011 09:16:22 -0700

> From: Michael Thomas [mailto:[email-masked]]
> Sent: Wednesday, May 25, 2011 7:03 AM
> To: Murray S. Kucherawy
> Subject: Re: [ietf-dkim] No signatures, bad signatures, cousin domains
> 

My use of Spamassassin for correlation of DKIM verification versus message classification is really only a proof-of-concept thing.  Certainly connecting this stuff to a much more robust spam feedback system like the one Cloudmark has would produce far better results.  It's somewhere on my to-do list.

---

From: Hector Santos <[email-masked]>
Date: Wed, 25 May 2011 14:04:45 -0400

Alessandro, with the undotting leading dot fix, I went back and adding 
code to adjust for this by undotting it in the C14N code and what a 
major difference compared to the failed rate listed before:

      Failure rates for level encoding type (OLD)

       Failure rates for encoding type (NEW)
+----------------------------------------------+
| enctype             total   bodyfail   pct   |
|----------------------------------------------|
| 8bit                31      22         71.0  |
| binary              5       1          20.0  |
| base64              32      3          9.4   |
| quoted-printable    1206    107        8.9   |
| 7bit                1654    85         5.1   |
|                     5512    188        3.4   |
+----------------------------------------------+

+----------------------------------------------+
|                       OLD          NEW       |
| hash                  bodyfail     bodyfail  |
|----------------------------------------------|
| relaxed/relaxed       1085         238       |
| relaxed/simple        276          56        |
| simple/simple         215          112       |
+----------------------------------------------+

Overall

   OLD: 18.7% failure
   NEW:  4.8% failure

and the major contributor to this is that I have no more 
facebookmail.com failures!

When I remove the domains I know, the rest is pretty much spam. :)

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 25 May 2011 14:14:18 -0400

On Wednesday, May 25, 2011 02:04:45 PM Hector Santos wrote:
...
> When I remove the domains I know, the rest is pretty much spam.
...

Isn't that pretty generally true, DKIM or no DKIM.  

Scott K

---

From: Hector Santos <[email-masked]>
Date: Wed, 25 May 2011 14:34:12 -0400

> Isn't that pretty generally true, DKIM or no DKIM.  

Sure, in general I would agree with that and most of it are single 
shot deals.

There seem to be a pattern among the different spammer domains and 
that shouldn't be a surprise when using templates.

---

From: "John R. Levine" <[email-masked]>
Date: 25 May 2011 16:06:33 -0400

> Thunderbird, and Evolution, and they all agree it's fine.

I went back and looked in more detail.  The problem appears to be that 
this mailing list wraps the signed body in a MIME multipart/mixed section 
including both the signed message and the unsigned footer.  Some MUAs look 
inside the mixed and see the signature, some don't.  For the ones that do, 
I haven't checked to see how if at all they distinguish the signed part 
from the unsigned when they show you the message (shades of all the l= 
arguments.)

So this tells me that existing mail software doesn't try very hard to 
recover signatures from modified messages, even for simple changes that 
don't need any guessing or heuristics to undo.  Why would anyone think 
that the situation with DKIM would be any different?

Regards,

---

From: Hector Santos <[email-masked]>
Date: Wed, 25 May 2011 17:11:00 -0400

John R. Levine wrote:
> this mailing list wraps the signed body in a MIME multipart/mixed 
> section including both the signed message and the unsigned footer.  Some 
> MUAs look inside the mixed and see the signature, some don't.  For the 
> ones that do, I haven't checked to see how if at all they distinguish 
> the signed part from the unsigned when they show you the message (shades 
> of all the l= arguments.)
> 

I think you were telling people for a while now to use it for some reason.

Anyway, I don't think the problem is as you state.

 From what I see, I think the issue here is no MIME Application Type 
association for the application and extension your message was made. 
Your email provides this MIME info:

Content-Type: APPLICATION/pkcs7-signature; name=smime.p7s
Content-Transfer-Encoding: BASE64
Content-Description: S/MIME Cryptographic Signature
Content-Disposition: attachment; filename=smime.p7s

Software that offer auto launching (generally with user permission 
necessary) using MIME Application association, can either have its own 
table or depends on the OS to provide it.

For example, under Windows, you can use a console window using the 
ASSOC command to get the file type for the extension:

    ASSOC .p7s

and it returns

.p7s=P7SFile

then you use the FTYPE command to see what application is responsible 
to handle this file type.

    FTYPE P7SFile

and it returns:

p7sfile=rundll32.exe cryptext.dll,CryptExtOpenPKCS7 %1

So if the email has an attachment file name/extension smime.p7s,  then 
it should allow the user (with permission) to load it by running the 
above rundll32.exe process passing the rest of the line as parameters 
and substituting %1 with the local temp storage FQFN for smime.p7s

Now, under TBIRD, it apparently is independent of the built-in Windows 
MIME association table in the registry.  So one would think that 
Outlook would not have a problem and indeed it doesn't; it displays a 
"seal icon" save button for the SMIME.P7S attachment file when viewing 
the message under OE.

Under TBIRD, you can click

     TOOLS | OPTIONS

and under the Attachments properties tab, you will see the View & Edit 
Associations button.   For me, it is empty list - no MIME associations 
so shown as a file name "PART 1.2"

Now why didn't it atleast show SMIME.P7S as a file attachment?   You 
did have:

    Content-Disposition: attachment; filename=smime.p7s

You should check bugzilla or asks the TBIRD folks why it doesn't at 
least show the actual file name as the attachment because even if 
TBIRD doesn't have its own table, when the user tries to save it, 
Windows will most likely make the shell association.

BTW, under OE, while it does show the SMIME.P7S icon attachment, the 
message display is blank.  That may be related to what you are talking 
about.  In any case, its all fubar.

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 26 May 2011 10:07:31 +0000

On 25 May 2011, at 21:06, John R. Levine wrote:

> I went back and looked in more detail.  The problem appears to be that this mailing list wraps the signed body in a MIME multipart/mixed section including both the signed message and the unsigned footer.  Some MUAs look inside the mixed and see the signature, some don't.  For the ones that do, I haven't checked to see how if at all they distinguish the signed part from the unsigned when they show you the message (shades of all the l= arguments.)
> 
> So this tells me that existing mail software doesn't try very hard to recover signatures from modified messages, even for simple changes that don't need any guessing or heuristics to undo.

My client found the signature, otherwise it would not have commented on its validity. It just wasn't able to verify it.

>  Why would anyone think that the situation with DKIM would be any different?

I don't know. I had the impression that you were claiming that S/MIME would work better than DKIM here. Perhaps it does, but it still doesn't seem to be bullet proof.

I think the long term solution would be for mailing list software to stop mucking around with the message body, and for MUAs to work better at exposing meta data added by lists (like the list-unsubscribe header).

My guess is that if the top five MUAs and the top ten webmail services were all to make good use of list-unsubscribe and list-id headers (and perhaps others), then many list operators would not feel the need to mess around with message bodies and subject lines.

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 07:46:06 -0400

Ian Eiloart wrote:

> I think the long term solution would be for mailing list software to 
> stop mucking around with the message body, and for MUAs to work 
> better at exposing meta data added by lists (like the list-unsubscribe 
> header).

This is the sort of improved "Mail Integration" information that 
should be in a MLM document.

I increasingly believe we (the mail industry) needs a "Mail 
Integration" document similar to the RFC1123 document (I called the 
Bible) that help put it all together for online hosting systems with 
integrated Telnet, FTP and SMTP services.  It was because of RFC1123 
that helped RFC2822/RFC2821 get done. In fact RFC22821 updated RFC1123.

We need a new document that deals all the new modern Mail Integration 
needs, and it can cover most and not exclusively:

    - Traditional Email
    - NNTP
    - POP3
    - IMAP
    - Web Mail
    - Mobile Mail
    - List Servers, Mailing List
    - MUA (offline and online)
    - SUBMIT (PORT 587)
    - Legacy Mail Issues
    - The "New Rules" (i.e. PTR requirements, DISCARDING, etc)
    - DKIM
    - EAI, IDN

etc. One of the things that I got out of RFC1123 was the Hosting 
Requirements under one bird eyes view.  This new "Mail Integration 
Hosting Requirements" document would do a similar thing to get the 
consistent and persistent expected behavior in the new modern mail 
network.

> My guess is that if the top five MUAs and the top ten webmail services 
> were all to make good use of list-unsubscribe and list-id headers (and 
> perhaps others), then many list operators would not feel the need to 
> mess around with message bodies and subject lines.

In principle, passthru mail should not be tampered, but MLM list mail 
are the industry accepted exception to this non-tampering tradition 
and today (at least in the USA), it is CAN-SPAM legal requirement to 
have a viewable OPT-OUT text shown to the user to satisfy the CAN-SPAM 
"capitalistic" legal right provided to the VENDOR to due business with 
users.

Nonetheless, today, the list operators already has the option to not 
change the body, but I can see if and when it was a "BCP" that end 
users will see unsubscribe information by their MUA based on the meta 
data, then list software developers may not make it a default to set 
text footers.

 From my (developer, vendor) point of view, we need to cater to all 
markets. So for example, these are our defaults when an list operator 
prepares a new list (showing ones that could alter the mail)

     [X] Add [List-Name] subject tag
     [X] Allow Attachments
     [_] Strip HMTL Content
     [X] Use List Address for Reply-To
     [X] Keep Original To: Addresses
     [_] Add Body Header
     [X] Add Body Footer
     [X] Add LIST-* headers

I can see a change where we offer an option for a new mindset group of 
list operator thinking with Keep vs Change feature:

    (_) Keep Original Submission Integrity
        [X] Add LIST-* headers
        [_] Use List Address for Reply-To

    (o) Mail Submission Change Options
        [X] Add [List-Name] subject tag
        [X] Allow Attachments
        [_] Strip HMTL Content
        [X] Use List Address for Reply-To
        [X] Keep Original To: Addresses
        [_] Add Body Header
        [X] Add Body Footer
        [X] Add LIST-* headers

But remember, when there is a "need to change" it also comes with the 
idea did we do it right, the need to justify the change, the whys and 
was all the key design issues taken into account.  Just consider a 
change request where we will now need to double (Keep vs Change) the 
internal name space for options.

In other words, from my point of view, if we are doing this for DKIM, 
then the DKIM spectrum of design requirements are considered as well.

I think, in this case, that means honoring originating signed 
submissions and if thats the case, then in my engineering opinion, all 
top level DKIM "Mail Integration" design aspects are considered as 
well, including honoring ADSP:

    [X] DKIM options

        [X] Exclude ADSP restricted domains membership
        [X] Reject ADSP restricted domain submissions

        [X] Validate signatures
            [X] Add Authentication-Results

        [_] Sign Message  [Click for Signer Options]
            [X] Strip Original Signatures

In other words, when the DKIM (re)signing option is checked:

     [X] Sign Message [Click for Signer Options]

then the exclusively mutual (radio) option is automatically unchecked.:

     (_) Keep Original Submission Integrity

because it doesn't matter any more when the MLM is always (re)signing.

Anyway, IMV, what people need is insights and let them make their own 
decisions based on their own needs, but overall, the same outcome in 
all cases should be the intended goal.

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 26 May 2011 13:00:13 +0000

On 26 May 2011, at 12:46, Hector Santos wrote:

> 
> In principle, passthru mail should not be tampered, but MLM list mail are the industry accepted exception to this non-tampering tradition and today (at least in the USA), it is CAN-SPAM legal requirement to have a viewable OPT-OUT text shown to the user to satisfy the CAN-SPAM "capitalistic" legal right provided to the VENDOR to due business with users.

Yes, and are similar requirements across the European Union. In the UK, the phrase is "easy to use". That's not the case if the user interface doesn't obviously expose it.

Still, we're straying off topic here, I guess.

---

From: "John R. Levine" <[email-masked]>
Date: 26 May 2011 09:40:27 -0400

> My client found the signature, otherwise it would not have commented on its validity. It just wasn't able to verify it.

Hmmn.  How does it like the copy of this message sent to you directly? 
The signature was definitely good on the way out.

> stop mucking around with the message body, and for MUAs to work better 
> at exposing meta data added by lists (like the list-unsubscribe header).

Actually, I think the long term solution is for people to stop pretending 
there is a problem.  Can you describe the operational problems you're 
experiencing here?  "Broken signatures" is a fact, not a problem. 
Mailing lists have worked quite well for 40 years with no signatures at 
all, making all sorts of random changes to the mail, so it has to be 
something more than that.

Also, if you're suggesting changes to list software, please explain why 
they would have greater benefits than the obvious and simple one to have 
lists add their own signature on the way out.

Regards,

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 09:42:38 -0400

Ian Eiloart wrote:
>> In principle, passthru mail should not be tampered, but MLM list mail are 
>> the industry accepted exception to this non-tampering tradition and today 
>> (at least in the USA), it is CAN-SPAM legal requirement to have a 
>> viewable OPT-OUT text shown to the user to satisfy the CAN-SPAM 
users.

> Yes, and are similar requirements across the European Union. In the UK, 
> the phrase is "easy to use". That's not the case if the user interface 
> doesn't obviously expose it.
> 

It was a background side point, but nevertheless an important point to 
keep in mind when a suggestion is made list operators is expected to 
move towards not changing list submissions during the distribution. 
Whether via a footer or via a meta-data header intelligent modern mail 
viewers make use of, it needs to be shown and this is what I have to 
consider.

My take (as a list software developer) is that if these smart mail 
viewer are expected to exist, then we did our job by providing 
guidance to domain signers sending mail to a list:

  - use the "l=" tag
  - not hash RFC5322.Subject (not actually written in spec)

That way the list server can continue two to three things:

  - Add the LIST-* headers
  - Add the text footer
  - Add the Subject tag [LIST-NAME]

and the smart MUA now can:

  - Use the meta-data (List-Unsubscribe), and
  - not show the text beyond "l=" body length.

So it appears we have all the ingredients in place actually.

But for the legacy MUA, the visual footer would satisfy the "business" 
requirements.

Note: I am just describing what I have to consider for our List Server 
product, I am not thinking as a List Operator only.

---

From: "John R. Levine" <[email-masked]>
Date: 26 May 2011 09:51:41 -0400

>> Thunderbird, and Evolution, and they all agree it's fine.

FYI, I checked copies coming through the list in Apple Mail 4.5, and it 
sees the signature, too.  I was mistaken about T'bird, 3.1.9 on the Mac 
and 3.1.10 on FreeBSD don't.

Just out of nosiness, if your MUA doesn't think this message is signed 
could you drop me a private note saying what it is?  Tnx.

> I went back and looked in more detail.  The problem appears to be that this 
> mailing list wraps the signed body in a MIME multipart/mixed section 
> inside the mixed and see the signature, some don't.  For the ones that do, I 
> haven't checked to see how if at all they distinguish the signed part from 
> the unsigned when they show you the message (shades of all the l= arguments.)
>
> So this tells me that existing mail software doesn't try very hard to recover 
> signatures from modified messages, even for simple changes that don't need 
> any guessing or heuristics to undo.  Why would anyone think that the 
> situation with DKIM would be any different?
>
> Regards,

Regards,

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 10:00:49 -0400

John R. Levine wrote:
> Actually, I think the long term solution is for people to stop 
> pretending there is a problem.  Can you describe the operational 
> problems you're experiencing here? 

eh, non-DKIM intermediaries?

> "Broken signatures" is a fact, not a problem. 

Sure it is.   You are saying there is a SOLUTION - resign it.

> Mailing lists have worked quite well for 40 years with no 
> signatures at all, making all sorts of random changes to the mail, so it 
> has to be something more than that.

But it doesn't have 40 years of dealing with DKIM or authentication 
issues. DKIM was around 40 years ago, Mailing List software would be a 
whole lot different today.

> lists add their own signature on the way out.

Not all list operators are agreeing with your general always resign 
"Pottery Theorem" model and/or take over the "responsibility" of the 
originating domain, copyright holder author mail.

The Broken Signature Resign solution is only one solution. It doesn't 
cover all the problems for one reason only - you can't assume everyone 
is going to resign yet alone add DKIM to their software.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 07:05:08 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Ian Eiloart
> Sent: Thursday, May 26, 2011 3:08 AM
> To: John R. Levine
> Cc: DKIM List
> Subject: Re: [ietf-dkim] DKIM Scouts, was 8bit downgrades
> 

The MLM document does say that such a thing is preferable, but not expected to become a reality any time soon.  So really, we've already been down this road.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 12:02:42 -0700

> Sent: Thursday, May 26, 2011 6:40 AM
> To: Ian Eiloart
> Cc: DKIM List

Applying the same logic: Email in general has been fine without DKIM for 40 years, so why do we need it?

Thinking in abstract terms: If you accept the premise that DKIM delivers a validated domain name as its payload, and that domain name represents an ADMD that takes "some" responsibility for a message, then it's not clear to me why one would claim it's not valuable to have two responsible parties instead of just one.  You can then evaluate both of those names and decide if either of them, or perhaps the combination of them, warrant additional filtering or, instead, priority handling.

The question really is: How valuable is this?  Or put another way: Is it worth the work to make the two identities available instead of only that of the MLM?  I suspect the answer is "yes" as it can only improve your accuracy.  The only remaining issue is how hard it will be to make that happen, and whether or not the payoff is big enough to offset the pain.  That, I think, is the real thing that needs to be evaluated.

Now, those are abstract terms.  When argued in terms of passing an author signature through an MLM given modern realities, it does indeed sound like it's not worthwhile, because in that particular context you're not likely to see the stuff you want to filter coming via such paths in the first place.

But now invert that thinking.  Let's say your domain manages to acquire a positive reputation, but now you and I are on a re-signing MLM whose domain has no reputation or maybe even a slightly negative one.  Your reputation could trump that of the list, or could improve that of the list by your participation in it, at least from my perspective.  But for that to happen, your signature has to survive.

I don't think that's a concept that should be discarded out of hand just because MLMs have been the way they are for a long time and they're in the way of such innovations.  Updating them even a little might enable a host of useful new applications.

---

From: Steve Atkins <[email-masked]>
Date: Thu, 26 May 2011 12:21:19 -0700

> The question really is: How valuable is this?  Or put another way: Is it worth the work to make the two identities available instead of only that of the MLM?  I suspect the answer is "yes" as it ca…

In my experience with traditional discussion MLMs (which is the situation we're talking about) if I trust the MLM, I generally don't care about who the participants are.

While you're absolutely right that in this case having identities of two responsible parties (original author and MLM) is more valuable than one (MLM). But I think the increase in value is somewhere between marginal and negligible, so unless it comes "for free" it's probably not that interesting to try and do. And when we're talking about DKIM identities it's definitely not something that will be easy to do (it may not even be possible without seriously compromising either DKIM's promises or an MLMs usability).

> But now invert that thinking.  Let's say your domain manages to acquire a positive reputation, but now you and I are on a re-signing MLM whose domain has no reputation or maybe even a slightly nega…

The value of traditional MLMs is the discussion, rather than the individual post. The quantum of value is the thread, rather than the email.

If the reputation of the MLM is poor enough that mail from it is not being delivered, trumping that with an authors reputation may get individual emails delivered - but not threads, so it doesn't really improve the value provided to the recipient (it probably decreases it - a mailing list that delivers one in ten posts to my inbox is less useful than one that delivers none at all).

Cheers,
  Steve

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 13:02:50 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Steve Atkins
> Sent: Thursday, May 26, 2011 12:21 PM
> To: DKIM List
> Subject: Re: [ietf-dkim] MLMs and signatures again
> 
> In my experience with traditional discussion MLMs (which is the
> situation we're talking about) if I trust the MLM, I generally don't
> care about who the participants are.

Good, that's useful data.

> If the reputation of the MLM is poor enough that mail from it is not
> being delivered, trumping that with an authors reputation may get
> individual emails delivered - but not threads, so it doesn't really
> improve the value provided to the recipient (it probably decreases it -
> a mailing list that delivers one in ten posts to my inbox is less
> useful than one that delivers none at all).

Perhaps an MLM's reputation is pulled up or down as the average of those of its participants, so if the MLM can attract "good" senders, suddenly entire threads start getting through.  But that would only be possible with signature survival.

I don't know, I'm mostly brainstorming here.  The abstract idea seems reasonable but the MLM instance of it carries with it so much baggage that it's perhaps the worst possible example.

---

From: Steve Atkins <[email-masked]>
Date: Thu, 26 May 2011 13:21:00 -0700

On May 26, 2011, at 1:13 PM, Franck Martin wrote:

> 
> 
> On 5/26/11 12:21 , "Steve Atkins" <[email-masked]> wrote:
>> 
>> In my experience with traditional discussion MLMs (which is the situation
>> we're talking about) if I trust the MLM, I generally don't care about who
>> the participants are.
> 
> True, but the system in charge of delivering the email to your mailbox,
> does not know about this trust. So how can we infer you have given this
> trust to the mailing list? Is your trust the same as some other person?

The mailing list needs to build and maintain a good reputation, just
the same as any other source of email. That I, the final recipient, trust it and
want the mail it sends will help it do that.

> 
> So the receiving MTA, sees messages with List-id: headers in direction to
> your mailbox. What it shall do? The Receiving MTA does not usually know
> you have subscribed to the mailing list...

Nor does it care.

If I signed up for a mailing list, I'm going to want to receive email from
it. I'm not going to report it as spam, I'm going to go looking for missing
mails (especially the initial COI challenge) in my spam folder.

All this behaviour will give the mailing list manager a good reputation
with my ISP. Similar behaviour by other recipients will also give it a
good reputation, all tied to the DKIM d= value.

This isn't a special case - it's just incoming email that's coming from
a sender with a good reputation, tied to the MLMs d= token.

> 1) as Murray says, It can infer it has to deliver (or not) the email based
> on other participants reputation to build a list reputation? side note:

Eh. If the signature of a particular sender happens to survive, and
that sender happens to have a better reputation than that of the list
then it might make some difference. It's not something I'd actually try
and do, though.

> do
> mail receivers treat mailing list differently than any other emails?

Generally not, I don't think, though MUAs do.

> 
> 2) do we need a mechanism to alert the receiving MTA that you have
> subscribed to a mailing list, and all messages should pass through?

Quite possibly, but that's really not DKIM related at all, more of an MUA
design decision.

Cheers,
  Steve

---

From: "John R. Levine" <[email-masked]>
Date: 26 May 2011 16:29:05 -0400

> Perhaps an MLM's reputation is pulled up or down as the average of those of its participants, so if the MLM can attract "good" senders, suddenly entire threads start getting through.  But that woul…

In my experience, the reputation of the list is unrelated to the 
reputation of its participants.  For example, in my filters I deliver mail 
from this list directly into the inbox without content filtering, even 
though I discard mail sent directly from a few of the subscribers.

With 40 years of experience with MLMs, a lot of experiments have already 
happened, and we should spend more time looking at the history rather than 
guessing what might happen under some hypothetical circumstances.  For 
example, we don't have to do experiments to find out whether people want 
an MUA to distingish between signed and unsigned parts of a message. 
We've already had partially signed messages (like this one, if you get it 
through the list) for over a decade, and MUAs don't care.  Either they 
don't see the signature at all (Thunderbird or Windows Live Mail), or they 
show the message without any particular distinction between the signed and 
unsigned parts (Evolution, Apple Mail, Alpine.)

If anyone's claiming that contributors' DKIM signatures on list mail are 
important, a good start would be to look at how PGP and S/MIME signatures 
have been treated during the many years they've been in use.  I don't see 
any harm in experiments like having an MLM adding a signed A-R header to 
the mail, since it doesn't break anything that works now, but I would want 
rather concrete evidence from anyone claiming that people pay any more 
attention than they do to S/MIME signatures now.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 13:36:38 -0700

> From: John R. Levine [mailto:[email-masked]]
> Sent: Thursday, May 26, 2011 1:29 PM
> To: Murray S. Kucherawy
> Cc: DKIM List

There are parties that want to do that experiment because they see potential in it.  (In fact they're doing it now through a non-standard hack; I need to ask them for results.)  What would probably be helpful is some decent description from them of why they think it's valuable and how they plan to use it.

You're absolutely right that nobody's cared up until now, but I'm not as sure as you are that this makes the question utterly uninteresting.

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 26 May 2011 13:37:36 -0700

On 5/26/2011 1:29 PM, John R. Levine wrote:
> In my experience, the reputation of the list is unrelated to the reputation of
> its participants.

Given how little DKIM-related reputation work has been done, deployed and 
heavily used so far, perhaps we should all be a bit cautious about taking 
existing practices and treating them as definitive of future needs and uses.

d/
-- 

   Dave Crocker

---

From: Franck Martin <[email-masked]>
Date: Thu, 26 May 2011 20:13:16 +0000

On 5/26/11 12:21 , "Steve Atkins" <[email-masked]> wrote:

>>> From: [email-masked]
>>>[mailto:[email-masked]] On Behalf Of John R. Levine
>> Thinking in abstract terms: If you accept the premise that DKIM
>>delivers a validated domain name as its payload, and that domain name
>>represents an ADMD that takes "some" responsibility for a message, then
>>it's not clear to me why one would claim it's not valuable to have two
>>responsible parties instead of just one.  You can then evaluate both of
>>those names and decide if either of them, or perhaps the combination of
>>them, warrant additional filtering or, instead, priority handling.
>> 
>> The question really is: How valuable is this?  Or put another way: Is
>>it worth the work to make the two identities available instead of only
>>that of the MLM?  I suspect the answer is "yes" as it can only improve
>>your accuracy.  The only remaining issue is how hard it will be to make
>>that happen, and whether or not the payoff is big enough to offset the
>>pain.  That, I think, is the real thing that needs to be evaluated.
>

1) as Murray says, It can infer it has to deliver (or not) the email based
on other participants reputation to build a list reputation? side note: do
mail receivers treat mailing list differently than any other emails?

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 16:50:45 -0400

Steve Atkins wrote:

> we're talking about) if I trust the MLM, I generally don't care about 
> who the participants are.

If by traditional, you mean the members are vetted with subscription
and confirmation, then this tends to be true.  But when not, when the
list or any group forum is anonymous in nature, history has told us
its get corrupted with junk and most people tend to dislike it.

---

From: Steve Atkins <[email-masked]>
Date: Thu, 26 May 2011 14:10:18 -0700

> its get corrupted with junk and most people tend to dislike it.

In that case the reputation of the MLM is poor, and I don't want to
receive email from it. I still don't care about who the participants
are.

The idea that people might sign up for a mailing list full of junk,
and hope that their spam filters / reputation engine will magically
pull the occasional gem out of it seems pretty unlikely. And that's
the premise behind there being value in tracking the reputation
of original authors in the case of their email being re-sent by a
MLM.

Cheers,
  Steve

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 17:41:33 -0400

> receive email from it. I still don't care about who the participants

But it is the participants that make up the quality of the public 
discussion group.  Sure, you're point is clear that you may not pay 
attention to the individuals.  But others do.

For the most part, most people will agree Google is a good intention 
and "reputable" company, not out to harm people, etc.  But do we 
always trust all their google groups because it was signed by google? 
  I don't think so.

In my opinion, its more about the particular list of interest one has 
and its make up of people that make it a quality discussion group. 
You (speaking in general) selected the list you wanted, so it should 
go without saying you like and trust the list.

Does certification change that? Sounds more like an marketing issue:

     Join our discussing list.  Approved by VeriSign as a
     Trusted List Vendor!

I think the focus has been lost because the real problem is how to 
deal with the unknowns signer. We are trying to figure out a way to 
get a 3rd party to vouch for them - a 3rd party you always trust.

I don't think DKIM changes much of one already subscribe to a list 
that he has interest in.

Now, if you were to say:

    I only will accept list mail when its signed because I expect
    it to be signed and anything else not signed "purported" to be
    by the list I will not accept or ignore.

Then we are talking about something else.

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 17:48:44 -0400

> mail receivers treat mailing list differently than any other emails?

Yes, it is auto white listed for acceptance. In other words, when the 
client issues:

     RCPT TO: <list-name @ list-host.com>

it will checked as an normal user (alias) and accepted.

> 2) do we need a mechanism to alert the receiving MTA that you 
> have subscribed to a mailing list, and all messages should pass 
> through?

Normally, once it accepted (1st question via RCPT TO:), then the mail 
is pass to a list server and it will check for member subscription.

I guess if the RECEIVER is a List Server SMTP Server, then its 
database will be easily accessible to do a member check at SMTP level.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 14:52:31 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Franck Martin
> Sent: Thursday, May 26, 2011 1:13 PM
> To: Steve Atkins; DKIM List
> side note: do

That's a local policy question.  Personally, I don't know of any that do.

> subscribed to a mailing list, and all messages should pass through?

Certainly a possible feature, but it seems like it won't scale very well.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 14:53:36 -0700

> Sent: Thursday, May 26, 2011 2:10 PM
> To: DKIM List

Let's say I route all traffic from list X to its own separate mailbox, but I also want my MUA to flag for special attention mail sent to that list by people I hold in high regard, for example, and I want that to be based on their accumulated reputations.  I either have to base that on something forgeable like From:, or on something reliable like "d=".  That doesn't seem magical to me.

It's a bit of a contrived example, but right now I would have to maintain that list manually; it would be nice to have it done automatically based on feedback I provide to a reputation system.

---

From: Franck Martin <[email-masked]>
Date: Thu, 26 May 2011 21:59:37 +0000

>it will checked as an normal user (alias) and accepted.

I meant as a receiver of mailing list email, does your MTA do something
special when it sees the List-id: header?

---

From: Steve Atkins <[email-masked]>
Date: Thu, 26 May 2011 15:19:36 -0700

On May 26, 2011, at 2:53 PM, Murray S. Kucherawy wrote:

> Let's say I route all traffic from list X to its own separate mailbox, but I also want my MUA to flag for special attention mail sent to that list by people I hold in high regard, for example, and I want that to be based on their accumulated reputations.  

That's relying on an awful lot of vaporware in the MUA, orthogonal to any sort of authentication. I don't think any MUAs really track sender reputation in any way[1].

> I either have to base that on something forgeable like From:, or on something reliable like "d=".  That doesn't seem magical to me.

Well, d= won't identify the original sender at all, in the case of individuals sending to a mailing list. It'll identify the domain of their ISP, nothing more.

> It's a bit of a contrived example, but right now I would have to maintain that list manually; it would be nice to have it done automatically based on feedback I provide to a reputation system.

Tunneling DKIM signatures through MLMs doesn't seem to be the missing bit of technology needed to do this.

If the MLM signs any email it sends then you have some level of trust in any information it annotates the mail with.

*If* it were possible to identify the original email author in some way (S/MIME, PGP, some private shared secret approach....) the MLM could annotate the mail with that information, and you could trust it enough to filter on. If the MLM doesn't have enough information to identify the original email author, it's unlikely you do either - whether there's a second DKIM signature or not.

Cheers,
  Steve

[1] It's something that'd be useful, though - it's been on my TODO list for about two years to add exactly this to our CRM system, via end-user thumbs-up / thumbs-down buttons.

---

From: "John Levine" <[email-masked]>
Date: 26 May 2011 22:21:10 -0000

>Let's say I route all traffic from list X to its own separate
>mailbox, but I also want my MUA to flag for special attention mail
>sent to that list by people I hold in high regard, for example, and I
>want that to be based on their accumulated reputations.  I either
>have to base that on something forgeable like From:, or on something
>reliable like "d=".  That doesn't seem magical to me.

In my experience, if a mailing list is worth delivering at all, the
addresses on the From: line are plenty reliable for bozo or anti-bozo
filtering, and don't require an extra magic step to decide whether the
signature is sufficiently related to the author's address.  A plan
that expects every contributor to have a separate d= reputation domain
seems pretty unlikely to work outside the lab.

Maybe I'm not not imaginative enough, but all the scenarios for
recipients using contributor signatures are either things we are doing
already without signatures, or things that nobody has shown any
interest in doing in the past several decades even though there were
other ways to do them.

I can think of some reasonable uses for contributor signatures at the
MLM, e.g., skip the verification step for adds or removes if enough
previous requests from the same signer were confirmed.  But not for
passing them through to the recipients.

As I've said, I'm not opposed to experiments so long as they don't
involve breaking things that work now.  So adding a signed A-R is
fine, removing signature tags and headers and footers is not.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 26 May 2011 22:23:55 -0000

>> reputation of its participants.

>existing practices and treating them as definitive of future needs and uses.

In case it wasn't clear, I wasn't referring to DKIM reputation.  I
whitelist mail from lists I'm subscribed to using List-ID or some
other bits of stable header text.  In theory evil people could forge
them, in practice it's never been a problem.  So having a DKIM
signature to be the stable text would be nice, but wouldn't
fundamentally change what I do already.

R's,
John

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 15:24:43 -0700

> Sent: Thursday, May 26, 2011 3:20 PM
> To: DKIM List
> That's relying on an awful lot of vaporware in the MUA, orthogonal to
> any sort of authentication. I don't think any MUAs really track sender
> reputation in any way[1].

It's not vapourware in general.  Such feedback systems exist, and could easily be tied to DKIM domains.

> Well, d= won't identify the original sender at all, in the case of
> individuals sending to a mailing list. It'll identify the domain of
> their ISP, nothing more.

Well, right.  You'd be basing decisions on validated DKIM "d=" values.

> If the MLM signs any email it sends then you have some level of trust
> in any information it annotates the mail with.

Yes, and A-R provides a mechanism for doing that as well.  It's mentioned in the MLM draft too.

> *If* it were possible to identify the original email author in some way
> (S/MIME, PGP, some private shared secret approach....) the MLM could
> annotate the mail with that information, and you could trust it enough
> to filter on. If the MLM doesn't have enough information to identify
> the original email author, it's unlikely you do either - whether
> there's a second DKIM signature or not.

Why the last part of that?

> [1] It's something that'd be useful, though - it's been on my TODO list
> for about two years to add exactly this to our CRM system, via end-user
> thumbs-up / thumbs-down buttons.

We have that at Cloudmark, and there's an open one as well.  I'm trying to figure out if and how such a system could be used when correlated with DKIM signatures.

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 18:31:25 -0400

First, lets tune down the "40" years thing. What email list system was 
around in 1970? Its more like 26 years with ListServ (circa 1985) 
among the first and leading the way for the rest of the list server 
developers.

There was, of course, list via X.400 but it was most of a "CC" like 
mailing list and that was more of exclusive entry - not for the public 
and I don't recall any real concerns about security other than being 
accused of being a SPY!

When talking about the public, BBSes and Fidonet predated the Internet 
and the Fidonet Echo Networking technology was the closest thing to 
having network-based public groupware/discussions system. Before that, 
probably CompuServ offers ideas of public groupware discussion areas 
with there GO groups. We had GO XPRESS.  You also have Prodigy with 
public discussion groups.  But outside of these fee base dialups into 
X.25 networks, BBSes were among the first public way to have social 
group telecommunications.

Anyway, needless to say, if DKIM was around even 50 years ago, or the 
idea of authenticated email was around, list system and the entire 
mail system would of taking on an entirely different path.  We are 
arguing it now. I don't see why we would not be arguing about it back 
then if it was around.

So all this about the the past is really a moot point.  We have DKIM 
now, today, and it doesn't fit with list systems or any system that 
has a natural integrity breaking process.  Unless all the list 
software and/or operators add Plug and Play hooks, to do the "Always 
Resign" thing you want, we will always have the problems for a very 
long time.

John R. Levine wrote:
>> Perhaps an MLM's reputation is pulled up or down as the average of 
>> those of its participants, so if the MLM can attract "good" senders, 
>> suddenly entire threads start getting through.  But that would only be 
>> possible with signature survival.
> 
> reputation of its participants.  For example, in my filters I deliver 
> mail from this list directly into the inbox without content filtering, 
> even though I discard mail sent directly from a few of the subscribers.
> 
> happened, and we should spend more time looking at the history rather 
> than guessing what might happen under some hypothetical circumstances.  
> For example, we don't have to do experiments to find out whether people 
> want an MUA to distingish between signed and unsigned parts of a 
> message. We've already had partially signed messages (like this one, if 
> you get it through the list) for over a decade, and MUAs don't care.  
> Either they don't see the signature at all (Thunderbird or Windows Live 
> Mail), or they show the message without any particular distinction 
> between the signed and unsigned parts (Evolution, Apple Mail, Alpine.)
> 
> important, a good start would be to look at how PGP and S/MIME 
> signatures have been treated during the many years they've been in use.  
> I don't see any harm in experiments like having an MLM adding a signed 
> A-R header to the mail, since it doesn't break anything that works now, 
> but I would want rather concrete evidence from anyone claiming that 
> people pay any more attention than they do to S/MIME signatures now.
> 
> Regards,

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 18:45:55 -0400

Murray S. Kucherawy wrote:
>> Franck Martin

Almost all SMTP receivers that have an acceptable table and if the 
operator has a separate list server running on the system, the list 
server can use this acceptance table to update it for the MTA server 
to use. Since RCPT TO checking is increasingly done, this is necessary 
otherwise the list address is unknown to the receiver.  If the mail is 
accepted unchecked, then a post smtp processor also checks the table. 
It could also queue the mail for a list which the list server is 
eyeballing and it does the checking.

> Certainly a possible feature, but it seems like it won't scale 

That is what was said in the past to not do RCPT TO checking. That 
changed with the advancements in hardware and Multi-threaded OSes.  As 
you know, database lookups are darn fast and scale.

The only reason we don't do it with our wcSMTP server and wcListServer 
is because the list server is sold separately and doesn't need our 
wcSMTP server.  But an integrated database hook can be added. 
Probably the other reason is that we don't want to have an official 
SMTP reject, and want the list server to create a non-bounce 
notification.  But I can see where this be a good idea to do now - 
SMTP level rejects with response text "User not member of so and so list."

---

From: Steve Atkins <[email-masked]>
Date: Thu, 26 May 2011 15:46:45 -0700

> It's not vapourware in general.  Such feedback systems exist, and could easily be tied to DKIM domains.

I don't think they exist at the MUA level, keyed on senders. I'd
be interested to hear about them if they do.

(There are bunches of end-user visible reputation systems that
have UI in the MUA, of course, but they don't track reputation
on a per-end-user basis, rather they feed end-user perception
into a shared reputation system).

> Well, right.  You'd be basing decisions on validated DKIM "d=" values.

Which isn't good enough to differentiate between [email-masked] and
[email-masked]. If Hector starts forging his From: address to pretend
to be Cleo, DKIM doesn't help me at all. If he doesn't then I'm probably
fine just keying on Cleo's From: field.

> Why the last part of that?

It's going to be a rare case where the final recipient can reliably authenticate
the original author of the email, while the MLM can't. (There are exceptions - 
but if a cooperating group of people are using untrusted
infrastructure to communicate, they're not going to be relying on DKIM,
rather they're going to be living on paranoia, cigarettes and OpenGPG).

Normally, if you can authenticate the original author then the MLM can do
so just as well, so you can reliably route email based on metadata added
by the MLM, rather than having to independently authenticate the original
author yourself.

Cheers,
  Steve

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 18:53:11 -0400

> special when it sees the List-id: header?

Yes. Some of my RBM (Rule Based Messaging) scripts at the DATA level 
check for specific LIST-ID among other things for acceptance or 
rejection. But thats local policy and not a stock script we distribute.

And now with DKIM, I am exploring similar scripts checking LIST-ID to 
match the signing side where the List-ID is used for signing specific 
list only when its going out.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 26 May 2011 19:06:40 -0400

On Thursday, May 26, 2011 03:21:19 PM Steve Atkins wrote:
> If the reputation of the MLM is poor enough that mail from it is not being
> delivered, trumping that with an authors reputation may get individual
> emails delivered - but not threads, so it doesn't really improve the value
> provided to the recipient (it probably decreases it - a mailing list that
> delivers one in ten posts to my inbox is less useful than one that
> delivers none at all).

I think this has it rather backwards.  If mail From (body From) a certain 
domain arrives 999 time with a valid DKIM signature and on the 1,000th time it 
arrives with either no signature or a broken one, then that's a negative 
anomaly in the mail stream that receivers are quite likely to take notice of.  
While ADSP is the public whipping boy for this, there are plenty of private 
efforts based on doing exactly this.

The question isn't do I trust the ML or not.  For domains with a non-trivial 
number of users the overall mail system will have no idea about what ML should 
be trusted or not.  The question is how harshly do I treat this message based 
on the lack of a good signature.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 16:14:01 -0700

> Sent: Thursday, May 26, 2011 3:47 PM
> To: DKIM List

Whether the reputation is made visible as a property in the UI versus in the accept/reject/discard decision at delivery time isn't an important distinction, is it?

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Thu, 26 May 2011 19:15:25 -0400

> From: [email-masked] [mailto:ietf-dkim-
> [email-masked]] On Behalf Of Scott Kitterman
> Sent: Thursday, May 26, 2011 7:07 PM
> being
> number of users the overall mail system will have no idea about what
ML
> should
> be trusted or not.  The question is how harshly do I treat this
message
> based

The other piece of the equation is how often do I see abusive mail
purporting to be from this domain with no signature while mail from this
domain that is normally signed has no significant problems.

Mike

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 16:40:17 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of MH Michael Hammer (5304)
> Sent: Thursday, May 26, 2011 4:15 PM
> To: Scott Kitterman; [email-masked]

I posted the results of some research on that very question earlier this week:

http://mipassoc.org/pipermail/ietf-dkim/2011q2/016656.html

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 26 May 2011 19:43:14 -0400

On Thursday, May 26, 2011 07:15:25 PM MH Michael Hammer (5304) wrote:
> > ML should
> > message based

True.  That could be a factor as well.  

Scott K

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 20:14:54 -0400

> domain that is normally signed has no significant problems.

That's an exclusive reject opportunistic question.

In other words, if I turn off my SMTP level rejects for all of our 
domain abuse, would DKIM take up that slack?

I'm going to do a quick scan just for today's log where we rejected 
mail purported to be from our domains us, santronics.com, 
winserver.com, isdg.net.  Remember, this is just today (May 26, 2011) 
and so far its 8PM EST:

MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM:<[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM:<[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>
MAIL FROM: <[email-masked]>

None of these are valid and they were all rejected via SPF and the 
same for fake HELO/EHLO domains.

Now, since we now signing all these three domains, the question is, if 
they were checked at the DATA level using my DKIM+ADSP/ATPS/ACL setup 
reject them?

Yes, 100%, I don't know if they were faked signers or they used 3rd 
party signers, or they were signed all, because they were accepted. 
But a DKIM policy that I have would of 100% rejected them all.

This is partly the reason I didn't like Sender-ID because it was a 
RFC5322 payload technology and SPF did the job at the SMTP level.  I 
had shown that over 82-84% of the time and it would been a waste in 
DATA overhead.

I also feel that is why DKIM is having a hard time - SPF did a lot of 
damage to its purpose in life.

In any case, we are not doing any REJECT/PASS handling based on DKIM 
yet, but I am going to try turning off SPF for my domains and see if I 
get the expected 100% "would-be" rejects based on DKIM and my ADSP 
policies.

---

From: Hector Santos <[email-masked]>
Date: Thu, 26 May 2011 20:27:24 -0400

BTW, these are our May Rejections stats:

    http://www.winserver.com/public/antispam/stats/stats-2011-May.wct
    http://www.winserver.com/public/spamstats.wct (since 2003)

The LMAP column is SPF and its been should a high +6% and I say high 
because only this year only has it been that high. Before that, it was 
in the 1-4% range.

So if most of the 6% SPF rejects are spoof attempts on our domains, 
then I have no reason to believe that DKIM plus our ADSP/ATPS/ASL 
policies would not yield the same result.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 26 May 2011 20:36:13 -0400

On Thursday, May 26, 2011 07:40:17 PM Murray S. Kucherawy wrote:
> > [mailto:[email-masked]] On Behalf Of MH Michael Hammer
> > (5304) Sent: Thursday, May 26, 2011 4:15 PM

My experience is it varies a lot by domain.  Some domains are phishing targets 
and some aren't.  If it's not a phishing target DKIM doesn't matter much 
either way.  If it is, then if they can manage to sign all their outbound mail 
signed/not signed gets to be useful.  So I don't think looking at global 
status is a very useful basis for deciding the question.

Scott K

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 26 May 2011 20:00:04 -0700

> Sent: Thursday, May 26, 2011 5:36 PM

So you'd rather I run this on some signing domains that aren't obvious phish targets?  I can do that.  If you have a few you think might be interesting, send me the names; if not, I can see if I can come up with some just based on the numbers.

And I can constrain it to a specific reporting site (e.g., my own) instead of all reporters if you think that gives a more interesting view.

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 26 May 2011 23:42:26 -0400

On Thursday, May 26, 2011 11:00:04 PM Murray S. Kucherawy wrote:
> > [mailto:[email-masked]] On Behalf Of Scott Kitterman
> > My experience is it varies a lot by domain.  Some domains are phishing
> > targets and some aren't.  If it's not a phishing target DKIM doesn't
> > matter much either way.  If it is, then if they can manage to sign all
> > their outbound mail signed/not signed gets to be useful.  So I don't
> > think looking at global status is a very useful basis for deciding the
> > question.
> 
> So you'd rather I run this on some signing domains that aren't obvious
> phish targets?  I can do that.  If you have a few you think might be
> interesting, send me the names; if not, I can see if I can come up with
> some just based on the numbers.
> 
> And I can constrain it to a specific reporting site (e.g., my own) instead
> of all reporters if you think that gives a more interesting view.

I was thinking the opposite.  Look at phish targets that sign pretty reliably.  
I'll contact you offlist with some ideas on which.

Scott K

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 May 2011 01:43:43 -0400

Murray S. Kucherawy wrote:
> So you'd rather I run this on some signing domains that aren't 
> obvious phish targets?  I can do that.  If you have a few you think 
> might be interesting, send me the names; if not, I can see if I can come 
> up with some just based on the numbers.
> 
> And I can constrain it to a specific reporting site (e.g., my own) 
> instead of all reporters if you think that gives a more interesting view.

This sounds like you are missing a point here.  But it might help to 
know a general makeup of the volume collection you have from the 
standpoint if it was already pre-filtered.  I guess you won't readily 
know that without asking your contributors, but it would be good know 
what level, if any, filtering was already done.

The reason why I ask is because many systems reject mail before it is 
accepted and this can mask the value of RFC5322 (payload) evaluations. 
In addition, it quite often very difficult (if not possible) to turn 
it off just to see how DKIM will work.

For your collection analysis, you will need a majority of the system 
with "always accept" first operations so that you can get the large 
spectrum of bad vs good mail. Then you will need a criteria for what 
is considered "bad."

Did you see my post where I showed how large the local hosted domain 
spoofing is a real problem?

For our receiver, we have 4 checks at the MAIL FROM: SMTP state:

    o CLHRP - Check Local Host Return Path
    o RPF   - Operator defined Return Path Filter rules
    o SPF
    o CBV   - Call Back Verifier

For CLHRP, if the domain part is a locally hosted domain, then the 
user account is checked. It must be a valid user account.

For RPF, operators create their own rules, and normally it could be a 
lightweight SPF-like conditions to help avoid the next SPF DNS based 
check for local domains:

     REJECT IF %RPD% = "santronics.com" AND %IP% !IN 208.247.131.*
     REJECT IF %RPD% = "winserver.com"  AND %IP% !IN 208.247.131.*
     REJECT IF %RPD% = "isdg.net"       AND %IP% !IN 208.247.131.*

After that SPF is checked and then CBV (which BTW is the highest 
rejection for us, bad return paths is a real problem).

Its not just me, any SPF domain will have the same high benefit of 
protected against local domain spoofs.  So with these MAIL FROM check 
alone, it can very well hide the exclusive value and benefits of DKIM 
for unauthorized signers or invalid/no signatures states.

BTW, one reason why we have such a high reject is because we were 
ISP/ESP in the 90's with our PPP and RADIUS servers and got of that 
business in 1998.  We had around 80K "Free Domain" user accounts and 
we still get a consistent 60% RCPT rejection of dirty or inactive 
accounts.  It has never let up.

I suspect the YAHOO, GOOGLE, AOL, if they are among your collection, 
they probably reject a lot today at the SMTP level.  In the past, many 
didn't and always accepted first for scalability reasons.  But 
machines are much faster today, software is more dynamic in its 
checking especially with the terrible accept/bounce problem everyone 
is trying to avoid with SMTP rejects.

I am just saying, it will help to know to some extent what is the make 
up of the collection you have from a pre-filter standpoint. If most of 
it is pre-filtered, then extracting the various value of DKIM is 
masked or lost.

---

From: "MH Michael Hammer (5304)" <[email-masked]>
Date: Fri, 27 May 2011 08:24:42 -0400

> Sent: Thursday, May 26, 2011 8:36 PM
> > > purporting to be from this domain with no signature while mail
from
> this
> > I posted the results of some research on that very question earlier
> this
> > week:
> >
> targets

Remember, it's not static, it's dynamic. What was a non-phished domain
yesterday could be a phished domain today or tomorrow. DKIM isn't a
magic bullet, it's one more tool in the toolbox. I've found that in
combination with SPF it works very nicely on double fail and none/fail
as far as catching badness with very little impact on legitimate mail.

Mike

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 27 May 2011 18:08:04 +0200

On 26/May/11 23:52, Murray S. Kucherawy wrote:
>> From: On Behalf Of Franck Martin
> 

Yes, desperately.

> Certainly a possible feature, but it seems like it won't scale very well.

Why not?  Of course, having a copy of each subscription record would
roughly double the database, globally.  "Twice" is scalable, though.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: "John R. Levine" <[email-masked]>
Date: 27 May 2011 12:29:40 -0400

> Why not?

If I were a spammer, I would tell the victim's MTA that the victim 
subscribed, then send the spam.

These days most subscriptions are entered on a web page, and if you're 
lucky the mailer will send a confirmation message with a URL that sends 
the subscriber back to the web page.  Where's the MTA going to get the 
subscriber info? The challenges in designing a protocol that neither makes 
unreasonable demands on users and MUAs nor is easily spoofed by hostile 
mailers seem insurmountable to me.  If you're planning to keep a 
reputation database of mailers who send credible subscription 
announcements, why not just whitelist their mail?

Since as far as I know nobody does this, it's a resarch topic, so I've 
directed replies to the ASRG.  See you there.

Regards,

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 27 May 2011 09:47:07 -0700

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Alessandro Vesely
> Sent: Friday, May 27, 2011 9:08 AM

An automated system to monitor mail flows to figure out lists to which users have subscribed or unsubscribed can't scale unless there are standards around how to do that, and everyone participates.  That's a high bar to set.

A manual system requires users to register lists they join or depart.  That's bound to be increasingly inaccurate over time.  And if every Gmail user subscribes to just two lists, that's an awfully large number of relationships to track and check on each message transiting their MTAs.

I could be wrong, but it sounds like a nightmare.

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Fri, 27 May 2011 10:17:17 -0700

> Sent: Thursday, May 26, 2011 10:44 PM
> This sounds like you are missing a point here.

And what point is that?

> But it might help to

All reporting sites are doing at least some RBL filtering, and all spam/not-spam flags are Spamassassin verdicts plus a few user-provided verdicts thrown in.

> spectrum of bad vs good mail. Then you will need a criteria for what

I think that's unnecessary.  If we can assume our reporting sites are typical, then the results are typically meaningful.  It just means the results have to be taken in the same context in which the data were collected, which seems reasonable to me.

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 27 May 2011 20:17:02 +0200

> subscribed, then send the spam.

Yes, but then the MTA would then know how to treat the corresponding
victim's complaint.

> subscriber info?

An http url could be advertised on the MX's EHLO reply.  That's a page
on the users' server, where they can confirm subscriptions along with
their credentials.  MLMs get digitally signed confirmations of COI.

Ways to accept non-interactive subscription notifications are also
necessary.  I agree they are somewhat more challenging.

For MLMs that present DKIM-signed List-* header fields, subscriptions
might be assumed, and tracked unilaterally that way.  But unsubscribe
attempts seem to be more difficult to do, in this case.

> The challenges in designing a protocol that neither makes 
> unreasonable demands on users and MUAs nor is easily spoofed by
> hostile mailers seem insurmountable to me.

MUAs wouldn't be much affected, except for allowing TiS buttons.

Users would gain uniform subscribe/unsubscribe pages.

> If you're planning to keep a reputation database of mailers who
> send credible subscription announcements, why not just whitelist
> their mail?

The ability to maintain such database would be improved.

> directed replies to the ASRG.  See you there.

Here I am.  I've kept a CC to DKIM list, to be removed in followups.

It is not the first time I bring up this idea on the ASRG.  It
addresses newsletters more than discussion lists, because of the way
subscriptions work.  In Europe, the resulting protocol could help
providing proofs of consent, which many European newsletters handle as
a checkbox on _manually_ signed forms.  Indeed, I'd consider it an
implementation of the Data Protection Directive.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 May 2011 15:50:41 -0400

Murray S. Kucherawy wrote:
>> Alessandro Vesely

> An automated system to monitor mail flows to figure out lists to 
> which users have subscribed or unsubscribed can't scale unless 
> there are standards around how to do that, and everyone participates.  
> That's a high bar to set.
> 
> A manual system requires users to register lists they join or depart.  
> That's bound to be increasingly inaccurate over time.  And if every Gmail 
> user subscribes to just two lists, that's an awfully large number of 
> relationships to track and check on each message transiting their MTAs.
> 

You would only deal with your own local side lists, not others and 
that will scale just fine.  The only thing is that it may be two 
separate applications (SMTP and MLS) so interfacing them would be an 
implementation question.  Generally, an accept or forwarding file is used.

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 May 2011 15:56:22 -0400

> the subscriber back to the web page.  Where's the MTA going to get the 

See below

> announcements, why not just whitelist their mail?

Does this include blacklisting the not credible?

> directed replies to the ASRG.  See you there.

Lets see if my MTA gets the non-subscriber info/notification with a 
target that includes the address you directed replies to.

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 May 2011 16:22:47 -0400

> as far as catching badness with very little impact on legitimate mail.

What sort of phishing are we talking about?  Identities or the context?

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 May 2011 16:31:27 -0400

> target that includes the address you directed replies to.

Here we go, the MTA automation is possible.

          -------- Reject Message: Cut here --------
Subject: Re: [ietf-dkim] MLMs and signatures again
Date: Fri, 27 May 2011 12:56:41 -0700
From: [email-masked]
To: [email-masked]

You are not allowed to post to this mailing list, and your message has
been automatically rejected.  If you think that your messages are
being rejected in error, contact the mailing list owner at
[email-masked].

---

From: Hector Santos <[email-masked]>
Date: Fri, 27 May 2011 17:09:31 -0400

> What sort of phishing are we talking about?  Identities or the context?

This is what I see in today's log or malicious spoofing and phishing 
of our three main domains (all rejected).

From: Rolex.com <[email-masked]>
From: [email-masked]
From: [email-masked]
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: [email-masked]
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: [email-masked]
From: Rolex.com <[email-masked]>
From: [email-masked]
From: [email-masked]
From: [email-masked]
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: [email-masked]
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: [email-masked]
From: [email-masked]
From: [email-masked]
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>
From: Rolex.com <[email-masked]>

Note the common sender using "rolex.com" user id part and I noticed 
the ones that don't have this, all of them where also from the 
rolex.com spammer.  So this just boils down to one spammer today doing 
this.

None of them were DKIM signed, but they would of been rejected as 
non-signed if the logic was enabled to reject on a failed ADSP.

---

From: "John R. Levine" <[email-masked]>
Date: 31 May 2011 11:06:51 -0400

> I didn't posit this as a problem. Others did. I jumped in at the point that you said s/mime was already a solution, with a message that proved otherwise.

It would be better to say that if there were a problem, and people wanted 
to solve it, the pieces are all there with S/MIME.  MUAs all know how to 
add S/MIME sigs to outgoing mail.  Mailman, which must be the most popular
MLM around, wraps the message body so that it preserves the signature. 
And some MUAs even look inside the wrapper to recognize and verify the 
wrapped signatures.

But a lot of MUAs don't, the ones that do make little effort to separate 
the signed from the unsigned text, and as far as I can tell, until I 
started poking at it last month, nobody knew or cared.

So, again, can someone tell me why this is so urgent a problem that we 
need to add special code to every DKIM signer and verifier in the world to 
solve it?

Regards,

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 31 May 2011 14:25:01 +0000

On 26 May 2011, at 14:40, John R. Levine wrote:

> Hmmn.  How does it like the copy of this message sent to you directly? The signature was definitely good on the way out.

That looks fine.

> Actually, I think the long term solution is for people to stop pretending there is a problem.  Can you describe the operational problems you're experiencing here?  "Broken signatures" is a fact, not a problem. Mailing lists have worked quite well for 40 years with no signatures at all, making all sorts of random changes to the mail, so it has to be something more than that.

I also said that - if there's a problem with packages arriving broken - then the solution lies in making the paths smoother rather than increasingly fancy suspension. 

> Also, if you're suggesting changes to list software, please explain why they would have greater benefits than the obvious and simple one to have lists add their own signature on the way out.

Well, I guess that there must be use cases where one would want to verify that the sender's message was unaltered. I accept that they're less common than with non-list mail, though.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 31 May 2011 14:31:34 +0000

> Also, if you're suggesting changes to list software, please explain why they would have greater benefits than the obvious and simple one to have lists add their own signature on the way out.

Oh, and actually, I'm not proposing changes to list software. The software that I use for lists is quite capable of passing unaltered bodies and subject lines. However, list operators don't like to do that because they have no other way of adding meta-data in a way that's visible to users.

What I'm suggesting is that MUAs should be modified to expose List-* headers in a way that's useful to end users. At that point, most of the motivation for mucking around with messages would disappear. Also, done well, the utility of the meta-data could be improved. Increased safe passage of DKIM signatures might be regarded as a positive benefit with respect to motivating DKIM deployment.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 31 May 2011 15:51:15 +0000

> That's relying on an awful lot of vaporware in the MUA, orthogonal to any sort of authentication. I don't think any MUAs really track sender reputation in any way[1].

Certainly Outlook with Exchange does. If you mark a message as spam, then you'll find future messages from the same sender will likely end up being delivered to your spam mailbox.