Subject: [ietf-dkim] Possible problem with "simple" body canonicalization -- trailing CRLFs

From: Eric Allman <[email-masked]>
Date: Thu, 13 Jul 2006 09:55:59 -0400

In doing an end-to-end read on -base this morning I came up with a 
possible difficulty with simple body canonicalization.

The description says that simple reduces CRLF 0*CRLF at the end of a 
message to a single CRLF.  But what if the message has no trailing 
CRLF at all?

Remember that in the sequence CRLF . CRLF, both CRLFs are part of the 
terminator, not part of the message.  Thus, a message reading:

        thanks <CRLF>
        . <CRLF>

in fact has no trailing CRLF on the message.  (This could also happen 
with BDAT, but that's not widely implemented.)

An approach to this might be to say that simple reduces 0*CRLF to a 
single CRLF (which is quite possibly what the current implementations 
actually do).

eric

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Tony Hansen <[email-masked]>
Date: Thu, 13 Jul 2006 10:12:29 -0400

This not a problem when using DATA. Check 2821 section 4.1.1.4; the
ending crlf.crlf was clarified as being the trailing crlf of the last
line of the message followed by the terminator sequence.

   Note that the first <CRLF> of this terminating sequence is also the
   <CRLF> that ends the final line of the data (message text) or, if
   there was no data, ends the DATA command itself.

You are correct that the problem exists when using BDAT.

My implementation uses the last CRLF in if it's there. If there is no
last CRLF, it does *not* add one.

	Tony Hansen
	[email-masked]

Eric Allman wrote:
> message to a single CRLF.  But what if the message has no trailing CRLF
> at all?
>

---

From: Eric Allman <[email-masked]>
Date: Thu, 13 Jul 2006 10:50:57 -0400

Great!  Thanks for the pointer.  So it's not a problem with DATA.

Do we want to worry about the BDAT case?  My suggestion would work 
for that (but break your implementation).

It also occurs to me that there is an assumption in DKIM (and 
probably a lot of other specs) that the mail body is text.  It 
doesn't have to be if you use BODY=BINARYMIME [RFC3030].  Is this 
worth considering?  It seems like it would either require redefining 
simple body canonicalization if you have a BINARYMIME message, or 
defining a new canonicalization ("binary"?) that allows absolutely no 
changes.  Of course, a new canonicalization can be added later.

eric

--On July 13, 2006 10:12:29 AM -0400 Tony Hansen <[email-masked]> wrote:

> ending crlf.crlf was clarified as being the trailing crlf of the
> last line of the message followed by the terminator sequence.
>
>    Note that the first <CRLF> of this terminating sequence is also
> the    <CRLF> that ends the final line of the data (message text)
> or, if    there was no data, ends the DATA command itself.
>
> My implementation uses the last CRLF in if it's there. If there is
> no last CRLF, it does *not* add one.
>
> 	Tony Hansen
> 	[email-masked]
>
> Eric Allman wrote:
>> The description says that simple reduces CRLF 0*CRLF at the end of
>> a message to a single CRLF.  But what if the message has no
>> trailing CRLF at all?
>>
>> Remember that in the sequence CRLF . CRLF, both CRLFs are part of
>> the terminator, not part of the message.  Thus, a message reading:
>>
>>        thanks <CRLF>
>>        . <CRLF>
>>
>> in fact has no trailing CRLF on the message.  (This could also
>> happen with BDAT, but that's not widely implemented.)
>>
>> single CRLF (which is quite possibly what the current
>> implementations actually do).
>

---

From: Mark Delany <[email-masked]>
Date: Thu, 13 Jul 2006 14:52:04 +0000

> message to a single CRLF.  But what if the message has no trailing 

[ reduce ]

That's an interesting catch. FWIW, the original intent was "remove and
append" rather than "reduce". That seems to be a drift from the
original "simple" definition - was that conscious? The original goal
being that every line fed into the verification always has a
terminating CRLF.

Mark.

---

From: "Hector Santos" <[email-masked]>
Date: Thu, 13 Jul 2006 13:42:12 -0400

----- Original Message -----
From: "Eric Allman" <[email-masked]>

> changes.  Of course, a new canonicalization can be added later.

This canonicalization concept seems to be a big PITA.  We need a consistent
and working concept out of the gate.   Is it not possible to get a single
straight forward method now?

Sure, new methods can be added in the future, but I don't have to tell you
about legacy and version control issues. Just consider that we already have
few active DKIM participants who are so dependent on alpha-ware designs and
are highly resistance to final design changes. :-)

I just seem to think that if we can't resolve this now, I doubt it will be
resolved later.

--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

---

From: Tony Hansen <[email-masked]>
Date: Thu, 13 Jul 2006 13:44:45 -0400

I'm sort of thinking out loud here.

One of the driving forces behind c=simple was to basically sign what was
there in its pristine form. In its simplest form, c=simple should have
just canonicalized the body exactly as it existed in the message.

Now there are a number of things that can occur to messages as they pass
through MSAs, MTAs, gateways, MDAs, mailing lists, etc. We essentially
prioritized those types of mungings and decided that we needed to
protect c=simple from at least one case. And that is, a fairly common
occurrence is that a message will obtain an extra blank line. (Perhaps
by MTAs incorrectly interpreting CRLF.CRLF? :-) ) So c=simple was
changed to handle this case by removing extra blank lines.

I can see us going two ways with this:
    *	Declare that if there no extra lines or even a line ending,
	treat the input as complete.
    *	Declare that if there is no ending CRLF, we should guarantee
	that there is one.

Note that the first is consistent with the  original intent of simple
(and matches what my code currently does). However, the second is
consistent with handling extra lines potentially being added to the
message somewhere during transition. Since we can't tell *if* an extra
line is *going* to be added to a message during transition, and we need
to make certain that the canonicalization on the signing side and the
verifying side see the same input no matter what happened in between, we
should go with the second method.

Eric, you convinced me. (And I'll change my code accordingly. :-) )

Now, as far as binary messages are concerned, we already have statements
saying that a message SHOULD be rendered in a 7bit-compatible MIME
format, which means one of 7bit, quoted-printable or base64. All of
these require textual line-based messages.

	Tony Hansen
	[email-masked]

Eric Allman wrote:
> Do we want to worry about the BDAT case?  My suggestion would work for
> that (but break your implementation).
> 
> It also occurs to me that there is an assumption in DKIM (and probably a
> lot of other specs) that the mail body is text.  It doesn't have to be
> if you use BODY=BINARYMIME [RFC3030].  Is this worth considering?  It
> seems like it would either require redefining simple body
> canonicalization if you have a BINARYMIME message, or defining a new
> canonicalization ("binary"?) that allows absolutely no changes.  Of
> course, a new canonicalization can be added later.
> 
> eric
> 
> 
>

---

From: Michael Thomas <[email-masked]>
Date: Wed, 19 Jul 2006 07:30:50 -0700

>last CRLF, it does *not* add one.
Having just worked through this myself, I'll say that Tony's interpretation
*has* to be the correct one otherwise an l= less than the body length won't
be correct. Namely:

"a b</body></html>" where l=3 is: "a b" not "a b<crlf>" or any other
weird interpretation. This is definitely worth a mention in the draft.

       Mike

---

From: Tony Hansen <[email-masked]>
Date: Wed, 19 Jul 2006 11:29:13 -0400

Adding l= into the mix makes no change to any of the arguments, as the
l= length is done off of the canonicalized form of the body. That is,
you conceptually canonicalize first, and then use the l= bytes from that
canonicalized form.

That is, for a body of "a b</body></html>", if this were canonicalized
to either "a b</body></html>" or "a b</body></html>CRLF", applying l=3
to that still reduces the body to "a b".

After posting the message below, I was convinced by the other arguments
that the *canonicalized* form of the message should always end with a
final CRLF. If you buy the argument that there are MTAs out there that
will potentially add extraneous CRLFs to a message, then the verifier
cannot differentiate between a message that has a CRLF on its last line
and one that didn't start that way and had one added by an MTA. So the
verifier *must* assume that that last CRLF is a valid part of the
message. Subsequently, this means that the signer must guarantee that
the canonicalized form of the body ends in a CRLF.

And I've changed my code accordingly to guarantee that the last line of
the body *does* have a CRLF after canonicalization.

	Tony Hansen
	[email-masked]

---

From: Mark Delany <[email-masked]>
Date: Wed, 19 Jul 2006 15:42:11 +0000

> weird interpretation. This is definitely worth a mention in the draft.

That would only be true if the signer forgets to add CRLF prior to
feeding into the signing code. In that case you would get an l=5, not
an l=3.

Mark.

---

From: Michael Thomas <[email-masked]>
Date: Wed, 19 Jul 2006 08:59:24 -0700

>the body *does* have a CRLF after canonicalization.
I must be missing something. The 821 input string of:

a b</body></html>CRLF
.
CRLF

already contains the final CRLF of the body, so there wouldn't be a 
situation
where you'd ever see an input string of the form: a b</body></html> (no 
crlf),
right? Or are you saying that's the case for DATA, but maybe not BDAT?

       Mike

---

From: Tony Hansen <[email-masked]>
Date: Wed, 19 Jul 2006 12:12:42 -0400

Exactly. If you're using something other than SMTP DATA, such as SMTP
BDAT, it is conceivable that the body could wind up without an ending
CRLF. Of course, the likelihood of both that *AND* the MTA adding an
extra CRLF during transmission does seem to me to be rather low.

So, do we worry about this potential case or not?

	Tony

Michael Thomas wrote:
> situation where you'd ever see an input string of the form: a
> b</body></html> (no crlf), right? Or are you saying that's the case
> for DATA, but maybe not BDAT?

---

From: Michael Thomas <[email-masked]>
Date: Wed, 19 Jul 2006 10:42:49 -0700

>So, do we worry about this potential case or not?

So not knowing anything about BDAT, isn't an MTA that added CRLF's to
binary data that it claims to understand (via ESMTP)  be somewhat... broken?

       Mike

---

From: John Levine <[email-masked]>
Date: 19 Jul 2006 20:30:57 -0000

>CRLF....

BDAT is part of the CHUNKING extension defined in experimental RFC
1830 in 1995.  It's been 11 years.  Does anyone use it?  Has anyone
even proposed making it standards track?  I don't ever recall running
into it, so I don't see any point in worrying about it.

We may well end up using the DKIM key infrastructure for signatures of
other things, but it's going to take a lot more than deciding which
line endings to ignore to make that happen.

R's,
John

PS: If anyone ever updates that RFC, it's now called Chongqing.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: [email-masked]
Date: Wed, 19 Jul 2006 14:20:30 -0700 (PDT)

> 1830 in 1995.

Your information is out of date. RFC 1830 was superceded by RFC 3030 in
December 2000. RFC 3030 is a proposed standard and I believe there has been
talk about advancing it to draft. I'm pretty sure there are enough
implementations.

> It's been 11 years.  Does anyone use it?

There's some use, but there are also incompliant implementations out there that
make unrestricted use of BDAT somewhat problematic. I currently have it enabled
on my home system after having added some workarounds for some compliance
issues, but a couple of folks are reporting having trouble sending me mail now
that i have it on. The problem appears to be BDAT-related so I'm probably going
to have to disable it. Given the limited traffic on my system this is not a
good sign.

> Has anyone even proposed making it standards track?

The question is, of course, moot, but FWIW the move to standards track was done
at the behest of the VPIM WG. I believe the LEMONADE profile calls for support
for both CHUNKING and BINARYMIME. (Although to be fair, I have yet to see a
single tangible result arise from any email service profile document other than
RFC 1123.)

> I don't ever recall running into it,

CHUNKING is definitely deployed to some extent. However, there are actually two
extensions: CHUNKING and BINARYMIME, and from what I've seen there's a lot less
support out there for BINARYMIME than for CHUNKING. Without BINARYMIME all you
have is a somewhat more efficient data transfer mechanism - you're still
dealing with 7bit or 8bit material. Stuff like the final line not being
terminated with CRLF is only supposed to be possible when BINARYMIME is used.

> so I don't see any point in worrying about it.

Well, one thing I do worry about is the assumption some folks keep making that
the transport infrastructure doesn't, or isn't supposed to, mess around with 
message content. Like it or not, operations like encoding downgrading (or
upgrading) are an explicit part of the email architecture and have been ever
since MIME came out. Getting this particular genie back in the bottle is
effectively impossible at this point, and the issues it may cause if we don't
get the applicability of DKIM right will make problems relating to bare CRs and
LFs pale in comparison.

This is why I keep pushing back on client use of DKIM. I think DKIM can be made
to work across boundaries between administrative domains, but any more than
that is asking for trouble.

				Ned

---

From: John L <[email-masked]>
Date: Wed, 19 Jul 2006 21:23:02 -0400 (EDT)

> December 2000.

Oops.

> since MIME came out. ...

I think we're in violent agreement here.  Every time a message passes 
through a relay, the relay modifies it somehow.  Ideally, the message 
would be DKIM signed by the sending domain's outgoing mail relay and the 
signature checked by the recipient's MX.  But since that's not always 
going to be possible, a design goal of DK and I hope DKIM is for 
signatures to survive typical relay behavior insofar as is practical.

To me this means that for stuff that is common, straightforward, and well 
understood, e.g., adding new headers, reordering some existing headers, 
and adding blank lines at the end, we have provisions to deal with that. 
For anything more complicated, forget it.  It's clear to me that for 
messages that don't end with CR LF (which is impossible in normal SMTP 
anyway) or that have bare CR or LF, the behavior of relay MTAs is varied 
and hard to predict, so the only useful advice we can give to people who 
want their signatures to work is Don't Do That.

If you want your signatures to work, be sure the message you're signing is 
as squeaky clean 2822 compliant as possible so as to give relay MTAs as 
little incentive as possible to make helpful modifications.  I realize 
that we have existing software and we can't always upgrade it, but if we 
want something that's designed to be resilient in the face of every known 
hostile MTA, we already have S/MIME and this is not it.  The existing 
simple canonicalization covers a large and useful set of relay MTA 
behavior, so I think we should declare victory and stop.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://johnlevine.com, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Jul 2006 01:04:12 -0400

> behavior, so I think we should declare victory and stop.

I have a general DKIM requirements question...

I have seen a combination of references to 822 and 2822 in recent discussions 
on the list.  Is the requirement that DKIM support both 822/2822 content (822 
being the current standard) or is the intent that DKIM is just required to 
support 2822 content?

I don't recall much discussion on this point.

Scott K

---

From: Barry Leiba <[email-masked]>
Date: Thu, 20 Jul 2006 10:58:16 -0400

> being the current standard) or is the intent that DKIM is just required to 

I believe there are two parts to the answer to that:

1. We refer to RFC 282x, as the current standard, and that's what we're 
aiming to support.

2. We're trying, to the extent we reasonably can, to deal with most of 
what's actually out there, which includes RFC-82x-compliant stuff as 
well as some small subset of odd or "broken" behaviour that we consider 
to be common enough to be worth working with.

Does anyone think that's not the right answer?

Barry

--
Barry Leiba, DKIM working group chair  ([email-masked])

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Jul 2006 08:51:09 -0700

Barry Leiba wrote:
>>      Is the requirement that DKIM support both
>> 822/2822 content (822 being the current standard) or is the intent
>> that DKIM is just required to support 2822 content?
> what's actually out there, ...
> 

I think your language describes things quite nicely.

I am pretty sure that DKIM does not have anything that cares about 822 vs. 2822.
 That is, it works for both.

So I have tended to view the dual-reference approach as a means of communicating
to folks that they do not have to worry about old-vs-new specifications for
message syntax/semantics.

d/

-- 

  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Eliot Lear <[email-masked]>
Date: Thu, 20 Jul 2006 18:45:18 +0200

> message syntax/semantics.

I agree but perhaps we need to be a bit more explicit about exactly what
you say above, Dave.

Eliot

---

From: Douglas Otis <[email-masked]>
Date: Thu, 20 Jul 2006 08:40:32 -0700

On Jul 20, 2006, at 7:58 AM, Barry Leiba wrote:

>> I have seen a combination of references to 822 and 2822 in recent  
>> discussions on the list.  Is the requirement that DKIM support  
>> both 822/2822 content (822 being the current standard) or is the  
>> intent that DKIM is just required to support 2822 content?
>
> 1. We refer to RFC 282x, as the current standard, and that's what  
> we're aiming to support.
>
> 2. We're trying, to the extent we reasonably can, to deal with most  
> of what's actually out there, which includes RFC-82x-compliant  
> stuff as well as some small subset of odd or "broken" behaviour  
> that we consider to be common enough to be worth working with.
>

RFC822 does not offer assurances with respect to header ordering,  
however there is not much caution included in this regard.

This note however which seems to suggest that RFC822 is viewed as  
anti-social:

INFORMATIVE ADMONITION:  Despite the fact that [RFC2822] permits
header fields to be reordered (with the exception of Received
header fields), reordering of signed header fields with multiple
instances by intermediate MTAs will cause DKIM signatures to be
broken; such anti-social behavior should be avoided.

-Doug

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 20 Jul 2006 14:07:57 -0400

On Thursday 20 July 2006 11:51, Dave Crocker wrote:
> I am pretty sure that DKIM does not have anything that cares about 822 vs.
> 2822. That is, it works for both.
>
> So I have tended to view the dual-reference approach as a means of
> communicating to folks that they do not have to worry about old-vs-new
> specifications for message syntax/semantics.
>
> d/

OK.  I may have mis-remembered, but I thought that one aspect of the naked CR 
discussion (which a spun this thread off of) was that a naked CR is allowed 
by 822, but not 2822.  So I think there is something that cares.

Also, I think what you are saying is different than what Barry is saying.  To 
paraphrase:

Barry - Design requirement is to support 2822, but we will try to deal with 
what is out there are much as we reasonably can (including 822).

Dave - Design requirement is to support 2822 and there aren't any 822/2822 
differences that matter, so by supporting 2822, we also support 822.

I think we need to have clarity on this point and it doesn't seem to me that 
we have it at this time.

Scott K

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Jul 2006 11:56:25 -0700

> I think we need to have clarity on this point and it doesn't seem to me that 

Indeed.

First, I apologize.  I was being entirely too cavalier in considering this
issue.  Second, I've gone back over the bare-CR discussion and I now think there
is indeed a difference between 822 and 2822 that is quite important for DKIM,
and that we need to deal with it explicitly.

Here's what I take from the discussion:

  1)  RFC 2822 generates text that is less subject to in-transit distortion than
RFC 822, by virtue of it's greater strictness about CR and LF.

  2)  There is no way to create a canonicalization algorithm that is completely
robust against in-transit distortions, and defining one that attempts to be
highly robust is likely to have the dual problems of excessive complexity and
inadequate robustness.  Complexity leads to errors and lack of robustness means
that the goal being used to justify the complexity is not attained.  Hence,
simpler algorithms are strongly preferred.

  3)  Adopting DKIM takes software modification, so we might as well include the
requirement that the message that is being signed be in a form that is less
subject to distortion.  To the extent that the message needs to be transformed,
that is the job of the 'local' system, prior to signing.

In sum:

     We should require that DKIM signers only sign messages that are RFC 2822
conformant.  (We will need a small amount of text that explains why.)

Comments?

d/
-- 

  Dave Crocker

---

From: John Levine <[email-masked]>
Date: 20 Jul 2006 21:46:20 -0000

>     We should require that DKIM signers only sign messages that are
>RFC 2822 conformant.  (We will need a small amount of text that
>explains why.)

I'd be inclined to say that messages to be signed SHOULD conform to
2822.  There are some 822 MTAs that produce nice clean messages that
pass through relays with little trouble, such as the one I'm using.

The explanatory text is indeed important to make it clear we're not just
being religious purists.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Jul 2006 16:35:42 -0700

> pass through relays with little trouble, such as the one I'm using.

Fair point.  Offhand I cannot think of a downside, by using SHOULD rather than
MUST.

The SHOULD asserts the importance quite clearly, and it is not as if doing 822
rather than 2822 can cause damage anywhere but to the verification of the
particular message.

d/
-- 

  Dave Crocker

---

From: Michael Thomas <[email-masked]>
Date: Thu, 20 Jul 2006 17:47:54 -0700

>rather than 2822 can cause damage anywhere but to the verification of the
I still have lingering worries about this naked CR problem. Since a lot of
it is due to windows file system thing considerations, you get a lot of 
sequences
like:

line^M^M^J

which will get transformed into:

line^M^J
^M^J

Will that transformation cause things to break? One thing for certain: 
consumers
of these messages are all of the sudden going to be seeing double spaced 
lines;
maybe just an annoyance, maybe worse. Who knows?  Likewise if you instead
deleted naked CR's. Will it cause problems? Who knows?

The thing is that this was perfectly legitimate 822 even if it wasn't 
entirely
anticiapated back then. And there's a lot of legacy out there that stopped
evolving in the 90's or maybe even earlier. Taking too cavalier an attitude
that 2822 is the answer seems to run into the same problems that Ned wrote
about: the email world is a wacky place, and we shouldn't be surprised
about being surprised. So not being accommodating of the pre-2822 world
seems like it has a pretty large risk of getting it wrong regardless of 
whether
we're "right". That bothers me.

       Mike

---

From: Mark Delany <[email-masked]>
Date: Fri, 21 Jul 2006 04:01:59 +0000

> Um, there's user experience, which last I heard sort of trumps everything.
> Can you say for absolute certain that the user experience of converting
> compliant 822 messages into compliant 2822 messages won't cause trouble?
> I sure can't. That at the very least ought to evoke some humility.

Sure. But a bucketload of such content is HTML now - of that I can
attest - all of which is entirely oblivious to stray whitespace.

Mark.

---

From: Michael Thomas <[email-masked]>
Date: Thu, 20 Jul 2006 20:20:42 -0700

Mark Delany wrote:

>>>Maybe, but if they're going through the usual MTAs, they're breaking 
>>>already so I can't get too worried about it. 
>>>      
>>>
>>That's not true. Sendmail does not change these strings.
>>    
>>
>
>Is that true on the SUBMIT front? or the MTA front?
>  
>

As far as I know, I run a submit server and it doesn't change anything.
As I said, I'm very interested in Eric's input here.

>>We really have 
>>absolutely
>>no clue as to whether this is going to be a problem.
>>    
>>
>
>I guess I'm a little confused about the problem here. For non-signed
>mail we don't care, yes?
>
>For signed mail, the spec is asking signers to ensure compliance with
>2822. If, as you say, sendmail passes through whatever bits it gets,
>then as long as signers ensure compliant messages, what problem are we
>trying to solve?
>  
>

>Put another way, my question is this: what evidence is there to
>suggest that stray characters are likely to be introduced *after*
>signing of an rfc2822 compliant mail?
>  
>
That's not the issue. The introduction is at submission. By the 
submitter. If changing
things after submission causes problems, then... As I've said many 
times, we don't
have the luxury of aspiration, we have the duty of dealing with the 
world as it is lest
we design something that is beside the point.

       Mike

---

From: Eliot Lear <[email-masked]>
Date: Fri, 21 Jul 2006 08:35:03 +0200

> Is that true on the SUBMIT front? or the MTA front?
Certainly NOT the MTA, and *even* if it does change on the SUBMIT front
who cares?  That's before signing, right?

Eliot

---

From: Dave Crocker <[email-masked]>
Date: Thu, 20 Jul 2006 23:39:27 -0700

> who cares?  That's before signing, right?

Not necessarily.

d/
-- 

  Dave Crocker

---

From: Michael Thomas <[email-masked]>
Date: Fri, 21 Jul 2006 06:08:54 -0700

>attest - all of which is entirely oblivious to stray whitespace.
The cases that I'm talking about most assuredly are not HTML. Again, it's
the user experience I worry about. If all of the sudden that changes for the
worse because you decided to turn on DKIM signing in your existing
deployment, who is going to get the blame? Who is going to have the onus
to deal with making the user experience right again? It's easy to say 
that the
applications shouldn't be sending non-2822 compliant mail, but I can say for
certain that in our environment that I haven't even a clue what those
applications are, let alone who owns them, let alone the ability to change
them. That's sort of an untenable position we're putting people in from a
deployment standpoint.

And since the subject is "does DKIM support 822 or only 2822", I'd say 
at this
point we'd have to say "only 2822".

       Mike

---

From: John Levine <[email-masked]>
Date: 21 Jul 2006 15:17:44 -0000

I've been perusing the spec, and was reminded that we still have the
simple body canon.  All it does is strip off trailing CR LF pairs, and
hash what's left.  Bare CRs?  Eight bit glop?  No problem.

If we do an implementor's guide, I'd have it say that if your goal is
simply to shoot a message in whatever form from one MTA to another,
use simple canon and it'll work.  On the other hand, if your goal is
to make your messages and signatures maximally tolerant of unknown relays
down the line, be sure it's perfectly 2822 conformant and use a relaxed
signature.

Those are both reasonable scenarios, we deal with both of them, so I don't
see any problems left to be solved.

R's,
John

PS: Sec 3.4.3 says to add a trailing CR LF if none is present, although
a message transmitted by SMTP has to have one.  But I don't think that's
worth fixing.

---

From: Michael Thomas <[email-masked]>
Date: Fri, 21 Jul 2006 08:25:08 -0700

>down the line, be sure it's perfectly 2822 conformant and use a relaxed
So we've been running relaxed body canonicalization on one of our production
servers, and I can't say that I've seen any evidence that it is any 
better than
simple.

       Mike

---

From: Michael Thomas <[email-masked]>
Date: Fri, 21 Jul 2006 08:52:04 -0700

John L wrote:

>> better than simple.
> So I guess the smashed CR's aren't a problem, are they?

Huh? Relaxed body canonicalization doesn't do anything different with
CR's than simple.

       Mike

---

From: John L <[email-masked]>
Date: Fri, 21 Jul 2006 11:50:12 -0400 (EDT)

> servers, and I can't say that I've seen any evidence that it is any better 
> than simple.

Regards,

---

From: Michael Thomas <[email-masked]>
Date: Fri, 21 Jul 2006 09:01:06 -0700

Tony Hansen wrote:

>What kinds of failures have you been seeing? Any thoughts on making
>relaxed better?
>  
>
The failures we've been seeing are all related to the naked CR problem. 
If we
made relaxed also canonicalize CR or LF as CRLF, it would certainly help,
and not have the uncertainty of whether forcing the actual body to be 2822
compliant will be harmful.

Other than that, it seems to be mainly mailing list related which I 
doubt could
ever be "fixed". The only other thing I've seen experimenting around is that
mailers will force a line break. I'm fairly certain that this is much more
a theoretical issue than a real one though.

       Mike

>>So we've been running relaxed body canonicalization on one of our 
>>production servers, and I can't say that I've seen any evidence that
>>it is any better than simple.
>>    
>>

---

From: "Hector Santos" <[email-masked]>
Date: Fri, 21 Jul 2006 12:35:58 -0400

----- Original Message -----
From: "Michael Thomas" <[email-masked]>
To: "Tony Hansen" <[email-masked]>

> If we made relaxed also canonicalize CR or LF as CRLF, it would certainly
> help, and not have the uncertainty of whether forcing the actual body to
> be 2822 compliant will be harmful.

It shouldn't be an issue forcing 2822 with original feed signings which are
those with no HOPs (No Received: lines, or 1 if its your SMTP receiver).

The only issue I can see is a MUA with an embedded or 3rd party DKIM hook.
But with a DKIM required for RFC 2822 minimum support, that shouldn't cause
an issue.  In addition, the MSA may not have to sign again with such
DKIM-aware MUAs.

---

From: Dave Crocker <[email-masked]>
Date: Sat, 22 Jul 2006 22:54:00 -0700

Eliot Lear wrote:
> Dave,
> What about section 8 of RFC-4409 and all those fun little changes that
> can be made?  We're even covered in there in 8.5.

The "not necessarily" referred to the claim that SUBMIT is automatically before
signing.  Signing may be done by any component in the AdMD, including the MUA.

d/

-- 

  Dave Crocker

---

From: [email-masked]
Date: Sun, 23 Jul 2006 07:44:39 -0700 (PDT)

> signing.  Signing may be done by any component in the AdMD, including the MUA.

But if it is done by the MUA, the problem is that fully comformant SUBMIT
agents or various sorts of other conforming intermediaries can and often will
wreck the signature. And this says nothing about non-conforming intermediaries,
which aren't exactly unheard of.

If you want another example, how about RFC 4141? You're a coauthor of that one.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Sun, 23 Jul 2006 08:08:30 -0700

Ned, et al,

[email-masked] wrote:
> wreck the signature. 

Yup.

But, then, I view that as merely an exemplar of the question of surviving
transit through intermediaries.  That the SUBMIT "intermediary" typically makes
significant changes merely makes this scenario more interesting to explore.

> And this says nothing about non-conforming intermediaries,

They are the core issue in deciding canonicalization trade-offs and I am
certainly not making light of the concerns.  However I do not believe I've
suggested anything that lessens or increases them as an issue for DKIM utility.

Again, MUA-based signing provides a more challenging example of the issue, but
this is a difference in degree, not in kind, IMO.

> If you want another example, how about RFC 4141? You're a coauthor of that one.

I think that content conversion *must* be viewed as a gateway function, and that
surviving gateways is already a challenge for DKIM.

Since you raise this example and I think it is interesting, let's explore it a
bit:

     The only way that I can think of surviving is to have the signature
evaluation be done before the conversion.  This means that conversion must take
place within the trusted AdMD in which the result of signature validation gets
used.  This can occur after content conversion, or the like, but must be within
the same trust boundary, so that the mechanism for communicating the validation
result -- such as with an Auth-Results header field -- can itself be trusted.

Thoughts?

d/

-- 

  Dave Crocker

---

From: John Levine <[email-masked]>
Date: 23 Jul 2006 16:51:01 -0000

There's a certain point at which the answer to questions has to be
"don't do that".

>> But if it is done by the MUA, the problem is that fully comformant
>> SUBMIT agents or various sorts of other conforming intermediaries
>> can and often will wreck the signature.

Seems to me that if the MUA does all the stuff in section 8 of RFC
4409 before it signs the message, the chances are pretty good that the
signature won't break.  Yes, in theory the SUBMIT agent might still
rewrite the headers and reformat the body, but my admittedly limited
experience suggests that for the most part they will leave valid parts
of a message alone.  So if you want to sign in the MUA, be sure your
MUA creates valid messages, or you use a shim that runs on the client
machine that cleans up and signs.

>I think that content conversion *must* be viewed as a gateway
>function, and that surviving gateways is already a challenge for
>DKIM.

Right. I think we all agree that we want DKIM signatures to survive
what one might call routine relay mutation.  Beyond that, the relay
needs to re-sign.

R's,
John

---

From: Eliot Lear <[email-masked]>
Date: Sun, 23 Jul 2006 19:20:02 +0200

> If you want another example, how about RFC 4141? You're a coauthor of that one.

Even fully compliant 2822 messages going in can have content tagged
going out.

Eliot

---

From: [email-masked]
Date: Sun, 23 Jul 2006 11:53:49 -0700 (PDT)

> significant changes merely makes this scenario more interesting to explore.

And dangerous to condone.

> suggested anything that lessens or increases them as an issue for DKIM

You specifically raised the point that DKIM could be applied prior to
submission. To the extent we suggest that this and other "internal" uses of
DKIM are feasible, my fear is that we raise expectations to an unreasonable
degree. What happens when users attempt to use DKIM this way, only to find that
the verification fails a significant fraction of the time?

> surviving gateways is already a challenge for DKIM.

True content conversion, sure. But the problem is there's an essentially
continuous range of actions from "convert this audio clip from stereo to mono"
or  to "add a disclaimer" to "convert from 8bit to quoted-printable" to "fix up
these headers". It is hard to know where to draw the line.

> Thoughts?

My view is that DKIM is designed to provide a boundary service between
administrative domains. (I suppose we could up with a different term than
administative domain here, but since the two will align more often than not I
prefer to stick to existing terminology.)

A given administrative domain may elect to provide some measure
of "internal" DKIM service (or they may not), but this isn't required
for DKIM to achieve its primary goals and may lead to expectation mismatches.

To the extent an administative domain elects to employ conversion services (and
far from becoming less common, we're seeing an upswing in the use of such
services), they need to position the use of DKIM to take such services into
account. This may mean that signatures need to be applied some distance from
where messages were injected into the infrastructure and verification may need
to occur some place prior to verification results actually being consumed.

In any case, I'm not suggesting that we forbid extended uses of DKIM, but I
think it is big mistake to actively promote them.

				Ned

---

From: <[email-masked]>
Date: Mon, 24 Jul 2006 09:07:25 -0400

On the face of this it looks like a third party is molesting the message
after signing but before delivery. If the third party does not currently
do DKIM then the signature will result in failure. If the third party is
DKIM aware then it could verify the signature, make needed changes then
re-sign the message and forward it to the recipient. This brings us back
to 3rd party signatures that break the original signature and into
policy issues.
Thanks,

Bill Oxley 
Messaging Engineer 
Cox Communications, Inc. 
Alpharetta GA 
404-847-6397 
[email-masked] 

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] On Behalf Of Dave Crocker
Sent: Sunday, July 23, 2006 11:09 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [ietf-dkim] 822/2822 or just 2822

Ned, et al,

> But if it is done by the MUA, the problem is that fully comformant
SUBMIT

Yup.

But, then, I view that as merely an exemplar of the question of
surviving
transit through intermediaries.  That the SUBMIT "intermediary"
typically makes
significant changes merely makes this scenario more interesting to
explore.

They are the core issue in deciding canonicalization trade-offs and I am
certainly not making light of the concerns.  However I do not believe
I've
suggested anything that lessens or increases them as an issue for DKIM
utility.

Again, MUA-based signing provides a more challenging example of the
issue, but
this is a difference in degree, not in kind, IMO.

> If you want another example, how about RFC 4141? You're a coauthor of
that one.

I think that content conversion *must* be viewed as a gateway function,
and that
surviving gateways is already a challenge for DKIM.

Since you raise this example and I think it is interesting, let's
explore it a
bit:

     The only way that I can think of surviving is to have the signature
evaluation be done before the conversion.  This means that conversion
must take
place within the trusted AdMD in which the result of signature
validation gets
used.  This can occur after content conversion, or the like, but must be
within
the same trust boundary, so that the mechanism for communicating the
validation
result -- such as with an Auth-Results header field -- can itself be
trusted.

Thoughts?

d/

-- 

  Dave Crocker

---

From: Douglas Otis <[email-masked]>
Date: Mon, 24 Jul 2006 07:35:59 -0700

On Sun, 2006-07-23 at 11:53 -0700, [email-masked] wrote:
> administrative domains. (I suppose we could up with a different term
> than administative domain here, but since the two will align more
> often than not I prefer to stick to existing terminology.)
> 
> To the extent an administative domain elects to employ conversion
> services (and far from becoming less common, we're seeing an upswing
> in the use of such services), they need to position the use of DKIM to
> take such services into account. This may mean that signatures need to
> be applied some distance from where messages were injected into the
> infrastructure and verification may need to occur some place prior to
> verification results actually being consumed.
> 
> In any case, I'm not suggesting that we forbid extended uses of DKIM,
> but I think it is big mistake to actively promote them.

Striving to allow the message to be verified at the MUA increases the
possible success of DKIM in offering the desired assurance.  While there
may be problems in some cases, many of these cases could be avoidable.
Signing at the MUA offers less value and will likely see a higher level
of failure.  There are many reasons to caution about signing at the MUA.

-Doug

---

From: Eliot Lear <[email-masked]>
Date: Mon, 24 Jul 2006 16:52:13 +0200

> of failure.  There are many reasons to caution about signing at the MUA.

I see nothing wrong with this, so long as caveats are explained
(anti-virus checks, etc), and more seriously when it comes to signing. 
One example would be a signature that says "this email brought to you by..."

Eliot

---

From: "Hector Santos" <[email-masked]>
Date: Mon, 24 Jul 2006 10:57:37 -0400

----- Original Message -----
From: <[email-masked]>
To: <[email-masked]>; <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, July 24, 2006 9:07 AM
Subject: RE: [ietf-dkim] 822/2822 or just 2822

> to 3rd party signatures that break the original signature and into

+1.

Question:

If it not possible to have a complete stripping of the CR/LF for hashing
purposes?  That would address this particular mixed bag EOL issue for both
the signer and verifier.

I believe from a security standpoint that this is not as secure as a SIMPLE.

So maybe we can two hashes:

  - a hash that is part of the signature using a STRIP canonicalization,
  - a hash of the original body,

The gives the verifier two new capabilities:

  - It will increase hash repeatability using the STRIP method, and
  - It provides a way to check for original mail integrity changes.

This can produce an interesting result with the DKIM signature calculated as
valid but with feedback that the originality of the message has change.

---

From: "Hector Santos" <[email-masked]>
Date: Mon, 24 Jul 2006 11:00:34 -0400

----- Original Message -----
From: "Douglas Otis" <[email-masked]>
To: <[email-masked]>

> of failure.  There are many reasons to caution about signing at the MUA.

If this is deemed to be desirable (no MUA signing), what mechanism would be
used to control it from happening?

A signed DKIM message with no HOP information?

Basically how would you distinguish an MUA vs. MTA?

---

From: [email-masked]
Date: Mon, 24 Jul 2006 08:03:37 -0700 (PDT)

> possible success of DKIM in offering the desired assurance.

You're assuming it will work well in this context. Everything I've learned
about large scale email deployments over the past 20 years says otherwise.

> While there

I'm sorry, but this is simply naive. Try telling, say. a bank that their policy
enforcement engine that is applied to all email can be "avoided".

> of failure.  There are many reasons to caution about signing at the MUA.

I agree that MUA signing is more problematic for a bunch of different reasons,
but this is only a question of degree. The same basic problems exist at both
ends.

				Ned

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 24 Jul 2006 16:13:03 +0100

Hi Hector,

Not that I follow this entire thread (anyone like to summarise?)
but just on this point:

> valid but with feedback that the originality of the message has change.

Given our current definition of "bh=" isn't this just the same
as inventing a new "eat all CR/LF" c14n alg. and including
two signatures - one with simple for the body and one with
the new c14n for the body?

I also think that the putative "eat all CR/LF" c14n might be
hard to get approved, given the obvious vulnerabilities it'd
create.

Stephen.

---

From: John Levine <[email-masked]>
Date: 24 Jul 2006 17:58:58 -0000

>hard to get approved, given the obvious vulnerabilities it'd

It's come up before, and did not get any traction.

In the spirit of rough consensus and running code, it seems to me that
people who want a new c14n scheme should implement it and at least try
it out on available message corpora and tell us what happened.

R's,
John

---

From: "Hector Santos" <[email-masked]>
Date: Mon, 24 Jul 2006 19:48:29 -0400

----- Original Message -----
From: "Stephen Farrell" <[email-masked]>
To: "Hector Santos" <[email-masked]>

>From my perspective, I can summarized it as a debate on the
essential question:

   Does the System fit into DKIM or does DKIM fit into the System?

> but just on this point:
>
>> This can produce an interesting result with the DKIM signature
>> calculated as valid but with feedback that the originality of
>> the message has change.
>

if bh= is based on the raw original content, then yes, it would be
the same thing.

> hard to get approved, given the obvious vulnerabilities it'd

I would like to see some documentation or references on this and its
conclusions.

Of course, if so, it is not addressable?

I mean, if STRIP provides a more robust and high reliability for validation
and can help DKIM from an implementation standpoint (DKIM fit into the
system), is there a way to protect against STRIP exploits thereby making the
method more acceptable?

The issue is very clear, hashing on original content and expecting this to
survive is on the low end of reliability spectrum. Nothing really matters in
DKIM if body hashing isn't deem reliable.

So "theoretically" hashing needs to be done with a canonicalization method
that is deemed to be robust and reliable in addressing on 822/2822 EOL
integrated infrastructure issues.

If and only if the DKIM signature is valid, the verifier can choose to check
the original content hash (hopefully based on the policy driven original
domain expectation for in-transient survivability or not).

John made a good suggestion for experimentation, that's always a good idea.
But it seems pretty obvious if the current hashing is problematic because of
unexpected cr/lf characters in the stream or lack there of, then removing
them from the DKIM hashing equation is a random simple understanding for a
solution.

But I would like to see some references on the vulnerabilities for his "eat
all cr/lf" CL4N method and then see if using a secondary original content
hash can help protect this scheme.

---

From: "Hector Santos" <[email-masked]>
Date: Mon, 24 Jul 2006 22:21:51 -0400

----- Original Message -----
From: "Stephen Farrell" <[email-masked]>
To: "Hector Santos" <[email-masked]>

> the new c14n for the body?

If I am wrong about this approach or there is serious security exploit with
this suggestion, please show how so I can nix the idea and move on. I don't
wish to waste time it isn't technically sound.

As a follow-up, I think if the bh= hashing is based on the original body,
then it would be the same idea of using STRIP or "Eat All CR/LF" concept.

We have put DKIM testing on the backburner since BASE-01, but I was
supportive of the bh= concept since its introduction.  The only that
scratched my mind what the separation idea and I think this issue now comes
to highlight the need to separate it.

I am reading the bh= specification now, and it seems to me that in order to
allow increase reliability, we might need an (optional) separate c= tag,
such as "bc=" to define a possible Body Canonicalization method.

I'm just winging it now, but here is something I see possible with backward
compatibility:

 B0 - body hash of original content

 BH - body hash based on STRIP method

      bc=STRIP required, with no alternative method, this serves as
      trigger for the verifier.  If bc=STRIP is not found, then
      we are back to BASE-04 logic, no need to calculate BO.

 B  - per spec, the hash of h= fields and constructed DKIM-SIG
      minus the BO tag

In short, if a domains wants to increase the survivability of the message
distribution, he would do the following:

SIGNER:

1) Add bc=STRIP tag for the DKIM-SIGNATURE.

2) Calculate the bh= body hash of STRIP message content.

3) Per spec, add bh=body-hash tag to the DKIM-SIGNATURE: header,
   and calculate the b= signature hash.

4) if BC=STRIP was enabled, calculate the BO= body
   hash of the raw "as is" content, and add bo=orig-body-hash
   tag to the DKIM-SIGNATURE: header

VERIFIER:

1) If the bc=STRIP tag is presence, this will trigger
   a body hash based on the STRIP method.

2) Per spec, verify the signature.

3) If valid, an local policy or domain policy drive option should be
   available to check the bo= tag.

I see as acceptable from my standpoint of having major concerns about the
open-end blasting of DKIM messages and its current high degree of
invalidity.

A) It gives us an important eliminating of 822/2822 CRLF related issues.
   This issue as it relates to CR/LF characters, is now resolved for all
   intent and purposes.

B) It increases the odds of successful DKIM validation that is
   currently plagued with body CRLF mutation issues.

C) And Finally, if the signature FAILS at this point, then it really doesn't
   matter what the original mail integrity issues are.  However, if the
   signature is valid, then we have an additional piece of information to
   determine of the mail content was altered.

This provides four result conditions:

   R1 - 100% validation (both signature and bo= valid)
   R2 - near 100% validation (signature valid, bo= failure)
   R3 - near 100% failure (signature invalid, bo= valid)
   R4 - 100% failure (both invalid signature and bo=)

If everything checks out successfully (R1), then this was a strong
validation with no in-transient mutations.

If the bo= is the only failure (R2), then does not necessary mean it was a
BAD message from a BAD ACTOR as long as the signature itself survived.  The
odds are good that this is condition due to transient mutations.

I would think that when the signature is invalid (R3 or R4), then this will
fall under the current specs considerations for invalid signatures.
However, a verifier may wish to check BO anyway to analyze if the header
hashing may have failed.

If there an exploit in any of this here, I don't see off hand, but the
verifier implementation should have the option NOT to hide the possible type
of message failure incidences.

In summary, the only changes I see to the base would be to offer a STRIP
method for the BH= coupled with a BO= hash that will help put a security
wrapper around authenticity protection.

   - Add bc=strip to DKIM-SIGNATURE header
   - Calculate bh= based on strip
   - Calculate b= signature
   - Add bo=orig-body-hash to DKIM-SIGNATURE header

Comments?

I personally like it and I think it will help me once again get excited
about restarting the DKIM project.   The current system has a higher cost in
software changes in our framework where we still have various legacy 822
mail processing and gateway considerations.  In addition, the current spec
introduces a redesign dilemma of where exactly do we fit DKIM in our
framework.  Using a STRIP and BO= to protect it, will help eliminate our
integration design considerations and more importantly lower implementation
cost.

---

From: Mark Delany <[email-masked]>
Date: Tue, 25 Jul 2006 03:36:52 +0000

> wish to waste time it isn't technically sound.

Well, one of the earlier DKIM nofws variants was nixed because it had
something similar. One of the concerns was that you can re-inject a
mail and *effectively* remove MIME boundaries and various MIME headers
as far as the UA is concerned, yet still have the email verify.

For example:

	--pWyiEgJYm5f9v55/
	Content-Type: image/jpeg
	Content-Transfer-Encoding: base64

can get turned into this:

	--pWyiEgJYm5f9v55/
	Content-Type: image/jpegContent-Transfer-Encoding: base64

Is that a problem? Probably not, but it makes me nervous because I
can't assess how UAs will react and I can't predict future MIME
headers.

> 1) Add bc=STRIP tag for the DKIM-SIGNATURE.

One thing that came out of the earlier nixing of the nofws variant is
that new canonicalizations are in the business of trying to prove a
negative - that something is safe. Just because no one identifies a
fault in a few days on a mailing list does not in any way mean that no
fault exists.

My general premise has always been that the more leeway you give, the
more risk there is. Consequently, folk proposing more leeway need to
demonstrate the cost/risk benefits more clearly than proposals that
afford less leeway.

Mark.

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 25 Jul 2006 01:02:05 -0400

----- Original Message -----
From: "Mark Delany" <[email-masked]>
To: <[email-masked]>
Sent: Monday, July 24, 2006 11:36 PM
Subject: Re: [ietf-dkim] Eat all CR/LF - STRIP Canonicalization Method

> can't assess how UAs will react and I can't predict future MIME

Not sure I follow.  The canoniculation should be transparent for hashing
purposes.  It should not be for correcting feeds.

If such content was change to the above,  you would be able to assest with
100% certainty all MIME processors (including our own) will fail since there
is no delimiter.  All MIME processors are not going be able to parse/read
this anymore.  Put another way, no would should expect the MIME processor,
whether it was a UA or some other middle waire mail processor, to be able to
parse the above.  There has to be atleast 1 delimiter.

The suggestion I am making removes the idea CR/LF alterations have taken
place from the DKIM hashing process.  The becomes an independent feed
concept because the canonicalization  is internal.  There is no
consideration for "correcting" a feed for other systems.

There are two parts:

- The BH is based on a transparent hashing as one ingredient for
   the DKIM signature, and

- A separate BO for detecting original content alternation.

The verifier will have the ability to report (provide result for a
presenters, etc), a  DKIM Signature is valid, but the original method
content has  been alterated.

In the current scheme, signature validation would fail.

In the old nofws,  it would survive, but there wasn't a way to determine
what actually failed. I believe this was the main point behind the new body
hash (bh=), as a way to determine if the body failed or the header failed.
This was why I added my support to it. It was a good idea.

But we now need to fine tune it because we have not yet to resolve how to
increase the surviiablity of the DKIM signature hashing during distibution.

> fault in a few days on a mailing list does not in any way mean that no

Agree, but I don't see any improvement in the current scheme.  It appears
nofws was removed because one made the determination that it was insecure.
Was there an approach to how to fix it instead?

> demonstrate the cost/risk benefits more clearly than proposals that

I understand this philosophy 100% and in general, practice it myself.

But we still have an engineering problem that will minimum the WIDE
acceptability of DKIM.  I am proposing a solution that might help without
having the security problem that was deemed with nofws.

I capped WIDE because DKIM is very good for a closed or exclusive signing
system where there is a strong two-way DKIM negotiation between "known"
end-points.

But as a wide adoption, I strongly believe DKIM runs a high risk on causing
more harm than good with an open-ended broadcasting of high potential
failures.

The only way to avoid it will be to ignore it  - completely.   Don't try to
support it,  just like we ignore all the Domainkey mail we see coming thru
our side.  With our testing, nearly all of them failed validation.  We saw
the "Fake Facsimiles" but for the most part it was all due to the mail
integrity.  It was impossible to determine why it really failed.  To me,
that is unacceptable

There needs to be a good reliable reason to begin DKIM implementation.  We
already did the cost assessment, and for us, it presented a very high impact
to implement DKIM as it currently defined.

I believe a little more fine tuning will eliminate and/or lower the cost,
and I think, this suggestion of stripping all CR/LF from the hashing will
atleast give us the ability make the validation more reliable. With the
additional, of the BO hash, that would resolve the authentication of the
mail content concerns.

What I see is this is a R1 and R2 condition that people will accept:

What we have now is a high signature invalid condition where most of the
failure is due to the hashing issue - a problem that can be solved and at
the same time help lower all the pressures of trying to change the world to
accommodate the current DKIM design.

---

From: Mark Delany <[email-masked]>
Date: Tue, 25 Jul 2006 05:37:14 +0000

> purposes.  It should not be for correcting feeds.

This is a replay issue:

o I get a verified email from paypal that has MIME attachments.
o I strip out the CR/LF characters to create the second form.
o I re-inject that mail into the system to some unsuspecting 
  bunny who has whitelisted paypal.
o Since STRIP will still verify such email, the bunny bypasses their
  protective layer because they trust paypal.

> is no delimiter.

Really? That's quite the generalization. Besides which, there is a
delimiter as the example removes one, not all delims. The case is
simply that the Content-Type: is now invalid. What can you say with
100% certainty about how all MIME parsers will deal with that?

> All MIME processors are not going be able to parse/read

That is my point. Code is buggy. DKIM verified mail from a trusted
domain will get through to a softer under-belly so a canonicalization
has to ask the question: if I allow modifications that get through a
softer underbelly what harm can I cause?

> consideration for "correcting" a feed for other systems.

Right. But a bad guy can modify the original content and it will still
verify with STRIP. This isn't about good-guys correcting, it's about
bad-guys modifying such things as general service notices from paypal
and replying them back to bunnies.

Mark.

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 25 Jul 2006 03:36:28 -0400

----- Original Message -----
From: "Mark Delany" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, July 25, 2006 1:37 AM
Subject: Re: [ietf-dkim] Eat all CR/LF - STRIP Canonicalization Method

>   protective layer because they trust paypal.

Mark, I don't see it.  Wouldn't a high value domain such as PAYPAL hashing
include From: and To: ?  If so, that cant change if the signature is to
survive this redirection.

At best, all I see done is a  mail redirection with nothing removed but
CR/LF characters from the body.  What is gained by the bad actor?

Besides, if the bunny is white listing paypal,  does it matter what is sent?
Is DKIM expected?   This all goes back to SSP related protections as well.
If I really wanted to fake out the bunny,  I would send it a message without
DKIM markings.

> 100% certainty about how all MIME parsers will deal with that?

With 100% certainly I do not expect any of my mail software engineering
peers to be correcting or even trying to correct or de-join or split MIME
headers that we joined by some mutation.   The point is, for this particular
scenario, its not a consideration how they will deal with it but with 100%
certainty, there will be problems and in most cases, the problems are
typically presented with customer compliants - "Hey, this message is not
displaying correctly."    At which point, you may investigate and you might
see that it bad line that no one is expected to reasonably understand or
deal with.   We do what we reasonbly can to deal with it for the customer
sake, but you can only go so far.    No one should expect anyone to be
dealing with 'bad formats."  They might do so, but thats besides the point.

> and replying them back to bunnies.

Ok.  But nothing as been done to address that.  Unless I missing something,
we are talking about increasing the survivability of the signature in the
current infrastructure where we are wasting alot of time discussing dead
horse issues regarding 822/2822 mutations.

If there is a security issue related to it that is more than what we have
now, I would like to see it.   Not saying there isn't one. I just have yet
to see it. If its there, would it be any worst than what we have now?  What
we don't have now is reliability of the signature hashing algorithm.

Well, anyway, thanks for comments. :-)

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 25 Jul 2006 11:28:16 -0400

----- Original Message -----
From: "Mark Delany" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, July 25, 2006 1:37 AM
Subject: Re: [ietf-dkim] Eat all CR/LF - STRIP Canonicalization Method

> verify with STRIP.

I missed this subtle point.

The only thing that could be modified by El Malo,  are CR/LF characters.
Nothing else can be modified without destroying the signature.

We lose nothing from a STRIP method, as long as the original BODY hash is
added to the scheme to help detect original body integrity changes of any
kind.

What is gained in increase survivability of the signature hashing, something
we don't have now with any degree of confidence.

Look,  I can't help but think if it was anyone else making this suggestion,
you wouldn't be able to kept up with this thread.    What a shame, it is
more important to push out a faulty spec than to fix the problem to make
DKIM more robust and acceptable.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 25 Jul 2006 17:04:12 +0100

Hi Hector,

> DKIM more robust and acceptable.

I don't think that's entirely fair. Your suggestion is really quite
close to nofws, which was found wanting from the security point of
view. In this case, I'm sure you can create messages that'd read
differently to a human under CR/LF manipulations. Signers IMO won't
accept such manipulations.

But if you disagree, that's certainly fine in which case I'd suggest
that the best thing would be to follow John Levine's idea to code it
up and try it out.

You can do this without requiring any change to base by including two 
signatures, the first say with simple c14n (so that bh= will be what
you're calling "bo="), and the 2nd with this new c14n. Then you can see
what happens. Your new c14n algorithm can be easily documented in an I-D
of its own, totally compatible with base. Were this c14n to prove a
success, then I'd imagine it'd be likely to be folded into a later
generation of the base RFC.

And since I don't think I've seen this said on the list (though it was
said at the mic in Montreal), there is a history here. Both S/MIME and
even moreso, XML signature had real difficulty with c14n, so this is an
area where a) we (at least I) expect problems, b) there are no silver
bullets (IMO) and so c) we'll only know we've gotten it right or wrong
after the base RFC issues. One consequence is that any new scheme will
be received with a certain amount of skepticism by those who've seen
c14n signing difficulties before.

So, why not write up your new c14n as an I-D and then see if folks
like it enough to write code.

Cheers,
Stephen.

---

From: "Hector Santos" <[email-masked]>
Date: Tue, 25 Jul 2006 16:11:12 -0400

Thanks Stephen, I will take your suggestions stated under advisement.