Subject: Fwd: RE: Last Call: 'S/MIME Version 3.1 Message Specification' to Proposed Standard

From: Russ Housley <[email-masked]>
Date: Thu, 22 Apr 2004 10:26:26 -0400

Does the S/MIME WG have a response to this comment?  I suspect so.  Please 
coordinate a response and send it to [email-masked] before Last Call expires.

Russ

>From: "Peter Hesse" <[email-masked]>
>To: <[email-masked]>, <[email-masked]>
>Cc: "'Blake Ramsdell'" <[email-masked]>
>Subject: RE: Last Call: 'S/MIME Version 3.1 Message Specification' to 
>Proposed Standard
>Date: Wed, 21 Apr 2004 09:34:50 -0400
>
>I am sorry for not catching this hadn't been handled earlier, but I would
>like to reiterate a request I made in November 2003 for this document.  The
>archived message can be found here:
>http://www.imc.org/ietf-smime/mail-archive/msg01800.html or, you can just
>read below
>
>--------
>I have recently run into a problem with signed emails not being able to be
>verified, because of the presence of the word "From" in the first columns of
>a line of the email message.  This email will serve as an example of this
>potential problem.  If your email client sees this message as signed but the
>signature is invalid, the next paragraph should start with the word
>"From"--see if it has been modified.
>
> >From appearing as the first characters after a blank line will result in
>some email delivery agents (such as sendmail or exim) escaping the
>word--"From" is replaced with ">From".   The reason for this behavior has to
>do with the UNIX mbox mail storage file format.  The mbox format stores
>multiple messages in one file, and the messages are separated by the word
>"From" as the first characters following a blank line.  Some mail delivery
>agents do not have this problem (i.e. Exchange), because they do not store
>messages in the mbox format.  Many do, however, resulting in a modification
>of the message and the signature being invalidated.
>
>I would like to request that this issue be more directly dealt with in
>son-of-RFC2633.  (Currently, it is mentioned in the example MIME-encoded
>message, but nowhere in the text.)  One recommendation might be to borrow
>from RFC2015 (MIME Security with PGP), which states:
>    Though not required, it is generally a good idea to use Quoted-
>    Printable encoding in the first step (writing out the data to be
>    signed in MIME canonical format) if any of the lines in the data
>    begin with "From ", and encode the "F".  This will avoid an MTA
>    inserting a ">" in front of the line, thus invalidating the
>    signature!
>
>Perhaps this might even be a SHOULD, although I will ask the group to weigh
>in on that.
>--------
>
>Thanks,
>
>--Peter Hesse

---

From: "Blake Ramsdell" <[email-masked]>
Date: Thu, 22 Apr 2004 18:07:14 -0700

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Russ Housley
> Sent: Thursday, April 22, 2004 7:26 AM
> To: [email-masked]
> Subject: Fwd: RE: Last Call: 'S/MIME Version 3.1 Message 
> Specification' to Proposed Standard 
> 

I may be missing the exact details here, so people can jump in where I'm
misguided.

Basically this is a multipart/signed message where a signature was
applied, and an entity that performed intermediate processing
deliberately violated section 2.1 of RFC1847 which states:

<verbatim>
The entire contents of the multipart/signed container must be treated
as opaque while it is in transit from an originator to a recipient.
Intermediate message transfer agents must not alter the content of a
multipart/signed in any way, including, but not limited to, changing
the content transfer encoding of the body part or any of its
encapsulated body parts.
</verbatim>

So is this "From mangling" problem prevalent enough to warrant special
treatment in the S/MIME spec is the question here, since it's already
been stated that you shouldn't do it. Based on the lack of discussion
and consensus when this issue was first brought up, we need to
understand the community feel better. That is, is this particular
mangling more prevalent than the other issues that aren't currently
discussed in the spec (are there whitespace issues we need to think
about, for instance)?

Blake

---

From: "Peter Hesse" <[email-masked]>
Date: Thu, 22 Apr 2004 22:26:59 -0400

Blake,

Perhaps the best case for the "From mangling" problem being prevalent is
found if you look at the IETF-S/MIME archive page and look at the first
message I sent:

http://www.imc.org/ietf-smime/mail-archive/msg01800.html

You will see the second complete paragraph begins with ">From".  I sent it
as "From".  So, whatever mail transfer agent received this message and
posted it to the archive was subject to this problem.

For the most part, any MTA system which uses the classic unix mail spool
will suffer from this problem.  Exim and sendmail both have options to turn
on "From escaping", and the common recommendation is typically to enable
this option.  (Details at bottom).  Generally, the majority of UNIX,
Solaris, BSD, and Linux based mail servers I have encountered suffer from
this problem.

Additionally, this problem occurs in unexpected places due to the prevalence
of HTML-enabled email applications, which send both HTML and plain-text
versions of email messages.  Although the word "From" may not appear at the
beginning of a line in the message, due to linebreaks being inserted by the
mailing application when sending, it might end up there anyway.  This can
get munged by an MTA, and it is often very difficult to determine that this
is what caused a digital signature to not be verified.  (The HTML gets
displayed to the reader and wasn't modified--the plaintext is what got
mangled.)

I think that adding the text from RFC2015 is an easy way to cover this
problem.  It states that handling this is "not required" but "generally a
good idea".  I invite you and the rest of the ietf-smime group to read about
this and consider it before we finalize son-of-2633.  Thanks,

--Peter Hesse

Links to additional information:
---------------------------------
Sendmail configuration:
http://www.sendmail.org/~ca/email/doc8.8/op-sh-5.html#sh-5.4
E        Escape lines beginning with From in the message with a `>' sign. 
 
EXIM configuration
http://www.us.exim.org/exim-html-3.30/doc/html/spec_15.html#SEC443
escape_string (appendfile)
Type: string
Default: ">From " 
 
On FAQs.org, the Linux Network Administrators Guide
(http://www.faqs.org/docs/linux_network/x-087-2-mail.message-format.html),
there is the quote:
"If you look into a local mailbox file, you may see each mail message
preceded by a "From" line (note: no colon). This is not an RFC-822 header;
it has been inserted by your mail software as a convenience to programs
reading the mailbox. To avoid potential trouble with lines in the message
body that also begin with "From," it has become standard procedure to escape
any such occurrence by preceding it with a > character."

+---------------------------------------------------------------+
| Peter Hesse                    [email-masked]     |
| Phone: (703)934-2031         Gemini Security Solutions, Inc.  |
| ICQ: 1942828                     www.geminisecurity.com       |
+---------------------------------------------------------------+
"Pay no attention to what the critics say; there has never been 
a statue set up in honor of a critic." --Jean Sibelius

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Blake Ramsdell
Sent: Thursday, April 22, 2004 9:07 PM
To: 'Russ Housley'; [email-masked]
Subject: RE: RE: Last Call: 'S/MIME Version 3.1 Message Specification' to
Proposed Standard 

> Does the S/MIME WG have a response to this comment?  I suspect so.  
> Please coordinate a response and send it to [email-masked] before Last 
> Call expires.

Basically this is a multipart/signed message where a signature was applied,
and an entity that performed intermediate processing deliberately violated
section 2.1 of RFC1847 which states:

<verbatim>
The entire contents of the multipart/signed container must be treated as
opaque while it is in transit from an originator to a recipient.
Intermediate message transfer agents must not alter the content of a
multipart/signed in any way, including, but not limited to, changing the
content transfer encoding of the body part or any of its encapsulated body
parts.
</verbatim>

So is this "From mangling" problem prevalent enough to warrant special
treatment in the S/MIME spec is the question here, since it's already been
stated that you shouldn't do it. Based on the lack of discussion and
consensus when this issue was first brought up, we need to understand the
community feel better. That is, is this particular mangling more prevalent
than the other issues that aren't currently discussed in the spec (are there
whitespace issues we need to think about, for instance)?

Blake

---

From: "Blake Ramsdell" <[email-masked]>
Date: Thu, 22 Apr 2004 20:21:26 -0700

> From: Peter Hesse [mailto:[email-masked]] 
> Sent: Thursday, April 22, 2004 7:27 PM
> To: 'Blake Ramsdell'; 'Russ Housley'; [email-masked]
> Subject: RE: RE: Last Call: 'S/MIME Version 3.1 Message 

Well, hang on, you had a particular case involving a particular set of
agents (including a) a user agent that wasn't smart enough to escape the
From at the desktop, and b) an MTA that chose to introduce the escape on
his side). The clients that I use happen to protect you from this
(independent of their S/MIME support), so personally, I haven't seen it
before.

I'm not saying that I can't conceive of a case where this happens, what
I'm saying is "if we add this, then are there other cases we need to
consider where MTAs munge content and how clients should defend against
that" as well as "is this issue important enough for us to call out
specifically".

In any case, the current path is a non-normative paragraph as Paul
suggested, which covers MTA munging in general.

Blake