Subject: Re: [smime] eContentType for detached signatures

From: Russ Housley <[email-masked]>
Date: Thu, 1 May 2014 14:37:24 -0400

Paul:

I still think that we did a reasonable thing in RFC 5485.

When I was looking at draft-flanagan-nonascii-01, it jumped out to me that id-ct-asciiTextWithCRLF would not be appropriate for such documents.  I suggested that Heather add an appendix to assign an object identifier for the UTF-8 documents that are described here.  That is a reasonable and consistent way forward.  There is no place to carry a media type in the detached signature.

I think id-ct-mime is appropriate for a content that is MIME encoded.  That is, the media type is carried in the content.

Russ

On Apr 30, 2014, at 7:25 PM, Paul Hoffman wrote:

> We agree that having a id-ct-mime would be a good thing. We also seem to agree that id-ct-asciiTextWithCRLF signals that the content must be be processed with the exact canonicalization specified in RFC 5485 before signing and validation. I was wrong when I said that I didn't see the need for the type for the content type you are focused on; I was actually looking at the other two types that were defined in RFC 5485, id-ct-xml and id-ct-pdf, which do not use the canonicalization rules given in the RFC. It was from there that I worry that this indicates that all types, not only those that need canonicalization, need to be defined, given that we have no definition for any.
> 
> This thread was prompted by Russ asking that, when Internet Drafts and RFCs no longer are expected to be all-ASCII, do we need another CMS content type. To me, this means making up a new canonicalization form for these, but we will also need to make one up for all the for other types that might be published. Further, it pointed out to me that the canonicalization that is defined in RFC 5485 is already fragile. It doesn't apply to two of the types defined in that RFC, and it makes a false assumption that all inputs are really ASCII (sometimes, non-ASCII characters slip through).
> 
> So, I still question whether or not RFC 5485 did the right thing in defining id-ct-xml and id-ct-pdf, and particularly in not defining something that could be called id-ct-unknown.
> 
> --Paul Hoffman

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 1 May 2014 14:46:17 -0700

> I still think that we did a reasonable thing in RFC 5485.

In retrospect, the general idea of "here's a marker for what needs to be canonicalized for signing (but not storage) and validating" is probably a good one. The implementation on RFC 5485 is confusing and possibly over-broad.

> I think id-ct-mime is appropriate for a content that is MIME encoded.  That is, the media type is carried in the content.

Jim and I discussed this a bit offline and came up with a reasonable way forward.

- Obsolete RFC 5485 with a new document of similar structure but more careful wording.

- Create a new CMS content type id-ct-canonicalizedText that is defined to solve the problems of line-ending changes that happen in FTP and in some browsers when saving files to disk as text *and that is all*. It would cover any text encoded in UTF-8. Do not include stripping of white space before line ends; if you do, fully define what characters are "white space" Make it very clear that this canonicalization is only for the signing and validation steps, *not* for storage on disk or conversion between attached and detached signatures.

- Create a new CMS content type id-ct-mime to assist processors that need to know the inner content type of non-text items.

- Discuss that non-text content should be marked with id-data.

- Deprecate all the id-ct values in RFC 5485. This will have nearly zero operational impact because the signatures are not long-term (and in fact are not being created today due to software bugs).

--Paul Hoffman

---

From: "Peter Rybar" <[email-masked]>
Date: Fri, 2 May 2014 12:40:45 +0200

Dear all,

When the binary data (included in a file) are not the same as the HASED binary data (included in a hashing procedure), because before hashing procedure the data are transformed into the canonical form, there must be also defined a new OID for a new canonicalMessageDigest because for OBJECT IDENTIFIER 1.2.840.113549.1.9.4 messageDigest are defined rules where the whole data are as input to the hashing procedure.

The solution, when the whole binary data are hashed but the type of the data interpretation/processing is important, for the protection of the type of signed data is used signed attribute id-aa-contentHint [RFC2634] where contentDescription field contains MIME header with MIME Content-Type:

      SEQUENCE {
       OBJECT IDENTIFIER 1.2.840.113549.1.9.16.2.4 contentHint
       SET {
        ContentHints SEQUENCE {
         contentDescription UTF8String 
MIME-Version: 1.0
Content-Type: application/vnd.oasis.opendocument.text; name="ATT00122.odt"
Content-Disposition: attachment; filename="ATT00122.odt"
         contentType OBJECT IDENTIFIER 1.2.840.113549.1.7.1 data
        }
       }
      }

It is also used for secure publication of web files with external CMS signature e.g. for a trusted list protection http://www.nbusr.sk/en/electronic-signature/signature-policies.1.html 

http://ep.nbusr.sk/kca/tsl/tsl.xml.p7s 

When some new rules are expected e.g. when id-ct-canonicalizedText will be defined then also id-aa-canonicalMessageDigest must be defined and used.

Proposed creation of a new CMS content type id-ct-mime can be more helpful when the document(s) will be signed indirectly.

Signed document identified by id-ct-mime will be DER or maybe a JSON encoded DataReferences type. When charset is UTF-8 then it is a binary file without canonicalization. Only ASCII txt can be processed with canonicalization.

The JSON proposal below, containing ASCII and UTF-8 data, can be also transformed as ASN.1:

DataReferences:
{
    "hash": {
        "OID": 		"2.16.840.1.101.3.4.2.1",
        "parameters": 	"",
        "name": 		"SHA256"
    }, 
    "fields": [ 
    {
      "data": {
        "url": "http://www.archive.eu/docs/?typ=123",
        "name": "invoice",
        "ext": "txt",
        "Content-Type": "text/plain"      
      },
      "canonicalization": {
        "OID": 		"1.2.840.113549.1.7.27",
        "name": 		"asciiTextWithCRLF"
      },      
      "hash": 		"MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=",
      "info":	"invoice 123"
    },
    {
      "data": {
        "url": "http://www.archive.eu/docs/?typ=432",
        "name": "invoice",
        "ext": "txt",
        "Content-Type": "text/plain; charset=UTF-8"
      },
      "hash":		"JspKEHtF3LyCalml/2mXuG1jLZ2NjdQCo2hkn+49k7o=",
      "info":	"invoice 432"
    }
    ]
} 
 
Peter Rybar
National Security Authority
Information Security and Electronic Signature Department
Budatinska 30, 850 07 Bratislava 57, Slovak Republic
tel.: +421 2 6869 2163
mob.: +421 903 993 708
fax: +421 2 6869 1701
e-mail: [email-masked]
e-mail: [email-masked]

-----Original Message-----
From: smime [mailto:[email-masked]] On Behalf Of Paul Hoffman
Sent: Thursday, May 01, 2014 11:46 PM
To: Russ Housley
Cc: IETF SMIME
Subject: Re: [smime] eContentType for detached signatures

--Paul Hoffman
_______________________________________________
smime mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/smime

---

From: Russ Housley <[email-masked]>
Date: Fri, 2 May 2014 10:22:41 -0400

Paul:

I do not agree.  When working on RFC 5485, the idea was to use a canonicalization that was already well defined and widely implemented.  That is why the FTP wire format for plain text was selected.

The detached signature should, in my opinion tell the type of the content being signed.  The use of id-data does not convey this information.

Your last point is incorrect.  There have been many I-D signatures that are correct using id-ct-asciiTextWithCRLF.  There are software bugs, and they are being worked, but some of the signatures are valid.

I have no problem with defining id-ct-mime for cases where the content is MIME encoded.  This object identifier would essentially mean, parse the content to extract the media type.

Russ

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 2 May 2014 07:52:34 -0700

On May 2, 2014, at 7:22 AM, Russ Housley <[email-masked]> wrote:

> I do not agree.  When working on RFC 5485, the idea was to use a canonicalization that was already well defined and widely implemented.  

NVT-ASCII is well defined and is not widely implemented, at least not implemented correctly.

> That is why the FTP wire format for plain text was selected.

Calling NVT-ASCII the "FTP wire format" is an overstatement. Many or most implementations only do the CRLF transposition, not the whitespace stripping.

> The detached signature should, in my opinion tell the type of the content being signed.  The use of id-data does not convey this information.

This means that anyone who wants to create a detached signature for a type of data that is not currently in the registry needs to write a specification and apply for one. It seems strange to believe that no one in the past five years has created a detached signature for, say, an MP3, a stand-alone HTML file or a JSON file.

> Your last point is incorrect.  There have been many I-D signatures that are correct using id-ct-asciiTextWithCRLF.  There are software bugs, and they are being worked, but some of the signatures ar…

Are you saying there will be significant negative operational impact of replacing those signatures with new ones? Given the "some" in that last sentence, I'm not sure I can imagine the problems.

> I have no problem with defining id-ct-mime for cases where the content is MIME encoded.  This object identifier would essentially mean, parse the content to extract the media type.

That was the least important proposal of the bunch for the work at hand.

--Paul Hoffman

---

From: Russ Housley <[email-masked]>
Date: Fri, 2 May 2014 11:01:06 -0400

> Are you saying there will be significant negative operational impact of replacing those signatures with new ones? Given the "some" in that last sentence, I'm not sure I can imagine the problems.

New signatures need to be generated for the I-D where there was a canonicalization problem.  The ones that did not have a canonicalization problem do not need new signatures.

Russ

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 2 May 2014 08:11:29 -0700

> New signatures need to be generated for the I-D where there was a canonicalization problem.  The ones that did not have a canonicalization problem do not need new signatures.

Quite true. We could have two different content types on the signatures, the old and the new. That seems silly, though, if no one is relying on the old signatures.

--Paul Hoffman

---

From: Russ Housley <[email-masked]>
Date: Fri, 2 May 2014 11:51:48 -0400

> Quite true. We could have two different content types on the signatures, the old and the new. That seems silly, though, if no one is relying on the old signatures.

What?  We cannot know that yet.

From RFC  5485:

   ...  At this point in time, it is important to support signature
   validation of expired Internet-Drafts that are obtained from non-IETF
   repositories.  Therefore, the appropriate value for such a signed
   attribute is unclear.  This specification allows an Internet-Draft
   and companion signature file to be stored anywhere without hindering
   signature validation.

The point is that the files can be gathers from anywhere on the Internet.  Then, the digital signature will show that the file has not been changed from the one posted by the IETF Secretariat.  At some point, I hope this signature validation will allow lawyers to perform this check and stop asking the IETF to make such confirmations in subpoenas.

Russ