Subject: Major backtracking on canonicalization

From: Paul Hoffman <[email-masked]>
Date: Wed, 6 Jul 2005 15:28:41 -0700

Greetings again. I gravely misunderstood XML Canonicalization, and as 
it has been explained to me now, XML Canonicalization would be a 
disaster for Atom: what we want is Exclusive XML Canonicalization. 
See <http://www.w3.org/TR/2002/REC-xml-exc-c14n-20020718/>.

What I didn't get was that in normal XML Canonicalization, the 
canonicalized version gets all the external definitions added as 
text; that doesn't happen in Exclusive XML Canonicalization. I 
thought that in normal XML Canonicalization, those definitions got 
assumed; I didn't realize that they got actually put in as text. Yuck.

(I cannot understand how the folks who put together XMLDigSig could 
allow normal XML Canonicalization to be even thought of, much less 
the only required form. What a mess.)

Now that I understand this better, I believe that our text should read:

[[ NEW ]]

    Section 6.5.1 of [W3C.REC-xmldsig-core-20020212] requires support
    for Canonical XML. However, many people believe that Canonical XML
    may be deprecated in the future, and many implementers do not use
    it because signed XML documents enclosed in other XML documents have
    their signatures broken. Thus, Atom Processors that verify signed
    Atom Documents MUST be able to canonicalize with Exclusive XML
    Canonicalization.

Does anyone object to that?

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: James M Snell <[email-masked]>
Date: Wed, 06 Jul 2005 16:21:34 -0700

> Does anyone object to that?
No objections.

- James

---

From: Tim Bray <[email-masked]>
Date: Wed, 06 Jul 2005 16:41:25 -0700

On Jul 6, 2005, at 3:28 PM, Paul Hoffman wrote:

>
> Greetings again. I gravely misunderstood XML Canonicalization, and  
> as it has been explained to me now, XML Canonicalization would be a  

Urgh, I should have caught this.  Correct.  +1 -Tim

---

From: "Bob Wyman" <[email-masked]>
Date: Wed, 6 Jul 2005 20:21:43 -0400

> Now that I understand this better, I believe that our text should read:

Thank you for catching this. You've saved us major pain!

        bob wyman

---

From: Danny Ayers <[email-masked]>
Date: Thu, 7 Jul 2005 11:00:27 +0200

> Thank you for catching this. You've saved us major pain!

+1

-- 

http://dannyayers.com

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 7 Jul 2005 10:23:43 -0400

We should go into a little more detail.

Are we specifying exclusive c14n with or without comments? My  
preference would be without.

As I understand it, inherited xml:lang and xml:base attributes aren't  
signed when you're using exclusive c14n. If we ended up allowing per- 
entry signatures, we need to give guidance that xml:lang and xml:base  
should be explicitly included in the signed content if they are  
important.

It may be helpful to give guidance about the usage of the  
InclusiveNamespaces PrefixList, especially with default namespaces.

It also might be good to give guidance to extension authors and  
publishers about use of namespaces that aren't visibly bound; e.g.,  
QNames in content.

More information at:
   http://www.w3.org/TR/2002/REC-xml-exc-c14n-20020718/#sec-Limitations

Another good reference is the WS-I Basic Security Profile;
   http://www.ws-i.org/Profiles/ 
BasicSecurityProfile-1.0.html#xmlSignatureAlgorithms

--
Mark Nottingham   Principal Technologist
Office of the CTO   BEA Systems

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 7 Jul 2005 08:36:27 -0700

>preference would be without.

Without. That is explicitly the default for 
<http://www.w3.org/TR/2002/REC-xml-exc-c14n-20020718/>.

>As I understand it, inherited xml:lang and xml:base attributes 
>aren't signed when you're using exclusive c14n. If we ended up 
>allowing per-entry signatures, we need to give guidance that 
>xml:lang and xml:base should be explicitly included in the signed 
>content if they are important.

Why? We are signing bags of bits. Why add those from the outside?

>InclusiveNamespaces PrefixList, especially with default namespaces.

The whole purpose of using exclusive XML is to not need to guess 
about what is and is not in the bag of bits being hashed.

--Paul Hoffman, Director

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 7 Jul 2005 13:56:42 -0400

On 07/07/2005, at 11:36 AM, Paul Hoffman wrote:

> Without. That is explicitly the default for <http://www.w3.org/TR/ 
> 2002/REC-xml-exc-c14n-20020718/>.

Where does it state that explicitly? There are two identifiers in  
section four; it would be best to reference the spec and the  
applicable identifier by name.

> Why? We are signing bags of bits. Why add those from the outside?

Exclusive canonicalisation itself says, in section 5.1;
> applications must carefully specify the XML (i.e., source,  
> fragment, and target) or define the node-set processing (i.e.,  
> removal, replacement, and insertion) with respect to default  
> namespace declarations (e.g., xmlns="") and XML attributes (e.g.,  
> xml:lang, xml:space, and xml:base).
Imagine that you sign an entry that relies on an feed-level xml:base  
of "http://www.example.com/". If you exclusively canonicalise, an  
attacker could introduce an xml:base of "http://www.evil- 
attacker.net/" into the signed entry without invalidating the  
signature, effectively rewriting all of the URIs inside it. Similar  
problems would occur with xml:lang, since we depend heavily on it  
(although I see xml:base as a more serious problem).

> about what is and is not in the bag of bits being hashed.

Right. People need to understand the implications of including or  
excluding particular pieces of information from that bag, as per above.

--
Mark Nottingham     http://www.mnot.net/

---

From: James M Snell <[email-masked]>
Date: Thu, 07 Jul 2005 11:13:52 -0700

+1

>> applications must carefully specify the XML (i.e., source,  fragment, 
>> and target) or define the node-set processing (i.e.,  removal, 
>> replacement, and insertion) with respect to default  namespace 
>> declarations (e.g., xmlns="") and XML attributes (e.g.,  xml:lang, 
>> xml:space, and xml:base).
>
Good catch.  I had missed this when I was looking at ex-c14n.  +1 on 
needing to explicitly state that xml:lang and xml:base and default 
namespace declarations

+1

- James

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 7 Jul 2005 11:12:25 -0700

At 1:56 PM -0400 7/7/05, Mark Nottingham wrote:
>Where does it state that explicitly?

"Just as with [XML-C14N] one may use the "#WithComments" parameter to 
include the serialization of XML comments." Meaning: if you don't 
include #WithComment, you don't get comments.

>There are two identifiers in section four; it would be best to 
>reference the spec and the applicable identifier by name.

Works for me: http://www.w3.org/2001/10/xml-exc-c14n#

>of "http://www.example.com/".

There are zillions of external things that might affect the 
*interpretation* of what is signed. We saw that in the discussion of 
signing entries with external references.

We are signing the bits only, not some interpretation of the bits. 
That is true for the xml:base, the xml:lang, the xml:something-else, 
and so on.

I have no problem with the implementer's guide (or the 
signing-and-encrypting guide, if it happens) to talk about the 
dangers of interpreting things not covered by the signature, but if 
we start a list in the base spec, readers will assume that the list 
is exhaustive. We can easily tell now that it won't be.

--Paul Hoffman, Director

---

From: Martin Duerst <[email-masked]>
Date: Fri, 08 Jul 2005 08:43:04 +0900

At 03:12 05/07/08, Paul Hoffman wrote:

 >We are signing the bits only, not some interpretation of the bits. That 
is true for the xml:base, the xml:lang, the xml:something-else, and so on.

Just a clarification that I may have made previously: XML Canonicalization
(both kinds) convert to UTF-8 from whatever encoding your document or
fragment is, so essentially, we are signing characters, not bits.

Regards,    Martin.

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 7 Jul 2005 17:14:52 -0700

>fragment is, so essentially, we are signing characters, not bits.

Er, right. And we change the bits again to deal with whitespace and 
attribute ordering and all those other bits.

We are signing canonicalized bits.

--Paul Hoffman, Director

---

From: Karl Dubost <[email-masked]>
Date: Mon, 11 Jul 2005 14:25:55 -0400

Le 05-07-07 à 14:12, Paul Hoffman a écrit :
> "Just as with [XML-C14N] one may use the "#WithComments" parameter  
> to include the serialization of XML comments." Meaning: if you  
> don't include #WithComment, you don't get comments.

Hmmmm. not sure.

[[[
4. Use in XML Security

Exclusive Canonicalization may be used as a Transform or  
CanonicalizationMethod algorithm in XML Digital Signature [XML-DSig]  
and XML Encryption [XML-Enc].

Identifier:
http://www.w3.org/2001/10/xml-exc-c14n#
http://www.w3.org/2001/10/xml-exc-c14n#WithComments

Just as with [XML-C14N] one may use the "#WithComments" parameter to  
include the serialization of XML comments. This algorithm also takes  
an optional explicit parameter of an empty InclusiveNamespaces  
element with a PrefixList attribute. The value of this attribute,  
which may be null, is a white space delimited list of namespace  
prefixes, and where #default indicates the default namespace, to be  
handled as per [XML-C14N]. The list is in NMTOKENS format (a white  
space separated list).

]]] - http://www.w3.org/TR/2002/REC-xml-exc-c14n-20020718/#sec-Intro

I read
     Just as with [XML-C14N] one may use the "#WithComments"
     parameter to include the serialization of XML comments.

So it's a MAY, but it doesn't say that when the parameter is not here  
that there's a MUST NOT include them.

Going a bit further, because it is said "Just as with [XML-C14N]". I  
follow the link

[[[
The second parameter of input to the XML canonicalization method is a  
boolean flag indicating whether or not comments should be included in  
the canonical form output by the XML canonicalization method. If a  
canonical form contains comments corresponding to the comment nodes  
in the input node-set, the result is called canonical XML with  
comments. Note that the XPath data model does not create comment  
nodes for comments appearing within the document type declaration  
(DTD). Implementations are REQUIRED to be capable of producing  
canonical XML excluding all comments that may have appeared in the  
input document or document subset. Support for canonical XML with  
comments is RECOMMENDED.
]]] - http://www.w3.org/TR/2001/REC-xml-c14n-20010315#DataModel

ok, so it's not really a default feature/behaviour but it seems  
something like a flag that you have to give.

-- 
Karl Dubost - http://www.w3.org/People/karl/
W3C Conformance Manager
*** Be Strict To Be Cool ***

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 11 Jul 2005 15:10:19 -0700

At 2:25 PM -0400 7/11/05, Karl Dubost wrote:
>So it's a MAY, but it doesn't say that when the parameter is not 
>here that there's a MUST NOT include them.

Hmmmm, that's not how I read it at all. Given that we are talking 
about a canonicalization spec that has to say exactly what is and is 
not in the converted, I would find it surprising to have something 
that means "this may or may not appear in the output, you don't know".

>something like a flag that you have to give.

Agree.

--Paul Hoffman, Director