Subject: PaceDigitalSignatures

From: Mark Nottingham <[email-masked]>
Date: Fri, 3 Sep 2004 20:52:04 -0700

PaceDigitalSignatures currently says:

> Because every Atom root element is an XML object, Atom can use 
> existing XML
> security services. If an Atom root element is signed, it MUST be 
> signed with
> XMLDSig [ref]. If an Atom root element is encrypted, it MUST be 
> encrypted with
> XMLEnc [ref]. Publishers MUST NOT sign any Atom content other than a 
> root element.
> Publishers SHOULD NOT sign or encrypt Atom content unless security 
> services
> are actually needed.
>
> Atom consumers MUST be able to parse XMLDSig content to extract the 
> Atom content,
> even if they cannot do any signature validation. Atom consumers MUST 
> NOT reject
> an Atom item simply because it is signed. The consuming program can 
> decide whether
> to alert the user (and how to alert the user) if the program cannot 
> verify the
> signature on an Atom root element.

The first two MUSTs have the effect of ruling out use of any other 
encryption or signature mechanism, ever. A pedantic reading might even 
rule out SSL and TLS. Was that the WGs intent? It seems odd to restrict 
the choice of signature and encryption mechanisms without actually 
specifying a mechanism to use them in Atom.

The second paragraph requires all atom consumers to support XML Digital 
Signatures. Is it the Working Group's intent to require this? While I'm 
sure there are scenarios where XML Digital Signatures and XML 
Encryption are required, it seems draconican to require every Atom 
consumer to support them, and will likely turn people away from Atom, 
especially when SSL/TLS is adequate for many situations (and I note we 
don't require SSL support, BTW).

Based on these concerns, I'm -1 on PaceDigitalSignatures.

--
Mark Nottingham     http://www.mnot.net/

---

From: Tim Bray <[email-masked]>
Date: Sat, 04 Sep 2004 00:27:59 -0700

On Sep 3, 2004, at 8:52 PM, Mark Nottingham wrote:

> The second paragraph requires all atom consumers to support XML 
> Digital Signatures. Is it the Working Group's intent to require this? 
> While I'm sure there are scenarios where XML Digital Signatures and 
> XML Encryption are required, it seems draconican to require every Atom 

Note that the consensus position (see the Work Queue rotation) was that 
only child signatures, not siblings or envelopes, be supported.  Thus 
the burden on software is light, it doesn't have to *use* the 
signature, it just has to be able to spot it and skip over.  Not rocket 
science.

> Based on these concerns, I'm -1 on PaceDigitalSignatures.

Have you reviewed the extensive discussion in the WG that got us to the 
consensus position? -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 4 Sep 2004 01:14:39 -0700

On Sep 4, 2004, at 12:27 AM, Tim Bray wrote:
>
> Have you reviewed the extensive discussion in the WG that got us to 
> the consensus position? -Tim

Yes. It seemed to be a long discussion of the relative merits of HTTP 
and WSSE authentication, and not much actual discussion of the Pace.

These requirements seem to be aimed at future extensions; e.g., if I 
want to define a "signedTitle" entry element or somesuch, this Pace 
wants to give advice on how to go about it. However, no such extensions 
are defined today, which raises a few questions;

* Are Atom processors required to recognise and decrypt XML 
Encryption's EncryptedData element at arbitrary places in an Atom 
document? The current language is unclear as to whether an element has 
to specify that it can/should/must be encrypted, or if any can be.

* Likewise, are extensions required to define how they use signatures, 
if they do?

* Are extensions to Atom, or Atom publishers, forbidden from using 
other signature and encryption mechanisms? "If an Atom root element is 
signed, it MUST be signed with XMLDSig [ref]. If an Atom root element 
is encrypted, it MUST be encrypted with XMLEnc [ref]." implies this. 
It's also impossible to enforce these requirements; how do you know an 
extension isn't a signature mechanism if you don't understand it?

* "Publishers SHOULD NOT sign or encrypt Atom content unless security 
services
are actually needed." is a specious requirement; how does one judge 
whether they're "actually needed"?

---

From: Mark Pilgrim <[email-masked]>
Date: Sat, 4 Sep 2004 09:58:48 -0400

> and WSSE authentication, and not much actual discussion of the Pace.

A more recent discussion is here:

http://imc.org/atom-syntax/mail-archive/msg08804.html
http://imc.org/atom-syntax/mail-archive/msg08807.html
http://imc.org/atom-syntax/mail-archive/msg08812.html
http://imc.org/atom-syntax/mail-archive/msg08813.html
http://imc.org/atom-syntax/mail-archive/msg08817.html
http://imc.org/atom-syntax/mail-archive/msg08819.html
http://imc.org/atom-syntax/mail-archive/msg08820.html
http://imc.org/atom-syntax/mail-archive/msg08822.html
http://imc.org/atom-syntax/mail-archive/msg08824.html
http://imc.org/atom-syntax/mail-archive/msg08814.html
http://imc.org/atom-syntax/mail-archive/msg08816.html
http://imc.org/atom-syntax/mail-archive/msg08876.html

-- 
Cheers,
-Mark

---

From: "Bob Wyman" <[email-masked]>
Date: Sat, 4 Sep 2004 16:22:01 -0400

I am concerned that PaceDigitalSignatures may make life difficult for
synthetic feed generators. Addressing this concern may lead to support for a
new level of function whose discussion I had been hoping to defer...
Nonetheless, here goes:

	In constructing synthetic feeds, a service like PubSub's, will often
find it necessary to augment the information in an entry. These
augmentations could interfere with signatures. For instance, as has been
noted on a number of times on this list, at PubSub.com, we insert a
<ps:source-feed> element into each entry we publish in order to propagate
with the entry that feed metadata which is needed by downstream processors.
The content of <ps:source-feed> is the same data which the current Atom spec
has in the atom:head element. (I would like to have atom:head in entries...)
It isn't obvious to me how we could continue to do this with signed entries
without some means of indicating what parts of the entry were written by the
original author, and thus should be considered when evaluating the
signature, and what parts where inserted by us and should thus be ignored
when evaluating the signature.
	In the future, we anticipate "augmenting" the entries we publish
with a great deal of information other than just that which will appear in
atom:head elements. Augmentations may include data such:
	1. The "LinkRank"[1] of the source feed.
	2. Indicators of an entry's "relevance" to the user's subscription
	3. "Concept" indicators determined by textual analysis (i.e. is the
entry talking about politics, sports, religion, business, computers, etc.?)
	4. Links to "similar" entries in other feeds

	If signatures were of the envelope type, then life would be much
easier for us -- however, I understand and support all the arguments that
say that dealing with envelopes would increase complexity more than is
justified. 
	Given that envelopes are not acceptable, how do we modify or augment
an entry without breaking its signature? Also, how do we sign our own
additions to the entry?

		bob wyman

[1] http://pubsub.com/linkranks

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 4 Sep 2004 13:23:47 -0700

Thanks; I hadn't seen those (as always, it would help if people put the 
full name of the Pace in the Subject:).

I agree with the drift of that thread, but I don't think it answers the 
questions in my last e-mail;

---

From: Tim Bray <[email-masked]>
Date: Sat, 04 Sep 2004 13:58:05 -0700

> an entry without breaking its signature? Also, how do we sign our own

By definition, I don't think you can.  The signature applies to what 
was there before you started enhancing it in the opinion of some 
authority who isn't you.  So either you get that authority to re-sign 
the enhanced version or you add another private structure that 
indicates what you added so that someone could potentially reverse the 
process and check the original signature.  I just don't see how 
envelope signatures or anything are going to help you if you're 
changing digitally-signed content.  Perhaps I'm missing something 
obvious.  -Tim

---

From: "Bob Wyman" <[email-masked]>
Date: Sat, 4 Sep 2004 17:49:52 -0400

Tim Bray wrote:
> Perhaps I'm missing something obvious.  -Tim
	I think the "obvious" missing thing would be to define an element
which was explicitly intended to carry bits which annotate the entry or were
otherwise created by persons or processes other then the creator or signer
of the entry -- for instance, a feed synthesizer. Given such an element, the
Atom specification would state that this element was not to be included when
evaluating signatures.
	e.g. In the following, the "annotation" element would, by
definition, be exclused from the signature computation. It could, of course,
be signed on its own -- independent of the entry content. Additionally,
there could be multiple <annotation> elements -- each inserted at a
different point in the channel between the publisher and the eventual
processor of the entry.

<entry>
  <annotation xmlns:ps="http://pubsub.com/xmlns">
    <ps:linkrank percentile="95"/>
    <ps:subject>Politics</ps:subject>
    <ps:relevance value="42"/>
    <ps:similar>http://example.com/similarthing.html</ps:similar>
  </annotation>
  ...
  <content>
  </content>
</entry>

	An alternative might be to introduce an attribute, such as "nosig"?
This attribute would instruct signature verifiers to exclude the so flagged
elements from signature verifications.
	Does this make sense? Note: I realize it looks a little ugly, but I
can't think of an alternative. I think both signatures and annotations
inserted in the channel both have value. One should not exclude the other.
It is unfortunate that they appear to conflict. I'm hoping that a suitable
solution can be found that permits both.

		bob wyman

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 4 Sep 2004 15:02:56 -0700

At 1:23 PM -0700 9/4/04, Mark Nottingham wrote:
>>document? The current language is unclear as to whether an element 
>>has to specify that it can/should/must be encrypted, or if any can 
>>be.

Good question. My guess is that the WG will say "no". Please leave 
the text as it is for the -02 draft, but add a parenthetical question 
there. We'll certainly deal with it.

>>* Likewise, are extensions required to define how they use 
>>signatures, if they do?

No.

>>other signature and encryption mechanisms? "If an Atom root element 
>>is signed, it MUST be signed with XMLDSig [ref]. If an Atom root 
>>element is encrypted, it MUST be encrypted with XMLEnc [ref]." 
>>implies this.

Note the use of the phrase "root element" in the requirements. They 
apply to root elements, not to extensions, not to publishers.

>>It's also impossible to enforce these requirements; how do you know 
>>an extension isn't a signature mechanism if you don't understand it?

It is impossible to enforce any requirement; that doesn't stop us 
from using the language.

>>* "Publishers SHOULD NOT sign or encrypt Atom content unless 
>>security services

The publisher. If you have a clearer way of saying "don't 
gratuitously sign everything just because you have the spare 
bandwidth", please propose the text.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 4 Sep 2004 14:43:32 -0700

At 4:22 PM -0400 9/4/04, Bob Wyman wrote:
>an entry without breaking its signature?

You can't, of course. That's what signatures are all about.

>  Also, how do we sign our own
>additions to the entry?

Remove the old now-broken signature and add your own.

--Paul Hoffman, Director

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Sat, 4 Sep 2004 16:03:36 -0700

>Atom specification would state that this element was not to be included when

Please specify this fully in a Pace. Some of us who have grappled 
with the same problem in email for a decade and never even got close; 
maybe you can do better than us.

>elements from signature verifications.

An unsigned attribute that says "ignore the signature" sounds like a 
trivial security attack vector.

--Paul Hoffman, Director

---

From: "Bob Wyman" <[email-masked]>
Date: Sat, 4 Sep 2004 20:30:10 -0400

Paul Hoffman wrote:
>>This attribute would instruct signature verifiers to exclude the so 
>>flagged elements from signature verifications.
> An unsigned attribute that says "ignore the signature" sounds
> like a trivial security attack vector.
	Sorry, I apparently wasn't clear in what I wrote. What I'm
trying to suggest is that the attribute "nosig" would indicate that
element in which it was found was *not* part of the signed content.
Perhaps I should have suggested an attribute name of "not-signed". 
	I don't see the security attack problem. Elements which have the
"not-signed" attribute should *not*, in any way, assume any of the trust
attributes that derive from elements which are, in fact, signed.

		bob wyman

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 4 Sep 2004 20:18:04 -0700

On Sep 4, 2004, at 3:02 PM, Paul Hoffman / IMC wrote:

> Good question. My guess is that the WG will say "no". Please leave the 
> text as it is for the -02 draft, but add a parenthetical question 

I was planning on incorporating the text regardless; could we put this 
Pace back on the 'revisit' bin, though?

> apply to root elements, not to extensions, not to publishers.

OK, If root element == document element (i.e., atom:feed in a feed doc, 
atom:entry in an entry doc), I think I see a way to clarify this and 
honour the intent here.

Thanks,

---

From: Eric Scheid <[email-masked]>
Date: Sun, 05 Sep 2004 13:24:34 +1000

> an entry without breaking its signature? Also, how do we sign our own

every entry has a unique id. the things being signed are the entries, not
the feed.

thus ... put the augmenting info into the feed, not the entries, and
associate each ps:entry-info to atom:entry via atom:id.

you can then also sign the ps:entry-info

it could look something like this:

<feed>
<head>
    ...
</head>
<ps:augments>
    <ps:entry-info ref="[atom:id#1]>
        ...
    </ps:entry-info>
    <ps:entry-info ref="[atom:id#2]>
        ...
    </ps:entry-info>
</ps:augments>
<entry>
    <id>[atom:id#1]</id>
    ...
</entry>
<entry>
    <id>[atom:id#2]</id>
    ...
</entry>
</feed>

e.

---

From: "Bob Wyman" <[email-masked]>
Date: Sun, 5 Sep 2004 18:52:11 -0400

Eric Scheid wrote:
> every entry has a unique id. the things being signed are the entries,
> not the feed. ... thus ... put the augmenting info into the feed, not the
> entries, and associate each ps:entry-info to atom:entry via atom:id.
	My concern here is that this introduces an entirely new concept to
Atom processing. (It is well known in other XML formats, but hasn't been
required in Atom so far.) Additionally, this method of linking things
together according to their ids is something which would be needed to
understood by even those processors that don't care about signatures. This
seems like a great deal of added complexity unless there is substantial
benefit and no reasonable alternatives.

<feed>
  <entry><id>[atom:id#1]</id>
    <augments>...</augments>
    <Signature>...</Signature>
    ...
  </entry>
</feed>

	Is semantically equivalent to what you suggest:

<feed>
  <augments><ps:entry-info ref="[atom:id#1]">...</augments>
  <entry><id>[atom:id#1]</id>
    <Signature>...</Signature>
  </entry>
</feed>

	The extra complexity of doing the association via ids just doesn't
seem justified.
	The much simpler approach that I am advocating can be made to work
with signed content by simply writing an additional canonicalization rule
that states: "Remove any <augments> tags from the element to be signed."
	It seems reasonable to me that the introduction of signatures should
not, if at all possible, constrain other design decisions. Thus, I think we
should avoid modifications to a potential "<augments>" element simply
because signatures might be used.
	Note: The "attribute-based" alternative to an augments tag would
require a canonicalization rule that stated: "Remove any elements which
contain the non-signed attribute. Thus, the following would be work well
with signatures:

<feed>
  <entry><id>[atom:id#1]</id>
    <foaf:Person notsigned>...</foaf:Person>
    <Signature>...</Signature>
    ...
  </entry>
</feed>
	In the case above, the <foaf:Person> element would have been added
by some "augmenting processor" in the channel and is flagged as not being
part of the signed content in the entry.

		bob wyman

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Mon, 6 Sep 2004 09:25:14 -0700

>Perhaps I should have suggested an attribute name of "not-signed".

That's a possibility. It sounds complicated for the receiving part, 
but possibly doable.

--Paul Hoffman, Director

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 6 Sep 2004 13:29:04 -0400

Paul Hoffman wrote:
>     That's a possibility. It sounds complicated for the receiving
> part, but possibly doable.
	Yes, this introduces an increase in complexity; however, I believe
that the increase is minimal. My assumption is that some form of
canonicalization must be performed in any case in order to create or verify
a signature. Thus, the addition of a single, simple rule ("Remove all
embedded elements that have "not-signed" attributes.") would not be a
significant increase in complexity for a system that was already doing other
canonicalizations anyway. 
	What I propose is, I think, more simple than other alternatives in
that it has *no* impact on processors that do not concern themselves with
signatures. The only impact in such a case is a few more bytes in the entry
that should be trivially easy to ignore.
	When we talk of extensibility measures, I suggest that we strongly
consider making a requirement that all elements that are added to an entry
by intermediate processors should insert the "not-signed" attribute to
indicate the added elements were not part of the original content. This will
add important information to entries (i.e. attribution -- or the lack of it)
as well as permit in-channel augmentation by intermediaries without breaking
signatures. (Removal or modification of the content of a signed element
would, of course, still break the signature. I see no way around that.)
	Having suggested that in-channel augmentations be flagged as
"not-signed", I think it is inevitable that others will wish to suggest that
even more meta-data be provided concerning the augmentations. For instance,
some might argue that an identification of the process which inserted the
data should be included. That discussion is, I think, best left for later.
	In any case, I think this is an effective and cheap means for
providing in-channel augmentations without breaking signatures. I would be
very interested in any explanations of why it would not work or what
problems it might cause. I may be missing something obvious.

		bob wyman

---

From: "The Purple Streak, Hilarie Orman" <[email-masked]>
Date: Mon, 6 Sep 2004 12:15:46 -0600

The "not-signed" stuff may turn out to be very fragile.  There's no
room for the signer and the verifier to disagree about how to remove
the "not-signed" portions, because even a single bit of difference
will invalidate the signature.  XML canonicaliziation already is
complicated and in some cases ill-defined.  The rules for handling
"not-signed" would make a bad matter worse.

What are the scoping rules for the tags?  Suppose someone wants to
include a signature over the entire item, including the "not-signed"
portions.  Do the "not-signed" tags apply to the innermost signature
only?

It also seems likely that people will tend to interpret the
"not-signed" portions inheriting some degree of legitimacy because
they are included in a signed item.  In a fuzzy way they will probably
assume that there's some kind of authentication involved, maybe weaker
than a digital signature, but still, something more than no
authentication.

Hilarie

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Mon, 6 Sep 2004 13:31:54 -0700

At 8:18 PM -0700 9/4/04, Mark Nottingham wrote:
>>>>document? The current language is unclear as to whether an 
>>>>element has to specify that it can/should/must be encrypted, or 
>>>>if any can be.
>>
>>the text as it is for the -02 draft, but add a parenthetical 
>>question there. We'll certainly deal with it.
>
>I was planning on incorporating the text regardless; could we put 
>this Pace back on the 'revisit' bin, though?

Instead of putting the whole Pace back on "revisit", I'll create a 
Pace with the parenthetical question and a proposed answer.

--Paul Hoffman, Director

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 6 Sep 2004 17:14:21 -0400

Hilarie Orman wrote:
> The "not-signed" stuff may turn out to be very fragile.
	It should be noted that I'm basing my discussion on ideas which are
discussed at some length in the specifications related to Signatures.[1] I'm
not actually proposing anything new here other than the adoption of
conventions that will permit more consistent and better-understood
application of the Signature specifications[2].

> What are the scoping rules for the tags?
	The scope of "not-signed" elements would be restricted to their
immediate parent elements and all descendents of their immediate parents.
	For instance, the following XPath Filter, when used in the
signature, would accomplish the same effect as what I'm suggesting:

<XPath Filter="subtract"
    xmlns="http://www.w3.org/2002/06/xmldsig-filter2">
      .//*[@not-signed]
</XPath>

(Note: I may have made a mistake in the syntax... But, you should get the
idea.)

> Suppose someone wants to include a signature over the entire item,
> including the "not-signed" portions.
	In this case, the easiest thing to do is to simply remove the
not-signed attribute from the element. But, that might cause problems...
	The problem comes in the case where two or more signatures exist for
a single element. One that signs the content exclusive of not-signed
elements and one or more that sign the original content plus one or more
not-signed elements. To handle this case (which I think will be rare), the
proposal would have to be extended to allow the signature to explicitly
reference the XPaths or ids of the not-signed elements that should be
included in the signature. i.e. a mechanism is needed to override the
not-signed attribute on a signature-by-signature basis. "Not-signed" would
mean "not-signed by default -- unless explicitly overridden.
	To include two not-signed elements that had the particular ids "one"
and "three", one might use something like this in the signature:

<Transform>
  <XPath Filter="intersect">.//* </XPath>
  <XPath Filter="subtract">.//*[@not-signed]</ XPath>
  <XPath Filter="union">id(one, three)</XPath>
</Transform
(Once again, please forgive syntax errors...)

> Do the "not-signed" tags apply to the innermost signature only?
	The consensus seems to be that Atom should only permit signatures as
child elements -- not envelopes. Thus, the "innermost" concept is a bit
weaker then it might be in other applications. In any case, the
canonicalization rule would say that all not-signed elements -- at all
levels -- should be removed (unless explicitly overridden) prior to
signature construction or verification.

> "not-signed" portions inheriting some degree of legitimacy
> because they are included in a signed item.
	There is always a risk that people who are ignorant of, or who
ignore, the details of a specification will make mistakes. However, I think
the risk is no greater than would exist if the in-channel augmentations are
kept in distinct elements. Assuming that in-channel augmentations *will* be
exist, the question is how best to represent them... I suggest that
explicitly flagging them is useful.

> There's no room for the signer and the verifier to disagree about
> how to remove the "not-signed" portions, because even a single bit
> of difference will invalidate the signature.
	This is a problem with all canonicalization rules. In this case, the
subject of "agreement" should not be left to the "signer and the verifier".
Rather, the specification of Atom would unambiguously define how the removal
was to be performed. The personal opinions of either the signer or the
verifier would be irrelevant. (Note: I assume that other canonicalization
rules will deal with questions such as how to remove any white-space that
might have been introduced in the process of first inserting and later
removing a not-signed element.)

	Perhaps it would be best to forgo modifying the canonicalization
rules and instead rely completely on the Filtering capabilities of the
Signature methods. However, even if this is the end result, I would still
argue that Atom should, at least, recommend a "best" and consistent method
of flagging content inserted by in-channel processes so that clients can
more easily figure out how to determine what is signed and what is not.
	Thanks for your comments. I hope I've adequately addressed your
concerns. Let me know if I haven't.

		bob wyman

[1] http://www.w3.org/TR/xmldsig-filter2/ This document contains a number of
examples of handling things "to be signed" and "not to be signed".
[2] http://www.w3c.org/Signature/