Subject: PaceMustUnderstandElement

From: Tim Bray <[email-masked]>
Date: Thu, 13 Jan 2005 10:36:43 -0800

+1.

The objections to this fall into two forms:
1. We don't have prior art in the syndication space that proves this is 
needed.
2. This is someone else's problem, e.g. SOAP

I can see both those arguments, but when I re-visit and re-read this, 
the implementation is so falling-off-a-log obvious, like one small 
subroutine, that if you're doing cost/benefit math, the cost is very 
close to zero.

And I can easily imagine lots of scenarios in which it would be 
extremely empowering for someone to design an extension and be 
confident that their feeds would be processed only by software that 
knows it.

And a decision to leave this out feels like hubris to me.  "We're so 
sure we got the core right that we can leave out an extremely cheap 
potentially-valuable extension point."  I doubt it.

  -Tim

---

From: Eric Scheid <[email-masked]>
Date: Fri, 14 Jan 2005 07:36:06 +1100

-1

I still have misgivings about the underlying model, mainly due to it's
binary nature. What happens if at some future point we want to rev the
definition of some element which has already been revved once before in it's
history? It already has the mU wart, and thus version 3 of that element
would be indistinguishable from version 2. Or do we introduce a
mustUnderstandAgain attribute?

e.

---

From: Graham <[email-masked]>
Date: Thu, 13 Jan 2005 20:51:23 +0000

On 13 Jan 2005, at 6:36 pm, Tim Bray wrote:

> 1. We don't have prior art in the syndication space that proves this 
> is needed.

3. We don't need it and it won't be implemented anywhere and it's 
extremely open to abuse and/or failure.

We are not having a mustUnderstand mechanism.

Graham

---

From: Antone Roundy <[email-masked]>
Date: Thu, 13 Jan 2005 14:56:15 -0700

On Thursday, January 13, 2005, at 01:51  PM, Graham wrote:
> 3. We don't need it
That's #1 above

> and it won't be implemented anywhere
Given the utter simplicity of implementation, I think it will be 
implemented in many clients.  If no publisher ever uses it, then that's 
very little loss.  (However, I don't expect it to be implemented 
universally.  How serious a problem will that be? )  If people make 
extensions and use Atom for uses that they wouldn't have lacking mU, 
then that could be a significant gain--we'll see.

> and it's extremely open to abuse
Could you explain?  Do you mean that people who don't implement support 
for certain extensions will feel abused?  If so, here's why I don't 
think <mU> makes the situation any worse--if anything it makes it 
better:

If a significant number of feeds, or if significant feeds, use an 
extension that really IS important to understand in order to process 
the feed in a non-bad way, clients that don't support it will get 
complaints whether they ignore it (no <mU>) or report that they can't 
process it (with <mU>).  If they report that they can't process it, the 
complaints will be along the lines of "I need to access this feed! 
Please add support (or I'll get a new feed reader)!"  If they ignore 
it, the complaints will be "This feed doesn't show up properly! Fix it 
(or I'll get a new feed reader)!" and "Your feed reader is causing 
problems with my system because it's partially processing my feed, but 
it isn't doing XYZ with extension ABC! Fix it you lazy, stinking, 
system-wrecking hacker!"

If the problem is a significant number of feeds, or significant feeds, 
marking something mU when it really doesn't hurt to not understand it, 
then yes, developers will get unnecessary complaints.  That could lead 
to:

A) More extensions being supported by more applications
B) Applications with the user-option to ignore mU (hopefully on a 
feed-by-feed and/or extension-by-extension basis)
C) Developers complaining to publishers
D) Developers publicly criticizing publishers
E) Developers telling their customers to complain to publishers
F) A few customers actually doing so
G) Some publishers getting their act together

A little pain, sure.  But how likely is it that the problems will 
outweigh the benefits?  We don't have prior experience to gauge either 
by.  I think the potential for pain is small enough to make the 
experiment worth the risk.

> and it's extremely open to failure.
Could you explain?

---

From: David Powell <[email-masked]>
Date: Thu, 13 Jan 2005 22:29:48 +0000

Does anyone have any example use cases for mustUnderstand?

-- 
Dave

---

From: Graham <[email-masked]>
Date: Thu, 13 Jan 2005 22:34:15 +0000

> That's #1 above

#1 is dependent on there not being prior art. Even if there were, we 
still wouldn't need it.

> implemented in many clients.

What's the value in implementing it? Yes its simple, but it means x% of 
feeds will now fail in my client, and I will get x% more complaints. 
What's the benefit to users?

What I'll be doing in Shrook if we have this, is ignoring extensions I 
know of but don't implement (and know that not supporting them does 
damage), and in all other cases processing the document.

>> and it's extremely open to abuse

I'm imagining someone thinking (or just deciding) they need to put all 
the extensions they use in mustUnderstand, whether they must be 
understood or not. If more than a couple of these feeds like this show 
up then clients will just give up paying any attention to mU.

>  If they report that they can't process it, the complaints will be 
> along the lines of "I need to access this feed! Please add support (or 
> I'll get a new feed reader)!"  If they ignore it, the complaints will 
> be "This feed doesn't show up properly! Fix it (or I'll get a new feed 
> reader)!".

Exactly, the feed not working properly will be obvious to the user 
whether it matters or not.

>> and it's extremely open to failure.

Most obviously, typos, incorrect implementations, poor testing etc. All 
parts of atom are susceptible to this, of course, but mU just creates 
another potential point of failure, and with no hope of recovery, since 
the processor must discard the whole document.

The main problem ince a possibly large percentage won't implement it, 
using mustUnderstand in a feed doesn't prevent whatever dire 
consequences there might be of ignoring the elements that must be 
understood, which makes the proposed system unable to serve its 
purpose.

---

From: Tim Bray <[email-masked]>
Date: Thu, 13 Jan 2005 14:46:06 -0800

> Does anyone have any example use cases for mustUnderstand?

1. A stream of financial disclosures from a public company in a 
highly-regulated industry.  The legislation is very clear that they may 
not say anything in public unaccompanied by disclaimers and 
limitation-of-liability statements.  The financial industry gets 
together an introduces an extension that requests clients to display 
these disclaimers in a fashion that meets the regulatory requirements.  
If Atom has MustUnderstand, compliant clients that can't do this will 
never fail to display the appropriate material, and this reduces the 
risk of litigation and makes it more likely that such feeds will be 
created.

2. A stream of information that uses a special-purpose 
digital-signature scheme to establish the authenticity of the 
information.  People should not act on this information without 
checking the signature.  A person using a conformant Atom client can be 
sure that they won't see anything that hasn't been checked.

  -Tim

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Thu, 13 Jan 2005 14:59:01 -0800

At 4:29 PM -0600 1/13/05, Roger B. wrote:
>All it will take is an A-lister or three on a mission to essentially
>force every general-use client developer to support whatever pet
>extension they're fired up about that week, no matter how
>ill-conceived.

Are you saying that Atom 1.0 should not be extendable at all? That 
is, should we take out the namespace mechanism altogether? I don't 
remember hearing support for that before.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "Roger B." <[email-masked]>
Date: Thu, 13 Jan 2005 16:29:40 -0600

> But how likely is it that the problems will
> outweigh the benefits?

Antone: Extremely, in my opinion. A big -1 on this one.

All it will take is an A-lister or three on a mission to essentially
force every general-use client developer to support whatever pet
extension they're fired up about that week, no matter how
ill-conceived. And offering a user-controlled bypass isn't an
option... I'm not ever gonna be in the mood to be sued by someone
because my app allowed the user to turn off a feed's bundle of "must
understand" DRM elements.

If a publisher's data requires non-core elements to be minimally
useful, then I'd say Atom is the wrong format for that data.

--
Roger Benningfield

---

From: Antone Roundy <[email-masked]>
Date: Thu, 13 Jan 2005 16:32:36 -0700

> understood, which makes the proposed system unable to serve its 
Okay, mU offers no guaranteed protection against bad side effects, and 
the spec text should make this very clear.  It would still be useful 
for USERS who want to ensure that their feed reader is processing feeds 
correctly, as in the examples Tim gave (sorry, no reference--it's not 
listed in the archive yet).

---

From: Antone Roundy <[email-masked]>
Date: Thu, 13 Jan 2005 16:42:18 -0700

> extension they're fired up about that week, no matter how
...or to turn an A-lister into a B-lister.  Yeah, if this happens, it 
will cause some pain.

But hey, maybe that's a good thing.  After all, we in the syndication 
community have been historically proven to love sniping, arguing and 
fighting.

> And offering a user-controlled bypass isn't an
Okay, so, as mentioned in my last email, mU is there to benefit the 
client, not to guarantee protection to the publisher.  Maybe we should 
change it to "should-understand" or "important"..., "SHOULD NOT process 
the document...", etc.

---

From: "Roger B." <[email-masked]>
Date: Thu, 13 Jan 2005 17:42:05 -0600

> Are you saying that Atom 1.0 should not be extendable at all?

Paul: Not at all. I'm simply saying that "must understand" gives
influential publishers the power to force the hands of developers,
something that RSS (wisely) doesn't provide.

In addition, I have a philosophical objection, which I touched on
briefly in my last message. Atom feeds should contain well-formed
entries... if an entry requires non-core elements to make it minimally
useful, I'd argue it isn't well-formed, and the use of Atom was
ill-advised.

--
Roger Benningfield

---

From: Graham <[email-masked]>
Date: Thu, 13 Jan 2005 23:46:32 +0000

On 13 Jan 2005, at 10:46 pm, Tim Bray wrote:

> highly-regulated industry.  The legislation is very clear that they 
> may not say anything in public unaccompanied by disclaimers and 

I can't imagine these people would use mustUnderstand to do that. If 
it's important, they'll want to put it in content too.

> checking the signature.  A person using a conformant Atom client can 
> be sure that they won't see anything that hasn't been checked.

Um, couldn't the malicious person easily remove the mustUnderstand 
element while they're adding spoofed information?

Can you try a bit harder with number 3?

Graham

---

From: Robert Sayre <[email-masked]>
Date: Thu, 13 Jan 2005 18:53:59 -0500

> risk of litigation and makes it more likely that such feeds will be 

So they write a spec that "requires display", and now any client that 
doesn't "display" stuff can't use it? This proposal seems uniquely prone 
to abuse by "morons" and "assholes."[0]

Why does the proposal equate "vocabularies" with XML Namespaces? I don't 
think the namespace spec does that(??). Is it possible to "understand a 
namespace"?

Also, this proposal does seem to have real implementation costs for 
programs that don't use streaming parsers directly. For example, 
SafariRSS parses feeds with an XQuery script. They also store entries 
forever, but maybe they don't version feed properties. What happens to 
their implementation if a feed intermittently contains these mU 
declarations? I'm confused by it.

Robert Sayre

[0] http://diveintomark.org/archives/2004/08/16/specs
     http://www.google.com/search?q=morons+and+assholes

---

From: Tim Bray <[email-masked]>
Date: Thu, 13 Jan 2005 16:16:07 -0800

The other Tim Bray, speaking as co-chair, observes that 
PaceMustUnderstandElement is hopelessly dead, with apparently no 
support and lots of detractors.  Thank you.  You may now return to your 
regularly scheduled programming. -Tim

---

From: Walter Underwood <[email-masked]>
Date: Thu, 13 Jan 2005 16:27:27 -0800

Excellent examples. Each of these could be handled without mustUnderstand.

Define an extension for entries. Put the restricted content inside the
extension. The extension would include the display constraints between
the content portions and the disclaimer or authentication portions.

This could mean duplicating some content elements inside the extension.
On the other hand, it would add support for restricted content instead
of redefining regular content as potentially restricted.

wunder

--On Thursday, January 13, 2005 02:46:06 PM -0800 Tim Bray <[email-masked]> wrote:

> 1. A stream of financial disclosures from a public company in a highly-regulated industry.  The legislation is very clear that they may not say anything in public unaccompanied by disclaimers and limitation-of-liability statements.  The financial industry gets together an introduces an extension that requests clients to display these disclaimers in a fashion that meets the regulatory requirements.  If Atom has MustUnderstand, compliant clients that can't do this will never fail to display the 
appropriate material, and this reduces the risk of litigation and makes it more likely that such feeds will be created.
>
> 2. A stream of information that uses a special-purpose digital-signature scheme to establish the authenticity of the information.  People should not act on this information without checking the signature.  A person using a conformant Atom client can be sure that they won't see anything that hasn't been checked.
>
>   -Tim
>

--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: Joe Gregorio <[email-masked]>
Date: Thu, 13 Jan 2005 19:35:06 -0500

> regularly scheduled programming. -Tim

Yay! I'm glad this goes down with more than me
as the lone detractor.

    -joe

-- 
Joe Gregorio        http://bitworking.org

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 14 Jan 2005 12:27:01 +0000

Sorry Tim, one more concern and I'm done.

I think mU introduces scenarios where content cannot be safely or 
properly processed without looking to the metadata for control codes, ie 
metadata no longer is advisory or supplementary information. I suspect 
this is a significant innovation in feed formats.

cheers
Bill

Bill de hÓra wrote:
> Hi Tim,
> I haven't decided yet. I would dearly like a sane way to inform the 
> downstream to Look Here. But I'm concerned; my expectation based on 
> experience with this sort of thing is that many people will think their 
> element must be understood and will not always consider that the other 
> data can be understood or useful without their specific extension - this 
> is a bit like the unwillingness to prioritize requirements in a software 
> project ("we have to have it all").
> 
> And it does seem to me to introduce barriers to content and/or 
> incidental complexity - I can't easily buy into the zero cost argument 
> when I consider the content. I believe mU might result in inaccessible 
> feeds and/or format extension wars fought by proxy.  I don't even want 
> to think about what nastiness could happen in DRM, copyright or inline 
> advertising scenarios. For example, I think mU could be used to 
> introduce machine processable rights declarations by stealth, something 
> we have already agreed to stay away from for now.
> 
> I guess my main concern is that mU is not so much a technical mechanism 
> for managing change, but a policy mechanism for controlling access.
> 
> cheers
> Bill
> 
>

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 14 Jan 2005 12:21:06 +0000

Hi Tim,

cheers
Bill

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 14 Jan 2005 14:30:43 +0000

Tim Bray wrote:
> PaceMustUnderstandElement is hopelessly dead, with apparently no support 
> and lots of detractors.  Thank you.  You may now return to your 

My comments went out before this hit my inbox, apologies.

cheers
Bill

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 21 Jan 2005 03:13:38 +0100

> 2. This is someone else's problem, e.g. SOAP

...Or this isn't such a big problem at all. Let's not go there. -1.

> potentially-valuable extension point."  I doubt it.

I don't think that's how it is. We all want Atom to be extensible in as  
many ways as possible.

-- 
Asbjørn Ulsberg     -=|=-    http://virtuelvis.com/quark/
«He's a loathsome offensive brute, yet I can't look away»