Subject: OK, enough with atom:id

From: Tim Bray <[email-masked]>
Date: Thu, 05 Aug 2004 11:00:22 -0700

The survey (http://intertwingly.net/wiki/pie/IDSurvey) is conclusive.

We have consensus that atom:id will be canonical URI per 
http://www.intertwingly.net/wiki/pie/PaceCanonicalIds

We also have consensus that we need strong language in our draft that 
atom:id must be present, globally unique, and immutable (even when an 
entry moves from category to category or feed to feed or wherever).  
There is plenty of good candidate language out there on the mailing 
list and in various Paces.

There is lots of room for proposing language for our drafts or an 
implementors' guide to make atom:id work better.  There have been 
various proposals to recommend some particular URI scheme or URN 
namespace for Atom; so far no specific scheme/namespace has managed to 
get anything like rough consensus.  Doesn't mean it can't happen, just 
means it hasn't yet.

To our editors: please feel free to go ahead and roll in the consensus 
language.
Sam, there might be a few Paces on the issues list that can now be 
closed?  -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 5 Aug 2004 14:59:35 -0700

While respecting that consensus has been determined, I'd like to 
register my views, since I missed the opportunity beforehand.

After reading the (extensive!) threads, my impression is that the use 
cases for atom:id have not been given due consideration. Namely, no-one 
has suggested that atom:id be used for anything except identifying 
whether two instances of a construct (e.g., an atom feed) refer to the 
same logical thing.

Tim did point out that URIs can and often are written on notes, 
billboards, etc. and copied down. I do not dispute this, but it does 
not follow that all URIs in all situations are exposed to people (and 
thus subject to transposition errors).

As such, we're using URIs in atom:id as pure identifiers; they're not 
used to locate anything (e.g., by network access), and they're AFAICT 
directly copied by machines when they're transposed.

If this is the case, it seems that the underlying requirements for 
atom:id are:
   - easy to compare two instances to determine if they're the same
   - easy to make globally unique

URIs fit the bill nicely.

However, as far as comparison goes, I don't see any reason to add the 
extra machinery of URI canonicalisation. Doing so adds complexity to 
the specification and implementations, and introduces a lot of 
scheme-specific logic. Comparing atom:id as a sequence of characters is 
much cheaper and simpler.

One concern raised about this approach was that implementations that 
use XML Schema (e.g., Java, C#) would have trouble because they 
automagically normalise URIs. In fact, I think this is an argument 
against using canonical URIs, because those implementations' 
interpretation of URI canonicalisation may not be the same as Atom's, 
and may diverge over time.

Instead, the XML Schema type of atom:id should be xs:string, and its 
value should be restricted to a URI -- any URI -- by specification 
prose.

Please note that I'm not saying that the current approach won't work -- 
it will. My objection is that this adds needless complexity to the Atom 
specification without corresponding benefit, and that it will make some 
feeds needlessly invalid.

If there were use cases where transposition errors of atom:id are 
likely, I'd be much more amenable to this proposal, but I haven't seen 
that discussion.

Thanks for listening,

--
Mark Nottingham     http://www.mnot.net/

---

From: Graham Parks <[email-masked]>
Date: Fri, 06 Aug 2004 18:14:17 +0100

Abso-frigging-lutely. No to canonicalization, Yes to everything Mark just said.

(I didn't bother with the survey because I was busy with other things, and also the stealth re-introduction of voting as "surveys" sucks)

Graham

---

From: Ken MacLeod <[email-masked]>
Date: 07 Aug 2004 14:58:16 -0500

> Abso-frigging-lutely. No to canonicalization, Yes to everything Mark

I'm not sure I understand where the difference is.

"Canonical IDs" says the spec text requires a URI and also requires
that the URIs be in "canonical" form to enable consumers to use only
string comparison level of equivalence.

Mark N and Graham seem to be indicating that string equivalence is the
goal and that [mentioning | recommending | requiring] canonical IDs to
achieve that goal is "less workable".

It seems to me the same thing is being stated from two different
directions.

** Regardless, I do agree that somewhere it should be noted that the
   XSD datatype should be "string" and not "URI" to enable string
   comparison and non-rewriting of URIs by XSD libraries.  The spec
   text prose will still be clear that the content must be a URI and
   is not an arbitrary string.

> (I didn't bother with the survey because I was busy with other
> things, and also the stealth re-introduction of voting as "surveys"
> sucks)

It's not voting.  The chairs need a way to determine consensus when
it's not clear from the messages themselves.  One technique is "straw
polls" (Tim may have used "survey" for one fewer wiki words in our
case).  The difference between voting and a straw poll is that no one
"wins" a straw poll simply by quantity of numbers.  The chairs still
use their reading of the straw poll to help determine where consensus
is, and is not.  At least that's my current understanding.

  -- Ken

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 8 Aug 2004 22:03:49 -0700

What does requiring a canonical URI buy us?

---

From: Sam Ruby <[email-masked]>
Date: Mon, 09 Aug 2004 08:38:37 -0400

> What does requiring a canonical URI buy us?

http://www.imc.org/atom-syntax/mail-archive/msg08261.html

- Sam Ruby

---

From: Graham Parks <[email-masked]>
Date: Mon, 09 Aug 2004 13:47:59 -0400

>string comparison level of equivalence.

All that's required to do string comparison is that there only be one form of each ID. One way to achieve this is Sam's, where we decide how everyone must canonicalize there URIs. The much more straightforward way is to have the form emitted by the original publisher be the canonical form of that ID. There is absolutely no valid reason for downstream software to change the ID into another form, so this works just as well and doesn't require us to invent or consumers to learn a set of complicated rules.

I don't actually see what problems Sam's canonical forms solves. I think the two were:
1) Human error in transcription. While it might correct a small subset of errors, it's not really that useful. There's also very few reasons for humans to be transcribing IDs ever.
2) URI classes that don't preserve exact form. There's no guarantee that these classes will follow the same rules as us, so surely to solve this problem we need to require URI level comparison.

Seriously Sam, why are we doing this?

Graham

---

From: Sam Ruby <[email-masked]>
Date: Mon, 09 Aug 2004 15:15:07 -0400

Graham Parks wrote:

>  2) URI classes that don't preserve exact form. There's no guarantee
> that these classes will follow the same rules as us, so surely to
> solve this problem we need to require URI level comparison.

If they produce the URI scheme in uppercase letters, their ids will be 
flagged as invalid.

If they provide the host in uppercase letters, their ids will be flagged 
as invalid.

If they ... etc., etc., etc.

  = = =

What I want is if there are two feeds, one in which an identifier is 
spelled as <id>a</id>, and in the other it is spelled as <id>a'</id>, 
then the comparison is either deterministic in that *every* clients gets 
the same result, *or* one or both feeds are declared to be invalid.

My fear is that by not precisely defining equivalence, we enable an arms 
race whereby one implementation uses a normalization rule during 
comparison, then some feeds are produced that work with that 
implementation, and then other consumers are required to reverse 
engineer the rule and implement aggressive canonicalization, etc.

Canonicallizing all possible schemes is a large task.  By comparison, 
producers of new entries know what schemes they are producing, generally 
have only to check a few simple rules (or let the validator do the 
checking for them), and often they will have to do nothing.  Clients, 
like shrook, also have to do nothing.

> Seriously Sam, why are we doing this?

http://www.imc.org/atom-syntax/mail-archive/msg08178.html
http://www.imc.org/atom-syntax/mail-archive/msg08108.html
http://www.imc.org/atom-syntax/mail-archive/msg08194.html

- Sam Ruby

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 9 Aug 2004 13:38:44 -0700

/me bangs head against wall.

There seems to be a circular argument here, along the lines of "we need 
to compensate for different forms of URIs because people might take 
advantage of the slack we give them to compensate for different forms 
of URIs..."  Why are we giving them the slack? I've heard "robots" and 
"spiders" mentioned, but I don't find that convincing if Atom IDs are 
neither to be dereferenced by machines nor transcribed by humans.

Can someone walk me through the soup-to-nuts scenario of how a 
non-canonical URI in an Atom feed causes problems for these use cases, 
if we specify a simple string comparison function? I've looked at the 
links Sam has provided and remain unconvinced.

Also,

> My fear is that by not precisely defining equivalence, we enable an 
> arms race whereby one implementation uses a normalization rule during 

I don't think anyone is advocating an imprecise definition of 
equivalence. My fear is that Atom is going to become complex and 
convoluted enough to understand that it'll serve as a barrier to 
adoption.

On Aug 9, 2004, at 12:15 PM, Sam Ruby wrote:

> If they provide the host in uppercase letters, their ids will be 
> flagged as invalid.
>
> then the comparison is either deterministic in that *every* clients 
> gets the same result, *or* one or both feeds are declared to be 
> invalid.
>
>
>
> producers of new entries know what schemes they are producing, 
> generally have only to check a few simple rules (or let the validator 
> do the checking for them), and often they will have to do nothing.  
> Clients, like shrook, also have to do nothing.
>

---

From: Dan Brickley <[email-masked]>
Date: Mon, 9 Aug 2004 17:59:45 -0400

I'm not so worried about the open-endedness of canonicalisation. 

There is nothing in the URI spec to stop URI schemes overlapping in
coverage, for example. The self-same thing might quite reasonably 
have multiple independent URIs (including urn:* URIs) which name it, and 
there's nothing we can do to legislate against that, since nobody 
(thankfully!) has a monopoly on naming. Is this frustrating for 
completists and perfectionists? you bet. Does it mean that all Atom-spec and 
URI-spec compliant software might not behave identically? yep. Is it a
business opportunity for hardworking canonicalisers? perhaps. A possible 
source of end-user frustration? certainly... But imho unavoidable if
outside of closed world / intranet environments. Canonicalisation 
*within a URI scheme* might be a more perfectable undertaking, for some 
schemes, but it's just part of a larger problem. Not a problem that 
keeps me awake at nights though...

Dan

---

From: David Powell <[email-masked]>
Date: Tue, 10 Aug 2004 01:14:25 +0100

> http://www.imc.org/atom-syntax/mail-archive/msg08194.html

I don't buy the argument that Atom spiders will want to "normalize"
Atom ids.

Web spiders normalize URLs to avoid having to fetch and process the
same resource several times. How does this apply to Atom ids? You
can't use an atom:id to directly retrieve an entry, so why would an
agent want to normalize it?

I can understand spiders normalizing the Feed URI, because it is used
for retrieval, but there is no reason to normalize the id.

If we say now that an Atom identifier requires character-by-character
comparison (as Namespaces and RDF do), then a republisher that
"normalizes" an atom id is broken.

Anyway, a reason not to require canonical URIs:

Publishers that have invested in RDF (such as RSS1.0), will have
already assigned URI identifiers to their publications. These URIs may
or may not fit whatever canonicalization rules that we come up with.
If we say that only canonical URIs are allowed, then publishers will
have to either change perfectly good ids to be canonicalized (bad), or
manage an additional canonicalized alias URI for use in Atom (bad).

(I am against requiring a specific URI scheme for ids for the same
reason.)

I'm not going to push this too hard, I'm sure Atom will work fine
whether we require URI canonicalization or not, but not requiring it
would save us a big chunk of text in the spec.

-- 
Dave

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Tue, 10 Aug 2004 22:51:07 +0200

> To our editors: please feel free to go ahead and roll in the consensus

What is the «consensus language», exactly?

> closed?

Which ones are those? (Just wondering.)

-- 
Asbjørn Ulsberg         -=|=-        [email-masked]
«He's a loathsome offensive brute, yet I can't look away»

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Tue, 10 Aug 2004 22:51:08 +0200

On 07 Aug 2004 14:58:16 -0500, Ken MacLeod <[email-masked]> wrote:

> It's not voting. [...] At least that's my current understanding.

Mine as well.

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Tue, 10 Aug 2004 15:33:48 -0700

We are still not talking about atom:ids for about a week. Tim and I 
will say when and how to start the discussion again.

--Paul Hoffman, Director
--Internet Mail Consortium