Subject: ID wording

From: Paul Hoffman / IMC <[email-masked]>
Date: Wed, 18 Aug 2004 13:57:48 -0700

Greetings again. Thanks for keeping the moratorium on ID discussion. 
I thought it would be best to do a sanity check to be sure people 
were in agreement on what it seems the consensus so far was.

- The format for IDs is going to be URIs, not plain strings.

- IDs must not change over time.

- Even though the format is a URI, the IDs are not expected to be 
dereferencable (although they might be).

- IDs MUST be compared character-by-character, no case-mapping, no 
%xx conversion, and so on.

- Receiving and gateway systems MUST NOT alter IDs in any way.

- Because IDs might be dereferenced, and because they must not be 
changed, it is a good idea to create them in a canonicalized form. 
There is no interoperability issues with this suggestion, so it is 
just a suggestion, not a SHOULD-level suggestion.

Does this match what people feel was the general consensus?

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Sam Ruby <[email-masked]>
Date: Wed, 18 Aug 2004 17:30:55 -0400

Not that all moratoriums are lifted, I've been quietly working on a 
series of paces which attempts to streamline and tighten a number of 
children of atom:entry elements:

   http://www.intertwingly.net/wiki/pie/NoInventions

Note: this are paces I've authored as a member of the working group, I'm 
not presenting this in my role as secretary.  Also, these are not a 
package, each Pace can be evaluated independently.

- Sam Ruby

---

From: Eric Scheid <[email-masked]>
Date: Thu, 19 Aug 2004 11:59:38 +1000

On 19/8/04 6:57 AM, "Paul Hoffman / IMC" <[email-masked]> wrote:

> what it seems the consensus so far was.
> 

+1

---

From: Martin Duerst <[email-masked]>
Date: Thu, 19 Aug 2004 13:09:55 +0900

Very good summary. +1.   Reagrds,   Martin.

At 13:57 04/08/18 -0700, Paul Hoffman / IMC wrote:

>Greetings again. Thanks for keeping the moratorium on ID discussion. I 
>thought it would be best to do a sanity check to be sure people were in 
>agreement on what it seems the consensus so far was.
>
>- IDs MUST be compared character-by-character, no case-mapping, no %xx 
>conversion, and so on.
>
>- Because IDs might be dereferenced, and because they must not be changed, 
>it is a good idea to create them in a canonicalized form. There is no 
>interoperability issues with this suggestion, so it is just a suggestion, 
>not a SHOULD-level suggestion.
>

---

From: Graham <[email-masked]>
Date: Wed, 18 Aug 2004 23:56:57 -0500

> just a suggestion, not a SHOULD-level suggestion.

Absolutely - see, that wasn't hard, was it? That's where the consensus 
has always been.

(I still don't understand the case for canonicalized ids - is there 
some reason for it beyond gateways that accidentally canonicalize not 
doing any damage?)

Graham

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 18 Aug 2004 22:17:09 -0700

On Aug 18, 2004, at 1:57 PM, Paul Hoffman / IMC wrote:

> thought it would be best to do a sanity check to be sure people were 
> in agreement on what it seems the consensus so far was.
>

+1 - well said.

--
Mark Nottingham     http://www.mnot.net/

---

From: Tim Bray <[email-masked]>
Date: Wed, 18 Aug 2004 23:18:02 -0700

> some reason for it beyond gateways that accidentally canonicalize not 

java.net.URI and the C# equivalent... a programmer sees that an atom:id 
is a URI and uses such things to store them, and well, they try to be 
smart.  Yes, the programmer probably shouldn't do that. -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 18 Aug 2004 23:49:44 -0700

Ah. Can we agree to add this to the list;

- If/when Atom is described with XML Schema, the ID should be a 
xs:string, not xs:anyURI. I.e., the content will be restricted to a URI 
in prose, not in the schema.

?

On Aug 18, 2004, at 11:18 PM, Tim Bray wrote:

> java.net.URI and the C# equivalent... a programmer sees that an 
> atom:id is a URI and uses such things to store them, and well, they 
> try to be smart.  Yes, the programmer probably shouldn't do that. -Tim
>

---

From: Tim Bray <[email-masked]>
Date: Wed, 18 Aug 2004 23:54:29 -0700

On Aug 18, 2004, at 11:49 PM, Mark Nottingham wrote:

> xs:string, not xs:anyURI. I.e., the content will be restricted to a 
> URI in prose, not in the schema.

I'm nervous about getting that specific... this feels like it belongs 
in an Implementor's Guide or some such.  I think we have agreement that 
we should provide schemas to help out (although not that they be 
normative), and a well-commented entry in such a schema seems like a 
good way to do it. -Tim

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 11:16:14 +0200

> Does this match what people feel was the general consensus?

Yes, very much so. But I'm not sure if any of your bullets include that  
the ID must be absolute, and not relative? If that is the  
canonicalization-part, then I feel this should be mentioned explicitly and  
be a MUST, not SHOULD. Absolute versus relative URI's as ID's is indeed a  
(big) interoperability issue.

-- 
Asbjørn Ulsberg         -=|=-        [email-masked]
«He's a loathsome offensive brute, yet I can't look away»

---

From: Julian Reschke <[email-masked]>
Date: Thu, 19 Aug 2004 11:38:54 +0200

Asbjørn Ulsberg wrote:

> 
> On Wed, 18 Aug 2004 13:57:48 -0700, Paul Hoffman / IMC 
> <[email-masked]>  wrote:
> 
> canonicalization-part, then I feel this should be mentioned explicitly 
> and  be a MUST, not SHOULD. Absolute versus relative URI's as ID's is 
> indeed a  (big) interoperability issue.

URIs always are absolute. Relative URIs are not URIs (duh!). See current 
discussion on http-uri mailing list.

:-)

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: David Powell <[email-masked]>
Date: Thu, 19 Aug 2004 10:59:24 +0100

> discussion on http-uri mailing list.

The other difference between a URI and a URI reference is that according to
RFC2396classic the "absoluteUri" term doesn't allow fragments to be used, in
RFC2396bis the "URI" term is introduced which does allow fragments.

I think that fragments should definitely be allowed in atom:id - they are used
widely in RDF, and if RDFers have already assigned URIRefs to their
publications, they aren't going to be pleased about having to assign another
set of id aliases for the benefit of Atom.

Check out PaceUriReferences where I proposed replacing the use of "URI" with
"URI Reference", and also proposed that atom:id should be an "absolute URI
Reference".

-- 
Dave

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 12:21:01 +0200

On Thu, 19 Aug 2004 11:38:54 +0200, Julian Reschke <[email-masked]>  
wrote:

> URIs always are absolute. Relative URIs are not URIs (duh!).

Hmkay. Is it not necessary to write something about this in the  
specification, or do you think everyone that knows what an URI is also  
knows that it can't be relative? I remember a lengthy discussion about  
this on this list a while ago, and it took a long time for people to agree  
that atom:id should not be relative (even though most people thought it  
should be an URI).

---

From: Bill de hÓra <[email-masked]>
Date: Thu, 19 Aug 2004 11:55:29 +0100

Tim Bray wrote:
> I'm nervous about getting that specific... this feels like it belongs in 
> an Implementor's Guide or some such.  

Sure it's specific, but's a real point of breakage. Also it only 
covers bindings off schema and not hand coded stuff (I'd be tempted 
to include some weasel words about using string types in programming 
languages).

Ok, how about either,

"XML Schema descriptions of Atom SHOULD prefer xsi:string to 
represent IDs. "

or, we just agree to remember that schema coming off this WG will 
xsi:string.

+1 to Paul's summary; good stuff.

cheers
Bill

---

From: Bill de hÓra <[email-masked]>
Date: Thu, 19 Aug 2004 12:23:57 +0100

I forgot:

> Ok, how about either,
> 
> "XML Schema descriptions of Atom SHOULD prefer xsi:string to represent 
> IDs. "

That's for assholes.

> or, we just agree to remember that schema coming off this WG will 

That's for morons.

If we do both, we're covered, right?

cheers
Bill

---

From: Anne van Kesteren <[email-masked]>
Date: Thu, 19 Aug 2004 13:30:23 +0200

> If we do both, we're covered, right?

We might need something for flamers[1]?

I don't really understand why it should be a xsi:string, since it really 
is a URI, not?

[1]<http://www.tbray.org/ongoing/When/200x/2004/08/18/MandA>

-- 
  Anne van Kesteren
  <http://annevankesteren.nl/>

---

From: Bill de hÓra <[email-masked]>
Date: Thu, 19 Aug 2004 12:45:00 +0100

> We might need something for flamers[1]?

Nah... that's just Tim not stamping out creativity. [1]

> I don't really understand why it should be a xsi:string, since it really 

It is, but we can't expect to get consistent equivalence calls 
across languages by the looks of things; strcmp seems to be the way 
to go for most implementors. I think Mark is thinking about cases 
where anyURI will be auto-marshalled onto a URI object rather than a 
String.

cheers
Bill

[1] http://www.tbray.org/ongoing/When/200x/2003/06/27/NoInventions

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 14:07:08 +0200

> I don't really understand why it should be a xsi:string, since it really  

The problem with having it an URI in programming languages, is that they  
normalize and do a lot of other smart things with the URI string. That  
makes the «byte by byte» comparison useless, since whenever you've loaded  
a URI into e.g. the System.Uri type in .NET, it is potentially not the  
same as the original string anymore.

It's somewhat the same as with dates. If we used dates for comparison,  
'2004-01-01T01:00:00+01:00' would be the same date as  
'2003-12-31T23:00:00-01:00', but they would not be the same string,  
compared byte by byte. (Not sure if this was a good example, but whatever  
:)

---

From: Tim Bray <[email-masked]>
Date: Thu, 19 Aug 2004 06:43:24 -0700

On Aug 19, 2004, at 3:55 AM, Bill de hÓra wrote:

>> in an Implementor's Guide or some such.
>
> Sure it's specific, but's a real point of breakage.

In my experience, RFCs which describe protocols rarely include 
discussion of the appropriate C or Java data types to hold things like 
IPv* addresses or DNS names. Now, you could argue that Atom is 
different from other protocols, or you could argue that schemas are 
different from programming language data types.  But it still feels a 
little weird to find this kind of implementation recommendation in the 
protocol description.  So...

> or, we just agree to remember that schema coming off this WG will 

I like that -Tim

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 19 Aug 2004 07:15:52 -0700 (PDT)

> that atom:id should not be relative (even though

Relative URIs not being URIs is a new stance that is
showing up on discussions about this issue on other
mailing lists. As far as I'm aware relative URIs have
been considered URIs for as long as they have existed. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #25
No matter how well it would perform, I will never construct any sort of machinery which is completely indestructible except for one small and virtually inaccessible vulnerable spot.

_______________________________
Do you Yahoo!?
Win 1 of 4,000 free domain names from Yahoo! Enter now.
http://promotions.yahoo.com/goldrush

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 19 Aug 2004 07:28:37 -0700 (PDT)

> Does this match what people feel was the general

Yes, except for the stuff about IDs not being relative
URIs being missing. 

=====

---

From: Sam Ruby <[email-masked]>
Date: Thu, 19 Aug 2004 11:15:08 -0400

http://www.intertwingly.net/wiki/pie/PaceIdConstruct

During the id discussion moratorium, I worked on a Pace that attempted 
to capture Paul's rendition of consensus as spec text.  It contains two 
notable additions:

    atom:id MUST NOT be a relative URI
    atom:head elements MUST contain an atom:id element

The former seems to be the consensus of this mailing list.  The latter 
simplifies the definition of atom:origin.  Either could be split off 
into a separate pace.

- Sam Ruby

---

From: Walter Underwood <[email-masked]>
Date: Thu, 19 Aug 2004 08:53:44 -0700

> some reason for it beyond gateways that accidentally canonicalize not

1) Consistency. it is good for other URIs in Atom to be canonical,
so why are IDs different? Inconsistency makes implementation more
error-prone.

2) Integration into multi-protocol systems. Atom will be used in
systems with information pulled from RSS, web spidering, doucument
databases, and those clients probably already canonicalize all URIs.
Why is it correct for all other URIs and forbidden for these? Another
source of error. It is hard enough merging data from HTML, MS Word,
and PDF without odd rules like this.

3) It simplifies the test cases. The requirement for no-change
copying requires inputs and outputs to test. A requirement for
canonical URIs does not. Test the output for canonical form.

4) It seems really weird to me that we require URIs, then compare
them like strings, then don't require the single step that makes that
comparison reliable under transformations. It sets off some sort of
design alarm in my head. Require apples and handle them like oranges.

If Atom is successful, it will be implemented by lots of people who
are not paying enough attention. Simple, consistent specs make the
protocol more likely to work in the wild.

I guess it is the robustness principle. To be conservative in what
we generate, use canonical URIs.

wunder
--
Walter Underwood
Principal Architect, Verity

---

From: Robert Sayre <[email-masked]>
Date: Thu, 19 Aug 2004 12:13:01 -0400

> http://www.intertwingly.net/wiki/pie/PaceIdConstruct

"An Identification construct is an element whose content identifies the 
parent element of the construct. If nothing else is stated for a given 
Identification construct, the following rules apply:"

It seems that Identification Constructs are intentionally being left 
open to extension, and the bulk of its semantics are optional. Why?

Robert Sayre

---

From: Anne van Kesteren <[email-masked]>
Date: Thu, 19 Aug 2004 18:23:22 +0200

> The problem with having it an URI in programming languages, is that 
> they  normalize and do a lot of other smart things with the URI string. 
> That  makes the «byte by byte» comparison useless, since whenever you've 
> loaded  a URI into e.g. the System.Uri type in .NET, it is potentially 
> not the  same as the original string anymore.
> 
> compared byte by byte. (Not sure if this was a good example, but 
> whatever  :)

Yeah, I understand it. But isn't this a limitation of XML Schema? You 
would expect that you could somehow declare that it is a URI (it is and 
would look semantically more appropriate), but you may not 
normalize/resolve it.

---

From: Tim Bray <[email-masked]>
Date: Thu, 19 Aug 2004 09:23:07 -0700

On Aug 19, 2004, at 8:15 AM, Sam Ruby wrote:

> to capture Paul's rendition of consensus as spec text.  It contains 
> two notable additions:
>

My judgment, based on what I've seen in the list, is that everything 
here has good (not "rough") consensus except for the final item: 
"atom:head MUST contain atom:id".  I agree about atom:origin, but as 
far as I can recall this is the first time this issue's really been 
brought to our attention.

So, in the spirit of grabbing consensus while we can and moving on, I 
think our editors should grab the text (except for the 
atom:head/atom:id bit) and plow it into their working-copy drafts 
forthwith.  -Tim

---

From: Sam Ruby <[email-masked]>
Date: Thu, 19 Aug 2004 12:38:04 -0400

> open to extension, and the bulk of its semantics are optional. Why?

That portion was lifted from PaceRecommendIdScheme.

I see no reason to explicitly call out the possibility of extension. 
Care to suggest an alternate wording?

I disagree that the "bulk of its semantics are optional".  I see the key 
semantics as being:

   * It MUST be universally unique, which means that it must be unique
     both at the time of creation and in the future.

   * It MUST NOT change over time, even if the parent feed or entry
     element is relocated, migrated, syndicated, republished, exported or
     imported.

- Sam Ruby

---

From: Dave Pawson <[email-masked]>
Date: Thu, 19 Aug 2004 18:06:15 +0100

On Thu, 2004-08-19 at 07:54, Tim Bray wrote:

>  I think we have agreement that 
> normative),

I don't recall any such agreement recently Tim? Was that some time ago?
I'm -1 on that.
If an atom instance can't be validated against a schema,
I for one will lose any interest in using it.

-- 
Regards DaveP.
XSLT&Docbook  FAQ
http://www.dpawson.co.uk/xsl

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 19 Aug 2004 10:02:28 -0700 (PDT)

> would look semantically more appropriate), but you

So there should be some way to declare a type as being
a URI but specifying that it shouldn't be treated like
URI? I'm not even sure what that means. Should there
also be a means for claiming that something is an
integer but specifying that you shouldn't use it for
arithmetic operations? 

=====

---

From: Robert Sayre <[email-masked]>
Date: Thu, 19 Aug 2004 13:15:05 -0400

Sam Ruby wrote:

> 
>>
>> "An Identification construct is an element whose content identifies 
>> the parent element of the construct. If nothing else is stated for a 
>> given Identification construct, the following rules apply:"

> Care to suggest an alternate wording?

An Identification construct is an element whose content identifies the 
parent element of the construct.  Its content MUST be a URI, subject to 
the following rules:

     * It conveys a permanent, universally unique identifier for its 
parent element
     * It MUST be universally unique, which means that it must be unique 
both at the time of creation and in the future.
     * It MUST NOT change over time, even if the parent feed or entry 
element is relocated, migrated, syndicated, republished, exported or 
imported.
     * atom:id MUST NOT be a relative URI (see "absoluteURI" in 
RFC-[RFC]2369, section 3)

> semantics as being...

They are only optional in the presence of the qualifier "If nothing else 
is stated...", so my wording removes that.

Robert Sayre

---

From: Anne van Kesteren <[email-masked]>
Date: Thu, 19 Aug 2004 19:17:03 +0200

>> Yeah, I understand it. But isn't this a limitation of XML Schema?
>> You would expect that you could somehow declare that it is a URI
>> (it is and would look semantically more appropriate), but you may
>> not normalize/resolve it.
> 
> So there should be some way to declare a type as being a URI but
> specifying that it shouldn't be treated like URI? I'm not even sure
> what that means. Should there also be a means for claiming that
> something is an integer but specifying that you shouldn't use it for 
> arithmetic operations?

You explain my problems with xs:string. Thanks. If it can't be described 
with xs:anyURI, why does ID have to be an URI?

---

From: Sam Ruby <[email-masked]>
Date: Thu, 19 Aug 2004 13:30:20 -0400

> RFC-[RFC]2369, section 3)

Thanks!  Committed.

- Sam Ruby

---

From: Dare Obasanjo <[email-masked]>
Date: Thu, 19 Aug 2004 10:35:22 -0700 (PDT)

> with xs:anyURI, why does ID have to be an URI?

IDs have to be globally unique. Many feel that URIs
are a cheap way for people to come up with globally
unique identifiers. The Web architecture is about
using URIs as identifiers (see RDF & XML namespaces). 

=====

		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 20:13:43 +0200

> http://www.intertwingly.net/wiki/pie/PaceIdConstruct

Ah, brilliant. +1. If there is lots of objections to the «MUST contain»  
wording in 4.2.6, I will have no problems with SHOULD.

---

From: Andreas Sewe <[email-masked]>
Date: Thu, 19 Aug 2004 20:13:31 +0200

Dare Obasanjo wrote:
 > Anne van Kesteren <[email-masked]> wrote:

Well, what should be possible is to declare that it is a string which 
has to _look_ like an URI.

> something is an integer but specifying that you shouldn't use it for 

So you might want to consider restricting xsd:string with the pattern 
facet, i.e. a probably long and ugly regex, such that the restricted 
type only allows strings respecting the URI syntax. And being a mere 
subtype of xsd:string it is quite clear that comparison has to be done 
character-by-character. Would this solve the problem?

Regards,

Andreas Sewe

---

From: Anne van Kesteren <[email-masked]>
Date: Thu, 19 Aug 2004 20:18:10 +0200

> http://www.intertwingly.net/wiki/pie/PaceIdConstruct

 From that page:

# http://www.example.org/rosé

That is not a valid URI. However, it is a valid IRI which will become a 
recommendation soon[1].

Is Atom going to do anything with IRIs?

[1]<http://www1.ietf.org/mail-archive/web/ietf-announce/current/msg00383.html>

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 20:51:57 +0200

On Thu, 19 Aug 2004 07:15:52 -0700 (PDT), Dare Obasanjo <[email-masked]>  
wrote:

> Relative URIs not being URIs is a new stance that is showing
> up on discussions about this issue on other mailing lists.

Okay. I haven't read that anywhere until just now.

> As far as I'm aware relative URIs have been considered URIs
> for as long as they have existed.

Yes, but I can both understand and agree with the notion of them not being  
so. Either way, I think the Atom specification needs to mention this fact,  
such as PaceIdConstruct[1] does perfectly: «atom:id MUST NOT be a relative  
URI».

____
[1] <url: http://www.intertwingly.net/wiki/pie/PaceIdConstruct>

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 20:55:35 +0200

> That is not a valid URI. However, it is a valid IRI which will become a  

I think that since IRI's are going to be a recommendation shortly, the  
pace should just replace all occurences of «URI» with «IRI». Then, the  
example will be correct as well.

The fact that atom:id is not going to be dereferencable makes it even  
easier to deploy IRI's, since they're only going to be stored as UTF-8  
somewhere, and not be used in some transport protocol layer to dereference  
resources.

---

From: Sam Ruby <[email-masked]>
Date: Thu, 19 Aug 2004 15:06:32 -0400

> wording in 4.2.6, I will have no problems with SHOULD.

If it is a SHOULD, then atom:origin can't depend on it being there. 
This was identified as an issue in the original discussion of 
atom:origin... what currently is spec'ed is not only surprising to some, 
but actually is broken.  See:

http://www.imc.org/atom-syntax/mail-archive/msg06269.html
http://www.imc.org/atom-syntax/mail-archive/msg07408.html
http://www.imc.org/atom-syntax/mail-archive/msg07432.html
http://www.imc.org/atom-syntax/mail-archive/msg07438.html

- Sam Ruby

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Thu, 19 Aug 2004 12:13:39 -0700

At 11:55 AM +0100 8/19/04, Bill de hÓra wrote:
>Ok, how about either,
>
>"XML Schema descriptions of Atom SHOULD prefer xsi:string to represent IDs. "

-1

>or, we just agree to remember that schema coming off this WG will xsi:string.

+1

If we have an implementer's guide, that guide should have a section 
on "implementing schemas for Atom", and "in schemas, IDs act like 
strings, not URIs" should be there.

--Paul Hoffman, Director

---

From: Sam Ruby <[email-masked]>
Date: Thu, 19 Aug 2004 15:23:45 -0400

Sam Ruby wrote:
>> Ah, brilliant. +1. If there is lots of objections to the «MUST 
>> contain»  wording in 4.2.6, I will have no problems with SHOULD.
> 
> If it is a SHOULD, then atom:origin can't depend on it being there. This 
> was identified as an issue in the original discussion of atom:origin... 
> what currently is spec'ed is not only surprising to some, but actually 
> is broken.  See:
> 

Actually, I should include the following one in the list, as it is the 
clearest indication of what should be done:

http://www.imc.org/atom-syntax/mail-archive/msg07413.html

- Sam Ruby

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Thu, 19 Aug 2004 13:26:01 -0700

At 8:18 PM +0200 8/19/04, Anne van Kesteren wrote:
>That is not a valid URI. However, it is a valid IRI which will 
>become a recommendation soon[1].

Saying "will become" in a context like this is premature. There are 
many hurdles between "IETF last call" and "accepted as a standard".

>Is Atom going to do anything with IRIs?

If they are accepted as a standard, this WG should certainly look at them.

At 8:55 PM +0200 8/19/04, Asbjørn Ulsberg wrote:
>I think that since IRI's are going to be a recommendation shortly, 
>the pace should just replace all occurences of «URI» with «IRI». 
>Then, the example will be correct as well.

If we follow what you suggest, and IRIs do not become a standard, 
then neither will Atom. I don't think that's what people here want.

--Paul Hoffman, Director

---

From: Tim Bray <[email-masked]>
Date: Thu, 19 Aug 2004 13:49:04 -0700

> atom:head/atom:id bit) and plow it into their working-copy drafts 

Oops, sorry, I'm getting pushback saying "you're going too fast."  So, 
while I do believe that Sam's draft language matches the consensus I've 
inferred from the last few weeks of email on this subject, I suppose we 
should hold on for a bit to give other people a chance to object if 
I've missed something (note one friendly amendment from Rob Sayre 
already).  -Tim

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 23:09:52 +0200

> If it is a SHOULD, then atom:origin can't depend on it being there.

I agree, so I'm +1 on it, unless everyone else is -1. I just don't want to  
halt consensus on it, that's all.

---

From: Sam Ruby <[email-masked]>
Date: Thu, 19 Aug 2004 17:17:08 -0400

Tim Bray wrote:
> should hold on for a bit to give other people a chance to object if I've 
> missed something (note one friendly amendment from Rob Sayre already).  

I also made a change (removed an invalid example) based on feedback from 
Anne van Kesteren [1].

At a minimum, atom:origin is very, very closely related to this issue as 
it involves the comparison of URIs; furthermore it currently is broken 
[2].  If we have time, we can discuss it now.  Otherwise, we can 
schedule it for later.

- Sam Ruby

[1] http://www.imc.org/atom-syntax/mail-archive/msg08628.html
[2] http://www.imc.org/atom-syntax/mail-archive/msg08640.html

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 19 Aug 2004 23:28:33 +0200

On Thu, 19 Aug 2004 13:26:01 -0700, Paul Hoffman / IMC <[email-masked]>  
wrote:

> If we follow what you suggest, and IRIs do not become a standard, then  
> neither will Atom. I don't think that's what people here want.

Okay, then let's just wait. I'm pretty confident IRIs will become standard  
before Atom, so whenever they are, Atom can reference them. Right?

In the meantime, the example with 'rosé' in a URI should probably be  
removed, since that is an invalid URI.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 19 Aug 2004 15:06:09 -0700

My intent was not that these words would show up in the spec; rather, 
that we'd just remember them when writing the schema.

Cheers,

On Aug 18, 2004, at 11:54 PM, Tim Bray wrote:

> in an Implementor's Guide or some such.  I think we have agreement 
> that we should provide schemas to help out (although not that they be

---

From: Bill de hÓra <[email-masked]>
Date: Thu, 19 Aug 2004 23:42:39 +0100

Dare Obasanjo wrote:
> 
> I'm not even sure what that means. Should there

Perhaps only in the presence of inconsistent or broken 
implementations of +.

cheers
Bill

---

From: Anne van Kesteren <[email-masked]>
Date: Fri, 20 Aug 2004 09:47:47 +0200

+1

+1

+1

> - IDs MUST be compared character-by-character, no case-mapping, no %xx 

-1. If they are URIs, why can't we treat them as such?

+1

---

From: Anne van Kesteren <[email-masked]>
Date: Fri, 20 Aug 2004 09:52:19 +0200

> http://www.intertwingly.net/wiki/pie/PaceIdConstruct

# URI references identifying entries and feeds are compared when
# determining whether a entry or feed is the same as one seen before.
# [Definition: The two URIs are treated as strings, and they are
# identical if and only if the strings are identical, that is, if they
# are the same sequence of characters. ] The comparison is
# case-sensitive, and no %-escaping is done or undone.
#
# A consequence of this is that URI references which are not identical
# in this sense may resolve to the same resource. Examples include URI
# references which differ only in case or %-escaping. Note that
# relative URIs are not allowed as ids. Replacement of XML character
# and entity references must be done before any comparison.

I don't see why we have to treat URIs as strings. Normalizing them as 
Mark Pilgrim explained[1] seems like a more sensible and correct thing 
to do. It just doesn't make sense to treat the following two example is 
being different:

  http://example.org/
  http://EXAMLE.org/

---

From: Anne van Kesteren <[email-masked]>
Date: Fri, 20 Aug 2004 10:01:07 +0200

> I also made a change (removed an invalid example) based on feedback 
> from Anne van Kesteren [1].

I still see that invalid URI. Actually, I see a lot of invalid URIs:

# If the entity eacute has been defined to be é, the atom:id elements
# below all contain the same URI reference, http://example.org/rosé.
#
# * <atom:id>http://example.org/rosé</atom:id>
# * <atom:id>http://example.org/ros&#xe9;</atom:id>
# * <atom:id>http://example.org/ros&#xE9;</atom:id>
# * <atom:id>http://example.org/ros&#233;</atom:id>
# * <atom:id>http://example.org/ros&eacute;</atom:id>

After the XML parser has done it's job, these are all invalid URIs, but 
valid IRIs.

I suggest that you remove that set of examples.

---

From: Anne van Kesteren <[email-masked]>
Date: Fri, 20 Aug 2004 10:00:41 +0200

> to do. It just doesn't make sense to treat the following two example is 

[1]<http://www.xml.com/pub/a/2004/08/18/pilgrim.html>

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 10:04:22 +0200

+1

> -1. If they are URIs, why can't we treat them as such?

Define "treat them as such". There are many ways to compare URIs, some 
of which require knowledge of the underlying scheme. What kind of 
comparison would you suggest instead of char-by-char?

Julian

---

From: Anne van Kesteren <[email-masked]>
Date: Fri, 20 Aug 2004 10:11:48 +0200

> comparison would you suggest instead of char-by-char?

The one which requires knowledge of the underlying scheme. Since in that 
case you are actually comparing URIs, not just the bytes that created 
the URI in which case you have used a simple string instead.

Mark Pilgrim does a better job than me:

  <http://www.xml.com/pub/a/2004/08/18/pilgrim.html>

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 10:19:23 +0200

>  <http://www.xml.com/pub/a/2004/08/18/pilgrim.html>

So, unless Atom requires one or several specific schemes, how is a 
generic recipient supposed to do that comparison?

BTW: I disagree with Mark's suggestion (or with the statement that there 
is consensus for) requiring publishers to canonicalize. As far as I can 
tell, there's still an ongoing debate over it.

As a matter of fact, XML-NS is using URIs for exactly the same purpose 
(identification and disambiguation), and guess what comparison method 
has been selected?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 20 Aug 2004 01:44:07 -0700

Hi Sam,

Looks good overall. A few questions and editorial notes;

> • It conveys a permanent, universally unique identifier for its parent 
> element

This is redundant, considering the text above.

> •  It MUST be universally unique, which means that it must be unique 
> •  It MUST NOT change over time, even if the parent feed or entry 

Is this a closed list, or are these examples? It may be good to make it 
something like "...is changed (e.g., ...)"

> •  atom:id MUST NOT be a relative URI (see "absoluteURI" in RFC-2369, 
> section 3)

RFC2396?

>  The URI scheme of an Identification construct MAY be a dereferencable 
> URL scheme (like HTTP), but MUST NOT be expected to be one. That means 
> that atom:id MUST NOT be expected to be dereferencable; it is just an 
> identifier.
>
>  If the identified resource is served dynamically, the content of an 
> Identification construct MUST be created only once and then stored 
> along with the resource. The content of an Identification construct 
> MUST NOT be created dynamically.

"Dynamic" isn't a good term to use in conjunction with the Web; it's 
not precisely defined and can be very misleading. Beyond that, I'm 
having difficulty parsing this; can you explain?

>  URI references identifying entries and feeds are compared when 
> determining whether a entry or feed is the same as one seen before. 
> [Definition: The two URIs are treated as strings, and they are 
> identical if and only if the strings are identical, that is, if they 
> are the same sequence of characters. ] The comparison is 
> case-sensitive, and no %-escaping is done or undone.
>
>  A consequence of this is that URI references which are not identical 
> in this sense may resolve to the same resource. Examples include URI 
> references which differ only in case or %-escaping. Note that relative 
> URIs are not allowed as ids. Replacement of XML character and entity 
> references must be done before any comparison.

It's likely I'll need to rewrite this for style, but I think the 
content is sound. Regarding the examples, I'm not crazy about putting 
so many in the middle of the draft, especially as a list; they'll tend 
to get in the way of the other content. A few possible ways around this 
(not mutually exclusive):
   - incorporate a variety of id's into the main example(s)
   - collapse them into prose rather than a list
   - add an appendix
   - put them in an implementer's guide
Thoughts?

>  As defined in "3.5 Identification Construct", atom:id, MUST NOT 
> change over time, even if other representations of the entry (such as 
> a web representation pointed to by the entry's atom:link element) are 
> relocated. For a given entry, the atom:id element's content MUST be 
> stable across all Atom documents published by the same entity. This 
> means that if the entry is represented in many different feeds 
> simultaneously, the atom:id of these entries MUST be the same. Also, 
> if the entry is relocated, migrated, syndicated, republished, exported 
> or imported, the atom:id MUST NOT change.

I'm OK with reminding people of the implications of this, but repeating 
the text wholesale doesn't add a lot of value, and restating 
requirements isn't good practice.

Cheers,

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 20 Aug 2004 01:55:35 -0700 (PDT)

--- Anne van Kesteren <[email-masked]> wrote:
> The one which requires knowledge of the underlying
> scheme. 

Interesting. So you expect every Atom client to know
the comparison rules of every possible URI scheme?
Heck, as Sam pointed out in
http://www.intertwingly.net/blog/2004/07/31/URI-Equivalence
Java & C# don't even agree on how to compare HTTP URIs
let alone arbitrary ones from exotic schemes. Allowing
anything besides char-by-char comparison is a recipe
for poor interoperability. 

The XML namespaces recommendation specifies character
by character comparison of namespace URIs for a
reason. 

=====

---

From: grayrest <[email-masked]>
Date: Fri, 20 Aug 2004 05:39:16 -0400

> having difficulty parsing this; can you explain?

Explanation by example: 

As a feed implementor for the popular http://example.com, I'm storing
Atom entries in my relational database. It would "waste disk space" to
store URIs, so when a feed is requested, I generate the required id by
taking a url (http://example.com/atom/examplefeed/) and sticking the
primary key for each entry onto the end
(http://example.com/atomid/examplefeed/132). This is fine, as I
control example.com and the PK for entries is guaranteed by my
database to be unique.

This is fine until the boss decides we need to be running Oracle
rather than SQLite. Because I'm lazy (ah, the programmer's virtue) and
I've already written an implementation, I'm going to do as little work
as possible. I'm certainly not going to re-read the spec and note that
I'll have to make sure my id stays the same and it's been a couple
years since I originally wrote the code. I'll just dump the database
(without PK, as Oracle can generate those) and change my database
connection string. Suddenly the UID for the feed changes, since the PK
for the new database is different. Chaos ensues and whatnot.

The example above is fairly contrived, but I can imagine a number of
similar examples (e.g. my company gets bought, we change our domain)
where the original author intended a value to be invariant but failed
to note it and someone else changes it. The wording is intended to
prevent programming errors of this sort by prodding developers to
waste storage space by storing the original generated id.

---

From: grayrest <[email-masked]>
Date: Fri, 20 Aug 2004 05:38:52 -0400

Explanation by example:

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 20 Aug 2004 10:47:24 +0100

>  <http://www.xml.com/pub/a/2004/08/18/pilgrim.html>

Here's a distinction that might be useful. For Atom the requirement 
is not that we have URIs for IDs, but that we have high quality IDs 
that are reasonably cheap to mint in a decentralized way. Specifying 
URIs is a means to that end. More precisely we're telling people we 
want them to reuse URI generators as ID generators.

So what we're saying in the document is something like this:

"Creation: You must use URI generators to *create* these keys - 
we're insisting you do that because URI generators are the cheapest 
way we know to mint stable IDs in a fairly simple and fairly 
decentralised way.

Comparison: Because IDs are just keys and because URI comparisons 
are scheme specific you must *compare* character by character - 
we're insisting you do that since we think it will be the most 
effective approach for you to make good matches against the keys."

cheers
Bill

cheers
bill

---

From: Sam Ruby <[email-masked]>
Date: Fri, 20 Aug 2004 07:15:36 -0400

> I suggest that you remove that set of examples.

Done.

- Sam Ruby

---

From: Sam Ruby <[email-masked]>
Date: Fri, 20 Aug 2004 07:32:31 -0400

> This is redundant, considering the text above.

OK, changed to:

     An Identification construct is an element whose content conveys a
     permanent, universally unique identifier the parent element of the
     construct.  Its content MUST be a URI, subject to the following

> something like "...is changed (e.g., ...)"

It is not a closed list, but I think the insertion of the word "changed" 
would be confusing... as the list is really realocated (with or without 
change), migrated (with or without change)...

>> •  atom:id MUST NOT be a relative URI (see "absoluteURI" in RFC-2369, 

2369bis and IRI are not yet approved.

> It's likely I'll need to rewrite this for style, but I think the content 
> is sound. Regarding the examples, I'm not crazy about putting so many in 
> the middle of the draft, especially as a list; they'll tend to get in 
> the way of the other content. A few possible ways around this (not 
> mutually exclusive):

Overall, I personally think the current The Atom Syndication Format is 
lacking in examples.  These examples were lifted directly from the 
Namespaces in XML 1.1 document.  Clearly, this is an area where there is 
signficant confusion.

- Sam Ruby

---

From: Mark Pilgrim <[email-masked]>
Date: Fri, 20 Aug 2004 08:09:03 -0400

> tell, there's still an ongoing debate over it.

Please note that that article was correct when it was submitted, a few
hours after this on-list announcement:

http://www.imc.org/atom-syntax/mail-archive/msg08317.html

Since then, it seems that the consensus has been ignored by a handful
of people who apparently couldn't be bothered to keep up with the list
during the weeks -- weeks! -- that we were discussing the issue, nor
could they be bothered to show up for a vote when a vote was called by
the working group chair.

Decisions are made by those who show up.  As far as I'm concerned,
this issue was settled on August 5, by the overwhelming margin of 11.8
to 0 to -7.  atom:id is a canonical URI of no particular scheme. 
Let's move on.

-- 
Cheers,
-Mark

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 14:33:38 +0200

> could they be bothered to show up for a vote when a vote was called by

Believe it or not, things like this happen. For instance, because people 
are on vacation. As far as I can tell, that survey lasted exactly one 
day (the version history for the pace indicates a creation date of 
August 5!), so how can it claim to reflect the WG's consensus in any way?

I remember multiple mails that have been asking for the rational for the 
"canonical" requirement; and as far as I can tell, I haven't seen a 
convincing explanation. In case I missed it please point me to it.

> to 0 to -7.  atom:id is a canonical URI of no particular scheme. 

Sorry, "rough consensus" doesn't mean having a quick vote and then 
telling people who raise objections to be silent. A survey is a *tool* 
that is supposed to detect consensus. If, afterwards, it turns out that 
there is indeed no consensus of all parts, then you should respect that.

So again: what's the motivation for "canonical"? People are supposed to 
store the identifiers faithfully (as strings) and compare them 
char-by-char, and that should be it.

Best regards, Julian

---

From: Mark Pilgrim <[email-masked]>
Date: Fri, 20 Aug 2004 09:02:55 -0400

> convincing explanation. In case I missed it please point me to it.

http://www.xml.com/pub/a/2004/08/18/pilgrim.html

> telling people who raise objections to be silent.

The quick vote exposed the fact that we already had overwhelming
consensus.  Which we still do, I might add; two weeks later, the wiki
vote stands at 11.8 to 4 to -11.  How long were you planning on using
the vacation excuse?

> A survey is a *tool* that is supposed to detect consensus.

And it did a bang-up job.

-- 
Cheers,
-Mark

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 15:19:19 +0200

> vote stands at 11.8 to 4 to -11.  How long were you planning on using

I'm not sure how relevant that Wiki is right now after results had 
already been posted. Just to make sure, I just added my comment there es 
well.

> And it did a bang-up job.

So how about getting back to the question I asked: "what's the rational 
for requiring the canonical form?". I assume you can answer that if 
you're so convinced about it...

Best regards, Julian

---

From: Robert Sayre <[email-masked]>
Date: Fri, 20 Aug 2004 09:31:07 -0400

Mark Pilgrim wrote:

 > consensus.  Which we still do

The beginning of this thread makes canonicalization a "suggestion".
Julian and Mark: do you agree that Paul's email captures consensus?

Robert Sayre

Paul Hoffman / IMC wrote:
> changed, it is a good idea to create them in a canonicalized form. There 
> is no interoperability issues with this suggestion, so it is just a 
> suggestion, not a SHOULD-level suggestion.
>

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 15:43:55 +0200

> Julian and Mark: do you agree that Paul's email captures consensus?

I agree with the summary, but I still don't understand how

"- Because IDs might be dereferenced, and because they must not be 
changed, it is a good idea to create them in a canonicalized form. There 
is no interoperability issues with this suggestion, so it is just a 
suggestion, not a SHOULD-level suggestion."

is supposed to help. As long as IDs are stored as-is, and they are 
always compared char-by-char, it doesn't make any difference whatsoever 
whether they are canonical or not. So at least the rational for that 
suggestion needs to be enhanced.

(yes, I likecanonicalization as well, but if spec says the same thing if 
we omit that suggestion, we should do that).

Best regards, Julian

---

From: Walter Underwood <[email-masked]>
Date: Fri, 20 Aug 2004 08:42:30 -0700

> (identification and disambiguation), and guess what comparison method

This is not the same as Atom. In XML-NS, this does not introduce
two ways to handle URIs, because XML-NS does not specify linking.

In Atom, this does introduce two ways to handle URIs, one way for
URIs as IDs and one way for URIs as hrefs.

As I said yesterday, we require apples (URIs), then require that they
are handled as oranges (strings).

wunder

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 20 Aug 2004 08:48:30 -0700

On Aug 20, 2004, at 4:32 AM, Sam Ruby wrote:
>> Is this a closed list, or are these examples? It may be good to make 
>> it something like "...is changed (e.g., ...)"
>
> It is not a closed list, but I think the insertion of the word 
> "changed" would be confusing... as the list is really realocated (with 
> or without change), migrated (with or without change)...

Right. I think the point you're getting at is that whatever an ID 
conceptually identifies doesn't change over any dimension, including 
time, just like a cool URI, and additionally, the identity isn't 
affected by changes in location.

The list you give is descriptive, which is good to communicate the gist 
of the requirement, but it doesn't cover everything, and therefore 
doesn't make a good requirement. How about "...is changed in content or 
location (e.g., ...)"?

>>> •  atom:id MUST NOT be a relative URI (see "absoluteURI" in 
>>> RFC-2369, section 3)
>> RFC2396?
>

RFC2369 is "The Use of URLs as Meta-Syntax for Core Mail List Commands 
and their Transport through Message Header Fields." I think you mean to 
refer to RFC2396, "Uniform Resource Identifiers (URI): Generic Syntax," 
no?

>> so many in the middle of the draft, especially as a list; they'll 
>> tend to get in the way of the other content. A few possible ways 
>> around this (not mutually exclusive):
> Namespaces in XML 1.1 document.  Clearly, this is an area where there 
> is signficant confusion.

I totally agree that we need more examples; these are just a bit 
verbose. There's a fine line between effective communication and 
restating what's already specified elsewhere.

Cheers,

---

From: Tim Bray <[email-masked]>
Date: Fri, 20 Aug 2004 09:18:37 -0700

On Aug 20, 2004, at 1:44 AM, Mark Nottingham wrote:

> to get in the way of the other content. A few possible ways around 
> this (not mutually exclusive):

I'm OK with edits for style (that's why we have editors), and Anne v.K. 
pointed out that some of the example URIs actually aren't, but on the 
other hand, I think it's better to err on the side of too many rather 
than too few examples (too many RFCs and W3C RECs go the other way).  
So please retain as many as possible. -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 20 Aug 2004 09:19:48 -0700

On Aug 20, 2004, at 2:38 AM, grayrest wrote:

> The example above is fairly contrived

On the evidence I've seen, it happens all the time -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 20 Aug 2004 09:34:38 -0700

That's helpful, thanks.

We need to tread very carefully here; this language tries to constrain 
implementation choices, and that's tricky business.

E.g., "stored along with the resource" -- what does that mean, 
precisely? Am I conformant if the ID is in the same database? The same 
table? The same sector of the disk?

WRT "dynamic," is this requirement violated if I can examine a resource 
and derive the ID from its data? Metadata?

Yes, this is a pedantic view, but that's the nature of standards; any 
wiggle room or ambiguity will be exploited and cause problems. OTOH, if 
we're too precise, we'll needlessly tie the hands of implementers; 
e.g., if someone understands and commits to the notion that ids have to 
be stable over time, why should their hands be tied with a MUST?

My gut feeling is that this is implementation advice, not a set of 
normative requirements.

---

From: Walter Underwood <[email-masked]>
Date: Fri, 20 Aug 2004 09:42:00 -0700

--On Friday, August 20, 2004 9:18 AM -0700 Tim Bray <[email-masked]> wrote:
>
> ... on the other hand, I think it's better to err on the side of too
> many rather than too few examples (too many RFCs and W3C RECs go the
> other way).  So please retain as many as possible. -Tim

+1

People have different learning styles. Some learn from rules, some
from examples. An effective spec needs full coverage in both styles.

wunder

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 19:19:25 +0200

> two ways to handle URIs, because XML-NS does not specify linking.

I don't understand that one. XML-NS uses globally unique identifiers 
using URI syntax. So does Atom. XML-NS namespace URIs may be 
referenceable. Same for Atom IDs.

Where's the difference?

> are handled as oranges (strings).

Because URIs are a very simple and cheap way to produce these strings. 
Can you suggest an alternative that has the same properties?

Julian

---

From: Walter Underwood <[email-masked]>
Date: Fri, 20 Aug 2004 10:32:17 -0700

--On Friday, August 20, 2004 7:19 PM +0200 Julian Reschke <[email-masked]> wrote:
> using URI syntax. So does Atom. XML-NS namespace URIs may be referenceable.
> Same for Atom IDs.
>
> Where's the difference?

Atom also uses URIs which are supposed to be followed (URLs).
XML-NS does not. That is the difference. This is not only about
IDs, this is about the Atom spec as a coherent whole. How easy
is Atom, not how easy are Atom IDs.

Also, the "URI is a string" behavior of XML-NS confuses people
every day. Maybe every hour, given how widespread XML is. XML-NS
might not be a good example of how to do URIs as IDs.

> Can you suggest an alternative that has the same properties?

No. That does not change the fact that Atom has two ways to
handle URIs and that the spec is more complicated because of that.

I'm not proposing non-URI IDs. I'm saying that a URI should be
treated like a URI whether it is an ID or not.

wunder

---

From: Julian Reschke <[email-masked]>
Date: Fri, 20 Aug 2004 20:06:22 +0200

Walter Underwood wrote:

> ...

OK,

so can you be a bit more specific about how we're not using the URI as a 
URI here?  Would you prefer a different way to compare two IDs? Which?

Julian

---

From: Walter Underwood <[email-masked]>
Date: Fri, 20 Aug 2004 12:23:49 -0700

> URI here?  Would you prefer a different way to compare two IDs? Which?

It is a structured string and we are ignoring the structure.
It like comparing two timestamps as strings. They might be
the same instant in different timezones, but string comparison
won't catch that.

wunder

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 20 Aug 2004 13:03:59 -0700 (PDT)

> the same instant in different timezones, but string

Instead of complaining why don't you suggest a
feasible alternative? 

=====

__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail

---

From: Walter Underwood <[email-masked]>
Date: Fri, 20 Aug 2004 15:11:11 -0700

I'll try this again, but referencing the wording directly.

> - The format for IDs is going to be URIs, not plain strings.

This is a change from Tim Bray's declaration of consensus on Aug 5.
That was for "canonical URIs", not "URIs".

There was one objection, on the grounds that there might be some
RDF out there with non-canonical URIs and that those might end up
in Atom feeds. We can address those by making URI a MUST and
canonical URI a SHOULD, explaining that the SHOULD is only for
legacy IDs.

We have talked about legacy data with LiveJournal dates, so making
exceptions for legacy IDs makes sense, but only if there really are
non-canonical URIs IDs in major implementations. If they are not common,
make canonical URI a MUST.

> - IDs MUST be compared character-by-character, no case-mapping, no %xx

This is safe for canonical URIs, though there is no need to
forbid fancier comparisons which also work on canonical URIs.
If canonical URIs are a SHOULD, then this is a MUST. If they
are a MUST, then this is implementation advice for fast compare,
but not a MUST for protocol correctness.

In short, we MUST have either canonical URIs or char-by-char
compares. The former is easier to test and probably more robust.
The latter is simple to implement and probably faster (no
canonicalization step).

Personally, I prefer robustness over premature optimization,
but I haven't seen consensus on that.

> - Receiving and gateway systems MUST NOT alter IDs in any way.

This makes sense for gateways. I don't see how it applies to
"receiving systems" whatever they are. Is there something that
gateways are supposed to alter? Shouldn't they leave entries
unaltered anyway?

Perhaps this is a requirement on how entries are stored when
they are not in protocol units? Maybe "IDs MUST NOT be altered
when stored, transmitted, or recieved".

wunder
--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: Julian Reschke <[email-masked]>
Date: Sat, 21 Aug 2004 10:29:02 +0200

> the same instant in different timezones, but string comparison

1) There's a problem with that comparison. For instance, for timestamps 
comforming to IS08601, there's a well-defined way to parse them and to 
compare them as dates. There simply isn't a similar comparison method 
for URLs, unless you restrict yourself to generic URI syntax.

2) Again: *Which* comparison method would you prefer over char-by-char?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Sat, 21 Aug 2004 10:34:08 +0200

> canonical URI a SHOULD, explaining that the SHOULD is only for

There were *several* objections. Mine was: it doesn't make any sense to 
require "canonical" unless that buys us something. What does it buy us?

> forbid fancier comparisons which also work on canonical URIs.

Can you expand on why this is "unsafe" for non-canonical URIs?

> but not a MUST for protocol correctness.

It is. Unless we forbid using multiples URIs that share the same 
canonical form, it is absolutely required that they are simply compared 
char-by-char.

> but I haven't seen consensus on that.

I prefer spec simplicity. The latter seems to be the simplest thing that 
the spec can say *and* that will work.

Best regards, Julian

---

From: Sam Ruby <[email-masked]>
Date: Sat, 21 Aug 2004 10:20:18 -0400

> require "canonical" unless that buys us something. What does it buy us?

Here's why I voted against the first column in IDSurvey:

http://www.imc.org/atom-syntax/mail-archive/msg08194.html
http://www.imc.org/atom-syntax/mail-archive/msg08229.html

- Sam Ruby

---

From: grayrest <[email-masked]>
Date: Sat, 21 Aug 2004 13:31:54 -0400

On Fri, 20 Aug 2004 09:18:37 -0700, Tim Bray <[email-masked]> wrote:
>  I think it's better to err on the side of too many rather
> So please retain as many as possible.

+1

At minimum, examples provide unit test cases that you don't have to
invent yourself.

Karl

---

From: Martin Duerst <[email-masked]>
Date: Sun, 22 Aug 2004 17:25:23 +0900

At 13:26 04/08/19 -0700, Paul Hoffman / IMC wrote:

>Saying "will become" in a context like this is premature. There are many 
>hurdles between "IETF last call" and "accepted as a standard".

I agree with Paul that the future is difficult to predict. However,
I think that 'accepted as a standard' isn't clear at all.
Who accepts? What kind of 'standard'? I think it's probably
better to say 'approved by the IESG as an IETF Proposed Standard'.
That's the next formal step after IETF Last Call, and is clearly
defined. Of course, both IRIs and Atom eventually hope to become
IETF (Full) Standard. But that will take quite some time in both
cases.

In addition, please note that you don't need to take a 'wait and see'
only attitude. If you think that it would be good for Atom to use IRIs,
it might be a good idea to make an IETF Last Call comment on the IRI spec
to say something like "I'd like IRIs to be used in Atom, so please
move this spec ahead".

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sun, 22 Aug 2004 17:37:05 +0900

>somewhere, and not be used in some transport protocol layer to dereference

Just a small point: There is no requirement to store IRIs as UTF-8.
IRIs are sequences of characters. In many cases they may be stored
in UTF-16, or something else.

Regards,    Martin.

---

From: Eric Scheid <[email-masked]>
Date: Mon, 23 Aug 2004 02:37:29 +1000

+1

If we don't even mention canonicalisation, not even as a suggestive SHOULD,
then we don't have to explain it in the spec. One less hurdle for the busy
developer.

Plus it weakens the argument that id comparisons must be done by string
comparison.

e.

---

From: Walter Underwood <[email-masked]>
Date: Sun, 22 Aug 2004 15:18:07 -0700

> for URLs, unless you restrict yourself to generic URI syntax.

2396bis, section 6.

  <http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#comparison>

> 2) Again: *Which* comparison method would you prefer over char-by-char?

I did not say I preferred another method. I said that the spec
does not need to restrict implementations to char-by-char if
canonical URIs are required.

Many libraries will provide an equals method for their URI objects,
and that is probably not char-by-char. Atom will require programmers
to avoid the obvious URI comparison routines and do something different.

Again, we require apples (URIs) and require that they be treated
as oranges (strings).

If we require URIs, we should require good URI practices.

wunder

---

From: Bill de hÓra <[email-masked]>
Date: Sun, 22 Aug 2004 23:34:40 +0100

> to avoid the obvious URI comparison routines and do something different.

I don't see the problem.

> Again, we require apples (URIs) and require that they be treated

Again, I don't see the problem:

Apples: key generation.
Oranges: key comparison.

cheers
Bill

---

From: Walter Underwood <[email-masked]>
Date: Sun, 22 Aug 2004 15:36:23 -0700

> If we require URIs, we should require good URI practices.

More accurately, we should not forbid good URI practices.

wunder

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 22 Aug 2004 16:01:02 -0700

Hi Martin,

Just curious -- what's the status of IRI support in implementations?

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Aug 2004 16:29:20 +0200

Sam Ruby wrote:
> 
> Julian Reschke wrote:
> 
>>
>> There were *several* objections. Mine was: it doesn't make any sense 
>> to require "canonical" unless that buys us something. What does it buy 
>> us?
> 
> 

I just reread those mails and fail to understand the issue.

If we state that atom IDs are strings, and are to be compared as 
strings, people who put them into URI objects (Tim's Java example) *and* 
use the objects' equality method simply make a programming mistake.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Aug 2004 16:35:13 +0200

>  <http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#comparison>

Yes. That chapter lists several comparison methods. Which one to use?

> canonical URIs are required.

That's incorrect. For instance,

	http://example.com

and

	http://example:com:80

will compare equal for some comparison methods although they are both in 
canonical form.

> to avoid the obvious URI comparison routines and do something different.

Yes.

> If we require URIs, we should require good URI practices.

Indeed. In fact, it's good practice to define which comparison to use 
because there are so many to choose from.  Char-By-Char comparison is 
one of the comparison methods that qualifies.

Best regards, Julian

---

From: Sam Ruby <[email-masked]>
Date: Mon, 23 Aug 2004 10:50:10 -0400

> will compare equal for some comparison methods although they are both in 

Assuming that the first colon in the second example is a typo, my 
reading of section 6.2.3 indicates that neither of the above URIs 
examples are expressed in the normal form for the "http" scheme.

- Sam Ruby

http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#normalize-scheme

---

From: Julian Reschke <[email-masked]>
Date: Mon, 23 Aug 2004 17:50:38 +0200

Sam Ruby wrote:

>> will compare equal for some comparison methods although they are both 
>> in canonical form.
> 
> 

Yes.

> examples are expressed in the normal form for the "http" scheme.

That's correct, but what does that have to do with the "canonical" form? 
And in case the spec would indeed require scheme-based normalization, 
how do you do that for a scheme you don't know?

Best regards, Julian

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Mon, 23 Aug 2004 22:52:50 +0200

> how do you do that for a scheme you don't know?

That's a good reason for requiring publishers to do the canonicalization  
and normalization (c14n11n :) and allowing consumers to compare atom:id  
char-by-char (as strings, not as URI's).

---

From: Julian Reschke <[email-masked]>
Date: Tue, 24 Aug 2004 08:15:41 +0200

Asbjørn Ulsberg wrote:
> 
> On Mon, 23 Aug 2004 17:50:38 +0200, Julian Reschke 
> That's a good reason for requiring publishers to do the 
> canonicalization  and normalization (c14n11n :) and allowing consumers 
> to compare atom:id  char-by-char (as strings, not as URI's).

No, that's a good reason to require just char-by-char comparison. This 
is all that's needed, and therefore this is all the spec should say 
about this topic.

Julian

---

From: Danny Ayers <[email-masked]>
Date: Tue, 24 Aug 2004 15:02:10 +0200

I think Paul's initial summary probably matches the consensus, and I
for one could live with id as described, so

+1.

I wonder perhaps whether it might be worth formally defining a
URIstring datatype in the Atom spec, as char-by-char comparison
doesn't match the definition of equivalence for URIs, hence it's a
different kind of thing.

The canonicalization idea sounds good, but does seem like the
introduction of something untried. Ok, so every HTTP server will
incorporate some canonicalization subsystem, but moving this low-level
stuff up a layer to Atom is a different story. So a SHOULD for it
sounds reasonable.

Cheers,
Danny.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Wed, 25 Aug 2004 21:42:03 +0200

On Tue, 24 Aug 2004 08:15:41 +0200, Julian Reschke <[email-masked]>  
wrote:

> No, that's a good reason to require just char-by-char comparison.

So the Atom spec shouldn't say anything about either normalization or  
canonicalization as something publishers SHOULD or MUST do with their  
atom:id URI's?

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Wed, 25 Aug 2004 22:13:03 +0200

On Fri, 20 Aug 2004 09:52:19 +0200, Anne van Kesteren  
<[email-masked]> wrote:

> to do.

I agree, but only if that burden can be put on the publisher. Consumers  
should be able to just do string comparison.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Wed, 25 Aug 2004 22:20:47 +0200

On Fri, 20 Aug 2004 07:32:31 -0400, Sam Ruby <[email-masked]>  
wrote:

> construct. [...]

Isn't it missing a word here? Shouldn't it be «...a permanent, universally  
unique identifier FOR the parent element...» or something?

> change), migrated (with or without change)...

Although I agree with Mark's comment, I agree with your assertions as  
well. It should try to include all kinds of re-allocations, changes etc.  
to the feed or entry -- even things we haven't thought of yet. Maybe we  
could say something like:

   It MUST NOT change over time, even if the parent feed or entry element
   is relocated, migrated, syndicated, republished, exported, imported
   or otherwise changed, copied or moved.

> lacking in examples.

I agree.

> Clearly, this is an area where there is signficant confusion.

Definately.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Aug 2004 22:22:41 +0200

Asbjørn Ulsberg wrote:
> 
> On Tue, 24 Aug 2004 08:15:41 +0200, Julian Reschke 
>>> canonicalization  and normalization (c14n11n :) and allowing 
>>> consumers  to compare atom:id  char-by-char (as strings, not as URI's).
>>
>>

Yes. Just state that they are plain strings using URI syntax 
(absoluteURI production in RFC2396) and that they are to be compared 
character-by-character. This is all that's needed for the desired 
syntax, and as far as I can tell, it's the simplest way to achieve it.

I'm not against canonicalization; but if the spec mentions it there 
should be a clear benefit in doing it. Saying "the IDs will work better 
when recipients break the spec when comparing them" IMHO is the wrong 
approach to spec writing.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Aug 2004 22:23:35 +0200

Asbjørn Ulsberg wrote:

>> Mark Pilgrim explained[1] seems like a more sensible and correct 
>> thing  to do.
> 
> 

If consumers compare them as strings anyway, there's no point in 
requiring canonicalization, right?

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Wed, 25 Aug 2004 22:32:05 +0200

On Fri, 20 Aug 2004 09:34:38 -0700, Mark Nottingham <[email-masked]> wrote:

> E.g., "stored along with the resource" -- what does that mean, precisely?

Good question, since the resource can be stored on whatever storage unit  
you'd like; from simple file systems to revisioned and high-tech content  
management systems RDBMS's.

> Am I conformant if the ID is in the same database?

I would say it needs to be stored in the same database row as the entry  
(if the entry is the «resource»). We could probably enumerate a couple of  
storage alternatives to get this clarified. What about adding the  
following text:

   If the resource is stored in a database, there should be a column
   representing atom:id that contains the full atom:id URI string.

   If the resource is stored in an XML file in the file system, there should
   be an XML element (or attribute) representing atom:id that contains the
   full atom:id URI string.

The wording here is a bit curved, but something along those lines would  
probably make it clearer what we want, no?

> and derive the ID from its data? Metadata?

Not if you only do it once. If you do it each time the resource is  
requested, you're violating the requirement. Maybe «dymanic» isn't the  
best word here. Something about «first time creation» should probably get  
into the text somehow.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 25 Aug 2004 22:31:26 +0200

> syntax, and as far as I can tell, it's the simplest way to achieve it.

Sorry:

s/syntax/semantics/

Julian

---

From: Mark Pilgrim <[email-masked]>
Date: Wed, 25 Aug 2004 17:14:47 -0400

> requiring canonicalization, right?

Some consumers will undoubtedly compare them as strings, since (I
believe) all consumers currently compare RSS's <guid> as strings. 
These consumers are wrong (but only sometimes, because <guid> is only
sometimes a URI), because comparing URIs is surprisingly difficult.

Some consumers will undoubtedly use their programming platform's fancy
URI class or equivalent, like this:

http://intertwingly.net/blog/2004/07/31/URI-Equivalence

We would like both of these classes of consumers to come up with the
same answer in all cases as to whether two atom:ids are the same.  I
believe that requiring canonical URIs for atom:id achieves that.  I
believe that no other solution achieves that without at least one part
of the process being counterintuitive and/or difficult for consumers.

You could...

1. Tell consumers who don't have fancy URI classes that they need to
perform a 10-step normalization process (which I detailed in my
article) to compare values WHOSE ONLY PURPOSE IN LIFE IS TO BE
COMPARED.  This is counterintuitive and difficult.

2. Tell consumers who have fancy URI classes that they can't use them,
despite clear language in the spec that atom:id MUST be a URI, and
despite the consumer being able to use their fancy URI class to do
other things with every other URI in Atom.  This is counterintuitive. 
This is part of the "principle of least surprise" I was talking about
in my article.

3. Tell consumers that they can compare atom:ids any way they like,
but they may come up with different answers about whether two atom:ids
are the same or different, and that that's somehow OK.  This is
counterintuitive and simply wrong (despite being advocated by our
working group chair).  I have made this case repeatedly, on-list and
off: if client A can make a reasonable case that two atom:ids are the
same, and client B can make a reasonable case that those two atom:ids
are different, then we have failed, and we may as well just pack it up
and go home.

-- 
Cheers,
-Mark

---

From: Mark Pilgrim <[email-masked]>
Date: Wed, 25 Aug 2004 17:28:45 -0400

On Wed, 25 Aug 2004 22:22:41 +0200, Julian Reschke
<[email-masked]> wrote:
> character-by-character.

What you are advocating is what I listed as alternate solution #2 in
my recent message: "atom:id MUST be a URI, except you can't use your
URI class to compare it... oh, and you can't do anything else with
atom:ids except compare them."  This is counterintuitive.

-- 
Cheers,
-Mark

---

From: Antone Roundy <[email-masked]>
Date: Wed, 25 Aug 2004 16:09:21 -0600

> This is part of the "principle of least surprise" I was talking about
Assuming we specify that ID's must be character-by-character 
comparable, we obviously won't have a problem with false negatives no 
matter what (non-ridiculous) method is used to compare, because even if 
that method mangles the IDs in unintended ways, it will mangle both 
sides of the comparison identically.

The issue is false positives, where two different IDs become the same 
after having been mangled.  Is this non-trivially more likely to occur 
than two people actually generating character-for-character identical 
IDs?

If my ID is http://www.geckotribe.com/blog/2004/8/25/1, it's clearly 
not going to get confused with someone else's 
http://www.foo.net:80/blog/2004/8/25/1 whether compared character by 
character or by any other method.  On the other hand, if one person is 
using the ID http://www.bloghost.com/2004/8/25/1 for their ID, and 
someone else is using http://www.bloghost.com:80/2004/8/25/1, then 
something is fundamentally wrong with the way they're generating their 
IDs.  They should be doing something like 
http://www.bloghost.com/blog-a/2004/8/25/1 and 
http://www.bloghost.com:80/blog-b/2004/8/25/1 anyway.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 00:15:49 +0200

Yes.

> This is part of the "principle of least surprise" I was talking about

Well, if the spec tells them that the ID is a *string* that has URI 
syntax, and that it needs to be compared as a string, I don't see a 
problem. As a matter of fact, this has *provably* worked for XML 
namespaces (show me one XML+NS processor that gets this wrong!), so why 
shouldn't it work for something else?

> are different, then we have failed, and we may as well just pack it up

No objection. Just stick with char-by-char comparison, and the spec will 
be as simple as possible, and there'll no way comparison results can vary.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 00:17:33 +0200

> atom:ids except compare them."  This is counterintuitive.

But that's the whole point of ids, right? What else except comparing for 
equality (or using them as a key, or doing other id-ish things...) do 
you want to do with them?

Julian

---

From: Dare Obasanjo <[email-masked]>
Date: Wed, 25 Aug 2004 15:29:48 -0700 (PDT)

> atom:ids except compare them."  This is

This is exactly how XML namespaces work. Are you
suggesting that what the W3C Namespaces in XML
recommendation specifies is wrong?[0]

[0] There is no right answer to this question. :) 

=====

---

From: Mark Pilgrim <[email-masked]>
Date: Wed, 25 Aug 2004 18:44:05 -0400

> you want to do with them?

I don't know how many ways I can repeat myself, so I give up.  Let's
do it Julian's way, and then spend the next ten years explaining to
people that using their platform's URI classes to compare URIs is
subtlely wrong.

-- 
Cheers,
-Mark

---

From: Eric Scheid <[email-masked]>
Date: Thu, 26 Aug 2004 09:02:05 +1000

> atom:ids except compare them."  This is counterintuitive.

No, he said they are *strings* that look[1] like URIs, not URIs that are to
be treated as strings. Different thing entirely.

e.

[1] "look like" = shorthand for "has special rules as to composition, rules
which just so happen to be the same as those for URI serialisation"

---

From: Graham Parks <[email-masked]>
Date: Wed, 25 Aug 2004 19:05:17 -0400

>people that using their platform's URI classes to compare URIs is

The problem is that there's more than one way to compare URIs, and not all URI classes do the same thing. For consistency's sake we need to specify one method, and we're specifying the simplest.

Canonicalization has been suggested as an alternative solution (right?). I see how that would work, but I much prefer the simplicity of the above solution. It is just a matter of personal preference, because from a technical standpoint either system can just as easily be messed up by a single broken implementation.

Graham

---

From: Martin Duerst <[email-masked]>
Date: Thu, 26 Aug 2004 08:27:53 +0900

>atom:ids except compare them."  This is counterintuitive.

It may sound counterintuitive, but the 'I' in 'URI' stands for
'Identifier'. For identifiers, comparision for equality is the
only thing that really counts. It answers the question: Is this
the one that I mean.

What else would you want to do with the URI in atom:id?
[Occasionally, you will be able to dereference it, but
in general, that won't work.]

Regards,   Martin.

---

From: Mark Pilgrim <[email-masked]>
Date: Wed, 25 Aug 2004 20:58:33 -0400

> only thing that really counts. It answers the question: Is this

I understand that their only purpose is comparison.  The
counterintuitive part is that they look exactly like URIs, but you
can't use your programming language's URI class to compare them.

I guess Julian doesn't use languages like Java and C#, but I hear
they're catching on.

-- 
Cheers,
-Mark

---

From: Graham <[email-masked]>
Date: Wed, 25 Aug 2004 18:36:18 -0700

> can't use your programming language's URI class to compare them.

But it's equally conterintuitive that they're defined as URI but I 
can't put a whole gamut of URIs in them because they don't fit a bunch 
of arbitrary rules I don't have time to learn. You can't win.

Graham

---

From: Sam Ruby <[email-masked]>
Date: Wed, 25 Aug 2004 22:33:51 -0400

Graham wrote:

> But it's equally conterintuitive that they're defined as URI but I can't 
> put a whole gamut of URIs in them because they don't fit a bunch of 
> arbitrary rules I don't have time to learn.

Exactly which word in *ANY* of the following says that you *CAN'T*?

   http://www.imc.org/atom-syntax/mail-archive/msg08572.html
   http://www.imc.org/atom-syntax/mail-archive/msg08760.html
   http://www.intertwingly.net/wiki/pie/PaceIdConstruct

- Sam Ruby

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 26 Aug 2004 08:25:07 +0200

On Wed, 25 Aug 2004 19:05:17 -0400, Graham Parks <[email-masked]> wrote:

> The problem is that there's more than one way to compare URIs, and not  
> all URI classes do the same thing. For consistency's sake we need to  
> specify one method, and we're specifying the simplest.

I agree. Consumers should do string comparison.

> Canonicalization has been suggested as an alternative solution (right?).

No, that's not an alternative. Consumers should do _nothing_ else with the  
URI than treat it as any other string. No normalization, no  
canonicalization, no nothing. This is up to the _publisher_ to do, so that  
all atom:id's are correctly canonicalized when the consumer is fetching  
them to do comparison.

> I see how that would work, but I much prefer the simplicity of the above  
> solution.

Simplicity for consumers is good in this area, for consumers it sucks,  
because it will create inconsistent atom:id URI's since they aren't  
required to be canonicalized.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 26 Aug 2004 08:29:47 +0200

> (absoluteURI production in RFC2396) and that they are to be compared  

This is wording for the consumers. What about wording for the producers?  
Is there any good reasons to not recommend or require publishers to put  
canonicalized URI's in atom:id?

> should be a clear benefit in doing it.

The clear benefit of recommending (SHOULD) or requiring (MUST) publishers  
to do it is to prevent inconsistent URI's in atom:id. Since atom:id is  
supposed to be immutable, that should be a goal, imho.

> Saying "the IDs will work better when recipients break the spec when
> comparing them"

I have never said that recipients (consumers) should do anything special  
with atom:id URI's. I have said that producers should.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 26 Aug 2004 08:31:05 +0200

> requiring canonicalization, right?

Of course there is. If not, producers might use 'http://EXAMPLE.COM/232'  
as atom:id one place, and 'http://example.com/232' another place. By URI  
rules, those are the same, as strings they're not.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 26 Aug 2004 08:44:38 +0200

> shouldn't it work for something else?

Because Bob doesn't have the same technical skills or knowledge as the  
people creating XML processors have. Bob will be a valuable consumer and  
producer of Atom feeds. Why shouldn't Bob get help from the specification  
to get this right? If the spec is silent on the issue, he will get it  
wrong.

Not to mention the ViewSourceClan. How do you expect them to consume Atom  
feeds correctly, if the publishers aren't gently forced to do things as  
interoperable and correctly as possible?

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 08:52:24 +0200

> I guess Julian doesn't use languages like Java and C#, but I hear

Please stop guessing.

Thanks, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 08:56:46 +0200

Asbjørn Ulsberg wrote:

> This is wording for the consumers. What about wording for the 
> producers?  Is there any good reasons to not recommend or require 
> publishers to put  canonicalized URI's in atom:id?

Yes, it doesn't help at all. Even if every atom ID is canonical, 
comparision results still can vary if people do not use identical 
comparison methods. On the other hand, if they do (and the method is 
string comparison), requiring canonicalization doesn't buy you anything 
and thus is useless.

> The clear benefit of recommending (SHOULD) or requiring (MUST) 
> publishers  to do it is to prevent inconsistent URI's in atom:id. Since 
> atom:id is  supposed to be immutable, that should be a goal, imho.

Define "inconsistent" in this context.

> I have never said that recipients (consumers) should do anything 
> special  with atom:id URI's. I have said that producers should.

Yes, but if consumers just do what the spec says, *it* *does* *not* 
*matter*.

Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 08:59:51 +0200

Asbjørn Ulsberg wrote:

> Of course there is. If not, producers might use 
> 'http://EXAMPLE.COM/232'  as atom:id one place, and 
> 'http://example.com/232' another place. By URI  rules, those are the 
> same, as strings they're not.

Incorrect. By *one* of many ways to compare URIs, they are the same. 
There are multiple ways to compare URIs, and we just need to pick one.

Speaking of which - how do you prevent people from using both

	http://example.com

and

	http://example.com:80

if you just require canonicalization? Both are canonical, but for some 
(scheme-dependant) comparison methods (such as in a URI class that knows 
about http) they will compare identical.

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 09:01:59 +0200

Asbjørn Ulsberg wrote:

> On Thu, 26 Aug 2004 00:15:49 +0200, Julian Reschke 
>> namespaces (show me one XML+NS processor that gets this wrong!), so 
>> why  shouldn't it work for something else?
> 
> 
> people creating XML processors have. Bob will be a valuable consumer 
> and  producer of Atom feeds. Why shouldn't Bob get help from the 
> specification  to get this right? If the spec is silent on the issue, he 
> will get it  wrong.

It should not be silent. It should be crystal-clear. The best way to do 
that is to make it *simple*. An id is to be compared 
character-by-character.

If this helps, add examples and a test suite.

> Not to mention the ViewSourceClan. How do you expect them to consume 
> Atom  feeds correctly, if the publishers aren't gently forced to do 
> things as  interoperable and correctly as possible?

How do you suggest to prevent the ViewSourceClan to re-use an atom ID 
(canonical or not) that they've seen in an existing document?

Julian

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 26 Aug 2004 09:07:53 +0200

On Thu, 26 Aug 2004 08:56:46 +0200, Julian Reschke <[email-masked]>  
wrote:

> comparison methods.

Who has been talking about different comparison methods? People should  
compare ID's in one way, at least internally. I really don't care whether  
they use URI or string comparison, and as long as the URI's are  
canonicalized, it won't matter either.

> On the other hand, if they do (and the method is string comparison),
> requiring canonicalization doesn't buy you anything and thus is useless.

What canonicalization on the producer's side buys you, is that  
'http://EXAMPLE.COM/1212' is an invalid or a should-not atom:id.

> Define "inconsistent" in this context.

'http://EXAMPLE.COM/1212' and 'http://example.com/1212' are the same  
URI's, but different strings. That's inconsistent. Getting rid of the  
uppercase version of those two (which you do by requiring  
canonicalization) makes atom:id much more resilient to such inconsistency  
errors.

> Yes, but if consumers just do what the spec says, *it* *does* *not*  

So in your book, 'http://EXAMPLE.COM/1212' and 'http://example.com/1212'  
are the same strings, or doesn't it matter that they're different?

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Thu, 26 Aug 2004 09:10:36 +0200

> Incorrect. By *one* of many ways to compare URIs, they are the same.

So now we don't only want people to compare atom:id as strings, but as  
case-insensitive strings. Correct?

> There are multiple ways to compare URIs, and we just need to pick one.

If we pick «compare as strings», casing matters.

> about http) they will compare identical.

'http://example.com:80' is imho not canonical in regard to the HTTP scheme  
rules. The scheme's default port should be stripped, which makes  
'http://example.com:80' into 'http://example.com'.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 09:20:49 +0200

> case-insensitive strings. Correct?

No.

> If we pick «compare as strings», casing matters.

What I was referring to is the method defined in 
<http://cvs.apache.org/viewcvs.cgi/*checkout*/ietf-uri/rev-2002/rfc2396bis.html#comparison-string> 
(case matters).

>> if you just require canonicalization? Both are canonical, but for 
>> some  (scheme-dependant) comparison methods (such as in a URI class 
>> that knows  about http) they will compare identical.
> 
> 
> 'http://example.com:80' is imho not canonical in regard to the HTTP 
> scheme  rules. The scheme's default port should be stripped, which 
> makes  'http://example.com:80' into 'http://example.com'.

But now you're speaking of a vague term, not the one defined in 
<http://cvs.apache.org/viewcvs.cgi/*checkout*/ietf-uri/rev-2002/rfc2396bis.html#canonical-form>. 
Where do I find the definition of "canonical" for an arbitrary URI 
scheme? What should the Atom spec actually say to require this 
scheme-dependant canonicalization?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 09:16:39 +0200

Asbjørn Ulsberg wrote:

> On Thu, 26 Aug 2004 08:56:46 +0200, Julian Reschke 
> compare ID's in one way, at least internally. I really don't care 
> whether  they use URI or string comparison, and as long as the URI's 
> are  canonicalized, it won't matter either.

a) There is *more* than one way to compare URIs. Some ways are 
scheme-dependant.

b) Unless we pick a specific method (and that by definition needs to be 
independant of scheme), comparison results *will* vary, and this has 
nothing to do whatsoever with canonicalization.

I just posted an example for that, and it would be appreciated if people 
would actually look at it and try to understand why requiring canonical 
URIs does NOT guarantee the same comparison results.

> canonicalization) makes atom:id much more resilient to such 
> inconsistency  errors.

Speaking of "sameness" requires an agreed-upon comparison method. 
RFC2396bis defines a "ladder" of comparison methods, and when you speak 
of "same" you'll need to say under which comparison.

If Atom defines string comparison 
(<http://cvs.apache.org/viewcvs.cgi/*checkout*/ietf-uri/rev-2002/rfc2396bis.html#comparison-string>), 
the two identifiers above are indeed *not* the same.

> So in your book, 'http://EXAMPLE.COM/1212' and 
> 'http://example.com/1212'  are the same strings, or doesn't it matter 
> that they're different?

They aren't the same strings, so they aren't the same IDs.

Julian

---

From: Bill de hÓra <[email-masked]>
Date: Thu, 26 Aug 2004 09:57:22 +0100

> I guess Julian doesn't use languages like Java and C#, but I hear

A major point of speccing strcmp is to protect developers from 
borked uricmp code.

I use both Java and C#, and sometimes I use both in the same system. 
You can use their URI.eq calls; indeed we can't stop people using 
their URI.eq calls. Just don't expect the match quality to be the 
same as if we were using String.eq calls. Java at least, has a 
history of borked URI-related code going all the way back to 1.0 
right up to 1.5 (~8 years). That's why a lot of people end up 
writing their own URI/URL classes in Java.

I agree it is counterintuitive. But it less counterintuitive than 
tracing down lost matches to a URI/L.eq library call, realizing you 
can't fix it by subclassing because the class is sealed, then 
realizing you'll have to wait years for the bug to be fixed (if 
ever), then realizing you options are dynamic proxies, or rolling 
your own, or changing your code to use someone else's URI class that 
  works (maybe). Ultimately the answer is to say to hell with with 
it and using String.eq and c14n. I've been through all this and it 
is a intensely stupifying and frustrating experience I wouldn't wish 
on anyone. Which is my way of saying not that we design against 
broken libraries, but that based on my experience, uricmp will be a 
significant burden that results in reduced data quality. Telling 
folks to use strcmp and/or having atom:id automapped onto string 
classes remains a major act of kindness on our part imo.

And no, this does not belong in an implementors guide. If we are 
unable to be decisive and as result inflict a burden on those 
mapping IDs onto their programming language constructs, or those 
manipulating Atom, then we need to explain very clearly the 
consequences of certain choices. If we punt, then we tell people how 
to cope with our ineffectiveness. No guts, no glory.

Winning scenario: mandate atom ids are compared are strings. Say 
stuff about using URIs to generate them. Comparison. Generation. 
Those are separate section headings folks. How hard do we have to 
make this?

cheers
Bill

---

From: "Michael Stillwell" <[email-masked]>
Date: Thu, 26 Aug 2004 11:36:25 +0100 (BST)

> atom:ids except compare them." This is counterintuitive.

Can't the canonicalisation issue be avoided by simply using a URI scheme
other than "http" for ids? (Or can't feed producers who dislike the
canonicalisation stuff individually choose to do so?)

Canonicalisation rules exist for the http scheme, but not for all
schemes, and URI classes will surely only attempt to canonicalise schemes
they know about. I can imagine that URI classes would consider
"http://example.com" and "http://EXAMPLE.COM" to be the same, for
example, but any that considers "atomid:example.com" and
"atomid:EXAMPLE.COM" to be so is very broken. i.e. define a URI of
scheme "atomid" to be "atomid:" + some string. (I realise this is almost
the same as just "some string," but some people seem really keen on URIs
as ids...)

RFC2696bis says (note the "with respect to their scheme"): "Those who
produce and make reference to URIs can reduce the cost of processing and
the risk of false negatives by consistently providing them in a form that
is reasonably canonical with respect to their scheme."
http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#canonical-form

(I also think it's counterintuitive for an id to look like something you
can type into your browser's location field; switching to (or
recommending) a different scheme would also eliminate this problem.)

--M.

--
http://beebo.org

---

From: Sam Ruby <[email-masked]>
Date: Thu, 26 Aug 2004 06:49:02 -0400

> about http) they will compare identical.

"For schemes that define a port, use an empty port if the default is 
desired"

http://www.intertwingly.net/wiki/pie/PaceIdConstruct
http://www.intertwingly.net/blog/2004/08/04/Urlnorm

The proposal is to put the former into the spec, the latter into the 
feedvalidator.  The expectation is that few will see this warning, and 
those that do are free to ignore it.

- Sam Ruby

P.S.: the Equals method in the System.Uri class in the .Net CLR will 
return True with the example URIs listed above.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 26 Aug 2004 13:06:20 +0200

Sam Ruby wrote:

>> (scheme-dependant) comparison methods (such as in a URI class that 
>> knows about http) they will compare identical.
> 
> 

OK,

this is something that goes beyond what RFC2396bis defines. Anyway; consider

	http://example.com:81

and

	http://example.com:081

No matter how hard you try defining a canonical form, there will always 
be room for IDs that have different "canonical" forms but will compare 
equal for some implementation that simply knows more about equivalence 
for that URI scheme than a generic canonicalization can specify.

> return True with the example URIs listed above.

Yes, but would it also for

	webcal://example.com

and

	webcal://example.com:80

?

The point being: assumptions about what URI classes will state on 
various platforms do not help. These may vary over time (releases), 
platforms, installed URI schemes. We should not rely on those, therefore 
we require simple string comparison. And if we do that, canonicalization 
does not give us anything in addition (except a more complex and harder 
to explain spec).

Julian

---

From: Danny Ayers <[email-masked]>
Date: Thu, 26 Aug 2004 16:25:36 +0200

> canonicalisation stuff individually choose to do so?)

Perhaps, though it would need the introduction of corresponding
URI-building tools in every producer. Which is probably about the same
complication as producing canonical URIs.
 
> the same as just "some string," but some people seem really keen on URIs

URIs are the identifiers of the web, Atom's primary environment.

> recommending) a different scheme would also eliminate this problem.)

How often do people type new things into the location field? I only
usually do it to get the name-completion bit of Firefox to remember
places I've already been. The counter-intuitive bit is giving
resources on the web (which usually have HTTP-retrievable
representations)  URIs that by design won't give access to a
representation.

I think the sweetspot between being 'correct' (using URIs according to
rfc2396bis, noting that many code libs are borked) and 'simple' is to
use the modified form of URI, where comparison is char-by-char. It
would have to be made clear that these aren't really URIs, and
probably advise producers to use the canonical form - it's in their
own interests not to flood aggregator windows with duplicates after
all.

Cheers,
Danny.

---

From: Walter Underwood <[email-masked]>
Date: Thu, 26 Aug 2004 08:11:01 -0700

> canonicalisation stuff individually choose to do so?)

Legacy data doesn't have IDs, and making one up can make things
worse (different IDs for the same entry, created in different
feed software). There are RDF feeds around with HTTP URI IDs.

So there is substantial advantage in allowing HTTP URIs as IDs.

wunder

---

From: Antone Roundy <[email-masked]>
Date: Thu, 26 Aug 2004 09:40:09 -0600

> requiring canonicalization, right?

What is our goal, and how to we maximize the probability of it being 
met?

Goal: to be able to recognize an entry that we've seen before as being 
the same entry when we see it again (whether in the same feed or not, 
whether modified or not, no false positives and no false negatives)

Required attributes of an id to meet this goal:
1) must be globally unique
2) must not change
3) must be able to compare ids unambiguously

How we maximize the probability of achieving those attributes, and thus 
the goal:
1) use URIs for ids
2) tell people not to alter existing ids under any circumstances
3) mandate string comparison

I think we can all agree on, or at least accept, the foregoing.

Possible barriers:
2) a) dynamic id generation yielding different ids after a change in 
the system that generates them (it gets moved to a new domain, a change 
to a library, etc.) b) dynamic id generation yielding different ids 
because multiple systems are independently generating ids differently 
from the same data (one uses a library that includes port numbers, the 
other doesn't, etc.)

Mandating that people store their IDs is an implementation detail that 
we don't want to get into (at least not most of us).  So how do we 
maximize the probability of dynamically generated ids being minted the 
same way each time?  By specifying not just canonicalization, but a 
specific set of canonicalization rules, hopefully prompting people to 
use code that they can be certain will generate ids according to those 
rules rather than trusting that the library they're using will do it 
right.  A specific set of canonicalization rules also enables feed 
validators to tell people that the way they're minting ids is wrong (or 
at least, violates a SHOULD--we need to accomodate ids originating from 
systems with looser constraints, right?), prompting them to improve 
their minting system making it more likely to remain consistent if one 
of its ingredients changes.

Questions:
1) Will requiring canonicalization according to a set of rules 
explicitly set out in the spec yield the benefits outlined above?
2) Will that be too onerous a burden for the benefits it yields?
3) Is there a better way to maximize the probability of meeting the 
goal?

---

From: Walter Underwood <[email-masked]>
Date: Thu, 26 Aug 2004 09:12:24 -0700

--On Thursday, August 26, 2004 8:59 AM +0200 Julian Reschke <[email-masked]> wrote:
> if you just require canonicalization? Both are canonical, ...

Neither one is canonical. Add a trailing slash, according to
this rule:

  For schemes that define an empty path to be equivalent to a path
  of "/", use "/".

Post to the URI working group list, suggesting that they add
default port elision to the canonicalization rules.

I will post to the URI WG about the rules not being clear enough,
since you missed that one. Please let me know whether you did
read the rules, so I can let them know (privately is fine, this
is off-topic for Atom).

wunder

---

From: Robert Sayre <[email-masked]>
Date: Thu, 26 Aug 2004 18:25:31 -0400

Tim Bray wrote:

> 
> I'm getting pushback saying "you're going too fast."  So, 
> should hold on for a bit to give other people a chance to object...

Sam has made several updates to this Pace based on feedback, and Mark N.
has proposed some reasonable changes:

http://www.imc.org/atom-syntax/mail-archive/msg08698.html
http://www.imc.org/atom-syntax/mail-archive/msg08691.html

The consensus remains the same and the Pace has been improved. This
"debate" could really use a new draft, IMHO.

Robert Sayre

---

From: Sam Ruby <[email-masked]>
Date: Thu, 26 Aug 2004 18:50:59 -0400

Robert Sayre wrote:
> 
> Tim Bray wrote:
> 
>> I'm getting pushback saying "you're going too fast."  So, while I do 
>> believe that Sam's draft language matches the consensus I've inferred 
>> from the last few weeks of email on this subject, I suppose we should 
>> hold on for a bit to give other people a chance to object...
> 

If the next draft were to omit the words "If the identified resource is 
served dynamically, the content of an Identification construct MUST be 
created only once and then stored along with the resource. The content 
of an Identification construct MUST NOT be created dynamically.", I 
would not object.  The requirement is covered elsewhere.

I am also OK with Mark's proposed "...is changed in content or location 
(e.g., ...)"

The examples were straight from the proposed Namespaces in XML 1.1 
document, so I gather that the editor of the Atom Syndication Format 
specification disagrees with the editors of the Namespaces in XML 1.1 
document on on the number and types of examples a document of this 
nature should have.  My leanings are to have more.

Yes, at this time, a new draft would be most welcome.

- Sam Ruby

---

From: Tim Bray <[email-masked]>
Date: Thu, 26 Aug 2004 16:32:22 -0700

> nature should have.  My leanings are to have more.

As it stands, PaceIdConstruct has exactly two examples.  I would be 
very unhappy if they were lost.

Paul spoke for the chairs here: 
http://imc.org/atom-syntax/mail-archive/msg08760.html - Graham had some 
issues around the wording and agreed, in the case where he doesn't like 
the way it shows up in the draft, to draft an improvement: 
http://imc.org/atom-syntax/mail-archive/msg08786.html

Since then, we've had another flurry of messages, but I still think 
letting MNot take a crack at working PaceIdConstruct into a format-02 
draft, then suggesting improvements to that, is the most constructive 
way forward.  -Tim

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 27 Aug 2004 08:19:48 +0200

On Thu, 26 Aug 2004 09:40:09 -0600, Antone Roundy <[email-masked]>  
wrote:

> By specifying not just canonicalization, but a specific set of
> canonicalization rules, hopefully prompting people to use code that
> they can be certain will generate ids according to those rules rather
> than trusting that the library they're using will do it right.

+1.

> explicitly set out in the spec yield the benefits outlined above?

I believe so, yes.

> 2) Will that be too onerous a burden for the benefits it yields?

Nah. It's easy to do by the few rules e.g. Mark has enumerated in his  
article. If atom:id canonicalization is only a SHOULD, these examples can  
go into an informative appendix as well, if people don't like to «clutter»  
the specification with it.

> 3) Is there a better way to maximize the probability of meeting the goal?

Not that I know of, at least.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 27 Aug 2004 09:13:19 +0200

> requiring canonicalization, right?

Yes, because if publishers don't canonicalize and normalize,  
'http://EXAMPLE.COM/1231' and 'http://example.com/1231' would be  
string-compared as two different entries, regardless of what the publisher  
really intended with those URI's. Most likely, the publisher wanted them  
to be the same.

If all publishers normalize, canonicalize and do everything in their power  
to keep atom:id stable and easy to compare, being an Atom consumer will be  
easy. If they don't, being an Atom consumer will be hard.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 27 Aug 2004 09:37:48 +0200

Asbjørn Ulsberg wrote:
> string-compared as two different entries, regardless of what the 
> publisher  really intended with those URI's. Most likely, the publisher 
> wanted them  to be the same.

Well, that's the publisher's problem. If they want two IDs to be the 
same, they'd better use the same string (as the spec tells them).

> If all publishers normalize, canonicalize and do everything in their 
> power  to keep atom:id stable and easy to compare, being an Atom 
> consumer will be  easy. If they don't, being an Atom consumer will be hard.

Incorrect. As long as IDs are compared as strings, and both producers 
and consumers are aware of that, there is no issue. Please don't make 
things more complicated than needed.

Julian

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 27 Aug 2004 08:19:48 +0200

+1.

I believe so, yes.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 27 Aug 2004 09:13:19 +0200

---

From: Julian Reschke <[email-masked]>
Date: Fri, 27 Aug 2004 09:37:48 +0200

Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 27 Aug 2004 14:14:55 +0200

> wanted them  to be the same.

If he wanted that, he made a mistake. It's better if this mistake has 
the same consequences in all aggregators, I would think.

> consumer will be  easy. If they don't, being an Atom consumer will be hard.

Atom IDs *are* simple to compare if we stick with char-by-char 
comparison. All recipients will behave exactly the same. Isn't this what 
interoperability is about?

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 27 Aug 2004 06:11:43 -0700 (PDT)

+1 
 
> things more complicated than needed.

+2 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #23
I will keep a special cache of low-tech weapons and train my troops in their use. That way -- even if the heroes manage to neutralize my power generator and/or render the standard-issue energy weapons useless -- my troops will not be overrun by a handful of savages armed with spears and rocks.

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 27 Aug 2004 18:27:41 +0200

> the same consequences in all aggregators, I would think.

It _would_ have the same consequence in all conforming aggregators, since  
the specification should say that atom:id should be char-by-char-compared.  
There is still no good reason not to require or recommend publishers to  
canonicalize their URI's.

> Atom IDs *are* simple to compare if we stick with char-by-char  

I am not against char-by-char comparison.

---

From: Julian Reschke <[email-masked]>
Date: Fri, 27 Aug 2004 18:29:27 +0200

Asbjørn Ulsberg wrote:

> On Fri, 27 Aug 2004 14:14:55 +0200, Julian Reschke 
> It _would_ have the same consequence in all conforming aggregators, 
> since  the specification should say that atom:id should be 
> char-by-char-compared.  There is still no good reason not to require or 
> recommend publishers to  canonicalize their URI's.

There's no reason to require a specific behaviour when it doesn't make 
any difference whatsoever whether people actually do it or not. This is 
about spec simplicity.

Best regards, Julian

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Fri, 27 Aug 2004 19:42:57 +0200

> Just a small point: There is no requirement to store IRIs as UTF-8.

Of course. My point wasn't that they were stored as UTF-8, but that most  
DNS servers and such currently have problems with UTF-8 URI's. But since  
atom:id is not dereferencable, what DNS's supports isn't very interesting.  
Which is why IRI's are easier to deploy in atom:id than in other,  
dereferencable URI's.