Subject: PaceFeedEquivalence

From: Tim Bray <[email-masked]>
Date: Thu, 29 Jul 2004 12:24:36 -0700

While the objective is fine, I don't believe we need a separate section 
on Feed Equivalence.  I propose:

(1) a note in both the Format and Protocol specs saying "when testing 
URIs for equivalence, implementors should consider the discussion in 
RFC2396bis section 6."  (currently 
http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#comparison).
(2) a note in section 4.2.6 of the format draft saying "since the same 
feed may be served from different URIs, and feeds may be accessed other 
than by dereferencing a URI, the <atom:id> in <atom:head> may be used 
to test whether two feeds are the same."

-Tim

---

From: Mark Pilgrim <[email-masked]>
Date: Thu, 29 Jul 2004 16:38:33 -0400

> http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#comparison).

Note that this is the first time we are actively tying ourselves to
2396bis (as opposed to PaceUriOrItsSuccessor, which only allowed us to
inherit it when it was ready).

Furthermore, are you rejecting Norman Walsh's proposal that "matches"
and "matching" be more specifically defined as per Section 6.2.1
"Simple String Comparison" of 2396bis?

-- 
Cheers,
-Mark

---

From: Tim Bray <[email-masked]>
Date: Thu, 29 Jul 2004 13:56:17 -0700

> inherit it when it was ready).

Good point... it's just that 2396bis has this useful section 6 that 
goes on and on about exactly this issue, which isn't in 2396classic at 
all.  On the other hand if we were nearly done, and 2396bis wasn't, I'd 
be in favor of nuking this section to get us across the finish line.

> "Simple String Comparison" of 2396bis?

Yes.  If I have two pointers to feeds:

http://www.example.com:80/feed.atom
http://www.Example.COM/feed.atom

I guarantee that some apps, in particular web spiders, are going to get 
very aggressive about schema-specific normalization and regard these as 
equivalent, and I don't see any reason to rule this out.  Norm?  -Tim

---

From: Mark Pilgrim <[email-masked]>
Date: Thu, 29 Jul 2004 17:05:37 -0400

> http://www.Example.COM/feed.atom

This has been discussed on-list before [1], but no language has made
it into a spec draft.

By a startling coincidence, less than 8 hours ago I opened an issue
tracker [2] on this exact issue for the feed validator.  My current
test cases are based on the examples in section 3.2.3 of RFC 2616 [3].
 Obviously I would be willing to amend the test cases to correspond
with whatever "URI equivalence" rules we state in the Atom spec.

[1] http://www.imc.org/atom-syntax/mail-archive/msg07662.html
[2] http://sourceforge.net/tracker/index.php?func=detail&aid=1000249&group_id=99943&atid=626803
[3] http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.2.3

-- 
Cheers,
-Mark

---

From: Walter Underwood <[email-masked]>
Date: Thu, 29 Jul 2004 15:39:19 -0700

> equivalent, and I don't see any reason to rule this out.  Norm?  -Tim

I'm not Norm, but I can confirm that. Normalization is done very
early, including %-escapes, ".." in paths, trimming fragments,
deleting session IDs, ...

wunder
--
Walter Underwood
Principal Architect, Verity

---

From: Joe Gregorio <[email-masked]>
Date: Thu, 29 Jul 2004 23:27:34 -0400

> equivalent, and I don't see any reason to rule this out.  Norm?  -Tim

This example is not relevant for several reasons.

First, "Simple String Comparison" is the strictest form
of URI comparison that you can do. That is, 
as a means of determining if two atom:ids are the same
it is the strictest test possible. Your example, 
and all other ways of comparing URIs, will be looser than
"Simple String Comparison". And that means that no
matter how you compare the URIs in practice, you will
always find the ones that are the same. I believe that
would be plus for interoperability.

That is, if I am a producer and I produce two feeds
with the same entry in each but they 
have atom:ids of

and

respectively then as a producer I should not be suprised
if an aggregator treats them as different entries.

Second, the assertion about spiders is a bit 
odd, are you asserting there are spiders out 
there that know nothing of the Atom format
yet will try to determine the equivalence of 
'entries' based on atom:id? Or are you asserting 
there are spiders that will know of the Atom format 
and will knowingly ignore the specification?

    -joe

-- 
Joe Gregorio        http://bitworking.org

---

From: Tim Bray <[email-masked]>
Date: Thu, 29 Jul 2004 21:10:59 -0700

> would be plus for interoperability.

Please try again Joe.  I must be having a stupid evening, I read that 
paragraph three times and I can't figure out what you're trying to say.

> if an aggregator treats them as different entries.

Right, and as a producer that would be bad practice.  The interesting 
(and not uncommon in my experience) case is when you pick those up on 
two different servers and are wondering if they're really the same.  
We're not going to *force* people to be heroic about doing 
scheme-specific comparison, but there's no point writing rules against 
it.

> and will knowingly ignore the specification?

I am 100% certain that there will be atom-savvy spiders, based on the 
fact that there are already RSS-savvy spiders.  Large scale spiders 
(I've written two) go to immense lengths to spot duplicate URLs because 
there are a lot of them out there, and you can do immense amounts of 
computation in the time it takes to fetch one, so it's very 
cost-effective.  I guarantee that every serious search-engine spider on 
the planet is already doing this kind of HTTP-specific smart 
comparison.  There's no earthly reason to write a rule against it.  It 
would also be unreasonable to *require* anything smarter than string 
comparison, but as URIs get passed around the infrastructure, shit 
happens, and we shouldn't get in the way of cleanup attempts. -Tim

---

From: Martin Duerst <[email-masked]>
Date: Fri, 30 Jul 2004 14:19:51 +0900

At 13:56 04/07/29 -0700, Tim Bray wrote:

>Good point... it's just that 2396bis has this useful section 6 that goes 
>on and on about exactly this issue, which isn't in 2396classic at all.  On 
>the other hand if we were nearly done, and 2396bis wasn't, I'd be in favor 
>of nuking this section to get us across the finish line.

I think that section is very useful (and there is an equivalent section
in the IRI draft). However, I think what we need for consistent
behavior is a clear decision on what equivalence we are going to use,
not a pointer to an on-and-ongoing discussion.

>equivalent, and I don't see any reason to rule this out.  Norm?  -Tim

I think web spiders might be allowed to do this. And for those cases
where a resolution is involved, the two above will obviously get us
the same result. However, for namespaces, we have to follow the Namespace
spec, and e.g. for atom:id, simple string comparison is much more
appropriate in my view.

Regards,     Martin.

---

From: Martin Duerst <[email-masked]>
Date: Fri, 30 Jul 2004 14:15:06 +0900

Hello Mark,

It would be good if your test cases were available without having
to log in on sourceforge.

Regards,    Martin.

At 17:05 04/07/29 -0400, Mark Pilgrim wrote:

>[2] 
>http://sourceforge.net/tracker/index.php?func=detail&aid=1000249&group_id=9 
>9943&atid=626803

---

From: Mark Pilgrim <[email-masked]>
Date: Fri, 30 Jul 2004 08:25:04 -0400

> to log in on sourceforge.

http://feedvalidator.org/testcases/atom/must/

The specific tests in question are

http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_2.xml
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_3.xml
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_4.xml
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_5.xml
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_6.xml
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_7.xml
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_8.xml

Note that I'm not sure the last one is a valid test.  I would
appreciate any guidance you might have on that.

-- 
Cheers,
-Mark

---

From: Joe Gregorio <[email-masked]>
Date: Fri, 30 Jul 2004 11:45:49 -0400

> happens, and we shouldn't get in the way of cleanup attempts. -Tim

I'll answer both of the above with a question: What kind of 
URI normalizations should be used when comparing
atom:id URIs for equivalence?

"Case Normalization"
"Percent-Encoding Normalization"
"Path Segment Normalization"
"Scheme-based Normalization"
"Protocol-based Normalization"

That is for the *consumer* side of 
atom:id. On the producer side should
we suggest/require the canonical form
of URIs[1]? And if so should we
augment that canonicalization
to include a unicode normalization form?

    -joe

[1] http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#canonical-form

---

From: Mark Pilgrim <[email-masked]>
Date: Fri, 30 Jul 2004 13:21:56 -0400

On Fri, 30 Jul 2004 11:45:49 -0400, Joe Gregorio <[email-masked]> wrote:
> What kind of

All of them except protocol normalization, which requires resource
retrieval to verify equivalence.

> "Case Normalization"

See http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_5.xml

> "Percent-Encoding Normalization"

See http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_6.xml
and http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_7.xml

> "Path Segment Normalization"

See http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_2.xml

> "Scheme-based Normalization"

See http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_3.xml
and http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_4.xml

> "Protocol-based Normalization"

No, because this requires retrieval.

Also, I wonder about
http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_8.xml
where two URIs differ only by Unicode normalization form after being
un-percent-encoded.  Are these two URIs the same or different?

> of URIs[1]?

I can definitely see us adding informational messages to the feed
validator to check for variations of these.

> And if so should we

An excellent question.  Tim?

-- 
Cheers,
-Mark

---

From: Tim Bray <[email-masked]>
Date: Fri, 30 Jul 2004 10:36:43 -0700

> retrieval to verify equivalence.

Er, by "should" do you mean SHOULD?  I think it's OK for software to do 
simple string comparison - particularly resource-challenged 
implementations - and I think it's also OK to do all the other things 
outlined.  I think that we should document the fact that those who mint 
and copy URIs can't count on other software doing anything more than a 
string comparison, so, to use the vernacular,
(a) Always use the same code when you're generating your own URIs, and
(b) don't fuck with the URIs you're given, store and pass them on the 
way you got them.

Given that, shit happens, which is why (in particular) robots will go 
to great lengths to determine when to different-looking URIs are 
actually the same.

> An excellent question.  Tim?

As above.  Don't require it, but don't try to forbid it either.  -Tim

---

From: Sam Ruby <[email-masked]>
Date: Fri, 30 Jul 2004 14:18:15 -0400

Tim Bray wrote:

> Given that, shit happens, which is why (in particular) robots will go to 
> great lengths to determine when to different-looking URIs are actually 
> the same.
> 
>>> And if so should we

Let's look at this from a producer side.  If "PlanetSun" ever has an 
Atom 1.0 feed, MUST the ids be byte for byte identical to the ones that 
these entries were based off of, or merely equivalent?

If we don't mandate byte for byte correspondence, then it is NOT OK for 
sofware to do simple string comparisons.  They must check for equivalence.

If we mandate byte for byte correspondence, then there is no need for 
clients to do anything other than byte for byte comparisons.  In fact, 
if they attempt to cannonicalize, then they may identify false matches.

I think we need to pick one.

- Sam Ruby

---

From: Martin Duerst <[email-masked]>
Date: Sat, 31 Jul 2004 09:21:38 +0900

My baseline opinion is still very strongly to only require
character-by-character (also called codepoint-by-codepoint)
comparison for ids. This is the one thing that is easiest
for all implementations to get right.

>to include a unicode normalization form?

No. If Atom is using URIs (which is still the case according
to the current spec), then that wouldn't work, because there
is no way for URIs to know about Unicode characters. If we
change to IRIs (which I hope we will soon), then the IRI
spec explicitly says that senders should get this right
(except for some very specific cases of dealing with legacy
encodings).

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sat, 31 Jul 2004 09:26:53 +0900

At 14:18 04/07/30 -0400, Sam Ruby wrote:

>Let's look at this from a producer side.  If "PlanetSun" ever has an Atom 
>1.0 feed, MUST the ids be byte for byte identical to the ones that these 
>entries were based off of, or merely equivalent?
>
>clients to do anything other than byte for byte comparisons.  In fact, if 
>they attempt to cannonicalize, then they may identify false matches.

Sam, I'm not sure what you are talking about here.

This is XML, there is no room for byte-for-byte equivalence at all.
The same ID can once be published in ASCII, once in UTF-16, and
once in EBCDIC, which will be different byte sequences, but for
XML, and also for URIs, they nevertheless will be the same sequence
of characters. The sequence of characters (also called codepoints)
is the absolute minimum we need, and it's what people mean when
they talk about simple string comparison, because in every parser
we know, the input gets converted to a single encoding before
any further work.

Regards,    Martin.

---

From: Tim Bray <[email-masked]>
Date: Fri, 30 Jul 2004 21:23:53 -0700

> for all implementations to get right.

I think we're all in agreement that this is all that should be 
required.  However, we should acknowledge that robots and that class of 
software will go further, and while it's not required there's nothing 
wrong with that. -Tim

---

From: Tim Bray <[email-masked]>
Date: Fri, 30 Jul 2004 21:26:39 -0700

> This is XML, there is no room for byte-for-byte equivalence at all.

I know I've said this too often, but anyone who cares please go read 
read http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html#comparison 
- section 6.2.1 covers this issue in great detail.  -Tim

---

From: Martin Duerst <[email-masked]>
Date: Sat, 31 Jul 2004 14:28:31 +0900

>http://feedvalidator.org/testcases/atom/must/

Thanks. Below is my take on each test.

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_2.xml

slash after authority: scheme-specific, correct

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_3.xml

colon but no port number: correct

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_4.xml

default port: scheme-specific, correct

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_5.xml

casing difference in reg-name: correct

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_6.xml

percent-encoding difference (~ vs. %7E): correct

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_7.xml

percent-encoding difference (~ vs. %7e): correct

>http://feedvalidator.org/testcases/atom/must/entry_id_duplicate_value_8.xml

http://example.com/%c7 http://example.com/C%cc%a7

I understand the intention: testing equivalence between precomposed
and decomposed C-cedilla. But that won't work. First, %c7 is a
C-cedilla encoded in iso-8859-1. Using iso-8859-1 in URIs is not
forbidden, but highly discouraged, and in any case, the URI has no
clue what the encoding is. The corresponding UTF-8 would be %C3%87.
[no, iso-8859-1 and UTF-8 are not the same, only US-ASCII is the
same in UTF-8]

Because URIs don't know what encoding is used, there is also absolutely
no way for them to try and do some equivalence between precomposed
and decomposed Unicode characters.

And even for IRIs, the spec is clear that the sender should use
NFC (or even NFKC) as much as possible, i.e. the IRI equivalent of
http://example.com/%C3%87 in the above case, and if that doesn't
happen, these are not treated as equivalent.

Regards,    Martin.

---

From: Mark Pilgrim <[email-masked]>
Date: Sat, 31 Jul 2004 09:42:11 -0400

On Sat, 31 Jul 2004 14:28:31 +0900, Martin Duerst <[email-masked]> wrote:
> C-cedilla encoded in iso-8859-1.

You're right, this was my mistake in constructing the test.  However,
you correctly determined my intention, so let me reconstruct the test
and ask again.

[1] http://example.com/%C3%87
[2] http://example.com/C%CC%A7

1 unescapes to two bytes, \xc3 \x87, which is the UTF-8 representation
of the Unicode code point U+00C7, which is the NFC form of the
character C-cedilla.

2 unescapes to three bytes, C \xcc \xa7, which is the UTF-8
representation of the Unicode code points U+0043 U+0327, which is the
NFKC form of the character C-cedilla.

After percent-decoding, converting from UTF-8 to Unicode, and
normalizing to Unicode Normalization Form C, these URIs are identical.

Assuming I got this test right this time (and used all the right
terminology, please correct me if I didn't), which of the following
statements are we making:

A) Everyone is expected to perform all of these steps, and therefore
everyone is required to treat these URIs as identical.
B) Some clients may perform all of these steps, but others may not,
and therefore some will treat these URIs as identical but others
won't.
C) No one should perform all of these steps, and therefore everyone is
required to treat these URIs as different.

-- 
Cheers,
-Mark

---

From: Mark Pilgrim <[email-masked]>
Date: Sat, 31 Jul 2004 10:07:09 -0400

> - section 6.2.1 covers this issue in great detail.  -Tim

I've read it, and it doesn't mention Unicode normalization forms,
hence my question in the other "URI equivalence" thread.

-- 
Cheers,
-Mark

---

From: Tim Bray <[email-masked]>
Date: Sat, 31 Jul 2004 07:57:56 -0700

> required to treat these URIs as different.

B, so producers SHOULD NOT produce variant spellings of the same URI.  
(B is what will happen regardless of what we write down, so we might as 
well live with that).  We could try for "MUST NOT produce variant 
spellings" if people are comfortable with that. -Tim

---

From: Mark Pilgrim <[email-masked]>
Date: Sat, 31 Jul 2004 12:44:26 -0400

On Sat, 31 Jul 2004 07:57:56 -0700, Tim Bray <[email-masked]> wrote:
> B

Oh. My. God.

> , so producers SHOULD NOT produce variant spellings of the same URI.

Or we could try mandating a URI scheme that is specifically designed
to be used as an identifier, not a
resource-location-and-oh-by-the-way-wouldn't-it-be-cool-to-use-it-as-an-identifer-too.
 I know I said just a few days ago that I was in favor of using any
URI scheme, but Jesus, http:// URIs are incredibly complex.  And then
saying that clients are free to pick and choose their own rules for
comparing them, is just ridiculous.

-- 
Cheers,
-Mark

---

From: Sam Ruby <[email-masked]>
Date: Sat, 31 Jul 2004 12:42:36 -0400

> hence my question in the other "URI equivalence" thread.

First, a relatively recent experience with unicode normalization forms 
and URIs: http://earthlingsoft.net/ssp/blog/2004/05/decomposition

Second, from rfc2396bis:

     URI comparison is performed in respect to some particular purpose,
     and software with differing purposes will often be subject to
     differing design trade-offs in regards to how much effort should be
     spent in reducing duplicate identifiers

Apparently, after weighing a similar set of tradeoffs, XML Namespaces 
opted for "Simple String Comparison".  In the context of atom:id, what 
are the costs and benefits of requiring anything more?

- Sam Ruby

---

From: Tim Bray <[email-masked]>
Date: Sat, 31 Jul 2004 10:12:31 -0700

> comparing them, is just ridiculous.

Actually, a lot of the normalizations are not dependent on HTTP.  These  
include at least monocasing the scheme and identity components,  
crushing a/./b and /a/../a/ in the path component, and %-escaping.   
HTTP adds knowledge about port 80 and some trailing-slash semantics  
(see 2396bis 6.2.3), not that much.  So if there's a problem here, it's  
not an HTTP problem.

I think that we can say that software MAY use straight  
codepoint-by-codepoint comparison and for that reason producers SHOULD  
(MUST?) consistently spell the URIs they generate and intermediate  
parties SHOULD (MUST?) NOT change the spelling of URIs they deal with.   
However, humans are in the loop: URIs are scribbled on napkins and  
painted on the side of buses and read out by politicians during their  
keynote acceptance speeches.  So spelling variations will inevitably  
creep in, and I guarantee that spider-class software will use extreme  
aggressiveness in normalizing the suckers, so there's no point issuing  
any MUST NOTs. -Tim

---

From: Tim Bray <[email-masked]>
Date: Sat, 31 Jul 2004 10:15:39 -0700

> are the costs and benefits of requiring anything more?

There is no benefit whatever in *requiring* anything more.  There is 
similarly no benefit in trying to rule out more aggressive 
normalization for those instances in which it is cost-effective. -Tim

---

From: Sam Ruby <[email-masked]>
Date: Sat, 31 Jul 2004 13:34:24 -0400

> aggressiveness in normalizing the suckers, so there's no point issuing  

 From a feedvalidator perspective, if we agree on enabling consumers to 
rely solely codepoint-by-codepoint comparisons to determine equality, it 
makes sense for the feedvalidator to produce:

1) an informational message (or perhaps a warning?) if an id does not
    match a canonical form (perhaps Atom should adopt rules that are
    consistent with rfc2396bis[1]?).

2) a stronger warning (or perhaps even an error?) if two ids are present
    in a feed that differ only based on one or more normalization rules.

I'm inclined to support the stronger (parenthetical) versions of all the 
above rules.

- Sam Ruby

[1]<http://www.gbiv.com/protocols/uri/rev-2002/draft-fielding-uri-rfc2396bis-03.html#canonical-form>

---

From: Tim Bray <[email-masked]>
Date: Sat, 31 Jul 2004 10:43:54 -0700

On Jul 31, 2004, at 10:34 AM, Sam Ruby wrote:

> rely solely codepoint-by-codepoint comparisons to determine equality, 
> it makes sense for the feedvalidator to produce:
>
> I'm inclined to support the stronger (parenthetical) versions of all 
> the above rules.

Me too. -Tim

---

From: Mark Pilgrim <[email-masked]>
Date: Sat, 31 Jul 2004 16:24:29 -0400

> NFKC form of the character C-cedilla.

Small correction to myself: #2 is the NFD form, not NFKC form.  See
http://www.unicode.org/charts/normalization/chart_Latin.html

-- 
Cheers,
-Mark

---

From: Sam Ruby <[email-masked]>
Date: Sat, 31 Jul 2004 16:28:05 -0400

> comparing them, is just ridiculous.

I took a look into the rules for how two existing client platforms 
compare uris, and I found the results to be surprising:

   http://www.intertwingly.net/blog/2004/07/31/URI-Equivalence

- Sam Ruby

---

From: Martin Duerst <[email-masked]>
Date: Sun, 01 Aug 2004 09:32:54 +0900

>of the Unicode code point U+00C7, which is the NFC form of the

Yes, this is the NFC form, and also the NFKC form.

>NFKC form of the character C-cedilla.

No, this is the NFD form, and also the NFKD form (as you correctly
pointed out in a followup).

>normalizing to Unicode Normalization Form C, these URIs are identical.

Yes. But I don't know any server that would do this. I also don't
know any client that would do this.

>everyone is required to treat these URIs as identical.

Wrong.

>and therefore some will treat these URIs as identical but others

Wrong. No client should preform these steps, because they are
not sure whether servers would perform them or not.

>required to treat these URIs as different.

Correct. It's very similar to http://example.com/C-CEDILLA
and http://example.com/c-cedilla. You have no clue whether the
server will treat them as the same, or as different. So just
treating them as the same on the client side would be totally
wrong. The IRI spec explains that in detail. See
http://www.w3.org/International/iri-edit/draft-duerst-iri.html#normalization.

The URI spec doesn't say anything about Unicode normalization,
because it only deals with US-ASCII. http://example.com/%C3%87
could be a C-cedilla encoded in UTF-8, or it could be windows-1252,
with an A-tilde followed by a double dagger. Or it could be some
other encoding. Even if we know this is an IRI, we can't assume
for sure that it's encoded as UTF-8, because for backwards
compatibility, all URIs are IRIs. We can only know what characters
these are if they appear directly (i.e. unescaped).

Regards,    Martin.

---

From: Martin Duerst <[email-masked]>
Date: Sun, 01 Aug 2004 09:34:21 +0900

>spellings" if people are comfortable with that. -Tim

Well, Tim, maybe in the widest sense, you are right. There will be
implementers that don't read the spec and will try to use this equivalence,
in the same way there are probably implementers out there that try to
use case equivalence even when they are not supposed to.

I could for example understand quite well that some spiders take
http://example.com/UPPER and http://example.com/upper to be equivalent,
even though no spec whatsoever licences that. In a spider context,
there is still some chance that these are the same, and therefore some
work can be saved and some other pages can be downloaded instead.
Spiders may take this risk. But that's a risk they should take on
their own, knowing for themselves that they may be wrong. There is
no spec that licenses it.

Similarly, there is no spec that licenses B above. Indeed, I do not
know any server or client that would behave according to B above.
(we probably have to get the Apple server to behave that way,
because they store filenames decomposed internally, but

The IRI spec is very clear that producers should not produce
variant spellings, and should use NFC, or even NFKC where applicable.

We can repeat that in the Atom spec once we have decided on IRIs.

Regards,    Martin.