Subject: URI Comparisons (something practical)

From: Danny Ayers <[email-masked]>
Date: Thu, 2 Sep 2004 21:32:35 +0200

Has anyone yet tried the equivalence tests implied in RFC2396bis with
the APIs they're using?

For example:

import java.net.URI;
import java.net.URL;

public class URITest {

    public static void main(String[] args) {
        try {
            URL urlA = new URL("http://example.org");

            URL urlB = new URL("HTTP://example.org"); 
            
            System.out.println(urlA.equals(urlB));
          
            URI uriA = new URI("http://example.org");

            URI uriB = new URI("HTTP://example.org");

            System.out.println(uriA.equals(uriB));

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Produces:

true
true

Other comparisons can be seen in PaceCanonicalIds.

I've changed my mind on canonicalization myself once already (it
sounded too much work at first), perhaps there's a way to consensus
through practical code..?

If we mandated char-by-char comparison alone (so HTTP://example.org
not equivalent to http://example.org) then the Java developer couldn't
use the built-in URI/URL classes directly. So (I hear you say) why not
just do .toString() prior to comparison? Well, say I, what about the
normalization of dots and so on? Is this confusing? Would it be any
more work to do this normalization when the ids were generated?

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: Mark Pilgrim <[email-masked]>
Date: Thu, 2 Sep 2004 15:53:23 -0400

> Has anyone yet tried the equivalence tests implied in RFC2396bis with

http://intertwingly.net/blog/2004/07/31/URI-Equivalence

-- 
Cheers,
-Mark

---

From: Jeremy Smith <[email-masked]>
Date: Thu, 02 Sep 2004 16:13:31 -0400

#!/usr/bin/perl
use warnings;
use strict;
use URI;

foreach my $result 
   (URI::eq('http://example.org',          'http://example.org/'        ),
    URI::eq('http://example.org',          'HTTP://example.org'         ),
    URI::eq('http://example.org',          'http://example.org:'        ),
    URI::eq('http://example.org',          'http://example.org:80'      ),
    URI::eq('http://example.org',          'http://Example.org'         ),
    URI::eq('http://example.org/~smith',   'http://example.org/%7Esmith'),
    URI::eq('http://example.org/~smith',   'http://example.org/%7esmith'),
    URI::eq('http://example.org/%7Esmith', 'http://example.org/%7esmith'),
    URI::eq('http://example.org/%C3%87',   'http://example.org/C%CC%A7' ), )
{
    print $result ? "true\n" : "false\n";
}

__OUTPUT__
true
true
true
true
true
true
true
true
false

---

From: Danny Ayers <[email-masked]>
Date: Thu, 2 Sep 2004 22:17:04 +0200

Thanks, I missed that.

Just a tad ironic that the C# version is more accurate, if it has to
be ditched in favour of char-by-char comparison.

-- 

http://dannyayers.com

---

From: Julian Reschke <[email-masked]>
Date: Thu, 02 Sep 2004 22:24:47 +0200

Jeremy Smith wrote:

> #!/usr/bin/perl
> use warnings;
> use strict;
> use URI;
> 
> foreach my $result   (URI::eq('http://example.org',          
> 'http://example.org/'        ),

Speaking of which -- why would anybody want the last comparison to 
evaluate to "true"?

Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: Mark Pilgrim <[email-masked]>
Date: Thu, 2 Sep 2004 18:39:29 -0400

> Speaking of which -- why would anybody want the last comparison to

Because they are percent-encoded representations of UTF-8
representations of the same character, for some definition of "same". 
The first is a fully composed Cedilla, the second is a fully
decomposed Cedilla (the letter C, followed by a combining character
that adds the tail).  Browsers would print these two characters
identically on screen, and Unicode-aware searching/sorting algorithms
would treat them as identical.

-- 
Cheers,
-Mark

---

From: Bill de hÓra <[email-masked]>
Date: Fri, 03 Sep 2004 01:38:01 +0100

> more work to do this normalization when the ids were generated?

It probably doesn't matter:

<fubar
     xmlns:f="http://www.example.com/GUID-uses-ABCDEF"
     xmlns:u="http://example.com/GUID-uses-ABCDEF"
     xmlns:b="http://www.example.com/guid-uses-abcdef"
     xmlns:a="http://WWW.EXAMPLE.COM/GUID-uses-ABCDEF"
     xmlns:r="http://WWW.EXAMPLE.COM/GUID-uses-ABCDEF/../../p0wn3d!">
   <f:f/>
   <u:u/>
   <b:b/>
   <a:a/>
   <r:r/>
</fubar>

that's ok XML; namespaces rules for URIs don't work like the web's. 
For atom:id I imagine we want something closer to RDF/namespaces 
than HTTP resolution.

In other words, URLs with screwy paths are ok for identification.

Personally I don't use the URI/L classes in Java for much. They've 
been  consistently buggy since forever. Pointing people towards 
toString() would be a kindness. Consider some more test goodness:

   import java.net.URI;
   import java.net.URL;
   public class URITest {

         URI uriA1 = new URI("http://EXAMPLE.org");
         URI uriB1 = new URI("HTTP://example.org");
         System.out.println(uriA1.equals(uriB1));

         URL urlA1 = new URL("http://example.ORG");
         URL urlB1 = new URL("HTTP://example.org");
         System.out.println(urlA1.equals(urlB1));

       } catch (Exception e) {e.printStackTrace();}
     }
   }

produces,

true
true
true
true

on 1.4.2_03-b02.

cheers
Bill

---

From: Julian Reschke <[email-masked]>
Date: Fri, 03 Sep 2004 09:10:49 +0200

> would treat them as identical.

Thanks. I (temporarily) forgot about the added complexity of Unicode 
normalization.

I'm still not sure whether I would want to see that level of processing 
when using URIs. URIs are ASCII, and there's no guarantee whatsoever 
that %-escaped sequences can be mapped back to UTF8 sequences (although 
that would have been nice in retrospective).

Best regards, Julian

---

From: Mark Pilgrim <[email-masked]>
Date: Fri, 3 Sep 2004 10:01:12 -0400

> that would have been nice in retrospective).

From http://www.gbiv.com/protocols/uri/rev-2002/draft-fielding-uri-rfc2396bis-06.html#identifying-data

"When a new URI scheme defines a component that represents textual
data consisting of characters from the Unicode character set [UCS],
the data should be encoded first as octets according to the UTF-8
character encoding [STD63], and then only those octets that do not
correspond to characters in the unreserved set should be
percent-encoded. For example, the character A would be represented as
"A", the character LATIN CAPITAL LETTER A WITH GRAVE would be
represented as "%C3%80", and the character KATAKANA LETTER A would be
represented as "%E3%82%A2"."

-- 
Cheers,
-Mark

---

From: Julian Reschke <[email-masked]>
Date: Fri, 03 Sep 2004 16:17:15 +0200

Mark Pilgrim wrote:

>...

True, but how does it apply to HTTP (and other "legacy" schemes)?

Julian

---

From: Martin Duerst <[email-masked]>
Date: Sun, 05 Sep 2004 11:22:24 +0900

At 09:10 04/09/03 +0200, Julian Reschke wrote:

>>representations of the same character, for some definition of "same". The 
>>first is a fully composed Cedilla, the second is a fully
>when using URIs.

I agree. There is no spec that requires that. The IRI spec has some
language that should result in the NFC/NFKC version of the URI being
produced in the first place, but there is always the chance that
a decomposition sneaks in somehow.

>URIs are ASCII, and there's no guarantee whatsoever that %-escaped 
>sequences can be mapped back to UTF8 sequences (although that would have 
>been nice in retrospective).

There is no guarantee. There is, however, a high chance that
%-encoded sequences that correspond to legal UTF-8 octet sequences
are actually originally UTF-8.

Regards,    Martin.