Subject: IRIs, URIs, and RFC 2396bis

From: Mark Pilgrim <[email-masked]>
Date: Wed, 7 Jul 2004 13:38:50 -0400

The Atom 0.3 draft [1] states that "Link constructs MUST have a href
attribute, whose value MUST be a URI [RFC2396]" [2].  Should this be
changed to "whose value MUST be a URI, as defined by RFC2396 or its
successor," so that Atom can benefit from RFC2396bis? [3]

Alternatively, we could state "whose value MUST be an IRI" [4].  After
reading all three specs (RFC 2396, RFC 2396bis, and the IRI draft), I
am not entirely sure what their relationship is.  RFC 2396bis appears
to obsolete RFC 2396, and IRIs appear to "have a dependency on RFC
2396bis" [5].  Currently Atom mandates RFC 2396 only.  I do not see
any discussion of this on the Atom wiki [6], and only one passing
mention of it in the list archives. [7]

Could someone with more experience shed some light on this situation?

[1] http://www.mnot.net/drafts/draft-nottingham-atom-format-02.html
[2] http://www.ietf.org/rfc/rfc2396.txt
[3] http://gbiv.com/protocols/uri/rev-2002/rfc2396bis.html
[4] http://www.w3.org/International/iri-edit/draft-duerst-iri.html
[5] http://lists.w3.org/Archives/Public/public-webarch-comments/2004Feb/0001.html
[6] http://intertwingly.net/wiki/pie/UniversalResourceIdentifier
[7] http://www.imc.org/atom-syntax/mail-archive/msg02941.html

-- 
Cheers,
-Mark

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Wed, 7 Jul 2004 11:16:20 -0700

>successor," so that Atom can benefit from RFC2396bis? [3]

Maybe.

>Alternatively, we could state "whose value MUST be an IRI" [4].

No. The IRI spec is not yet an RFC, and it is not at all clear when 
it will be. There is lots of contention about what it means and how 
it should be implemented. Basing our work on IRIs is very, very risky 
for no perceptible value.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Walter Underwood <[email-masked]>
Date: Wed, 07 Jul 2004 11:44:23 -0700

--On Wednesday, July 7, 2004 1:38 PM -0400 Mark Pilgrim <[email-masked]> wrote:
>
> After reading all three specs (RFC 2396, RFC 2396bis, and the IRI draft),
> I am not entirely sure what their relationship is.

Same here. The IRI spec is especially confusing.

wunder
--
Walter Underwood
Principal Architect, Verity

---

From: Tim Bray <[email-masked]>
Date: Wed, 07 Jul 2004 11:47:21 -0700

> successor," so that Atom can benefit from RFC2396bis? [3]

+1. 2396bis is very nearly finished, not controversial as far as I 
know, and immensely better than "2396 Classic".

> Alternatively, we could state "whose value MUST be an IRI" [4].

Less cooked, more controversial, I'd be nervous about this.

>   After
> am not entirely sure what their relationship is.

You are *not* the only one.

- Tim Bray, Director of Web Technologies, Sun Microsystems
   +1-877-305-0889 http://www.tbray.org/ongoing/
   AIM: MarkupPedant

---

From: Martin Duerst <[email-masked]>
Date: Thu, 08 Jul 2004 11:33:28 +0900

Hello Mark,

Thanks for asking these questions.

>Alternatively, we could state "whose value MUST be an IRI" [4].

In my view, this is the right thing to do.

What would be the actual difference?

For the users of the Atom format, it would allow to put into
the href attribute International Domain Names (IDNs) and path/document
names with non-ASCII characters, rather than to escape/encrypt them
as punicode (for IDNs) and %-escapes (for the rest).

For implementers, it would mean that whenever you dereference one of
these hrefs (i.e. follow a link), the following is needed:
1) If you don't work with UTF-8 internally (e.g. UTF-16), then
    convert the attribute to UTF-8. From then on, make sure your
    code is 8-bit clean.
2) Before handing it over to your URI resolver, escape non-ASCII
    characters with %HH.
3) Before resolving the domain name, revert the %-encoding, and
    hand over the domain name to an IDN library (various open-source
    implementations already available) for conversion to punycode.

1) is about one call to a library function that if you do XML, you'll
have around anyway. 2) is a few lines of code. 3) is a few lines
of code plus something like 200k in library code, mostly due to the
tables needed for nameprep. In terms of implementation, this should
be on the order of a few minutes to a few hours, depending on coding
skills. In terms of footprint, the IDN library can be a problem e.g.
on mobile phones, but should not be an issue on bigger systems.
I have done 3) for Amaya, and it was really no big deal.

I'd be glad to work with any project implementing Atom to help them
do IRIs; in my experience, such things go extremely easily if you
put have two people work together, one familliar with the existing
code and the other familliar with the addition being made.

I'd also be very glad to help with tests; Mark, if you can send
me a very simple example of an Atom feed with a few hrefs, and
the documents it points to, I can convert that to an example using
IRIs, and make sure the documents are correctly hosted so that
the tests are not affected by side-effects.

>am not entirely sure what their relationship is.

I hope that my mail to Tim explains this well enough. If not,
I'm glad to try again.

>RFC 2396bis appears
>2396bis" [5].

IRIs not only appear to have a dependency on 2396bis (which is not
an RFC, and if and when it becomes an RFC, it will have a new number),
they actually have a dependency on 2396bis. The IRI syntax parallels
the URI syntax, and it was simply thought a bad idea to leave out
all the fixes to the URI spec from the IRI spec.

Regards,    Martin.

>Currently Atom mandates RFC 2396 only.  I do not see
>[5] 
>http://lists.w3.org/Archives/Public/public-webarch-comments/2004Feb/0001.html

---

From: Martin Duerst <[email-masked]>
Date: Thu, 08 Jul 2004 10:42:08 +0900

Hello Tim, others,

At 11:47 04/07/07 -0700, Tim Bray wrote:

>+1. 2396bis is very nearly finished, not controversial as far as I know, 
>and immensely better than "2396 Classic".

I agree. The main problem is timing, but the list of issues that the
Atom spec has to go through and agree is vastly longer than the issue
remaining to be resolved for 2396bis, so I don't think this is an issue.

>Less cooked,

I'm very surprised to hear this! It is correct to say that the
IRI spec is less cooked than the URI spec, but the cooking of
the IRI spec started around 1997, so I don't really think 'not
enough cooking' is a valid argument. Also, the IRI spec has been
submitted to the IESG for further processing and approval after
a successful 2-week mailing list last call. Of course, this does
not guarantee that the IESG will approve the IRI spec, and there
is no guarantee when it will be published as an RFC, but the same
(and more) can easily be said for Atom at this point in time.

>more controversial,

Given that Opera, Safari, and Mozilla actually implement IRIs,
and that IE implements the parts it was able to implement when it
was put out,..., and given that the XML spec specifies what amounts
to IRIs (but wasn't called IRI at the time the XML spec was baked)
for system identifiers, and specs such as XLink and XML Schema
do the same, I'm not really sure what the big 'controversy' is
all about. Also, given that it took about 1 (read 'one') test case
to for example get Opera and Mozilla on the right track, and given
that I have never received any serious questions or comments regarding
the spec from actual implementers (as opposed to people just reading
the spec).

>I'd be nervous about this.

Would some test cases be able to reduce this nervousness?

>You are *not* the only one.

Okay. Maybe the following helps:

- RFC 2396bis is an update to RFC 2396, mostly finished, but still
   with one known open issue.

- The IRI spec defines IRIs as different from, but related, to URIs.
   IRIs can contain non-ASCII characters, URIs can't.
   IRIs can be used as *protocol elements* (see e.g. the list of XML
   specs above, and that's what Atom would do), or they can be used
   as *presentation elements* (e.g. if you type an URI with a long
   sequence of %-escapes into some browsers such as Opera, it will
   convert this for you in the location/address bar to meaningful
   non-ASCII characters if this is possible according to the IRI spec).

Please feel free to ask actual technical questions (rather than
just throwing non-technical conjectures into the air), or to
comment on the IRI spec (preferably copying [email-masked]
if you think something in the IRI spec needs improvement).

Regards,    Martin.

---

From: Mark Baker <[email-masked]>
Date: Wed, 7 Jul 2004 23:12:31 -0400

On Wed, Jul 07, 2004 at 01:38:50PM -0400, Mark Pilgrim wrote:
> Should this be

It's my understanding that "or its successor" is implied in RFCs.
That is, when one RFC obsoletes another, all references to the
obsoleted RFC are considered to be updated.

2026 doesn't say, but I'm sure I've heard that on at least a couple
of occasions.  Paul?

Mark.
-- 
Mark Baker.   Ottawa, Ontario, CANADA.        http://www.markbaker.ca

  Seeking work on large scale application/data integration projects
  and/or the enabling infrastructure for same.

---

From: Walter Underwood <[email-masked]>
Date: Wed, 07 Jul 2004 20:17:11 -0700

I don't agree with "2396 or its successor", because that means
the Atom spec changes when other specs change. What happens
when "2396ter" comes out?

We should specify "2396bis" and update the references when the
that spec is ready.

wunder

---

From: Julian Reschke <[email-masked]>
Date: Thu, 08 Jul 2004 08:34:10 +0200

> 2026 doesn't say, but I'm sure I've heard that on at least a couple

I've heard that as well; but it would be nice if somebody could clarify. 
  For instance, if RFCxx refers to a specific feature of RFCyy, and that 
feature gets dropped in RFCyy-bis, what does this mean?

Regards, Julian

-- 
<green/>bytes GmbH -- http://www.greenbytes.de -- tel:+492512807760

---

From: "Bob Wyman" <[email-masked]>
Date: Thu, 8 Jul 2004 11:21:38 -0400

> It's my understanding that "or its successor" is implied in RFCs.
	This would not be a good policy. It would mean that any RFC that has
a dependency on another would have an indeterminate definition. The
maintainers of such a specification would have no ability to manage the
evolution of their specification since it could, in essence, by modified at
anytime by some other group. RFC's are immutable and version-free for a
reason -- a good one. Once you know the content and meaning of RFC-XXXX, you
can go to your grave knowing that it is now and ever more shall be the same.
	The "or its successor" language should be explicitly provided if
desired and then only in special cases. I think the language code case is,
in fact, one of the cases where we can assume that a link to "successors"
is, in fact, safe. In many other cases, such an open ended dependency would
be chaotic and would probably result in people begin forced to copy the
content of other specifications into their own rather than referencing the
other specs.

		bob wyman

---

From: Mark Baker <[email-masked]>
Date: Thu, 8 Jul 2004 19:26:23 -0400

On Thu, Jul 08, 2004 at 11:21:38AM -0400, Bob Wyman wrote:
> a dependency on another would have an indeterminate definition.

Or it would mean that documents which obsolete other documents would
need to be developed with the utmost care for compatibility issues.
Oh wait, that's what happens. 8-)

Mark.

---

From: Sam Ruby <[email-masked]>
Date: Fri, 09 Jul 2004 07:13:23 -0400

> Oh wait, that's what happens. 8-)

Counter example, from The Atom Syndication Format 0.3 (PRE-DRAFT) [1]:

     the canonical serialization of an Atom document is XML 1.0, and this
     is the only serialization that can be identified with the
     "application/atom+xml" media type.

- Sam Ruby

[1]<http://www.mnot.net/drafts/draft-nottingham-atom-format-02.html#rfc.section.2>

---

From: Mark Baker <[email-masked]>
Date: Fri, 9 Jul 2004 07:43:49 -0400

Sorry, I just meant RFCs which obsolete other RFCs.  And by "that's
what happens", I mean that's what I've observed in the IETF in the past
viz a viz RFCs 2068/2616 and 2396/2396bis.

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Fri, 9 Jul 2004 15:43:16 -0700

>2026 doesn't say, but I'm sure I've heard that on at least a couple

Sorry for the delay in getting to this. Talking about references to 
RFCs in general here.

If we say "or its successor", we get that. If we don't say it, we 
point *exactly* to the RFC in question. Successors are *not* implied.

This is very important for many reasons. Successors might change some 
bits-on-the-wire; that would be Very Bad for us. Successors might 
change only some unclear parts, but those changes might be things 
that we as a group thought we understood.

That is why very few RFCs ever refer to other "and its successors" of an RFC.

In the specific case, if we point to RFC2396, that's what we get. If 
we point to the Internet Draft of the successor to RFC 2396, that's 
what we get. In the latter case, we also cannot finish with Atom 
until that document is finished. And before you say "oh, but 
rfc2396bis is far ahead of us", remember that they have been far 
ahead of us for years and are still not finished.

This is one of the un-fun things of making standards: guessing which 
thing that we rely on are going to be out of the oven before us. Tim 
and I will look at this later in the baking process.

--Paul Hoffman, Director

---

From: Mark Baker <[email-masked]>
Date: Sun, 11 Jul 2004 07:33:25 -0400

Hi,

I've just been chatting with Roy, and he agrees that this issue isn't
as simple as either "successors are implied" or "successors are not
implied".  Luckily, we can avoid that discussion if, as he suggests,
we use a reference to 2396bis only rather than 2396 *and* its
successor.  Directions to the RFC editor in this respect will also
have to be provided.

As you say though Paul, 2396bis' schedule isn't clear.  So perhaps we
can do this for now as what, IMO, seems like the best way forward, but
be prepared to change back to 2396 if it faulters (which I can't
believe will happen, but stranger things have happened).

Mark.