Subject: Re: [whatwg] New URL Standard from Anne van Kesteren on 2012-09-24 ([email-masked] from September 2012)

From: Ian Hickson <[email-masked]>
Date: Wed, 17 Oct 2012 23:57:38 +0000 (UTC)

On Mon, 24 Sep 2012, Peter Saint-Andre wrote:
>
> It's also not clear to me what the WHATWG HTML Living Standard [1] 
> really means by "willful violation"
> http://www.whatwg.org/specs/web-apps/current-work/multipage/introduction.html#willful-violation

Can you elaborate on how the definition you cite there in the spec is 
unclear? I'm not sure how to make it clearer.

> e.g., is it just an allowance for APIs and browser software to not be 
> completely strict about processing some input for the sake of backward 
> compatibility with existing (messy) web content, or is it an active 
> attempt at redefining core protocols?

The spec seems pretty clear that it's the latter ("conflicting needs have 
led to this specification violating the requirements of these other 
specifications").

> However, it is interesting that the willful violations are not limited 
> to RFC 3986: the spec also mentions willful violations of RFC 2046, RFC 
> 2616, RFC 2781, RFC 5322, EcmaScript, XPath, XSLT, and Unicode. Quite a 
> list...

Yeah. Turns out we (the Web standards community) haven't been doing such a 
great job of making our specificatiosn match reality. :-(

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

---

From: Noah Mendelsohn <[email-masked]>
Date: Wed, 17 Oct 2012 21:25:18 -0400

On 10/17/2012 7:57 PM, Ian Hickson wrote:
> great job of making our specificatiosn match reality.:-(

Um, true... but it's also the case that the implementation community hasn't 
over the years been doing as much as might be best to make reality match 
the specifications. The new specs we're writing now would like have been a 
lot thinner and cleaner if they had.

Noah

---

From: Ian Hickson <[email-masked]>
Date: Thu, 18 Oct 2012 01:29:13 +0000 (UTC)

On Wed, 17 Oct 2012, Noah Mendelsohn wrote:
> > Yeah. Turns out we (the Web standards community) haven't been doing 
> > such a great job of making our specificatiosn match reality.:-(
> 
> Um, true... but it's also the case that the implementation community 
> hasn't over the years been doing as much as might be best to make 
> reality match the specifications. The new specs we're writing now would 
> like have been a lot thinner and cleaner if they had.

At least on the browser front, having been on those front lines for much 
of my career, I don't really know what more we could have done.

But in any case, that's somewhat irrelevant now, since it's too late to 
ge back and change it.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 22 Oct 2012 17:03:08 +0100

Hi Noah,

On 18/10/2012 02:25, Noah Mendelsohn wrote:
>> such a

However, I think the concern here is that if certain IETF specs need to be
updated to match the real world, that work needs to be done in the IETF.
It can't be done by reference in a W3C document; it has to be done by value
in an IETF document.

So it seems to me there is an open liaison question here and I'd like to hear
from the IETF/W3C liaison how this will be resolved.

Regards
    Brian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 22 Oct 2012 20:59:41 +0200

On 2012-10-22 19:55, Ian Hickson wrote:
> On Mon, 22 Oct 2012, Brian E Carpenter wrote:
>>> reality match the specifications. The new specs we're writing now
>>> would like have been a lot thinner and cleaner if they had.
>>
>> However, I think the concern here is that if certain IETF specs need to
>> be updated to match the real world, that work needs to be done in the
>> IETF.
>
> I couldn't agree more! We've been waiting for four years for the URI
> working group to get their act together and fix the URL mess. Nothing has
> happened. We lost patience and are now doing it ourselves.
> ...

Clarifying: there is no URI Working Group, and as far as I can tell, 
there is no consensus that there is a "mess" to fix related to URIs. 
There *is* indeed a *IRI* Working Group which planned to define things 
that occur in HTML links (including sanitizing whitespace and dealing 
with the problems with non-ASCII characters), and *that* WG indeed 
hasn't delivered yet.

Best regards, Julian

---

From: Tim Bray <[email-masked]>
Date: Mon, 22 Oct 2012 15:11:38 -0700

On Mon, Oct 22, 2012 at 2:46 PM, Ian Hickson <[email-masked]> wrote:

> On Mon, 22 Oct 2012, Roy T. Fielding wrote:
>>
>> What you are insisting on defining as a "URL" is the input to the
>> process of making a hypertext reference (the arbitrary string typed into
>> a dialog or placed inside an href/src attribute)
>
> Or placed on a command line to wget(1), or put in an RDFa triple store, or
> in transmitted in an HTTP Location: header, or...

It seems reasonable that someone should write rules for dealing with the
kinds of errors that are observed to occur in links as embedded in resource
representations AKA HTML pages.  It also seems reasonable that WHATWG, who
speak (if I understand correctly) for browser builders, can write those
rules.

The notion that curl, or an HTTP cache manager, or an XML namespace
processor, is going to be routing around errors, strikes me on the face of
it as being wrong.  One of the main uses I put curl to is making sure I
have the URL exactly right before I drop it into chat or whatever.   -T

---

From: Julian Reschke <[email-masked]>
Date: Tue, 23 Oct 2012 00:12:10 +0200

On 2012-10-22 23:46, Ian Hickson wrote:
> On Mon, 22 Oct 2012, Julian Reschke wrote:
>>>
>>> working group to get their act together and fix the URL mess. Nothing
>>> has happened. We lost patience and are now doing it ourselves. ...
>>
> Whoever. The people complaining that it should be done at the IETF haven't
> done any work. That's the complaint. Until they do the work, complaining
> that we're doing it instead is going to fall on deaf ears and be met with
> the rolling of eyeballs.

This always was about venue, not people. If people want to "fix" or 
"augment" URIs/IRIs, they should come over to the IETF. That's where the 
specs live. The IETF is open to anyone, works async on mailing lists, 
and doesn't require any membership fees. I don't think there's any 
standards body that is *more* open to individuals.

But yes, you may have to convince a few people outside the WhatWG. 
That's a feature. It means more review from people outside the browser 
ecosystem.

> The specs don't define everything that implementations have to do to be
> interoperable. If the IETF doesn't think that's a problem, then that's
> fine, but then y'all shouldn't be surprised when people who _do_ think
> that's a problem try and fix it.

Yes, please fix *that*, but *just* that without messing with the basics 
without consensus/review.

So yes: if you feel you need to make \ to equivalent to /, that may be 
ok (as \ isn't valid anyway). But changing the reference resolution 
algorithm for valid URI/reference pairs is something entirely different. 
*If* it needs to be done, it needs to be done within the scope of the 
URI spec.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Oct 2012 09:14:26 +1100

On 23/10/2012, at 9:11 AM, Tim Bray <[email-masked]> wrote:

> It seems reasonable that someone should write rules for dealing with the kinds of errors that are observed to occur in links as embedded in resource representations AKA HTML pages.  It also seems reasonable that WHATWG, who speak (if I understand correctly) for browser builders, can write those rules.

+1. The work seems obviously necessary, the confusion is around why this is being couched in terms of redefining URIs, rather than just defining it as "how you get to a URI (or an unrecoverable error) from an arbitrary string."

Cheers,

--
Mark Nottingham   http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Oct 2012 09:39:44 +1100

On 23/10/2012, at 9:35 AM, Ian Hickson <[email-masked]> wrote:

> Consensus isn't a value I hold highly, but review of Anne's work is 
> welcome.
> 
> If the IETF community didn't want Anne to do this work, then the IETF 
> community should have done it. Having not done it, having not even 
> understood that the problem exists, means the IETF has lost the 
> credibility it needs to claim that this is in the IETF's domain.
> 
> You don't get to claim authority over an area while at the same time 
> telling someone else "please fix that" for the hard work that comes with 
> that area. The reality is, he who does the hard work, gets the authority.

All very interesting, but please address the point that's now been made repeatedly -- why is it necessary for you to redefine URIs, rather than doing as we suggest?

Regards,

---

From: Tim Bray <[email-masked]>
Date: Mon, 22 Oct 2012 16:17:29 -0700

On Mon, Oct 22, 2012 at 3:35 PM, Ian Hickson <[email-masked]> wrote:

> > processor, is going to be routing around errors, strikes me on the face
> > of it as being wrong.  One of the main uses I put curl to is making sure
> > I have the URL exactly right before I drop it into chat or whatever.
>
>    $ wget 'http://example.com/a b'
>    --2012-10-23 00:27:43--  http://example.com/a%20b
>
>    # test.cgi returns a 301 with "Location: a b"
>    $ curl -L
> http://damowmow.com/playground/demos/url/in-http-headers/test.cgi
>    This file is:
> http://damowmow.com/playground/demos/url/in-http-headers/a%20b
>
> Software does this stuff already. We need to make sure it does it
> interoperably.
>

Hmm.  I went to tbray.org and made a file at '$ROOT_DIR/tmp/a b' - note the
space.

Then I did

curl -I 'http://www.tbray.org/tmp/a%20b'
curl -I 'http://www.tbray.org/tmp/a b'

Curl, quite properly, doesn’t fuck with what I ask it, and revealed a very
interesting fact: That my Apache httpd returns 200 for both of these, but,
with, uh, interesting variations, amounting to what I think is quite
possibly a bug.  I also pasted the version with the space into the nearest
Web browser, and it quite properly auto-corrected to a%20b.

I think it’s a bug that curl is claiming the 301 pointed at "a%20b" not "a
b".  Because suppose it had pointed at "a%20b" - I don’t want middleware
lying to me.

It seems like a good idea to document the steps by which "a b" pasted in
becomes "a%20b" in the address bar. But I don’t see the relevance outside
human-authored strings.  -T

>
>
> On Tue, 23 Oct 2012, Julian Reschke wrote:
> >
> > "augment" URIs/IRIs, they should come over to the IETF.
>
> I think the person doing the work has the prerogative to do it wherever he
> or she wants to do it. Maybe the IETF should consider why Anne isn't doing
> it in the IETF.
>
>
> > > The specs don't define everything that implementations have to do to
> > > be interoperable. If the IETF doesn't think that's a problem, then
> > > that's fine, but then y'all shouldn't be surprised when people who
> > > _do_ think that's a problem try and fix it.
> >

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Oct 2012 10:32:54 +1100

On 23/10/2012, at 10:25 AM, Ian Hickson <[email-masked]> wrote:
> 
> What exactly do you suggest? 
> 
> Doing the work but at the IETF? See my reply to James.

Don't much care about the venue, as long as there's *some* coordination / communication.

> Waiting for the IETF to do the work? We did, and timed out.

Understood, and unfortunate. Arguably, you waited longer than the timeout.

> Not doing the work? That doesn't lead to interop.

Absolutely - again, I don't see anyone suggesting that. Do I smell straw?

> Doing the work as a diff spec? That's what we did for a while, but it 
> doesn't work. Having to reference three specs (pre-parse, IRI, URI) just 
> to parse and resolve a URL is not what leads to implementors having a good 
> time and thus not what leads to interop.

Really? You're comfortable with the current weight and depth of the HTML5 spec, but balk at a pre-processing step for URIs? Seriously?

The underlying point that people seem to be making is that there's legitimate need for URIs to be a separate concept from "strings that will become URIs." By collapsing them into one thing, you're doing those folks a disservice. Browser implementers may not care, but it's pretty obvious that lots of other people do.

BTW, it doesn't have to be a separate spec, although it probably would benefit from being one. Browser implementers already have to reference TCP, IP, DNS, and likely tens to hundreds of other specs to get what they want done -- unless you have bigger plans?

Regards,

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Oct 2012 10:52:55 +1100

On 23/10/2012, at 10:40 AM, Ian Hickson <[email-masked]> wrote:

> On Tue, 23 Oct 2012, Mark Nottingham wrote:
>> 
> Everyone is welcome to participate in the WHATWG list.

As they are on the IETF list. The difference is that the WHATWG is run by an unelected board of "members" - <http://www.whatwg.org/charter>. 

> There's no reason to have three specs when one suffices.

OK. 

> The underlying point that people seem to be making is that there's 
>> legitimate need for URIs to be a separate concept from "strings that 
>> will become URIs."
> 
> Anne's spec will define "valid URL", which addressed that need.

Why not define (or reuse) a separate term for the input stream, and leave "URL" alone?

> By collapsing them into one thing, you're doing those folks a 
>> disservice.
> 
> They are not collapsed into one thing.

OK, good.

>> Browser implementers may not care, but it's pretty obvious that lots of 
>> other people do.
> 
> Browser implementors aren't particularly special here.

No, but your arguments are often coloured by your perspective -- just as everyone else's are. 

> BTW, it doesn't have to be a separate spec, although it probably would 
>> benefit from being one. Browser implementers already have to reference 
>> TCP, IP, DNS, and likely tens to hundreds of other specs to get what 
>> they want done -- unless you have bigger plans?
> 
> The difference is that the DNS implementor doesn't need to implement TCP, 
> he uses TCP (and UDP) and builds on it. And so on. Whereas here we're 
> talking about one thing, URLs, being specified in one place vs three.

OK.

If I believed that Anne was willing to and capable of re-specifying RFC3986 in such a way that the definition, syntax and semantics of URLs (or whatever they ends up being called) doesn't change at all, I'd be less concerned. 

However, that doesn't seem very likely, especially when he isn't engaging with the folks that wrote that spec (especially, Roy). 

RFC3986 is referenced by a LOT of technologies, not just Web browsers, not just HTML. Replacing it unilaterally with input from the browser / HTML community from an implementer perspective is very likely to break most of them.

As such, they won't use your new spec, and we'll be living in a world where there will be two definitions of "URL" -- the IETF one and the WHATWG one (or perhaps the W3C one, I can never remember the relationship there). 

That seems a pretty bad tradeoff for the benefits you're getting -- a slightly easier-to-read spec for browser implementers (a relatively tiny audience).

Regards,

---

From: Tim Bray <[email-masked]>
Date: Mon, 22 Oct 2012 17:15:41 -0700

One more data point... I work on Web software all the time and have for
many years; in recent years mostly at the REST (app-to-app HTTP
conversations) rather than browser-wrangling level.  I’d have to say that
URI interoperability problems haven’t come near getting into the list of
top-20 pain points.   So either my experience is wildly untypical, or maybe
it’s a combination of being a little bit lucky, and that the pain which
exists is highly concentrated in the browser space.  -T

On Mon, Oct 22, 2012 at 5:05 PM, Ian Hickson <[email-masked]> wrote:

> > As they are on the IETF list. The difference is that the WHATWG is run
> > by an unelected board of "members" - <http://www.whatwg.org/charter>.
>
> "Run" is a bit of a strong word. There's basically no non-public activity
> from the charter members.
>
>
> Because everyone calls these things URLs (except STD 66).
>
>
> > >> Browser implementers may not care, but it's pretty obvious that lots
> > >> of other people do.
> > >
> Which arguments in particular are we talking about here? I've mostly been
> talking about curl, wget, GoogleBot, Perl libraries, etc.
>
>
> > If I believed that Anne was willing to and capable of re-specifying
> > RFC3986 in such a way that the definition, syntax and semantics of URLs
> > (or whatever they ends up being called) doesn't change at all, I'd be
> > less concerned.
> >
> > However, that doesn't seem very likely, especially when he isn't
> > engaging with the folks that wrote that spec (especially, Roy).
> >
> > RFC3986 is referenced by a LOT of technologies, not just Web browsers,
> > not just HTML. Replacing it unilaterally with input from the browser /
> > HTML community from an implementer perspective is very likely to break
> > most of them.
>
> I suspect it will break nothing, but I guess we'll find out.
>
> I don't really understand how it _could_ break anything, so long as the
> processing of IRI and URIs as defined by IETF is the same in the WHATWG
> spec, except where software already differs with the IETF specs.
>
> Do you have a concrete example I could study?
>
>
> > As such, they won't use your new spec, and we'll be living in a world
> > where there will be two definitions of "URL" -- the IETF one and the
> > WHATWG one [...].
> >
> > That seems a pretty bad tradeoff for the benefits you're getting -- a
> > slightly easier-to-read spec for browser implementers (a relatively tiny
> > audience).
>
> If you have any concrete concerns, please don't hesitate to e-mail the
> WHATWG list, showing the specific examples you're worried about. Browsers
> are but one of many implementation classes that are relevant.
>
> --

---

From: Julian Reschke <[email-masked]>
Date: Tue, 23 Oct 2012 08:26:55 +0200

> spec, except where software already differs with the IETF specs.

Define "software". *All* software? How do you test that?

> Do you have a concrete example I could study?

Do you?

This brings me back to something I've been asking for many times: a 
*concrete* list of things that are "broken" in RFC 3986 (as opposed to 
be "undefined").

Best regards, Julian

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 17:55:42 +0000 (UTC)

I'm sure Anne would love nothing more than to be able to work on something 
more interesting that this. But at the end of the day, someone has to do 
it, and y'all aren't doing it.

This should not come as a surprise to anyone, the IETF and W3C have been 
discussing this matter at last as far back as 2008.

> It can't be done by reference in a W3C document; it has to be done by 
> value in an IETF document.

Currently it's only being done in a WHATWG document.

---

From: Noah Mendelsohn <[email-masked]>
Date: Mon, 22 Oct 2012 14:29:50 -0400

> be updated to match the real world, that work needs to be done in the

That's certainly an important concern, but it's not the one I had in mind 
when I made my comment.

Rather, while Ian is right that our new specifications must match today's 
"reality", we should also be sobered by the price we're paying for years of 
implementations that do things like sniffing. Yes, given that it's become 
widespread, documenting ways to do it interoperably and securely is good. 
I'm not convinced that implementing sniffing in widely-used user agents was 
the right choice way back when.

So, let's try harder than we have in the past to not introduce yet more 
such cruftiness, even if we do have to document what's there. Maybe we 
should have been a little more hesitant to do things like having both SVG 
and Canvas as ways of doing graphics. Perhaps that overlapping function is 
ultimately justified, but there's a long term cost. We're moving too far 
away from KISS IMO.

HTML5 is great work, but the spec is very, very thick. Keeping things as 
small and simple as we can moving forward is really what I had in mind.

Noah

---

From: "Roy T. Fielding" <[email-masked]>
Date: Mon, 22 Oct 2012 14:00:14 -0700

What you are insisting on defining as a "URL" is the input to the
process of making a hypertext reference (the arbitrary string typed
into a dialog or placed inside an href/src attribute), whereas the
IETF standards define the output of that process as a uniform
addressing syntax for use on the Internet by every application that
makes use of Web addresses.  Browsers implement both the input
processing and the output URI standards.  HTML must define the
input processing, either within the spec or by reference to a
new spec. The IETF defined the output standards for what goes
on the wire.

I would love to have a single standard for hypertext references.
However, they are not URLs, they are not implemented consistently
across HTML elements, and even within single elements they are
not implemented consistently across browsers.  It is a worthwhile
area to pursue further consistency, but only if you stop referring
to them as something they are not and never will be. "" is not a URL.

> discussing this matter at last as far back as 2008.

Yes, we have been discussing it since 1994.  It would be nice
if you would take the advice already received and define
references in HTML, including the algorithms for converting them
into URI references (for DOM and network usage) and IRI
references (for display).

There is no need for any willful violations, since you aren't
even defining the same things.

....Roy

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 21:46:56 +0000 (UTC)

> whereas the IETF standards define the output of that process as a 
> uniform addressing syntax for use on the Internet by every application 
> that makes use of Web addresses.

That's what Anne is specifying, because, despite your claims, STD 66 
doesn't actually define that. It only defines processing for valid 
strings, not invalid ones.

> Browsers implement both the input processing and the output URI 
> standards.  HTML must define the input processing, either within the 
> spec or by reference to a new spec.

This has nothing particularly to do with HTML, HTML is just one of many 
many contexts in which URLs are found.

> "" is not a URL.

Whether it's a valid URL or not is besides the point (it's like people 
claiming that invalid XML files aren't XML, a pointless argument). We 
still have to define the processing for such a string when software is to 
process it as a URL.

> > I'm sure Anne would love nothing more than to be able to work on 
> > something more interesting that this. But at the end of the day, 
> > someone has to do it, and y'all aren't doing it.
> > 
> > This should not come as a surprise to anyone, the IETF and W3C have 
> > been discussing this matter at last as far back as 2008.
> 
> Yes, we have been discussing it since 1994.

19 years of not fixing the problem, then. Shouldn't come as a surprise 
that someone has finally gotten around to doing it.

> It would be nice if you would take the advice already received and 
> define references in HTML, including the algorithms for converting them 
> into URI references (for DOM and network usage) and IRI references (for 
> display).

As far as HTML goes, my plan is to remove all references to STD 66 and 
rely entirely on Anne's work, so that there doesn't have to be any 
preprocessing nonsense. Then the STD 66 RFCs are entirely irrelevant.

---

From: Jan Algermissen <[email-masked]>
Date: Tue, 23 Oct 2012 00:04:42 +0200

Ian,

On Oct 22, 2012, at 11:46 PM, Ian Hickson wrote:

> Whether it's a valid URL 

the question if not whether an empty string is a valid URI.

The point is that what you and Anne are addressing is parsing of URI *References* not URIs.

Roy, IIUC, meant that "" is a URI *reference* - it is not a valid or invalid URI.

This is why any references to fixing or aligning URI syntax with reality is besides the point and not neccessary. All that you (we) deal with is URI references and how to parse them to yield valid URIs.

Jan

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 22:35:16 +0000 (UTC)

> The point is that what you and Anne are addressing is parsing of URI 

Anne's spec defines how you get from any arbitrary string (plus a base 
URL) to a data structure with fields like scheme, hostname, port, path, 
etc. The input can be absolute, completely invalid, the empty string, 
whatever.

> This is why any references to fixing or aligning URI syntax with reality 
> is besides the point and not neccessary. All that you (we) deal with is 
> URI references and how to parse them to yield valid URIs.

That's certainly part of the required work, yes. It's not all of it.

   # test.cgi returns a 301 with "Location: a b"
   $ curl -L http://damowmow.com/playground/demos/url/in-http-headers/test.cgi
   This file is: http://damowmow.com/playground/demos/url/in-http-headers/a%20b

---

From: James M Snell <[email-masked]>
Date: Mon, 22 Oct 2012 15:51:06 -0700

> etc. The input can be absolute, completely invalid, the empty string,
Sounds useful but does not sound really like Anne's spec is "defining
URLs", however. Clarifying the language in your spec ought to resolve any
possible confusion.

> That's certainly part of the required work, yes. It's not all of it.
Is there a list of issues that you and Anne are working from for this? If
there indeed is a need to update the URI/IRI RFC's to address specific
problems I'm sure it wouldn't take much effort to draft up an I-D. I'd be
more than willing to help out with such an effort.

> [snip]
Indeed. Good question: Anne, is there are particular reason why you chose
not to pursue this work as an I-D? Let's get that particular issue resolved.

- James

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 23:16:19 +0000 (UTC)

On Mon, 22 Oct 2012, James M Snell wrote:
>
> Is there a list of issues that you and Anne are working from for this? 
> If there indeed is a need to update the URI/IRI RFC's to address 
> specific problems I'm sure it wouldn't take much effort to draft up an 
> I-D. I'd be more than willing to help out with such an effort.

See the spec. He's documented what needs to be done. The big one that he 
hasn't done yet is defining "valid URL" (the "Writing" section), which is 
the bulk of what STD 66 and the IRI RFC do.

> > Maybe the IETF should consider why Anne isn't doing it in the IETF.
>
> Indeed. Good question: Anne, is there are particular reason why you 
> chose not to pursue this work as an I-D? Let's get that particular issue 
> resolved.

(I don't think Anne is on any of the cc'ed mailing lists.)

I can't speak for Anne, but having experienced the IETF via the hybi work, 
my own opinion is that the main reason I wouldn't work with the IETF is 
that the community these days values consensus over technical value and 
running code, and the culture in the IETF doesn't value the kind of 
specification style that IMHO leads to better interop. For example, this 
very thraed -- we're having to argue to convince people that defining 
error handling is even a valuable thing to do. I have no interest in 
attempting to get anything done in an environment where that's the level 
at which the conversation starts.

Note that this isn't the only work recently that has gone from the IETF to 
the WHATWG. For example, the MIME Sniffing specification, which was 
originally in the HTML spec, was moved to the IETF by Adam Barth, who has 
done other work in the IETF successfully (e.g. Cookies). But in the end, 
Adam gave up on specifying MIME sniffing in the IETF and moved it out, and 
now it's found a home at the WHATWG:

   http://mimesniff.spec.whatwg.org/

The character encodings registry at IETF/IANA has also recently been, to 
some extent, suplanted by work at the WHATWG after failed attempts at 
getting the specs and registries at the IETF/IANA into better shape:

   http://encoding.spec.whatwg.org/

I spoke to Anne briefly and he pointed me to:

   http://wiki.whatwg.org/wiki/IETF

...which gives a very brief synopsis of why he's given up on the IETF.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Oct 2012 10:18:47 +1100

On 23/10/2012, at 10:16 AM, Ian Hickson <[email-masked]> wrote:

> error handling is even a valuable thing to do.

Wait - who's making that argument? References, please.

> I have no interest in

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 23:25:45 +0000 (UTC)

On Tue, 23 Oct 2012, Mark Nottingham wrote:
> > telling someone else "please fix that" for the hard work that comes 
> > with that area. The reality is, he who does the hard work, gets the 
> > authority.
> 
> All very interesting, but please address the point that's now been made 
> repeatedly -- why is it necessary for you to redefine URIs, rather than 
> doing as we suggest?

What else do you suggest?

On Mon, 22 Oct 2012, Tim Bray wrote:
> Hmm.  I went to tbray.org and made a file at '$ROOT_DIR/tmp/a b' - note 
> the space.
> 
> Then I did
> 
> Curl, quite properly, doesn't fuck with what I ask it

Instead it makes an invalid HTTP request. Your offensive language 
notwithstanding, that means wget and curl don't interoperate. This is bad. 
This is what we want to fix.

> and revealed a very interesting fact: That my Apache httpd returns 200 
> for both of these, but, with, uh, interesting variations, amounting to 
> what I think is quite possibly a bug.

How could it be a bug, since there's no spec that says how to handle a URL 
with spaces in it?

> I also pasted the version with the space into the nearest Web browser, 
> and it quite properly auto-corrected to a%20b.

Quite properly according to whom? There's no spec that defines this.

> I think it’s a bug that curl is claiming the 301 pointed at "a%20b" not 
> "a b".

You're wrong, but only because the de facto standard of "most software 
does it that way" says so. No IETF spec does. That's the problem.

> Because suppose it had pointed at "a%20b" - I don’t want middleware 
> lying to me.

What you want isn't really the issue. Compatibility with deployed code is 
the issue.

> becomes "a%20b" in the address bar. But I don’t see the relevance 
> outside human-authored strings.

All the strings in question are human-authored.

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 23:31:44 +0000 (UTC)

On Tue, 23 Oct 2012, Mark Nottingham wrote:
> > I can't speak for Anne, but having experienced the IETF via the hybi 
> > work, my own opinion is that the main reason I wouldn't work with the 
> > IETF is that the community these days values consensus over technical 
> > value and running code, and the culture in the IETF doesn't value the 
> > kind of specification style that IMHO leads to better interop. For 
> > example, this very thraed -- we're having to argue to convince people 
> > that defining error handling is even a valuable thing to do.
> 
> Wait - who's making that argument?

Me.

> References, please.

This very thread is evidence enough, but see also the complete disinterest 
in fixing the URL specs, or the reaction abarth got from MIME sniffing, or 
the disaster that was hybi, or this complete disinterest in fixing the 
problem with encodings:

   http://mail.apps.ietf.org/ietf/charsets/threads.html#01830
   http://mail.apps.ietf.org/ietf/charsets/threads.html#02027
   http://mail.apps.ietf.org/ietf/charsets/threads.html#02034

...or the way IANA registrations for MIME types get handled, or HTTP bis' 
reaction to browser feedback, or the way process is put ahead of progress 
(there's no way to fix an RFC once it's published, even errata are often 
rejected), or the lack of any testing culture...

I understand that you disagree that most of those were a problem. But the 
original question was "why don't you work at IETF", and that's the answer. 
It may be that you conclude that it's a good thing, therefore, that I and 
others don't work at the IETF, but in that case you shouldn't complain 
when we go and do stuff outside the IETF.

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 23:40:51 +0000 (UTC)

> > doesn't work. Having to reference three specs (pre-parse, IRI, URI) 
> > just to parse and resolve a URL is not what leads to implementors 
> > having a good time and thus not what leads to interop.
> 
> Really? You're comfortable with the current weight and depth of the 
> HTML5 spec, but balk at a pre-processing step for URIs? Seriously?

Good lord, no. Who's comfortable with the HTML spec's size?

Unfortunately, the size of the HTML spec is dictated by the complexity of 
the platform it is describing.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 23 Oct 2012 10:42:18 +1100

>> Wait - who's making that argument?

So, you're saying that you can't work in this environment (*fans self*) because of the arguments you're making?

OK.

> in fixing the URL specs

Also? I thought that was what we were talking about...

> or the reaction abarth got from MIME sniffing

AIUI Adam walked away from it because two people expressed individual concerns about it. Had he stuck with it, I'm personally convinced it would have gotten through pretty easily.

> or 
> the disaster that was hybi

I personally think websockets was a bad idea from the start, so I'll refrain from further comment.

> or this complete disinterest in fixing the 
> problem with encodings:
> 

No comment, would have to look into it.

> ...or the way IANA registrations for MIME types get handled

... the process for which was recently revised, based partially on those experiences.

> or HTTP bis' reaction to browser feedback

As far as I know, we addressed all of that feedback to the satisfaction of those who brought it. If you believe otherwise, we're currently in WGLC.

> or the way process is put ahead of progress 
> I understand that you disagree that most of those were a problem.

Oh, no, I agree that there are some pretty serious problems here. 

> But the 

Again, I'm not stuffed about the venue, and you can do what you like. However, when the *W3C* does things that interact with IETF technologies, we coordinate to make sure that there aren't overlaps, conflicts, etc. 

Regards,

---

From: Ian Hickson <[email-masked]>
Date: Mon, 22 Oct 2012 23:59:00 +0000 (UTC)

> because of the arguments you're making?

I'm saying this is why I don't want to work here, yes.

> >> References, please.
> > 
> > This very thread is evidence enough, but see also the complete 
> > disinterest in fixing the URL specs
> 

The disinterest spans many years. This thread demonstrates not so much the 
disinterest so much as the lack of understanding that there's a problem to 
fix, which is a bit of a different issue, though equally frustrating and 
equally discouraging.

> AIUI Adam walked away from it because two people expressed individual 
> concerns about it. Had he stuck with it, I'm personally convinced it 
> would have gotten through pretty easily.

The question here is "why do people walk away from the IETF or not 
participate in the IETF". Adam walked away. It doesn't matter whether he 
would have been able to get his stuff done if he'd stayed -- the fact is 
he walked away, so if you're looking for reasons why people walked away, 
it's a relevant data point.

Imagine you're trying to sell me a car, and I think the car is ugly, so I 
don't buy it. Then you're trying to figure out why people like me don't 
buy the car, and you ask me, and I say it's ugly. It doesn't matter if you 
think it's the prettiest thing in the world; if your goal is to sell me 
the car, then I can't think it's ugly.

Now if your goal _isn't_ to sell me that car, then that's fine.

> > or the disaster that was hybi
> 

Whether WebSockets is a good idea or not is besides the point. The point 
is that the hybi group was not a pleasant experience for me. If I were to 
be in a position to do Web Sockets again, I would decline the opportunity 
to do it through the IETF. Doing it through the IETF made the work take a 
year longer than it would have, made the protocol less secure (the WG 
removed a number of defense-in-depth features), and made the spec a mess 
(it's a mishmash of different editing styles). Plus, the group _still_ 
hasn't done multiplexing, which some of the vendors said was a prereq to 
implementation, something which, prior to the IETF getting involved, was 
only 3 to 6 months out on the roadmap.

> ... the process for which was recently revised, based partially on those 

If it was revised more than about 2 weeks ago, the problems aren't solved, 
based on what I've seen in the past 2 weeks.

> As far as I know, we addressed all of that feedback to the satisfaction 
> of those who brought it. If you believe otherwise, we're currently in 
> WGLC.

There's a number of people who raised feedback who gave up trying to get 
it addressed, but I haven't been following this closely enough to tell you 
what those are, because I gave up too.

> > But the original question was "why don't you work at IETF", and that's 
> > the answer. It may be that you conclude that it's a good thing, 
> > therefore, that I and others don't work at the IETF, but in that case 
> > you shouldn't complain when we go and do stuff outside the IETF.
> 
> Again, I'm not stuffed about the venue, and you can do what you like. 
> However, when the *W3C* does things that interact with IETF 
> technologies, we coordinate to make sure that there aren't overlaps, 
> conflicts, etc.

Yeah, well, I've mostly given up on the W3C, too. :-)

Anyway, this URL work isn't happening at the W3C.

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 00:05:09 +0000 (UTC)

---

From: mike amundsen <[email-masked]>
Date: Mon, 22 Oct 2012 20:32:51 -0400

<snip>
I’d have to say that URI interoperability problems haven’t come near
getting into the list of top-20 pain points.
</snip>
I can't recall the last time i experienced "URI interoperability problems"
across various user agents/implementations on the public Internet. My
problems w/ browser implementations is another thing.

In this particular case (URIs), I applaud Anne's attempt to fix the broken
way browsers handle these strings within their own executable code (i.e
"browsers (apart from Chrome) do not unescape URL escapes."[1]).

However, I find the announcement that he plans to change the names of
things along the way (i.e. "And yes, the plan is to do away with IRI/URI
and just call them all URLs"[1]) a waste of time.

Fixing the code can happen regardless of naming. Do that and do it now.

Thanks.

[1] http://annevankesteren.nl/2012/09/url-equivalence

mca+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 01:02:02 +0000 (UTC)

On Mon, 22 Oct 2012, Tim Bray wrote:
> conversations) rather than browser-wrangling level.  I’d have to say 
> that URI interoperability problems haven’t come near getting into the 
> list of top-20 pain points.  So either my experience is wildly 
> untypical, or maybe it’s a combination of being a little bit lucky, and 
> that the pain which exists is highly concentrated in the browser space.  

The importance of error handling goes up dramatically as the number of 
participants in a space increases. When one is writing private software 
where you write the server and the client, one is unlikely to run into any 
problems that one would attribute to the specs. When one is writing 
software with a few hundred participants, these kinds of errors occur, but 
it's trivial to deal with them by telling the offending people to follow 
the spec. When one is dealing with trillions of items, it becomes 
impossible to fix the problems, and error-handling becomes necessary.

The Web is a classic example of the latter, so browser vendors and authors 
of software that interacts with the Web, e.g. Web search engine software 
(GoogleBot), Web mirroring software (wget), etc, often run into it.

---

From: Graham Klyne <[email-masked]>
Date: Tue, 23 Oct 2012 09:31:46 +0100

On 22/10/2012 23:35, Ian Hickson wrote:
> Consensus isn't a value I hold highly,

!

#g
--

---

From: John Cowan <[email-masked]>
Date: Tue, 23 Oct 2012 09:23:02 -0400

> >Consensus isn't a value I hold highly,

Just call him Frank Sinatra.

-- 
John Cowan   [email-masked]    http://ccil.org/~cowan
I am he that buries his friends alive and drowns them and draws them
alive again from the water. I came from the end of a bag, but no bag
went over me.  I am the friend of bears and the guest of eagles. I am
Ringwinner and Luckwearer; and I am Barrel-rider.  --Bilbo to Smaug

---

From: Julian Reschke <[email-masked]>
Date: Tue, 23 Oct 2012 15:27:00 +0200

> removed a number of defense-in-depth features), and made the spec a mess

And, as far as I can tell, fixed a security problem in the original 
design (which caused some UA implementers to actually disable what they 
were shipping at that time): <http://w2spconf.com/2011/papers/websocket.pdf>

> only 3 to 6 months out on the roadmap.

Indeed, but then wasn't it you arguing *against* having it in the base 
spec? (see 
<http://www.ietf.org/mail-archive/web/hybi/current/msg00239.html>)

Best regards, Julian

---

From: Ted Hardie <[email-masked]>
Date: Tue, 23 Oct 2012 09:44:53 -0700

On Mon, Oct 22, 2012 at 2:46 PM, Ian Hickson <[email-masked]> wrote:
> done any work. That's the complaint. Until they do the work,

Handing things off to the IETF and saying "please go do this work" has
a very low success rate, because that's not how the IETF works.  The
IETF works by bringing together folks interested in solving a
particular problem and putting them in a larger context; that context
can help those working on a point solution see other aspects of their
problem space.  It also provides a set of processes which can be
useful for decision making when the trade-offs may involve different
folks' oxes being gored.

In this case, the concern is that defining what you are doing as a
revision of the URL standard outside the IETF will:

* lose that larger context
* use a process which has a bias toward browser viewpoints, which
raises concerns about trade-offs outside that space
* generate a fork, either directly or in the creation of two
communities which understand URL to be either a subset of URI in the
STD 66 sense or the "input string to web identifier" sense that Anne's
work uses.

It's tedious when people say "you should come here and do the work",
and I apologize that I'm about to say it.  But for work which
redefines IETF standards, the IETF is really the place to do it, and
preserving that context is important to making sure that the
communities of use retain a single standard.  I share your frustration
with the pace of work on related topics, but I urge you to put energy
into the process rather than simply appropriating the term.

If you choose not call what you're doing a "URL" but by some other
term ("fleen" is my favorite), then the issue does not arise; at most,
someone needs to later define how a fleen and URL relate, but that's
much less likely to cause confusion.

My two cents, as an individual as always,

Ted

---

From: Ted Hardie <[email-masked]>
Date: Tue, 23 Oct 2012 15:02:27 -0700

On Tue, Oct 23, 2012 at 2:34 PM, Ian Hickson <[email-masked]> wrote:
>> term ("fleen" is my favorite), then the issue does not arise
>
> Since the IETF doesn't call it a URL anyway, I don't see the problem with
> terminology.
>

Please see RFC 3986, Section 1.1.3, which says:

   "A URI can be further classified as a locator, a name, or both.  The
   term "Uniform Resource Locator" (URL) refers to the subset of URIs
   that, in addition to identifying a resource, provide a means of
   locating the resource by describing its primary access mechanism
   (e.g., its network "location").  The term "Uniform Resource Name"
   (URN) has been used historically to refer to both URIs under the
   "urn" scheme [RFC2141], which are required to remain globally unique
   and persistent even when the resource ceases to exist or becomes
   unavailable, and to any other URI with the properties of a name."

While the document does recommend the use of the more generic term
"URI" it defines URL clearly and in a way that is significantly
different from that used in Anne's document and which you have
described as the aim of your work.

Note also that Anne's document contains this text:

"Align RFC 3986 and RFC 3987 with contemporary implementations and
obsolete them in the process."

Unless you get buy-in from the community that produced RFC 3986 and
RFC 3987, the production of this document *will* result in a fork, and
that is damaging to the Internet.  I urge you to pick a different term
(several far more useful ones than fleen have been suggested) and
avoid this needless conflict.  The WhatWG choosing to redefine IETF
standards is not contributing to a better web; it's simply making it
less clear for those outside the small cabal of standards workers what
they should do when faced with a URL.  Un-marked context shifts are
likely, and likely to be bad.  Avoiding them by picking a new term is
both easy and appropriate.

My personal opinion, as always,

regards,

Ted Hardie

---

From: Ted Hardie <[email-masked]>
Date: Tue, 23 Oct 2012 17:10:19 -0700

On Tue, Oct 23, 2012 at 4:51 PM, Ian Hickson <[email-masked]> wrote:
> Having multiple specs means an implementor has to refer to multiple specs
> to implement one algorithm, which is not a way to get interoperability.
> Bugs creep in much faster when implementors have to switch between specs
> just in the implementation of one algorithm.
>

First, do you have data that supports this assertion?

Second, multiple folks in this conversation have asserted that the
right way to approach this is to have *two* algorithms.  The first is
"method to get from string to URI"  and the second is "Process URI".
It is not obvious that those need to be in the same document, any more
than the processing of DNS names needs to be described in the same
document of URLs whose schemes include DNS names.

(In case it is not obvious, it is the string-which-may-become-a-URI
that I have referred to as a "fleen" in previous notes).

It also seems far more likely to me that bugs will creep in from
re-defining a known algorithm (the "process URI" bit from the pair
above) than from the separation of that from a different operation.
If the results of the rewording would be different operations the, as
I have noted before, you really should use different terms and admit
to the fork.

My personal opinion, as has been noted,

regards,

Ted Hardie

---

From: Jari Arkko <[email-masked]>
Date: Wed, 24 Oct 2012 13:36:34 +0300

Ted, Ian,

> Un-marked context shifts are

FWIW, I agree with Ted's advice above.

> My personal opinion, as always,

Mine too.

Jari

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 24 Oct 2012 11:39:54 +0100

On 23/10/2012 00:32, Mark Nottingham wrote:
...
> The underlying point that people seem to be making is that there's legitimate need for URIs to be a separate concept from "strings that will become URIs." By collapsing them into one thing, you're…

Thanks for bringing this point out. It was explained to me in 1993 by TBL and
Robert Cailliau that URLs (the only term used then, I think) should never
be typed in by a user, and preferably never even seen by a user. It's because
that doctrine was abandoned a year or so later that we have this problem
today. I think there would be value in a document making this clear, as
a framework for clearly separating the specification of what is allowable
as a URI on the wire from what is acceptable as a user input string (UIS?).

UIS to URI conversion may well end up as a heuristic algorithm.

Regards

     Brian

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 24 Oct 2012 13:18:41 +0100

> FWIW, I agree with Ted's advice above.

Further to that, some guy's fine tool [1] says that
RFC 3986 is referenced by 193 other RFCs. Google
scholar says that RFC 3986 has 2090 citations. [2]
While that's nowhere near the full story, it is
nonetheless significant.

Buggering about with a spec like that whilst only
considering a limited context seems hugely dumb to
me, no matter how good a case you seem to have for
changing bits and pieces of it. So far, I've not seen
the argument for changing 3986, but there do seem
to be good arguments for additional things such as
error-handling specific to the browser context.

I honestly don't get the argument for forking,
and I agree with Ted that that's what's at issue
and would be stupid. I don't know that inventing a
new term is that good an approach either really,
it'd seem better to me to try specify the additional
stuff needed in the whatwg context and then see if
there's any change to 3986 needed and if so, then
propose those changes as an update to the RFC,
following the IETF process. That is not what I
see happening now and I do see an apparent intent
to create the stupid fork that Ted identified.

Getting an update to 3986 through the IETF process
will I'm sure be a huge PITA for whoever does
propose it, since there are so many other dependent
specs. But that's life, and its quite do-able, if
done by someone who's able to handle that kind of
work.

S.

[1] http://www.arkko.com/tools/allstats/citations-rfc3986.html
[2]
http://scholar.google.com/scholar?cites=6121683772362091274&as_sdt=2005&sciodt=0,5&hl=en

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 21:12:17 +0000 (UTC)

On Tue, 23 Oct 2012, Julian Reschke wrote:
> were shipping at that time): 
> <http://w2spconf.com/2011/papers/websocket.pdf>

The security issue in question was already fixed in the draft by the time 
that paper came out.

> > hasn't done multiplexing, which some of the vendors said was a prereq 
> > to implementation, something which, prior to the IETF getting 
> > involved, was only 3 to 6 months out on the roadmap. ...
> 
> spec? (see <http://www.ietf.org/mail-archive/web/hybi/current/msg00239.html>)

I was arguing against having it in the first version, which I had planned 
for Q3 2009 IIRC, and was planning on defining it as an extension protocol 
in early 2010 (I even had a strawman ready). The hybi group argued and 
argued and argued and argued and then decided to not have it in the first 
version, which they ended up doing in Q4 2011, and still haven't done the 
extension. So yeah, I stand by my point above.

---

From: James M Snell <[email-masked]>
Date: Tue, 23 Oct 2012 14:32:13 -0700

It should be quite clear to everyone that the horse is quite dead at this
point. Any further beating is entirely unnecessary. So let's wrap it up
with this: the whatwg's spec language around urls has the potential to
cause confusion among implementers, so please consider reworking that
language to avoid such confusion. Period, end of story.
 On Oct 23, 2012 2:12 PM, "Ian Hickson" <[email-masked]> wrote:

> > spec? (see <
> http://www.ietf.org/mail-archive/web/hybi/current/msg00239.html>)
>

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 21:34:05 +0000 (UTC)

On Tue, 23 Oct 2012, Ted Hardie wrote:
> >> > I couldn't agree more! We've been waiting for four years for the 
> >> > URI working group to get their act together and fix the URL mess. 
> >> > Nothing has happened. We lost patience and are now doing it 
> >> > ourselves. ...
> >>
> > Whoever. The people complaining that it should be done at the IETF 
> > haven't done any work. That's the complaint. Until they do the work,
> 
> Handing things off to the IETF and saying "please go do this work" has a 
> very low success rate, because that's not how the IETF works.

Ok, but "we disagree with you, do it differently" is not how it works 
either.

> The IETF works by bringing together folks interested in solving a 
> particular problem and putting them in a larger context

The problem here is that the bulk of the people here don't seem interested 
in solving this particular problem. In fact, many don't seem to even think 
that there is a problem to solve.

> * lose that larger context

By which I presume you mean "you'll miss good feedback".

My experience with the Web Sockets work at the IETF is that we got way 
better feedback (in terms of technical usefulness, relevance, and style) 
when it was at the WHATWG than when it was at the IETF. All the good 
feedback we got at the IETF was from people who were already participating 
in the WHATWG.

So I'm not sure that this is a given.

> * use a process which has a bias toward browser viewpoints, which raises 
> concerns about trade-offs outside that space

Working at the IETF results in the opposite concern, of course.

You're all welcome to participate in the WHATWG.

> * generate a fork, either directly or in the creation of two communities 
> which understand URL to be either a subset of URI in the STD 66 sense or 
> the "input string to web identifier" sense that Anne's work uses.

If Anne does a good job, then STD 66 can just be updated to contain Anne's 
work, and then there's no fork to worry about. If Anne does a poor job, 
then people will ignore it, and there's still no fork to worry about. The 
only way there could be a problem is if Anne does a good job, so people 
pay attention to it, but the IETF for political reasons refuses to 
acknowledge it. That's up to the IETF.

> It's tedious when people say "you should come here and do the work", and 
> I apologize that I'm about to say it.  But for work which redefines IETF 
> standards, the IETF is really the place to do it, and preserving that 
> context is important to making sure that the communities of use retain a 
> single standard.  I share your frustration with the pace of work on 
> related topics, but I urge you to put energy into the process rather 
> than simply appropriating the term.

The problem is that we have lost hope that the IETF is somewhere where 
this work could even take place. There's too much stop energy (just look 
at this thread -- Tim just said we were all wasting our time and should 
all shut up!).

But suppose we did try. Let's in fact try: Hi guys, we need to fix STD 66 
because it doesn't define error handling. Also it seems to be about time 
to merge IRIs and URIs together since implementing IRIs on top of URIs 
leads to a confusing implementation experience. We can call the result 
just "URLs" since that's what the majority of people call these things. 
Anne has a proposal for how to do it:

   http://url.spec.whatwg.org/

Does anyone object to us updating STD 66 to using this once it's ready? 
How do we go about it?

---

From: Peter Saint-Andre <[email-masked]>
Date: Tue, 23 Oct 2012 14:36:52 -0700

+1

On 10/23/12 2:32 PM, James M Snell wrote:
> It should be quite clear to everyone that the horse is quite dead at
> this point. Any further beating is entirely unnecessary. So let's wrap
> it up with this: the whatwg's spec language around urls has the
> potential to cause confusion among implementers, so please consider
> reworking that language to avoid such confusion. Period, end of story.

---

From: Jan Algermissen <[email-masked]>
Date: Tue, 23 Oct 2012 23:51:12 +0200

On Oct 23, 2012, at 11:34 PM, Ian Hickson <[email-masked]> wrote:

>  Let's in fact try: Hi guys, we need to fix STD 66 
> because it doesn't define error handling. 

Help me, I am just not getting it:

Why do you insist on 'fixing STD 66'?

What is the reason you are not willing to reframe the problem to 'fixing how we get from the provided string -the input to the reference construction process- to a STD-66-valid result'?

To me this is really what you are aiming at and dropping the 'fix the URI spec' language would get everyone on board immediately in my perception.

Jan

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 22:43:22 +0000 (UTC)

On Tue, 23 Oct 2012, Jan Algermissen wrote:
> > Let's in fact try: Hi guys, we need to fix STD 66 because it doesn't 
> > define error handling.
> 
> What is the reason you are not willing to reframe the problem to 'fixing 
> how we get from the provided string -the input to the reference 
> construction process- to a STD-66-valid result'?

Because that's not a good way to write specs. Implementors shouldn't have 
to read three separate specs to implement one algorithm. The definition 
for Base64 isn't spread into tree separate RFCs. You don't put the HTML 
parser in a different spec than the HTML elements.

A spec for this kind of thing should define the following:

 - The conformance requirements for authors so that they can use the 
   technology in a manner that avoids likely pitfalls

 - A processing model for each relevant implementation conformance class 
   (software) that defines how you take the input and use it

In the case of these string, that means, to a first approximation:

 - A definition of what the valid syntax of these strings is.

 - A definition of how you get from one of these strings, whether valid or 
   not, to the information you need to process it, in particular, for 
   e.g. strings that reference specific files:
    - the scheme (what protocol you're going to be using)
    - the hostname and port of the remote host
    - the path and query string to pass to that host
    - the fragment identifier

So there should just be one spec, not three (IRIs, URIs, and the error 
handling).

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 22:47:55 +0000 (UTC)

On Tue, 23 Oct 2012, Ted Hardie wrote:
> 
> Unless you get buy-in from the community that produced RFC 3986 and RFC 
> 3987, the production of this document *will* result in a fork, and that 
> is damaging to the Internet.

I'm trying to work with y'all to see how we can update these specs instead 
of having to do it elsewhere.

> I urge you to pick a different term (several far more useful ones than 
> fleen have been suggested) and avoid this needless conflict.

What term we use isn't going to have any effect on whether it harms the 
Internet. If curl decides it should use WHATWGRLs or whatever and wget 
decides it should use IRIs or whatever, and the specs aren't compatible, 
then wget and curl won't interoperate.

Then again, they already don't interoperate -- that's the problem we're 
trying to fix -- so maybe the fork won't harm the Internet any more than 
STD 66 already is. Might even help matters, if curl and wget both decide 
to follow the WHATWGRLs spec (or whatever), instead of STD 66, and as a 
result start interoperating.

$ curl 'http://example.com/a b' # fetches "/a b" from example.com
$ wget 'http://example.com/a b' # fetches "/a%20b" from example.com

---

From: Jan Algermissen <[email-masked]>
Date: Wed, 24 Oct 2012 00:52:40 +0200

> A spec for this kind of thing should define the following:

Then, how about going from 'fixing STD 66' to

'augmenting STD 66 with how we get from the provided string -the input to the reference construction process- to a valid URI'?

(Personally, I do not see any problems with having one spec defining the valid output and one spec defining how to get from input to valid output. But that is a discussion that can be easily separated from the current one.)

What matters is that nothing of the existing URI spec *changes*.

Can you agree on that?

Jan

---

From: Christophe Lauret <[email-masked]>
Date: Wed, 24 Oct 2012 10:09:54 +1100

Jan wrote:

Help me, I am just not getting it:
> To me this is really what you are aiming at and dropping the 'fix the URI
> spec' language would get everyone on board immediately in my perception.

I completely agree - I don't understand the insistence on "fixing" STD 66.

As a Web developer who's had to write code multiple times to handle URIs in
very different contexts, I actually *like* the constraints in STD 66, there
are many instances where it is simpler to assume that the error handling
has been done prior and simply reject an invalid URI. By reducing the space
of valid addresses around rigid rules to match more closely the set of
addressable resources, it reduces my scope of work.

I understand that it isn't practical in some contexts (e.g. browsers) where
it is inevitable that strings that are not considered valid URI references
will crop up and it is desirable for everyone to deal with these in a
consistent manner. So, I applaud Anne and Ian for wanting to standardize
the error handling and tackling this problem head on.
But why not do it as a separate spec?

Increasing the space of valid addresses, when the set of addressable
resources is not actually increasing only means more complex parsing rules.
I accept that it is a requirement in some contexts, but it is
also completely unnecessary in others where coding against STD 66 is just
fine if not more desirable.

Personally, I don't have any qualms in calling that larger set of addresses
"URLs" if it is what most people use, and leave STD 66 and URIs are as they
are.

Christophe-

(I've used 'address' as a short for a "string which should ultimately
resolve as a valid STD 66 result")

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 23:47:54 +0000 (UTC)

> What matters is that nothing of the existing URI spec *changes*.

Do you mean the actual text, or the normative meaning of the text?

---

From: Ian Hickson <[email-masked]>
Date: Tue, 23 Oct 2012 23:51:09 +0000 (UTC)

On Wed, 24 Oct 2012, Christophe Lauret wrote:
> 
> As a Web developer who's had to write code multiple times to handle URIs 
> in very different contexts, I actually *like* the constraints in STD 66, 
> there are many instances where it is simpler to assume that the error 
> handling has been done prior and simply reject an invalid URI.

I think we can agree that the error handling should be, at the option of 
the software developer, either to handle the input as defined by the 
spec's algorithms, or to abort and not handle the input at all.

> resources is not actually increasing only means more complex parsing rules.

I'm not saying we should increase the space of valid addresses. The de 
facto parsing rules are already complicated by de facto requirements for 
handling errors, so defining those doesn't increase complexity either 
(especially if such behaviour is left as optional, as discussed above.)

---

From: Ian Hickson <[email-masked]>
Date: Wed, 24 Oct 2012 01:16:31 +0000 (UTC)

On Tue, 23 Oct 2012, Ted Hardie wrote:
> > Having multiple specs means an implementor has to refer to multiple 
> > specs to implement one algorithm, which is not a way to get 
> > interoperability. Bugs creep in much faster when implementors have to 
> > switch between specs just in the implementation of one algorithm.
> 

No.

> Second, multiple folks in this conversation have asserted that the right 
> way to approach this is to have *two* algorithms. The first is "method 
> to get from string to URI"  and the second is "Process URI".

That would be inefficient, so isn't likely to be a solid implementation 
strategy in many environments. I don't see any reason to do it this way.

> above) than from the separation of that from a different operation.

That's what regression tests and review are for. Obviously if we rewrote 
the algorithms and introduced bugs and just left them as is, we'd be 
pretty awfully incompetent. Nobody is suggesting that the work will be 
done before we have reached a point where the new spec is as good or 
better than the existing specs.

> If the results of the rewording would be different operations then, as I 
> have noted before, you really should use different terms and admit to 
> the fork.

There's no desire to make the new spec incompatible with existing 
software. That would not be a useful spec.

---

From: "Manger, James H" <[email-masked]>
Date: Wed, 24 Oct 2012 12:59:05 +1100

> From: Ian Hickson [mailto:[email-masked]]
> I think we can agree that the error handling should be, at the option
> of the software developer, either to handle the input as defined by the

Currently, I don't think url.spec.whatwg.org distinguishes between strings that are
valid URLs and strings that can be interpreted as URLs by applying its standardised error handling. Consequently, error handling cannot be at the option of the software developer as you cannot tell which bits are error handling.

This might be why some are unhappy with url.spec.whatwg.org.

url.spec.whatwg.org does have separate "Writing" and "Parsing" sections. Perhaps the implicit idea is that any output of the "Writing" section is a valid URL (that all URL-processing software should handle). The "Parsing" section accepts more strings than can be created by the "Writing" section. The difference is the error handling. It's OK for a software developer not to parse this difference if it makes its parser simpler, safer, or that is the way its parser works today.

--
James Manger

---

From: David Sheets <[email-masked]>
Date: Tue, 23 Oct 2012 20:49:52 -0700

> spec's algorithms, or to abort and not handle the input at all.

Yes, input is handled according to the specs' algorithmS.

> just in the implementation of one algorithm.

One algorithm? There seem to be several functions...

- URI reference parsing (parse : scheme -> string -> raw uri_ref)
- URI reference normalization (normalize : raw uri_ref -> normal uri_ref)
- absolute URI predicate (absp : normal uri_ref -> absolute uri_ref option)
- URI resolution (resolve : absolute uri_ref -> _ uri_ref -> absolute uri_ref)

Of course, some of these may be composed in any given implementation.
In the case of a/@href and img/@src, it appears to be something like
(one_algorithm = (resolve base_uri) . normalize . parse (scheme
base_uri)) is in use.

A good way to get interop is to thoroughly define each function and
supply implementors with test cases for each processing stage
(one_algorithm's test cases define some tests for parse, normalize,
and resolve as well).

Some systems use more than the simple function composition of web browsers...

> I'm not saying we should increase the space of valid addresses.

Anne's current draft increases the space of valid addresses. This
isn't obvious as Anne's draft lacks a grammar and URI component
alphabets. You support Anne's draft and its philosophy, therefore you
are saying the space of valid addresses should be expanded.

Here is an example of a grammar extension that STD 66 disallows but
WHATWGRL allows:
<http://www.rfc-editor.org/errata_search.php?rfc=3986&eid=3330>

> The de facto parsing rules are already complicated by de facto requirements for

*parse* is separate from *normalize* is separate from checking if a
reference is absolute (*absp*) is separate from *resolve*.

Why don't we have a discussion about the functions and types involved
in URI processing?

Why don't we discuss expanding allowable alphabets and production rules?

David

---

From: John Cowan <[email-masked]>
Date: Tue, 23 Oct 2012 23:59:55 -0400

> are saying the space of valid addresses should be expanded.

Before confusion is worse confounded, Anne's draft does not extend the space
of valid addresses, but rather provides processing for both valid and
invalid addresses.  As such, it extends the space of what may be called
processable or usable addresses.

-- 
"The serene chaos that is Courage, and the phenomenon   [email-masked]
of Unopened Consciousness have been known to the        John Cowan
Great World eons longer than Extaboulism."
"Why is that?" the woman inquired.
"Because I just made that word up", the Master said wisely.
        --Kehlog Albran, The Profit             http://www.ccil.org/~cowan

---

From: David Sheets <[email-masked]>
Date: Tue, 23 Oct 2012 21:20:17 -0700

> processable or usable addresses.

In the version of the spec that I am reading
<http://url.spec.whatwg.org/>, I see definition of an "invalid flag"
<http://url.spec.whatwg.org/#invalid-flag> and a "valid attribute"
<http://url.spec.whatwg.org/#dom-url-valid>.

It would appear that <##> is a valid WHATWGRL but an invalid URI reference.

Under WHATWGRL processing (and as is found in extant browsers), <##>
is handled as it is literally written and not through coercion to a
"valid" address from a "usable" address.

Given these facts, I don't understand how Anne's spec doesn't extend
the space of valid addresses. If the spec says that input is output
without modification and that input is outside of the STD 66 space,
doesn't that expand the space of valid addresses?

WHATWGRLs with "[" and "]" in path, query, and fragment are allowed as well.

David

---

From: Ian Hickson <[email-masked]>
Date: Wed, 24 Oct 2012 05:05:44 +0000 (UTC)

On Wed, 24 Oct 2012, Manger, James H wrote:
> 
> Currently, I don't think url.spec.whatwg.org distinguishes between 
> strings that are valid URLs and strings that can be interpreted as URLs 
> by applying its standardised error handling. Consequently, error 
> handling cannot be at the option of the software developer as you cannot 
> tell which bits are error handling.

Well first, the whole point of discussions like this is to work out what 
the specs _should_ say; if the specs were perfect then there wouldn't be 
any need for discussion.

But second, I believe it's already Anne's intention to add to the parsing 
algorithm the ability to abort whenever the URL isn't conforming, he just 
hasn't done that yet because he hasn't specced what's conforming in the 
first place.

> - URI resolution (resolve : absolute uri_ref -> _ uri_ref -> absolute uri_ref)

I don't understand what your four algorithms are supposed to be.

There's just one algorithm as far as I can tell -- it takes as input an 
arbitrary string and a base URL object, and returns a normalised absolute 
URL object, where a "URL object" is a conceptual construct consisting of 
the components scheme, userinfo, host, port, path, query, and 
fragment, which can be serialised together into a string form.

(I guess you could count the serialiser as a second algorithm, in which 
case there's two.)

> Anne's current draft increases the space of valid addresses.

No, Anne hasn't finished defining conformance yet. (He just started 
today.)

You may be getting confused by the "invalid flag", which doesn't mean the 
input is non-conforming, but means that the input is uninterpretable.

> > The de facto parsing rules are already complicated by de facto 
> > requirements for handling errors, so defining those doesn't increase 
> > complexity either (especially if such behaviour is left as optional, 
> > as discussed above.)
> 

No, it doesn't have to be. That's actually a more complicated way of 
looking at it than necessary, IMHO.

> Why don't we have a discussion about the functions and types involved in 
> URI processing?
>

Personally I think this kind of open-ended approach is not a good way to 
write specs. Better is to put forward concrete use cases, technical data, 
etc, and let the spec editor take all that into account and turn it into a 
standard. Arguing about what precise alphabets are allowed and whether to 
spec something using prose or production rules is just bikeshedding.

---

From: "Manger, James H" <[email-masked]>
Date: Wed, 24 Oct 2012 17:41:16 +1100

> > strings that are valid URLs and strings that can be interpreted as
> > URLs by applying its standardised error handling. Consequently, error
> > handling cannot be at the option of the software developer as you
> > cannot tell which bits are error handling.

> Well first, the whole point of discussions like this is to work out
> what the specs _should_ say; if the specs were perfect then there
> wouldn't be any need for discussion.
> 
> But second, I believe it's already Anne's intention to add to the
> parsing algorithm the ability to abort whenever the URL isn't
> conforming, he just hasn't done that yet because he hasn't specced
> what's conforming in the first place.

That is good to hear. There is no hint about this in the current text/outline. There is an "invalid" flag in the current text -- but that is for strings that are so broken no error handling can resurrect a URL. There is no mention of a separate "conforming" flag, even if the rules for when to set it are yet to be fixed (though it should have been easy to say conforming=conforming-as-per-rfc3987/3987 if that was the intention).

Assuming this is Anne's intention, then 1 spec for URI/IRI/error-handling would be helpful. I'm not sure that parsing rules with conforming/non-conforming branches would be pretty, but perhaps this isn't necessary if what a conforming URL is is clear from other parts of the spec.

--
James Manger

---

From: Anne van Kesteren <[email-masked]>
Date: Wed, 24 Oct 2012 10:14:25 +0200

> That is good to hear. There is no hint about this in the current text/outline. There is an "invalid" flag in the current text -- but that is for strings that are so broken no error handling can res…

Thanks for pointing that out. I renamed it to "fatal error flag" and
added an issue about the ability to halt on the first (non-fatal)
error. Ian is right that it's not defined yet because conformance is
not defined. And yes, as you say I've yet to figure out if branches in
the parser section is easy enough to do. If it is I think it's
worthwhile because it makes implementing a conformance checker (or
strict parser) much more straightforward.

-- 
http://annevankesteren.nl/

---

From: Jan Algermissen <[email-masked]>
Date: Wed, 24 Oct 2012 13:04:25 +0200

> Do you mean the actual text, or the normative meaning of the text?

I ideally mean the actual text, but it might be that there is some overlap in the construction algorithms - I am not expert enough there to judge that.

The point really was to make it very clear that *additional* stuff is going to be said and that existing implementations that follow the URI spec strictly remain conforming.

Jan

---

From: John C Klensin <[email-masked]>
Date: Wed, 24 Oct 2012 07:38:41 -0400

--On Wednesday, October 24, 2012 11:39 +0100 Brian E Carpenter
<[email-masked]> wrote:

>> The underlying point that people seem to be making is that
>> there's legitimate need for URIs to be a separate concept
>> from "strings that will become URIs." By collapsing them into
>> one thing, you're doing those folks a disservice. Browser
>> implementers may not care, but it's pretty obvious that lots
>> of other people do.
> 
> Thanks for bringing this point out. It was explained to me in
> 1993 by TBL and Robert Cailliau that URLs (the only term used
> then, I think) should never be typed in by a user, and
> preferably never even seen by a user. It's because that
> doctrine was abandoned a year or so later that we have this
> problem today. I think there would be value in a document
> making this clear, as a framework for clearly separating the
> specification of what is allowable as a URI on the wire from
> what is acceptable as a user input string (UIS?).
> 

Very useful perspective, IMO.

Seen that way, IRIs might then be considered a different flavor
of UIS.  Less heuristic than some, but not a flavor of URI.

best,
   john

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 24 Oct 2012 17:53:07 +0200

On Oct 24, 2012, at 06:20, David Sheets <[email-masked]> wrote:

> WHATWGRL

Hey, call them EARLs.  Error-tolerant web-Address Repairing Labels or whatever.
(Just not URLs, that term is already taken in the Web.)

Grüße, Carsten

---

From: Melinda Shore <[email-masked]>
Date: Wed, 24 Oct 2012 08:59:24 -0800

On 10/23/12 2:52 PM, Jan Algermissen wrote:
> 'augmenting STD 66 with how we get from the provided string -the
> input to the reference construction process- to a valid URI'?

[ ... ]

> What matters is that nothing of the existing URI spec *changes*.

W00t!  Uh, that is to say, +1.

Melinda

---

From: Ian Hickson <[email-masked]>
Date: Wed, 24 Oct 2012 17:02:16 +0000 (UTC)

On Wed, 24 Oct 2012, Jan Algermissen wrote:
> I ideally mean the actual text, but it might be that there is some 
> overlap in the construction algorithms - I am not expert enough there to 
> judge that.

Well I definitely don't think we should constrain a spec editor to being 
forced to use text he didn't even write, that seems like a very poor way 
to write a spec. Especially given that here the text is already spread 
across two specs (URI and IRI).

I think it makes sense to be conservative and say that URL synatx 
conformance requirements should probably not change from what the IRI spec 
says today unless there's a really compelling reason, though.

> The point really was to make it very clear that *additional* stuff is 
> going to be said and that existing implementations that follow the URI 
> spec strictly remain conforming.

Well unless there's a very good reason (e.g. following the current specs 
involves a security vulnerability or something like that) then I'd think 
that was a reasonably strong technical requirement, sure. But that's 
independent of how the spec is written. It's trivial to write a spec that 
uses existing text while making all existing implementations 
non-conforming, for example (just add a line that says "implementations 
MUST NOT do what the following section says" or something...).

---

From: "Roy T. Fielding" <[email-masked]>
Date: Wed, 24 Oct 2012 12:10:59 -0700

On Oct 24, 2012, at 3:39 AM, Brian E Carpenter wrote:
> Robert Cailliau that URLs (the only term used then, I think)

As a historical footnote, the term URL was created by the same
BOF that created the Uniform Resource Identifiers working group
at the IETF meeting in July 1992.

The early Web protocol specs had used the term "network address".

The term "Document Identifiers" came from Brewster Kahle and was
later used in a call for proposals by the Coalition for Networked
Information's Architectures & Standards Working Group, which in
turn led to TimBL propose Web addresses as Universal Document
Identifiers for a BOF at IETF 24 (Cambridge, MA).  Somewhere
in that BOF discussion, the URI working group was proposed and
TimBL's proposal was renamed Uniform Resource Locators
to distinguish it from other ideas for URNs
[see IETF 24 proceedings, p.184, and the following link].

 ftp://ftp.ietf.org/ietf/92jul/udi-minutes-92jul.txt

TimBL had originally specified that addresses in HREF could be
provided in full or partial form.  The IETF removed the partial
form, leading to all sorts of bad decisions regarding syntax,
and so I revived it in 1994 as Relative URLs [RFC1808].  That
spec is the only one that came close to defining what Anne
is trying to do here -- a single parsing standard for
potentially relative references. 

It is easy to claim that the merging of syntax specs that
created RFC2396 lost some value when the parsing standard was
replaced by a non-normative appendix.  However, it was discussed
extensively at the time, including with the browser developers,
and there was simply nothing common enough to make standard.
The best I could do for 2396 and 3986 was to include a
regular expression that accepts all strings and parses them
into the component parts.

I have absolutely no problem with writing a proposed standard
for parsing references, particularly if browser developers are
willing to adhere to one.  However, it is not a redefinition of
URLs, nor does it make sense for error-correcting transformations
(like pct-encoding embedded spaces) to be "the standard" for
parsing when there are plenty of applications that string
parse references for the sake of generating invalid test cases
(e.g., the example attributed to curl).

It is not non-interoperable behavior to parse input data
differently depending on the context in which it is entered.
What matters is that the context be properly documented to
indicate what pre/post-processing is applied, just as we
expect a browser's combined search/location dialog bars to
be documented as not merely URL-entry forms (or be banned
due to the privacy leakage of incremental search results).

....Roy

---

From: David Morris <[email-masked]>
Date: Wed, 24 Oct 2012 16:16:57 -0700 (PDT)

> just in the implementation of one algorithm.

That is true to the extent that all developers have the same objective but 
we've already established that there at least two case:
a) Products which can assume correct URI (STD 66) syntax and reject errors
b) Products which need to handle human mangled input

By writing merging (a) and (b)->(a) you make life more difficult and hence
error prone for (a). I would suggest that the developers of (b) will be
better served by a clear specification of (a) w/o the (b)->(a) concerns
because that will improve their ability to validate URIs generated by
their logic as well as have meaningful discussions with folks who
assume (a).

---

From: David Sheets <[email-masked]>
Date: Wed, 24 Oct 2012 20:37:28 -0700

> the specs _should_ say; if the specs were perfect then there wouldn't be

Good! Let's have a discussion about what the spec should say.

> I don't understand what your four algorithms are supposed to be.

Ian, these are common descriptors (and function signatures).

Here are (longer) prose descriptions for those unfamiliar with
standard functional notation:

*parse* is a function which takes the contextual scheme and a string
to be parsed and produces a structure of unnormalized reference
components.
*normalize* is a function which takes a structure of unnormalized
reference components and produces a structure of normalized reference
components (lower-casing scheme, lower-casing host for some schemes,
collapsing default ports, coercing invalid codepoints, etc).
*absp* is a function which takes a structure of normalized reference
components and potentially produces a structure of normalized
reference components which is guaranteed to be absolute (or nothing:
in JS, this roughly corresponds to nullable).
*resolve* is a function which takes a URI structure and a reference
component structure and produces a URI structure corresponding to the
reference resolution of the second argument against the first (base)
argument.

See my original message for how these compose into your one_algorithm.

> fragment, which can be serialised together into a string form.

How is the arbitrary string deconstructed? How is the result
normalized? What constitutes an absolute reference? How does a
reference resolve against a base URI?

> No, Anne hasn't finished defining conformance yet. (He just started

This is a political dodge to delay the inevitable discussion of
address space expansion.

>From what I have read of WHATWG's intentions and discussed with you
and others, you are codifying current browser behavior for
'interoperability'. Current browsers happily consume and emit URIs
that are invalid per STD 66.

<http://url.spec.whatwg.org/#writing> presently says:
"A fragment is "#", followed by any URL unit that is not one of
U+0009, U+000A, and U+000D."
This is larger than STD 66's space of valid addresses.

> looking at it than necessary, IMHO.

Why use several simple, flexible sharp tools when you could use a
single complicated, monolithic blunt tool?

Why do you insist on producing a single, brittle, opaque function when
you could produce several simply-defined functions that actually model
the data type transformations?

Vendors are, of course, always free to implement an optimized
composition for their specific use cases.

> write specs.

The specs already exist and use these formalisms successfully. Why do
you think discussions about the model of the problem-space are
'open-ended'? Why are you trying to stop a potentially productive
discussion?

> Better is to put forward concrete use cases, technical data,
> standard.

Is <https://github.com/dsheets/ocaml-uri/blob/master/lib/uri.ml#L108>
correct or should safe_chars_for_fragment include '#'?

Whatever 'standard' you produce will require me to exert significant
effort on your One Giant Algorithm to factor it into its proper
components and reconcile it with competing standards for my users. I
have applications that use each of the above functions separately.

> Arguing about what precise alphabets are allowed and whether to

I can only conclude that you understand neither the value of precision
nor the meaning of "bikeshedding".

You are not constructing anything remotely comparable to a nuclear reactor.

I am expressing a genuine desire to discuss the actual technical
content of the relevant specifications in the most precise and concise
way possible.

I am losing confidence in your technical leadership.

Sincerely,

David Sheets

---

From: Anne van Kesteren <[email-masked]>
Date: Thu, 25 Oct 2012 15:54:43 +0200

> that are invalid per STD 66.

Correct, that does not mean valid input is similarly relaxed. That is
also not the case for HTML for example. Literally anything produces a
tree of sorts, but far from all input is considered valid.

> This is larger than STD 66's space of valid addresses.

I aligned it with IRI now, apart from private Unicode ranges. Not
really sure why we should ban them in one place and not in another.

We discussed how to write the algorithms on the WHATWG list before and
again I challenge you to write out your approach and convince the
world it's better. I'm not interested in doing that work for you.

---

From: Roberto Peon <[email-masked]>
Date: Thu, 25 Oct 2012 10:45:47 -0700

There are obviously orthogonal problems here.
If we were doing this as code, these would be separate functions and most
everyone would agree that it would make both testing and understanding
easier.

Why is it different with specs? The hardest part of specs is choosing which
one is right. The second hardest part is dealing with ambiguity (and the
larger the function, the more difficult figuring out intention, at least
IMHO)

-=R