Subject: HTTP URI in the form of "http://example.com?query"

From: Zhong Yu <[email-masked]>
Date: Mon, 3 Jun 2013 18:04:50 -0500

The question is whether this is a valid HTTP URI:

    http://example.com?query

According to RFC2616, it is invalid, a slash before the question mark
is mandatory(i.e. http://example.com/?query)

According to the latest bis draft, it is valid. The draft adopts the
generic URI syntax of RFC, which permits this kind of URI.

Any reason for this spec change? It seems risky; some old programs may
not be able to accept such URIs.

Zhong Yu

---

From: Julian Reschke <[email-masked]>
Date: Tue, 04 Jun 2013 08:52:49 +0200

On 2013-06-04 01:04, Zhong Yu wrote:
> ...

And then, others do.

Do you have any evidence of things breaking?

In general, I prefer consistency in edge cases like these.

Best regards, Julian

---

From: Roberto Peon <[email-masked]>
Date: Tue, 4 Jun 2013 00:33:38 -0700

It seems less likely to be parsed correctly without the slash-- I know of
many regexps which will fail to correctly parse the example above.
-=R

---

From: Julian Reschke <[email-masked]>
Date: Tue, 04 Jun 2013 09:43:03 +0200

On 2013-06-04 09:33, Roberto Peon wrote:
> It seems less likely to be parsed correctly without the slash-- I know
> of many regexps which will fail to correctly parse the example above.
> -=R
> ...

There's even a regexp in RFC 3986; everybody using that already accepts 
that format.

That being said: what exactly is the breakage? Can you give a concrete 
example?

Best regards, Julian

---

From: Roberto Peon <[email-masked]>
Date: Tue, 4 Jun 2013 00:50:28 -0700

A search for regular expression (or synonym) and url will bring up numerous
examples which would be broken by this change.
It is certainly not every one, but numerous, nonetheless.

Here is one example:
http://net.tutsplus.com/tutorials/other/8-regular-expressions-you-should-know/
-=R

---

From: Julian Reschke <[email-masked]>
Date: Tue, 04 Jun 2013 10:04:23 +0200

On 2013-06-04 09:50, Roberto Peon wrote:
> A search for regular expression (or synonym) and url will bring up
> numerous examples which would be broken by this change.

Yes, but what's the exact breakage except for one component not 
processing that edge case? It's an edge case after all?

Me thinks it's better to be (a) consistent with generic URI parsing and 
(b) what important components already do (UAs, http servers etc).

Best regards, Julian

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Tue, 04 Jun 2013 12:08:01 +0200

>is mandatory(i.e. http://example.com/?query)

That may well be unintentional, RFC 2396 certainly allows for the path
to be omitted and RFC 2616 talks about an "empty abs_path" as opposed to
an omitted abs_path in section 3.2.3; RFC 1945 does the same. RFC 1808
does not allow it though, so quite possibly RFC 2616 failed to account
for the change between RFC 1808 and RFC 2396.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 4 Jun 2013 12:19:19 +0200 (CEST)

On Tue, 4 Jun 2013, Julian Reschke wrote:

> Yes, but what's the exact breakage except for one component not processing 
> that edge case? It's an edge case after all?

I only recently got a bug report for curl as when we got an input such as that 
and passed it to a particular (unspecified) proxy it would reject it as 
illegal.

Thus curl nowadays will insert a slash before passing on the URL to a proxy.

-- 

  / daniel.haxx.se

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Jun 2013 10:38:41 -0500

How about we add some cautionary notes on this kind of URIs:
1. consumers must accept them
2. producers should not produce them
3. a middle man should add the slash before forwarding it to others.

Zhong Yu

On Tue, Jun 4, 2013 at 5:19 AM, Daniel Stenberg <[email-masked]> wrote:
> I only recently got a bug report for curl as when we got an input such as
> that and passed it to a particular (unspecified) proxy it would reject it as
> illegal.
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 4 Jun 2013 09:00:22 -0700

No, the URI is correctly handled by all relevant implementations.
There is no opportunity for error handling while parsing a URI,
so it is useless to create corner cases where a parser doesn't
know what string is associated with which component.

The ABNF in 2616 was known to be buggy since before it was even
published, so I have no interest in considering it further.

The definition of http URIs in practice is that they accept any
string, parse according to the regex in 3986, and fix such things
as an empty path as part of the request handling.  The only part
of HTTP that needs to consider this case is the creation of a
request-target, and we already have that requirement in the spec.

....Roy

---

From: Zhong Yu <[email-masked]>
Date: Tue, 4 Jun 2013 11:45:42 -0500

All right, everybody must be able to accept the URI. (Though privately
I'll never give that URI to anyone)

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 5 Jun 2013 07:36:00 +0200

Hi Julian,

> processing that edge case? It's an edge case after all?

Interesting case, at least it breaks haproxy's path extraction, which
relies on 2616. When you need to check the path from a request, haproxy
does this :

   1) skip the scheme and "://"
   2) skip user:pass@host:port
   3) look for the first "/"
   4) return everything from the first "/" to the first "?" or end of
      the string.

So "http://example.com?query=foo/bar" will return "/bar" as the path of
the request instead of an empty string or "/". BTW, is "/" supposed to
be the abspath here, or just something empty ? I'm asking because haproxy
returns a pointer to the beginning of the string and a length, so if the
response is "/", we don't have it in this request, so probably the best
thing to do would be to "fix" the request to insert the "/" before "?".

> (b) what important components already do (UAs, http servers etc).

I agree. Using a generic parser is important in that it avoids future
incompatibilities as in the example above which was based on 2616.

Best regards,
Willy

---

From: Zhong Yu <[email-masked]>
Date: Wed, 5 Jun 2013 11:06:24 -0500

> thing to do would be to "fix" the request to insert the "/" before "?".

It's probably a good idea to insert the "/"

http://tools.ietf.org/html/rfc3986#section-6.2.3

   In general, a URI that uses the generic syntax for authority with an
   empty path should be normalized to a path of "/".

http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-22#section-2.7.3

   When not being used in
   absolute form as the request target of an OPTIONS request, an empty
   path component is equivalent to an absolute path of "/", so the
   normal form is to provide a path of "/" instead.

I still think this problem deserves a warning note in the spec for the
interest of interoperability. It is apparently something that human
implementers may overlook.

Zhong Yu