Subject: Re: Issue 196, was: #110: how to determine what entity a response carries

From: Julian Reschke <[email-masked]>
Date: Tue, 13 Apr 2010 16:41:18 +0200

On 15.10.2009 06:07, Mark Nottingham wrote:
> Flipping this to editorial; I don't think there's a design aspect to
> this one (it's just specifying something already present more clearly).
> ...

Here's proposed text for the definition of "Effective Request URI", with 
questions inlined:

--
4.3.  Effective Request URI

    The term "Effective Request URI" is introduced in order to abstract
    away the various syntactical forms a request can take.

    If the request-target is an absolute-URI, then the Effective Request
    URI is the request-target. [[effrequri-scheme: What about the case
    where the given scheme name and the transport disagree? --jre]]

    If the request-target uses the path-absolute (plus optional query)
    syntax, then the Effective Request URI is constructed by
    concatenating

    o  the scheme name: "http" if the request was received over an
       insecure TCP connection, or "https" when received over SSK/
       TLS-secured TCP connection, [[effrequri-othertransports: Need to
       mention other future transports here? --jre]]

    o  the character sequence "://",

    o  the host name, as determined by the rules in Section 4.2,
       [[effrequri-nohost: How do we deal with undefined host names?
       --jre]] and

    o  the request-target obtained from the Request-Line.

    Otherwise, when request-target is an asterisk ("*") or uses the
    authority form, the Effective Request URI is undefined.

    Example: the Effective Request URI for the message

      GET /pub/WWW/TheProject.html HTTP/1.1
      Host: www.example.org

    (received over an insecure TCP connection) is "http", plus "://",
    plus the host name "www.example.org", plus the request-target "/pub/
    WWW/TheProject.html", thus
    "http://www.example.org/pub/WWW/TheProject.html".

    [[effrequri-compare: Need to declare comparison?  Can we re-use the
    comparison defined in Section 2.6.3? --jre]]
--

(see 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.diff>).

Feedback appreciated,

Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Apr 2010 17:17:07 +0200

> Here's proposed text for the definition of "Effective Request URI", with

I did some more work, and started to consider how this effects Part 2 
(see 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.2.diff>).

As expected, it works well for "Identifying the Resource Associated with 
a Representation" 
(<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-09.html#rfc.section.6.1>).

However, in general it's not as easy as simply replacing 
"request-target" by "Effective Request URI".

The reason for this is that with the currently proposed definition, 
"Effective Request Target" is undefined for certain requests, such as 
when request-target is "*" or just an authority component.

Should we re-consider this, mapping the request

   OPTIONS * HTTP/1.1
   Host: example.org

to the Effective Request URI

   http://example.org

(note no trailing slash)?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 14 Apr 2010 17:42:16 +0200

> when request-target is "*" or just an authority component.

Related to that, 
<http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-09.html#rfc.section.4.1.2> 
says:

- snip -
If a proxy receives a request without any path in the request-target and 
the method specified is capable of supporting the asterisk form of 
request-target, then the last proxy on the request chain MUST forward 
the request with "*" as the final request-target.

For example, the request

   OPTIONS http://www.example.org:8001 HTTP/1.1

would be forwarded by the proxy as

   OPTIONS * HTTP/1.1
   Host: www.example.org:8001

after connecting to port 8001 of host "www.example.org".
- snip --

But what if the method does not support the asterisk? What is supposed 
to happen for:

   GET http://www.example.org:8001 HTTP/1.1

?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 15 Apr 2010 12:03:31 +1000

I think it would be helpful to have an example with an explicit port number. Also, the proposed text refers to 'host name', which isn't quite right.

On 14/04/2010, at 12:41 AM, Julian Reschke wrote:

> Here's proposed text for the definition of "Effective Request URI", with questions inlined:
> 
> --
> (see <http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.diff>).
> 
> Feedback appreciated,
> 
> Julian

--
Mark Nottingham     http://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Thu, 15 Apr 2010 14:13:14 +0200

> I think it would be helpful to have an example with an explicit port number. Also, the proposed text refers to 'host name', which isn't quite right.

Ack. Update at 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.diff>. 
Now uses "authority" instead of "host":

-- snip --

    If the request-target is an absolute-URI, then the Effective Request
    URI is the request-target. [[effrequri-scheme: What about the case
    where the given scheme name and the transport disagree?  What if
    actual and specified port disagree? --jre]]

    o  the authority component, as specified in the Host header
       (Section 9.4) and determined by the rules in Section 4.2,
       [[effrequri-nohost: How do we deal with undefined hosts? --jre]]
       and

      GET /pub/WWW/TheProject.html HTTP/1.1
      Host: www.example.org:8080

    (received over an insecure TCP connection) is "http", plus "://",
    plus the authority component "www.example.org:8080", plus the
    request-target "/pub/WWW/TheProject.html", thus
    "http://www.example.org:8080/pub/WWW/TheProject.html".

-- snip --

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Thu, 22 Apr 2010 16:55:02 +0200

> (note no trailing slash)?

For now I have added this to the proposed patch (which is 
work-in-progress), see 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.3.diff>; 
the definition now reads:

- snip -

    If the request-target uses the path-absolute (plus optional query)
    syntax or if it is just the asterisk "*", then the Effective Request
    URI is constructed by concatenating

    o  the scheme name: "http" if the request was received over an
       insecure TCP connection, or "https" when received over SSL/
       TLS-secured TCP connection, [[effrequri-othertransports: Need to
       mention other future transports here? --jre]]

    o  the request-target obtained from the Request-Line, unless the
       request-target is just the asterisk "*".

    Otherwise, when request-target uses the authority form, the Effective
    Request URI is undefined.

    Example 1: the Effective Request URI for the message

    Example 2: the Effective Request URI for the message

      GET * HTTP/1.1
      Host: www.example.org

    (received over an SSL/TLS secured TCP connection) is "https", plus
    "://", plus the authority component "www.example.org", thus
    "https://www.example.org".

       [[eru-options: Need to point out that we are not defining an URI
       for OPTIONS * here. --jre]]

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Apr 2010 11:46:47 +1000

On 23/04/2010, at 12:55 AM, Julian Reschke wrote:
> For now I have added this to the proposed patch (which is work-in-progress), see <http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.3.diff>; the definition now reads:
> 
> - snip -

Suggested introduction:

HTTP requests often do not carry the absolute-URI [ref to 3986] for they are intended; instead, the value needs to be inferred from the request-target, Host header and other context. The result of this process is the "Effective Request URI."

>   actual and specified port disagree? --jre]]

It's perfectly acceptable to send a request for http://foo.com to a proxy via HTTPS on a different port. I think this is a separate problem; what happens when a server gets an effective request URI that they don't / can't / won't handle, and how much error correction do they attempt? I think both are implementation-specific.

>      mention other future transports here? --jre]]

I think that's their problem; they can update this RFC.

>      [[effrequri-nohost: How do we deal with undefined hosts? --jre]]

p1 2.6.1 already has

> http-URI = "http:" "//" authority path-abempty [ "?" query ]

[...]
> The host MUST NOT be empty; if an "http" URI is received with an empty host, then it MUST be rejected as invalid.

>   comparison defined in Section 2.6.3? --jre]]

I'd hope so... any reason why not?

Cheers,

---

From: "Thomson, Martin" <[email-masked]>
Date: Fri, 23 Apr 2010 11:46:05 +0800

Mark: 
> Suggested introduction:
> 
> HTTP requests often do not carry the absolute-URI [ref to 3986] for
> they are intended; instead, the value needs to be inferred from the
> request-target, Host header and other context. The result of this
> process is the "Effective Request URI."

The first part of this sentence doesn't parse for me.  Perhaps:

  HTTP requests often do not carry an absolute URI [RFC3986].  The absolute URI is instead inferred from the... 

[[No need to use ABNF symbols in prose, unless you are referring directly to an ABNF symbol, as you do in the next part.]]

Or was there something else that you intended...to say?

--Martin

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 Apr 2010 12:57:24 +0200

On 23.04.2010 03:46, Mark Nottingham wrote:
>> For now I have added this to the proposed patch (which is work-in-progress), see<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.3.diff>; the definition now reads:
>>
>> - snip -

Sounds good to me.

> It's perfectly acceptable to send a request for http://foo.com to a proxy via HTTPS on a different port. I think this is a separate problem; what happens when a server gets an effective request URI…

So do we need to state that?

> I think that's their problem; they can update this RFC.

OK, so we'll stay silent on this for now.

>> The host MUST NOT be empty; if an "http" URI is received with an empty host, then it MUST be rejected as invalid.

4.2 currently says:

"Recipients of an HTTP/1.0 request that lacks a Host header field MAY 
attempt to use heuristics (e.g., examination of the URI path for 
something unique to a particular host) in order to determine what exact 
resource is being requested."

So it appears we need to cover this case as well.

> I'd hope so... any reason why not?

I'll have to check :-)

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Mon, 26 Apr 2010 13:00:42 +0200

>    HTTP requests often do not carry an absolute URI [RFC3986].  The absolute URI is instead inferred from the...

Oh.

Proposal:

"HTTP requests often do not carry the absolute URI ([RFC3986], Section 
4.3) for the resource they are intended for; instead, the value needs to 
be inferred from the request-target, Host header and other context. The 
result of this process is the "Effective Request URI"."

(I think have the ABNF components from which the URI is derived from is 
fine).

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 Apr 2010 21:08:40 +1000

Works for me.

On 26/04/2010, at 9:00 PM, Julian Reschke wrote:

> "HTTP requests often do not carry the absolute URI ([RFC3986], Section 4.3) for the resource they are intended for; instead, the value needs to be inferred from the request-target, Host header and other context. The result of this process is the "Effective Request URI"."
> 
> (I think have the ABNF components from which the URI is derived from is fine).
> 
> Best regards, Julian

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 27 Apr 2010 08:02:42 +0800

> result of this process is the "Effective Request URI"."

Much clearer, thanks.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 27 Apr 2010 15:06:03 +0200

> HTTP requests often do not carry the absolute-URI [ref to 3986] for they are intended; instead, the value needs to be inferred from the request-target, Host header and other context. The result of…

I have attached a new work-in-progress patch to issue 196 
(<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/196/i196.4.diff>); 
the intro now reads:

    HTTP requests often do not carry the absolute URI ([RFC3986], Section
    4.3) for the resource they are intended for; instead, the value needs
    to be inferred from the request-target, Host header and other
    context.  The result of this process is the "Effective Request URI".

> It's perfectly acceptable to send a request for http://foo.com to a proxy via HTTPS on a different port. I think this is a separate problem; what happens when a server gets an effective request URI…

OK.

> I think that's their problem; they can update this RFC.

OK.

>> The host MUST NOT be empty; if an "http" URI is received with an empty host, then it MUST be rejected as invalid.

Rephrased the reminder to:

    o  the authority component, as specified in the Host header
       (Section 9.4) and determined by the rules in Section 4.2,
       [[effrequri-nohost: Do we need to include the handling of missing
       hosts in HTTP/1.0 messages, as described in Section 4.2? --jre]]
       and

> I'd hope so... any reason why not?

Yes. 2.6.3 normalizes the empty path to "/", which we can' do here (or 
can we?). Proposed text:

    Effective Request URIs are compared using the rules described in
    Section 2.6.3, except that empty path components MUST NOT be treated
    as equivalent to an absolute path of "/".

Best regards, Julian