Subject: HTTP redirect section draft-ietf-httpbis-bcp56bis-08

From: Ori Finkelman <[email-masked]>
Date: Tue, 13 Nov 2018 11:12:12 +0200

Hi Mark,
This draft seems a good place to clarify to redirection issues we have
discussed in HTTPbis and how they are used wrongly in HLS and DASH clients.
Reading the "Redirection" section 4.6.1 of this draft it is does not
clarify that a a user agent that automatically followed a redirect should
treat that a request as a new request and not as an "extension" of the
original request, and therefore the base URI of the retrieved object is the
final retrieval URI.
This may also get some readers confused with the text in
https://tools.ietf.org/html/draft-ietf-httpbis-semantics-03#section-9.4.3

That says:
Since the redirection might be altered on occasion, the client ought to
continue to use the effective request URI for future requests.

An implementation might wrongly interpret that and conclude that if
subsequent requests to that resource should go back to the effective (that
means the original ?) request URI, then the base URI of the retrieved
object is still the original URI, even though it was redirected.

I was hoping that draft-ietf-httpbis-bcp56bis can be a good place to
clarify that and give the right guidance to protocols like HLS and DASH.

Hope this make sense.

Thanks,
Ori

-- 

*Ori Finkelman*Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 |
[email-masked]

---

From: Julian Reschke <[email-masked]>
Date: Tue, 13 Nov 2018 11:08:16 +0100

On 2018-11-13 10:12, Ori Finkelman wrote:
> clarify that a a user agent that automatically followed a redirect 
> should treat that a request as a new request and not as an "extension" 
> of the original request, and therefore the base URI of the retrieved 
> object is the final retrieval URI.
 > ...

FWIW, and AFAIU, the only real confusion mentioned in your report was 
about mixing up "embedding a URI (reference)" with "embedding a 
resource" (which has nothing to do with redirects at all).

Also, can you point out any language in existing specs that makes it 
sound as if the base URI is not the request URI of the final request 
after redirects?

Best regards, Julian

---

From: Ori Finkelman <[email-masked]>
Date: Tue, 13 Nov 2018 12:18:00 +0200

Hi Julian,
My reference was the https://tools.ietf.org/html/rfc7231#section-6.4.3

...Since the redirection might be altered on occasion, the client
ought to continue to use the
   effective request URI for future requests.

This can be wrongly interpreted as "the redirect does not change the base
URI of the object".
I fully agree this is not the case and that reading it such is
misinterpretation, but since I have seen many such wrong interpretations I
thought it would be good to clarify.
As this draft bcp56bis is about helping protocols to use HTTP properly, I
thought it may be a good place to clarify.

Thanks,
Ori

--

---

From: Daniel Stenberg <[email-masked]>
Date: Tue, 13 Nov 2018 11:30:32 +0100 (CET)

On Tue, 13 Nov 2018, Ori Finkelman wrote:

> > ...Since the redirection might be altered on occasion, the client ought to 

Can you clarify how you can come to that conclusion? Or perhaps suggest an 
alternative phrasing that would've helped to avoid that mistake?

Base URI/URL isn't even used as a term in the document (barring a reference in 
the appendix to changes from 2616).

-- 

  / daniel.haxx.se

---

From: Ori Finkelman <[email-masked]>
Date: Tue, 13 Nov 2018 13:32:59 +0200

Hi Daniel,
Indeed the base URI is not mentioned,
https://tools.ietf.org/html/rfc7231#section-6.4.3 refers to the effective
request URI.
So, what is the effective request URI ? from this phrasing my understanding
is that it is the original URI before redirection (as the context is that
the the redirection might be altered).

If it's the original URI, then this is, IMHO,  what is causing confusion,
as some reads it as "you should not use the redirected URL for anything
other than the local retrieval of that specific request".
This, is in contradiction with
https://tools.ietf.org/html/rfc3986#section-5.1.3, but the confusion is
still there.

I am trying to have a clarification so I can go back with that to the video
players community and promote fixes to align with the correct behavior.

What I would suggest as phrasing (without taking a stand where to put it):
"Clients automatically following redirect must note that the redirected
request is a separate request and as such, the URI of the retrieved object
after redirection is the redirection URI"

Thanks,
Ori

On Tue, Nov 13, 2018 at 12:33 PM Daniel Stenberg <[email-masked]> wrote:

> ought to

--

---

From: Julian Reschke <[email-masked]>
Date: Tue, 13 Nov 2018 13:51:48 +0100

On 2018-11-13 12:32, Ori Finkelman wrote:
> https://tools.ietf.org/html/rfc7231#section-6.4.3 refers to the 
> effective request URI.
> So, what is the effective request URI ? from this phrasing my 
> understanding is that it is the original URI before redirection (as the 
> context is that the the redirection might be altered).
> 
> If it's the original URI, then this is, IMHO,  what is causing 
> confusion, as some reads it as "you should not use the redirected URL 
> for anything other than the local retrieval of that specific request".
> I am trying to have a clarification so I can go back with that to the 
> video players community and promote fixes to align with the correct 
> behavior.
> 
> request is a separate request and as such, the URI of the retrieved 
> object after redirection is the redirection URI"
> 
> Thanks,
> Ori

...once again: does this fix the issue that you reported? Wasn't that 
really about the confusion about "embedding a resource"?

Best regards, Julian

---

From: Ori Finkelman <[email-masked]>
Date: Tue, 13 Nov 2018 15:02:43 +0200

Hi Julian,
Ideally I would seek clarifications for both the meaning of a redirect and
for the base URI establishment (the confusion about embedding resource).
But, while the embedding issue I think we can explain well enough using the
inputs we got from you and Patrick, the redirect IMHO is still
confusing due to the
"the client ought to continue to use the effective request URI for future
requests"

Clarifying both issues would make it very clear that anything else is not
conforming with the RFC and would help conclude the debates in the player
development community.

Thanks,
Ori

--

---

From: Ori Finkelman <[email-masked]>
Date: Wed, 14 Nov 2018 11:10:56 +0200

Hi Chris,
I don't think 301 is the right solution for CDN as we don't want the
redirection to be permanent.
Also, the main problem is not what happens when client requests the same
playlist again (if it goes to the original link it's not that bad), but
what happens when it uses the URI of this playlist in order to resolve
relative references that are parsed from that playlist.

As Julian explain, section 5.1 of RFC 3986 states it should use the
playlist retrieval URI, after redirect, as the base URI for computing URIa
of the relative references found within the playlist.

The discussion here is whether this is clear enough for the reader if the
RFCs (7231 and 3986 together).

My feeling is that the opinion here is that it is clear enough and there is
no place for further clarifications ?
We can publish guidelines for video player developers according to that.

Thanks,
Ori

On Wed, Nov 14, 2018 at 2:37 AM Chris Lemmons <[email-masked]> wrote:

> So, if I understand the question rightly, what is unclear is what a
> client ought to do when a given reference is to be used more than
> once.
>
> So, if a client is processing an entity and finds a reference, which
> it processes according to the rules for either absolute or relative
> references, it normally just requests it. If the request comes back
> with a 302, it follows the redirect once and obtains content from the
> new location. The question then is what URL should the client use if
> it needs to request the content again (as one might in an HLS or DASH
> scenario).
>
> I think RFC 7231 is relatively clear:
>     Since the redirection might be altered on occasion, the client
> ought to continue to use the effective request URI for future
> requests.
> "Effective request URI" is clearly the original URI, before
> redirection. This text is intended to contrast with the previous
> section, 301:
>     [...] the target resource has been assigned a new permanent URI
> and any future references to this resource ought to use one of the
> enclosed URIs. Clients with link-editing capabilities ought to
> automatically re-link references to the effective request URI to one
> or more of the new references sent by the server, where possible.
>
> If you wanted the client to always use the new address, it sounds like
> the answer should be 301, not 302. 301s are cacheable and
> Cache-Control headers can be used to suggest how clients ought to
> cache things. I don't think it's possible to tell the client they must
> use the new address from then on, though.
>
> On Tue, Nov 13, 2018 at 6:07 AM Ori Finkelman <[email-masked]> wrote:
> >
> > Hi Julian,
> > Ideally I would seek clarifications for both the meaning of a redirect
> and for the base URI establishment (the confusion about embedding resource).
> > But, while the embedding issue I think we can explain well enough using
> the inputs we got from you and Patrick, the redirect IMHO is still
> confusing due to the
> > "the client ought to continue to use the effective request URI for
> future requests"
> >
> > Clarifying both issues would make it very clear that anything else is
> not conforming with the RFC and would help conclude the debates in the
> player development community.
> >
> > Thanks,
> > Ori
> >
> >
> > Ori Finkelman
> > Qwilt | Work: +972-72-2221647 | Mobile: +972-52-3832189 | [email-masked]
>

--

---

From: Chris Lemmons <[email-masked]>
Date: Tue, 13 Nov 2018 17:37:32 -0700

On Tue, Nov 13, 2018 at 6:07 AM Ori Finkelman <[email-masked]> wrote:
>
> Hi Julian,
> Ideally I would seek clarifications for both the meaning of a redirect and for the base URI establishment (the confusion about embedding resource).
> But, while the embedding issue I think we can explain well enough using the inputs we got from you and Patrick, the redirect IMHO is still confusing due to the
> "the client ought to continue to use the effective request URI for future requests"
>
> Clarifying both issues would make it very clear that anything else is not conforming with the RFC and would help conclude the debates in the player development community.
>
> Thanks,
> Ori
>
>
> On Tue, Nov 13, 2018 at 2:51 PM Julian Reschke <[email-masked]> wrote:
>>