Subject: Re: Redirection to Other IP Addresses

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 00:02:57 -0700

Hi Jack,

Potentially, we can extend RFC 7838, as I mentioned in my proposal
<https://docs.google.com/document/d/1gtF6Nq3iPe44515BfsU18dAxfCYOvQaekiezK8FEHu0/edit#heading=h.e8fyx0s8f7wy>,
to define the behavior of client and server when "alt-svc" is used in
conjunction with a special status code. This is still a change/extension to
the existing protocol. I feel 4xx is not good for this purpose, since this
is not a client error. I think a 3XX is more appropriate.

Thanks!

Bin

On Wed, Jul 31, 2019 at 10:52 PM Jack Firth <[email-masked]> wrote:

> Versus with "alt-svc", the server will serve the content for the current
>> request, client will finish receiving the response and MAYBE connect to the
>> new IP for the next request.
>
>
> Could you return a 4xx error with the Alt-Svc header set, and a body
> message that tells clients they must use the Alt-Svc if they don't want to
> get a 4xx? Or even a generic 300? It seems reasonable to me for a CDN
> server to refuse to serve requests it knows will be prohibitively
> expensive, while providing clients with Alt-Svc as a way to find a
> less-expensive alternative.
>
> On Wed, Jul 31, 2019 at 8:55 PM Bin Ni <[email-masked]> wrote:
>
>> Hi Chris,
>>
>> There are a few caveats in your reasoning:
>> 1.  It does not have to be some "accept header". It can be the
>> "User-Agent" header as I mentioned, for example, chrome version > 100. Or
>> on contract. For example, some CDN customers have full control of the
>> client software. They just tell the CDN provider that "you can enable the
>> 312 redirection on all of our domains".
>> 2. Even when it does rely on some "accept header", there is still a
>> critical difference from "alt-svc":
>>     In this proposal, the current request will not be served. The client
>> will get a 312 and forced to reconnect to the new IP, similar to the 30X
>> redirection.
>>     Versus with "alt-svc", the server will serve the content for the
>> current request, client will finish receiving the response and MAYBE
>> connect to the new IP for the next request.
>>
>> Hope this is more clear. Please don't hesitate with more questions!
>> Thanks!
>>
>> Bin
>>
>> On Wed, Jul 31, 2019 at 6:49 PM Chris Lemmons <[email-masked]> wrote:
>>
>>> So, the typical mechanism for that would be an accept header of some
>>> sort. But if clients are opting into the redirect, then the redirect is
>>> effectively optional. Any client that would set the accept header can
>>> instead just support alt-svc today and choose to redirect.
>>>
>>> On Wed, Jul 31, 2019 at 7:39 PM Bin Ni <[email-masked]> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Great question!
>>>> The solution is that the server will only return the new status code
>>>> 312 if it is sure the client can support it.
>>>> The information can be from the User-Agent header, or some other
>>>> request header.
>>>> Or communicated through some other channel, for example, on a paper
>>>> contract.
>>>>
>>>> Thanks!
>>>>
>>>> Bin
>>>>
>>>>
>>>> On Wed, Jul 31, 2019 at 2:36 PM Chris Lemmons <[email-masked]>
>>>> wrote:
>>>>
>>>>> So I have to wonder about the end usefulness from an implementation
>>>>> perspective. Part of why alt-svc works is that it's optional, so
>>>>> servers can use them as optimization but everything else still works.
>>>>>
>>>>> If you have a new protocol that means basically "alt-svc, but
>>>>> mandatory", it means the CDN, load balancer, or similar service simply
>>>>> wouldn't work for any client that didn't understand the new value.
>>>>> There are a _lot_ of http clients out there. This would be a fairly
>>>>> high barrier to adoption, which would create a chicken-and-egg problem
>>>>> that would be tough to solve.
>>>>>
>>>>> On Tue, Jul 30, 2019 at 5:53 PM Bin Ni <[email-masked]> wrote:
>>>>> >
>>>>> > Hi Amos and All,
>>>>> >
>>>>> > Regarding the 30X redirect across different cache servers, it is
>>>>> already used by many big CDN companies that I know of.
>>>>> > It is proven to make the system faster without much burden on the
>>>>> front-end layer which you are concerned.
>>>>> > But 30X has the limitations I mentioned. This is why I'm proposing
>>>>> this new type of redirection to address the limitations.
>>>>> >
>>>>> https://docs.google.com/document/d/1gtF6Nq3iPe44515BfsU18dAxfCYOvQaekiezK8FEHu0/edit?usp=sharing
>>>>> > So it is not a question that this proposal will be useful or not.
>>>>> > I know it will at least be very useful to those CDNs.
>>>>> >
>>>>> > Thanks for your comments.
>>>>> > Please let me know if you have any questions.
>>>>> >
>>>>> > Bin
>>>>> >
>>>>> > On Tue, Jul 30, 2019 at 12:38 AM Amos Jeffries <[email-masked]>
>>>>> wrote:
>>>>> >>
>>>>> >> On 30/07/19 7:02 am, Bin Ni wrote:
>>>>> >> > Yes, what we want is a way to force a "deterministic behavior
>>>>> from the
>>>>> >> > client", just like all the 30X redirections today.
>>>>> >> >
>>>>> >> > Let me give a few more cases in which this can be helpful:
>>>>> >> > 1. A client in North America is returned a server IP in Europe by
>>>>> the
>>>>> >> > DNS. The server then wants to direct the client to another server
>>>>> in
>>>>> >> > North America for better performance.
>>>>> >> > 2. The content of a website is hashed to multiple servers based
>>>>> on URL.
>>>>> >> > These multiple servers may not even be in the same datacenter.
>>>>> The DNS
>>>>> >> > does not have this information and may return any IP to any query
>>>>> of the
>>>>> >> > website's hostname.  Each server will calculate the hash for each
>>>>> >> > request and redirect client to the correct server that has the
>>>>> content.
>>>>> >> > This is quite common for CDN.
>>>>> >>
>>>>> >> It is common for good reason: efficiency.
>>>>> >>
>>>>> >> There is a secondary level of efficiency that comes from the
>>>>> redirects
>>>>> >> being actual HTTP 30x redirects. Having large objects at different
>>>>> URL
>>>>> >> entirely provides for a different CDN or caching layer closer to the
>>>>> >> client to provide the large object contents. DNS can be (often is)
>>>>> >> involved in that layer to provide the closest server IP.
>>>>> >>
>>>>> >> As proposed so far your mechanism would flatten this two-tier
>>>>> structure.
>>>>> >> Forcing the frontend layer (now only layer) to be involved in
>>>>> deciding
>>>>> >> the specific hardware location of individual objects / resources.
>>>>> >>  Making the frontend machinery store more information and do more
>>>>> work
>>>>> >> per-request is not going to make the system faster, quite the
>>>>> opposite.
>>>>> >>
>>>>> >>
>>>>> >> By separating the work into the three layers: frontend LB, cache,
>>>>> and
>>>>> >> origin. Each CDN layer gets some orders of magnitude increase in
>>>>> >> performance / capacity:
>>>>> >>  - origin able to handle/generate some few thousand responses per
>>>>> second,
>>>>> >>  - cache able to re-distribute those as static objects at line
>>>>> speed for
>>>>> >> an order or two magnitude more than origins,
>>>>> >>  - frontend LB able to handle millions of the small ~1KB
>>>>> >> request/response pairs for redirection spreading that high load
>>>>> across
>>>>> >> the lower layers.
>>>>> >>
>>>>> >>
>>>>> >> AYJ
>>>>> >>
>>>>> >
>>>>>
>>>>
>>>>

-- 

Bin Ni
VP of Engineering

Connecting users with content...it's that simple.

Office: +1-888-847-9851 <(888)%20847-9851>

 <https://twitter.com/Team_Quantil>  
<https://plus.google.com/+Quantil_team/>  
<https://www.linkedin.com/company/quantil>

The information contained in this email may be confidential and/or legally
privileged. It has been sent for the sole use of the intended recipient(s).
If the reader of this message is not an intended recipient, you are hereby
notified that any unauthorized review, use, disclosure, dissemination,
distribution, or copying of this communication, or any of its contents, is
strictly prohibited. If you have received this communication in error,
please reply to the sender and destroy all copies of the message. To
contact us directly, send to QUANTIL, INC. at 1919 S Bascom Ave #600,
Campbell, CA 95008
<https://maps.google.com/?q=1919+S+Bascom+Ave+%23600,+Campbell,+CA+95008&entry=gmail&source=g>,
or visit our website at www.quantil.com. <https://www.quantil.com/>

---

From: Jack Firth <[email-masked]>
Date: Thu, 1 Aug 2019 00:07:14 -0700

Yes you could make a status code, but I don't follow why you need a new
status code instead of reusing a generic 300 response with body explaining
they need to follow Alt-Svc. A special status code requires clients to
implement new behavior, and if they're doing that already can't they just
implement Alt-Svc support instead? As for 4xx, it sounds like a client
error to me in the same way that 429 Too Many Requests is: the client asked
for something correctly, but did so in a way that places too great a burden
on the server.

On Thu, Aug 1, 2019 at 12:03 AM Bin Ni <[email-masked]> wrote:

>>>>>> > On Tue, Jul 30, 2019 at 12:38 AM Amos Jeffries <
>>>>>> [email-masked]> wrote:
>>>>>> >>
>>>>>> >> > 1. A client in North America is returned a server IP in Europe
>>>>>> by the
>>>>>> >> > DNS. The server then wants to direct the client to another
>>>>>> server in
>>>>>> >> > does not have this information and may return any IP to any
>>>>>> query of the

---

From: Jack Firth <[email-masked]>
Date: Wed, 31 Jul 2019 22:52:07 -0700

On Wed, Jul 31, 2019 at 8:55 PM Bin Ni <[email-masked]> wrote:

>>> The solution is that the server will only return the new status code 312
>>> if it is sure the client can support it.
>>> The information can be from the User-Agent header, or some other request
>>> header.
>>>> >> > Yes, what we want is a way to force a "deterministic behavior from
>>>> the
>>>> >> > 2. The content of a website is hashed to multiple servers based on
>>>> URL.
>>>> >> > These multiple servers may not even be in the same datacenter. The
>>>> DNS
>>>> >> By separating the work into the three layers: frontend LB, cache, and
>>>> >>  - cache able to re-distribute those as static objects at line speed
>>>> for

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 00:30:18 -0700

Hi Jack,

Actually, 300 was also my first reaction when Julian brought up "alt-svc".
Technically, the main benefit of using a new status code vs reusing an
existing status code is easier statistics.
People can tell what happened by only look at the status code in the access
log.
I think this is the main purpose of defining different status code in the
first place.

Bin

On Thu, Aug 1, 2019 at 12:07 AM Jack Firth <[email-masked]> wrote:

>>>>     In this proposal, the current request will not be served. The
>>>> client will get a 312 and forced to reconnect to the new IP, similar to the
>>>> 30X redirection.
>>>>>>> >> being actual HTTP 30x redirects. Having large objects at
>>>>>>> different URL
>> The information contained in this email may be confidential and/or
>> legally privileged. It has been sent for the sole use of the intended
>> recipient(s). If the reader of this message is not an intended recipient,
>> you are hereby notified that any unauthorized review, use, disclosure,
>> dissemination, distribution, or copying of this communication, or any of
>> its contents, is strictly prohibited. If you have received this
>> communication in error, please reply to the sender and destroy all copies
>> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
>> Bascom Ave #600, Campbell, CA 95008

-- 

Bin Ni
VP of Engineering

---

From: Daniel Stenberg <[email-masked]>
Date: Thu, 1 Aug 2019 09:56:28 +0200 (CEST)

On Thu, 1 Aug 2019, Jack Firth wrote:

> implement Alt-Svc support instead?

No, because Bin wants to (as I understand it) force the next request that uses 
that origin to *forcibly* use the new alternative IP address. I'm pretty sure 
most alt-svc implementations will not do that but will rather reuse existing 
connections (since the spec allows that and it is much faster) and then switch 
to the alternative only once a connection to that has been setup...

-- 

  / daniel.haxx.se

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 01:49:58 -0700

Let me try to explain my proposal with some human language :)
1. the "Alt-Svc" behavior:
Client: Hi Server-1.1.1.1, can you send me the movie XXX?
Server-1.1.1.1: Here is the movie, but next time, you'd better ask server
2.2.2.2 for this movie.
(It then took 2 hours to deliver the movie)
2. my proposed behavior:
Client: Hi Server-1.1.1.1, can you send me the movie XXX?
Server-1.1.1.1: Sorry, I can't give you the movie, you need to ask server
2.2.2.2 for this movie.
Client: Hi Server-2.2.2.2, can you send me the movie XXX?
Server-2.2.2.2: Here is the movie.
(It then took 0.5 hours to deliver the movie, because server-2.2.2.2 is
closer to the client, or less loaded)

Bin

-- 

Bin Ni
VP of Engineering

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 01 Aug 2019 19:13:53 +1000

On Thu, Aug 1, 2019, at 18:53, Bin Ni wrote:
> Server-1.1.1.1: Here is the movie, but next time, you'd better ask 
> server 2.2.2.2 for this movie.
> Server-1.1.1.1: Sorry, I can't give you the movie, you need to ask 
> server 2.2.2.2 for this movie.

How is this not a 302 status code?

---

From: Daniel Stenberg <[email-masked]>
Date: Thu, 1 Aug 2019 11:18:47 +0200 (CEST)

> closer to the client, or less loaded)

If we for a moment play with the idea that we'd do something like this, then I 
think it should be aligned with and work together with Alt-Svc in a better way 
than what is currently proposed...

There's no max-age/TTL. For how long is the user-agent supposed to consider 
the alternative IP addresses as the only ones that the given origin has? 
Forever? Only for the next single connect (attempt)?

Are the alternative IPs supposed to be used for the entire origin or for that 
specific URI only?

A 3xx redirect without a Location: header? Wouldn't it make more sense and 
work more similar to existing 3xx redirects if it also sends a Location:? Then 
existing clients that don't understand 312 might have a higher chance of at 
least doing something sensible.

If a client gets this response and starts downloading huge content from the 
new IP and the client then opens a second connection to the origin in a second 
tab. Which IPs is that supposed to use? The original ones or the redirected 
ones?

Requring user-agent snooping for a server to figure out if the feature works 
or not is a totally broken idea and I think this detail needs to be worked out 
for this idea to be considered for real.

My personal preference is probably to add some sort of "urgency" thing to 
alt-svc instead of this 312 plus several headers, so that a client can be told 
that it should switch sooner rather than later.

-- 

  / daniel.haxx.se

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 02:55:50 -0700

Hi Martin,

I explained this in my very first email:
3xx breaks cookie and https.

Bin

-- 

Bin Ni
VP of Engineering

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 03:12:16 -0700

Hi Daniel,

At high level, my proposal is in every other way the same as today's 30X
redirection.
With this in mind, the answer to your questions are:
1. In general, the alternate IP should only be used once for the next
single request.
But there is nothing to prevent the clients from remembering it, which is
OK.
Just like there is nothing to prevent a client to disregard the DNS TTL.
They do it with their own risk.
2. This proposal is to fix some limitations of the 30X with Location header.
Not very helpful to make it work together with the Location header.
3. We are not requiring every server and every client to support this
proposal.
For the ones who find it to be useful, the "extra burden" is a non-issue.

Thanks!

Bin

-- 

Bin Ni
VP of Engineering

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 01 Aug 2019 21:08:16 +1000

> 3xx breaks cookie and https.

You can give the second server a discrete name (so that it can use HTTPS) and you can pass any state it requires in the URL (obviating the need for using cookies to pass state).

---

From: Patrick McManus <[email-masked]>
Date: Thu, 1 Aug 2019 09:13:42 -0400

Use of Alt-Svc instead of Location in a 3xx makes some sense to me for this
very edge case of a single large xfer. Server side resource balancing is an
important property, and if you have some really big resources it can really
make the system unresponsive and paying the latency penalty that alt-svc
normally avoids might be worth the cost... avoiding the state where you
have the same resource under a lot of different URIs is a good goal.

Its going to be hard to bootstrap in a backwards compatible way though

On Thu, Aug 1, 2019 at 7:11 AM Martin Thomson <[email-masked]> wrote:

> You can give the second server a discrete name (so that it can use HTTPS)
> and you can pass any state it requires in the URL (obviating the need for
> using cookies to pass state).
>
>

---

From: "W. Felix Handte" <[email-masked]>
Date: Thu, 1 Aug 2019 12:37:09 -0400

Bin,

I've been following along on this discussion and it's still not clear to 
me why 30X doesn't solve this use case. Take for example a request and 
response as follows.

   GET /large_file HTTP/1.1
   Host: cdn.com

To which the server responds with

   HTTP/1.1 307
   Location: https://singapore.geo.cdn.com/large_file

Or even

   HTTP/1.1 307
   Location: https://123_45_67_89.ip.cdn.com/large_file

Maybe I'm missing something, but as I understand it, HTTPS and Cookies 
should work with the above (assuming you have wildcard certs for 
*.geo.cdn.com and/or *.ip.cdn.com, and have set your cookies with 
domain=.cdn.com). And it otherwise seems to accomplish exactly your intent.

Can you explain in a little more detail why you believe something along 
those lines wouldn't solve your need?

Thanks,
Felix

On 8/1/19 6:12 AM, Bin Ni wrote:
> But there is nothing to prevent the clients from remembering it, which 
> is OK.
> On Thu, Aug 1, 2019 at 2:18 AM Daniel Stenberg <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>        / daniel.haxx.se <http://daniel.haxx.se>
> 
> 
> 
> -- 
> 
> Bin Ni
> VP of Engineering
> 
> Quantil
> 
> Office: +1-888-847-9851 <tel:(888)%20847-9851>
> 
> Tweeter <https://twitter.com/Team_Quantil> Google Plus 
> <https://plus.google.com/+Quantil_team/> Linked In 
> recipient(s). If the reader of this message is not an intended 
> recipient, you are hereby notified that any unauthorized review, use, 
> disclosure, dissemination, distribution, or copying of this 
> communication, or any of its contents, is strictly prohibited. If you 
> have received this communication in error, please reply to the sender 
> and destroy all copies of the message. To contact us directly, send to 
> QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 11:45:26 -0700

Thanks for your comments Patrick.
For whoever finds this proposal interesting, backward compatibility should
not be an issue.
The benefits should be way greater than the "burden" of checking client
capability.

Bin

On Thu, Aug 1, 2019 at 6:13 AM Patrick McManus <[email-masked]> wrote:

> Use of Alt-Svc instead of Location in a 3xx makes some sense to me for
> this very edge case of a single large xfer. Server side resource balancing
> is an important property, and if you have some really big resources it can
> really make the system unresponsive and paying the latency penalty that
> alt-svc normally avoids might be worth the cost... avoiding the state where
> you have the same resource under a lot of different URIs is a good goal.
>

-- 

Bin Ni
VP of Engineering

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 11:55:24 -0700

Hi Felix,

What you described is exactly what my company (you probably already figured
out that I work for a CDN company)
is providing to our customers today. The problems are:
1. In your example, the first host "cdn.com" is the CDN customer's
hostname. They usually can't provide us with a "*.geo.cdn.com" wildcard
cert. Some of them requires EV certificate, which does not even support
wildcard.
2. Cookies are often targeting a specific hostname. We can't ask all our
customer to change the business logic of their web application to make sure
all cookies are targeting the entire domain.

Hope this helps. Any more questions?

Bin

On Thu, Aug 1, 2019 at 9:37 AM W. Felix Handte <[email-masked]> wrote:

> > <
> https://maps.google.com/?q=1919+S+Bascom+Ave+%23600,+Campbell,+CA+95008&entry=gmail&source=g>,
>

-- 

Bin Ni
VP of Engineering

---

From: Bin Ni <[email-masked]>
Date: Thu, 1 Aug 2019 12:06:48 -0700

Hi Martin,

I think you are talking about the same thing as Felix. Hopefully my answer
to him resolves your question?
Thanks.

Bin

-- 

Bin Ni
VP of Engineering

---

From: Bin Ni <[email-masked]>
Date: Thu, 8 Aug 2019 17:01:25 -0700

Hi All,

Thanks you all for the discussion in the past few weeks.
Hopefully I have addressed all the questions and concerns.
Now I wonder what is the procedure to make it a formal RFC?
Can someone point me a direction?
Thanks!

Bin

On Tue, Jul 23, 2019 at 4:32 PM Bin Ni <[email-masked]> wrote:

> Hi All,
>
> We trying to propose an addition to the HTTP protocol to support
> redirecting client to other IP addresses.
>
> The background is many website owners and CDN providers want the web or
> proxy servers to be able to participate in load balancing. For example,
> when the server is overloaded, redirect the request to some other servers;
> or based on the geo information of client IP address, send the client to a
> server that can provide better performance. Compare to the DNS-based load
> balancing, this method is more real time and more accurate. It can also
> take the URL, headers or other components in the HTTP request into
> consideration.
>
> What they can use today is the 30X redirection, which has at least the
> following limitations:
> 1. breaks the cookie, if you 30x redirect to another hostname or IP
> address.
> 2. breaks https if you redirect to another IP address.
>
> We need another way of telling the client/browser to only change the IP
> address but keep everything else the same.
>
> We put the proposal in a google doc at:
>
> You can post your questions and comment in the document or reply to this
> email.
> If this is not the correct way to submit a proposal, someone please point
> me to the right direction.
> Thanks!
> --
> Bin Ni
> www.quantil.com
>
>

-- 

Bin Ni
VP of Engineering

---

From: Erik Nygren <[email-masked]>
Date: Thu, 8 Aug 2019 21:27:15 -0400

Another directional approach for this use-case would be out-of-band content
encoding.
For example:

       https://tools.ietf.org/id/draft-reschke-http-oob-encoding-12.html

It is possible that this could cover at least some of the use-cases.
It has the benefit that the client signals support (via Accept-Encoding).
The target won't get cookies, but authenticators could be passed in the URL
target.
It also has some better properties for not having to fully trust the
redirect target
and not having to have the same cert on the redirect target.

    Erik

On Thu, Aug 1, 2019 at 2:58 PM Bin Ni <[email-masked]> wrote:

> Hi Felix,
>
> What you described is exactly what my company (you probably already
> figured out that I work for a CDN company)

---

From: Erik Nygren <[email-masked]>
Date: Thu, 8 Aug 2019 21:55:51 -0400

On Thu, Aug 1, 2019 at 5:21 AM Daniel Stenberg <[email-masked]> wrote:

> than what is currently proposed..
>

Agreed that if we go down the Alt-Svc-like-road that this needs to play well
with Alt-Svc.  A major difference is that Alt-Svc is scoped to Origins
(ie, all requests for that Origin are encouraged to use the Alt-Svc).
But in this case the scope is presumably for individual URLs.
If the client asks for a bunch of different objects it is possible
that each one of them might want to be directed to a different
alternative service, and some of these might be running in-parallel.

It seems like on the Alt-Svc route there are two design questions
that must be addressed:

1) How to signal client support
2) How to allow different alternative services to be used for different
URIs.

For #1, maybe there is a new request header?
"Accept-Redirect-to-Alternative" ?

For #2, maybe we allow for labeled alternatives?  For example:

GET /abc/foo345678.mp4
Host: videos.example.com

HTTP/1.1 312 Redirect URI to Alternative
Alt-Svc: h2="cdn-node-98765.example.net:443"; ma=3600;
label="cdn-cluster-98765"
Alt-Svc: h3="cdn-node-98765.example.net:443"; ma=3600;
label="cdn-cluster-98765"
Alt-Svc-Location-for-URI: "cdn-cluster-98765"

In this case there would be an additional cache of Alt-Svcs keyed by
{origin,label}.
On receiving one of these redirects, a client supporting this synchronously
look for
a connection to an Alt-Svc keyed by {"https://videos.example.com
","cdn-cluster-98765"}
and then re-issue the request there (sending along the Alt-Used request
header as well
which would somehow want to include the label used.

Presumably other URIs for the origin would not use the labelled Alt-Svc
unless
specifically redirected.

Another way to handle #2 would be to allow Alt-Svc to be scoped to URI
patterns,
so the "synchronous redirect to alternative" could include a set of Alt-Svc
entries with:    scope="/abc/*"

That said for the above, I think that something along the lines of an
out-of-band
encoding solves a bunch of these use-cases in a more flexible manner.
Alt-Svc requires the alternatives all have the same degree of trust.
Being able for an origin to say "get the bytes from this other URL but
use integrity-validation and decryption information provided as part
of this response" opens up many more use-cases.

       Erik

---

From: Bin Ni <[email-masked]>
Date: Tue, 13 Aug 2019 01:09:39 -0700

Hi Erik,

Thanks for your thoughts.
Since we will use a dedicated status code 312, I think we can simply use
"cache-control" to set the cache behavior, just like how 30X is handled
today.

Bin

On Thu, Aug 8, 2019 at 6:56 PM Erik Nygren <[email-masked]> wrote:

> On receiving one of these redirects, a client supporting this
> synchronously look for

-- 

Bin Ni
VP of Engineering

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 14 Aug 2019 11:58:44 +1000

Hi Bin,

Just to make sure -- please don't use status codes without having them standardised, and don't use a concrete number in your proposal, as it might have to change, which can cause interoperability problems.

From RFC7231 <https://tools.ietf.org/html/rfc7231#section-8.2.2>:

>    Proposals for new status codes that are not yet widely deployed ought
>    to avoid allocating a specific number for the code until there is
>    clear consensus that it will be registered; instead, early drafts can
>    use a notation such as "4NN", or "3N0" .. "3N9", to indicate the
>    class of the proposed status code(s) without consuming a number
>    prematurely.

Cheers,

> On 13 Aug 2019, at 6:09 pm, Bin Ni <[email-masked]> wrote:
> 
> Hi Erik,
> 
> Since we will use a dedicated status code 312, I think we can simply use "cache-control" to set the cache behavior, just like how 30X is handled today.
> 
> Bin
> 
> 2) How to allow different alternative services to be used for different URIs.
> 
> For #1, maybe there is a new request header?  "Accept-Redirect-to-Alternative" ?
> 
> Alt-Svc: h2="cdn-node-98765.example.net:443"; ma=3600; label="cdn-cluster-98765"
> Alt-Svc: h3="cdn-node-98765.example.net:443"; ma=3600; label="cdn-cluster-98765"
> In this case there would be an additional cache of Alt-Svcs keyed by {origin,label}.
> On receiving one of these redirects, a client supporting this synchronously look for 
> a connection to an Alt-Svc keyed by {"https://videos.example.com","cdn-cluster-98765"}
> and then re-issue the request there (sending along the Alt-Used request header as well 
> Presumably other URIs for the origin would not use the labelled Alt-Svc unless 
> specifically redirected.  
> 
> Another way to handle #2 would be to allow Alt-Svc to be scoped to URI patterns,
> That said for the above, I think that something along the lines of an out-of-band
> Office: +1-888-847-9851
> The information contained in this email may be confidential and/or legally privileged. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please reply to the sender and destroy all copies of the message. To contact us directly, send to QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008, or visit our website at www.quantil.com.
> 
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Mike Bishop <[email-masked]>
Date: Wed, 14 Aug 2019 14:34:46 +0000

I second this – given that you’re looking for a forced redirect, an out-of-band response coupled with an Alt-Svc to the other host should provide the right behavior.  The first request goes OOB and the client gets the payload from the alternate; the subsequent requests can be made directly to the alternative, since it also has a cert for the origin.

From: Erik Nygren <[email-masked]>
Sent: Thursday, August 8, 2019 9:27 PM
To: Bin Ni <[email-masked]>
Cc: W. Felix Handte <[email-masked]>; HTTP Working Group <[email-masked]>
Subject: Re: Redirection to Other IP Addresses

Another directional approach for this use-case would be out-of-band content encoding.
For example:

It is possible that this could cover at least some of the use-cases.
It has the benefit that the client signals support (via Accept-Encoding).
The target won't get cookies, but authenticators could be passed in the URL target.
It also has some better properties for not having to fully trust the redirect target
and not having to have the same cert on the redirect target.

    Erik

On Thu, Aug 1, 2019 at 2:58 PM Bin Ni <[email-masked]<mailto:[email-masked]>> wrote:
Hi Felix,

What you described is exactly what my company (you probably already figured out that I work for a CDN company)
is providing to our customers today. The problems are:
1. In your example, the first host "cdn.com<http://cdn.com>" is the CDN customer's hostname. They usually can't provide us with a "*.geo.cdn.com<http://geo.cdn.com>" wildcard cert. Some of them requires EV certificate, which does not even support wildcard.
2. Cookies are often targeting a specific hostname. We can't ask all our customer to change the business logic of their web application to make sure all cookies are targeting the entire domain.

Bin

On Thu, Aug 1, 2019 at 9:37 AM W. Felix Handte <[email-masked]<mailto:[email-masked]>> wrote:
Bin,

   GET /large_file HTTP/1.1
   Host: cdn.com<http://cdn.com>

   HTTP/1.1 307

Or even

   HTTP/1.1 307

Maybe I'm missing something, but as I understand it, HTTPS and Cookies
should work with the above (assuming you have wildcard certs for
*.geo.cdn.com<http://geo.cdn.com> and/or *.ip.cdn.com<http://ip.cdn.com>, and have set your cookies with
domain=.cdn.com<http://cdn.com>). And it otherwise seems to accomplish exactly your intent.

Thanks,
Felix

On 8/1/19 6:12 AM, Bin Ni wrote:
> On Thu, Aug 1, 2019 at 2:18 AM Daniel Stenberg <[email-masked]<mailto:[email-masked]>
> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:
>
>        / daniel.haxx.se<http://daniel.haxx.se> <http://daniel.haxx.se>
>
>
>
> --
>
> Bin Ni
> VP of Engineering
>
> Quantil
>
> or visit our website at www.quantil.com<http://www.quantil.com>. <https://www.quantil.com/>
>

--

Bin Ni
VP of Engineering
[Image removed by sender. Quantil]

Office: +1-888-847-9851<tel:(888)%20847-9851>

[Image removed by sender. Tweeter]<https://twitter.com/Team_Quantil>  [Image removed by sender. Google Plus] <https://plus.google.com/+Quantil_team/>   [Image removed by sender. Linked In] <https://www.linkedin.com/company/quantil>

The information contained in this email may be confidential and/or legally privileged. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please reply to the sender and destroy all copies of the message. To contact us directly, send to QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008<https://maps.google.com/?q=1919+S+Bascom+Ave+%23600,+Campbell,+CA+95008&entry=gmail&source=g>, or visit our website at www.quantil.com.<https://www.quantil.com/>

---

From: Bin Ni <[email-masked]>
Date: Wed, 14 Aug 2019 12:32:19 -0700

Hi Mark,

Thanks for the reminder. Yes, 312 is just an example. I was sloppy in the
last email to mention this.

Bin

On Tue, Aug 13, 2019 at 6:59 PM Mark Nottingham <[email-masked]> wrote:

> Hi Bin,
>
> Just to make sure -- please don't use status codes without having them
> standardised, and don't use a concrete number in your proposal, as it might
> have to change, which can cause interoperability problems.
>
> Bascom Ave #600, Campbell, CA 95008, or visit our website at
> www.quantil.com.
> >
> >
>
> --

-- 

Bin Ni
VP of Engineering

---

From: Bin Ni <[email-masked]>
Date: Wed, 14 Aug 2019 12:38:09 -0700

Hi Mike and Eric,

If looks to me that if I use oob encoding for my use case, it will be the
same as 30X redirection.
I won't be able to achieve my goal of "only change the IP address,
everything else remain the same".

Bin

On Wed, Aug 14, 2019 at 7:34 AM Mike Bishop <[email-masked]> wrote:

> I second this – given that you’re looking for a forced redirect, an
> out-of-band response coupled with an Alt-Svc to the other host should
> provide the right behavior.  The first request goes OOB and the client gets
> the payload from the alternate; the subsequent requests can be made
> directly to the alternative, since it also has a cert for the origin.
>
>
>
> *From:* Erik Nygren <[email-masked]>
> *Sent:* Thursday, August 8, 2019 9:27 PM
> *To:* Bin Ni <[email-masked]>
> *Cc:* W. Felix Handte <[email-masked]>; HTTP Working Group <
> [email-masked]>
> *Subject:* Re: Redirection to Other IP Addresses
>
>
>
> <https://twitter.com/Team_Quantil>  [image: Image removed by sender.
> Google Plus] <https://plus.google.com/+Quantil_team/>  [image: Image
> removed by sender. Linked In] <https://www.linkedin.com/company/quantil>
>

-- 

Bin Ni
VP of Engineering

---

From: Erik Nygren <[email-masked]>
Date: Wed, 14 Aug 2019 16:11:04 -0400

You get something in the middle ground.  The Origin doesn't change, but
cookies don't get sent without extra work.
But you gain the ability to decouple the server certificate and trust and
integrity from the Origin.
They do solve slightly different but heavily overlapping problems.

It may make sense to take a step back and enumerate the set of problems and
use-cases
that would be good to solve and see which ones there is interest in
addressing, and then
finding a solution that covers this set.

In order for any effort on this front to be worthwhile it will also require
getting interest and buy-in to collaborate and implement from both server
operators and client implementations, with the client implementations
being the one that is going to be especially critical to success.
(Even Alt-Svc has had limited client implementation to-date.)

     Erik

---

From: Bin Ni <[email-masked]>
Date: Wed, 14 Aug 2019 15:49:52 -0700

Hi Erik and All,

My use case is we have a bunch of reverse proxy servers all over the world
(just like any other CDN providers).
Our customers deploy proxy policies to all those servers.
Assume company ABC is one of our customers, they may configure all our
servers to serve https://download.abc.com, with the certificate for "
download.abc.com" installed on each of the servers.
They also configure their DNS to point www.abc.com to a CNAME provided by
us, such that my company essentially takes over the resolution of that
hostname.
When an end-user is visiting download.abc.com, the hostname is resolved by
our DNS server to one of the server's IP addresses.
This IP address is chosen based on some algorithm. But this algorithm has
some limitations which may not always return an IP that is optimal for the
end user.
When the end user reaches that IP to download some object, for example
https://download.abc.com/movie.mp4 (potentially with some cookie targeting "
download.abc.com"), an algorithm running on that server
finds out that another server would provide better performance and want to
redirect the end user to the new server to download the object.
Right now what we can do is sending 30X redirection to the end user with
something like:
Location: https://{new IP}.mycompany.com/download.abc.com/movie.mp4
As I mentioned in earlier posts, this current method breaks cookie and
company ABC's client (maybe a mobile App) may refuse to be directed to a
different domain.

Hope this is a clear enough description of the use case. Please let me know
of any questions.
Thanks!

Bin

On Wed, Aug 14, 2019 at 1:11 PM Erik Nygren <[email-masked]> wrote:

> It may make sense to take a step back and enumerate the set of problems
> and use-cases
>>> >     Are the alternative IPs supposed to be used for the entire origin
>>> or
>>> >     for that
>>> >     specific URI only?
>>> >

-- 

Bin Ni
VP of Engineering

---

From: craigt <[email-masked]>
Date: Thu, 15 Aug 2019 13:59:22 +0100

Just catching up on this thread:

Bin, It's great to hear of others wanting a solution in this area....

I set up a proof of concept for something very similar to this using
Alt-Svc, and it worked really well (on Firefox). The preferred path
was almost always set up before the bulk download was initiated due to
companion assets. The problem was that most user agents only
implemented protocol upgrade via Alt-Svc rather than 'connection
migration' or almost in this case 'http traffic engineering'.
Alt-Svc also has other useful properties when used in conjunction with
single address endpoints like failback to origin and multiple
alternates.

The use case described here, which was very similar to my own at the
time, has a 'lowest common capability' issue so the OPTIONAL aspects
of Alt-Svc make it unusable without support from the bulk of clients.
However, given Alt-Svc has become near essential for protocol upgrade
and partially supported by all, how can we encourage client
implementors to work towards:

"Therefore, if a client supporting this specification becomes aware of
   an alternative service, the client SHOULD use that alternative
   service for all requests to the associated origin as soon as it is
   available"

Or a method of signalling that Alt-Svc/equivalent is not optional in $case....

On Wed, 14 Aug 2019 at 23:53, Bin Ni <[email-masked]> wrote:
>
> Hi Erik and All,
>
> My use case is we have a bunch of reverse proxy servers all over the world (just like any other CDN providers).
> Assume company ABC is one of our customers, they may configure all our servers to serve https://download.abc.com, with the certificate for "download..abc.com" installed on each of the servers.
> They also configure their DNS to point www.abc.com to a CNAME provided by us, such that my company essentially takes over the resolution of that hostname.
> When an end-user is visiting download.abc.com, the hostname is resolved by our DNS server to one of the server's IP addresses.
> This IP address is chosen based on some algorithm. But this algorithm has some limitations which may not always return an IP that is optimal for the end user.
> When the end user reaches that IP to download some object, for example https://download.abc.com/movie.mp4 (potentially with some cookie targeting "download.abc.com"), an algorithm running on that server
> finds out that another server would provide better performance and want to redirect the end user to the new server to download the object.
> Right now what we can do is sending 30X redirection to the end user with something like:
> As I mentioned in earlier posts, this current method breaks cookie and company ABC's client (maybe a mobile App) may refuse to be directed to a different domain.
>
> Hope this is a clear enough description of the use case. Please let me know of any questions.
> Thanks!
>
> Bin
>
>> You get something in the middle ground.  The Origin doesn't change, but cookies don't get sent without extra work.
>> But you gain the ability to decouple the server certificate and trust and integrity from the Origin.
>> It may make sense to take a step back and enumerate the set of problems and use-cases
>> that would be good to solve and see which ones there is interest in addressing, and then
>>> If looks to me that if I use oob encoding for my use case, it will be the same as 30X redirection.
>>> I won't be able to achieve my goal of "only change the IP address, everything else remain the same".
>>>
>>> Bin
>>>
>>>
>>>> 1. In your example, the first host "cdn.com" is the CDN customer's hostname. They usually can't provide us with a "*.geo.cdn.com" wildcard cert. Some of them requires EV certificate, which does not even support wildcard.
>>>>

---

From: Bin Ni <[email-masked]>
Date: Wed, 21 Aug 2019 02:08:10 -0700

Yes, the key of my proposal is a "forced", non-optional redirect, just like
30X, which is fundamentally different from Alt-Svc.
Another problem with Alt-Svc is that the "protocol-id" is mandatory.
If we introduce "forced Alt-Svc" the server also needs to make sure the
client supports the protocol, which is more work for the server.
We need to make the "protocol-id" to the optional.

Bin

On Thu, Aug 15, 2019 at 5:59 AM craigt <[email-masked]> wrote:

> Or a method of signalling that Alt-Svc/equivalent is not optional in
> $case...
>
>
> > My use case is we have a bunch of reverse proxy servers all over the
> world (just like any other CDN providers).
> > They also configure their DNS to point www.abc.com to a CNAME provided
> by us, such that my company essentially takes over the resolution of that
> hostname.
> > When an end-user is visiting download.abc.com, the hostname is resolved
> by our DNS server to one of the server's IP addresses.
> > This IP address is chosen based on some algorithm. But this algorithm
> has some limitations which may not always return an IP that is optimal for
> the end user.
> https://download.abc.com/movie.mp4 (potentially with some cookie
> targeting "download.abc.com"), an algorithm running on that server
> > finds out that another server would provide better performance and want
> to redirect the end user to the new server to download the object.
> >>>> 2. Cookies are often targeting a specific hostname. We can't ask all
> our customer to change the business logic of their web application to make
> sure all cookies are targeting the entire domain.
> >>>>
> >>>>
> >>>>
> >>>> >      > Server-1.1.1.1: Sorry, I can't give you the movie, you need
> to
> >>>> >     ask server
> >>>> >      > 2.2.2.2 for this movie.
> >>>> >     If we for a moment play with the idea that we'd do something
> like
> >>>> >     this, then I
> >>>> >     think it should be aligned with and work together with Alt-Svc
> in a
> >>>> >     better way
> >>>> >     There's no max-age/TTL. For how long is the user-agent supposed
> to
> >>>> >     consider
> >>>> >     the alternative IP addresses as the only ones that the given
> origin
> >>>> >     has?
> >>>> >     Are the alternative IPs supposed to be used for the entire
> origin or
> >>>> >     for that
> >>>> >     specific URI only?
> >>>> >
> >>>> >     If a client gets this response and starts downloading huge
> content
> >>>> >     from the
> >>>> >     new IP and the client then opens a second connection to the
> origin
> >>>> >     in a second
> >>>> >     or not is a totally broken idea and I think this detail needs
> to be
> >>>> >     worked out
> >>>> >     alt-svc instead of this 312 plus several headers, so that a
> client
> >>>> >     can be told
> https://maps.google.com/?q=1919+S+Bascom+Ave+%23600,+Campbell,+CA+95008&entry=gmail&source=g
> >,

-- 

Bin Ni
VP of Engineering

---

From: Mike Bishop <[email-masked]>
Date: Wed, 21 Aug 2019 16:31:52 +0000

Which, again, makes it seem like an OOB encoding is a useful route.  The body’s location is given as a URI; the protocol used to retrieve the resource can vary, because URIs don’t specify protocols.  There are ways around the Cookie issue, and if you’re modifying the client to support this new protocol then you’re also going to modify it to support the target being on a different domain.

From: Bin Ni <[email-masked]>
Sent: Wednesday, August 21, 2019 5:08 AM
To: craigt <[email-masked]>
Cc: Erik Nygren <[email-masked]>; Mike Bishop <[email-masked]>; W. Felix Handte <[email-masked]>; HTTP Working Group <[email-masked]>
Subject: Re: Redirection to Other IP Addresses

Yes, the key of my proposal is a "forced", non-optional redirect, just like 30X, which is fundamentally different from Alt-Svc.
Another problem with Alt-Svc is that the "protocol-id" is mandatory.
If we introduce "forced Alt-Svc" the server also needs to make sure the client supports the protocol, which is more work for the server.
We need to make the "protocol-id" to the optional.

Bin

On Thu, Aug 15, 2019 at 5:59 AM craigt <[email-masked]<mailto:[email-masked]>> wrote:
Just catching up on this thread:

Or a method of signalling that Alt-Svc/equivalent is not optional in $case...

On Wed, 14 Aug 2019 at 23:53, Bin Ni <[email-masked]<mailto:[email-masked]>> wrote:
> Assume company ABC is one of our customers, they may configure all our servers to serve https://download.abc.com, with the certificate for "download.abc.com<http://download.abc.com>" installed on each of the servers.
> They also configure their DNS to point www.abc.com<http://www.abc.com> to a CNAME provided by us, such that my company essentially takes over the resolution of that hostname.
> When an end-user is visiting download.abc.com<http://download.abc.com>, the hostname is resolved by our DNS server to one of the server's IP addresses.
> When the end user reaches that IP to download some object, for example https://download.abc.com/movie.mp4 (potentially with some cookie targeting "download.abc.com<http://download.abc.com>"), an algorithm running on that server
> Location: https://{new IP}.mycompany.com/download.abc.com/movie.mp4<http://mycompany.com/download.abc.com/movie.mp4>
> On Wed, Aug 14, 2019 at 1:11 PM Erik Nygren <[email-masked]<mailto:[email-masked]>> wrote:
>>
>> On Wed, Aug 14, 2019 at 3:38 PM Bin Ni <[email-masked]<mailto:[email-masked]>> wrote:
>>>
>>> Hi Mike and Eric,
>>>
>>> On Wed, Aug 14, 2019 at 7:34 AM Mike Bishop <[email-masked]<mailto:[email-masked]>> wrote:
>>>>
>>>> From: Erik Nygren <[email-masked]<mailto:[email-masked]>>
>>>> To: Bin Ni <[email-masked]<mailto:[email-masked]>>
>>>> Cc: W. Felix Handte <[email-masked]<mailto:[email-masked]>>; HTTP Working Group <[email-masked]<mailto:[email-masked]>>
>>>> The information contained in this email may be confidential and/or legally privileged. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please reply to the sender and destroy all copies of the message. To contact us directly, send to QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008, or visit our website at www.quantil.com<http://www.quantil.com>.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Bin Ni
>>> VP of Engineering
>>>
>>> The information contained in this email may be confidential and/or legally privileged. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please reply to the sender and destroy all copies of the message. To contact us directly, send to QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008, or visit our website at www.quantil.com<http://www.quantil.com>.
>
>
>
> --
>
> Bin Ni
> VP of Engineering
>
> The information contained in this email may be confidential and/or legally privileged. It has been sent for the sole use of the intended recipient(s). If the reader of this message is not an intended recipient, you are hereby notified that any unauthorized review, use, disclosure, dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please reply to the sender and destroy all copies of the message. To contact us directly, send to QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008, or visit our website at www.quantil.com<http://www.quantil.com>.

--

Bin Ni
VP of Engineering

---

From: Rob Sayre <[email-masked]>
Date: Wed, 21 Aug 2019 15:42:13 -0700

> redirecting client to other IP addresses.

Well, this is a very interesting proposal. I'm having trouble imagining
ways it could be misused.

thanks,
Rob

---

From: Bin Ni <[email-masked]>
Date: Wed, 21 Aug 2019 16:25:41 -0700

Hi Rob,

Do you see any security holes in doing this?

Bin

-- 

Bin Ni
VP of Engineering

---

From: Bin Ni <[email-masked]>
Date: Wed, 21 Aug 2019 16:24:48 -0700

Hi Mike,

The thing is there is no way around the cookie issue.
As a CDN provider, we can't ask every customer to change how their web
applications use Cookie AND/OR allow their requests to be redirected to a
different domain. Those are major changes to their business logic.

Bin

On Wed, Aug 21, 2019 at 9:31 AM Mike Bishop <[email-masked]> wrote:

> Which, again, makes it seem like an OOB encoding is a useful route.  The
> body’s location is given as a URI; the protocol used to retrieve the
> resource can vary, because URIs don’t specify protocols.  There are ways
> around the Cookie issue, and if you’re modifying the client to support this
> new protocol then you’re also going to modify it to support the target
> being on a different domain.
>
>
>
> *From:* Bin Ni <[email-masked]>
> *Sent:* Wednesday, August 21, 2019 5:08 AM
> *To:* craigt <[email-masked]>
> *Cc:* Erik Nygren <[email-masked]>; Mike Bishop <
> [email-masked]>; W. Felix Handte <[email-masked]>; HTTP Working
> Group <[email-masked]>
> Yes, the key of my proposal is a "forced", non-optional redirect, just
> like 30X, which is fundamentally different from Alt-Svc.
>

-- 

Bin Ni
VP of Engineering

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 22 Aug 2019 14:18:34 +1000

My .02 - 

Any new protocol element along these lines needs to be optional, to make it backwards-compatible with existing clients.

Introducing a new status code here is application-visible, when you really want behaviour that is hidden. That's one of the big advantages of Alt-Svc, I think.

So, a flag on Alt-Svc that basically says "I really want you to use this alt-svc for any new requests, even if you have to wait for that connection to spin up" would help.

What you'd also need is a way to convey that information to the client for *this* response (since the Alt-Svc header only affects requests after the response it occurs upon).

421 Misdirected request would do that, but it means that clients that don't support 421 or Alt-Svc with a different hostname won't be able to get anything for that URL (see above: re optional).

Modifying the semantics of the new Alt-Svc flag to "I really want you to use this alt-svc for *this* response and any new requests, even if you have to wait for that connection to spin up" might do it.

A 1xx response that contains the Alt-Svc + new flag, followed by a short pause is kind of interesting here; it gives the client the opportunity to sever the connection without wasting (much) bandwidth.

Or you could just put the Alt-Svc + new flag onto the normal response, and pace the start of the response a bit (for ~1RT) to a similar effect.

The biggest barrier to adoption here by far is getting clients to adopt something; Alt-Svc was specified some time ago now, and is still not available cross-host in Chrome. The other thing to look at is how much surface area you're adding -- and therefore, how much security analysis is necessary. Reusing Alt-Svc seems like a good start there.

Cheers,

> On 15 Aug 2019, at 10:59 pm, craigt <[email-masked]> wrote:
>

---

From: Erik Nygren <[email-masked]>
Date: Thu, 22 Aug 2019 08:40:18 -0400

Bin, I'm still not sure if a major part of your use-case is to use
different Alternative Services for different URLs within the same Origin?
That seems like an important input requirement differentiating between a
number of the different options here.  It's also the one that changes the
semantics of Alt-Svc most dramatically over what they are today.  In
addition to being optional, Alt-Svc is also scoped to the Origin which
presumably simplifies both client implementation but also significantly
simplifies semantics.

For example, if a client is requesting a bunch of URLs in parallel from the
same origin
(eg, different large streaming movies), would we want them each to be able
to go to different places?  That seems like a major use-case not covered by
the current Alt-Svc.

A precursor to Alt-Svc covered a bunch of these, although the semantics
were not as fleshed out as they could have been:
draft-nygren-httpbis-connection-redirect-00
That both had a way to scope Alt-Svc to a Path as well as an attribute
indicating that this should be handled immediately:

   2.  The Path specified in the Alt-Server-Policy is a prefix of the
       request-path for the subsequent request (including the case where
       they string-compare equal).
   [...]
   Depending on the policy-when-value the behavior should change:
      * "immediate" - the client SHOULD abort this HTTP request and re-
         issue the request to an alt-server.  This MUST NOT be returned in
         response to non-idempotent requests (such as POST/PUT).
      * "next" = the client SHOULD continue with any in-flight HTTP
         requests but should in-parallel establish a connection to the alt-
         server for use in subsequent requests.
The impact on client complexity was a big reason these were abandoned
in Alt-Svc.

To Mark's point, Alt-Svc also hasn't gotten critical mass of good
client implementations yet, even so.
Handling clients not supporting this was also a challenge (ie, even
with the "immediate" label
as a hint it was less clear what to do about the content itself, such
as whether to return it,
do a 100 response, or pause some for clients to disconnect).

Extrapolating on Mike's OOB encoding suggestion, what about allowing
the Alternative-Service to be provided as a hint along with the OOB
response and scoped to that particular OOB-encoded URL?

Somewhere back in time or in some mail thread I had a proposal which
did this for this purpose.

For example, if the client signaled support through a header and the
response was:

HTTP/1.1 200 OK
Content-Encoding: out-of-band

Vary: Accept-Encoding

{
    "sr": [
      { "r" : "/c/bae27c36-fa6a-11e4-ae5d-00059a3c7a00",
               "alt-svc": [ "h2=\"cdn-node-75.example.net:443\"" ] }
    ]
}

Where in this case the alt-svc was scoped purely to the particular
OOB-encoded URL response.
In the case where the origin matches the origin returning the OOB
response, cookies can be sent.

All of these do add lots of complexity to client connection pool management
as connections no longer
can be keyed purely off of Origin but also need some way to index into some
Alt-Svc or path or URL property as well.
The impact on code complexity for connection management (and a need to
refactor) is a major reason why
one major client never added Alt-Svc support.  Even more complexity here
might make it less likely?

Overall this is a direction that I suspect that a number of
server-side / CDN providers would be quite interested in.
What would it take to get general-purpose client software interested?
Are any here interested? Without them it is an academic exercise.

Another reason to combine with OOB-encoding is that there are enough
other things that can be done with it (especially in cases
where cookies are NOT send and where the server CDN serving the
content is not trusted to see cookies or even the content itself)
to make clients potentially more interested.

       Erik

On Thu, Aug 22, 2019 at 12:18 AM Mark Nottingham <[email-masked]> wrote:

> My .02 -
>
> Any new protocol element along these lines needs to be optional, to make
> it backwards-compatible with existing clients.
>
> Introducing a new status code here is application-visible, when you really
> want behaviour that is hidden. That's one of the big advantages of Alt-Svc,
> I think.
>
> So, a flag on Alt-Svc that basically says "I really want you to use this
> alt-svc for any new requests, even if you have to wait for that connection
> to spin up" would help.
>
> What you'd also need is a way to convey that information to the client for
> *this* response (since the Alt-Svc header only affects requests after the
> response it occurs upon).
>
> 421 Misdirected request would do that, but it means that clients that
> don't support 421 or Alt-Svc with a different hostname won't be able to get
> anything for that URL (see above: re optional).
>
> Modifying the semantics of the new Alt-Svc flag to "I really want you to
> use this alt-svc for *this* response and any new requests, even if you have
> to wait for that connection to spin up" might do it.
>
> A 1xx response that contains the Alt-Svc + new flag, followed by a short
> pause is kind of interesting here; it gives the client the opportunity to
> sever the connection without wasting (much) bandwidth.
>
> Or you could just put the Alt-Svc + new flag onto the normal response, and
> pace the start of the response a bit (for ~1RT) to a similar effect.
>
> The biggest barrier to adoption here by far is getting clients to adopt
> something; Alt-Svc was specified some time ago now, and is still not
> available cross-host in Chrome. The other thing to look at is how much
> surface area you're adding -- and therefore, how much security analysis is
> necessary. Reusing Alt-Svc seems like a good start there.
>
> Cheers,
>
>
> $case....
> servers to serve https://download.abc.com, with the certificate for
> "download..abc.com" installed on each of the servers.
> >> When an end-user is visiting download.abc.com, the hostname is
> resolved by our DNS server to one of the server's IP addresses.
> >>> You get something in the middle ground.  The Origin doesn't change,
> but cookies don't get sent without extra work.
> >>>>>>    or not is a totally broken idea and I think this detail needs to
> be
> >>>>>>    worked out

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 22 Aug 2019 14:45:07 +0100

Hi Erik,

Thanks for sharing, I wasn't aware of
draft-nygren-httpbis-connection-redirect-00 or some of the background to
Alt-Svc. I think there are some attractive properties to being able to
direct based on path but acknowledge the complexities.

One use case I worked on was to do layer 7 direction of adaptive video
streaming clients to different IPs based on the quality of the video [1].
For example, when a client requests a single video segment at HD quality,
all future requests for that quality should go elsewhere. We built a PoC
based on Alt-Svc and it would have been preferable to keep the DASH URL
pattern that encodes the representation detail in the path. Instead,
because of Alt-Svc's origin-level redirection, we had to map this
information into the host, which has several downsides for larger
deployments. At the time, I penciled out what it might look like to add
path-based pattern matching to Alt-Svc, and I probably came up with a
similar answer to what you describe.

[1] draft-pardue-mcast-quic-05,which specifies how Alt-Svc can be used to
advertise the properties of QUIC packets and HTTP/3 payloads being sent to
multicast IP addresses.

On Thu, Aug 22, 2019 at 1:44 PM Erik Nygren <[email-masked]> wrote:

> For example, if a client is requesting a bunch of URLs in parallel from
> the same origin
> The impact on client complexity was a big reason these were abandoned in Alt-Svc.
>
> To Mark's point, Alt-Svc also hasn't gotten critical mass of good client implementations yet, even so.
> Handling clients not supporting this was also a challenge (ie, even with the "immediate" label
> as a hint it was less clear what to do about the content itself, such as whether to return it,
> the Alternative-Service to be provided as a hint along with the OOB response and scoped to that particular OOB-encoded URL?
>
> Somewhere back in time or in some mail thread I had a proposal which did this for this purpose.
>
> For example, if the client signaled support through a header and the response was:
>
> HTTP/1.1 200 OK
> Where in this case the alt-svc was scoped purely to the particular OOB-encoded URL response.
> In the case where the origin matches the origin returning the OOB response, cookies can be sent.
>
> All of these do add lots of complexity to client connection pool
> management as connections no longer
> can be keyed purely off of Origin but also need some way to index into
> some Alt-Svc or path or URL property as well.
> Overall this is a direction that I suspect that a number of server-side / CDN providers would be quite interested in.
> What would it take to get general-purpose client software interested?  Are any here interested? Without them it is an academic exercise.
>
> Another reason to combine with OOB-encoding is that there are enough other things that can be done with it (especially in cases
> where cookies are NOT send and where the server CDN serving the content is not trusted to see cookies or even the content itself)
>> Introducing a new status code here is application-visible, when you
>> really want behaviour that is hidden. That's one of the big advantages of
>> Alt-Svc, I think.
>>
>> What you'd also need is a way to convey that information to the client
>> for *this* response (since the Alt-Svc header only affects requests after
>> the response it occurs upon).
>>
>> Or you could just put the Alt-Svc + new flag onto the normal response,
>> and pace the start of the response a bit (for ~1RT) to a similar effect.
>>
>> >> They also configure their DNS to point www.abc.com to a CNAME
>> provided by us, such that my company essentially takes over the resolution
>> of that hostname.
>> >> finds out that another server would provide better performance and
>> want to redirect the end user to the new server to download the object.
>> >>>>> 2. Cookies are often targeting a specific hostname. We can't ask
>> all our customer to change the business logic of their web application to
>> make sure all cookies are targeting the entire domain.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> or visit our website at www.quantil.com. <https://www.quantil.com/
>> >
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>>
>> >>>>> Bin Ni
>> >>>>> VP of Engineering
>> >>>>>