Subject: Clarification on Alternative Service Connection Failures

From: Samuel Hurst <[email-masked]>
Date: Fri, 29 Sep 2017 12:10:52 +0100

Hi,

I've got another question about HTTP Alternative Services [RFC7838]. In
section 2.4, it says the following:

> if the connection to the alternative service fails or is unresponsive,
> the client MAY fall back to using the origin or another alternative
> service

What actually counts as a failure in the connection? Is it only for
transport/network layer connection failures, or could an application
layer failure also count?

For example, I make a request to a http origin and it responds with a
valid Alt-Svc header indicating that there is a https alternative.
Connecting to the https origin succeeds and I can make my request.
However, that https alternative has been misconfigured or is otherwise
having issues and it returns a 500 Internal Server Error response for my
request. Would that be considered a failure of the connection?

In addition, if the request to another alternative or back to the origin
succeeds, would it then be wise to drop the failing alternative from my
Alt-Svc cache?

The way I've read it, I think that a 500 series response would be
considered a failure, and I would mark it as a bad alternative in my
cache and then make another request to the next highest priority
alternative (or back to the insecure origin if the application is okay
with this).

Best Regards,
Sam

---

From: Patrick McManus <[email-masked]>
Date: Fri, 29 Sep 2017 13:27:22 +0000 (UTC)

The text is referring to any failure - the client MAY want to try a
different alternative in that case. Its just saying don't feel pinned down
by an Alt-Svc announcement just because you started to use it - you can
always use a different one (or the default) if you think it would work
better. A 500 seems a reasonable input to that choice to me.

---

From: Samuel Hurst <[email-masked]>
Date: Fri, 29 Sep 2017 14:33:31 +0100

On 29/09/17 14:27, Patrick McManus wrote:
> different alternative in that case. Its just saying don't feel pinned
> down by an Alt-Svc announcement just because you started to use it - you
> can always use a different one (or the default) if you think it would
> work better. A 500 seems a reasonable input to that choice to me.

Thanks for the clarification, that's exactly what I thought but just
wanted to make sure.

-Sam

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 29 Sep 2017 12:50:28 -0700

On Fri, Sep 29, 2017 at 12:48 PM, Ryan Hamilton <[email-masked]> wrote:

> On Fri, Sep 29, 2017 at 6:27 AM, Patrick McManus <[email-masked]>
> wrote:
>
> ​Wow, really? Hm. While I understand your reasoning here, I don't love it.
> It seems like we might want to have more concrete advice to browsers about
> what kinds of failures should count as a failure. Otherwise we'll end up
> with different behavior in different browsers for no terribly good reason.
> For example, what about 4xx errors? Should all 4xx and all 5xx errors
> count? What behavior should servers be able to count on? Is it "safe" to
> advertise an alternative which is not current reachable (I think so). What
> about one which 404s for all requests, or one which 500s every request?
> Should this be considered "safe" for a server to advertise?
>
> I'm not sure I have a terribly well thought out idea here, but this makes
> me a bit nervous.
>

​One more thought. Should non-idempotent requests be resent in the case
where an alternative 500s?​

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 29 Sep 2017 12:48:58 -0700

---

From: Lucas Pardue <[email-masked]>
Date: Fri, 29 Sep 2017 21:11:12 +0000

I'm inferring from the responses that Firefox and Chrome currently don't do much in this regard for handling all variations of errors. Although perhaps 421 is special because there is explicit text describing what to do.

Of course, it is not just browsers that are affected by this. Curl is looking to add Alt-Svc support [1], and in our case Sam's been working on an Alt-Svc component for software/intermediaries with no real user facing UI.

I'm not sure how general the advice could be but I think it could be useful (a BCP?). RFC 7838 already calls out that "it is advantageous for clients to behave in a predictable way when alternative services are used by servers".

I'd hazard a guess that QUIC's ongoing development and deployment will see more people coming to Alt-Svc. The easy thing to do is to just scrape response headers for the presence of "Alt-Svc: hq=..." and then start switching action, it's feasible this might not match the expectations of the server deployment. E.g. short cache lifetimes that are ignored the client.

I'm also reminded of previous thoughts I've had about documentation of strategies for opening up a connection to the alternative in the background, and whether the client is restricted to only selecting a single alternative (i'm sure its in the RFC but can't find that particular line right now).

Lucas

[1] https://github.com/curl/curl/wiki/QUIC
________________________________
From: Ryan Hamilton [[email-masked]]
Sent: 29 September 2017 20:50
To: Patrick McManus
Cc: Sam Hurst-RD; HTTP Working Group
Subject: Re: Clarification on Alternative Service Connection Failures

On Fri, Sep 29, 2017 at 12:48 PM, Ryan Hamilton <[email-masked]<mailto:[email-masked]>> wrote:
On Fri, Sep 29, 2017 at 6:27 AM, Patrick McManus <[email-masked]<mailto:[email-masked]>> wrote:
The text is referring to any failure - the client MAY want to try a different alternative in that case. Its just saying don't feel pinned down by an Alt-Svc announcement just because you started to use it - you can always use a different one (or the default) if you think it would work better. A 500 seems a reasonable input to that choice to me.

​Wow, really? Hm. While I understand your reasoning here, I don't love it. It seems like we might want to have more concrete advice to browsers about what kinds of failures should count as a failure. Otherwise we'll end up with different behavior in different browsers for no terribly good reason. For example, what about 4xx errors? Should all 4xx and all 5xx errors count? What behavior should servers be able to count on? Is it "safe" to advertise an alternative which is not current reachable (I think so). What about one which 404s for all requests, or one which 500s every request? Should this be considered "safe" for a server to advertise?

I'm not sure I have a terribly well thought out idea here, but this makes me a bit nervous.

​One more thought. Should non-idempotent requests be resent in the case where an alternative 500s?​

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 29 Sep 2017 14:38:16 -0700

On Fri, Sep 29, 2017 at 2:11 PM, Lucas Pardue <[email-masked]>
wrote:

> I'm inferring from the responses that Firefox and Chrome currently don't
> do much in this regard for handling all variations of errors. Although
> perhaps 421 is special because there is explicit text describing what to do.
>

I can't speak to the behavior of Firefox, but Chrome basically only stops
using an alternative where there are connection errors. HTTP response codes
do not factor into this decision at all. It wouldn't be terribly hard to
implement, of course. But when we implemented this logic we thought that
connection failures were the "right" kind of failures to look at.
Conveniently, since they happen when the connection is established, it
means that idempotency of requests does not come into play.

> Of course, it is not just browsers that are affected by this. Curl is
> looking to add Alt-Svc support [1], and in our case Sam's been working on
> an Alt-Svc component for software/intermediaries with no real user facing
> UI.
>
> I'm not sure how general the advice could be but I think it could be
> useful (a BCP?). RFC 7838 already calls out that "it is advantageous for
> clients to behave in a predictable way when alternative services are used
> by servers".
>
> I'd hazard a guess that QUIC's ongoing development and deployment will see
> more people coming to Alt-Svc. The easy thing to do is to just scrape
> response headers for the presence of "Alt-Svc: hq=..." and then start
> switching action, it's feasible this might not match the expectations of
> the server deployment. E.g. short cache lifetimes that are ignored the
> client.
>
> I'm also reminded of previous thoughts I've had about documentation of
> strategies for opening up a connection to the alternative in the
> background, and whether the client is restricted to only selecting a single
> alternative (i'm sure its in the RFC but can't find that particular line
> right now).
>

​Document this seems sensible. That being Chrome only connects in the
background when it has a connection open to the​ original server. If it
does not have an open connection then it'll only open a connection to the
alternative, unless that fails. This is because we expect alternatives to
mostly work and hence we don't want to waste resources creating two
connections when one would suffice.

---

From: Patrick McManus <[email-masked]>
Date: Fri, 29 Sep 2017 22:03:49 +0000 (UTC)

> ​Wow, really? Hm. While I understand your reasoning here, I don't love it.

I think I should underscore my point - these hosts form a set of equivalent
alternates. If one of them isn't working well for you - using another one
is a sensible thing. Errors that seem to be about the host, rather than the
resource, seem like valid input into that decision. There is never a
directive that you have to use any particular alternate.. the ordering
indicates the server's preference but the client is not bound by that.

> For example, what about 4xx errors?
>

other than the special case of 421, that's not related to the host or
connection.. it should be a function of the origin and the request. so I
wouldn't think that's a reasonable input to the algorithm. But it wouldn't
be out of spec to panic and flush the alt-svc cache on seeing it (though
not very helpful either).

>   Is it "safe" to advertise an alternative which is not current reachable
> (I think so).
>

I think so too. That's definitely a connection error :)

> What about one which 404s for all requests, or one which 500s every
> request? Should this be considered "safe" for a server to advertise?
>

500 is an unexpected error on the server, that's pretty different than a
404.

---

From: Patrick McManus <[email-masked]>
Date: Fri, 29 Sep 2017 22:07:27 +0000 (UTC)

> where an alternative 500s?​
The answer is the same whether or not there is an alternate.  The
alternates are semantically equivalent to each other.

You can say that its inconsistent to say that and also that a 500 might
make you flush the alt-svc entry from the cache, but you can flush the
entry any time you don't like it (too slow for example).

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 29 Sep 2017 15:21:33 -0700

On Fri, Sep 29, 2017 at 3:03 PM, Patrick McManus <[email-masked]>
wrote:

> I think I should underscore my point - these hosts form a set of
> equivalent alternates. If one of them isn't working well for you - using
> another one is a sensible thing. Errors that seem to be about the host,
> rather than the resource, seem like valid input into that decision. There
> is never a directive that you have to use any particular alternate.. the
> ordering indicates the server's preference but the client is not bound by
> that.
>
>

​Hm. I see your point. I guess I'm accustomed to thinking of 2xx and 3xx
and "good" and 4xx and 5xx as "dammit why is the server broken". (I've seen
404s happening because two different servers behind the same LB were
running out of sync software and depending on which you hit you'd get a 200
or a 404. But that's relatively pathological.)  Semantically I guess 5xx
really is in a different class here. Ok, I'm with you.
http://crbug.com/770379

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 29 Sep 2017 15:26:27 -0700

> alternates are semantically equivalent to each other.

It's true that they're semantically equivalent. Totally with you.

But pre-Alt-Svc we would never automatically retry an even idempotent
request on receipt ​of a 5xx. But I think you're suggesting that it in an
Alt-Svc world, that might be something a browser would do?

---

From: Patrick McManus <[email-masked]>
Date: Fri, 29 Sep 2017 23:33:02 +0000 (UTC)

> Alt-Svc world, that might be something a browser would do?

you have defeated my plan of not answering the question..

I don't really know that the alt-svc vs default-svc makes a difference to
me.. the real question is one of replay risk and that answer seems equally
opaque in each circumstance because you have no idea what the source of the
unexpected internal error might be.

fwiw 5xx does not actually impact firefox's alt-svc cache today.. but it
seems like the right change to make.

---

From: Ryan Hamilton <[email-masked]>
Date: Fri, 29 Sep 2017 19:44:27 -0700

> you have defeated my plan of not answering the question..

​\o/
​

> unexpected internal error might be.

​Fair point. I confess, my understanding of Alt-Svc is quite colored by my
experience with QUIC. Since many (~5%, irrc) users do not have UDP
connectivity, when QUIC is a specified Alt-Svc, connection errors are
expected and need to be transparent to the user. Since QUIC has 0-RTT, this
also means that we end up needing to transparently "retry" requests which
were sent on failed 0-RTT connections. So in my mind marking an alternative
server as not working is synonymous with retry, but that doesn't really
need to be the case. And your point about all the alternatives being
semantically equivalent makes me that that perhaps a 5xx should not trigger
a retry, but may well cause us to stop using the alternative.

> seems like the right change to make.

​Ditto. That reminds me. So when Chrome decides to stop using an
alternative, it marks that alternative as broken for 5 minutes (subject to
exponential backoff). Does Firefox do something similar? (This is, of
course, an artifact of QUIC in which if a user has no UDP connectivity but
the server keeps advertising a QUIC alternative, we wouldn't want to keep
attempting QUIC connections in the background)

Cheers,

Ryan
​