Subject: Re: Indicating Chosen Service #443

From: Erik Nygren <[email-masked]>
Date: Thu, 17 Apr 2014 18:54:19 -0400

On Mon, Mar 31, 2014 at 12:37 PM, Martin Thomson
<[email-masked]>wrote:

> >From @mnot in #443:
>
> It's likely necessary for the server to know which protocol the user
> agent has chosen, and perhaps even the hostname (for load balancing).
> At the very least, there should be a flag in SETTINGS that indicates
> that an alternate service has been used.
>
> >From me:
>
> How is it that the combination of the protocol that is actually in
> use, and maybe Via, is not sufficient?
>
> As in, you get a connection from a client, using HTTP/2.  That's
> usually a good sign that they are connecting via HTTP/2.
>
> Presumably you know the names you can be reached by (it isn't always
> true that you know what DNS records point to you, conceded, but let's
> pretend that you aren't operating completely blind).  But that doesn't
> matter, because what's actually important is what domains you are
> claiming to be authoritative for, which you have to know.
>
>
So, why?
This is particularly important when using the ALTSVC frame to send traffic
to a different hostname but the same protocol, especially if any individual
server might have many hostnames by which it could be reached, and if the
hostname sets are overlapping.  The ALTSVC hostname is routing information,
and if the recipient of the connection doesn't have this information it can
be problematic to avoid loops, provide stickiness, troubleshoot, and know
where load is coming from for load-balancing purposes.

For example, if you return "ALTSVC: port=443, proto=h2t, host=
us-east-1.example.com, max-age=360" for www.example.com then it's highly
valuable for the server at us-east-1.example.com to know that it was
reached that way (as it might also have us-east-2.example.com and
us-east-1.example.org pointing to it).  This would allow it to: 1) know
that it shouldn't return another ALTSVC frame immediately (loop
prevention); 2) give some sense for other preferable ALTSVC host values
should be for nearby session affinity if returning another ALTSVC before a
GOAWAY/DRAIN; 3) allow it to log how clients got there for diagnostics
purposes; and 4) allow it to report load properly associated with
us-east-1.example.com, especially if other DNS names could have been used
to reach it.

Not all scenarios will require all of these.  However, not passing through
routing information to servers that can be reached multiple ways has turned
out to be unfortunate down the road for multi-tenant situations (lack of
manditory Host header in the first HTTP versions, lack of SNI in TLS, etc).

           Erik

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 25 Apr 2014 11:23:25 +1000

I think it'd be helpful to talk about concrete proposals here.

AFAICT there are a number of ways to do this, based on discussion to date:

a) A new HTTP request header, to be sent upon the first request (at least); e.g., 
   Service: h2;alt.example.com:443

b) A new frame type (maybe SVC?) whose payload is a serialisation of the alternative service tuple

c) A flag to send in SETTINGS that indicates an alternative service is in use

Each of these has its downsides; (a) feels like a layer violation, but is easy to do; (b) might be considered "heavy", but is probably the "correct" way to do this, and (c) is very easy to do, but doesn't provide all of the information.

Are there any others?

What do people think about the tradeoffs above, as well as doing this in general?

Cheers,

On 18 Apr 2014, at 8:54 am, Erik Nygren <[email-masked]> wrote:

> On Mon, Mar 31, 2014 at 12:37 PM, Martin Thomson <[email-masked]> wrote:
> >From @mnot in #443:
> 
> This is particularly important when using the ALTSVC frame to send traffic to a different hostname but the same protocol, especially if any individual server might have many hostnames by which it could be reached, and if the hostname sets are overlapping.  The ALTSVC hostname is routing information, and if the recipient of the connection doesn't have this information it can be problematic to avoid loops, provide stickiness, troubleshoot, and know where load is coming from for load-balancing purposes.
> 
> For example, if you return "ALTSVC: port=443, proto=h2t, host=us-east-1.example.com, max-age=360" for www.example.com then it's highly valuable for the server at us-east-1.example.com to know that it was reached that way (as it might also have us-east-2.example.com and us-east-1.example.org pointing to it).  This would allow it to: 1) know that it shouldn't return another ALTSVC frame immediately (loop prevention); 2) give some sense for other preferable ALTSVC host values should be for nearby session affinity if returning another ALTSVC before a GOAWAY/DRAIN; 3) allow it to log how clients got there for diagnostics purposes; and 4) allow it to report load properly associated with us-east-1.example.com, especially if other DNS names could have been used to reach it.  
> 
> Not all scenarios will require all of these.  However, not passing through routing information to servers that can be reached multiple ways has turned out to be unfortunate down the road for multi-tenant situations (lack of manditory Host header in the first HTTP versions, lack of SNI in TLS, etc).
> 
>            Erik
> 

--
Mark Nottingham   http://www.mnot.net/

---

From: Martin Thomson <[email-masked]>
Date: Thu, 24 Apr 2014 21:43:45 -0700

> What do people think about the tradeoffs above, as well as doing this in general?

I'm still not 100% convinced.  I guess that it depends on how complex
you imagine the redirection chain will ultimately be.  For cases where
alt-svc is moving clients from a "first-hit" server to the "right"
server, then it's probably not that useful.  For load balancing
scenarios, it's fairly straightforward for clients to detect
flip-flopping and other pathological scenarios (similar in many
respects to redirect loop detection; the difference there being that
Referer and occasionally URL rewrites offer the server some hope of
finding problems).

If we do this, let's do it right.  A frame type that directly mirrors
ALTSVC (yeah, SVC might work) - maybe omitting only the expiration
time - is probably best.  That way a client can say to a server, I'm
asking about this origin, because I received the following ALTSVC
information.  You know, a client could just use ALTSVC.  Apart from
one thing....

We might want to examine this general mechanism for privacy purposes.
We don't need yet another channel for client fingerprinting.

---

From: Erik Nygren <[email-masked]>
Date: Fri, 25 Apr 2014 10:08:55 -0400

As an option (d), which might be rejected out of hand:

d) a HEADER sent early in the connection on streamid 0 with something like:
     { ":svc", "h2;alt.example.com:443" }

I do wonder if we'll keep running into other cases where we want
connection-level key/value extensibility without needing to add new frame
types each time?

     Erik

Sent from my mobile device
On Apr 24, 2014 9:22 PM, "Mark Nottingham" <[email-masked]> wrote:

> Each of these has its downsides; (a) feels like a layer violation, but is
> easy to do; (b) might be considered "heavy", but is probably the "correct"
> way to do this, and (c) is very easy to do, but doesn't provide all of the
> information.
>
> Are there any others?
>
> > This is particularly important when using the ALTSVC frame to send
> traffic to a different hostname but the same protocol, especially if any
> individual server might have many hostnames by which it could be reached,
> and if the hostname sets are overlapping.  The ALTSVC hostname is routing
> information, and if the recipient of the connection doesn't have this
> information it can be problematic to avoid loops, provide stickiness,
> troubleshoot, and know where load is coming from for load-balancing
> purposes.
> >
> > Not all scenarios will require all of these.  However, not passing
> through routing information to servers that can be reached multiple ways
> has turned out to be unfortunate down the road for multi-tenant situations
> (lack of manditory Host header in the first HTTP versions, lack of SNI in
> TLS, etc).
> >
> >            Erik
> >
>
> --

---

From: Martin Thomson <[email-masked]>
Date: Fri, 25 Apr 2014 09:12:28 -0700

>      { ":svc", "h2;alt.example.com:443" }

That's probably not a good idea.  HEADERS already means a lot of
different things, in particular it's used to open a stream.  So that
might be a way to open the door to a whole plethora of bugs.  Or at
least complexity.

> connection-level key/value extensibility without needing to add new frame

I think that I've seen enough evidence to suggest that we might want
to at least leave the issue of extensibility open until the next
meeting.

---

From: Erik Nygren <[email-masked]>
Date: Fri, 25 Apr 2014 13:42:21 -0400

When we've started mocking up what we might implement here, it seems to be
impossible to prevent fairly common pathological situations without at
least a flag, at least when there is any overlap between the servers
handling initial and altsvc requests (which seems likely in some cases).
The hacks around this (different IPs, setting cookies and hoping they match
up, etc) get ugly fast.

I'd agree on what you suggest around the "do it right".  (ie, a frame that
directly mirrors the ALTSVC that was used.)

There are certainly risks around client fingerprinting if abused, although
particularly concerned clients could ignore the ALTSVC sent by the server.

---

From: Martin Thomson <[email-masked]>
Date: Fri, 25 Apr 2014 11:05:19 -0700

> particularly concerned clients could ignore the ALTSVC sent by the server.

That's the extreme option.  I'm thinking that we might just be able to
scrub the expiry time and maybe even allow the first domain component
to be replaced with '*', so that wildcard certificate owners are
unable to use 'somehighentropylabel.example.com' as their ALTSVC in
the interests of tracking.  I need to think about that last one a
little more, it might be that it's overkill.

---

From: Erik Nygren <[email-masked]>
Date: Fri, 25 Apr 2014 14:08:49 -0400

> little more, it might be that it's overkill.

Given that the ALTSVC name isn't associated with the cert name, the
highentropylabel could exist across multiple labels.  There are also
legitimate uses for wanting to know the first domain component (eg, if it
is routing information or the name of the server cluster the user was sent
to).

---

From: Martin Thomson <[email-masked]>
Date: Fri, 25 Apr 2014 11:40:35 -0700

On 25 April 2014 11:08, Erik Nygren <[email-masked]> wrote:
> legitimate uses for wanting to know the first domain component (eg, if it is
> routing information or the name of the server cluster the user was sent to).

Yes, I thought of that, but there are practical limits to what can be
done for labels that can't be covered by a wildcard.  It's certainly
imperfect.

Perhaps the best thing to do is provide security (privacy)
considerations on the matter.  I think that we can safely scrub the
expiration time.

I think that ideally, I'd like to make the indicator optional.  What
are the consequences for you if some clients follow ALTSVC but don't
provide an indicator?

---

From: Erik Nygren <[email-masked]>
Date: Fri, 25 Apr 2014 15:00:32 -0400

Consequences of not getting the indicator:
1) We'd likely return another ALTSVC response immediately under some/many
circumstances
2) Some parts of load reporting and load balancing become harder