Subject: Eric Rescorla's Discuss on draft-ietf-httpbis-cdn-loop-01: (with DISCUSS and COMMENT)

From: Eric Rescorla <[email-masked]>
Date: Thu, 20 Dec 2018 05:19:47 -0800

Eric Rescorla has entered the following ballot position for
draft-ietf-httpbis-cdn-loop-01: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-httpbis-cdn-loop/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Rich version of this review at:
https://mozphab-ietf.devsvcdev.mozaws.net/D12072

This seems like can be easily fixed, but I do think it needs to be
fixed.

DETAIL
S 2.
>      header if necessary).
>   
>      The token identifies the CDN as a whole.  Chosen token values SHOULD
>      be unique enough that a collision with other CDNs is unlikely.
>      Optionally, the token can have semicolon-separated key/value
>      parameters, to accommodate additional information for the CDN's use.

I don't know how to understand "unique enough" as a conformance
requirement. I think you need to specify something specific here, like
"globally unique" or some other scope. I don't insist that you provide
a construction algorithm, though obviously that would be good.

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

S 1.
>      in a "loop" accidentally; because routing is achieved through a
>      combination of DNS and forwarding rules, and site configurations are
>      sometimes complex and managed by several parties.
>   
>      When this happens, it is difficult to debug.  Additionally, it
>      sometimes isn't accidental; loops between multiple CDNs be used as an

can be used

S 2.
>      CDN-Loop = #cdn-id
>      cdn-id   = token *( OWS ";" OWS parameter )
>   
>      Conforming Content Delivery Networks SHOULD add a value to this
>      header field to all requests they generate or forward (creating the
>      header if necessary).

Can this header only go in a request?

S 3.
>      through configuration) and servers (including intermediaries) SHOULD
>      NOT use it for other purposes.
>   
>   3.  Security Considerations
>   
>      The threat model that the CDN-Loop header field addresses is a

As Alissa points out, this also potentially leaks the CDN you use,
even if that would otherwise be hidden. For instance, suppose that a
request goes A -> B -> C but B is hidden (doesn't add anything to the
headers). If you know B's token, then you can tell if this is the case
or not., by injecting it yourself and seeing if you get service. Seems
like you should document this.

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 10:27:31 +1100

Hello EKR,

> On 21 Dec 2018, at 12:19 am, Eric Rescorla <[email-masked]> wrote:

> a construction algorithm, though obviously that would be good.

There are relatively few CDNs, and they're pretty aware of each other. I'm not convinced a strict / algorithmic means of identifying them will add value; consider the name spaces of things like HTTP headers, and how little the registry is actually used, and how few collisions problems we actually encounter. Also, I wanted this field to be someone flexible so that a CDN could choose to use multiple tokens if it wanted to; e.g., some use a hierarchy, others use multiple "networks" with different configurations.

If this text is causing so much heartburn, I suggest we just remove the sentence (and probably remove "as a whole.").

>>     sometimes isn't accidental; loops between multiple CDNs be used as an

Already fixed.

> Can this header only go in a request?

Yes.

> like you should document this.

It leaks the CDN the origin configures to the origin -- what attack are you concerned about here?

--
Mark Nottingham   https://www.mnot.net/

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 20 Dec 2018 15:45:13 -0800

On Thu, Dec 20, 2018 at 3:27 PM Mark Nottingham <[email-masked]> wrote:

> There are relatively few CDNs, and they're pretty aware of each other.

That doesn't seem like a design constraint we should start with.

I'm not convinced a strict / algorithmic means of identifying them will add
> value; consider the name spaces of things like HTTP headers, and how little
> the registry is actually used, and how few collisions problems we actually
> encounter. Also, I wanted this field to be someone flexible so that a CDN
> could choose to use multiple tokens if it wanted to; e.g., some use a
> hierarchy, others use multiple "networks" with different configurations.
>
> If this text is causing so much heartburn, I suggest we just remove the
> sentence (and probably remove "as a whole.").
>
>

I'm not going to be any happier about that. I don't think you need to
provide a construction, but if the requirement is that they be unique in
some fashion, then you need to state what that fashion is.

> > Can this header only go in a request?

OK. Can you say MUST NOT in a response?

> you concerned about here?

If the client forges it, it leaks it to the client, no?

-Ekr

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 10:59:38 +1100

> On 21 Dec 2018, at 10:45 am, Eric Rescorla <[email-masked]> wrote:
> 
> I'm not convinced a strict / algorithmic means of identifying them will add value; consider the name spaces of things like HTTP headers, and how little the registry is actually used, and how few collisions problems we actually encounter. Also, I wanted this field to be someone flexible so that a CDN could choose to use multiple tokens if it wanted to; e.g., some use a hierarchy, others use multiple "networks" with different configurations.
> 
> I'm not going to be any happier about that. I don't think you need to provide a construction, but if the requirement is that they be unique in some fashion, then you need to state what that fashion is.

I was saying that we should remove the requirement. This is similar to Via's ability to use a pseudonym; there is no uniqueness requirement there (we trust that the motivation of the party generating it to make it useful is sufficient).

> OK. Can you say MUST NOT in a response?

It's explicitly called out as a request header. We don't place such requirements on many other request-only headers (e.g., Host).

> If the client forges it, it leaks it to the client, no?

Can you be more specific -- what do you mean by "client" and "it" here?

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 20 Dec 2018 16:17:48 -0800

On Thu, Dec 20, 2018 at 3:59 PM Mark Nottingham <[email-masked]> wrote:

> > I'm not convinced a strict / algorithmic means of identifying them will
> add value; consider the name spaces of things like HTTP headers, and how
> little the registry is actually used, and how few collisions problems we
> actually encounter. Also, I wanted this field to be someone flexible so
> that a CDN could choose to use multiple tokens if it wanted to; e.g., some
> use a hierarchy, others use multiple "networks" with different
> configurations.
> >
> I was saying that we should remove the requirement. This is similar to
> Via's ability to use a pseudonym; there is no uniqueness requirement there
> (we trust that the motivation of the party generating it to make it useful
> is sufficient).
>

I'm not persuaded this true. Collisions here seem to create an obvious
interop problem, and so the specification should tell you how to avoid them.

> It's explicitly called out as a request header. We don't place such
> requirements on many other request-only headers (e.g., Host).
>

All right.

> Can you be more specific -- what do you mean by "client" and "it" here?

We have a situation with two alternate topologies:

1.  A -> B -> Origin
2.  A ->  Origin

The original HTTP client (i.e., an external attacker on the Internet) sends
a request with a CDN-Loop header containing B. In topology (1) this causes
some kind of failure and in topology (2) it does not, thus leaking the
topology.

-Ekr

---

From: Adam Roach <[email-masked]>
Date: Thu, 20 Dec 2018 18:40:19 -0600

On 12/20/18 6:17 PM, Eric Rescorla wrote:
>
>
> On Thu, Dec 20, 2018 at 3:59 PM Mark Nottingham <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>
>     > On 21 Dec 2018, at 10:45 am, Eric Rescorla <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > I'm not convinced a strict / algorithmic means of identifying
>     them will add value; consider the name spaces of things like HTTP
>     headers, and how little the registry is actually used, and how few
>     collisions problems we actually encounter. Also, I wanted this
>     field to be someone flexible so that a CDN could choose to use
>     multiple tokens if it wanted to; e.g., some use a hierarchy,
>     others use multiple "networks" with different configurations.
>     >
>     > If this text is causing so much heartburn, I suggest we just
>     remove the sentence (and probably remove "as a whole.").
>     >
>     >
>     > I'm not going to be any happier about that. I don't think you
>     need to provide a construction, but if the requirement is that
>     they be unique in some fashion, then you need to state what that
>     fashion is.
>
>     I was saying that we should remove the requirement. This is
>     similar to Via's ability to use a pseudonym; there is no
>     uniqueness requirement there (we trust that the motivation of the
>     party generating it to make it useful is sufficient).
>
>

I agree with EKR here. The more I think about the situation -- and 
Mark's responses in particular ("There are relatively few CDNs, and 
they're pretty aware of each other") -- the more I'm convinced that, 
while the proposed solution *probably* works in 2018, it may well fail 
in 2028, depending on how the CDN market evolves. If history is any 
guide, engineers are pretty terrible futurists. The best we can do is 
plan for scaling beyond that which we can presently imagine (cf. IPv4).

I think some means of collision avoidance here is required prior to 
document publication.

/a

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 12:13:19 +1100

On 21 Dec 2018, at 11:17 am, Eric Rescorla <[email-masked]> wrote:
> The original HTTP client (i.e., an external attacker on the Internet) sends a request with a CDN-Loop header containing B. In topology (1) this causes some kind of failure and in topology (2) it does not, thus leaking the topology.

Ah - so you're saying that 'A' is also a CDN, not the user-agent?

If the answer is 'yes', I understand; will add some text.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 12:19:02 +1100

On 21 Dec 2018, at 11:40 am, Adam Roach <[email-masked]> wrote:
> 
> I agree with EKR here. The more I think about the situation -- and Mark's responses in particular ("There are relatively few CDNs, and they're pretty aware of each other") -- the more I'm convinced that, while the proposed solution *probably* works in 2018, it may well fail in 2028, depending on how the CDN market evolves. If history is any guide, engineers are pretty terrible futurists. The best we can do is plan for scaling beyond that which we can presently imagine (cf. IPv4). 
> 
> I think some means of collision avoidance here is required prior to document publication.

I agree engineers are terrible futurists; however (and perhaps as a result), we're really really good at over-engineering things for anticipated problems that don't eventuate. My viewpoint here is informed by observing how many Web-related registries operate (or fail to).

The current specification allows us to insert more stringent requirements in the future if necessary. Personally, I think that's sufficient. 

However, if you insist, I think the lowest-impact way to address this is to adopt a structure similar to that used in Via, (hostname under control of the CDN or pseudonym) and encourage (but not require) use of a hostname.

Would that be sufficient for you?

If we do that, we might want to revisit the example, since its use of 'host' might be confusing to readers.

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 13:40:11 +1100

> On 21 Dec 2018, at 12:13 pm, Mark Nottingham <[email-masked]> wrote:
> 

I've added:

"""
A CDN's use of the CDN-Loop header field might expose its presence. For example, if CDN A is configured to forward its requests to CDN B for a given origin, CDN B's presence can be revealed if it behaves differently based upon the presence of the CDN-Loop header field.
"""

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 20 Dec 2018 20:44:24 -0600

> A CDN's use of the CDN-Loop header field might expose its presence. For example, if CDN A is configured to forward its requests to CDN B for a given origin, CDN B's presence can be revealed if it b…

Sorry for being slow to catch up.

It's not entirely clear to me that you actually need A to be a CDN for this
to work -- if I want to know if CDN B is used to serve a given origin,
can't I just tell my browser to add B's CDN-Loop header field to the
request?  (With the idea being that if that change causes me to not get
content, then I know CDN B is in use.)

-Benjamin

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 13:46:32 +1100

> On 21 Dec 2018, at 1:44 pm, Benjamin Kaduk <[email-masked]> wrote:
> 

I'm not following you. There are any number of ways to determine that a given CDN is configured to serve a given origin; how does CDN-Loop specifically change this?

Cheers,

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 20 Dec 2018 20:52:19 -0600

> I'm not following you. There are any number of ways to determine that a given CDN is configured to serve a given origin; how does CDN-Loop specifically change this?

I seem to just be saying that CDN-Loop adds one more.

(And to be clear, I'm not saying that your added text is wrong.)

-Benjamin

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 13:55:12 +1100

> On 21 Dec 2018, at 1:52 pm, Benjamin Kaduk <[email-masked]> wrote:
> 

Could you walk me through it? I'm still not seeing it.

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 20 Dec 2018 20:59:13 -0600

> Could you walk me through it? I'm still not seeing it. 

Sure, though it's probably getting to be not worth our respective time...

I want to know if CDN B serves site X.  I configure site Y as an origin
for my own site behind CDN B and observe what CDN-Loop header values I
receive at Y.  I make a request to site X twice, once with that CDN-Loop
header and once without.  If they give the same response, I conclude that X
does not use CDN B.  If the request with CDN-Loop header doesn't get
content back, I conclude that X uses CDN B.

-Benjamin

---

From: Adam Roach <[email-masked]>
Date: Thu, 20 Dec 2018 22:34:04 -0600

> I agree engineers are terrible futurists; however (and perhaps as a result), we're really really good at over-engineering things for anticipated problems that don't eventuate. My viewpoint here is…

No one here is asking for a registry.

> However, if you insist, I think the lowest-impact way to address this is to adopt a structure similar to that used in Via, (hostname under control of the CDN or pseudonym) and encourage (but not re…

It's probably enough to say that it has to end in a domain name under 
control of the CDN, but your proposal works as well.

> If we do that, we might want to revisit the example, since its use of 'host' might be confusing to readers.

I always assume that spec changes that impact examples imply changes in 
the corresponding examples, so yes. Thanks.

/a

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 21 Dec 2018 16:04:00 +1100

See:
  https://github.com/httpwg/http-extensions/commit/59505dbd1883

> On 21 Dec 2018, at 3:34 pm, Adam Roach <[email-masked]> wrote:
> 
> It's probably enough to say that it has to end in a domain name under control of the CDN, but your proposal works as well.
> 
> 
> I always assume that spec changes that impact examples imply changes in the corresponding examples, so yes. Thanks.
> 
> 
> /a
> 
>