Subject: [CDNi] upstream cdn enforce content policies

From: Johan Rydberg <[email-masked]>
Date: Thu, 04 Aug 2011 11:52:46 +0200

After following the discussions for a while, I have a question:

Instead of constructing a complex metadata exchange protocol,
why not force the upstream CDN to enforce any prolicy rules that
the CSP has put on the content?

In the case of DNS-based routing, the CDN can answer the DNS
lookup with the address of one of its own CDNI request routers
that will enforce any policies before redirecting (HTTP 302) to
a downstream CDN.

The upside to this is that redirects between CDNs will always be
HTTP 302, regardless if the initial mechanism was DNS-based.
Also, the complex metadata protocol can be elimited, minimizing
the scope of the CDNI work.

-
Johan Rydberg

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 4 Aug 2011 11:10:52 +0100

Johan,

> the CSP has put on the content?

That would not provide sufficient coverage to ensure that the policy is enforced as clients may attempt to request content directly from a downstream CDN, bypassing the upstream CDN.

> the scope of the CDNI work.

Even if one could avoid the need for a downstream CDN to apply some policies, this wouldn't eliminate the need to exchange CDNI metadata as other CDNI metadata such as rate limiting, whether an encrypted channel is required, where to obtain the content from etc. is still required.

---

From: Johan Rydberg <[email-masked]>
Date: Thu, 04 Aug 2011 12:16:24 +0200

> That would not provide sufficient coverage to ensure that the policy is enforced as clients may attempt to request content directly from a downstream CDN, bypassing the upstream CDN.
There must be ways to make sure that the client first passes through the 
upstream
CDN.

I have a feeling that if the interconnect protocols are not limited in 
scope, this
CDNI effort will either (1) fail or (2) be in some "design phase" for ever.

> Even if one could avoid the need for a downstream CDN to apply some policies, this wouldn't eliminate the need to exchange CDNI metadata as other CDNI metadata such as rate limiting, whether an enc…
It sounds like most of those could be folded into the request routing 
protocol.
Rate limiting and encryption might not be attributes of the content, but 
rather
of the client.  (Some customers pay more, and get a higher bitrate for 
example)

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 4 Aug 2011 11:54:42 +0100

Johan,

On 4 Aug 2011, at 11:16, Johan Rydberg wrote:

> There must be ways to make sure that the client first passes through the upstream
> CDN.

I don't see how you can guarantee that behaviour.

For example, how do you handle the upstream request router being the only policy enforcement point if a client (after being redirected by a request router) holds a persistent HTTP connection open to the surrogate and uses it to request multiple pieces of content?

> I have a feeling that if the interconnect protocols are not limited in scope, this

There are two parts to that: (1) what interfaces does CDNI need to support and (2) what is the scope of data/functionality that needs to be incorporated into each interfaces.

Removing the metadata interface (and stuffing the resultant functionality into request routing as you suggest below) doesn't reduce the design work required it just moves it to a different interface.

I agree that we need to constrain the scope of data/functionality we incorporate into each interface to a pragmatic set that is capable of delivering the required services so as not to end up "boiling the ocean" but hiding metadata functionality in another interface doesn't achieve that.
  
> It sounds like most of those could be folded into the request routing protocol.
> Rate limiting and encryption might not be attributes of the content, but rather
> of the client.  (Some customers pay more, and get a higher bitrate for example)

You can't assume that request routing has happened for every request (see above) and even if you could, doing so would introduce a degree of coupling between surrogates and request routers that IMO is undesirable.

Ben

---

From: "Woundy, Richard" <[email-masked]>
Date: Thu, 4 Aug 2011 14:34:40 +0000

A couple of observations, as an individual and not a working group chair.

> Instead of constructing a complex metadata exchange protocol

How about we construct a *simple* metadata exchange protocol? :)

I believe we are anticipating this protocol to be XML/JSON over HTTP.

> I have a feeling that if the interconnect protocols are not limited in scope, this CDNI effort will either (1) fail or (2) be in some "design phase" for ever.

CDNI was approved as a WG in late June. Now it is early August and we're already in danger of failing? Hmmm.

It feels to me like we are debating theoretical scenarios. Johan, if you feel strongly about eliminating this interface, maybe you could write up your proposal as an internet-draft that we can discuss and debate? We also need to ensure that your proposal meets the use cases identified by this group.

-- Rich

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Johan Rydberg
Sent: Thursday, August 04, 2011 6:16 AM
To: Ben Niven-Jenkins
Cc: [email-masked]
Subject: Re: [CDNi] upstream cdn enforce content policies

_______________________________________________
CDNi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/cdni

---

From: Johan Rydberg <[email-masked]>
Date: Thu, 04 Aug 2011 16:48:34 +0200

Richard,

I just got a bit scared when I saw discussions about features that "may 
be good to have"
or "it would be nice to support".

I guess all I wanted to say was that everyone would benefit from a 
minimized scoop.

---

From: "Woundy, Richard" <[email-masked]>
Date: Thu, 4 Aug 2011 15:02:39 +0000

OK, fair enough. There are several participants that want CDNI to be "as simple as possible, but no simpler" -- including me. :)

-----Original Message-----
From: Johan Rydberg [mailto:[email-masked]] 
Sent: Thursday, August 04, 2011 10:49 AM
To: Woundy, Richard
Cc: Ben Niven-Jenkins; [email-masked]
Subject: Re: [CDNi] upstream cdn enforce content policies

Richard,

---

From: "Peterson, Larry" <[email-masked]>
Date: Thu, 4 Aug 2011 11:30:32 -0400

Picking up on Johan's minimized scope thread...

I too have been puzzled by the metadata interface. If you read the language 
carefully, you'll see there's a caveat that this interface might exist in various 
forms, presumably as a combination of other interfaces and mechanisms,
for example:
   o the uCDN doesn't route the request to the dCDN unless policy permits;
   o the uCDN sets HTTP caching directives consistent with policy and the
      dCDN obeys those directives; and
   o the uCDN invokes purge on dCDN should the policy change.

But that generous interpretation aside, the high level issue seems to be how
access control policy is enforced. In a single CDN it's natural for this policy to 
be enforced by the CDN, and there's likely a metadata-like interface by which
the CSP expresses this policy to that CDN.

The question is: does interconnection imply that uCDN further delegates that
responsibility for enforcing access control policy to the dCDN, or does the uCDN
retain that locus of control for itself. The later design, which essentially treats 
the dCDN as it would any other cache (at least with respect to metadata), seems 
the simplest to me.

Larry

---

From: Bhumip Khasnabish <[email-masked]>
Date: Thu, 4 Aug 2011 11:59:16 -0400

once handed over to dCDN, uCDN should probably be not  tracking any
further.. unless the dCDN allows it. If it does, things will definitely get
more complicated because different uCDN may have different requirements..
which may be out of scope..

On Thu, Aug 4, 2011 at 11:30 AM, Peterson, Larry <[email-masked]>wrote:

> scope, this CDNI effort will either (1) fail or (2) be in some "design
> phase" for ever.
> >>>
> >> It feels to me like we are debating theoretical scenarios. Johan, if you
> feel strongly about eliminating this interface, maybe you could write up
> your proposal as an internet-draft that we can discuss and debate? We also
> need to ensure that your proposal meets the use cases identified by this
> group.
> >>
> >> -- Rich
> >>
> >>> That would not provide sufficient coverage to ensure that the policy is
> enforced as clients may attempt to request content directly from a
> downstream CDN, bypassing the upstream CDN.
> >>>
> >>>
> >>> Even if one could avoid the need for a downstream CDN to apply some
> policies, this wouldn't eliminate the need to exchange CDNI metadata as
> other CDNI metadata such as rate limiting, whether an encrypted channel is
> required, where to obtain the content from etc. is still required.
> >>>
> >>>

---

From: Kevin J Ma <[email-masked]>
Date: Thu, 4 Aug 2011 15:00:20 -0400

> retain that locus of control for itself. 

I think the issue is more the threat that a CSP may have an agreement with
the uCDN, which it has paid for, which may not be enforceable through dCDNs.
It is not clear that a uCDN can always enforce those policies.  It is not that
hard to figure out where the content is really coming from (i.e., the dCDN).
Will CSPs, who tend to be risk averse, just disallow delegation?

Sticking with the minimalist approach, I still think there is value in a
standardized method of exchanging opaque metadata.  If some CDNs agree 
amongst themselves (outside of CDNI) to enforce policies, they will most
likely need to exchange metadata?  Having a well defined protocol to do so
seems like an important first step?

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Peterson, Larry
> Sent: Thursday, August 04, 2011 11:31 AM
> To: Johan Rydberg
> Cc: [email-masked]
> >> CDNI was approved as a WG in late June. Now it is early August and
> we're already in danger of failing? Hmmm.
> >>
> >> It feels to me like we are debating theoretical scenarios. Johan, if
> you feel strongly about eliminating this interface, maybe you could write
> up your proposal as an internet-draft that we can discuss and debate? We
> also need to ensure that your proposal meets the use cases identified by
> this group.
> >>
> >> -- Rich
> >>
> >>> That would not provide sufficient coverage to ensure that the policy
> is enforced as clients may attempt to request content directly from a

---

From: "Peterson, Larry" <[email-masked]>
Date: Thu, 4 Aug 2011 15:14:42 -0400

Good questions... In my mind, the metadata interface is used to delegate
responsibility from uCDN to dCDN. The CSP still has to trust that the dCDN
behaves according to the metadata it's passed. Alternatively, the CSP trusts
that the uCDN -- with which it has a contractual agreement -- to "stay in
the loop" and keep a tighter leash on what the dCDN does (e.g., make a
decision on each request it routes, issues purge commands when necessary,
and adds cache directives as necessary). Yes, we have to trust the dCDN to
observe cache directives, so you could argue both strategies involve trusting
the dCDN, but you could also argue that the latter approach narrows the
window of vulnerability if trust is violated.

As to the position that "we just as well define the interface in case some pair
of CDNs want to use it"... sure, but every bit of mechanism comes at some cost
in complexity.

Larry

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 4 Aug 2011 12:39:48 -0700

Hi Kevin,

I agree with you here.

We need to define metadata for content to have the CSP delegate the
policies, which could include the content not leaving the uCDN, or being
passed to a defined set of dCDN's.

Thanks,
Vishwas

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Fri, 5 Aug 2011 00:19:56 +0100

Larry,

> window of vulnerability if trust is violated.

Can you expand on that argument because I can't see how the latter approach narrows the window of vulnerability without introducing an undesirable level of coupling between the upstream & downstream CDNs?

Thanks
Ben

---

From: "Peterson, Larry" <[email-masked]>
Date: Fri, 5 Aug 2011 08:45:06 -0400

Ben,

I think the phrasing of your question makes my point. In one model,
there remains a strong coupling. The uCDN is involved in every request 
routing decision (for example), and hence, retains the ability to interject
itself (and its customer's will) on content delivery. In the other model,
the uCDN delegates (through metadata) all responsibility to the dCDN.

Is that level of coupling undesirable? In m view, no. Certainly I think we
can agree the uCDN is going to be involved in request routing since the
URL is going to contain a domain name that the uCDN is going to have
deal with one way or another. I hope we can also agree that the dCDN
is going to need to respect caching directives, so I don't see the problem
with taking advantage of that.

Let me turn this around and talk about why I'm worried that the metadata
interface is going to be tricky... and maybe people will correct me on this.
My sense is that metadata is in a semantically rich space (at least relative 
to a CDN's pure caching function), and so reaching agreement on universal
CDN metadata is going to be a challenge. 

Larry

---

From: ZhouZhipeng <[email-masked]>
Date: Fri, 05 Aug 2011 08:40:15 +0000

I agree with Richard that for CDNI it is just in an early stage and the use cases and requirements are still being discussed.
As for the metadata, my feeling is the question includes two parts: what may be contained in the metadata; how to use the metadata.
The conclusion needs time to be given at last.
Zhipeng

-----邮件原件-----
发件人: Woundy, Richard [mailto:[email-masked]] 
发送时间: 2011年8月4日 22:35
收件人: Johan Rydberg; Ben Niven-Jenkins
抄送: [email-masked]
主题: Re: [CDNi] upstream cdn enforce content policies

-- Rich

---

From: "Woundy, Richard" <[email-masked]>
Date: Fri, 5 Aug 2011 17:10:07 +0000

Speaking personally...

> My sense is that metadata is in a semantically rich space (at least relative to a CDN's pure caching function), and so reaching agreement on universal CDN metadata is going to be a challenge.

Maybe we are aiming for the wrong goal. Instead of agreement on "universal" CDN metadata, we should be aiming for "core" CDN metadata -- what do we agree we need (as a consensus)?

If there is "universal" metadata that we *might* need, we could leverage vendor extensions to the initial metadata interface, and/or include them in a future version of the specification.

If we cannot agree on any "core" metadata, then we should closely examine to see if we still need the metadata interface.

-- Rich

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Peterson, Larry
Sent: Friday, August 05, 2011 8:45 AM
To: Ben Niven-Jenkins
Cc: [email-masked]
Subject: Re: [CDNi] upstream cdn enforce content policies

Ben,

Larry

---

From: Scott Wainner <[email-masked]>
Date: Mon, 08 Aug 2011 00:36:35 -0400

Richard,

     I too am confused about the use of the Metadata interface to relay 
metadata.  Seems the term metadata is overloaded and can mean 'anything 
we deem relevant' which will get us in trouble.  In looking through the 
requirements document, I can see concrete evidence of specific 
information in the following:

R50: acquisition protocol and uri of the content to be pre-positioned or 
dynamically acquired
R51: list of candidate Origin sources
R58: metadata exchange status
R59: policies; presumably policies associated with an object or set of 
objects
R60: authorization checks

Perhaps these elements represent a few of the 'core' metadata of which 
we speak.

When I think of metadata, I usually think of it in the context of 
metadata for a specific object.  For example, a media file will need 
metadata that describes the use of the media (offering date, expiration 
date, black-out locations, trick mode restrictions, transcode 
restrictions, cipher requirements, etc.).  The metadata is not in the 
media; the metadata must be associated with the media.  This is 
typically done by packaging the media and metadata together.  If we're 
talking about dynamic acquisition or pre-positioning of the media only, 
then we will need some means of relaying the associated metadata.

The capabilities of the dCDN seemed to be defined in the Routing Request 
requirements.  Need to delineate where the policies in R59 are handled 
.. Routing Request or Metadata.

Scott

On 8/5/11 1:10 PM, Woundy, Richard wrote:
>
> Speaking personally...
>
> > My sense is that metadata is in a semantically rich space (at least 
> relative to a CDN's pure caching function), and so reaching agreement 
> on universal CDN metadata is going to be a challenge.
>
> Maybe we are aiming for the wrong goal. Instead of agreement on 
> "universal" CDN metadata, we should be aiming for "core" CDN metadata 
> -- what do we agree we need (as a consensus)?
>
> If there is "universal" metadata that we *might* need, we could 
> leverage vendor extensions to the initial metadata interface, and/or 
> include them in a future version of the specification.
>
> If we cannot agree on any "core" metadata, then we should closely 
> examine to see if we still need the metadata interface.
>
> -- Rich
>
> > Can you expand on that argument because I can't see how the latter 
> approach narrows the window of vulnerability without introducing an 
> undesirable level of coupling between the upstream & downstream CDNs?
> >
> > Thanks
> > Ben
> >
> >>
> >>>> carefully, you'll see there's a caveat that this interface might 
> exist in
> >>>> various
> >>>> But that generous interpretation aside, the high level issue 
> seems to be
> >>>> how
> >>>> access control policy is enforced. In a single CDN it's natural 
> for this
> >>>> policy to
> >>>> be enforced by the CDN, and there's likely a metadata-like 
> interface by
> >>>> which
> >>>> The question is: does interconnection imply that uCDN further 
> delegates
> >>>> that
> >>>> responsibility for enforcing access control policy to the dCDN, 
> or does
> >>>> the uCDN
> >>>> the dCDN as it would any other cache (at least with respect to 
> metadata),
> >>>> seems
> >>>> the simplest to me.
> >>>>
> >>>> Larry
> >>>>
> >>>>>> There must be ways to make sure that the client first passes 
> through
> >>>> the
> >>>>>> upstream
> >>>>>> CDN.
> >>>>>>
> >>>>>> CDNI effort will either (1) fail or (2) be in some "design 
> phase" for
> >>>> ever.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> Rate limiting and encryption might not be attributes of the 
> content,
> >>>> but
> >>>>>> rather