Subject: [CDNi] comments on requirements

From: Kevin J Ma <[email-masked]>
Date: Fri, 4 Mar 2011 16:15:20 -0500

Hi all,

  Enclosed below are comments and questions on the requirements as promised.

thanx!

--  Kevin J. Ma

Requirements Comments/Questions:

1. "R13  The CDNI Control API SHOULD allow the Downstream CDN to
         communicate to the Upstream CDN aggregate information ..."

   Should there also be a requirement that the Upstream CDN be able to
   negotiate global SLA and policy data to the Downstream CDN (e.g.,
   purge latency or replication latency), or is that considered to be
   content metadata?

2. "R33  The CDNI Request-Routing loop prevention mechanism SHOULD allow
         routing of the request (as opposed to the request loop being
         simply interrupted without routing the request).

   a) Who determines which metrics to use in selecting a CDN to service
      the request?  The CDNI Control API allows communication of metrics
      (R13 and R20), but an administrator needs to be able to set the
      priority of metrics or configure a static default?  The specific
      metrics (e.g., proximity, bandwidth, cost) may be too detailed for
      this document, but perhaps this requirement should refer to a
      "policy-based" routing of a failed request?

   b) In the case where a delivery SLA is being enforced and none of the
      CDNs can support it, is that a case of loop prevention?  In this
      specific case, should the request be given best effort, or should
      an error be generated?  Volation of the SLA may not be prudent?

3. "R37  The CDNI Request-Routing API MUST allow the Downstream CDN to
         include the following information in the response to the
         Upstream CDN: ..."

   In the case where a user solicited what would have been a Downstream
   CDN, or a non-Authoritative CDN (e.g., due to caching of a URL, or
   local NSP proxying policy), is there a requirement that the
   Downstream CDN request permission from and/or notify the previously
   Upstream CDN or Authoritative CDN that it wishes to service the
   request?  If so, should there be a request routing requirement?

4. "R49  The CDNI Metadata API MUST allow removal of obsolete CDNI
         Metadata from the Downstream CDN (this could, for example, be
         achieved via an explicit removal request from the Upstream CDN
         or via expiration of a Time-To-Live associated to the Metadata)."

   I am concerned about what the lack of metadata implies

   e.g., if geo-blocking or window metadata existed and then expired
   due to a metadata TTL, would that imply that the CDN nolonger has to
   enforce geo-blocking or the window?

   The enforcement of actions that are implied by certain metadata must
   have default states with implied default actions?  There may be a
   need for making certain metadata mandatory for all deployments?
   It might be necessary to have compliance levels for different actions?

5. "R51  The CDNI Metadata API MUST allow association of CDNI Metadata at
         the granularity of an object set.  This is necessary to achieve
         scalable distribution of metadata when a large number of objects
         share the same distribution policy.

   Does this imply a precedence structure for medadata, i.e., if
   metadata exists for a object, but that object belongs to a group,
   and a different value for the same metadata is applied to the group,
   will the object metadata take precedence?  What happens when
   overlapping metadata heirarchies conflict?

6 "R52  The CDNI Metadata API MUST provide indication by the Downstream
        CDN to the Upstream CDN of whether the CDNI metadata (and
        corresponding future request redirections) is accepted or
        rejected.  When rejected, the CDNI Metadata API MUST allow the
        Downstream CDN to provide information about the cause of the
        rejection."

   Notification of rejection seems like a logging concern, not a
   metadata concern?

7. "R61  The CDNI Logging API MUST provide logging of deliveries to User
         Agents performed by the Downstream CDN as a result of request
         redirection by the Upstream CDN."

   Should there be a requirement for logging asynchronous purges and
   metadata expirations as well?

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 10 Mar 2011 15:37:13 +0100

Hi Kevin,

>   purge latency or replication latency), or is that considered to be

As discussed as part of your comments on the Problem Statement, we propose propose that detailed SLA parameters are negotiated/established out-of-band and thus not negotiated/advertised in-band.

>      the request?

I see this as the prerogative of the Upstream CDN. It is his choice to serve itself or delegate the serving to another CDN, it is his choice to pick the other CDN, it is his choice to use whatever policy/algorithm/criteria to make such decision.

As discussed in other CDNI requirements, the CDNI APIs may allow a Downstream CDN to advertise some coarse information about its capabilities/state or (possibly in the initial scope or possibly in the longer term) to advertise some dynamic information about footprint/"costs"/etc that can be taken into account by the Upstream CDN in making the decision. But the decision would still be up to the uCDN.

>  The CDNI Control API allows communication of metrics

I am assuming that the CDN selection during routing/rerouting of the request (in the presence, or not, of a potential routing loop) is again entirely up to the Upstream CDN. So the requirement on the CDNI protocol/mechanism is not about the actual decision but rather about providing the sufficient information so the loop can be detected/prevented and so that the Upstream CDN has a chance to modifying its CDN selection so the request is eventually served. 

>      an error be generated?  Volation of the SLA may not be prudent?

As discussed, we don't expect (at least initially) detailed SLA advertisement/negotiation.

>   request?

I think that depends on the authorisation-checks/validation rules that are associated with the content (as define in the Distribution Metadata).

We currently have the following requirement under the CDNI Metadata API section:
"
   R54  The Metadata that can be distributed by the CDNI Metadata API
        MUST allow signaling of authorization checks and validation that
        are to be performed by the surrogate before delivery.  For
        example, this could potentially include:

        *   need to validate URI signed information (e.g.  Expiry time,
            Client IP address).
"
I see this as covering the case you describe above (ie if content Distribution metadata indicate that no check is to be performed, then the cache in dCDN can serve directly, if the metadata indicates that more specific checks/validation are needed, then the dCDN cache must enforce that).

>  If so, should there be a request routing requirement?

I think it is covered by R54. Let us know if you feel something additional is needed.

>   enforce geo-blocking or the window?

I would say that the situation is the same whether geo-blocking/window metadata disappears after metadata TTL expiry of where it has not been advertised in the first place.

>   have default states with implied default actions?  

I think I agree with that.
Clearly a CDN must always know what to do for a content for which no geo-blocking/window metadata has been explicitly provided.
This may be achieved in different ways. For example:
	* a default default could be defined (eg in the absence of anything else, a CDN must assume that content is accessible everywhere at any time)
	* a mechanism to signal an alternate default (eg in the absence of anything else, assume content is accessible nowhere )
	* a mechanism to signal an alternate default for a collection of contents (eg in the absence of per-content metadata, assume content is accessible only in HongKong at anytime)

I think this could be summarized by saying that there is a notion of inheritance in metadata and when a lower level disappears, the higher level takes over (eg if per-content metadata disappears, then the per-content collection metadata takes over. eg if the per-content collection metadata disappears then the explicit default takes over, ...).
Would adding a requirement along those lines address your concern?

> There may be a
>   will the object metadata take precedence?  

Oh well, exactly what I just discussed above. So yes, I would personally expect such precedence.

> What happens when

I suggest we leave the answer to that question for the solution/protocol work.
But we could add a requirement to deal with that situation.

>   Notification of rejection seems like a logging concern, not a

FAir point. But to allow the Upstream CDN to log that some metadata was rejected/why, it seems useful to pass that info in the CDNi API.

>   Should there be a requirement for logging asynchronous purges

I would think so.
Unless we hear otherwise, we will add a requirement to that effect. 

> and

What functional entity would you see logging metadata expiration: the CDNI gateway supporting the CDNI Metadata API? or each surrogate in teh dCDN that expire some metadata?

Thanks

Francois

> 
> 
> 
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni

---

From: Kevin J Ma <[email-masked]>
Date: Thu, 10 Mar 2011 10:51:22 -0500

Hi Francois,

> I think it is covered by R54. Let us know if you feel something additional is needed.

  I am fine with it being covered under R54.

  I like your summary.  A requirement along those lines would be great.

thanx.

--  Kevin J. Ma

From: Francois Le Faucheur [mailto:[email-masked]]
Sent: Thursday, March 10, 2011 9:37 AM
To: Kevin J Ma
Cc: Francois Le Faucheur; [email-masked]
Subject: Re: [CDNi] comments on requirements

Hi Kevin,

Hi all,

thanx!

--  Kevin J. Ma

There may be a

What happens when

I would think so.

and

Thanks

Francois

_______________________________________________
CDNi mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/cdni

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 10 Mar 2011 16:20:07 +0000

Francois, Kevin,

> Unless we hear otherwise, we will add a requirement to that effect. 

I think you want to be careful here in not mixing up different types of things into a single "log". IMO if content purge requests are made via a Control API/protocol then the Control API/protocol should report on their success/failure (which could be done asynchronously). Mixing in "log lines" related to content purge with "log lines" related to delivery is not the right approach IMO.

> What functional entity would you see logging metadata expiration: the CDNI gateway supporting the CDNI Metadata API? or each surrogate in teh dCDN that expire some metadata?

If Kevin intends "metadata expiration" to mean a Control API/protocol action to invalidate metadata then see above (IMO it should be via the Control API/protocol) if you mean the natural expiration of the TTL (or equivalent) on a metadata record I personally see no need to log that at all - I am not aware of any CDN product on the market today that logs the natural expiration of a metadata TTL.

Ben

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 10 Mar 2011 17:32:17 +0100

On 10 Mar 2011, at 17:20, Ben Niven-Jenkins wrote:

> I think you want to be careful here in not mixing up different types of things into a single "log". IMO if content purge requests are made via a Control API/protocol then the Control API/protocol should report on their success/failure (which could be done asynchronously).

So the Control API would possibly involve:
	* a response that means "I have understood your purge request and will do my best to get it enforced" , and 
	* a response that means "I have now triggered the corresponding purge in relevant caches and I have explicit confirmation that content has been removed".

Is this what you mean?

Thanks

Francois

> Mixing in "log lines" related to content purge with "log lines" related to delivery is not the right approach IMO.
> 
>>> and

---

From: Kevin J Ma <[email-masked]>
Date: Thu, 10 Mar 2011 11:39:27 -0500

Ben/Francois,

  In the case of purge, my concern was for cascaded CDNs.  If the purge
  goes to an authoritative CDN, does the response from the downstream
  CDNs imply that all cascaded CDNs below that have been purged?  If this
  is the case, then I think Ben's suggestion of leaving it as part of the
  Control API response is fine.  I think we want to be able to know when
  all purges have completed.

  In the case of metadata TTL, certain metadata it probably will not make
  sense to set a TTL, but I was considering a case where the TTL is set
  incorrectly or inadvertently, specific on metadata with implied actions.
  Thinking about traceability in an error condition, e.g., if content
  suddenly disappears or becomes available when it should not have, would
  there be a benefit in having a log at the CDN gateway level?
  Debuggability may be beyond the scope of the current discussion?

thanx.

--  Kevin J. Ma

> -----Original Message-----
> From: Ben Niven-Jenkins [mailto:[email-masked]]
> Sent: Thursday, March 10, 2011 11:20 AM
> To: Francois Le Faucheur
> Cc: Kevin J Ma; [email-masked]
> I think you want to be careful here in not mixing up different types of things
> into a single "log". IMO if content purge requests are made via a Control
> API/protocol then the Control API/protocol should report on their
> success/failure (which could be done asynchronously). Mixing in "log lines"
> related to content purge with "log lines" related to delivery is not the right
> approach IMO.
> 
> >> and
> > What functional entity would you see logging metadata expiration: the CDNI
> gateway supporting the CDNI Metadata API? or each surrogate in teh dCDN that
> expire some metadata?
> 
> If Kevin intends "metadata expiration" to mean a Control API/protocol action
> to invalidate metadata then see above (IMO it should be via the Control
> API/protocol) if you mean the natural expiration of the TTL (or equivalent) on
> a metadata record I personally see no need to log that at all - I am not aware
> of any CDN product on the market today that logs the natural expiration of a
> metadata TTL.
> 
> Ben
>

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 10 Mar 2011 17:16:53 +0000

Francois,

We are probably getting into solution mode, but see inline.

> 	* a response that means "I have now triggered the corresponding purge in relevant caches and I have explicit confirmation that content has been removed".

My personal preference would be for the Control API/protocol to be a RESTful WebService. There are of course other ways to implement it.

The entity wishing to purge something (the "Purger") would HTTP POST a ControlRequest (e.g. purge content X) and be returned a HTTP 202 response with a Content-Location referring to a URI Y.

The 202 response means "I have accepted your request and will do my best to get it enforced" and the Content-Location Y means "go to URI Y to check the status of your request".

The "Purger" could subsequently perform HTTP GETs to URI Y to check whether the purge had completed or not. With Interconnect an upstream CDN could cascade the original purge request and store the returned URI (Z). If the upstream CDN receives a GET to Y it could make a GET to Z to check on the progress in the downstream CDN (it could also periodically perform GETs to Z to check the status for its own purposes if it wanted to).

FWIW our product supports a similar approach today to enable CSPs to pre-publish content into the CDN.

Ben

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 10 Mar 2011 17:26:10 +0000

On 10 Mar 2011, at 16:39, Kevin J Ma wrote:

>  CDNs imply that all cascaded CDNs below that have been purged?

IMO, Yes. It is upto the CDN that delegated the request to ensure it reports on the total of (i) its status plus (ii) the status of any CDNs it has delegated the request to.

The fact that downstream CDNs have been used should be largely transparent to anyone upstream of the delegating CDN IMO.

>  If this

I think this may depend on what you mean by TTL. IMO the TTL of the metadata just states how long that metadata record is valid for before it needs to be revalidated/re-obtained. The metadata TTL expiring does not imply the content should be purged just that the metadata needs to be re-validated and possibly re-obtained. In that way it is similar to the behaviour of the HTTP Expires header/max-age directive.

If there is a requirement to no longer offer content after a specific time that should be an explicit property of the content contained in the metdata record, it should not be inferred from the TTL of the metadata record.

This is because the TTL of the metadata record is a property of the metadata record itself and not a property of the content that the metadata record refers to (if that makes sense).

Ben

---

From: Kevin J Ma <[email-masked]>
Date: Thu, 10 Mar 2011 13:42:28 -0500

That makes sense.  thanx.

> Sent: Thursday, March 10, 2011 12:26 PM
> To: Kevin J Ma
> IMO, Yes. It is upto the CDN that delegated the request to ensure it reports
> on the total of (i) its status plus (ii) the status of any CDNs it has
> delegated the request to.
> 
> The fact that downstream CDNs have been used should be largely transparent to
> anyone upstream of the delegating CDN IMO.
> 
> >  If this
> I think this may depend on what you mean by TTL. IMO the TTL of the metadata
> just states how long that metadata record is valid for before it needs to be
> revalidated/re-obtained. The metadata TTL expiring does not imply the content
> should be purged just that the metadata needs to be re-validated and possibly
> re-obtained. In that way it is similar to the behaviour of the HTTP Expires
> header/max-age directive.
> 
> If there is a requirement to no longer offer content after a specific time
> that should be an explicit property of the content contained in the metdata
> record, it should not be inferred from the TTL of the metadata record.
> 
> This is because the TTL of the metadata record is a property of the metadata
> record itself and not a property of the content that the metadata record
> refers to (if that makes sense).
> 
> 
> Ben
> 
> 
> > thanx.
> >
> > --  Kevin J. Ma
> >

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 10 Mar 2011 21:53:12 +0100

On 10 Mar 2011, at 18:16, Ben Niven-Jenkins wrote:

> Francois,
> 
> We are probably getting into solution mode,

This is helpful IMO.

Wrt requirements I-D, I propose to:
	* in the CDNI Logging API section: not add any requirement related to content purge
	* in the Control API section: right after the requirement indicating that the uCDN can trigger actions (such as Content Purge) in the dCDN, add a requirement indicating that "the CDNI Control API MUST allow the downstream CDN to report on the completion of these actions, in a manner appropriate for the action (e.g. synchronously or asynchronously)."

Francois

> but see inline.

Francois Le Faucheur
Distinguished Engineer
Corporate Development
[email-masked]
Phone: +33 49 723 2619
Mobile: +33 6 19 98 50 90

Cisco Systems France
Greenside
400 Ave de Roumanille
06410 Sophia Antipolis
France
Cisco.com

 Think before you print.

This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message.

Cisco Systems France, Société à responsabiité limitée, Rue Camille Desmoulins – Imm Atlantis Zac Forum Seine Ilot 7 92130 Issy les Moulineaux, Au capital de 91.470 €, 349 166 561 RCS Nanterre, Directeur de la publication: Jean-Luc Michel Givone.

For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 10 Mar 2011 22:12:22 +0100

>   I like your summary.  A requirement along those lines would be great.

Here is what I propose to add:

Rx	The CDNI Metadata API MUST support multiple levels of inheritance with precedence to more specific metadata. For example, the CDNI Metadata API may support metadata that is applicable to any content, metadata that is applicable to a content collection and metadata that is applicable to an individual content where content level metadata overrides content collection metadata that overrides metadata for any content.

Ry	The CDNI Metadata API MUST ensure that conflicting metadata with overlapping scope are prevented or deterministically handled.

Francois

Francois Le Faucheur
Distinguished Engineer
Corporate Development
[email-masked]
Phone: +33 49 723 2619

Cisco Systems France
Greenside
400 Ave de Roumanille
06410 Sophia Antipolis
France
Cisco.com

 Think before you print.

---

From: Kevin J Ma <[email-masked]>
Date: Thu, 10 Mar 2011 16:45:18 -0500

sounds good.  thanx.

From: Francois Le Faucheur [mailto:[email-masked]]
Sent: Thursday, March 10, 2011 4:12 PM
To: Kevin J Ma
Cc: Francois Le Faucheur; [email-masked]
Subject: Re: [CDNi] comments on requirements

Hi Francois,

Francois

thanx.

--  Kevin J. Ma

From: Francois Le Faucheur [mailto:[email-masked]]
Sent: Thursday, March 10, 2011 9:37 AM
To: Kevin J Ma
Cc: Francois Le Faucheur; [email-masked]<mailto:[email-masked]>
Subject: Re: [CDNi] comments on requirements

Hi Kevin,

Hi all,

thanx!

--  Kevin J. Ma

There may be a

What happens when

I would think so.

and

Thanks

Francois

Francois Le Faucheur
Distinguished Engineer
Corporate Development
[email-masked]<mailto:[email-masked]>
Phone: +33 49 723 2619
Mobile: +33 6 19 98 50 90

Cisco Systems France
Greenside
400 Ave de Roumanille
06410 Sophia Antipolis
France
Cisco.com<http://www.cisco.com>

 Think before you print.

---

From: David R Oran <[email-masked]>
Date: Thu, 10 Mar 2011 19:12:28 -0500

> Ry      The CDNI Metadata API MUST ensure that conflicting metadata with overlapping scope are prevented or deterministically handled.
Two small nits:

1. "more specific" is possibly ill-defined when content collections are considered, unless:
(a) collections are defined by enumerable sets as opposed to fuzzy sets
(b) precedence is a total order, rather than a partial order.
It also needs to be made clear if the inheritance is only by atomic metadata item, or something more complicated.

2. "deterministically handled" can mean either or both of:
a) if I hand the same set of conflicting metadata to the API N times, the resulting conflict resolution is identical
b) if I hand the same set of conflicting metadata to two different instances of the CDNI metadata API, the different instances return identical results.

(a) requires a each CDNI API instance have a deterministic conflict resolution algorithm.
(b) requires all CDNI API implementations have isomorphic conflict resolution algorithms.

I'm not sure if we need (a) or (b), or (a)+(b), or neither, if nothing really bad happens as long as certain loop-prevention constraints are met without enforcing determinism.

Despite what I say in both of the above, as long as these requirements aren't going to be submitted to the international criminal court or the pedantic rule interpretation bureau, I'm OK with sweeping these small points under the rug.

But I will keep a little "I told you so" card if the ambiguity ever comes back to bite us. :-)

---

From: Francois Le Faucheur <[email-masked]>
Date: Fri, 11 Mar 2011 14:04:02 +0100

> But I will keep a little "I told you so" card if the ambiguity ever comes back to bite us. :-)

I'll go for the "under the rug" for now. But hopefully we'll get to discuss those before they bite us. 
Let's bring up these detailed points if/when we start the actual metadata distribution protocol definition/specification. We could also then decide to reflect some of the conclusions into the requirements document (or decide that this level of functionality is beyond the scope of the CDNI requirements document anyways). 

Thanks

Francois

Francois Le Faucheur
Distinguished Engineer
Corporate Development
[email-masked]
Phone: +33 49 723 2619

Cisco Systems France
Greenside
400 Ave de Roumanille
06410 Sophia Antipolis
France
Cisco.com

 Think before you print.