Subject: [CDNi] CDNI Requirements I-D

From: Francois Le Faucheur <[email-masked]>
Date: Wed, 26 Jan 2011 13:03:03 +0100

Folks,

We've posted a "CDNI Requirements" I-D:
https://datatracker.ietf.org/doc/draft-lefaucheur-cdni-requirements/

Please, consider this as a first cut whose aim is to facilitate discussion and convergence on what are "urgent/important" requirements (that would have to be addressed in a potential initial WG charter) versus what are "less-urgent/less-important" requirements (that would be left for future rechartering or left out of scope altogether).

Cheers,

Francois, Mahesh, Grant & Yiu

Begin forwarded message:

> From: IETF I-D Submission Tool <[email-masked]>
> Date: 26 January 2011 12:51:46 CET
> To: [email-masked]
> Cc: [email-masked], [email-masked], [email-masked]
> Subject: New Version Notification for draft-lefaucheur-cdni-requirements-00 
> 
> 
> A new version of I-D, draft-lefaucheur-cdni-requirements-00.txt has been successfully submitted by Francois Le Faucheur and posted to the IETF repository.
> 
> Filename:	 draft-lefaucheur-cdni-requirements
> Revision:	 00
> Title:		 Content Distribution Network Interconnection (CDNI) Requirements
> Creation_date:	 2011-01-26
> WG ID:		 Independent Submission
> Number_of_pages: 22
> 
> Abstract:
> Content Delivery Networks (CDNs) are frequently used for large-scale
> content delivery.  As a result, existing CDN providers are scaling up
> their infrastructure and many Network Service Providers (NSPs) are
> deploying their own CDNs.  There is a requirement for interconnecting
> standalone CDNs so that their collective CDN footprint can be
> leveraged for the end-to-end delivery of content from Content Service
> Providers (CSPs) to end users.  However, no standards or open
> specifications currently exist to facilitate such CDN
> interconnection.
> 
> The "CDN Interconnect Problem Statement" Internet-Draft outlines the
> problem area for the IETF with a view towards creating a working
> group.  This working group would work on an interoperable and
> scalable solution for CDN interconnection.  The goal of the present
> document is to start outlining the requirements for such a "CDN
> Interconnection" solution.
> 
> 
> 
> The IETF Secretariat.
> 
>

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Fri, 28 Jan 2011 15:45:51 +0000

Francois,

> https://datatracker.ietf.org/doc/draft-lefaucheur-cdni-requirements/

Thanks for starting the process of gathering & documenting requirements for CDNI.

I've included below some comments on the document. There is a lot of material in the document and I need to spend some more time thinking about parts of it (e.g. which API may be the best place to address certain requirements), so please just treat my comments as a first pass.

In most cases I have quoted the text of the requirement as context and then put my comments underneath.

Comments:

   R3  The CDNI solution MUST NOT require to expose the intra-CDN
       information across CDNs (e.g.  Surrogates topology, surrogates
       status, cached content, ...) for effective and efficient delivery
       of the content.

I would suggest a rewording to:

"The CDNI solution MUST NOT require intra-CDN information to be exposed to other CDNs for effective and efficient delivery of the content. Examples of intra-CDN information include surrogate topology, surrogate status, cached content, etc."

   R7  The CDNI solution MAY support acquisition across CDNs based on
       other protocols than HTTP.

s/other protocols than/protocols other than/

   R8  The CDNI solution SHOULD support cascaded CDN redirection (CDN1
       redirects to CDN2 that redirects to CDN3) to an arbitrary number
       of levels.

Based on Haibin’s e-mail may be include an additional requirement that it should be possible to limit the number of levels. Same comment applies to R10.

   R13  The CDNI Control API SHOULD allow the Downstream CDN to
        communicate to the Upstream CDN aggregate information on the
        Downstream CDN capabilities, resources and affinities (i.e.
        Preferences or cost).  This information can be taken into
        account by the Upstream CDN Request Routing system in its CDN
        Selection decisions.  This information could, for example,
        include:
        *   supported content types and delivery protocols
        *   footprint (e.g. layer-3 coverage)
        *   a set of metrics/attributes (e.g.  Streaming bandwidth,
            storage resources, distribution and delivery priority)
        *   a set of affinities (e.g.  Preferences, indication of
            distribution/delivery fees)
        *   information to facilitate request redirection (e.g.
            Reachability information of Downstream CDN Request Routing
            system). 

I am not sure what the exact split should be but I’d suggest “information to facilitate request redirection” fits under the CDNI Request Routing API and much of the other examples may fit best under the CDNI Metadata API.

My thinking is that the CDNI Metadata API is really about distributing information about an identifiable object (e.g. metadata on a piece of content, geo-blocking policy, etc.) and for example we could treat information about footprint in a very similar way to how we would treat distribution of IP block details for geo-blocking. As such information is needed by more than just request routers Metdata API seems a better fit.

   R16  The CDNI Control API MUST allow the Upstream CDN to request the
        Downstream CDN (and potential cascaded Downstream CDNs - if
        cascaded CDNs are supported by the solution) to interrupt
        delivery of an object (or object set) and trigger specific cache
        actions or behaviors in the Downstream CDN surrogates (e.g.
        Object removal from cache, revalidation).  [Editor's note: this
        functionality is currently described in
        draft-jenkins-cdni-problem-statement-01 under the Metadata API
        but probably fits better under the Control API].

I think “interrupt delivery of an object” may imply a real-time requirement on a downstream CDN to perform these actions and I do not think we need that.

It might be clearer to break out the individual requirements in R16 as bullets, e.g:

“The CDNI Control API MUST allow the Upstream CDN to request that the Downstream CDN (and potential cascaded Downstream CDNs – if cascaded CDNs are supported by the solution) perform the following actions on an object or object set:
  * Mark an object(s) and/or its CDNI metadata as “stale” and revalidate them before they are delivered again.
  * Remove an object(s) and/or its CDNI metadata from its surrogates and any storage."

   R19  The CDNI Control API SHOULD allow bootstrapping of the Request-
        Routing API.  For example, this can potentially include:
        *   negotiation of the Request-Routing method (e.g.  DNS vs
            HTTP, if more than one method is specified)

I am not sure that negotiation of the Request Routing method is possible. The Request Router that responds to the User Agent/DNS resolver has to respond using the protocol it received the request on, so the choice of Request Routing method is driven by the protocol the UA connected to the Request Router on.

   R23  The CDNI Control API SHOULD support virtualization of the
        Downstream CDN, so that the Downstream CDN appears as multiple
        virtual Downstream CDN.  In that case, the information discussed
        in the previous requirement MUST be complemented with a Virtual
        CDN Identifier that is unique in the context of the pair of CDNs
        on both sides of the CDNI Control API.

I am unsure about this requirement as I am not sure exactly what it is asking for, is there a use case that would illustrate it by way of example?

Also the second sentence reads like a particular solution rather than a pure requirement.

   R25  The CDNI Control API SHOULD allow bootstrapping of the Content
        Acquisition API.  This information could, for example, include:
        *   negotiation of the Content Acquisition protocol to be used
            (e.g.  HTTP, HTTPS, FTP, ATIS C2), with some granularity
            (e.g.  On a per content type basis).

   R26  The CDNI Control API SHOULD allow the Upstream CDN to distribute
        the information necessary to bootstrap delivery authorization
        mechanisms to be performed by Downstream CDN.  This information
        could, for example, include:
        *   information necessary for surrogates to perform URI
            signature based validation

For R25 & R26 I think the role of the CDNI Control API is to exchange/negotiate what protocols & authorization mechanisms a CDN supports. Communication of the specific protocol/auth mechanism to use & associated details (credentials etc.) should be on a per object (or object set) basis via the CDNI Metadata API.

R27 - I’d suggest removing the three bullets marked with ‘+’ as they’re very solution specific and I think the requirement is clear enough from the preceding text.

   R33  The CDNI Request-Routing loop prevention mechanism SHOULD allow
        routing of the request (as opposed to the request loop being
        simply interrupted without routing the request).

It’s not clear to me what “SHOULD allow routing of the request” means, could you provide an example of how it may work?

   R34  The CDNI Request-Routing API SHOULD support optional enforcement
        of a limit on the number of successive CDN redirections for a
        given request.

“SHOULD support optional enforcement” - Does this mean the CDNI Request Routing API SHOULD support a mechanism but that honouring the limit is optional? I think just “SHOULD support enforcement” is maybe what you mean. Same comment also applies to R41.

   R38  The CDNI Request-Routing architecture MUST support a mechanism
        by which a Downstream CDN can notify the Upstream CDN when it is
        unable or unwilling to serve a request, so the Upstream CDN can
        react accordingly (e.g.  Select another Downstream CDN, or serve
        the request itself).

Is this the same as the “status code” in R37?

R42 - s/endusers/end users/

   R43  The CDNI Metadata API SHOULD support a pre-positioning model
        whereby the Upstream CDN requests the Downstream CDN to acquire
        content and associated distribution metadata (and possibly
        position those into Downstream CDN surrogates) at an appropriate
        time (now or a specified future time), before the content is
        actually requested by endusers.  [Editor's Note: how much
        influence the Upstream CDN ought to have on pre-positioning of
        the content on surrogates inside the Downstream CDN is TBD].
        [Editor's note: this functionality is currently described in
        draft-jenkins-cdni-problem-statement-01 under the Metadata API.
        Should it be moved under the Control API (since it is not
        strictly about exchange of metadata but more about triggering
        multiple actions in downstream CDN, one of them being possibly
        to get metadata) ?].

The “trigger” is probably better suited to the Control API but the actual exchange should be via the metadata API, otherwise we will end up having two different ways to distribute metadata and I think we want to avoid that situation.

R44/R45 - I am not sure we need R44 & R45. R42 & R43 are sufficient. If we have a method to dynamically acquire metadata (R42) and a method to trigger a downstream CDN to go collect some metadata (R43) I think we have R44 and R45 covered with the granularity of how much is communicated in advance controlled by the “trigger”.

   R46  Whether in the pre-positioning model or a dynamic acquisition
        model, the CDNI Metadata API MUST provide the necessary
        information to allow the Downstream CDN to acquire the content
        from the Upstream CDN (e.g.  Acquisition protocol and Uniform
        Resource Identifier in Upstream CDN- or rules to construct this
        URI).

I’d prefer this to say “upstream source” to allow for deploymwent flexibility. In many cases I can see the upstream source would be the upstream CDN but I would rather we didn’t restrict ourselves to that being the only option, e.g. if CDN1 is redirecting to CDN2 because CDN1 is out of capacity then requiring CDN2 to acquire content from CDN1 may not make sense.

   R47  The CDNI Metadata API MUST also provide the necessary
        information to allow the Downstream CDN to attempt to acquire
        the content from the content Origin Server, in case it can not
        be obtained from the Upstream CDN (e.g.  Because of a failure
        scenario).  Note that the content Origin Server may or may not
        be willing to serve the content to the Downstream CDN since the
        Content Provider may not have a direct agreement/relationship
        with the Downstream CDN for delivery of this content.

In one way this kind of addresses my previous comment but my preference would be to combine the two requirements into a single requirement as follows:

“Whether in the pre-positioning model or a dynamic acquisition model, the CDNI Metadata API MUST provide the necessary information to allow the Downstream CDN to acquire the content from one or more upstream sources. Note that some upstream sources (e.g. the content Origin Server) may or may not be willing to serve the content to the Downstream CDN, if this policy is known to the upstream CDN then it MAY omit those sources when exchanging CDNI metadata.

   R59  The CDNI Metadata API SHOULD allow signalling of the Virtual CDN
        identifier to be used in the Downstream CDN for distribution and
        delivery of the corresponding content (or content set).

I commented on R23 that a “Virtual CDN identifier” is too solution specific at this time, so I would also suggest removing this requirement.
R55 appears to be a duplicate of R55 so I suggest it is deleted.

Section 11 - You misspelt my surname :-) s/Nivens/Niven/

Thanks
Ben

---

From: Francois Le Faucheur <[email-masked]>
Date: Mon, 31 Jan 2011 13:42:45 +0100

Hi Ben,

> "The CDNI solution MUST NOT require intra-CDN information to be exposed to other CDNs for effective and efficient delivery of the content. Examples of intra-CDN information include surrogate topolo…

OK.

> s/other protocols than/protocols other than/

OK. I assume same goes for R6.

> Based on Haibin’s e-mail may be include an additional requirement that it should be possible to limit the number of levels.

This is covered in the "Request-Routing section" by R34:
"   
R34  The CDNI Request-Routing API SHOULD support optional enforcement
        of a limit on the number of successive CDN redirections for a
        given request.
"
or do you feel this is broader than Request-Routing and should be moved to the "Generic Requirements" section 

> Same comment applies to R10.
> 
> 

The requirement was intended to bring up "information to facilitate request redirection”. In fact, we should just mention exactly that at the beginning of the requirement.  To me, pretty much all of the examples listed above fit in that category.
I agree this should be move to the "Request-Routing API" section. This is in line with the recent thread on the list. I think this applies also to R14, R15, R20, R21 and R22.
 
> My thinking is that the CDNI Metadata API is really about distributing information about an identifiable object (e.g. metadata on a piece of content, geo-blocking policy, etc.) and for example we c…

Interesting. I see your point about a similar potential need to communicate "footprint" descriptions for the sake of conveying geo-blocking content distribution policies (as part of CDNI Metadata) and for the sake of conveying CDN footprint coverage (as part of CDNI Request Routing API).
But I think we can just make a note of that and make sure that if/when we get to interface specification, a single mechanisms is defined to convey such footprint description. Then CDNI Metadata interface can convey information that lists allowed/blocked footprints, while Request-Routing interface can convey information that associates capabilities/attributes/etc to footprints.

In other words, my suggestion is that we keep the discussion on requirements to convey geo-blocking under CDNI Metadata interface and requirements to convey CDN coverage/capabilities/attributes as part of the Request Routing interface, and just consider as a potential encoding optimization the fact those two may share some common information elements aka a "footprint").

Makes sense? 

>  * Remove an object(s) and/or its CDNI metadata from its surrogates and any storage."

agreed.

> I am not sure that negotiation of the Request Routing method is possible. The Request Router that responds to the User Agent/DNS resolver has to respond using the protocol it received the request o…

I'll come back on this one later.

> I am unsure about this requirement as I am not sure exactly what it is asking for, is there a use case that would illustrate it by way of example?

a Downstream CDN may offer multiple delivery services. For example, it could present a virtual CDN that covers one country only (which may be of interest to other CDNs that just need coverage of that country) or it could present a virtual CDN that covers one continent (including the previous country). Alternatively, it could present two virtual CDNs, one comprising a given set of resources, and another one with a larger set of resources.

> Also the second sentence reads like a particular solution rather than a pure requirement.

Agreed. We'll remove it.

> For R25 & R26 I think the role of the CDNI Control API is to exchange/negotiate what protocols & authorization mechanisms a CDN supports. Communication of the specific protocol/auth mechanism to us…

I agree this is a better split.

I propose to reword as:
"
Rx:	The CDNI Control API SHOULD allow bootstrapping of the Content Acquisition API. This could, for example, include exchange and negotiation of the Content Acquisition protocols to be used across the CDNs (e.g. HTTP, HTTPS, FTP, ATIS C2).

Ry:	The CDNI Control API SHOULD allow exchange and negotiation of delivery authorization mechanisms to be supported across the CDNs (e.g. URI signature based validation).
"

> R27 - I’d suggest removing the three bullets marked with ‘+’ as they’re very solution specific and I think the requirement is clear enough from the preceding text.

This was only mentioned as a (fairly obvious) example to make it clear we don't have to invent everything. But I am fine with removing it since it seems clear to all.

> It’s not clear to me what “SHOULD allow routing of the request” means, could you provide an example of how it may work?

Say, there is a redirection loop (CDN1-->CDN2-->CDN3--->CDN1), the solution could have a pure loop detection mechanism which will result in a given request/query being dropped when it starts looping. The result would be that the request is never honored and the user never gets the content. The objective of the requirement is to say that we want more than that: we want to make sure that the loop is detected and the loop is effectively prevented so that the user will get the content somehow.
Just for example, CDN3 could realize that it is about to redirect a request to CDN1, while the request had been redirected earlier via CDN1, so CDN3 may change its "normal" decision and instead serve the request itself or redirect to another CDN that is not in the "loop".

> “SHOULD support optional enforcement” - Does this mean the CDNI Request Routing API SHOULD support a mechanism but that honouring the limit is optional?

No. It is intended to mean that CDNI Request Routing API should support a mechanism but invoking the mechanism is optional.

I propose we reword as:
"The CDNI Request-Routing API SHOULD support an optional mechanism enforcing a limit on the number of successive CDN redirections for a given request."

> I think just “SHOULD support enforcement” is maybe what you mean. Same comment also applies to R41.
> 
> 

Yes. I think R37 is more specific than R38, so I propose we remove R38.

> R42 - s/endusers/end users/

ack.

> The “trigger” is probably better suited to the Control API but the actual exchange should be via the metadata API, otherwise we will end up having two different ways to distribute metadata and I th…

Agreed. I propose to:

*1) add a requirement in the Control API section:
"
Rx:	The CDNI Control API SHOULD support triggering and control for a pre-positioning model whereby the Upstream CDN requests the Downstream CDN to acquire content and/or associated distribution metadata (and possibly position those into Downstream CDN surrogates) at an appropriate time (now or a specified future time), before the content is actually requested by endusers. Note that the actual exchange of metadata and the actual content acquisition are outside the scope of the Control API: those are supported, respectively, via the CDNI Metadata API and the Content Acquisition API. [Editor's Note: how much influence the Upstream CDN ought to have on pre-positioning of the content on surrogates inside the Downstream CDN is TBD].
"

*2)  reword R42 and R43 to:
"
Rx:	The CDNI Metadata API MUST allow the Upstream CDN to provide the Downstream CDN with content distribution metadata of inter-CDN scope.

Ry:	The CDNI Metadata API MUST support exchange of CDNI metadata for both the dynamic acqusition model (whereby the Downstream CDN surrogates acquire content when it is actually requested by end users) and the pre-positioning model (whereby the Upstream CDN requests the Downstream CDN to acquire content and/or associated distribution metadata before the content is actually requested by endusers). [Editor's Note: how much influence the Upstream CDN ought to have on pre-positioning of the content on surrogates inside the Downstream CDN is TBD].
" 

> R44/R45 - I am not sure we need R44 & R45. R42 & R43 are sufficient. If we have a method to dynamically acquire metadata (R42) and a method to trigger a downstream CDN to go collect some metadata (…

I don't think R44 and R45 really follow from R42 and R43.
R42 and R43 just say that there needs to be a way to pass metadata and that this should work both for dynamic acquisition and for prepositioning scenarios. It does not imply much on whether this is achieved by pushing all metadata, pushing some allowing to pull the rest, issuing a trigger that allows the pull.

> I’d prefer this to say “upstream source” to allow for deploymwent flexibility. In many cases I can see the upstream source would be the upstream CDN but I would rather we didn’t restrict ourselves…

absolutely. See more below.
> In one way this kind of addresses my previous comment

Right.

> but my preference would be to combine the two requirements into a single requirement as follows:
> 

I propose to expand a bit and spread over two requirements:

"
Rx:	Whether in the pre-positioning model or a dynamic acquisition model, the CDNI Metadata API MUST provide the necessary information to allow the Downstream CDN to acquire the content from an upstream source (e.g. Acquisition protocol and Uniform Resource Identifier in Upstream CDN- or rules to construct this URI). 

Ry:The CDNI metadata MUST allow signaling of one or more upstream sources, where each upstream source can be in the Upstream CDN, in another CDN, the CSP origin server or any arbitrary source designated by the Upstream CDN. Note that some upstream sources (e.g. the content origin server) may or may not be willing to serve the content to the Downstream CDN, if this policy is known to the upstream CDN then it may omit those sources when exchanging CDNI metadata.
"

(it says "may omit" and mot "MAY omit" because the document defines requirement on the protocol itself, not so much how an implementation must/should/may use the protocol knobs).

> I commented on R23 that a “Virtual CDN identifier” is too solution specific at this time, so I would also suggest removing this requirement.

This requirement was intended to indicate that Virtual CDN would also impact the metadata API. But rather than have this show up in every API, I propose we have a single requirement on Virtual CDN in the Generic Requirement (as a SHOUDL for "Beyond Initial scope").

> R55 appears to be a duplicate of R55 so I suggest it is deleted.

I think you mean R58.
The trick is that cascaded CDNs may or may not be kept within Initial scope.
So R55 says: if cascaded CDNs are in initial scope then, the requirement MUST be met in initial scope.
R58 says that : no matter what, the requirement is to be met in subsequent scope (ie beyond initial scope).

> Section 11 - You misspelt my surname :-) s/Nivens/Niven/

Sorry about that. Fixed now.

Francois