Subject: Re: [CDNi] Control and Metadata interfaces

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 1 Nov 2011 15:10:52 +0000

Emile, Colleagues,

On 31 Oct 2011, at 22:36, <[email-masked]> <[email-masked]> wrote:
> Hence I propose the following:
> -          The control interface is split in 2 parts:
> o   bootstrapping of the CDN interconnexion;
> o   control of the contents distribution;
> -          The control of the contents distribution and the Metadata interface share the same framework (datamodel parts, protocols…);
> -          The framework document specifies precisely the bootstrapping operations and the framework;

I agree that as we work through the CDNI solution components and how they fit together the exact functionality placed under each interface needs to be firmed up compared to what we have currently in the Problem Statement and Bruce's framework draft.

I shared some thoughts with some folks offline about the CDNI Control interface and the two functions of bootstrapping & control of content/metadata and where they might best fit in the context of CDNI interfaces but as you bring it up here I'll repeat some of my thoughts on the list.

Bootstrapping/(fairly) static capability advertisement:

I tend to think that the CDNI Metadata is essentially (content service level) configuration information and the CDNI Metadata interface is about distributing configuration between interconnected CDNs.

In the case of bootstrapping of the CDNI interfaces and communicating static (or fairly static) information about capabilities and policies they can also (IMO) be thought of as essentially CDN/CDNI level configuration information.

Although the current proposals for CDNI Metadata data models & interface specifications only address CDNI Metadata at present I think it should be possible to build a data model for CDNI bootstrapping/capabilities that can reuse, for example, the interface specification and encoding in proposed draft-jenkins-cdni-metadata. New objects (and associated relationships & media types) would need to be specified but that should be fairly straightforward and the base interface proposed could be reused for both content service configuration (aka CDNI Metadata) and CDN/CDNI configuration (aka bootstrapping/capabilities).

So it would seem to me a reasonable proposal to move the bootstrapping/capability elements currently nominally placed under "CDNI Control" to become part of the CDNI Metadata interface

Control of content:

In the case of control of content although draft-davie-cdni-framework-01 puts forward the view that "a purge is equivalent to a statement that the availability window for that content has now expired" I think the requirements placed on purging/invalidating/pre-positioning content require functionality not covered by the current CDNI Metadata proposals.

In contrast to the distribution of CDNI Metadata which is a dCDN discovering (or being told) the content service level configuration it needs, invalidating/purging content is the uCDN requesting the dCDN perform some task on its behalf (e.g. purge this item of actual content).

A key factor that IMO separates purge/pre-position from CDNI Metadata is the timing & feedback aspects. When a dCDN pulls CDNI Metadata from a uCDN (or when a uCDN pushes CDNI Metadata into a dCDN) it's reasonable for the uCDN to assume the dCDN has some mechanism to distribute that so it is available to Surrogates when they require it. In the case of push (via a HTTP POST for e.g.) the dCDN's response of "201 Created" is saying "I got that metadata record and created a resource for it on the server you POSTed to" it says nothing about whether or not the Metadata was distributed to the rest of the system (same for pull via GET except the acceptance by the dCDN is implicit).

In the case of purge/pre-position the uCDN will want explicit feedback to know that the purge/pre-position "task" it has requested has been distributed within the dCDN and completed by the dCDN. The dCDN's "201 Created" response would be saying "I got that request for purge and I have created a resource [task] for it on the server you POSTed to". If we're being RESTful the Location: header in the 201 response would link to a resource that could be queried by the uCDN to check the progress of the task. The nature of purge/pre-position requests means they may take some time to complete and so trying to overload the meaning of the 201 with "I got that task and I've completed it" doesn't seem realistic to me.

Therefore purge/invalidate/pre-position of actual content (as opposed to CDNI Metadata) is in my mind a separate beast to distributing the actual CDNI Metadata and requires a different behaviour from the interface used to implement purge/invalidate/pre-position from that used to distribute CDNI Metadata.

Whether we incorporate that different behaviour under the banner of a "CDNI Control interface" or an extended/richer "CDNI Metadata interface" is to some extent semantics but I do think it needs something more than is required purely for the distribution of CDNI Metadata between interconnected CDNs.

Ben

---

From: <[email-masked]>
Date: Wed, 2 Nov 2011 16:00:37 +0100

Hi Ben and Bruce,

See inline

> -----Message d'origine-----
> De : Ben Niven-Jenkins [mailto:[email-masked]]
> Envoyé : mardi 1 novembre 2011 16:11
> À : STEPHAN Emile RD-CORE-LAN
> Cc : [email-masked]; [email-masked];
> [email-masked]
> Objet : Re: [CDNi] Control and Metadata interfaces
> 
> Emile, Colleagues,
> 
> > -          The control of the contents distribution and the Metadata
> interface share the same framework (datamodel parts, protocols...);
> > -          The framework document specifies precisely the
> bootstrapping operations and the framework;
> 
> I agree that as we work through the CDNI solution components and how
> they fit together the exact functionality placed under each interface
> needs to be firmed up compared to what we have currently in the Problem
> Statement and Bruce's framework draft.

IMO, this is preventing to go in the detail of the works. Hence IMO we must asap specify the bootstrapping and the information elements that must be exchanged during this step.

> 
> I shared some thoughts with some folks offline about the CDNI Control
> interface and the two functions of bootstrapping & control of
> content/metadata and where they might best fit in the context of CDNI
> interfaces but as you bring it up here I'll repeat some of my thoughts
> on the list.
> 

I share your approach but in draft-stephan-cdni-usecase-metadata we clearly indentify that an object can be really static in a situation and extremely dynamic in another. This is why I promote that the control interface and the metadata interface share the same datamodeling framework and a subset of objects too.

> 
> I tend to think that the CDNI Metadata is essentially (content service
> level) configuration information and the CDNI Metadata interface is
> about distributing configuration between interconnected CDNs.
> 
> In the case of bootstrapping of the CDNI interfaces and communicating
> static (or fairly static) information about capabilities and policies
> they can also (IMO) be thought of as essentially CDN/CDNI level
> configuration information.
> 
> Although the current proposals for CDNI Metadata data models &
> interface specifications only address CDNI Metadata at present I think
> it should be possible to build a data model for CDNI
> bootstrapping/capabilities that can reuse, for example, the interface
> specification and encoding in proposed draft-jenkins-cdni-metadata. New
> objects (and associated relationships & media types) would need to be
> specified but that should be fairly straightforward and the base
> interface proposed could be reused for both content service
> configuration (aka CDNI Metadata) and CDN/CDNI configuration (aka
> bootstrapping/capabilities).
> 
> So it would seem to me a reasonable proposal to move the
> bootstrapping/capability elements currently nominally placed under
> "CDNI Control" to become part of the CDNI Metadata interface
> 
> 
> Control of content:
> 
> In the case of control of content although draft-davie-cdni-framework-
> 01 puts forward the view that "a purge is equivalent to a statement
> that the availability window for that content has now expired" I think
> the requirements placed on purging/invalidating/pre-positioning content
> require functionality not covered by the current CDNI Metadata
> proposals.
 
I agree. CSP require explicit purge. This is an operation of control. 

> 
> In contrast to the distribution of CDNI Metadata which is a dCDN
> discovering (or being told) the content service level configuration it
> needs, invalidating/purging content is the uCDN requesting the dCDN
> perform some task on its behalf (e.g. purge this item of actual
> content).
> 
> A key factor that IMO separates purge/pre-position from CDNI Metadata
> is the timing & feedback aspects. When a dCDN pulls CDNI Metadata from
> a uCDN (or when a uCDN pushes CDNI Metadata into a dCDN) it's
> reasonable for the uCDN to assume the dCDN has some mechanism to
> distribute that so it is available to Surrogates when they require it.
> In the case of push (via a HTTP POST for e.g.) the dCDN's response of
> "201 Created" is saying "I got that metadata record and created a
> resource for it on the server you POSTed to" it says nothing about
> whether or not the Metadata was distributed to the rest of the system
> (same for pull via GET except the acceptance by the dCDN is implicit).
> 
> In the case of purge/pre-position the uCDN will want explicit feedback
> to know that the purge/pre-position "task" it has requested has been
> distributed within the dCDN and completed by the dCDN. The dCDN's "201
> Created" response would be saying "I got that request for purge and I
> have created a resource [task] for it on the server you POSTed to". If
> we're being RESTful the Location: header in the 201 response would link
> to a resource that could be queried by the uCDN to check the progress
> of the task. The nature of purge/pre-position requests means they may
> take some time to complete and so trying to overload the meaning of the
> 201 with "I got that task and I've completed it" doesn't seem realistic
> to me.
> 
> Therefore purge/invalidate/pre-position of actual content (as opposed
> to CDNI Metadata) is in my mind a separate beast to distributing the
> actual CDNI Metadata and requires a different behaviour from the
> interface used to implement purge/invalidate/pre-position from that
> used to distribute CDNI Metadata.
> 
> Whether we incorporate that different behaviour under the banner of a
> "CDNI Control interface" or an extended/richer "CDNI Metadata
> interface" is to some extent semantics but I do think it needs
> something more than is required purely for the distribution of CDNI
> Metadata between interconnected CDNs.
> 
> Ben
> 

Here is my proposal refined:
  - The framework document splits the control interface in 2 parts:
    o   bootstrapping of the CDN interconnection;
    o   control of the contents distribution;
  - The control of the contents distribution and the Metadata interface share the same datamodeling framework (datamodel parts, protocols...);
  - The framework document specifies precisely the bootstrapping (operations and information elements)
  - The framework document specifies the datamodeling framework;

Regards
Emile

---

From: Alan Kavanagh <[email-masked]>
Date: Thu, 3 Nov 2011 15:28:00 -0400

Hi Francois

Thanks for putting some light on this, most appreciated.

I fully agree that Content Adaptation is definitely an optimization in terms of delivery from the CSP/origin Server to the CDN Provider itself, send 1 copy and not 10 etc saves on bandwidth which implies you need smaller pipes and don't have to worry about the transcoding etc, makes perfect sense. However i do believe its important for CDN_I to include this and ill explain why. A lot of CSP's don't do content adaptation as they either don't have the expertise or don't want to manage that side of the content they produce, they only want to have it distributed to a wide variety of devices etc and they give that job of Content Adaptation to the CDN Provider to take care of as part of their business agreement between the Content Provider and Distributor and this model is growing quite a lot as im sure all CDN vendors alike here would agree with and don't see that diminishing.

Similarly as new codec's and end devices with different screen density's come to the market i believe the CDN is better equipped to handle this than the CSP. The CDN does the Content Adaptation for lots of CSP's it has service agreements with so they typically do this as opposed to have lots of CSP's do this themselves, its not just transport savings but also CAPEX/OPEX and expertise the CSP may not have. The savings here is for the CSP not the end user as i note you have below and agree, but the cost burden here is the CSP in the first place and same also between CDSP's.....transport/bandwidth is never free and when optimization which translates into cost savings can be achieve its imperative it is used.

I think Richard had a good proposal which was to bring some presentations to Taipei or next meeting as time is short to the group and take it from that on how to proceed. Ill try and work with others to provide input on some use cases describing this as i feel it will help the discussion and help it conclude i hope :-)

Alan

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Francois Le Faucheur
Sent: October-28-11 5:49 AM
To: [email-masked]
Subject: [CDNi] Content Adaptation

Folks,

Speaking as a co-chair.

I have been through the recent exchange on content adaptation (aka the poorly-titled "Re: [CDNi] I-D Action: draft-ietf-cdni-use-cases-00.txt" thread) and wanted to share a few thoughts.

First, some messages might have given the impression that content adaptation was required for a CDNI solution to be able to deal with multiple types of terminal/resolution/sizes etc. So at the risk of restating the obvious, let's be very explicit that this is obviously not the case. A very simple/common way to support multiple types of terminal/resolution/sizes in one CDN, or in a multi-CDN environment, is to have different contents/URIs for each profile.
So content adaptation is not _required_ to be supported by CDNI, but is a candidate optimisation to do some of the same things in a different way.

Second, some messages suggested that content adaptation was a very strong win (aka transfer 1 flow instead of 10 flows across CDNs). As Richard pointed out already, this ignores a number of considerations such as the cost associated with performing the actual content adaptation downstream. Anyways, I think content adaptation may be better seen as a trade-off that can indeed be positive in some environments, rather than an obvious huge win.
Also, without the content adaptation optimization, while the content maybe passed across CDNs multiple times (in different formats), it is worth noting that the smaller res/size/quality representations may actually amount to much smaller size/bandwidth than the highest res/size/quality, so it is not quite a multiplying factor.
Lastly, while the content adaptation optimization may reduce further the inter-CDN transfer from 1-10 flows down to  1 flow, the base CDNI solution allows a content to be transferred across CDNS 1-10 times total instead of once per user accessing the content (as would be the case if the uCDN pumps the same traffic to every users because it cannot interoperate with the dCDN) which can result in a gain factor of 10s, 100s, 1000s,.....

I have the impression that the key arguments have now been exposed (some of them even repeatedly) through the long thread. So I would suggest to pause that discussion, unless _new_ arguments need to be brought into the discussion, and then we can poll for consensus in Taipei and then on the list.

Thanks

Francois

PS: Please think of using specific Subject lines for the various threads and don't hesitate to branch off with a different "Subject" when an initial thread morphs into a different topic. This helps the discussion.

---

From: Francois Le Faucheur <[email-masked]>
Date: Thu, 3 Nov 2011 21:01:09 +0100

Hi Alan,

> I fully agree that Content Adaptation is definitely an optimization in terms of delivery from the CSP/origin Server to the CDN Provider itself, send 1 copy and not 10 etc saves on bandwidth which i…

Right, but having the CSP delegate content adaptation to "the CDNs" does not translate in a requirement for CDNI to support content adaptation. Specifically, the content adaptation can be performed by the authoritative CDN (aka the upmost CDN). CDNI support of content adaptation is only needed if the dCDN needs to do further content adaptation beyond what the uCDN has already done.
In fact, Richard has already made that point:
> Speaking as an individual, I can find a lot of examples where downstream content adaptation is not needed, and any adaptation is really part of the CP - uCDN interface (which is out of the scope of our charter).

The point you make above seems like a good argument for why it would be good to support content adaptation in a CSP-uCDN interface but not an argument for why it is needed in CDNI interfaces.

Francois

> I think Richard had a good proposal which was to bring some presentations to Taipei or next meeting
> as time is short to the group and take it from that on how to proceed. Ill try and work with others to provide input on some use cases describing this as i feel it will help the discussion and help it conclude i hope :-)
>  
> Alan
>

---

From: HeXiaoyan <[email-masked]>
Date: Fri, 04 Nov 2011 17:09:43 +0800

Hi Stephan and Ben,

Following your discussion on capability advertisement, here is my view,

> > "CDNI Control" to become part of the CDNI Metadata interface

I did not find any explicit capability exchange requirement under the CDNI
Control interface except REQ-2 under the Request-Routing one,
so are you proposing to move REQ-2 to CDNI Control/Metadata interface or
anything else?

In my understanding, bootstrapping of CDNI interfaces means getting some
info necessary for CDNI startup and the info used for bootstrapping are
usually static, as some of the capability info of a dCDN change occasionally
and need to be updated to the upCDN from time to time , e.g. UsedDeliveryBW
defined in section 4.2.1 of
draft-xiaoyan-cdni-request-routing-protocol-00.txt, hence I don't think
capability advertisement should be part of bootstrapping and it is not
appropriate to put it under the Control interface.

Putting capability advertisement function under the MI scope is not so
appropriate either, the current definition of CDNI Metadata if pretty clear,
as Ben said, it is kind of content service configuration data, the MI is
used to exchange such data, I don't understand why we need to mud up the
concept of CDNI Metadata with a totally different type of data, capability
of a CDN. In practice CDN today, the service configuration interface and the
capability report interface are two different, keep consistent with that
would be good.  

>From above, keep the capability exchange function in Request Routing
interface to facilitate request redirection as present or put it under
another new introduced independent interface is preferred.    
Thanks.

Best Regards
Xiaoyan(Susan) He

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Wednesday, November 02, 2011 11:01 PM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]
> Subject: Re: [CDNi] Control and Metadata interfaces
> 
> Hi Ben and Bruce,
> 
> See inline
> 
> IMO, this is preventing to go in the detail of the works. Hence IMO we
must
> asap specify the bootstrapping and the information elements that must be
> exchanged during this step.
> 
> >
> I share your approach but in draft-stephan-cdni-usecase-metadata we
clearly
> indentify that an object can be really static in a situation and extremely
> dynamic in another. This is why I promote that the control interface and
the
> metadata interface share the same datamodeling framework and a subset of
> objects too.
> 
> 
> >
>   - The control of the contents distribution and the Metadata interface
share
> the same datamodeling framework (datamodel parts, protocols...);
>   - The framework document specifies precisely the bootstrapping
(operations
> and information elements)
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni