Subject: Re: [CDNi] FW: New Version Notification for draft-liu-cdni-metadata-interface-01.txt

From: Kevin J Ma <[email-masked]>
Date: Thu, 29 Mar 2012 12:16:03 -0400

Hi Xiaoyan,

  I had a questions about condition precedence.  The draft states that the
  "last condition metadata group with matching conditions takes effect".
  I assume that applies to each individual metadata, i.e., that different
  metadata may come from different condition groups, and be applied to a
  given content request?

  The "last"-based approach gives precedence to newer rules?  To properly
  order the list, a uCDN may be required to re-evaluate and replacing all
  conditions whenever a new condition is created, which may not scale?
  Is there a precedence order for the different condition fields?  And is
  there a best match criteria for the condition value types?  That might
  be an alternative?

  Applying different sets of metadata to a given content item is an
  interesting idea, however, imo, applying sets of metadata to
  sets of content is easier to comprehend (especially with large
  amounts of content and overlapping metadata), i.e., keep URI separate
  and always match URI first before applying other condition metadata;
  condition metadata would only apply to specific URIs?

thanx.

--  Kevin J. Ma

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> HeXiaoyan
> Sent: Tuesday, December 20, 2011 3:56 AM
> To: [email-masked]
> Subject: [CDNi] FW: New Version Notification for draft-liu-cdni-metadata-
> interface-01.txt
> 
> Hi all,
> I've uploaded an updated version of draft-liu-cdni-metadata-interface with
> some typos corrected.
> http://www.ietf.org/id/draft-liu-cdni-metadata-interface-01.txt
> 
> Your comments are welcome.
> 
> Wish you all a happy holiday season!
> 
> Best Regards
> Xiaoyan(Susan) He
> 
> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, December 20, 2011 3:22 PM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked];
> [email-masked]
> Subject: New Version Notification for draft-liu-cdni-metadata-interface-
> 01.txt
> 
> A new version of I-D, draft-liu-cdni-metadata-interface-01.txt has been
> successfully submitted by Xiaoyan He and posted to the IETF repository.
> 
> Filename:	 draft-liu-cdni-metadata-interface
> Revision:	 01
> Title:		 Content Distribution Network Interconnection (CDNI)
> Metadata Interface
> Creation date:	 2011-12-20
> WG ID:		 Individual Submission
> Number of pages: 22
> 
> Abstract:
>   The Metadata Interface is one of the core interfaces defined by the
>   IETF Content Distribution Network Interconnection (CDNI) Working
>   Group. The primary function of the CDNI Metadata Interface is to
>   allow interconnected CDNs to communicate metadata in order to ensure
>   content delivery and content acquisition. This document defines the
>   metadata exchange protocol between an upstream CDN and a downstream
>   CDN for the content delivery. It also defines metadata objects
>   especially condition metadata object for CDNI.
> 
> 
> 
> 
> The IETF Secretariat
> 
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni

---

From: HeXiaoyan <[email-masked]>
Date: Thu, 29 Mar 2012 19:08:34 +0000

Hi Kevin,
Thanks for the comments.
Please see my response inline.  

________________________________________
发件人: Kevin J Ma [[email-masked]]
发送时间: 2012年3月30日 0:16
到: HeXiaoyan; [email-masked]
主题: RE: [CDNi] FW: New Version Notification for draft-liu-cdni-metadata-interface-01.txt

Hi Xiaoyan,

  I had a questions about condition precedence.  The draft states that the
  "last condition metadata group with matching conditions takes effect".
  I assume that applies to each individual metadata, i.e., that different
  metadata may come from different condition groups, and be applied to a
  given content request?
[Xiaoyan]  Don't quite understand your concer here, but let me try to give some explaination here.
If you look at the data model, the top level object is "Delivery Domian",
which means that the basic granularity of metadata application scope is a delivery domain,
i.e. all contents belonging to that domain, not a specific content. Actually, we belive metadata of
contents belonging to a same domain will overlap in much, we describe these metadata
via DomainMetadata Group in our document, which means you do not need to extract most of 
the metadata from different condition groups. They should be there only in one group, named DomainMetadataGroup. 
The condition metadata only applied for contents in the domain which has special characteristics and some of
its metadata needs to be reset beyond default values contained in DomainMetadataGroup.
 And the intention of multiple CONDITION GROUP is to reset  metadata requiring different conditions match against performance, it is not explicitly exclude
the case to reset same metadata, but "last condition metadata group takes effect that is a extreme case for reseting same metadata via multiple condition group I think.

  The "last"-based approach gives precedence to newer rules?
[Xiaoyan] Right, but only under the case that the latter condition group are trying to reset a metadata that already reseted by a previous condition group. If the 
latter condition group is used to reset different metadata than the previous ones, then "precedence" does not make sense.

  To properly order the list, a uCDN may be required to re-evaluate and replacing all 
  conditions whenever a new condition is created, which may not scale?
[Xiaoyan] This is not true. A uCDN does not need to do that at all.   
Let's take an example , if there are already two condition group, Condition Group1 reset metadata 1 once condition A and B are met. Conditin Group 2 reset
metadata 2 once condition C, D, E are met.  Now you want to create a new condition F for metadata 1, if you do not require condition A and B 
are met  simultaneously, just create a new Condition Group 3 and put it later than Condition Group1, that's enough.  If you require condition A , B and the new 
created condition F should be met simutaneously to reset metadata1, then just add condition F into condition group 1.   uCDN does not need re-evaluate and
replacing all conditions.

  Is there a precedence order for the different condition fields?  And is
  there a best match criteria for the condition value types?  That might
  be an alternative?
[Xiaoyan] There is no precedence order for the different condition fields. Multiple conditions
with different condition fields in ONE CONDITION GROUP are used in AND relation.  

  Applying different sets of metadata to a given content item is an
  interesting idea, however, imo, applying sets of metadata to
  sets of content is easier to comprehend (especially with large
  amounts of content and overlapping metadata), 

[Xiaoyan] The main idea of this draft is applying same domain  metadata to
all contents belonging to a delivery domain,  besides that we give an mechanism that refine some kind of metadata of some special contents 
in the same domain. I think we are doing the same thing, applying sets of metadata to sets of content. 
And the merit of our proposal is  we can save the volume of metadata very largely and provide good extensibility and flexibility  .

i.e., keep URI separate
  and always match URI first before applying other condition metadata;
  condition metadata would only apply to specific URIs?
[Xiaoyan] We do match URI first, in our proposal, once a dCDN receives a content request,
hostname of the URI must first be checked to find out the metadata needs to be applied.
Hope this helps.

thanx.

--  Kevin J. Ma

---

From: Kevin J Ma <[email-masked]>
Date: Fri, 30 Mar 2012 13:36:14 -0400

Hi Xiaoyan,

  some additional comments:

> all contents belonging to a delivery domain,  besides that we give an
> mechanism that refine some kind of metadata of some special contents
> in the same domain.

  I understand that you want to reduce the amount of metadata, by relying
  on default metadata, however, I do not think it is realistic to assume
  that a single metadata value will apply to all content within a domain.
  We handle thousands of pieces of content for our customers, and most of
  them have different availability windows.  Each would need a condition.

> hostname of the URI must first be checked to find out the metadata needs

  Matching just the hostname is not the same as matching the full request URI.
  I do not think applying metadata to just the FDQN is sufficient in all cases.
  Applying default metadata to a domain is just a degenerate case of full URI
  matching where: URI = /*.

> [Xiaoyan] This is not true. A uCDN does not need to do that at all.

  The definition of the condition object is very flexible.  The high
  degree of flexibility allows for much more complex combinations of
  conditions than the ones you describe.  When you get to thousands of
  condition objects with regex matches theres a high chance for error.

> [Xiaoyan]  Don't quite understand your concer here,

  My question was about the order of processing in the condition list.
  I think we need to be very precise on ordering, so that all dCDNs can
  identically and deterministically apply the metadata.

  Given metadata objects: m1, m2, m3
  Assume the domainlist sets: m1 = A, m2 = B, m3 = C
  Assume a condition c1 which sets: m1 = W and m2 = X if the URI matches */foo/*
  Assume a condition c2 which sets: m1 = Y if the URI matches */bar/*  
  Assume a condition c3 which sets: m2 = Z if the userAgent matches *iPhone*

  Given a requst for the URI: /foo/bar/baz.txt

  The order of the conditions clearly matters:

  If the condition creation order was: c1, c2, c3
    an iPhone would get: m1 = Y , m2 = Z, m3 = C
    an iPad would get: m1 = Y, m2 = X, m3 = C

  If the condition creation order was: c3, c2, c1
    an iPhone would get: m1 = W , m2 = X, m3 = C
    an iPad would get: m1 = W, m2 = X, m3 = C

  If one of the conditions (c3) was new, and the existing conditions (c1, c2)
  were already embedded in a list of thousands of other conditions, properly
  ordering them may not be as simple as just adding to the end of the list,
  especially if the regex expressions overlap with other existing conditions?

thanx.

--  Kevin J. Ma

> From: HeXiaoyan [mailto:[email-masked]]
> Sent: Thursday, March 29, 2012 3:09 PM
> To: Kevin J Ma; [email-masked]
> Subject: Re: [CDNi] FW: New Version Notification for draft-liu-cdni-
> metadata-interface-01.txt
> 
> Hi Kevin,
> Thanks for the comments.
>  And the intention of multiple CONDITION GROUP is to reset  metadata
> requiring different conditions match against performance, it is not
> explicitly exclude
> the case to reset same metadata, but "last condition metadata group takes
> effect that is a extreme case for reseting same metadata via multiple
> condition group I think.
> 
> [Xiaoyan] Right, but only under the case that the latter condition group
> are trying to reset a metadata that already reseted by a previous
> condition group. If the
> latter condition group is used to reset different metadata than the
> previous ones, then "precedence" does not make sense.
> 
> 
> Let's take an example , if there are already two condition group,
> Condition Group1 reset metadata 1 once condition A and B are met. Conditin
> Group 2 reset
> metadata 2 once condition C, D, E are met.  Now you want to create a new
> condition F for metadata 1, if you do not require condition A and B
> are met  simultaneously, just create a new Condition Group 3 and put it
> later than Condition Group1, that's enough.  If you require condition A ,
> B and the new
> created condition F should be met simutaneously to reset metadata1, then
> just add condition F into condition group 1.   uCDN does not need re-
> evaluate and
> all contents belonging to a delivery domain,  besides that we give an
> mechanism that refine some kind of metadata of some special contents
> And the merit of our proposal is  we can save the volume of metadata very
> largely and provide good extensibility and flexibility  .
> 
> i.e., keep URI separate