Subject: Re: [CDNi] 答复: Re: Fwd: draft-jin-cdni-content-deduplication-optimization-02.txt

From: Kevin J Ma <[email-masked]>
Date: Thu, 2 Aug 2012 01:04:23 -0400

Hi Mian and Bhumip,

  When you say CSP-unique, does that include encoding-unique?
  If a CDN or other intermediary modifies the content in some way, how
  is that reflected in the content ID, and how would that be enforced?

  There are organizations which are looking at content identification
  (http://eidr.org/ comes to mind).  Does CDNI need to develop its own
  "authoritative 'Entity'" for managing content IDs, or do you feel that
 there are existing services/protocols that would qualify as candidate
  'Entities'?

thanx.

--  Kevin J. Ma

From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Wednesday, August 01, 2012 2:52 AM
To: [email-masked]; [email-masked]; [email-masked]; 'Bhumip Khasnabish'; [email-masked]
Subject: [CDNi] 答复: Re: Fwd: draft-jin-cdni-content-deduplication-optimization-02.txt

Dear Theodore,

Thank you for your comments and sharing with us your proposals. Could you please share the documents you mentioned below with me? Based on what you explained in previous email, I have some questions for clarification:
1. We both consider it necessary to have a unique content identifier for a content object. Our opinion in this draft is that this content ID is CSP-unique. Could you please explain more about the uniqueness of the UID in your proposal?
2. By applying the mechanism you proposed below, we can guarantee the content is uniquely stored in one CDN. And according to my analysis on the basis of your explanation, the content replication is checked by comparing the CURL which is similar to the current mechanism in CDNi work, correct me if my understanding is wrong. If so, when redirection occurs between uCDNs and dCDN, the URL would be changed and thus be different with the original one - CURL. And if a dCDN is connected with two different uCDNs and two end users request the same content from these two uCDNs respectively, the two uCDNs may generate different redirection URLs to the dCDN. In this case we think the content duplication issue still exists. How do you think? If my understanding is wrong, could you please explain more on how to consider this issue by using your proposal?

Thank you again~

[email-masked]<mailto:[email-masked]> 写于 2012-08-01 02:00:23:

> Dear Bhumip,
>
> Some ideas and considerations on the draft.
>
> We have done some work in the area of de-duplication of content
> (Please see Th. Zahariadis, E. Quacchio, “Fast content-aware
> delivery in overlay networks,” IEEE COMSOC MMTC E-Letter, Vol.6, No.
> 7, July 2011, pp. 44-47))
>
> Actually, we believe that it is needed to define Unique IDs (UID)
> per content object/chunk in order to:
> a) Avoid extended data replications at the CDN level and minimize
> the load to find the correct object.
> b) Detect and retrieve very fast the content. This should be fast
> enough to allow even seamless real-time video streaming by
> retrieving video chunks
> c) Be backwards compatible with todays’ URLs (in case the
> content/chunk is not found, you should be able to go to the original site).
>
> In order to meet the (a), UID should be always associated with the
> content object itself (e.g. encapsulated in the object) or be based
> on unique characteristics of the content object (e.g. a set of low
> level descriptors). However, (b), poses that this could be
> calculated once (or sometimes), but should not be calculated or
> generated each time the object is requested; instead it should be
> “carried” and “extracted” in most cases.  On the other hand, due to
> backwards compatibility needs (c), we should not change the standard
> file format (e.g. we could not encapsulate UID or low level
> descriptor in the content object).
>
> One solution could be to create a wrapper that would encapsulate the
> UID whenever the content object enters the CDN and extracts that at
> the time that the content object leaves the CDN, but this would
> increase the complexity and processing time.
>
> Instead, we propose to use as UID a formal file name format. UID
> will be a string concatenation in the format:
>
> CM-CID-filename.ext
>
> where:
> • CM is a “Content Marker”
> • CID is a content signature, which could be a self-certifying
> identifier e.g. MD5 or SHA-1 based-hash function on the file's
> content or even a combination of searchable low level descriptors. ,
> which guarantees an easy and fast detection that this name is a UID
> • the original filename, which is used for making UID easily
> recognized by humans and avoid complex self-certifying names.
>
> Whenever new content is published or stored at the CDN, the object
> filename could be renamed to a UID and the URL to a Content URL
> (CURL) with the
> following format:
>
> http://www. website.com/…/CM-CID-filename.ext
>
> The CURL is a URI, which is designed to enable caching mechanisms
> and trigger CDN-related functionality (e.g. accessing the CDN
> overlay network and querying for locally or “nearby” cached
> content). It should also be emphasized that from the CURL, the
> original URL and the UID may be easily extracted, while on the other
> hand it is fully backwards compatible with existing browsers and no
> modifications are needed. In this way, we may seamlessly support any
> kind of data available in the Internet (text, images, various types
> of audio or video), while in case the content object is not cached
> in the CDN, we can always go back to the original source URL.
>
> Best regards,
> Theodore
>
> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> On Behalf Of
> Bhumip Khasnabish
> Sent: Tuesday, July 31, 2012 8:44 PM
> To: [email-masked]<mailto:[email-masked]>
> Subject: [CDNi] Fwd: draft-jin-cdni-content-deduplication-optimization-02.txt
>
>
> To: i-d-announce at ietf.org
> Subject: I-D Action: draft-jin-cdni-content-deduplication-optimization-02.txt
> From: internet-drafts at ietf.org
> Date: Sun, 29 Jul 2012 19:29:14 -0700
> Delivered-to: i-d-announce at ietfa.amsl.com
> List-archive: <http://www.ietf.org/mail-archive/web/i-d-announce>
> List-help: <mailto:[email-masked]?subject=help>
> List-id: Internet Draft Announcements only <i-d-announce.ietf.org>
> List-post: <mailto:[email-masked]>
> List-subscribe: <https://www.ietf.org/mailman/listinfo/i-d-announce>, <
> mailto:[email-masked]?subject=subscribe>
> List-unsubscribe: <https://www.ietf.org/mailman/options/i-d-announce>, <
> mailto:[email-masked]?subject=unsubscribe>
> Reply-to: internet-drafts at ietf.org
>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>
>
>         Title           : Content De-duplication for CDNi Optimization
>         Author(s)       : WeiYi Jin
>                           Mian Li
>                           Bhumip Khasnabish
>         Filename        : draft-jin-cdni-content-deduplication-
> optimization-02.txt
>         Pages           : 17
>         Date            : 2012-07-29
>
> Abstract:
>    Recent explosive growth of content delivery/distribution networks
>    (CDNs) and their interconnection are causing unintended repetition of
>    content storage in the same dCDN.  This can be avoided by using a
>    suitable de-duplication mechanism.  This document explores the
>    scenarios which create the problems, and then discusses the
>    approaches to eliminate the duplicated transmission of the same
>    content from uCDN(s) to dCDN in CDNi networks.  To implement the
>    optimization, some enhancements to the CDNi metadata model and
>    interface are required.
>
>
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-jin-cdni-content-
> deduplication-optimization
>
> There's also a htmlized version available at:
> http://tools.ietf.org/html/draft-jin-cdni-content-deduplication-
> optimization-02
>
> A diff from previous version is available at:
> http://tools.ietf.org/rfcdiff?url2=draft-jin-cdni-content-
> deduplication-optimization-02
>
>
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>  _______________________________________________
> CDNi mailing list
> [email-masked]<mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/cdni

---

From: [email-masked]
Date: Thu, 2 Aug 2012 14:22:30 +0800

Dear Kevin,

Thank you for you comments. Please see my opinions below:

P.S. dear all, due to the wrong operation of my mailbox, 
[email-masked] mailling list was added in this email loop by 
mistake. So if you will reply to this topic, do please remove this mailing 
list. Sorry for your inconvenience and thank you for you consideration~

>   When you say CSP-unique, does that include encoding-unique?
[Li Mian]: when I say CSP-unique, I mean the content ID assigned for a 
content object is unique within one CSP scope.
>[Li Mian]: Do you mean that a CDN or other intermediary modifies the 
content itself? Correct me if my understanding is wrong. As for me, this 
situation would not happen 'coz CDN is only responsible for delivering the 
content to the user. It is not likely to modify the content itself.
>[Li Mian]: actually we do not enforce to develop a new entity to assign 
and manage the Content ID for CDNi case. Both existing 
orgs/services/protocols and new ones can take charge of this. We are also 
seeking the proper method for Content ID mechanism and management. We 
would welcome your suggestions on this if you are also interested in this. 
Thank you~
> To: [email-masked]; [email-masked]; [email-masked]; 
> 'Bhumip Khasnabish'; [email-masked]
> Thank you for your comments and sharing with us your proposals. 
> Could you please share the documents you mentioned below with me? 
> Based on what you explained in previous email, I have some questions
> for clarification: 
> 1. We both consider it necessary to have a unique content identifier
> for a content object. Our opinion in this draft is that this content
> ID is CSP-unique. Could you please explain more about the uniqueness
> of the UID in your proposal? 
> 2. By applying the mechanism you proposed below, we can guarantee 
> the content is uniquely stored in one CDN. And according to my 
> analysis on the basis of your explanation, the content replication 
> is checked by comparing the CURL which is similar to the current 
> mechanism in CDNi work, correct me if my understanding is wrong. If 
> so, when redirection occurs between uCDNs and dCDN, the URL would be
> changed and thus be different with the original one - CURL. And if a
> dCDN is connected with two different uCDNs and two end users request
> the same content from these two uCDNs respectively, the two uCDNs 
> may generate different redirection URLs to the dCDN. In this case we
> think the content duplication issue still exists. How do you think? 
> If my understanding is wrong, could you please explain more on how 
> to consider this issue by using your proposal? 
> 
> Thank you again~ 
> 
> [email-masked] 写于 2012-08-01 02:00:23:
> 
> > Dear Bhumip, 
> > 
> > From: [email-masked] [mailto:[email-masked]] On Behalf 
Of 
> > To: [email-masked]

> > [email-masked]