Subject: Re: [CDNi] CDNI Triggers interface

From: Rob Murray <[email-masked]>
Date: Wed, 4 Apr 2012 21:46:25 +0000

Hi Kevin,

Apologies for the delayed response, and thank you for clarifying that use
case.

I'd be interested to hear from others, but my initial thinking is that
triggered activity is most likely to be pushed into the network quite
quickly, and happen in parallel with other triggers. So, unless dCDN
decides it's overloaded, each trigger will probably not spend much time in
"pending" state. If that's the case, offering the appearance of
finer-grained control over the scheduling may not add much. But, that
said, a priority indication would be an easy thing to add to the protocol
(now or later), if it's thought to be useful.

The subject of trigger ordering was discussed further in Paris, outside
the WG session. In particular - Matt Caulfield pointed out that in a
cascade of CDNs, careful ordering of request initiation by uCDN into dCDN
could be lost when dCDN passes the triggers downstream.

I think we concluded that a trigger request should be allowed to contain
an ordered list of triggers. Failure of any one of the triggers will be
reported as failure of the whole request, and dCDN must pass on ordered
triggers to its dCDNs as ordered triggers. Internally however, each CDN
may still process those triggers in parallel, so the new semantics of
content invalidate/purge up to the time the request was received still
apply. Under this scheme, to replace some content, you'd bundle an
invalidate/purge with a preposition in a single request. That eliminates
the window where newly acquired content could be invalidated/purged.

I'll update the draft to reflect that, and the other comments received.

Many thanks,
Rob.

On 28/03/2012 15:40, "Kevin J Ma" <[email-masked]> wrote:

>Hi Rob,
>
>  Wrt priorities, I was thinking about a case where there are a large
>number
>  of trigger requests, possibly initiated by end customers, e.g., the uCDN
>  provides a web portal to allow customers to purge/invalidate content.
>Not
>  all customers/administrators are created equal, and not wanting to
>delete,
>  remember, and repost every trigger, it might be useful to have
>priorities.
>  I agree that we may not want triggers to be pre-emptive, and the dCDN
>may
>  have its own priorities, but if a uCDN and dCDN negotiate some
>understanding,
>  outside the scope of CDNI, would a priority field be useful?
>
>thanx.
>
>--  Kevin J. Ma
>
>> -----Original Message-----
>> From: Rob Murray [mailto:[email-masked]]
>> Sent: Wednesday, March 28, 2012 6:25 AM
>> To: Kevin J Ma; Kent Leung (kleung); [email-masked]
>> Subject: Re: [CDNi] CDNI Triggers interface
>> 
>> Hi Kevin, and thanks - responses inline...
>> 
>> On 28/03/2012 02:08, "Kevin J Ma" <[email-masked]> wrote:
>> 
>> >Hi Rob,
>> >
>> >  I think the doc does a good job of clearly defining the space.
>> >  I had a couple of questions:
>> >
>> >  - Section 4.4 mentions returning 404/410 on subsequent requests for a
>> >    resource after deletion.  I assume this also applies to section 4.3
>> >    and explicit deletion?  Might be worth clarifying in section 4.3?
>> 
>> Yes, will do.
>> 
>> >  - Along the lines of Matt and Kent's questions about request
>>ordering,
>> >    I was wondering about repeat requests.  The current set of triggers
>> >    are fairly idemopotent, so perhaps it would only be an optimization
>> >    for them, but are there concerns for duplicate requests or requests
>> >    with overlapping urls/patterns wrt any pending or active triggers?
>> 
>> I think the interaction we want to avoid is between invalidate/purge and
>> preposition, I hope my suggestion for using trigger "ctime" to indicate
>> which data should be invalidated/purged addresses that? And, as you say,
>> the current operations are fairly idempotent.
>> 
>> Each uCDN can only affect its own data, so it should be in full control
>>of
>> any interactions. If it asks dCDN to do something silly with overlapping
>> URLs/paterns, I don't think dCDN should try to second-guess it.
>> 
>> >  - For trigger creation, would it make sense to have a priority field,
>> >    to allow uCDNs to influence the order in which dCDNs process
>> triggers?
>> 
>> Whether it's possible to influence the order of processing will depend a
>> lot on the implementation in dCDN. It may or may not be possible to
>> stop/pause/pace triggered activity on any given cache, or across the
>> network. Allowing the uCDN to set a priority might give an illusion of
>> control that it doesn't have.
>> 
>> Do you have a particular use-case in mind? One that springs to mind
>>would
>> be that uCDN has made a time-consuming request, then wants to inject
>> something more urgent. In this case it has the option of deleting the
>> original trigger (dCDN may or may not stop work on it) and re-creating
>>it
>> later, the idempotence of the operations helps us again there.
>> 
>> Overall, I think it's probably best to keep things simple and leave
>> scheduling/ordering of triggers under uCDN control by expecting it to
>>make
>> requests and wait for completions in an order it likes.
>> 
>> 
>> >thanx.
>> >
>> >--  Kevin J. Ma
>> >
>> >> -----Original Message-----
>> >> From: [email-masked] [mailto:[email-masked]] On Behalf
>>Of
>> >> Rob Murray
>> >> Sent: Sunday, March 25, 2012 7:43 AM
>> >> To: Kent Leung (kleung); [email-masked]
>> >> Subject: Re: [CDNi] CDNI Triggers interface
>> >>
>> >> Thanks Kent - replies inline ...
>> >>
>> >> On 22/03/2012 18:11, "Kent Leung (kleung)" <[email-masked]> wrote:
>> >>
>> >> >Hi Rob. Nice write-up. Here are some comments on the draft.
>> >> >
>> >> >1. Sect. 2: Nit for "The trigger may request action on either
>>metadata
>> >> >or on content". Reword to "The trigger may request action on either
>> >> >metadata or content"?
>> >>
>> >> Will do.
>> >>
>> >> >2. Sect. 2: Any thoughts on the scalability aspect of using RESTful
>> web
>> >> >service assuming there will be many dCDNs providing delivery service
>> >>for
>> >> >a uCDN? These triggers will likely apply on all the dCDNs.
>> >>
>> >> Since uCDN has the contract with the Content Owner (or with another
>> uCDN
>> >> that does), it has ultimate responsibility for ensuring that triggers
>> >>are
>> >> acted upon - I think it will need to track which dCDNs have its data
>> and
>> >> which have completed its triggers, for auditing purposes if nothing
>> >>else?
>> >> Because it's got that responsibility, uCDN will want a way to
>> >>interrogate
>> >> dCDN for status of triggers so, correspondingly, dCDN will need to
>> track
>> >> and report state of triggers.
>> >>
>> >> So, I'd say per-trigger, per-interconnect state will exist at both
>>ends
>> >> of the interface whether it's RESTful or not. Does that answer the
>> >> concern? I can add some descriptive text to the draft along those
>>lines
>> >>if
>> >> it does.
>> >>
>> >> Do you have a feel for the numbers of interconnects a given CDN is
>> >>likely
>> >> to have? I can't find an indication in the problem-statement or
>> >> requirements - but I'd imagine each CDN will have a relatively small
>> >> number
>> >> of interconnect agreements, perhaps up to low tens? Generally I'd
>>think
>> >> CDNs will have many fewer interconnects than Delivery Surrogates, but
>> an
>> >> exception might be a "broker" CDN that owns content but doesn't do
>>the
>> >> actual delivery itself. In that case, I'd guess we're not talking
>>about
>> >> thousands of interconnects but perhaps it might get into hundreds? I
>> >> don't have any actual facts or data to cloud my thinking though (!),
>>so
>> >> I'd be interested to hear other views.
>> >>
>> >> >3. Sect. 4: Typo in "For example, is anticipated that decisions on
>>use
>> >> >of HTTPS for other CDNI interfaces will be adopted for Triggers."
>> >>
>> >> Will fix.
>> >>
>> >> >4. Sect. 4.1: The Trigger Request must be processed in the sequence
>>as
>> >> >triggered by the uCDN? Is there a message sequencing method defined?
>> >>For
>> >> >example, uCDN wants to purge a specific content, then preposition
>>that
>> >> >content. What happens when the message arrives at dCDN out of order?
>> >>
>> >> Good point, thanks, I've not covered that.
>> >>
>> >> If we define the invalidate/purge triggers to mean "invalidate or
>>purge
>> >> data obtained before this trigger was created" (before "ctime" of the
>> >> Trigger Status Resource), I think that solves the problem. For
>>example,
>> >>a
>> >> use-case for "invalidate" followed by "preposition" would be an
>> >>emergency
>> >> fix to some metadata or content - with this change, by issuing
>>triggers
>> >>in
>> >> order after the data is repaired, uCDN can be sure incorrect data is
>> >> deleted. But, data acquired from the same location after the repair
>> >> (either as a result of pre-positioning or normal operation) will not
>> >>need
>> >> to be re-fetched. The invalidate/purge and the preposition triggers
>>can
>> >> run concurrently.
>> >>
>> >> I makes the semantics of invalidate/purge cleaner anyway. If
>>(repaired)
>> >> data can still obtained from affected URL, it'd be hard for uCDN to
>> >> guarantee that when a purge completes no data from affected URLs
>>exists
>> >> in dCDN. Much easier for it to say that data acquired before a given
>> >> time has been invalidated/purged from the network.
>> >>
>> >> >5. Sect. 4.2: Nit for ".. to cheaply check for change .." Maybe " ..
>> to
>> >> >inherently check for change .."?
>> >>
>> >> How about "... to check for change in status of a resource or
>> collection
>> >> of resources without re-fetching the whole resource or collection."
>> >>
>> >> >
>> >> >6. Sect 4.2: Reword " to indicate the frequency it would like uCDN
>>to
>> >> >poll at." to " to indicate the frequency of polling by the uCDN"?
>> >>
>> >> How about "The dCDN should use the cache control headers for
>>responses
>> >>to
>> >> GETs for Trigger Status Resources and Collections to indicate the
>> >> frequency at which it recommends uCDN should poll for change."
>> >>
>> >> >Kent
>> >> >
>> >> >-----Original Message-----
>> >> >From: [email-masked] [mailto:[email-masked]] On Behalf
>> Of
>> >> >Rob Murray
>> >> >Sent: Wednesday, February 29, 2012 1:25 PM
>> >> >To: [email-masked]
>> >> >Subject: [CDNi] CDNI Triggers interface
>> >> >
>> >> >Hi all,
>> >> >
>> >> >I've just uploaded a new draft that proposes a CDNI Triggers
>>interface
>> >> >...
>> >> >
>> >> >    http://datatracker.ietf.org/doc/draft-murray-cdni-triggers/
>> >> >
>> >> >
>> >> >There was some discussion at the last WG meeting about whether
>> triggers
>> >> >fit best as part of the control or metadata interface - the
>>suggestion
>> >> >was
>> >> >that concrete proposals for the interface might help us spot
>> >>commonality
>> >> >with one or the other, so let's see!
>> >> >
>> >> >Please take a look, your comments are welcome.
>> >> >
>> >> >Best regards,
>> >> >Rob.
>> >> >
>> >> >
>> >> >On 29/02/2012 19:37, "[email-masked]"
>> >> ><[email-masked]>
>> >> >wrote:
>> >> >
>> >> >>A new version of I-D, draft-murray-cdni-triggers-00.txt has been
>> >> >>successfully submitted by Rob Murray and posted to the IETF
>> >>repository.
>> >> >>
>> >> >>Filename:	 draft-murray-cdni-triggers
>> >> >>Revision:	 00
>> >> >>Title:		 CDN Interconnect Triggers
>> >> >>Creation date:	 2012-02-29
>> >> >>WG ID:		 Individual Submission
>> >> >>Number of pages: 33
>> >> >>
>> >> >>Abstract:
>> >> >>   This document proposes a mechanism for a CDN to trigger activity
>> in
>> >> >>   an interconnected CDN that is configured to deliver content on
>>its
>> >> >>   behalf.  The upstream CDN can use this mechanism to request that
>> >>the
>> >> >>   downstream CDN pre-positions metadata or content, or that it re-
>> >> >>   validate or purge metadata or content.  The upstream CDN can
>> >>monitor
>> >> >>   the status of activity that it has triggered in the downstream
>> CDN.
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>The IETF Secretariat
>> >> >
>> >> >_______________________________________________
>> >> >CDNi mailing list
>> >> >[email-masked]
>> >> >https://www.ietf.org/mailman/listinfo/cdni
>> >>
>> >> _______________________________________________
>> >> CDNi mailing list
>> >> [email-masked]
>> >> https://www.ietf.org/mailman/listinfo/cdni
>

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Wed, 4 Apr 2012 23:58:08 +0100

Rob, Kevin,

> (now or later), if it's thought to be useful.

Ordering triggers is one thing, having priorities is another.

What does priority mean in the context of a trigger (i.e. what is it prioritised against)?

- Priority against other triggers from the same user?
- Priority from other triggers from the same uCDN (which might have been created by different services/users of that uCDN)?
- Priority against triggers from a different uCDN?

I think it quickly becomes a non-straightforward problem to scope and define the semantics necessary for interoperability and I'm far from convinced we need to go there (I've never worked with a CDN that allows their equivalent of triggers to be prioritised but I haven't worked with them all).

My view is the ordering within a trigger request should be sufficient for our needs and let's avoid the can of worms that would be priorities for triggers until we are certain we actually need that added complexity.

Ben