Subject: [CDNi] footprint and capability mechanisms

From: "Peterson, Jon" <[email-masked]>
Date: Thu, 6 Feb 2014 18:41:05 +0000

Following our discussion about the FCI mechanism in Vancouver, I took a
look at two drafts out there today: draft-ma and draft-seedorf. In short,
I think both express promising directions, and both have a long way to go
before they will describe fully fleshed-out mechanisms. At a high level,
their architectures are similar: both are about pushing JSON objects that
describe footprints and capability via HTTP. It¹s often when the choices
are so similar that it¹s hardest to figure out which is the right approach.

One of the prominent distinctions between the two is that Kevin specifies
a capability-sharing system with an optional footprint, whereas Jan
specifies a footprint-sharing system with the option of capabilities.
Jan¹s choice is an artifact of using ALTO as a basis, which already has a
concept of sharing 'network maps' in this fashion, and can be extended to
add further information to these maps such as capability. Kevin works from
the ground up on a bespoke system for sharing capabilities, which may not
ever involve sharing footprint data. I¹ve turned this over in my head a
bunch of times, and I¹m not really sure there¹s any practical advantage
one approach has over the other. One of the non-obvious reasons why is
that the concept of a PID (the groupings of network elements) in ALTO is
so abstract that it serves as a layer of indirection, and can serve as a
placeholder for a footprint that¹s effectively specified implicitly.

An important feature we want this mechanism to have is incremental FCI
updates. Both Kevin and Jan have a story about this, though I think both
will require substantial work to advance. Kevin, for example, sketches a
system of primitives (replace, include, exclude) to allow a dCDN to update
a uCDN about changes, based on tracking update sequencing with a new
'CDNI-FCI-Seq' HTTP header. State freshness of web resources is a pretty
thoroughly-studied problem, though, and there are other HTTP facilities
(Etags, say) that address this. I think it¹s pretty unlikely that our
problem is enough of a special case that we could convince Application
folks here at the IETF that we need to standardize a custom HTTP header
for it. Jan¹s draft, on the other hand, references existing ALTO work on
incremental updates, in particular draft-schwan-alto-incr-updates. That
draft contains a good overview of the different approaches, including HTTP
If-Modified-Since and Etags, as well as looking at JSON-specific
incremental change techniques like JSON Patch (RFC6902). Sounds great, but
- draft-schwan is currently an expired I-D (for like a year), and it is
really more of a survey than a draft that makes a concrete recommendation.
This work would need to be resumed and made far more concrete for this to
be a viable approach.

Asynchronous updates are another very important feature. We don¹t want to
force uCDNs to poll dCDNs, we want dCDNs to be capable of asynchronously
notifying uCDNs when a change to network state happens. Kevin again has a
sketch of a solution here, where capability information can either be
acquired by a GET from the uCDN or a POST by the dCDN. While as described
it is very lightweight, I suspect it¹s probably a bit too lightweight: I¹d
want to know a lot more about how resource identifiers are discovered or
formulated, and a ton more about error-handling and security. This
mechanism almost certainly should be ReST-based, as it will then inherit
many of the right principles. ALTO, based on Jan¹s draft, of course is
already ReST-based, and has exhaustive sections on JSON encoding, errors,
security, and the way to formulate queries via POSTs. What it lacks is an
asynchronous publication capability. Once again, there are some I-Ds we
can point to, but nothing I¹d consider very concrete. Worse still, even
the capability data that ALTO might carry is based on a -00 I-D
(draft-roome-alto-pid-properties) which is very sketchy, to the point
where providing even a mock-up of the JSON encoding for capabilities in
ALTO is probably impossible today.

So where does this leave us? Fleshing out either of these approaches will
take work. We don¹t want to do needless work articulating two proposals
where we only need one. It is however difficult to have a serious
discussion about the advantages of either mechanism when they are only
specified to this limited degree. Ideally some kind of merger would be
possible here, and the sooner we can get to that the better. I don¹t think
the baseline JSON formats for rendering either footprints or capability
will ultimately vary significantly in the two proposals. It then just
becomes a question of whether it is more practical to start tabula rasa,
as Kevin does, or to build on an existing mechanism, as Jan does. ALTO
comes with baggage, but that baggage has accumulated from several years
now of comment and review, and surely a lot of those components (like
error handling, encoding, security) would have to be incorporate into
Kevin¹s draft if we went forward with it. I suspect that would ultimately
be a longer path, but neither of these paths are particularly short.

Jon Peterson
Neustar, Inc.

---

From: Jan Seedorf <[email-masked]>
Date: Fri, 7 Feb 2014 09:01:22 +0000

Dear Jon,

Thank you for the good and fair summary below, this is pretty much the status quo. Just one comment: the reason why some of the necessary ALTO extensions have been delayed is that the ADs/chairs have disallowed such work to be discussed before the core ALTO protocol has been published; just recently green light for continuing work on extensions has been given.

What in my view it boils down to (as you highlight below):
-- ALTO has error handling, encoding, security worked out (pro for using ALTO)
-- incremental and asynchronous updates for ALTO is early stage; in fact the ALTO WG is right now re-chartering to add those two items to the charter item list, early work exists, but it will take some time for those specs to be ready (con for using ALTO)

What Richard and I have in mind was to drive these very-soon-to-be-on-the-chartered ALTO extensions (i.e. incremental/asynchronous updates) by the concrete CDNI FCI use case. In other words we intend to define a solution for CDNI/FCI and kick that back into the ALTO WG. But no doubt, having these ALTO extensions done in ALTO will likely take longer than the current CDNI milestone for the FCI solution spec.

Kevin, all: anything you want to share to the discussion?

 - Jan

> -----Original Message-----
> From: CDNi [mailto:[email-masked]] On Behalf Of Peterson, Jon
> Sent: Thursday, February 06, 2014 7:41 PM
> To: [email-masked]
> Subject: [CDNi] footprint and capability mechanisms
> 
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni

---

From: Kevin J Ma <[email-masked]>
Date: Sun, 9 Feb 2014 23:47:45 -0500

I think my question is still about the actual call flows for the ALTO
implementation, and whether there are any significant inefficiencies?
Though, to Jon's point, both approaches are going to need more work,
so it's hard to assess from the current state.

I believe Matt had an action from Vancouver to also do an analysis?

--  Kevin J. Ma

> From: CDNi [mailto:[email-masked]] On Behalf Of Jan Seedorf
> Sent: Friday, February 07, 2014 4:01 AM
> To: Peterson, Jon; [email-masked]
> Subject: Re: [CDNi] footprint and capability mechanisms
> 
> Dear Jon,
> 
> Thank you for the good and fair summary below, this is pretty much the
> status quo. Just one comment: the reason why some of the necessary ALTO
> extensions have been delayed is that the ADs/chairs have disallowed such
> work to be discussed before the core ALTO protocol has been published;
> just recently green light for continuing work on extensions has been
> given.
> 
> -- incremental and asynchronous updates for ALTO is early stage; in fact
> the ALTO WG is right now re-chartering to add those two items to the
> charter item list, early work exists, but it will take some time for those
> specs to be ready (con for using ALTO)
> 
> What Richard and I have in mind was to drive these very-soon-to-be-on-the-
> chartered ALTO extensions (i.e. incremental/asynchronous updates) by the
> concrete CDNI FCI use case. In other words we intend to define a solution
> for CDNI/FCI and kick that back into the ALTO WG. But no doubt, having
> these ALTO extensions done in ALTO will likely take longer than the
> current CDNI milestone for the FCI solution spec.
>

---

From: "Matt Caulfield (mcaulfie)" <[email-masked]>
Date: Mon, 10 Feb 2014 12:40:37 +0000

Yes - that action is still in my queue. 

Expect my analysis sometime in the next week or so.

Thanks,
Matt

-----Original Message-----
From: CDNi [mailto:[email-masked]] On Behalf Of Kevin J Ma
Sent: Sunday, February 09, 2014 11:48 PM
To: Jan Seedorf; Peterson, Jon; [email-masked]
Subject: Re: [CDNi] footprint and capability mechanisms

I think my question is still about the actual call flows for the ALTO implementation, and whether there are any significant inefficiencies?
Though, to Jon's point, both approaches are going to need more work, so it's hard to assess from the current state.

--  Kevin J. Ma

> status quo. Just one comment: the reason why some of the necessary 
> ALTO extensions have been delayed is that the ADs/chairs have 
> disallowed such work to be discussed before the core ALTO protocol has 
> been published; just recently green light for continuing work on 
> extensions has been given.
> 
> -- ALTO has error handling, encoding, security worked out (pro for 
> using
> ALTO)
> -- incremental and asynchronous updates for ALTO is early stage; in 
> fact the ALTO WG is right now re-chartering to add those two items to 
> the charter item list, early work exists, but it will take some time 
> for those specs to be ready (con for using ALTO)
> 
> What Richard and I have in mind was to drive these 
> very-soon-to-be-on-the- chartered ALTO extensions (i.e. 
> incremental/asynchronous updates) by the concrete CDNI FCI use case. 
> In other words we intend to define a solution for CDNI/FCI and kick 
> that back into the ALTO WG. But no doubt, having these ALTO extensions 
> done in ALTO will likely take longer than the current CDNI milestone for the FCI solution spec.
> 
> > Following our discussion about the FCI mechanism in Vancouver, I 
> > took a look at two drafts out there today: draft-ma and 
> > draft-seedorf. In
> short,
> > I think both express promising directions, and both have a long way 
> > to
> go
> > before they will describe fully fleshed-out mechanisms. At a high 
> > level, their architectures are similar: both are about pushing JSON 
> > objects
> that
> > describe footprints and capability via HTTP. It¹s often when the 
> > choices are so similar that it¹s hardest to figure out which is the 
> > right
> approach.
> >
> > Jan¹s choice is an artifact of using ALTO as a basis, which already 
> > has
> a
> > concept of sharing 'network maps' in this fashion, and can be 
> > extended
> to
> > add further information to these maps such as capability. Kevin 
> > works
> from
> > the ground up on a bespoke system for sharing capabilities, which 
> > may
> not
> > ever involve sharing footprint data. I¹ve turned this over in my 
> > head a bunch of times, and I¹m not really sure there¹s any practical 
> > advantage one approach has over the other. One of the non-obvious 
> > reasons why is that the concept of a PID (the groupings of network 
> > elements) in ALTO is so abstract that it serves as a layer of 
> > indirection, and can serve as a placeholder for a footprint that¹s effectively specified implicitly.
> >
> > An important feature we want this mechanism to have is incremental 
> > FCI updates. Both Kevin and Jan have a story about this, though I 
> > think both will require substantial work to advance. Kevin, for 
> > example, sketches a system of primitives (replace, include, exclude) 
> > to allow a dCDN to
> update
> > 'CDNI-FCI-Seq' HTTP header. State freshness of web resources is a 
> > pretty thoroughly-studied problem, though, and there are other HTTP 
> > facilities (Etags, say) that address this. I think it¹s pretty 
> > unlikely that our problem is enough of a special case that we could 
> > convince Application folks here at the IETF that we need to 
> > standardize a custom HTTP header for it. Jan¹s draft, on the other 
> > hand, references existing ALTO work on incremental updates, in 
> > particular draft-schwan-alto-incr-updates. That draft contains a 
> > good overview of the different approaches, including
> HTTP
> > incremental change techniques like JSON Patch (RFC6902). Sounds 
> > great,
> but
> > - draft-schwan is currently an expired I-D (for like a year), and it 
> > is really more of a survey than a draft that makes a concrete
> recommendation.
> > This work would need to be resumed and made far more concrete for 
> > this
> to
> > be a viable approach.
> >
> > Asynchronous updates are another very important feature. We don¹t 
> > want to force uCDNs to poll dCDNs, we want dCDNs to be capable of 
> > asynchronously notifying uCDNs when a change to network state 
> > happens. Kevin again has
> a
> > sketch of a solution here, where capability information can either 
> > be acquired by a GET from the uCDN or a POST by the dCDN. While as
> described
> > want to know a lot more about how resource identifiers are 
> > discovered or formulated, and a ton more about error-handling and 
> > security. This mechanism almost certainly should be ReST-based, as 
> > it will then inherit many of the right principles. ALTO, based on 
> > Jan¹s draft, of course is already ReST-based, and has exhaustive 
> > sections on JSON encoding,
> errors,
> > security, and the way to formulate queries via POSTs. What it lacks 
> > is
> an
> > asynchronous publication capability. Once again, there are some I-Ds 
> > we can point to, but nothing I¹d consider very concrete. Worse 
> > still, even the capability data that ALTO might carry is based on a 
> > -00 I-D
> > (draft-roome-alto-pid-properties) which is very sketchy, to the 
> > point where providing even a mock-up of the JSON encoding for 
> > capabilities in ALTO is probably impossible today.
> >
> > take work. We don¹t want to do needless work articulating two 
> > proposals where we only need one. It is however difficult to have a 
> > serious discussion about the advantages of either mechanism when 
> > they are only specified to this limited degree. Ideally some kind of 
> > merger would be possible here, and the sooner we can get to that the 
> > better. I don¹t
> think
> > the baseline JSON formats for rendering either footprints or 
> > capability will ultimately vary significantly in the two proposals. 
> > It then just becomes a question of whether it is more practical to 
> > start tabula rasa, as Kevin does, or to build on an existing 
> > mechanism, as Jan does. ALTO comes with baggage, but that baggage 
> > has accumulated from several years now of comment and review, and 
> > surely a lot of those components (like error handling, encoding, 
> > security) would have to be incorporate into Kevin¹s draft if we went 
> > forward with it. I suspect that would
> ultimately

---

From: "Peterson, Jon" <[email-masked]>
Date: Mon, 10 Feb 2014 19:55:40 +0000

Do you mean inefficiencies beyond the need for asynchronous and
incremental updates? This is the discussion I think we need to have.

Jon Peterson
Neustar, Inc.

On 2/9/14, 8:47 PM, "Kevin J Ma" <[email-masked]> wrote:

>>very-soon-to-be-on-the-
>>level,
>>choices
>>pretty
>> > notifying uCDNs when a change to network state happens. Kevin again
>>has
>> a
>>we
>>proposals
>>capability

---

From: Scott Wainner <[email-masked]>
Date: Mon, 10 Feb 2014 15:35:03 -0500

> placeholder for a footprint that¹s effectively specified implicitly.
My opinion is:

1) the capabilities must match before we even care about the properties 
of the footprint
2) the properties of the footprint are likely defined in a contract 
(i.e. global coverage, regional coverage)
3) retraction of a portion of footprint probably violates a contractual 
obligation; therefore, no CDN would announce it
4) the advertisement of a footprint (that conforms to a contract) is 
probably just a means of automating the CDN selection by the uCDN which 
already has the ability to statically assign a footprint to a dCDN based 
on the contract
5) the uCDN probably has many other rules that it must consider before 
footprint comes into play (i.e. geo-location, quotas, performance, 
time-of-day, etc.).  For example, footprint becomes somewhat irrelevant 
if the performance of the dCDN doesn't meet an SLA.  It doesn't matter 
how close the dCDN is to the client.  If the dCDN can't deliver the 
content in a timely manner or with sufficient throughput, the uCDN 
should pick a different dCDN.

> This work would need to be resumed and made far more concrete for this to
... or deferred so we can get a basic Footprint and Capabilities 
Interface specified.  Most likely the dCDN will want to advertise global 
coverage (to maximize their perceived utility).  The exception is an ISP 
CDN that only wants to serve content for their own clients.  Thus, a 
specific footprint is defined which probably correlates with the ISP's 
address space advertised via E-BGP.
> ALTO is probably impossible today.
If we simplify the requirements, we might conclude that a RESTful POST 
from the dCDN that describes the capabilities and the associated 
footprint is all we need.  What the uCDN needs to know is the 
availability of the dCDN's ability to route content requests. The 
recursive mode is no problem, but the iterative mode may be a problem.  
Having a periodic HTTP POST from the dCDN to the uCDN may serve as a 
'keep-alive' for a given capability and footprint.

We can focus on enhancing the granularity of the advertisement in a 
subsequent release.
> .
Scott

=======================
  Scott Wainner
   Cisco, Distinguished SE
   USSP Verizon
   +1-703-484-5820
   [email-masked]
=======================
_____ .:|:.:|:. _______

---

From: Kevin J Ma <[email-masked]>
Date: Mon, 10 Feb 2014 16:08:40 -0500

Basically that's what I meant.
Initialization as well as asynchronous/incremental updates.

> From: Peterson, Jon [mailto:[email-masked]]
> Sent: Monday, February 10, 2014 2:56 PM
> To: Kevin J Ma; Jan Seedorf; [email-masked]
> Subject: Re: footprint and capability mechanisms
> 
> 
> fact
> FCI
> want
> >> > it is very lightweight, I suspect it¹s probably a bit too
> lightweight:
> >> I¹d

---

From: "Y. Richard Yang" <[email-masked]>
Date: Mon, 10 Feb 2014 17:12:58 -0500

Hi Kevin,

Regarding incremental updates, a short answer is that one may use JSON
Patch (RFC6902). If we emphasize reusability (in IETF), then this is the
way to go. There are discussions among the authors that we can improve the
encoding efficiency using a customized incremental updates encoding. This
can be important for a large-scale update, e.g., outage to a large number
of end hosts. But this may or may not be crucial for cdni.

Let's see some more details on the overhead. Jan's ALTO-based design uses
two maps:

- The footprint map (ALTO Network Map, which assigns each (PID) name to its
set of endhosts)
- The capability map (the PID property map, which maps each PID name to its
current set of capabilities).

A use case I see that the FCI interface provides is that the dCDN provides
notifications about failures/temporary outage to the uCDN. Hence, let's
consider the overhead of the updates in such settings.

Case 1: No change to the Network Map. It is just that the capabilities of
some named network regions change. Then, only the capability map changes.
Updating the capability map should be small, because I anticipate (based on
BGP data size) that the largest amount of data will be IP prefixes. To be
concrete, if a region, say south-France (or whatever ever city named in the
Network Map), is down/overloaded, the update is just an update to the
mapped capabilities of the PID. No IP prefixes appear on the wire. When the
outage/overload is over, update is again simple, with no IP prefixes on the
wire.

Case 2: There is a need to change the Network Map. We can identify several
key cases. One: an IP address moved (e.g., IP address relocation), so that
an IP address moved from south-France to north-France. This will be a
delete of the IP address from the old PID and an insertion of the same to
the new PID. If the mapping from PID to capability does not change, we have
small overhead. Second: the Network Map is not well designed such as that
it is too large a region, and there is a partial failure of the region on a
given capability. Then the change can be large in that these failed IP
addresses will need to be removed from the old PID and moved to a new PID;
the capabilities of the new PID will need to defined. However, this
overhead is hard to reduce. If IP prefixes are the largest data items on
the wire, then the Network Map should define "atoms" to avoid individual IP
prefixes appears on the wire.

Is this the overhead that you are referring to?

Richard

On Mon, Feb 10, 2014 at 4:08 PM, Kevin J Ma <[email-masked]>wrote:

> ALTO
> took
> head
> > >> > that the concept of a PID (the groupings of network elements) in
> ALTO
> > >>is
> > >> > for it. Jan¹s draft, on the other hand, references existing ALTO
> work
> > >>on
> > >> > draft contains a good overview of the different approaches,
> including
> > >> HTTP
> be
> discovered
> > >>or
> point

-- 
-- 
 =====================================
| Y. Richard Yang <[email-masked]>   |
| Professor of Computer Science       |
| http://www.cs.yale.edu/~yry/        |
 =====================================

---

From: "Peterson, Jon" <[email-masked]>
Date: Tue, 11 Feb 2014 00:42:38 +0000

Good points, a few notes inline.

Jon Peterson
Neustar, Inc.

>should pick a different dCDN.

I'm mostly on the same page as you (especially about 3), though I suspect
that your point 2 applies as well to point 1: the capabilities will also
largely be a matter for the contracts. I heartily concur that decisions in
the future, like decisions today, will depend mostly on factors like you
describe in 5: quotas, performance and so on.

>address space advertised via E-BGP.

Agreed that incremental updates don't need to be a day-one feature, and
that dCDNs are likely to advertise global coverage. If everyone could have
agreed two years ago that footprint just meant what prefixes you advertise
in BGP... we would have been done two years ago.

>>[snip]

I'm not sure I can go that far. If we grant that FCI is worth sharing at
all, I think it's still useful for a uCDN to get able to send a GET to the
dCDN to request its FCI state. It doesn't incur any additional complexity
that I can see. Moreover, you do need to talk about things like error
handling, and the encoding and processing of the FCI information itself,
for this to be more than just a sketch. It's not just "use REST and we're
done."

>We can focus on enhancing the granularity of the advertisement in a

You are certainly right to counsel that we not require that either of
these proposals do everything out of the gate.