Subject: Re: [Idr] draft on virtual aggregation

From: Paul Francis <[email-masked]>
Date: Mon, 7 Jul 2008 16:47:29 -0400

Its posted by IETF now.  At
http://www.ietf.org/internet-drafts/draft-francis-idr-intra-va-00.txt

PF

> -----Original Message-----
> From: Paul Francis
> Sent: Monday, July 07, 2008 12:53 PM
> To: [email-masked]'
> Subject: draft on virtual aggregation
> 
> 
> Gang,
> 
> At the following URL is a draft on virtual aggregation that I'm posting
> to IETF (it'll show up in a day or two), and which I'll present at IDR
> in Dublin.
> 
>  http://www.cs.cornell.edu/people/francis/draft-francis-idr-intra-va-
> 00.txt
> 
> Title and abstract are below.  I hope to create a work item on this in
> IDR.  I would characterize this as falling under the general charter of
> scaling BGP.
> 
> Any comments and discussion on this prior to Dublin is of course
> greatly appreciated.
> 
> PF
> 
> 
> Title:  Intra-Domain Virtual Aggregation
> 
> 
>    Virtual Aggregation (VA) is a technique for shrinking the DFZ FIB
>    size in routers (both IPv4 and IPv6).  This allows ISPs to extend
> the
>    lifetime of existing routers, and allows router vendors to build
> FIBs
>    with much less concern about the growth of the DFZ routing table.
> VA
>    does not shrink the size of the RIB.  VA may be deployed
> autonomously
>    by an ISP (cooperation between ISPs is not required).  While VA can
>    be deployed without changes to existing routers, doing so requires
>    significant new management tasks.  This document describes changes
> to
>    routers and BGP that greatly simplify the operation of VA.

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: Daniel Ginsburg <[email-masked]>
Date: Wed, 09 Jul 2008 01:21:36 +0400

> http://www.ietf.org/internet-drafts/draft-francis-idr-intra-va-00.txt

Paul,

I think that there're few problems with the draft.

One is with PHP. If a router is a border VA router, it must perform 
*ultimate*-hop-popping, not penultimate as section 3.2.3 of the draft 
states, since the router is egress for an LSP (as per RFC5036 
terminology). I.e. it means that a border VA router must not advertise 
implicit (or explicit) null label for FECs corresponding to external 
next-hops. Also section 3.2.3 implies that external next-hops must be 
carried inside AS' IGP, and IMHO the draft should explicitly say so.

Another problem is mixing legacy hop-by-hop forwarding and VA. Consider 
the following topology.

     10
A--------B
|        |
|100     | 10
|        |
C--------D------E
    10       10

A, B, D are VA routers. A is APR. B and E are legacy routers capable of 
only hop-by-hop forwarding, i.e. not doing MPLS. Link metrics are shown.

Packets ingress the topology at A. They get forwarded to B as plain IP 
since B is not an LSR. B forwards plain IP packets to D. D is a VA 
router and has suppressed the specific routes, thus it sends the packets 
back to A.

-- 
dg

---

From: Paul Francis <[email-masked]>
Date: Tue, 8 Jul 2008 19:01:35 -0400

Thanks for your comments Dan.  Replies inline.  

> From: Daniel Ginsburg [mailto:[email-masked]]
> Sent: Tuesday, July 08, 2008 5:22 PM
> To: Paul Francis
> Cc: [email-masked]
> Subject: Re: [Idr] draft on virtual aggregation
> 
> Paul Francis wrote:

Ok, I'll dig back into RFC5036.  I was assuming that, since the external
router is the target of the tunnel, but since the border router detunnels,
then this meant PHP.  

> next-hops. 

I'm afraid that I'm not up enough on MPLS terminology to even know what the
above means.  My bad.  I'll figure it out.

> Also section 3.2.3 implies that external next-hops must be

Good point---will add this in the next version.  BTW, of course another
approach would be to use stacked labels, with the outer label targeting the
border router, and the inner label identifying the external peer.  I'd love
to hear what people think of this as an alternative (or additional) approach.

> router and has suppressed the specific routes, thus it sends the

Yes.  This is why the draft states (clearly, I hope!) that legacy routers
must at least do MPLS.  If this is a problem (i.e., there are legacy routers
that don't do MPLS), then I believe that there is a way around this problem,
but it is a bit complex and I'd like to avoid it.

PF

---

From: Daniel Ginsburg <[email-masked]>
Date: Wed, 09 Jul 2008 04:12:43 +0400

> router is the target of the tunnel, but since the border router detunnels,

Since the border router is the first one which assigns a label to a 
prefix, I think it qualifies as ultimate hop. Just in case: I assume 
that the draft doesn't require speaking LDP to an external peer, right?

> above means.  My bad.  I'll figure it out.

It simply means that a border router must assign a separate non null 
label for each external next-hop to be able avoid doing L3 lookup for 
any of egressing packets and switch the packets solely on that label. I 
think this is what you intended in your draft.

I think that there might be some implementation difficulties when 
there're multiple external next-hops on a multiple access link, but I'll 
leave this to others, who are far more knowledgeable than me about 
current implementations, to comment.

> to hear what people think of this as an alternative (or additional) approach.

Well, you might need to extend LDP to be able to do that. Probably, 
procedures outlined in Kireeti Kompella's presentation 
(http://www.isocore.com/mpls2007/cd/Presentations/132%20Kireeti%20Kompella.pdf) 
might help. Though I'm not sure about standardization status of this 
approach.

> but it is a bit complex and I'd like to avoid it.

As I read the draft you still reserve a possibility of throwing 
hop-by-hop forwarding routers into the mix (section 2.1 and 3.2.1). 
IMHO, it is the best to avoid complexities of such configuration and 
strictly require all the routers to do MPLS.

---

From: Paul Francis <[email-masked]>
Date: Wed, 9 Jul 2008 09:56:50 -0400

> Sent: Tuesday, July 08, 2008 8:13 PM
> To: Paul Francis
> Cc: [email-masked]

Right.

> think this is what you intended in your draft.

Exactly. 

> might help. Though I'm not sure about standardization status of this

Actually I had more in mind to do it in the style of MPLS-VPNs.  That is,
carry the inner label in a new BGP multi-protocol extension created for this
purpose.  

> strictly require all the routers to do MPLS.

Ah, I see the confusion.  Yes, it is certainly my intent that legacy routers
must absolutely do MPLS and maintain the full set of tunnels.

That text is referring to the (hypothetical) case where some legacy router
has a tunnel to the next hop, but nevertheless chooses to use its
IGP-resolved interface rather than its tunnel interface.  This strikes me as
weird behavior, but I didn't want to assume that there is no router anywhere
that might do this (or might accidently be configured to do this), so I
mention it in the document.  If nobody knows of any router that does this,
I'm more than happy to take it out of the text to help avoid confusion..

PF

---

From: Daniel Ginsburg <[email-masked]>
Date: Wed, 09 Jul 2008 19:09:01 +0400

Paul,

I made another pass through the draft and I have few more comments, if 
you don't mind.

Section 3.2.2.2:
 > The LOCAL_PREF attribute is optional.

A small nit. As per RFC4271, section 5.1.5, LOCAL_PREF is included in 
all iBGP updates, so it can't be optional.

 >    4.  If a packet is received at the APR whose best match is the VP
 >        (i.e. it matches the VP but not any sub-prefixes within the VP),
 >        then the packet MUST be discarded (see Section 3.2.2.1).

Some ASes relay on default route to route some traffic while carrying 
[almost] full BGP table. One of notable examples is so called 
"disconnected backbone" setup. If virtual aggregation is implemented as 
described, it would break default route forwarding.

Section 3.2.2.3:
 >    The router MUST advertise the VP to its iBGP peers.  The router MUST
 >    NOT advertise the VP to eBGP peers.  (This should go without saying,
 >    since the Extended Communities T bit is set to be non-transitive.)

T-bit will only prevent extcommunity from propagating to eBGP neighbors. 
The prefix itself still will be advertised. I think NO_EXPORT should be 
always attached to VP as a safety measure.

Section 4.2:
There're some possible externally visible traffic engineering issues. AS 
with VA deployed might choose different exit points as compared to AS 
without VA.
These issues are similar to that of route reflectors deployments.

Section 4.3:
 >    Likewise, routers can bootstrap VA by first bringing up the IGP, then
 >    establish LSPs, then establish routes to all required sub-prefixes,
 >    and then finally advertise VPs.

Hmm, wouldn't delayed VP advertisement require non-APR routers to 
temporary install all the specific prefixes? If so, FIB overflow might 
occur. Some platforms react badly to such cases (i.e. overloading CPU, 
starving routing protocols and possibly crash), so it may destabilize 
the network.

-- 
dg

---

From: Paul Francis <[email-masked]>
Date: Thu, 10 Jul 2008 08:44:50 -0400

Dan, these are good comments.   My replies inline.

PF

> Sent: Wednesday, July 09, 2008 11:09 AM
> To: Paul Francis
> Cc: [email-masked]

Ok.

> described, it would break default route forwarding.

Good catch, we forgot about this case.  One observation here is that if the
router with the default is legacy, then it can probably run outside the scope
of this spec...that is, it doesn't make tunnels or anything...it just
defaults to the "backbone", which can be running VA.  Another observation is
that, if it is running VA, then it doesn't really need the default
approach...it can use VA to shrink its FIB (which has the advantage that it
can still advertise the full DFRT with its external peers).

I think nevertheless you can run VA and still have the default, though not
sure why you'd do this.  My understanding of disconnected backbone is that
you need to be pretty careful about which routers use the default....i.e.
only routers that connect to customers that themselves don't require the
complete DFRT, and don't transit traffic that isn't to or from said customer.
So as long as this is still the case, I'd think you could use a default route
in combination with VA, in which case the "MUST be discarded" clause is too
strict.  (Actually, if I think about it, it is probably unnecessarily strict
in the first place.)

> always attached to VP as a safety measure.

You mean, the T-bit will cause the border router to strip the ext-community
attribute from the route, and then send the route over eBGP?  I guess that
makes sense.  So we definitely need the NO_EXPORT attribute as well.  Will
fix. 

> These issues are similar to that of route reflectors deployments.

I don't see why VA would choose a different exit point.  VA doesn't change
the BGP decision process at all.  If a router selected a particular next-hop
without VA, it would select the same next-hop with VA.  Perhaps what you mean
is the following?

Say you had the following topology:

A--B--C--|--X
   |
   D-----|--Y

where X and Y are external peers.  In the case without VA, A gets a packet,
picks X as the exit, forwards the packet to B, which decides to pick Y as the
next hop (can this happen?).  If this is VA, however, once A picks X, it
tunnels the packet to X, so B can't divert it to a different exit.  If this
case occurs, then you are right that VA can cause a change in exit...

> starving routing protocols and possibly crash), so it may destabilize

Very nice catch!  I think the only good way to fix this is to require that
all VA routers be statically configured with the complete set of VPs.  It is
easy to conjure up scenarios where a router simply won't know all the VPs at
the point in time at which it needs to start loading up its FIB.  If the
router statically knows all the VPs, then it has a basis for deciding which
FIB entries to suppress, and avoiding FIB overflow.

PF

---

From: Daniel Ginsburg <[email-masked]>
Date: Tue, 15 Jul 2008 19:43:16 +0400

> case occurs, then you are right that VA can cause a change in exit...

Yes, it can happen. Assume the following link metrics:

      1
A--B---C--|--X
    |
    |10
    |
    D------|--Y

Without VA, A would choose X as the exit point, D would choose Y. Now
with VA, D is APR. D still chooses Y, but A doesn't carry a specific
route and tunnels packets to D, and the packets now exit to Y.

> FIB entries to suppress, and avoiding FIB overflow.

Hmm, now if both APRs and non-APR are statically configured with VPs, 
what's left to the proposed protocol extension?

-- 
dg

---

From: Paul Francis <[email-masked]>
Date: Tue, 15 Jul 2008 12:15:34 -0400

inline

> Sent: Tuesday, July 15, 2008 11:43 AM
> To: Paul Francis
> Cc: [email-masked]
> >> There're some possible externally visible traffic engineering
> issues.
> >> AS

Ok.

> 
> >> Section 4.3:
> >>  >    Likewise, routers can bootstrap VA by first bringing up the
> IGP,
> >> then

Heh!  Will have to think about this.  But maybe in fact we don't need the
extension.  For instance, say an ISP wants to configure a new VP.  It would
first configure the APRs for that VP, which would start advertising the VP
route (without the new attribute, but with NO_EXPORT).  Subsequently, it
could reconfigure the non-APRs (one by one) so that they now know that the VP
is in fact a VP, and could start suppressing sub-prefixes...off-hand seems
ok...

PF