Subject: [Idr] Fwd: I-D ACTION:draft-pmohapat-idr-acceptown-community-01.txt

From: Danny McPherson <[email-masked]>
Date: Tue, 29 Apr 2008 22:38:46 -0600

Surprisingly, I don't recall seeing this draft discussed here yet.

In short, I think the is a really bad idea.  It's bad enough the route
reflection spec was changed from 1966 to 2796 to permit an RR
to reflect routes to a client even if they were learned from that
client - arguably, to enable an implementation optimization, but
for this to recommend a well-known community and further recommend
disabling RFC 1966 "suppression" on the RR if the BGP community
is present in order to save configuration overhead on the PEs is
going a bit overboard.

-danny

Begin forwarded message:

> From: [email-masked]
> Date: April 25, 2008 3:30:01 PM MDT
> To: [email-masked]
> Subject: I-D ACTION:draft-pmohapat-idr-acceptown-community-01.txt
> Reply-To: [email-masked]
>
> A New Internet-Draft is available from the on-line Internet-Drafts
> directories.
>
>
> 	Title		: BGP ACCEPT_OWN Well-known Community Attribute
> 	Author(s)	: J. Uttaro, P. Mohapatra, D. Smith, R. Raszuk, J. Scudder
> 	Filename	: draft-pmohapat-idr-acceptown-community-01.txt
> 	Pages		: 8
> 	Date		: 2008-4-25
> 	
> It may be useful for a BGP speaker in an autonomous system to receive
>   and accept its own advertised route from a route reflector with more
>   fine-grained route control. For example, the route reflector can
>   change certain attributes of a route as desired, and then re-
>   advertise it back to the originator. Though it is possible to  
> perform
>   such policy control directly at the originator, it may be
>   operationally cumbersome in a network with a large number of border
>   routers having complex BGP policies.
>
>   This draft defines a new and well-known BGP community value,
>   ACCEPT_OWN, that signals a BGP speaker to accept an UPDATE message
>   and process the associated routes even when the ORIGINATOR_ID or the
>   NEXT_HOP matches that of the receiving speaker.
>
> A URL for this Internet-Draft is:
> http://www.ietf.org/internet-drafts/draft-pmohapat-idr-acceptown-community-01.txt
>
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>
> Below is the data which will enable a MIME compliant mail reader
> implementation to automatically retrieve the ASCII version of the
> Internet-Draft.
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: "Ilya Varlashkin" <[email-masked]>
Date: Wed, 30 Apr 2008 11:26:08 +0200

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Danny McPherson
> Sent: Wednesday, April 30, 2008 6:39 AM
> To: idr idr
> Subject: [Idr] Fwd: I-D 
> ACTION:draft-pmohapat-idr-acceptown-community-01.txt
> 
> 	
> In short, I think the is a really bad idea.  It's bad enough 
> the route reflection spec was changed from 1966 to 2796 to 
> permit an RR to reflect routes to a client even if they were 
> learned from that client - arguably, to enable an 
> implementation optimization, but for this to recommend a 
> well-known community and further recommend disabling RFC 1966 
> "suppression" on the RR if the BGP community is present in 
> order to save configuration overhead on the PEs is going a 
> bit overboard.
> 

I share Danny's view regarding this proposal. To me it seems like
ACCEPT_OWN doesn't add anything what is not possible today, while
potentially creates opportunities for new bugs in the heart of BGP route
handling both on RR and edge routers; as well as, from operational
prospective, undermining stable operations of the route reflector(s). 

Often operations of the core network and the edge are split between
different teams, and route reflectors are often operated by the core
team who needs to ensure that network runs fine as whole, once setup
RR's are usually left intact. ACCEPT_OWN implies that policies regarding
inter-VPN route import/export will shift from the edge to the core,
while still being operated by edge/access team. Due to misconfiguration
on RR during manipulation of inter-VPN policies, there will be risk to
affect routing stability of the whole network, which is bad idea.

Usually RR's do not have specific VPN's defined and just relay routes
based on generic BGP rules. ACCEPT_OWN will require RR to be aware at
least of some VPN's (many in fact, else it doesn't make sense). This
will require extra work on RR part, extra data structures, code, meaning
more opportunities for things to go wrong.

Next, it's quite possible that for the path optimisations BGP clients
will peer not only with RR, but also between themselves and
client-to-client reflection is disabled. This will break integrity of
the routing information within AS since RR will send modified updates
only to non-clients, while clients within the cluster will still have
original unmodified information about given prefix.

Last, same functionality as offered by ACCEPT_OWN is reasonably easy
implemented on PE via generic policies that are setup once VRF
provisioned. Complexity that is necessary to mark prefixes with
ACCEPT_OWN community is on par with complexity necessary to tag prefixes
with something else and use generic policies locally on PE. So at the
end of the day, moving policies to RR isn't magic bullet that will
eliminate need for any work on PE, and on the other hand efficient
generic policies can make PE configuration as easy as it would be done
on RR.

Kind regards,
iLya

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Apr 2008 11:30:01 -0400

Danny:

I get that you have a bad feeling about the idea.  Can you provide  
some specifics, though?  As far as I can tell, the proposed extension  
is safe (for the same value of "safe" as the rest of the  
protocol...).  See also my comments below regarding defaulting it off.

Ilya:

The main point of your comments seems to be "I would never configure  
my network that way".  You also make some assumptions about the  
organizational structure of the network operator.  Of course, nobody  
would ever try to force you to adopt this model.  The question is why  
you think others shouldn't be allowed to configure their networks that  
way.

With regard to some of your other points, I think the proposed feature  
should be only be enabled if configured, i.e. the default should be  
the current behavior.  The draft should be updated to say this  
explicitly.  As long as it defaults to off, usage of the feature  
becomes a "consenting adults" thing.  As for your observations  
regarding specific deployments (e.g., disabled client-to-client  
reflection) I think it's reasonable to ask that the network operator  
know what they're doing before enabling the feature and will avoid  
using configurations which are broken!  This is no different than the  
situation today, since the possibility to configure the protocol in  
broken ways has always existed, nor is it unique to BGP.

> more opportunities for things to go wrong.

In fact, very little new RR code would be required to take advantage  
of the feature.  The RR needs to be able to match on an RT, add an RT,  
and add the well-known community.  Matching and adding RTs (which are  
just extended communities) and adding other communities is existing  
functionality.  This requires additional configuration, but little or  
no new code.

Regards,

--John

---

From: "Ilya Varlashkin" <[email-masked]>
Date: Wed, 30 Apr 2008 17:54:44 +0200

John, 

> you think others shouldn't be allowed to configure their 

I do not worry that someone will force me, nor do I want to tell others
how to configure and operate their network. But adding this specific
functionality will require changes in one of the core BGP parts. 

> becomes a "consenting adults" thing.  As for your observations  

Even if this feature is off by default, it's still addition of the new
code to the core part of BGP. So there is risk that things will go wrong
with implementing the code (I do not worry about enabling feature since
I don't have to enable it) and this may destabilise current BGP
functionality. Since hardly any vendor will ship two versions of
router's software - one with and one without new code, it's obvious that
one day even networks that don't need this functionality will have to
become field testers of the new BGP code, and this is exactly part I
worry about.

> of the feature.  The RR needs to be able to match on an RT, 

Ok I see you point for this one.

Kind regards,
iLya

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 10:06:41 -0600

> protocol...).  See also my comments below regarding defaulting it off.

Take, for example, the case where the client doesn't know how
to handle the new community and some routing information loop
results.

I saw this occur in a real network when vendor 'n' changed their
RR behavior to that of 2796 and reflected routes back to clients
while the clients, vendor 'm', didn't know they were supposed to
drop the routes.

Additional configuration would be required to avoid the above,
and even more care would now be required to setup congruent
or overlay BGP topologies - while localized configuration is what
you're trying to avoid on the PEs in the application provided.

I suspect there are other scenarios where this could occur,
in particular if hierarchal RRs are used, etc..,

> functionality.  This requires additional configuration, but little or

Right, it's the client that needs most of the code..  Of course,
more than likely, the clients and the RRs will be on the same
code base - unless there are address family overlays constructed,
and then we're adding one more variable to the mis-configuration
opportunity mix, when some clients are older and can't support
newer code, then I've got a mix of topology and behavior types
that didn't previously exist.

How much configuration overhead is actually being saved
on the PEs that justifies changing basic BGP specification
behavior and introducing even more redundant network
state?

-danny

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Apr 2008 13:45:53 -0400

> to handle the new community and some routing information loop

The nice thing is that the feature is backward compatible and fails  
safe -- if the client doesn't know how to handle the community, then  
it sees itself in the ORIGINATOR_ID and drops the route.  No route, no  
routing information loop.  Granted the behavior of dropping the route  
is undesired, but not horrible.

> while the clients, vendor 'm', didn't know they were supposed to

Interesting -- one would have expected the vendor 'm' router to drop  
the routes even if it didn't understand ORIGINATOR_ID, due to the  
NEXT_HOP check.  I guess if the NEXT_HOP were getting rewritten...

But in any event this particular case is academic because of the "fail  
safe" behavior discussed above -- any client that interoperates with a  
2796 RR will fail safe.  (Any client that doesn't interoperate with a  
2796 RR is arguably pretty broken at this point.)

> behavior

One of my co-authors may want to address this point.  From my PoV,  
suffice it to say there's real demand for the feature.

> and introducing even more redundant network
> state?

What is the redundant network state you're referring to?  The "address  
family overlays" I snipped out of the quote?  Something else?

--John

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 14:12:18 -0600

> is undesired, but not horrible.

Even your draft disagrees with this, else it wouldn't mention
the disabling RFC 1966 behavior (as opposed to the discard
based on ORIGINATOR_ID on the client that RFC 2796
introduced -  for no valid _protocol reason, mind you, which
annoys me considerably).  Along this vein, I would be quite
interested in someone providing an explanation of why this
behavior was changed from 1966 to 2796....

> 2796 RR is arguably pretty broken at this point.)

Right, but see my point above...  This assumes all clients employ
the discard behavior of 2796.

> suffice it to say there's real demand for the feature.

I'm sure there is demand to make 2547 configuration simpler, but
that doesn't mean it's something we should change fundamental
BGP behavior to facilitate.  I would be interested in here from your
co-authors..

> family overlays" I snipped out of the quote?

No, I've come to accept the AF overlays..   What troubles me is the fact
that we're using BGP to enable a RR to modify an update learned from
a client, and to reflect that update back to that client - just does  
seem
good routing protocol hygiene to me, to save configuration overhead
on the client, when it'll surely be further required on the RRs.

-danny

---

From: Enke Chen <[email-masked]>
Date: Wed, 30 Apr 2008 13:37:43 -0700

Hi, Danny:

Regarding the specific change from 1966 to RFC 2796 that you mentioned, 
the reason was to make the spec. be consistent with the deployment 
(which has the behavior since Day 1 of the RR era).   AFAIK, there is 
nothing unusual about it in the iterative process for a protocol, and it 
is not alone. Take RFC 4271 as an example, one of the changes is the 
following:

     UPDATE Message Error subcode 7 (AS Routing Loop) has been deprecated.

The change reflects deployment and allows for implementation flexibility.

-- Enke

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 14:40:05 -0600

On Apr 30, 2008, at 2:37 PM, Enke Chen wrote:

> Hi, Danny:
>
> Regarding the specific change from 1966 to RFC 2796 that you  
> mentioned, the reason was to make the spec. be consistent with the  
> deployment (which has the behavior since Day 1 of the RR era).

Not all implementations, else I'd have not seen conflicts ~10
years ago between two of the most prominent implementations.

> AFAIK, there is nothing unusual about it in the iterative process  
> for a protocol, and it is not alone. Take RFC 4271 as an example,  
> one of the changes is the following:
>

Flexibility?  That's a gratuitous choice of words...

-danny

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Apr 2008 16:44:45 -0400

On Apr 30, 2008, at 4:12 PM, Danny McPherson wrote:
> the disabling RFC 1966 behavior

Not so.  The reason for that is to allow RR hierarchy to be used.   
Since the point of the draft is that the route's originator has to see  
it, it's important that RR's transparently pass it through.  I'm at a  
loss to see how you can interpret this as a tacit disagreement with my  
statement above.

> (as opposed to the discard

I suggest you take it up with that RFC's authors or check the archives  
-- I don't know the answer.  (I would say that implementability can be  
a "valid_protocol reason" though.)

> the discard behavior of 2796.

Well, that OR they properly implement the base spec and drop routes  
which have themselves as NEXT_HOP.

Do you really think that a client which doesn't interoperate with 2796  
RRs is viable in a network at all?  Especially one which also violates  
the base spec by not checking the NEXT_HOP?

> BGP behavior to facilitate.  I would be interested in here from your

I'm sure one of them will chime in.

> on the client, when it'll surely be further required on the RRs.

OK, I don't see where this is "redundant network state" to speak of.   
Well, I guess for each route treated in this way there's one extra  
route exported from the RR to the route's originator -- but then again  
that is a route which the RR has to export to the rest of the network  
anyway.  So the RR does in effect no extra work.  The PE receiving the  
route does store a little extra stuff, but then again there would be  
some state for that route if it was locally exported between VRFs  
too.  Seems to me as though state -- both dynamic and configuration --  
is more-or-less a wash.

--John

---

From: Enke Chen <[email-masked]>
Date: Wed, 30 Apr 2008 13:49:11 -0700

> years ago between two of the most prominent implementations.

So would you prefer declaring the major deployment as "illegal", and 
somehow having all the deployment rolled back?

>
>> AFAIK, there is nothing unusual about it in the iterative process for 
>> a protocol, and it is not alone. Take RFC 4271 as an example, one of 
>> the changes is the following:
>>

It's meant for the "AS Routing loop", although it equally applies to the 
RR case.

-- Enke

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 14:53:00 -0600

> somehow having all the deployment rolled back?

Sure..  I don't like the idea of changing specs to accommodate
implementation optimizations.

> It's meant for the "AS Routing loop", although it equally applies to  

No, that to accommodate use cases that are themselves
arguably hacks.  It wasn't meant to allow AN implementation
to be optimized.

-danny

---

From: Enke Chen <[email-masked]>
Date: Wed, 30 Apr 2008 13:56:28 -0700

> implementation optimizations.

welcome to the real world :-)

-- Enke

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 15:19:31 -0600

> loss to see how you can interpret this as a tacit disagreement with my

Then why the 1966 text at all, that was my point.

> a "valid_protocol reason" though.)

Heh, that's the high road :-)

> the base spec by not checking the NEXT_HOP?

I've seen it before in a real network.

> too.  Seems to me as though state -- both dynamic and configuration --

Well, upon thinking about this a bit more, it also seems silly to
me that a network would be architected in such a way as to allow
problems with an 'upstream' RR to affect connectivity between
two VRFs on the same PE.  I'd think to an operator trying to ensure
availability would be worth the configuration overhead, even if
some folks consider the extra 'state' a wash.

-danny

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 15:31:28 -0600

> welcome to the real world :-)

I do understand the reasoning behind this, although I
don't agree with it.

In the real world, operators are complaining about DFZ
sizes (unique routes), and routing scalability, and churn,
little implementation tweaks that surface as base spec
changes like this have serious implications.

Consider this 'optimization', for example.  Now, most
clusters have at least 2-3 RRs, and many clients.  If you've
got a single client that has 50k external paths that it sees
as best, and that client advertises them to the RRs, and
the RRs reflect them back, just so that the client can discard
them, then unless I'm confused, that little optimization just cost
you 150k (50k routes * 3 RRs, reflected back to client they
were learned from) worth update processing resources on
the clients.  Not to mention implications on churn, or additional
layers of RR hierarchy, or the dynamics this introduces in a
REAL network.

 From a network-level perspective I don't see much of an
optimization at all, quite the contrary, actually.

-danny

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Apr 2008 18:03:05 -0400

> I've seen it before in a real network.

And did you think that client was behaving appropriately then?  If so,  
why did you think it was OK for the client to violate the base spec?   
Also, you mentioned that your "real network" example was from ~10  
years ago, so while it's interesting... do you think such clients are  
still likely to exist *today*?

> some folks consider the extra 'state' a wash.

I guess different operators have different views about where in the  
optimization space they want to live.  For that matter, if the  
(redundant pair of) RRs goes south, the PE's connectivity to the rest  
of the world is broken anyway, local connectivity is just a special  
case.  I can understand why the survival of that special case when the  
rest of the world is broken may not be seen as a deal-breaker.

--John

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Apr 2008 15:04:33 -0700

Hi Danny,

In your below mail you are missing one fundamental practical data.

It is very cheap and easy to drop the unneeded route on inbound while it 
is very CPU intensive to build separate updates to each peer.

With current link bandwith the amount of traffic generated due to that 
is just white noise.

IMHO based on the most common implementations at that time that was the 
main reason for the RR spec change.

**

Now back to accept-own community I really see nothing wrong with it. If 
you are trying to say this is bad just because some implementations to 
not support 2796 that's I am afraid wrong avenue.

There are practical applications for this enhancement of which one is 
described in the draft.

As general rule of thumb I think there is need to support more 
automation in the network provisioning among many providers. There is 
also a demand in the market for more dynamic behavior of running 
applications. This is just right on this spot attempt to support one of 
them.

Cheers,
R.

PS. As to the comment from Ilya that this draft may break BGP 
implementations ... one needs to realize that conditional one more check 
during inbound processing requires negligible amount of new BGP code.

And I would like to point out that in most BGP implementations I am 
familiar with every month amount of code changes which go in even in the 
fundamental parts of BGP assuming one may freeze all IDR work is much 
much much higher then such draft would require.

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 16:27:44 -0600

On Apr 30, 2008, at 4:03 PM, John G. Scudder wrote:
> And did you think that client was behaving appropriately then?

No, neither did your current employer.  But the other vendor
pointed to the spec changes and said they were compliant and
the client needs to know it's a client and needs to drop the
extraneous updates.

I'd be keen on revamping 2796 again to remove that local
optimization, personally.  I'm no fan of expanding upon it.

> If so,

Depends on whether I think the spec change was
appropriate or not.  Again, local optimizations that
result in much more system-wide state I'm not a big
fan of.

> (redundant pair of)

If they're redundant, then there's dual the amount of
extraneous state, and they may still share a common
switched link layer or the like.

> RRs goes south, the PE's connectivity to the rest

Perhaps.  I wouldn't buy such a service from an operator
that designed their network in this manner, you might (although
I doubt it :-), but fair enough.

I believe folks that are implementing this may want to help
their customers understand this case, and I think the
draft would benefit from it's discussion as well.

-danny

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 16:39:08 -0600

> In your below mail you are missing one fundamental practical data.

No, I'm not missing this, I understand it, I know why the spec
change was made.

> is very CPU intensive to build separate updates to each peer.

Really?  Where's the line drawn?  What about when there are
one million updates being dropped?  I could probably make a
case for this today?  What about legit updates queued behind
these?  I don't believe introducing extraneous routing system
level state to allow more efficient update generation is a valid
excuse.

> With current link bandwith the amount of traffic generated due to that

Traffic wasn't the issue.  It's more CPU, memory, etc..  Just
because it's available doesn't mean it's free.

> main reason for the RR spec change.

"Implementations" should be "implementation" there, not the
plural.

> not support 2796 that's I am afraid wrong avenue.

Perhaps, if you're in the business of selling CPUs and memory.

> There are practical applications for this enhancement of which one is

Yes, one that introduces even local PE VRF-VRF connectivity
reliance on upstream RRs..

> applications. This is just right on this spot attempt to support one  

Perhaps, but I think it ugly because of both clean routing hygiene
issues, and the network architectural fault considerations I've already
outlined.

> during inbound processing requires negligible amount of new BGP code.

Negligible, can you quantify that?  How many routes?  How much
churn?

> much much higher then such draft would require.

I don't understand what point you're making here.

-danny

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Apr 2008 18:52:22 -0400

> because it's available doesn't mean it's free.

Regarding memory, it needn't be consumed for a this-is-my-own-route  
path.  (It may be, but this is an implementation decision.)

Regarding CPU, remember that you want to optimize the bottleneck,  
nothing else matters really.  In most cases, the RR is the control  
plane bottleneck.  Loading it up more to protect the CPUs of non- 
bottleneck devices seems like an odd design decision.

> "Implementations" should be "implementation" there, not the

FWIW I'm aware of at least three implementations that independently  
arrived at this same "optimization".

> Perhaps, if you're in the business of selling CPUs and memory.

I don't really see the connection.

> reliance on upstream RRs..

As we've discussed already.  I don't think this point is any different?

--John

---

From: Brian Dickson <[email-masked]>
Date: Wed, 30 Apr 2008 19:20:33 -0400

> path.  (It may be, but this is an implementation decision.)

You need to be careful about steady-state memory usage vs 
high-water-mark memory usage.
Processing updates chews memory *prior* to installation of entries in 
various RIBs/FIBs, and
unless you have an excess of CPU power (and can handle line-rate packets 
in the slow path),
it isn't possible to feasibly pre-filter without degrading performance.

And not pre-filtering means memory gets hit on the transient prior to 
discarding this-is-my-own-route.

> bottleneck devices seems like an odd design decision.

The red flag has gone up. Using the term "*the* bottleneck" means you 
don't understand the nature
of complex systems. Yes, there will *always* be a bottleneck. However, 
solving for one bottleneck
will *always* move the bottleneck to some other place. It may be 
acceptable for a stable state with
several equally-pernicious bottlenecks affecting the system, or to move 
the bottleneck to a portion of
the solution space that scales best (e.g. one that scales as O(n log n) 
vs one that scales as O(n^2).

The two other points I'd like to make, are, that:
(1) saying "in most cases" is the same as saying "it's not always the 
case". Solving for one case without
considering other cases is very short-sighted and can cause unforseen 
consequences. It is much better
to *consider* those cases, explicitly, and discuss whether and to what 
degree they would get affected; and

(2) keep in mind - an RR is *strictly* control plane. It is possible for 
it to reside on a device that also does
data plane stuff. However, if RR performance is a huge deal, throwing 
*computer* hardware at it, rather
than *router* hardware at it, is an entirely feasible solution - with 
likely higher (by orders of magnitude)
ceiling on performance that can be bought, and likely at much lower 
price point than equivalent router vendor
RR solutions. RR can achieve equivalent topological solutions (and thus 
avoid topological variance versus
router-based RR's) by sitting as a one-armed device hanging off a core 
router.

There is something to very much consider, however:
RR's are fundamentally required to achieve suitable scaling on big 
non-VPN networks.
Please keep those networks in mind when messing about with core BGP 
protocols.

(Never mind that I'm working on something to *really* mess with those 
same core BGP protocols... but with good reason. :-) )

Brian Dickson

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 30 Apr 2008 16:30:33 -0700

Hi Danny,

>  > It is very cheap and easy to drop the unneeded route on inbound 
>  > while it is very CPU intensive to build separate updates to each peer.
> 
> these?

Let me address your concerns one by one ...

* The line is drawn when you have 100s of PEs to send the same 
information to.

* The milion updates to be dropped .... Clearly you must be talking 
about VPN routes. If so there are hardly single PE with milion routes to 
  make such a case today. Total number of routes yes but this draft is 
orthogonal to this. You are complaining about the 2796 case again here.

* Each PE can today receive only legit updates with or without of this 
draft including RRs support for 2796. There are well know standard based 
technics for this example: rfc4684.

Can you tell us how your argument holds when you support rfc4684 ?

> because it's available doesn't mean it's free.

Memory is just a transient memory. If your CPU suffers with inbound 
filtering I think there are much bigger issues behind the scenes.

Keep in mind that discussed here draft does not increase number of 
routes each PE would receive even by a single one. All of your points 
Danny flame the issue of reflecting back the information to the client 
which were originated by it before. Perhaps this is a valid theoretical 
complain but it is just out of this topic. And as said before this is 
all for VPN space where this problem has already been addressed by rfc4684.

> reliance on upstream RRs..

You are missing the big picture :) This is to provide functional 
consistency ... RR will propagate the same very update to all PE - which 
very often are in hundreds or maybe thousands of today's large 
providers. Only one of them originated the route .. appending this new 
community will not have any impact in all of the PEs .. only in the 
originator which will accept the route and perform required action which 
will be consistent with all other PEs in the network.

> issues, and the network architectural fault considerations I've already

Recently we have had hussars bashing "BGP overload" ... now in 2008 
perhaps we see a new trend "routing hygiene". So can you spell what is 
so unhygienic in this draft ? All points you have brought so far talk 
about solved issue (in the VPN space) if reflecting unneeded routes to 
PEs ... but non of your points refer to this draft. Can you put in 
points how this draft make this any worse ?

Cheers,
R.

---

From: Danny McPherson <[email-masked]>
Date: Wed, 30 Apr 2008 17:52:42 -0600

Regarding all your other text, I don't have any new points
to make and haven't changed my position, so I'll avoid
further discussion on that.

> points how this draft make this any worse ?

It's "be liberal in what you accept, conservative in what you
send", NOT "be liberal in what you send because others
are liberal in what they accept" :-)

I understand the utility of your proposal, I understand why
you selected the mechanism you did.  My concern is that all
these tweaks have system-wide implications, and we all
need to be cognizant of that when proposing solutions to
problems operators have.

-danny

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 30 Apr 2008 21:15:07 -0400

I've changed the subject line since we have drifted far afield from  
discussion of draft-pmohapat-idr-acceptown-community-01.txt.

On Apr 30, 2008, at 7:20 PM, Brian Dickson wrote:
>> Regarding memory, it needn't be consumed for a this-is-my-own- 
>> route  path.  (It may be, but this is an implementation decision.)
>
> You need to be careful about steady-state memory usage vs high-water- 
> mark memory usage.
> Processing updates chews memory *prior* to installation of entries  
> in various RIBs/FIBs, and
> unless you have an excess of CPU power (and can handle line-rate  
> packets in the slow path),
> And not pre-filtering means memory gets hit on the transient prior  
> to discarding this-is-my-own-route.

I have in-depth familiarity with several well-known BGP  
implementations, and I haven't the slightest idea what you're talking  
about.

> The red flag has gone up. Using the term "*the* bottleneck" means  
> you don't understand the nature
> of complex systems.

Tempting though it may be to pick up the gauntlet and enter into a  
full scale flame war, I hope you won't mind if I try to confine myself  
to the substance of your remarks rather than their tone.

> Yes, there will *always* be a bottleneck. However, solving for one  
> bottleneck
> several equally-pernicious bottlenecks affecting the system, or to  
> move the bottleneck to a portion of
> the solution space that scales best (e.g. one that scales as O(n log  
> n) vs one that scales as O(n^2).

Yes, this is algorithms 101.  I was making a rather simple point  
regarding networks as systems, viz the fact that if you've got one RR  
feeding a large number of clients, the RR usually needs to process a  
lot more stuff than the clients do.  It has N >> 1 peers (the clients  
and non-client peers); they have M << N peers (the RR, likely a  
redundant RR, and probably a smaller number of external peers).   
Granted that this isn't universally true, but one has to design to  
something and this would seem to be the common case.

It's rather grandiose to describe the issue under discussion (which,  
I'll remind you, is a tangent having to do with route reflection and  
the differences between RFC 1966 and RFC 2796) as an attempt to "solve  
for [the] bottleneck".  While a "solution" would be nice, I think most  
would settle for just increasing the bottleneck's diameter. :-)  In  
any event, this is water which ran under the bridge a long time ago.

> (1) saying "in most cases" is the same as saying "it's not always  
> the case".

Well, yeah.

> Solving for one case without
> considering other cases is very short-sighted and can cause  
> unforseen consequences.

Thank you for this insight, I will strive to bear it in mind.

> It is much better
> to *consider* those cases, explicitly, and discuss whether and to  
> what degree they would get affected; and

Please feel free to redesign route reflection to be optimal in all  
cases if you have the time to spare!  Absent a really interesting  
point being raised, I'll probably refrain from further discussion of  
this topic myself though, since as I mentioned, this is water under  
the bridge.

> (2) keep in mind - an RR is *strictly* control plane. It is possible  
> for it to reside on a device that also does
> data plane stuff. However, if RR performance is a huge deal,  
> throwing *computer* hardware at it, rather
> than *router* hardware at it, is an entirely feasible solution -  
> with likely higher (by orders of magnitude)
> RR solutions. RR can achieve equivalent topological solutions (and  
> thus avoid topological variance versus
> router-based RR's) by sitting as a one-armed device hanging off a  
> core router.

OK.

> RR's are fundamentally required

If you are going to insist on precision in the use of language, you  
should be careful with expressions like "fundamentally required".

> to achieve suitable scaling on big non-VPN networks.

I have some familiarity with the networks you are speaking of.  And to  
repeat one more time lest the confusion continue, the draft originally  
under discussion isn't about route reflection other than tangentially.

> (Never mind that I'm working on something to *really* mess with  
> those same core BGP protocols...

Yes, the first draft was interesting and I look forward to seeing the  
next draft.

> but with good reason. :-) )

That remains to be proven though it's an intriguing hypothesis.

--John

---

From: Brian Dickson <[email-masked]>
Date: Wed, 30 Apr 2008 22:52:30 -0400

> to the substance of your remarks rather than their tone.

Thanks. My comment was using "you" in the sense of "one", rather than 
you personally. No flame war was intended. :-)

> Yes, this is algorithms 101.

I wish it *were* 101, sadly, I suspect it is more along the lines of 
601, i.e. second year grad level, based on the frequency with which I 
see poorly designed (rather than merely poorly implemented) systems.

> I was making a rather simple point regarding networks as systems, viz 
> the fact that if you've got one RR feeding a large number of clients, 
> the RR usually needs to process a lot more stuff than the clients do.  
> It has N >> 1 peers (the clients and non-client peers); they have M << 
> N peers (the RR, likely a redundant RR, and probably a smaller number 
> of external peers).  Granted that this isn't universally true, but one 
> has to design to something and this would seem to be the common case.

There's more than one way to improve the performance and scalability of 
RR's.

For instance, in most of the networks I've built and run, and many 
others that I'm quite familiar with, there wasn't client-client 
reflection - for both performance concerns, as well as for the 
topological aspects, where in order to guarantee the RR makes a suitably 
sensible choice, it needs to also be in the data path.
If you have clients that see each other directly, and that bypass the 
box that is the RR, then client-client reflection can do seriously weird 
stuff even when you know what you're doing.

And, it also matters where you do policy stuff. If you put your policy 
stuff on the edges of your AS, then the RR's can be almost trivial, 
without touching updates beyond adding/changing the RR-specific bits, 
and possibly seeing further optimizations based on peer-groups.

My experience has been that for similar-class boxes acting as core and 
border routers, and core routers doing double duty as RRs, that you get 
the best bang for the buck when N is about the same as M. And not just 
because of RR scaling issues, but also because of the proportion of 
revenue-generating ports and boxes versus overhead ports and boxes.

> this topic myself though, since as I mentioned, this is water under 

Ha ha ha! :-)

(I actually did start looking into RR design alternatives, and gained a 
real respect for the original authors of BGP. The implementation is 
easy, only because the underlying design got things right. Any deviation 
from the basics on how BGP works, gets really hairy really fast.)

> should be careful with expressions like "fundamentally required".

I am (careful, that is).

For IBGP, there are obvious points at which the operational cost and 
overhead of having a full mesh, becomes excessive.
Without RRs, there are O(n^2) peering sessions, all of which are 
required to be properly configured to guarantee things work properly.
For a border router with (worst case) a single backhaul link to the rest 
of the AS, for every inbound EBGP update, there need to be N IBGP updates.
At the observed rate of thrash, for any given pipe size there is a 
corresponding value N at which updates more than fill the pipe, leading 
to congestive network collapse.

And, regardless of network size and thrash rate, as long as the pipe 
size is big enough to handle a single update stream without regularly 
congesting, an RR topology can be built to handle the network size and 
all updates.

That's what I mean when I say "fundamentally required" - that it is both 
necessary and sufficient, for arbitrarily large networks, to have RRs 
deployed to avoid congestive collapse.

But, I'm positive you already know all of what I just wrote.
(The words were for the benefit of anyone else that might wonder what I 
meant by "fundamentally required".)

>> same core BGP protocols...
>

Thanks. (I've been trying to get permission from the authors of the 
"add-paths" draft to re-use it and augment it.)

> That remains to be proven though it's an intriguing hypothesis.

I agree, and I hope to be able to produce convincing demonstrations of 
it, in due time.

I realize that until something of substance is available, it's academic; 
it took a bit of work to convince myself that it worked.

Brian