Subject: [Idr] Re: BGP issues

From: Robert Raszuk <[email-masked]>
Date: Wed, 09 Jan 2008 05:02:39 -0800

Hi Iljitsch,

As we are discussing BGP i am redirecting this to idr list for IDR WG to 
comment or work on.

>> But I would like to also see the real list of what are the issue in 
>> BGP we are trying to address ... Is this point-to-point model, is this 
>> TCP transport, is this update format ... No one yet stated what is 
>> broken. And when we do not know what is broken .. or which link of BGP 
>> chain is fragile it is quite a challenge to design new thing around 
>> the unknown.
> 
> In addition to the fact that it's possible to configure iBGP such that 
> you can have routing loops?

Could you provide an example ?

> it suffers from some of the problems inherent to distance vector
> routing, such as slow convergence

BGP can converge today in subseconds. More over BGP routes with correct 
implementation may converge in prefix independent manner. It is not that 
BGP is slow. The vast majority of end to end convergence is a product of 
failure localization, failure detection time and failure propagation 
time. BGP can be made even today to propagate more then one path within 
given AS when best external functionality is in place. Ask your 
favourite vendor for details. More over there are also proposals 
discussed at IDR to send more then best path in BGP.

 > Another problem area for
> BGP is the fact that all processing happens on a per-prefix basis: there
> is no way to communicate reachability or policy changes except to update
> all impacted prefixes. 

Not true. There is proposal for BGP aggregate withdraw. Prefixes can be 
aggregated itself by it's mask. There ways to distribute policies 
without attaching them to the prefixes (example: rt-constrain).

> BGP is extremely agnostic as to the underlying
> path selection algorithm in order to accommodate as much policy control
> as possible.

Accommodating policies is not a BGP protocol requirement but it comes 
from operators using this protocol to fit their operational needs.

> Unfortunately, this makes it very hard to predict BGP's
> behavior and the default behavior (especially with today's rather flat
> AS hierarchy) is more often than not suboptimal. BGP allows harmful
> policies that keep the protocol from converging to a stable state.

Again this is not BGP which allows to configure harmful policies. 
Policies can be harmful indeed as there is no mechanism or in any way 
control on them. If in IGP you block LSA flooding on all links of the 
other routers in IGP peers would you call this an OSPF fault that your 
router is somehow not seen by the rest of them ?

 > Lack
> of workable aggregation mechanisms means that once an address block is
> deaggregated, it's almost impossible to get rid of the resulting long
> prefixes, leading to excessive growth of the internet's global routing
> table. 

Correct. I think we agree that scalable multi-homing should be solved. 
Perhaps a way to do that is to use PI address spaces, perhaps shim6, or 
berhaps just to decouple sending all in one flat BGP as is today. 
Protocol IMHO is ready for 2 tier hierarchy with no protocol changes 
required.

> Coarseness of the only available end-to-end metric (the AS path)
> pushes operators to deaggregation for traffic engineering purposes. 

What other metrics would you recommend to add to BGP and how that would 
that reduce deaggregation ?

> The
> way BGP operates within a single AS requires an additional intra-domain
> routing protocol 

IMHO this is a feature not a bug. In fact for convergence reasons this 
is a very useful feature to have fast IGP.

> and suboptimal engineering tradeoffs by requiring
> having a full mesh between all BGP routers within the AS or having route
> reflectors or a confederation. 

That problem has been solved today. Within any domain you can run 
encpasulation of your choice between ASBRs and non of the routers in the 
core need to have any BGP routes.

> There is no validation of routing
> information beyond the next hop. 

Are you alluding to S-BGP/so-BGP proposals ? If not what other 
validation would be required ?

> A BGP speaker only communicates its
> best path (if any) to a neighbor, with no way to tie additional
> information to the nonexistence of a path and no way to accomplish type
> of service routing or install backup paths. 

Discussed that one above.

 > Paths must be explicitly
> revoked, which in practice requires a BGP speaker to keep track of which
> paths were communicated to which peer. 

Not today. Today you revoke the prefix not the path. When the ability to 
send more then best path is added true .. paths would have to be 
explicitly revoked. Any other proposal in RRG does the very same. 
Readvertisement of EID with new list of locators is an explicit 
modification.

> BGP requires fairly extensive
> configuration (setting up filters) before it's useful.

It could be just one line if you like to send/accept everything to/from 
your peer.

Cheers,
R.

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 11 Jan 2008 17:29:01 +0100

On 9 jan 2008, at 14:02, Robert Raszuk wrote:

> As we are discussing BGP i am redirecting this to idr list for IDR  
> WG to comment or work on.

Ah, I saw that you wrote a reply but I couldn't find it at first.  :-)

>> In addition to the fact that it's possible to configure iBGP such  
>> that you can have routing loops?

> Could you provide an example ?

R1e - R2i - R3i - R4e

If R1 and R4 both prefer their eBGP path towards a certain destination  
and then R2 prefers the path over R4 while R3 prefers the path over  
R1, packets will loop between R2 and R3. Note that this configuration  
only happens when applying policy to iBGP sessios or in unfortunate  
route reflector placement.

> BGP can converge today in subseconds.

Sure, but it can also take many minutes, especially with a min-route- 
advertisement timer in effect in many places and if the update is a  
withdrawl. Now I've argued that this shouldn't be too problematic  
because if you're revoking your advertisement that means you're  
unreachable anyway, but not everyone agrees with that. (Not good if  
you are revoking a more specific and you want the aggregate to attract  
the traffic.)

One reason that BGP is getting chattier is because updates tend to be  
multiplied if there are several interconnects between two ASes.

> Not true. There is proposal for BGP aggregate withdraw.

I'm talking about what's out there, not about what's proposed. Very  
little of what is proposed makes it to wide deployment.

> Accommodating policies is not a BGP protocol requirement but it  
> comes from operators using this protocol to fit their operational  
> needs.

Sure, but in my opinion, BGP takes it a bit too far, it's too easy to  
shoot yourself in the foot. Or, in other words: a routing protocol  
that allows loops is doing something wrong.

> Policies can be harmful indeed as there is no mechanism or in any  
> way control on them. If in IGP you block LSA flooding on all links  
> of the other routers in IGP peers would you call this an OSPF fault  
> that your router is somehow not seen by the rest of them ?

The difference is that in OSPF filtering LSAs is not supported by  
reasonable implementations because that is incompatible with the SPF  
architecture, while in BGP policies are fully supported by design,  
even some harmful ones.

> Correct. I think we agree that scalable multi-homing should be solved.

And scalable traffic engineering. Remember that at least 200000  
prefixes in the DFZ are NOT the result of multihoming using the one  
AS / one prefix model.

> What other metrics would you recommend to add to BGP and how that  
> would that reduce deaggregation ?

I would like to see a value that is increased by every AS (twice, for  
good measure: once on reception, once on transmission), but not by  
one, but by a value (say) between 1 and 127, with a default in the  
middle (11). So where you'd see an AS path length 3 today, you'd see a  
metric value 66 if nobody in the path changed the default. But if  
someone wants to make this path just a little better or a little  
worse, they can make it 65 or 67. This allows finer grained control  
that the AS path so it reduces the need for deaggregation for TE  
purposes.

> IMHO this is a feature not a bug. In fact for convergence reasons  
> this is a very useful feature to have fast IGP.

It also adds tons of complexity, especially if you don't want to have  
a full iBGP mesh. I'm not saying you shouldn't have an IGP, I'm saying  
you shouldn't _have_ to have an IGP.

> Are you alluding to S-BGP/so-BGP proposals ?

Yes, to their problem statement, at least. Not sure if drowning  
everything in sticky crypto syrup is the best solution, though.

> Not today. Today you revoke the prefix not the path.

Path in the sense of "a prefix learned over a specific eBGP session".

> When the ability to send more then best path is added true .. paths  
> would have to be explicitly revoked. Any other proposal in RRG does  
> the very same.

It would be better if implementations wouldn't have to keep track of  
what they sent where. However, this is probably relatively easy to fix  
("start now" .... "forget all the prefixes that I haven't refreshed  
since "start now"") but the fix will probably be hard to deploy.

> It could be just one line if you like to send/accept everything to/ 
> from your peer.

Yes, and I'm happy this setup is so easy to configure every single  
time I have any use for it.  </sarcasm>

Iljitsch

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 11 Jan 2008 08:57:53 -0800

Hi Iljitsch,

> and then R2 prefers the path over R4 while R3 prefers the path over R1, 
> packets will loop between R2 and R3. Note that this configuration only 
> happens when applying policy to iBGP sessios or in unfortunate route 
> reflector placement.

Altering best path selection on IBGP routers is a bad idea when you have 
hop by hop switching.

> Sure, but it can also take many minutes, especially with a 
> min-route-advertisement timer in effect in many places and if the update 
> is a withdrawl. Now I've argued that this shouldn't be too problematic 
> unreachable anyway, but not everyone agrees with that. (Not good if you 
> are revoking a more specific and you want the aggregate to attract the 
> traffic.)
> 

Just to clarify here .. what I call convergence is in fact reachability 
restoration. And that is in vast majority of cases just the thing within 
your AS or at most to your peering AS.

As long as reachability is restored (even if temporarily in an 
suboptimal fashion) does not impact user's traffic. Then BGP can take as 
long as it likes to "converge" to an optimal path as long as 
reachability is in place. That IMHO should be the focus.

> little of what is proposed makes it to wide deployment.

If you say that very little on what is proposed or what vendors ship 
makes it to the actual wide deployment what could we say about something 
which is not even proposed yet ? Should we sit and do nothing for years 
to come ???

> shoot yourself in the foot. Or, in other words: a routing protocol that 
> allows loops is doing something wrong.

See the problem is that loops can form with bad policies. And it would 
not be that hard to detect them algorithmically. The problem is that no 
operators wants to share his polices to others. So while you could quite 
easy come out with the intra-domain policy verifier .. doing the same 
inter-domain is a non technical challenge. I don't think BGP or any 
other protocol would do any magic if there is no data at hand to work on.

> good measure: once on reception, once on transmission), but not by one, 
> but by a value (say) between 1 and 127, with a default in the middle 
> (11). So where you'd see an AS path length 3 today, you'd see a metric 
> value 66 if nobody in the path changed the default. But if someone wants 
> to make this path just a little better or a little worse, they can make 
> it 65 or 67. This allows finer grained control that the AS path so it 
> reduces the need for deaggregation for TE purposes.

Hmmmm that's an idea ... Perhaps you should write a brief draft on it ?

> It also adds tons of complexity, especially if you don't want to have a 
> full iBGP mesh. I'm not saying you shouldn't have an IGP, I'm saying you 
> shouldn't _have_ to have an IGP.

IMHO recursive protocol should recurse via something. And for 
intra-domain reachability restoration bgp is not the best choice. I saw 
in the past some attempts to use iBGP as an IGP but it is IMHO a very 
bad idea limited to very few specific topologies.

The issue is that intra and inter domain requirements for routing 
protocol are very different and diverse. It is a challenge to come up 
with a huge protocol to do all .. as such would not do anything very 
good. On that grounds I do believe that we should have precise tools for 
  different jobs. For the same reason I have a big workshop in my garage 
not to have bend wires, cut wood or unscrew car bolts all with one set 
of pliers.

> since "start now"") but the fix will probably be hard to deploy.

Then you would loose the huge benefit of BGP to be an incremental protocol !

Cheers,
R.

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Fri, 11 Jan 2008 12:53:18 -0500

Hi Iljitsch,

Good points. However, we should not mix up 'what's deployed' with
'what's a allowed by the protocol'.

By all means, the BGP traffic engineering and BGP deaggregation points
are worthy of further discussion. 

> that the AS path so it reduces the need for deaggregation for TE  

I think that you are alluding to enabling each transient AS to attach
its relevant weight/preference for each BGP prefix. This furhter enables
each AS to get end-to-end metric for each prefix with or without the
AS_PATH. Correct?

While I agree that it may help wrt TE, but it doesn't fully solve the
deaggregation. For example, a network may advertise one chunk of
prefixes (bunch of /24s) to one AS and antoher chunk (bunch of /24s) to
another AS to receive the traffic accordingly from different Ases,
though the network could have advertised one aggregate /23 (say).

> everything in sticky crypto syrup is the best solution, though.

I agree with you to this point. Perhaps, you would prefer something like
the IRV -
http://www3.ietf.org/proceedings/02nov/slides/rpsec-6/index.html

Regarding other points, please see inline,

> >> that you can have routing loops?

Your example to create routing loop in BGP by (mis)using policies is
somewhat valid.  But, one can create loops by just configuring static
routes. It is quite subjective to characterize what's easy to shoot and
what's not.

BGP policies provide a toolkit to help making educated designs. Yes,
they are good if used appropriately, and bad, if not used appropriately.
That's why there are guidelines. The policies are meant to aid the one's
preference of tweaking the result of protocol at one's own expense. 

I think that the better question would be - should the protocol
claim/advertise the reachability for a certain prefix, when the
reachability may not exist in the forwarding plane? 

> architecture, while in BGP policies are fully supported by design,  

This is an apple to orange comparision. Firstly, the OSPF specification
(RFC2328 section 3) allows for unidentical link-state database in an AS
(due to introduction of areas).

Secondly, independent of filtering, one can infuse a policy to deny OSPF
to install a route in the routing table. So, while the neighbors can
send the traffic for that route, the router will either drop it or
forward it as per the less specific route. One may end up with loop.
Again, nothing to do with protocol itself.

> withdrawl. Now I've argued that this shouldn't be too problematic  

Well, the min-route-advertisement timer is meant to help in reacting to
every routing change faster or slower. Well, one may argue that it is
advantageous to tune the timer according to one's preference and one's
understanding of the network(s).

> a full iBGP mesh. I'm not saying you shouldn't have an IGP, 
> I'm saying you shouldn't _have_ to have an IGP.

Agreed. 
BGP mustn't mandate the usage of IGP, and it doesn't.
One can use static routes to reach the next-hops, and obviate IGP
altogether.
~~~~~~~~~~~~~

Cheers,
Rajiv

> -----Original Message-----
> From: Iljitsch van Beijnum [mailto:[email-masked]] 
> Sent: Friday, January 11, 2008 11:29 AM
> To: [email-masked]
> Cc: idr
> Subject: [Idr] Re: BGP issues
> 
> >> BGP is the fact that all processing happens on a 
> per-prefix basis:  
> >> there
> >> of workable aggregation mechanisms means that once an 
> address block  
> >> is
> >> prefixes, leading to excessive growth of the internet's global  
> >> routing table.
> 
> >> revoked, which in practice requires a BGP speaker to keep 
> track of  
> >> which

---

From: "John G. Scudder" <[email-masked]>
Date: Mon, 14 Jan 2008 15:54:55 -0500

On Jan 11, 2008, at 11:29 AM, Iljitsch van Beijnum wrote:
> However, this is probably relatively easy to fix ("start now" ....  
> "forget all the prefixes that I haven't refreshed since "start  
> now"") but the fix will probably be hard to deploy.

If you squint hard enough you will realize that this is available  
today under the name "graceful restart".

--John

---

From: Paul Jakma <[email-masked]>
Date: Thu, 17 Jan 2008 12:07:54 +0000 (GMT)

On Fri, 11 Jan 2008, Iljitsch van Beijnum wrote:

> I would like to see a value that is increased by every AS (twice, 
> for good measure: once on reception, once on transmission),

That's one to add to the "AS_PATH reform" list possibly (encoding 
AS_PATHs as {ASN,metric} has been proposed before).

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
Fortune:
Few things are harder to put up with than the annoyance of a good example.
 		-- "Mark Twain, Pudd'nhead Wilson's Calendar"