Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0 routes

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 03:18:40 +0200 (EET)

On Mon, 1 Mar 2004, Curtis Villamizar wrote:
> Yes I was following.  You just didn't read the paragraph that was
> immediately above this one, which you also conveniently chopped off.
> The paragraph you omitted said:
> 
>    Then the aggregate is locally generated and it is very easy to
>    exclude the parts of an aggregate that you generated but are not
>    covered by more specific from the reachability test and it makes
>    perfectly good sense to do so.  If other prefixes in the AS match
>    that aggregate then the aggregate is generated but locally it is
>    installed as a discard route.
> 
> I'm agreeing with the original point just mentioning that the original
> point wasn't clear.

It was (or _is_) not clear whether you consider this to be the status 
quo or not.

My interpretation of the spec (and a vendor's, as well) is that this 
is not considered at all in the current spec.  And if we disagree, it 
calls for EVEN MORE LOUDLY for a clarification!

> > > No need to modify the spec.  If you've installed a special type of
> > > route in the kernel that is there explicitly to indicate
> > > non-reachability, then that route should not be allowed to indicate
> > > reachability.  I dont' think we really need to say that.
> > 
> > You would think so, but the current spec does not say so.  
> 
> The bgp3-23 test just mentions route.  If you interpret that to
> include discard routes you simply have a bug in your implementation.
> It is quite easy to just look for something in the FIB radix trie and
> if you find something declare that is satisfied the reachability
> criteria without looking at what it was you found (a discard route).

No -- the current spec is just looking up a route -- it takes no 
stance what kind of 'route' that is.  This needs to be specified.  

Of course, it'd be nice to fix the implementation I'm having problems 
with.  But being altruistic, and wanting to fix this problem for 
others as well, this needs to be specified, somehow, in the spec to 
avoid this issue being misinterpreted.

My assumption is that the implementation must work properly, if
implemented to the letter of the spec.  That's not the case now.  
Anything that requires the implementor to "use non-obvious
interpretation of the spec" is a potential missing (or badly written)
item from the spec.

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 03:18:40 +0200 (EET)

---

From: Pedro Roque Marques <[email-masked]>
Date: Mon, 1 Mar 2004 17:56:09 -0800 (PST)

Pekka Savola writes:

> is not considered at all in the current spec.  And if we disagree,
> it calls for EVEN MORE LOUDLY for a clarification!

[...]

> stance what kind of 'route' that is.

By your own admission the spec doesn't require clarification. It is
clear that it doesn't understand the next-hop type of a route.

> This needs to be specified.

What you mean by this is that you believe that the current behaviour
should be modified...

Even if we where to agree that the behaviour you and Curtis propose is
more desirable than the current one, it isn't feasible to modify the
behaviour of current systems.

The modification you propose also isn't suficient to address the
problem that you point out whenever your aggregate route has a
reachable next-hop.

Examples where this could happen:

- User wants to forward all traffic that hits the default route to a
remote box that analyzes traffic... i.e. 0/0 to an LSP to a traffic
collector.

- Network is divided in areas but has global IGP. Each area border
generates a summary of address in the are that is propagated across
the network. i.e. outside the area you see an aggregate via BGP and
more specific /32s.

  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Mon, 1 Mar 2004 17:56:09 -0800 (PST)

[...]

  Pedro.

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 06:15:51 +0200 (EET)

> clear that it doesn't understand the next-hop type of a route.

Some people seem to disagree.  I personally am of the opinion that the
spec can be read both ways, but *seems* to indicate that the intent is
to be "route -type agnostic".

> What you mean by this is that you believe that the current behaviour

Modified (if the WG thinks your interpretation is correct) is
preferable.  Clarified is required.

> problem that you point out whenever your aggregate route has a

As stated, that can be avoided by operational procedures (by putting
the discard routes in every box).  This really can't.

> remote box that analyzes traffic... i.e. 0/0 to an LSP to a traffic

Of course, this can happen.  Then the BGP nexthops matching the 
default remain 'resolvable'.  But that seems quite obvious: if you 
have a "working" default route, it will be considered by the 
resolvability process.

Of course, in the long run, there is a need to specify the 
resolvability in a more fine-grained manner, but the "discard route" 
scenario seems to be a non-obvious problem which must be fixed 
quickly.

I don't have many preferences on the way it's solved, as long as it
gets fixed soon.  And if fixing would take time, and it's for any
purpose delayed from the bgp4-xx spec for now, we must clearly specify
the expected behaviour in the spec.

> the network. i.e. outside the area you see an aggregate via BGP and

"Don't do that". :-)

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 06:15:51 +0200 (EET)

"Don't do that". :-)

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 02 Mar 2004 10:42:03 -0500

In message <[email-masked]>, Pedro Roque Mar
> remote box that analyzes traffic... i.e. 0/0 to an LSP to a traffic

That's a default route that forwards traffic, not a discard route.

> the network. i.e. outside the area you see an aggregate via BGP and

That route also forwards traffic.  It is not a discard route.  On the
aggregator itself, there had better be a more specific route or if not
the routes should be considered infeasible.

If the aggregator which also happens to be an EIBGP router where
confederations are used then the aggregator is also passing the BGP
routes from the border router that went down.  When the hole in the
aggregate occurs if the discard route is allowed to pass the
reachability test, then the BGP routes remain in the rest of the AS
until the BGP session times out - a very long time.  This quite
obviously is bad for SLAs and if some of those routes had alternates
there might be some unhappy customers (you could also argue that if
they didn't have alternates they'd be even more unhappy).  If the
discard does not cause the reachability test to pass, then the BGP
routes advertised by the border router that went down go down in IGP
time (seconds and possibly sub-second - YMMV).  An occasional fault
and recovery in seconds doesn't violate most SLAs and customers also
don't see substantial impact.

If the IGP aggregates, as it does in OSPF areas, but the aggregators
are not either RRs or confed boundaries, then the test for
reachability does not occur on the aggregator and that can be a
problem.  The only current solution is to not aggregate the addresses
used for the infrastructure within the AS, but aggregate those
prefixes outside the AS.  There really is not alternative.  Once the
aggregate is formed information is lost that is needed to quickly
withdraw the routes due to the reachability change.

This is certainly a topic for a BCP but we don't need to put anything
in the BGP spec.

... that said, I think I'll go back and check our implementation.

>   Pedro.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 02 Mar 2004 10:42:03 -0500

Curtis

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Mar 2004 11:13:31 +0200 (EET)

Hello, it's me again :)..

Now that the on-list thread has apparently died, and the off-list
discussions as well, I think it would be important to understand how
to move forward, and to try to gauge whether everyone is aware of
which issue we're talking about.

As there was a lot of confusion at first on the actual problem, I 
won't try to summarize the thread, because at different times, people 
probably had entirely different assumptions of the problem.

The bottom line is this: in very many cases you cannot avoid having to
insert some some kind of discard/reject or null0 route for a specific
prefix on a router (e.g., to generate a default route to some peers).  
When the next-hop of some other router, on that *same* router, goes
through that "discard" route, the next-hop is still considered valid.
This causes rather obvious BGP timeout problems.

The non-problem is when you have to redistribute that discard route in
your routing system, whether in IGP or BGP, and the other routers
cannot tell that's it's a discard route as it's learned through a 
routing protocol.  This can be rather easily avoided by configuring 
the discard routes everywhere, so I don't think this needs to be 
solved at this point.

Now.. what do people think about this problem -- what should we do 
(pick one or multiple):

 1) try to specify some behaviour in the bgp spec to avoid this issue; 
   for example:
  1.a) state that if the route towards the next-hop leads to packets 
being discarded by the router, SHOULD consider the next-hop ineligible
  1.b) state that the implementation SHOULD support being able to 
configure which routes to use in in the next-hop eligibility detection 
process
  1.c) to be worked out later..
  1.d) something else -- what?

 2) defer the solution until later

 3) clarify the operational problem, to make the reader/implementer 
aware of it
  3.a) in this revision of bgp spec
  3.b) in some other documents(s)

 4) other -- what ?

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Mar 2004 11:13:31 +0200 (EET)

4) other -- what ?

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Mar 2004 01:48:48 -0800

Pekka,

 > which issue we're talking about.

The problem did not died but was solved. The solution was that this is 
implementation detail what next hops are used for route resolution as a 
matter of the protocol owner as well as what mask length is desired. It 
was also clearly concluded by a few that BGP spec is not the place to 
add this and that you may rather issue a standalone draft describing 
operational implications on discard next hops and possibly recommend to 
implementations some new functionality.

 > This causes rather obvious BGP timeout problems.

Well I would say that this is rather not so obvious to have next hops 
for the routes causing any BGP timeouts. I have not see anyone opening a 
BGP session or configuring update src to a null0 route. I am not even 
sure what you are trying to say above.

Cheers,
R.

 > Pekka Savola wrote:

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Mar 2004 01:48:48 -0800

Pekka,

Cheers,
R.

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 12 Mar 2004 04:55:04 -0500

In message <[email-masked]>, Pekka Savola w
>   3.b) in some other documents(s)

In this ongoing discussion, we should be clear about what the
consequence is.

IBGP learned routes from a peer that has gone down do not get
withdrawn on the IGP route withdraw due to the coverage of the next
hop by an aggregate.  Those IBGP learned routes are withdrawn after
the BGP session times out which is rather a long time by today's
standards.

This is mostly a usage issue which arises when a provider's
infrastructure is covered by an aggregate that is advertised within
that provider's infrastructure.  This problem is easily avoided,
except in providers (or other BGP usage) where a default route is
made available or where address space of the provider is taken from a
larger aggregate of another provider.

With some optional configuration a router can exclude routes from
consideration in matching the BGP next-hop resolvable test.  If the
router does so, the provider doesn't need to work around the problem.

I think the best way to handle this (from the standards standpoint) is
to write a separate draft.  The draft would provide a better "BGP next
hop resolvable test" but does not match any current implementation so
it does not belong in the BGP spec.  If changes are made to any
implementations they would remain fully interoperable with those that
go by the letter of the spec without any "better resolvable test".

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 12 Mar 2004 04:55:04 -0500

Curtis

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Mar 2004 11:56:46 +0200 (EET)

On Fri, 12 Mar 2004, Robert Raszuk wrote:
> matter of the protocol owner as well as what mask length is desired. 

First, if it was decided that this is an implementation detail, it 
must IMHO be explicitly called out as such.

Second, I disagree with this being an "implementation detail".  This
has significant operational implications, and the behaviour must be as 
consistent as possible.

Dusting this off under the carpet doesn't seem to be reasonable.

> sure what you are trying to say above.

Sorry for bad wording.  The intent was to say that when another router
fails, and the next-hop points to that router, if the next-hop is not
made ineligible, the routes going through that router will only be
removed when the BGP timeout expires... causing very long
unavailability.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Mar 2004 02:09:47 -0800

Pekka,

> removed when the BGP timeout expires... causing very long

So all you are asking here is faster next hop state detection and fast 
route validatation. Very valid request, but again I fail to see why BGP 
spec should mandate how fast one wishes to validate his next hops. All 
major implementations I am aware about do it fast enough or allow one to 
configure how fast you want to validate the next hops.

Anyhow you can for sure propose the text and min timers this should 
happen and send to the list for review.

Btw .. bgp session timeout does not have to be only triggered by BGP 
keepalives :) If you monitor some other work in the IETF BFD could be 
one trigger, IGP another etc ... so it could be very fast indeed ;)

Cheers,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 12 Mar 2004 02:09:47 -0800

Pekka,

Cheers,
R.

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Mar 2004 12:20:55 +0200 (EET)

> configure how fast you want to validate the next hops.

No, the validation time (i.e., how often you check whether a nexthop
has become ineligible) is not the point.  The point is that the
validation process, no matter how often or seldom is run, reports
"valid!" for these routes which go through a discard next-hop.  
Rigging the timers changes nothing.

> one trigger, IGP another etc ... so it could be very fast indeed ;)

FWIW, Multi-hop case is currently basically out of scope for BFD.  
But that's a separate topic.

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Mar 2004 12:20:55 +0200 (EET)

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 12 Mar 2004 14:13:18 -0500

> go by the letter of the spec without any "better resolvable test".

I would agree with Curtis.  This is an operational/deployment issue
and a draft covering such issues - and not just this one - may
be a Good Thing.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 12 Mar 2004 14:13:18 -0500

-- 
Jeff Haas 
NextHop Technologies

---

From: Pekka Savola <[email-masked]>
Date: Sat, 13 Mar 2004 09:57:15 +0200 (EET)

> and a draft covering such issues - and not just this one - may

There was already an off-list discussion about this, but I think I 
should clarify the issue here:

  The local router knows that route R2 is a discard/null0 route, and
  the nexthop of route R1 goes through R2 after the more specific
  route has been withdrawn from IGP.  Route R1 is considered valid 
  even though the router knows it goes through a discard route.

Many consider this an implementation bug; some have already said
they'd fix this, so the argument "there is no implementation" does not
hold.

What I want is that this gets fixed in the spec, because the spec can
be read both ways.  I want to avoid "local optimizations" by
implementations, and fix this in the specification.  It would be
irresponsible to have purposely ambiguous specifications.

One possible way to fix this could be, in 9.1.2.1:

----
   The route resolvability condition is defined as follows.

      1. A route Rte1, referencing only the intermediate network
      address, is considered resolvable if the Routing Table contains at
      least one resolvable route Rte2 that matches Rte1's intermediate
      network address and is not recursively resolved (directly or indi-
      rectly) through Rte1. If multiple matching routes are available,
      only the longest matching route SHOULD be considered.

      2. Routes referencing interfaces (with or without intermediate
      addresses) are considered resolvable if the state of the refer-
      enced interface is up and IP processing is enabled on this inter-
      face.
----

Add a clarifying paragraph afterwards:

   If a route's (Rte1) next-hop (Rte2) or next-hop interface would 
   lead to packets being discarded by the same router, the route 
   (Rte1) SHOULD be considered unresolvable.

or:
 
   When considering resolvable routes or referenced interfaces,
   if packets through the route would be discarded by the router,
   the route SHOULD be considered unresolvable.

---

From: Pekka Savola <[email-masked]>
Date: Sat, 13 Mar 2004 09:57:15 +0200 (EET)

----

or:

---

From: Benjamin Black <[email-masked]>
Date: Fri, 12 Mar 2004 10:05:44 -0800

Which one of your options leads to "this isn't something that concerns 
BGP"?

Ben

---

From: Benjamin Black <[email-masked]>
Date: Fri, 12 Mar 2004 10:05:44 -0800

Ben

---

From: Pekka Savola <[email-masked]>
Date: Fri, 12 Mar 2004 11:56:46 +0200 (EET)