Subject: RE: [Idr] issue: bgp4-23: route resolvability and discard/null0 routes

From: Parantap Lahiri <[email-masked]>
Date: Mon, 01 Mar 2004 12:25:27 -0500

If you aggregate the prefixes just towards EBGP neighbors it definitely does
solve the problem, but the issue is you will have to touch all your ebgp
sessions once you get a new block to aggregate. Operator motto is have a
design which requires least touching. Of course that can be equated to
laziness, but it depends on which side of fence you are on.

>This isn't a problem with the BGP protocol.  Its a problem with the
>way some providers configure their networks (incorrectly) that is
>easily solved by not doing it that way, or can be solved by adding a
>config knob to the router to exclude aggregates from this AS from
>consideration in the reachability test or exclude specific prefixes.
>This is certainly something that can be addressed in a separate
>document.

This is exactly what I suggested in the 2nd paragraph of my original mail.

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Monday, March 01, 2004 11:46 AM
To: Parantap Lahiri
Cc: [email-masked]; 'Pekka Savola'; 'John G. Scudder'; [email-masked]
Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0
routes

In message <[email-masked]>, Parantap Lahiri
writes
:
> Curtis,
> 
> I was referring to a complete separate address space for the secondary
> loopbacks and the idea was not to aggregate that space. It would be only
in
> IGP as discrete /32s, and if the node is gone, the /32 will be gone, thus
> immediately invalidating the routes originated by it. The rest of the
world
> could still reach the box using the primary loopback address, which would
be
> aggregated and would be in BGP. 
> 
> Anyway, my initial message was delayed by the moderator and I found John
> Scudder referring to this option in one of his later mails. So I think its
> covered. 
> 
> -Parantap   

The way we did this when I was doing operations was we aggregated and
sent the aggregate to EBGP peers.  Since we didn't have the aggregate
for our own address allocation within our AS this never came up.  This
also was not a problem because the loopback by definition cannot
become unreachable, only interface addresses can become unreachable.

IBGP peers should be announcing their loopback, which can never go
down unless the router becomes completely unreachable.  If the
provider's aggregate is only sent outside the AS, then this is fine.
I personally felt as an operator that if another provider is to *lazy*
to configure the aggregate statement on each EBGP peer, and instead
chooses to announce that aggreate into IBGP, then they get what they
deserve.  Working for a vendor now, I have to take the position that
the customer can be as arbitrarily lazy as they like and thats just
fine.

This isn't a problem with the BGP protocol.  Its a problem with the
way some providers configure their networks (incorrectly) that is
easily solved by not doing it that way, or can be solved by adding a
config knob to the router to exclude aggreates from this AS from
consideration in the reachability test or exclude specific prefixes.
This is certainly something that can be addressed in a separate
document.

Curtis

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 01 Mar 2004 11:46:06 -0500

In message <[email-masked]>, Parantap Lahiri writes
:
> loopbacks and the idea was not to aggregate that space. It would be only in
> immediately invalidating the routes originated by it. The rest of the world
> could still reach the box using the primary loopback address, which would be

Curtis

---

From: Parantap Lahiri <[email-masked]>
Date: Mon, 01 Mar 2004 10:33:11 -0500

Curtis,

-Parantap   

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Saturday, February 28, 2004 1:24 PM
To: Parantap Lahiri
Cc: 'Pekka Savola'; 'John G. Scudder'; [email-masked]
Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0
routes

In message <[email-masked]>, Parantap Lahiri
writes
:
> The problem Pekka is describing is definitely a very valid problem which
> exists. I am not sure though if the fix is in some design modifications or
> in fixing the spec itself. 
> 
> The ways to fix it by design modification would be ...
> 
> The router R1 in question resets the BGP next-hop to some secondary
loopback
> address which is different from the primary loopback (part of the
prefix/16)
> and the secondary loopback address range is only in IGP as discrete /32s,
> this problem might be mitigated. Though it's going to load the IGP with a
> new set of IP prefixes and may be problematic in some situations since
some
> IGP protocols have limited capability to carry large number of IP
prefixes. 
> 
> The other way to solve it might be a vendor implemented knob, where a
> particular address range (like prefix/16) is never considered while doing
> BGP next hop resolution. Though, I am not sure if this knob would be a
part
> of the spec.

I'm not sure what that has to do with discard routes unless you are
talking about generating an aggregate and the next hop falling into
the unused portion of the aggregate.  If you allow the next hop to
resolved to the unused portion of an aggregated, your implementation
is simply broken.  You can't forward on a aggregate you generate, you
can only forward on the components that make up the aggregate and if
you miss this distinction in your BGP implementation, that is just an
implementation mistake.

If OTOH, some other router did the aggregation and sent the aggregate
into the IGP or IBGP, then the problem is not one that needs to be
fixed in the spec.  If allowing BGP learned routes to resolve BGP next
hops, which is commonly configured, another knob that can be added is
to exclude routes in which the aggregator is in the same AS.  If that
was not sufficient you'd need a knob to exclude specific prefixes.
This is clearly outside of the BGP spec.  It seems to me to be more a
case of adding configuration knobs to cover up the problems caused by
misconfiguration.

Curtis

---

From: Parantap Lahiri <[email-masked]>
Date: Mon, 01 Mar 2004 15:30:18 -0500

Not to beat a dead horse here but in some major vendor implementations, you
can't send an "aggregate" if it is not in your routing table. And it defeats
the purpose.

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Monday, March 01, 2004 3:02 PM
To: Parantap Lahiri
Cc: [email-masked]; 'Pekka Savola'; 'John G. Scudder'; [email-masked]
Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0
routes

In message <[email-masked]>, Parantap Lahiri
writes
> laziness, but it depends on which side of fence you are on.

How many different blocks of addresses do you put your IBGP loopbacks
in?  Last I counted it was one.  You don't have to treat your customer
aggregates this way if you don't want to, just the aggregate that
covers your backbone.  And btw - with machine generated configs there
is no problem generating aggregates on each EBGP speaking router - its
only when you try to hand configure your 1,000+ routers (or whatever).

Anyway, we already seemed to agree that this at most ends up in a
separate new document.

Curtis

---

From: Parantap Lahiri <[email-masked]>
Date: Mon, 01 Mar 2004 15:30:18 -0500

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Monday, March 01, 2004 3:02 PM
To: Parantap Lahiri
Cc: [email-masked]; 'Pekka Savola'; 'John G. Scudder'; [email-masked]
Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0
routes

Curtis

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 01:41:41 +0200 (EET)

On Mon, 1 Mar 2004, Curtis Villamizar wrote:
> I suppose this is the original reason that the discard route was
> mentioned, but with a failure to mention the entire context.  You can
> manually configure discard routes but that is generally only done in
> very unusual circumstances, such as a DoS attack to a specific
> destination.

I guess you weren't following.  Discard routes are set up because
folks want to announce that aggregate route to their peers.  That's
not a "very unusual circumstance".

> No need to modify the spec.  If you've installed a special type of
> route in the kernel that is there explicitly to indicate
> non-reachability, then that route should not be allowed to indicate
> reachability.  I dont' think we really need to say that.

You would think so, but the current spec does not say so.  

Maybe a problem is that a discard route should not fail the 
resolvability check when propagated as a DDoS prevention hack, 
referred earlier.

IMHO, excluding these routes seems obvious to me, but as it wasn't 
obvious to the implementors and they hadn't done it, it MUST be in the 
spec in one form or another.  Whether it should be discussed at more 
length in some other document (Targeting BCP or PS, I don't know).

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 01:41:41 +0200 (EET)

---

From: Pedro Roque Marques <[email-masked]>
Date: Mon, 1 Mar 2004 16:17:14 -0800 (PST)

> IMHO, excluding these routes seems obvious to me

And to me it seems obvious that a route is a route, regardless of
next-hop type. The later being an implementation specific concept and
not something to be addressed at the spec level...

It is clear that what 'seems obvious' is highly subjective to the
person in question.

>, but as it wasn't
> obvious to the implementors and they hadn't done it, it MUST be in
> the spec in one form or another.

imho, it MUST NOT be in the BGP specification in one way or
another. The BGP specification deals w/ the protocol definition, not
with how to implement it. 

  Pedro.

---

From: Pedro Roque Marques <[email-masked]>
Date: Mon, 1 Mar 2004 16:17:14 -0800 (PST)

Pedro.

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 1 Mar 2004 19:27:32 -0500

Pekka,

> not a "very unusual circumstance".

If I'm following you correctly, here's an example of the condition that
you're talking about.

  C
 /|
A |
 \|
  B

B has prefix D1 - 192/17
C has prefix D2 - 192.128/17

B and C advertises these routes with loopback addresses.  (next-hop-self)

A synthesizes an aggregate, 192/16 using B and C's routes as contributors.
A thus installs 192/16 in it's FIB as a reject route per accepted
practice.

C is nuked from space in a green fireball.

Prior to BGP figuring out that C has gone away, C's loopback is no longer
advertised, leaving B to use the aggregate from A to calculate the
resolved nexthop for D2.  Thus, B has a route to D2 via A, which will
eventually reach a reject route.

Is this your scenario?

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 1 Mar 2004 19:27:32 -0500

Pekka,

  C
 /|
A |
 \|
  B

B has prefix D1 - 192/17

Is this your scenario?

-- 
Jeff Haas 
NextHop Technologies

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 01 Mar 2004 15:02:13 -0500

In message <[email-masked]>, Parantap Lahiri writes

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 01 Mar 2004 15:44:45 -0500

In message <[email-masked]>, Parantap Lahiri writes
> can't send an "aggregate" if it is not in your routing table. And it defeats

Then the aggregate is locally generated and it is very easy to exclude
the parts of an aggregate that you generated but are not covered by
more specific from the reachability test and it makes perfectly good
sense to do so.  If other prefixes in the AS match that aggregate then
the aggregate is generated but locally it is installed as a discard
route.  

I suppose this is the original reason that the discard route was
mentioned, but with a failure to mention the entire context.  You can
manually configure discard routes but that is generally only done in
very unusual circumstances, such as a DoS attack to a specific
destination.  Even then it doesn't make a whole lot of sense to allow
such a discard route to be used in the resolvability test.

Curtis

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 1 Mar 2004 19:44:27 -0500

On Mon, Mar 01, 2004 at 03:44:45PM -0500, Curtis Villamizar wrote:
> mentioned, but with a failure to mention the entire context.

It took me a second to remember, but to dredge up the ghost from
the last time this thread came around:

rfc 1519, section 4.2.2:

:   4.2.  Rules for route advertisement
: 
[...]
:       2.   A routing domain which performs summarization of multiple
:            routes must discard packets which match the summarization but
:            do not match any of the explicit routes which makes up the
:            summarization. This is necessary to prevent routing loops in
:            the presence of less-specific information (such as a default
:            route).  Implementation note - one simple way to implement
:            this rule would be for the border router to maintain a "sink"
:            route for each of its aggregations. By the rule of longest
:            match, this would cause all traffic destined to components of
:            the aggregation which are not explicitly known to be
:            discarded.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 1 Mar 2004 19:44:27 -0500

rfc 1519, section 4.2.2:

-- 
Jeff Haas 
NextHop Technologies

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 02:52:57 +0200 (EET)

> A thus installs 192/16 in it's FIB as a reject route per accepted

We're doing "manual aggregation" by adding the 192/16 manually as a 
discard route, and filtering out the more specifics when advertising 
to peers, but the end result is the same.
 
> eventually reach a reject route.

Yep -- there two flavors here:
 1) looking from A's perspective -- the route to C is known to be a 
reject route by A.
 2) looking from B's perspective -- the route to C goes to A, where 
the packets are rejected. B does not, in BGP, know that the route is a 
reject route at A.

1) is the root cause; 2) can be worked around by installing "reject"  
routes on all boxes.

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 1 Mar 2004 20:09:14 -0500

Please excuse the long citation.

> to peers, but the end result is the same.

A, B and C are all within a given AS.
Regardless of whether A propagates B and C's more specifics to peers
outside of this AS isn't important to this particular scenario.

To be clear, we may consider A, B and C to be egress routers for this
AS and B and C will propagate the aggregate synthesized by A and learned
via iBGP.  Note that as has been mentioned multiple times, this is not
common practice - C and B should be configured to synthesize the aggregate
as well, but this will work "ok" without it.

>  1) looking from A's perspective -- the route to C is known to be a 

We're fine here.

> the packets are rejected. B does not, in BGP, know that the route is a 

And thus packets that arrive at B will be propagated to A and be
consumed by the reject route.

And this is not fine how?

The ideal scenario is that each core router within the AS knows all
of the routes to networks within an AS.  Once a packet hits one of
these routers, that router will generate ICMP destination unreachable.

In our scenario above, we provide a routing sink that attracts packets
that otherwise would be generating ICMP further into the network,
but the right thing continues to happen.  If we're being clever,
we may be doing this intentionally with a default sink as an attempt
to collect statistics about packets being sent to hosts within our
network that appear outside advertised aggregates for purposes of 
intrusion detection or something else.

At no time does this break routing.  What the above scenario indicates
is that the network isn't being configured well.

Ideally (and please feel to shoot this down based on operational 
experience):
o Aggregation happens at your AS border routers.  Every single one of them.
o You do not inject your aggregates into your iBGP.
o You do not advertised more specifics covered by an announced
  aggregate unless you are doing intentional traffic engineering.
  If you are, you're preferably working with your upstream ISPs to
  make sure those routes aren't propagated "too far" in attempt to keep
  the Internet routing table clean.  One example is tagging the routes
  with the NO_EXPORT community.

However, if you fail in one of the above, routing will still work.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 1 Mar 2004 20:09:14 -0500

We're fine here.

-- 
Jeff Haas 
NextHop Technologies

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 03:37:35 +0200 (EET)

> >  1) looking from A's perspective -- the route to C is known to be a 

We'll, not sure what you mean.  I don't think "we're fine here" is a 
good choice of words, because that's the actual problem I was 
reporting -- one which is not covered by the specification!

> And this is not fine how?

I can accept this behaviour, but 1), above is unacceptable.

> these routers, that router will generate ICMP destination unreachable.

Agreed.

> intrusion detection or something else.

But such a packet sink breaks route resolvability -- all the routers
which do not have the reject route -- assuming that problem 1) above
will be fixed -- will continue to have their iBGP sessions up.  This 
is, AFAICS, in conflict with two of your goals below.
 
> is that the network isn't being configured well.

I think you're assuming that 1) is a non-problem.  It is a real 
problem.  But assuming it isn't..
 
> o You do not inject your aggregates into your iBGP.

These are desirable goals, yes.  Actually, they are just one goal,
worded differently -- if you don't aggregate everywhere, you have to
have the aggregates in your iBGP, no question about it.  Excluding
misconfigurations (such as unintentional aggregate advertisement to
iBGP unnecesarily), practically, either you break zero or you break
both.

Practically, these are not always feasible.  E.g., we use private ASns
in access networks, so each core router connecting to an access
network has EBGP and would have to aggregate (we're just advertising
the default).  This is "aggregation in all of the borders" is a bit
cumbersome, but we could add it if that would make the network "more
correct" and make it work better.

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 03:37:35 +0200 (EET)

Agreed.

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 2 Mar 2004 10:15:44 -0500

On Mon, Mar 01, 2004 at 08:40:52PM -0500, Curtis Villamizar wrote:
> If A installs the discard route but not both components, then A's
> implementation of aggregation is broken.

The installation of the more specifics was presumed.  
I was talking about BGP aggregation.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 2 Mar 2004 10:15:44 -0500

-- 
Jeff Haas 
NextHop Technologies

---

From: Parantap Lahiri <[email-masked]>
Date: Mon, 01 Mar 2004 12:25:27 -0500

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Monday, March 01, 2004 11:46 AM
To: Parantap Lahiri
Cc: [email-masked]; 'Pekka Savola'; 'John G. Scudder'; [email-masked]
Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0
routes

in
world
be

Curtis

---

From: Parantap Lahiri <[email-masked]>
Date: Mon, 01 Mar 2004 10:33:11 -0500

Curtis,

-Parantap   

-----Original Message-----
From: Curtis Villamizar [mailto:[email-masked]] 
Sent: Saturday, February 28, 2004 1:24 PM
To: Parantap Lahiri
Cc: 'Pekka Savola'; 'John G. Scudder'; [email-masked]
Subject: Re: [Idr] issue: bgp4-23: route resolvability and discard/null0
routes

loopback
prefix/16)
some
prefixes. 
part

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 01 Mar 2004 15:44:45 -0500

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 01 Mar 2004 15:02:13 -0500

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 01 Mar 2004 11:46:06 -0500

Curtis

---

From: Pekka Savola <[email-masked]>
Date: Tue, 2 Mar 2004 02:52:57 +0200 (EET)