Subject: [Idr] rfc1863 - route-server rfc support

From: Paul Jakma <[email-masked]>
Date: Fri, 3 Oct 2003 13:49:30 +0100 (IST)

Hi,

I'm wondering about RFC1863 and how widely it is supported. In 
particular support for the client-mode aspects of the RFC, in 
particular the 2 following details:

- 1863 specifies that a Route-Server (RS) can withdraw a specific 
announcement by sending the full announcement to the client but with 
the next-hop address in the next-hop attribute set to one of the 
addresses of the client.

- presumably the RS-client can use the Advertiser attribute (which an
RFC1863 RS should include in updates), or any other (if included), to
discriminate between which announcements for that prefix the
withdrawal was referring to.

How many BGP implementations support these 2 things? Ie recognition 
of advertiser attribute and of next-hop-self withdrawal? (and can 
they properly discriminate withdrawals?)

It seems 1863 is the only thing out there for route-serving which 
does not require policy to be defined on the RS itself. However, i 
can find no mention of support for this RFC in the documentation of 
major BGP implementations, other than in the RFC citations.

Thanks in advance.

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
	warning: do not ever send email to [email-masked]
Fortune:
It's a naive, domestic operating system without any breeding, but I
think you'll be amused by its presumption.

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Paul Jakma <[email-masked]>
Date: Mon, 6 Oct 2003 01:54:56 +0100 (IST)

Hi Manav,

On Fri, 3 Oct 2003, Manav Bhatia wrote:

> Hi Paul,
> 
> There are severe scaling problems with RFC 1863 and hence you wont find
> many implementations supporting this (Path Attributes 12 and 13).

Ah, ok. Out of curiosity: what are the scaling problems?

Then my next question is:

- is there any portable way to announce a prefix multiple times to a 
peer and be able to withdraw a /specific/ announcement? (implicit 
withdraw?)

Essentially, is it possible to create a transparent route-server 
which does /not/ have to do path-selection on behalf of client ebgp 
peers (which requires the route-server to be configured with all 
client peers policies). Ie a route-server which simply passes on 
all announcements received to all 'client' peers? Passing on the 
announcements of prefixes isnt a problem, but i'm not sure how an RS 
would discriminate withrawals (without withdrawing a prefix then 
reannouncing remaining valid announcements - but that is not feasible 
because of stability).

thanks in advance,

> Regards,
> Manav

PS: I'm not receiving mail from this list, though I thought I had 
subscribed.

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
	warning: do not ever send email to [email-masked]
Fortune:
The difference between a career and a job is about 20 hours a week.

---

From: Paul Jakma <[email-masked]>
Date: Mon, 6 Oct 2003 01:54:56 +0100 (IST)

Hi Manav,

thanks in advance,

regards,

---

From: Ian Campbell <[email-masked]>
Date: Mon, 6 Oct 2003 15:33:37 +0100

Thanks for that Manav,
	Very interesting, shows I should check this list a little more
often! I can see how this could be useful for creation of route server
clusters (which are calculating best path for clients) as well as the
original intended application. The reason for clusters in our case is to
provide a way of scaling numbers of peers up by distributing CPU resources
(for best path calculation), as well as to a limited extent memory
resources. 

I can also see how our present software might be modified to allow peers
greater control of their own policy by using the draft attributes as you
intend (not too much of a jump!). Has there been any other interest shown so
far?

Regards,

Ian Campbell
Band-X Ltd.

-----Original Message-----
From: Manav Bhatia [mailto:[email-masked]]
Sent: 06 October 2003 14:02
To: Ian Campbell
Cc: [email-masked]
Subject: Re: [Idr] rfc1863 - route-server rfc support

Ian,

> Using the standard protocol there is no way to selectively withdraw a
> prefix, as there is no way to associate a withdraw message with a set of
> attributes. That is even if you could get the end implementation to not
> overwrite the current attribute information for a prefix (from a
particular
> peer) on update.

You can do this with
http://www.ietf.org/internet-drafts/draft-bhatia-ecmp-routes-in-bgp-00.txt

To see how it may do that refer to
http://www.ietf.org/proceedings/03jul/slides/idr-4/index.html

http://www.ietf.org/proceedings/03jul/slides/idr-4/sld13.HTM
shows how one may selectively withdraw a prefix associated with a set of
Path Attributes.

Regards,
Manav

[clipped]

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 6 Oct 2003 12:04:01 -0400

> peer and be able to withdraw a /specific/ announcement? (implicit 

If you are talking strictly within the context of a route server,
then 1863 has a mechanism already in it.  However, 1863 has
more than most people want out of a route server cluster in many
cases.  Recent proposals, including Manav's seem a bit cleaner.

An interesting question is whether those proposals have been
considered in a proxy-route-selection environment.  1863 was designed
specifically for those requirements.

> client peers policies).

Manav's draft, also draft-walton-bgp-add-paths

> PS: I'm not receiving mail from this list, though I thought I had 

You may wish to verify you're subscribed to [email-masked].

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 6 Oct 2003 12:04:01 -0400

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 6 Oct 2003 12:09:20 -0400

Paul,

On Fri, Oct 03, 2003 at 01:49:30PM +0100, Paul Jakma wrote:
> does not require policy to be defined on the RS itself.

A route-server is generally meant to be a proxy-routing concentrator.
If you aren't having it carry out your policy for you, you're just
looking for something that can send multiple route instances.
(See other e-mails on this thread for examples.)

> However, i 

Based on my conversations with others, I believe there was one or
two implementations of 1863, but it was not widely deployed.  The most
widely deployed route server environment to my knowledge was Merit's
RSng project (http://www.rsng.net for some archives) and its predecessor
project (which I have no documentation or history for).

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 6 Oct 2003 12:09:20 -0400

Paul,

-- 
Jeff Haas 
NextHop Technologies

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 10:52:00 +0100 (IST)

> A route-server is generally meant to be a proxy-routing

Well, there's 2 types, AIUI:

- 'policy aware/path selecting': ie does path-selection on behalf of
the client (in accordance with its policies). This tends to require
keeping a seperate table for each client. Ie policy must be applied
/before/ path selection.

(to apply policy afterwards may mean the path-selected is not 
applicable due to client policy. however, there may be other paths 
which /are/ applicable.)

This type of RS needs no modifications to client BGP implementations. 
The RS never needs to have more than announcement outstanding to a 
client at any one time. (ie to the client the RS is a normal eBGP 
peer).

- 'announcement reflector': One that passes on all announcements to 
peers without doing any kind of best-path selection, possibly, but 
not neccesarily applying policy before sendign announcement to the 
client. 

The latter is what I understand 1863 to provide. However, given the
impossiblity of discriminating between multiple previous
announcements when withdrawing a prefix in BGP, the latter is not
possible without some additional mechanisms (eg the NEXT_HOP set to
client address mechanism + ADVERTISER attribute described in 1863).

AIUI anyway. :)

> looking for something that can send multiple route instances. (See
> other e-mails on this thread for examples.)

Yes, but its not possible with how BGP implementations stand today.

> two implementations of 1863, but it was not widely deployed.

Shame. It seems an OK RFC. (except for the LIST message and the
RS-Cluster stuff - which i think would be impossible to implement
reliably.  Never mind unneccessary - clients could just put up with
receiving the same announcement from multiple RSes).

> The most widely deployed route server environment to my knowledge
> was Merit's RSng project (http://www.rsng.net for some archives)
> and its predecessor project (which I have no documentation or
> history for).

Indeed, AFAICT, RSng is of the 'policy aware' or 'path selecting' 
type.

However, I believe many operators would like to see a mechanism that 
allows an RS to be implemented that does /not/ require the RS to be 
policy-aware. Ie because of the overhead of keeping RSes configured 
with client policy, and, possibly more important, because policy can 
be commercially sensitive and many operators of potential RS-client 
peers do not wish to have to register their policy with a 3rd party 
(eg an IX operating an RS).

So it would be nice to have some means to allow implementing an RS
that was 'dumb' and simply reflected on /all/ announcements and
withdrawals.

I'll have a look at Manav's ECMP_NEXT_HOP draft and offer comment.

thanks!

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
	warning: do not ever send email to [email-masked]
Fortune:
Running Windows on a Pentium is like having a brand new Porsche but only
be able to drive backwards with the handbrake on.
(Unknown source)

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 10:52:00 +0100 (IST)

AIUI anyway. :)

thanks!

regards,

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 10:00:37 -0400

> keeping a seperate table for each client. Ie policy must be applied

Sort of.

To give a specific counter-example, a rsd "view" may be the same
for all peers of that view.  Route-selection was intended to be
done on a per-view basis.

> The latter is what I understand 1863 to provide.

The language of the RFC is unclear enough that this is a valid
assumption.  However, where's the benefit?  You minimize the
overhead of peering sessions, but that is it.

The RFC does allow for route servers to share routes and distribute
load thus making the proxy-routing case a bit easier to deal with.

> Yes, but its not possible with how BGP implementations stand today.

In the protocol, no.  In practice, people run virtual router instances
and multiple peering sessions to accomplish some of this where
it is seen as necessary.

> policy-aware.

What exactly are you trying to accomplish?

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 10:00:37 -0400

Sort of.

-- 
Jeff Haas 
NextHop Technologies

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 15:13:11 +0100 (IST)

> done on a per-view basis.

Yes, you could do it that way. Table per peer can be implemented by 
having one view per peering pairs.

> assumption.

Seems reasonably clear in section 4.3.2:

"Route servers relay all acquired routes unaltered to their clients"

:)

> However, where's the benefit?  You minimize the overhead of peering
> sessions, but that is it.

That may be the only overhead you want to minimise, yes.

> load thus making the proxy-routing case a bit easier to deal with.

Yes, though that bit of the RFC is a bit over-complex imho. Much 
simpler just to let the clients go "oh, i received that already from 
the other RS, ignore" in my mind. (given that clients probably would 
have to be able to cope with condition anyway, as the RS-clustering 
stuff (LIST message, lists of clients) seems a bit fragile to me).

> In the protocol, no.  In practice, people run virtual router
> instances and multiple peering sessions to accomplish some of this
> where it is seen as necessary.

yes. but thats high overhead :)

> What exactly are you trying to accomplish?

The spirit of RFC 1863, ie route-servers which operate along the vein
of the first paragraph of Sec. 4.3.2 in RFC1863 - ie pass on all
announcements and withdrawals without doing path-selection.

But it doesnt appear possible as things stand.

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
	warning: do not ever send email to [email-masked]
Fortune:
The Shuttle is now going five times the sound of speed.
		-- Dan Rather, first landing of Columbia

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 15:13:11 +0100 (IST)

:)

regards,

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 10:21:17 -0400

> "Route servers relay all acquired routes unaltered to their clients"

Indeed.

A client *could* be another route server. :-)

> stuff (LIST message, lists of clients) seems a bit fragile to me).

The failover and coordination issues are likely some of the reasons
why this never really caught on.  However, it does allow for you
to avoid receiving multiple duplicate route instances from a peer
which you can get even from just redundant route-reflector connections.

If you're trying to minimize for memory, this may be ideal.  

In the context of RSd/RSng, in practice, being at one step removed
in the forwarding plane from the router announcing was a serious
operational issue.  Blackholing due to ATM PVC issues was way too
common.  Having to coordinate between route servers for failover
would likely be more of an issue than just building your network to
hold the extra route instances from your live backup.

> But it doesnt appear possible as things stand.

Understood.  It will be interesting to see how the two current proposals
fare.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 10:21:17 -0400

Indeed.

-- 
Jeff Haas 
NextHop Technologies

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 15:41:36 +0100 (IST)

> A client *could* be another route server. :-)

yes :)

> why this never really caught on.  However, it does allow for you to
> avoid receiving multiple duplicate route instances from a peer

Just specifying that 1863 style RS-clients should be prepared for 
duplicates and to deal with it probably would have been the simplest 
solution. (perfect is the enemy of good :) )

> If you're trying to minimize for memory, this may be ideal.

I'm not too sure of the operational reasoning actually. I suspect
commercial sensitivity of client policy and operational simplicity of
not having to centralise that policy is the main factor in wanting
1863 style route-servers.

> common.

eek yes. though, could one not regard this problem as one of an 
IGP / network architecture nature. (make sure there are multiple, 
robust paths between peering addresses). A lot of IXes these days 
seem to be 100 or GigE based anyway - not ATM.

> Having to coordinate between route servers for failover would
> likely be more of an issue than just building your network to hold
> the extra route instances from your live backup.

Yeah, co-ordination between route-servers would be a killer. So just 
dont do it is what I'd say. Let the client cope with and discard 
duplicates. :)

> Understood.  It will be interesting to see how the two current
> proposals fare.

yes, it will be.

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
	warning: do not ever send email to [email-masked]
Fortune:
You can be replaced by this computer.

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 15:41:36 +0100 (IST)

yes :)

yes, it will be.

regards,

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 10:54:38 -0400

> seem to be 100 or GigE based anyway - not ATM.

The issue is a more general one.  If you receive a route from someone
else and they are not in the forwarding path, you are much less
sure that the route will not be a blackhole.  This is true even of
several third-party nexthop scnenarios.

Admittedly, most of this was caused by ATM issues and a broadcast
media would be less prone to the problem.

> dont do it is what I'd say. Let the client cope with and discard 

Depends on the problem you are wanting to solve.  You want all the
route instances, others want to proxy their policy.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 10:54:38 -0400

-- 
Jeff Haas 
NextHop Technologies

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 16:09:24 +0100 (IST)

On Tue, 7 Oct 2003, Jeffrey Haas wrote:

> The issue is a more general one.  If you receive a route from
> someone else and they are not in the forwarding path, you are much
> less sure that the route will not be a blackhole. 

True, yes.

Though, if the next-hops are /not/ on the broadcast media, then one 
could ensure that a received BGP route's next-hop would have a 
corresponding static in IGP. If not, pick another route which /does/ 
have a valid next-hop in IGP.

Though, this only would work for the 1863 style route-server, where 
client-peers are fully informed and have multiple routes to choose 
from.

For the RSng style 'path-selecting' RS, indeed, the RS might consider 
a next-hop valid and pass on what it sees as best-path, but the 
client might not have a route to the next-hop.

> This is true even of several third-party nexthop scnenarios.

Can you think of any in the 1863 style RS model? It seems to me that 
as the client receives all announcements, 1863 would be more robust 
to reachability problems. (if we presume the BGP next-hops are 
distributed as IGP statics, and the RS-client peers participate in 
the IGP).

> media would be less prone to the problem.

indeed :)

> route instances, others want to proxy their policy.

Absolutely. Though, 1863-style seems more robust (if you avoid trying 
to co-ordinate the route-servers).

regards,
-- 
Paul Jakma	[email-masked]	[email-masked]	Key ID: 64A2FF6A
	warning: do not ever send email to [email-masked]
Fortune:
Peace cannot be kept by force; it can only be achieved by understanding.
		-- Albert Einstein

---

From: Paul Jakma <[email-masked]>
Date: Tue, 7 Oct 2003 16:09:24 +0100 (IST)

True, yes.

indeed :)

regards,

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 11:19:26 -0400

> have a valid next-hop in IGP.

The situation I was describing is a simple third-party nexthop
case.  Thus, the nexthop you give me needs to be on the same logical
network as the one we both share.

Using statics to resolve iBGP nexthops can lead to similar blackhole
issues if not done carefully.

> client-peers are fully informed and have multiple routes to choose 

This happens even in normal BGP. 

> client might not have a route to the next-hop.

If the (non-route server) RS client can't resolve the nexthop,
then it should not consider it a feasible route for route selection
purposes.

> to co-ordinate the route-servers).

Given that we're at a point of stating opinions, I wish to make myself
clear:  I have no direct experience with 1863 and thus can't comment
on how well it worked beyond a theoretical case.  However, if we
were to want to solve the same problem *today*, I wouldn't use it.
I think we have better ideas at this point.

-- 
Jeff Haas 
NextHop Technologies

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 7 Oct 2003 11:19:26 -0400

-- 
Jeff Haas 
NextHop Technologies