Subject: Re: Number of Routes and network stability

From: Brett White <[email-masked]>
Date: Wed, 16 Mar 2005 11:35:51 +1100

Hi Mitchell,

You say that a router shouldn't have an issue with 3000 links to other routers in an area.  What I am interested in is how you see the breakdown of this figure on a per interface basis?  Do you expect that the device should be able to support these 3000 links on a single interface or spread across say 128 ports?

Regards,
 
Brett White

-----Original Message-----
From: Mailing List [mailto:[email-masked]] On Behalf Of Erblichs
Sent: Friday, 3 December 2004 13:49
To: [email-masked]
Subject: Re: Number of Routes and network stability

Andrew Smith,

        I don't think I agree with 90% of this.

        Available memory has to do with the max size of
        the LSDB, siumultaneous rexmit queues, etc, THAN
        the max number of routers supported in a area.

        Assuming that enough memory is available, link-stability
        and dead-router interval are more important than CPU
        capability wrt network stability. Next comes with the
        number & sizing of input queues, link-speed, and network
        topology.

        If the number of hellos and OSPF control pkts
        overload a CPU, then I hate to see the SPF timeframes,
        or what would happen if a large number of pkts
        are targeted to the router.

        If one says that they can't handle close to 3000
        simultaneous links with other routers in a area,
        then IMO, they have a few issues with their code.

        I am NOT stating that 3000 simultaneous full adjs can
        be formed. But if you are dropping adjs because their
        are too many routers in the area and you thus are
        dropping (or delaing the processing of) hellos, then
        their is something wrong with the code or hardware.

        Support for a large number of routers within a area
        more has to do with IMO, the ability to decrease the
        hello interval and hello multiplier and still not drop
        adjs. Secondly, each router then COULD support fewer
        links and a failed router would have less of a topology
        effect.

        Mitchell Erblich
        -----------------

> Andrew Smith wrote:
>
> The recommendation for number of routers per area is usually derived 
> by considering the amount of processor power and memory available to 
> the routers in an area. Usually you consider the lowest common 
> denominator. This means the smallest router (the router with the 
> fewest resources) limits the size of the area.
>
> A good rule of thumb is no more than 100 routers per area. If you try 
> to keep an initial design to 50-70 routers that usually allows room to 
> grow and recover from a network crisis.
>
> The vendors that say they can do 3000 routers per area are giving a 
> scalability number for that box. This particular router can achieve 
> 3000 routers per area "under the proper circumstances". This is good 
> to know for how much the box can take, not how high to scale your OSPF 
> areas.
>
> Regards,
> Andy Smith
>
> -----Original Message-----
> From: Mailing List [mailto:[email-masked]] On Behalf Of 
> Jing Shen
> Sent: Thursday, December 02, 2004 4:46 AM
> To: [email-masked]
> Subject: Number of Routes and network stability
>
> Hi,
>
> Some vendors recommended that number of routers within one OSPF area 
> should not exceed 50, some others recommended  number of routes within 
> one area should not exceed 3000.
>
> To my understanding, OSPF stability is directly related to link 
> stability, radius of an OSPF area and OSPF implementation( router 
> hardware, methodology of link status processing). But, is there any 
> research done on this topic? or how are those recommendation derived ?
>
> thanks
>
> Jing Shen
>
> _________________________________________________________
> Do You Yahoo!?
> ×¢²áÊÀ½çÒ»Á÷Æ·ÖÊµÄÑÅ»¢Ãâ·ÑµçÓÊ
> http://cn.rd.yahoo.com/mail_cn/tag/1g/*http://cn.mail.yahoo.com/

---

From: Erblichs <[email-masked]>
Date: Wed, 16 Mar 2005 01:13:23 -0800

Brett & group,

	3000 is a fairly conservative figure. If we first start
	with say just 3000 hellos per 10 secs, then we are at
	say 300 * y hello pkts per sec. "y" is based  that the hello
	pkt could be multiple MTU (1500 bytes) in size and are
	the number of necessary hello frag and non-fragmented
	pkts.

	If we are dealing with 10Gb interfaces that are Eth
	with 1500 bytes/pkt, then we can traverse approx 83k
	seg/pkts each sec. Forgeting about Inter Pkt Gap and
	such, as we are dealing with approximates.

	So, 600 or 900 pkts per sec within a 83k pkt timeframe
	is noise. If we drop to say a 1Gb, then we are about
	8.3K pkts/sec and the 900 starts to become significant,
	because we still need to add updates. Yes, I am assuming
	that we aren't dealing with DNA LSAs and that they will
	need refreshing.

	If you can't process just 10% of your wirespeed
	traffic with the standard slow-path for control
	pkts, then .... So, I am assuming at least this rate,
	and with a large number of interfaces the actual
	% rate decreases with an increasing aggreaated
	link bandwidth.

	So, If we spread this load accross 128 interfaces, it
	becomes noise again. Approx the same amount of protocol
	parsing work exists independently of the number of
	interfaces.

	However, the issue then becomes how we
	proccess the pkts when we have a fair number of interfaces.
	Do we poll and hopefully process a decent number of
	pkts per interface or are we interrupt driven where we 
	can do a minimal amount of work at interrupt level?

	Bottom line is that as we are able to deal with a larger
	number of simultaneious nbrs/peers, part of the overhead,
	of adj is dealing with the hello's, and that instead
	of just theorectly increasing nbrs, we can realisticly
	decrease the hello interval to 1 per sec or in some draft
	RFCs consider supporting subsec hello intervals.

	So, did this make sense and answer you question?? 

	Mitchell Erblich
	---------------------