Subject: [Idr] why has 4096 bytes limit on BGP messages size?

From: "Fenggen Jia" <[email-masked] >
Date: Thu, 14 Jun 2007 13:01:47 +0800

hi,all,
      I am wondering why BGP has 4096 bytes limit on all its messages,should we leave it to the implementator to decide,thanks!
Yours,Fenggen

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Tony Li <[email-masked]>
Date: Wed, 13 Jun 2007 23:24:50 -0700

On Jun 13, 2007, at 10:01 PM, Fenggen Jia wrote:

> hi,all,
>       I am wondering why BGP has 4096 bytes limit on all its  
> messages,should we leave it to the implementator to decide,thanks!

Well, the problem with that is that a sender could then create an  
arbitrarily long message, which could be arbitrarily problematic for  
a receiver.

Tony

---

From: Pekka Savola <[email-masked]>
Date: Thu, 14 Jun 2007 18:59:47 +0300 (EEST)

On Wed, 13 Jun 2007, Tony Li wrote:
> Well, the problem with that is that a sender could then create an arbitrarily 
> long message, which could be arbitrarily problematic for a receiver.

While this is an interesting historical perspective, I'm not sure how 
relevant it is anymore.  How does BGP differ from other TCP 
applications (almost none of which specify the maximum byte limit in 
the application)?  Are routers so low-end that they cannot allocate 
more than 4K memory per message, or that they cannot determine the 
amount of memory to allocate depending on the message length? :-)

(The header fields already tell the length of the message, so 
allocating 4K buffer for each BGP message (when most messages are much 
smaller than 4K) is probably a waste in many cases.)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Danny McPherson <[email-masked]>
Date: Thu, 14 Jun 2007 16:40:31 -0600

On Jun 14, 2007, at 9:59 AM, Pekka Savola wrote:
> allocating 4K buffer for each BGP message (when most messages are  
> much smaller than 4K) is probably a waste in many cases.)

Pekka,
Do you have some references to data supporting your comment here
regarding most messages being smaller than 4K?

-danny

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jun 2007 01:14:01 -0400

> smaller than 4K) is probably a waste in many cases.)

Just from an efficiency standpoint it makes sense to read much more
than 4KB from the BGP socket.  If the router is busy, having a 16KB or
larger read buffer lets TCP do its thing more efficiently.  You then
typically get N BGP updates plus some fraction of an update which has
to be shuffled in the buffer and another large read issued.  If the
most you have to shuffle is 4KB that might be a good thing.  Less
system calls is generally better favoring big reads though it depends
on the OS.  This tends to matter when a big routing transient happens.

Older very memory challenged implementations may have allocated 4KB
and did a small read to get the header and then another read to get
the rest of the update but that can be fairly inefficient.  Back when
BGP was invented some routers had 4MB or less and a 68020 or worse (a
PC-RT).

4KB is big enough that the overhead of parsing the header and the
attributes more than once when a large number of prefixes have the
same attributes is not a big penalty.  That is something that anyone
doing an implementation could verify or dispute after doing a program
execution profile.

Curtis

---

From: Pekka Savola <[email-masked]>
Date: Fri, 15 Jun 2007 08:48:02 +0300 (EEST)

On Thu, 14 Jun 2007, Danny McPherson wrote:
>> (The header fields already tell the length of the message, so allocating 4K 
>> buffer for each BGP message (when most messages are much smaller than 4K) 
>> is probably a waste in many cases.)
>
> Pekka,

Yes.  I've sometimes taken debugs logs of BGP when a DFZ table flapped 
and looked at the packets.  Here's a plot of BGP Update size versus 
occurrences (in this about 10-15min trace, 87K BGP updates).  BGP 
Updates are sent by a Juniper T-series router.  The smallest is 26B, 
the largest 4095B.  Top 20 sizes (6995 through 1229 occurrences) are 
all between 58 and 95 bytes.

(Not sure if the small attachment gets on the list, but if not, I'll 
post it somewhere.)

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jun 2007 01:51:34 -0400

In message <[email-masked]>
> regarding most messages being smaller than 4K?

I looked into exactly this more than a decade ago.  Its not likely to
have changed much.

Most of the time you get an occasional BGP keepalive.  This is
uninteresting.

What is interesting is when a transient occurs.  You have to engineer
for the stress conditions and this is it for a BGP implementation.

At that time many peers sent a large number of prefixes with the same
AS path.  These got packed into multiple full BGP updates and a few
partial updates.  Then there were AS paths (and other attributes, but
mostly AS path) that had a smaller number of prefixes.

If the BGP sender is efficient then lots of small updates are packed
into a single write.  TCP will deliver this as a stream of TCP
segments making use of the full MSS.  On the receiving end one big
read will pick up multiple BGP updates.  The BGP adj-in RIB processing
is done (which is very minimal) and a read is immediately reposted.
This way the flow of BGP updates goes quickly.  By the time the BGP
adj-out processing happens most or all of the updates have been
transferred.  A possible slow step is getting the route changes jammed
into the forwarding cards.  This too has improved.

So during a transient most of the reads are delivering either full BGP
updates or multiple partially filled updates.  If the read buffer is
larger than 4KB this is even more efficient as multiple full BGP
updates can be read in a single read.

Curtis

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 15 Jun 2007 08:53:16 -0600

Pekka:

What is the complexity of the BGP peering at your DFZ? Can you give an
overview of the type of updates this represents?  

Without that information, it is difficult to quantify your data point. 

Millions of routers in several virtual routers with RD tags may have a
different flow pattern of updates. 

Thanks,

Sue

-----Original Message-----
From: Pekka Savola [mailto:[email-masked]] 
Sent: Friday, June 15, 2007 1:48 AM
To: Danny McPherson
Cc: idr List Routing
Subject: Re: [Idr] why has 4096 bytes limit on BGP messages size?

>> (The header fields already tell the length of the message, so
allocating 4K

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 15 Jun 2007 09:02:43 -0600

Pekka:

Oops - I shouldn't type before my coffee in the morning. Please change
the question to the following information: 

Millions of routes in several virtual routers with RD tags may have a
different flow pattern of updates. 

Thanks,

-----Original Message-----
From: Susan Hares [mailto:[email-masked]] 
Sent: Friday, June 15, 2007 10:53 AM
To: Pekka Savola; Danny McPherson
Cc: idr List Routing
Subject: RE: [Idr] why has 4096 bytes limit on BGP messages size?

Pekka:

Thanks,

Sue

-----Original Message-----
From: Pekka Savola [mailto:[email-masked]] 
Sent: Friday, June 15, 2007 1:48 AM
To: Danny McPherson
Cc: idr List Routing
Subject: Re: [Idr] why has 4096 bytes limit on BGP messages size?

allocating 4K