Subject: Re: Re: [Idr] why has 4096 bytes limit on BGP messages size?

From: "Fenggen Jia" <[email-masked] >
Date: Thu, 14 Jun 2007 18:58:06 +0800

Then why not other limit,how is 4096 bytes derived, thanks.BTW,whether it's a good pratice in protocol devise to limit message sizes?I see in ripV1 that the maximum PDU length is 576bytes its reason is to avoid IP fragmentation,but in ripng the PDU length is determined only by the link MTU,and in snmp i also doesn't see the message size limit. 
Yours,Fenggen
>
>On Jun 13, 2007, at 10:01 PM, Fenggen Jia wrote:
>
>> hi,all,
>>       I am wondering why BGP has 4096 bytes limit on all its  
>> messages,should we leave it to the implementator to decide,thanks!
>
>
>Well, the problem with that is that a sender could then create an  
>arbitrarily long message, which could be arbitrarily problematic for  
>a receiver.
>
>Tony
>

_______________________________________________
Idr mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/idr

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 14 Jun 2007 09:53:40 -0400

On Thu, Jun 14, 2007 at 06:58:06PM +0800, Fenggen Jia wrote:
> Then why not other limit,how is 4096 bytes derived, thanks.

It wouldn't shock me if it had to do with the MTU of a FDDI link but
that's complete speculation.

Aside from the 4k limit, here's some things to keep in mind when writing
protocols.  Mostly IMO:
- Messages encoded in some sort of TLV format probably need a full PDU
  kept in memory at the same time.  This is because a sane
  implementation will validate the PDU contents against TLV constraints
  (either implied by the TLV or by the protocol) before doing *anything*
  to it.
- Having a fixed sized buffer means you can allocate it *once* and be
  done with it.  
- You're going to have one of these buffers for each of your peering
  sessions.  You don't want it to be large.
- You want the PDU to be "big enough" to be able to transport your
  information usefully.  If you're regularly exceeding a 4K message in
  BGP, you're probably not passing around Internet reachability.

-- Jeff

---

From: Enke Chen <[email-masked]>
Date: Thu, 14 Jun 2007 08:50:43 -0700

Fenggen Jia wrote:
> Then why not other limit,how is 4096 bytes derived, thanks.BTW,whether it's a good pratice in protocol devise to limit message sizes?

Do not know the precise reason for the number. I suspect that it's
chosen as it's large enough. It turns out to be a good number - it has
certainly withstood the test of time.

If there is ever a legitimate need to increase the limit in the future,
that can be done easily with the capability advertisement from the
protocol perspective.

-- Enke

> I see in ripV1 that the maximum PDU length is 576bytes its reason is to avoid IP fragmentation,but in ripng the PDU length is determined only by the link MTU,and in snmp i also doesn't see the message size limit. 
> Yours,Fenggen
>

---

From: "Vishwas Manral" <[email-masked]>
Date: Thu, 14 Jun 2007 11:27:53 -0700

Hi Jeff,

I had a few doubts.

>   (either implied by the TLV or by the protocol) before doing *anything*
Are you saying keeping/ allocating a PDU bigger than 4k bytes is a problem?

> - Having a fixed sized buffer means you can allocate it *once* and be
If I remember correctly what glibc does, it does not make a brk system
call everytime, it itself keeps a pool of memory, and does not give it
back to the OS. I think allocating a few buffers of different sizes
for such a design would help.

>   sessions.  You don't want it to be large.
Its again a memory management issue of an implementation right. We
could allocate a central pool and extract memory from there.

Thanks,
Vishwas

---

From: Erblichs <[email-masked]>
Date: Thu, 14 Jun 2007 15:25:11 -0700

Group,

	If one decided to pre-allocate fix buffer sizes,
	 then..
	What is wrong with a pre-allocating a buffer size 
	 of say 128bytes and assume that it will cover 90+%
	 of your needs and on the other 10% of the time,
	 allocate a buffer larger on demand??

	I believe this is legacy..

	However, if one likes to look at memory allocators
	then the "Slab Allocator" would be one I would suggest
	to look at first.

	Mitchell Erblich
	----------------

---

From: Danny McPherson <[email-masked]>
Date: Thu, 14 Jun 2007 16:36:52 -0600

In RFC 1105 the maximum message size was 1024 octets.  I suspect
because RFC 1163 added the capability to support multiple prefixes
(i.e., compression/update packing) they bumped the size to 4096. Note
that RFC 1163's Appendix 2 states the UPDATE format was changed
to support multiple "path attributes", when I think they actually meant
multiple "prefixes".

As for why it was constrained in the first place, RFC 1164 talks to this
a bit in section 6.3.  In particular, I suspect the size was constrained
initially because of this:

    Due to the stream nature of TCP, all the data for received messages
    does not necessarily arrive at the same time
    [...]
    This can make it difficult to process the data as messages,
    especially on systems such as BSD Unix where it is not possible to
    determine how much data has been received but not yet processed.

One might hypothesize that the selection of the original value of 1024
had something to the 4.2 BSD-derived systems TCP segment size
common settings of 1024.

RFC 1267 incorporated some of the RFC 1164 text directly, in section
5.2.

Of course, I'm just a youngster compared to some of the old folk here,
especially Yakov, so I'll happily defer  ;-)

-danny

---
RFC 1267

5.2  Processing Messages on a Stream Protocol

    BGP uses TCP as a transport mechanism.  Due to the stream nature of
    TCP, all the data for received messages does not necessarily arrive
    at the same time. This can make it difficult to process the data as
    messages, especially on systems such as BSD Unix where it is not
    possible to determine how much data has been received but not yet
    processed.

    One method that can be used in this situation is to first try to  
read
    just the message header. For the KEEPALIVE message type, this is a
    complete message; for other message types, the header should  
first be
    verified, in particular the total length. If all checks are
    successful, the specified length, minus the size of the message
    header is the amount of data left to read. An implementation that
    would "hang" the routing information process while trying to read
    from a peer could set up a message buffer (4096 bytes) per peer and
    fill it with data as available until a complete message has been
    received.

---

From: Erblichs <[email-masked]>
Date: Thu, 14 Jun 2007 15:58:53 -0700

Danny McPherson, eta al,

	Let me take a stab at this, but do your own investigation..

	IMO, one item is due to the use of TCP by BGP. After one
	RTO timeout, most implimentations use a slow-start after
	idle. To preserve TCP's fast transmit and power of two
	mem allocation the stardard link's MTU is 1.5K, times four
	with a reduction to the closest power of two, thus 4K.

	Oh, and the recieve window size is normally 4 x MTU which
	is then capped at 6K for slow-start.

	Mitchell Erblich
	----------------

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jun 2007 01:20:50 -0400

In message <[email-masked]>
> could allocate a central pool and extract memory from there.

Yes and no.  Dynamic allocation is a wonderful thing but you have to
allocate memory and post a read on every BGP session simultaneously.
There is no way to know which one will have updates to send next.  You
should post a big enough read that TCP and your BGP implementation can
operate efficiently if you instantaneously go from quiescent to a
torrent of BGP updates.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jun 2007 01:36:37 -0400

In message <[email-masked]>
> especially Yakov, so I'll happily defer  ;-)

Danny,

The code back then in rcp_routed and gated, the research reference
implementations (and running code on the NSFNET), was not written by
experts in BSD socket programming.  The code had problems with partial
reads of a BGP update and had even more trouble with a block on the
write (just gave up and reset the BGP session).  It did sort of work
though and implementations have gotten better.

Anyway, we still get lots of prefixes into a BGP update so unless
there is some motivation to increase the 4KB update size limit why
bother?  Maybe it would be more efficient to collect up all the prefix
changes with a common set of attributes and send the whole thing no
matter how big.  If so, lets continue the conversation when someone
has strong evidence that this is the case.  Until then...

Regards,

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Jun 2007 02:05:42 -0400

> 	is then capped at 6K for slow-start.

Not even close.

The BGP update size has nothing to do with the TCP socket buffer size
that you set with setsockopt SO_SNDBUF and SO_RCVBUF.

The BGP update size can be 4K and the socket buffer size can be 16KB
or 64KB (or larger with modern TCP).  Or it could be smaller but you
wouldn't want to do that.

And the MTU of concern then was FDDI or DS3/HSSI (4372? something >4KB).

Also the MTU and TCP MSS are not quite the same thing.

Curtis