Subject: Re: [Idr] draft-ymbk-bgp-extended-messages-01

From: Jakob Heitz <[email-masked]>
Date: Sun, 10 Apr 2011 14:15:21 -0400

The length field follows the marker. If the first byte of the length field can be 0xff, it reduces the value of the marker. The value of the marker is to validate the beginning of a message. This is good both in protocol operation as well as for debugging.

To preserve the value of the marker, I propose the maximum message length to be 65279 (0xfeff).

--
Jakob Heitz.

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 10 Apr 2011 20:23:12 +0200

Hi Jakob,

Per RFC4271 length field indicates the total length of the message 
including the header which consists of the marker, length field itself 
as well as message type octet.

       Length:

          This 2-octet unsigned integer indicates the total length of the
          message, including the header in octets.

Conclusion:

Length includes the fixed marker field hence it will not reduce it's size.

Many thx,
R.

> The length field follows the marker. If the first byte of the length
> field can be 0xff, it reduces the value of the marker. The value of
> the marker is to validate the beginning of a message. This is good
> both in protocol operation as well as for debugging.
>
> -- Jakob Heitz. _______________________________________________ Idr
> mailing list [email-masked] https://www.ietf.org/mailman/listinfo/idr
>

---

From: Jakob Heitz <[email-masked]>
Date: Sun, 10 Apr 2011 14:50:21 -0400

Yeah, I know. Thats not what I meant.

You've never debugged a TCP stream before, have you?

--
Jakob Heitz.

---

From: Jeff Wheeler <[email-masked]>
Date: Sun, 10 Apr 2011 14:50:43 -0400

> Length includes the fixed marker field hence it will not reduce it's size.

Where Jakob writes "value" I am certain he means "usefulness" for
debugging, as if the marker is a magic string that could be used to
easily split up messages in a debug dump.  He doesn't mean there is
some upper bound on the length of the message caused by the marker
field.

The marker's intent was actually confused, as you can determine
somewhat easily by skimming the references to marker in the relevant
RFCs.  It could be a magic 16-bit field for detecting "loss of sync"
(RFC saying this, not me) or it could be for BGP's own authentication
mechanism (again, the specs talking.)  In reality, I think it's pretty
much useless as there aren't implementations taking advantage of it
for authentication, and messages are not encoded in such a way that
the value 0xFFFF cannot be present in a message.

Since marker actually cannot be used reliably to find message
boundaries for operation or debugging, I disagree with Jakob's
reasoning.  The maximum message length is arbitrary within the bounds
of a 16 bit field.

-- 
Jeff S Wheeler <[email-masked]>
Sr Network Operator  /  Innovative Network Concepts

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 10 Apr 2011 20:59:35 +0200

> You've never debugged a TCP stream before, have you?

Indeed. I am not that fast.

I always capture it first then debug the captured file. Finding marker 
demarkations is quite easy even if first octet length field is 0xFF. You 
just match on 16 octets of all 1s.

I really see no problem even if the 17th octet will also be all 1s.

Cheers.
R.

> On Apr 10, 2011, at 11:23 AM, "Robert Raszuk"<[email-masked]>  wrote:
>
>> Hi Jakob,
>>

---

From: Jakob Heitz <[email-masked]>
Date: Sun, 10 Apr 2011 15:00:01 -0400

I don't use the marker to sync the TCP stream. I use the length field. I do use the marker to verify.

--
Jakob Heitz.

On Apr 10, 2011, at 11:50 AM, "Jeff Wheeler" <[email-masked]> wrote:

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: Ilya Varlashkin <[email-masked]>
Date: Sun, 10 Apr 2011 21:30:32 +0200

> I don't use the marker to sync the TCP stream. I use the length field. I do use the marker to verify.

are you talking about situation where layer(s) above TCP lost message boundary and you're trying to find it again using length and marker?

/iLya

---

From: Jakob Heitz <[email-masked]>
Date: Sun, 10 Apr 2011 15:47:12 -0400

Yes, like that. I use the length field to find the message boundary and use the marker to verify it. Verifying things helps to avoid bugs.

--
Jakob Heitz.

---

From: Jeff Wheeler <[email-masked]>
Date: Sun, 10 Apr 2011 16:01:51 -0400

> I don't use the marker to sync the TCP stream. I use the length field. I do use the marker to verify.

I understand what you meant in your earlier post.  The debugging
method you describe is unreliable since the message encoding does not
ensure that the same bit-string would not be present within a message.
 Consider a message length mis-calculation that lands the message
parser at the start of an inner-message sent by the neighbor as a
debugging mechanism, or which lands you within a prefix, router ID,
community, or other field which can have arbitrary value.

Even a 16-bit CRC field in place of the marker would be superior (yet
still not totally "reliable") for the debugging method you describe,
but that isn't what we have.  Changing the message bit encoding would
allow for your method to become reliable by protecting the magic from
collision within the message data, but I doubt if anyone would think
that to be reasonable in terms of cost/benefit.

---

From: Jakob Heitz <[email-masked]>
Date: Sun, 10 Apr 2011 17:56:37 -0400

Please understand the word "verify"

--
Jakob Heitz. 510-566-2901

---

From: Jeff Wheeler <[email-masked]>
Date: Sun, 10 Apr 2011 20:06:17 -0400

> Please understand the word "verify"

Your method doesn't "verify" that the parser is beginning at start of
a message.  It can only detect some error conditions, and in others,
it will produce a false positive.  If you go back and consider your
own idea to encapsulate whole BGP messages within outer-messages for
remote-neighbor diagnostic purposes, you should immediately understand
why this is the case -- even if you have not yet considered the
possibility of an erroneous message length causing the parser to start
at an unlucky offset within an unlucky message payload.

What you are asking for is the length field not be allowed to be the
same two-octet value as the marker, assuming the marker is a constant,
for detecting "message length off by +1 or +2" errors.  There are
certainly other "off by N" cases where this will produce false
positives, depending on the message payload.  It is not guaranteed to
detect "off by -1 or -2" either.

It should be understood that this would only aid in diagnosing a
limited set of error circumstances.  It's really giving you a little
bit more context for "guessing," not "verification," as it can and
sometimes will produce the wrong result.  IMO your suggestion is not
without merit, but it works sometimes, not all the time.

There is no way to reliably recover "stream sync" in the BGP protocol,
if the sender is sending incorrect message lengths or truncated
message payload.  To say "verify" in the context of ensuring you are
parsing from the start of a message implies this, and that is simply
not correct, even diagnostically.

---

From: Jakob Heitz <[email-masked]>
Date: Sun, 10 Apr 2011 22:22:59 -0400

Verify:

If verification fails, you know it's bad.
If it passes, you have a better chance it's good.
Not 100%.

This has value. It helps to make code robust.

It's like, you provide a photo ID when you pay
with a credit card. If the photo isn't you,
you fail. If the photo resembles you, you might
still have forged it or disguised yourself.
Photo IDs are used anyway.

--
Jakob Heitz.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Jeff Wheeler
> Sent: Sunday, April 10, 2011 5:06 PM
> To: [email-masked]
> Subject: Re: [Idr] draft-ymbk-bgp-extended-messages-01
>

---

From: Randy Bush <[email-masked]>
Date: Tue, 12 Apr 2011 06:07:11 -0700

> field. I do use the marker to verify.

this would all be simpler if bgp used a reliable transport such as tcp

randy

---

From: Jeff Wheeler <[email-masked]>
Date: Tue, 12 Apr 2011 12:07:56 -0400

> this would all be simpler if bgp used a reliable transport such as tcp

If the BGP implementation on one side of a session makes an error in
calculating the length of a message, or truncates a message which
should be sent, "stream sync" is lost.  These posts have all been
about the upper-bound of the message length field's affect on the
ability to debug this situation.

If you haven't done much programming with [ length + data | length +
data ] type transactions over TCP, I understand how you could be
confused about this.  Everyone knows the message stream is transported
over TCP.  No one is suggesting any attempt to try to improve upon the
already satisfactory reliability of the session transport.

To make this simple, let's say the marker is always "Z," and the
current message length can be 0-9A-H.  Data inside the message can be
any number or letter.  Right now, this is basically how BGP works; I
can send you:
Z5HELLOZ5OOPSZ4OKAYZ5WORLD
and you can diagnostically detect "off by 1" message lengths received
from the neighbor, because your parser will always be looking at the
length of the next message, which must be preceded by a Z, and may not
itself be a Z.

If the maximum message length is allowed to be 65535 (or really,
greater than 65535-256) it is essentially possible for the message
length itself to be "Z," so you could not detect "off by 1" in some
cases, such as:
Z5OOPSZZSUPERCALIFRAGILISTICEXPIALIDOCIOUS
Why?  Because the parser, in dealing with the above messages, would
think the second message had length S followed by data UPERCALI... and
believe the third letter Z was actually a marker, when in fact the
second one is the marker and the third Z is the length of the second
message.

As I have posted before, I believe Jakob's suggestion has merit but
it's difficult to understand why if you do not have any implementation
experience.  It's worth wrapping your head around before you form an
opinion.

---

From: Jakob Heitz <[email-masked]>
Date: Tue, 12 Apr 2011 12:45:45 -0400

> this would all be simpler if bgp used a reliable transport such as tcp

I think you meant SCTP.

What's stopping you from running yours on SCTP?

--
Jakob Heitz.

---

From: Randy Bush <[email-masked]>
Date: Tue, 12 Apr 2011 12:41:00 -0700

>> this would all be simpler if bgp used a reliable transport such as tcp

i did not.  i meant precisely what i wrote.

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 12 Apr 2011 22:03:41 +0200

Hi Randy/all,

I think the main question here to ask is do we really need an ability to 
detect message boundaries at BGP level or not.

Historically when any BGP message was bad (errors of any sort etc ...) 
we dropped the session and restarted. Life was good.

Now there is more and more proposals to break this original design as 
the price seems too high to impact 1000s users with one attribute error.

With this in mind what Jakob and Jeff are saying does make sense. In 
fact I think it will make sense even more when we see someone to propose 
not to drop the session under the cases if BGP update msg boundaries 
were lost. Those are BGP level errors .. not transport errors like your 
reply may have had suggested.

So we either never allow first length octet to be 0xFF or make sure 
there is some spacing of few octets of all zeros between BGP messages. 
Then parsing on N x 0x00 + 16 x 0xFF would have a chance to determine 
the BGP message boundary. As a side note the replay of message would not 
contain the spacing hence it would not be misinterpreted by any parsing 
routines.

Rgs,
R.

---

From: Keyur Patel <[email-masked]>
Date: Tue, 12 Apr 2011 13:29:52 -0700

Robert,

BGP message length errors are hard to recover from particularly if these
errors are due to some form of corruption. One never knows what else is
messed up once such a corruption happens. If you ever want to solve such
cases you probably want to consider soft-notify like mechanisms although
they don't solve all cases.

Reserving first octet of length field is not sufficient as it doesn't detect
all the cases.

Regards,
Keyur

---

From: Randy Bush <[email-masked]>
Date: Tue, 12 Apr 2011 13:30:40 -0700

Damn shame we can't use reliable transport and a length field

randy, on iPhone

On Apr 12, 2011, at 13:03, Robert Raszuk <[email-masked]> wrote:

> Hi Randy/all,
> 
> I think the main question here to ask is do we really need an ability to detect message boundaries at BGP level or not.
> 
> Historically when any BGP message was bad (errors of any sort etc ...) we dropped the session and restarted. Life was good.
> 
> Now there is more and more proposals to break this original design as the price seems too high to impact 1000s users with one attribute error.
> 
> With this in mind what Jakob and Jeff are saying does make sense. In fact I think it will make sense even more when we see someone to propose not to drop the session under the cases if BGP update msg boundaries were lost. Those are BGP level errors .. not transport errors like your reply may have had suggested.
> 
> So we either never allow first length octet to be 0xFF or make sure there is some spacing of few octets of all zeros between BGP messages. Then parsing on N x 0x00 + 16 x 0xFF would have a chance to determine the BGP message boundary. As a side note the replay of message would not contain the spacing hence it would not be misinterpreted by any parsing routines.
> 
> Rgs,
> R.
> 
>

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 12 Apr 2011 22:37:29 +0200

Hi Keyur,

 > Reserving first octet of length field is not sufficient as it doesn't
 > detect all the cases.

Indeed .. hence my suggestion of extra zeros spacing between messages. 
Then length field could be of any value. What do you and others think 
about it ?

The alternative would be to always drop the session/inject soft-notify 
etc ... if message boundaries can not be determined.

Cheers,
R.

> On 4/12/11 1:03 PM, "Robert Raszuk"<[email-masked]>  wrote:
>
>> Hi Randy/all,
>>

---

From: Keyur Patel <[email-masked]>
Date: Tue, 12 Apr 2011 14:14:38 -0700

Hi Robert,

Sure. IMHO, we should NOT be limiting length field for this purpose.
Anything else probably should be handled as a separate draft.

Regards,
Keyur

---

From: Ilya Varlashkin <[email-masked]>
Date: Tue, 12 Apr 2011 23:39:39 +0200

On Apr 12, 2011, at 22:37 , Robert Raszuk wrote:

> Indeed .. hence my suggestion of extra zeros spacing between messages. Then length field could be of any value. What do you and others think about it ?
> 

As an alternative to non-FF octet following the marker, we could add some padding attribute (provided we use MP-BGP style Attr 14/15 for conveying NLRI) that would be last attribute in an UPDATE message and contain sufficient number of zeroes. Ironically this will be similar to decreasing max value of length, but may be more helpful in recovering lost stream sync. So if stream sync is common issue, then one way or another effective max message size will be decreased. Whether to use length field for that purpose or adding padding, people more close to the code should say what's more efficient/useful. From protocol clarity perspective length field should not be cannibalized (thinking of analogy with Type/Length in Ethernet header).

> The alternative would be to always drop the session/inject soft-notify etc ... if message boundaries can not be determined.
> 
Yet another alternative - send 5 KEEPALIVES one after another :-) N.B.: In every joke there is a part which is a joke.

Cheers,
iLya

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 13 Apr 2011 00:12:11 +0200

Hi Ilya,

>> Indeed .. hence my suggestion of extra zeros spacing between
>> messages. Then length field could be of any value. What do you and
>> others think about it ?
>
> As an alternative to non-FF octet following the marker, we could add
> some padding attribute (provided we use MP-BGP style Attr 14/15 for
> conveying NLRI) that would be last attribute in an UPDATE message and
> contain sufficient number of zeroes.

The entire trick here is that such spacing should not be within the 
update message boundaries as if it would all bets are off.

>> The alternative would be to always drop the session/inject
>> soft-notify etc ... if message boundaries can not be determined.
>>
> Yet another alternative - send 5 KEEPALIVES one after another :-)
> N.B.: In every joke there is a part which is a joke.

That is in fact not that much of a joke ! The inter message spacing I 
had in mind was just to be a new BGP message called ZERO Msg. It would 
be of fixed size and consist of only N octets of all zeros (Anti-Marker).

But before we go there I think folks on the list need to really express 
if BGP message boundary parsing at the BGP level is really required. 
Otherwise checking even 16 octets of all 1s followed by the msg length 
should be sufficient especially as an extra safeguard regardless of what 
the 17th octet contains.

Cheers,
R.

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 12 Apr 2011 19:46:06 -0400

> ability to debug this situation.

Getting a TLV protocol right is not rocket science.

I think everyone uses the length to figure out where the next packet
should start and then checks for the marker as you've described.

Have BGP implementation really become that bad lately?

> already satisfactory reliability of the session transport.

[... example snipped ...]

btw - I don't think Randy is at all confused about this.  Terse
sarcasm doesn't always get through email as intended (maybe we need a
new MIME type text/sarcasm - oh - more multipart messages, never mind)

Perhaps it would be best to 1) remove the possibility of
unidirectional extended message capability described in the draft and
2) bound the size of normal messages such that an entire message can
easily fit into a NOTIFICATION message (ie 65K - 256).

Reinventing HDLC in BGP (or PPP byte stuffing) seems like a bad idea.
An implementation that can't count bytes and build a TLV right
deserves a CEASE (and a long hold down).

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 12 Apr 2011 20:05:36 -0400

In message <[email-masked]>
> etc ... if message boundaries can not be determined.

If you really wanted it to work you would need to reinvent PPP byte
stuffing for BGP.  And *no* I don't want to go there.

Curtis

---

From: Shane Amante <[email-masked]>
Date: Tue, 12 Apr 2011 19:51:36 -0600

> Damn shame we can't use reliable transport and a length field

... and, TCP-MD5 on top of that.

-shane

---

From: Randy Bush <[email-masked]>
Date: Wed, 13 Apr 2011 19:41:10 +0900

> Getting a TLV protocol right is not rocket science.

no, but understanding it seems to be

fwiw, i do not have serious problems with the capability being
bi-directional.  my co-authors may.

randy

---

From: Randy Bush <[email-masked]>
Date: Wed, 13 Apr 2011 19:44:53 +0900

> stuffing for BGP.  And *no* I don't want to go there.

those who want to put start/end-record flags in the stream should be
forced to run and debug slip for a month.

randy

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 13 Apr 2011 15:28:12 -0400

> bi-directional.  my co-authors may.

What I meant by bi-directional is both sides have to be able and
willing to receive extended-messages for either side to send them.
That way an entire packet can always be sent in the reverse direction
when indicating that something was wrong with it.

Curtis

---

From: Keyur Patel <[email-masked]>
Date: Wed, 13 Apr 2011 12:31:57 -0700

Curtis,

> when indicating that something was wrong with it.

If a BGP sending speaker sees a need to receive complete encapsulated
packet, it can always send original packet with size less than 65K. You
don't need capability to be *bi-directional*. There are cases where
asymmetric capability may be needed.

Regards,
Keyur

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Apr 2011 04:36:34 +0900

> What I meant by bi-directional

was pretty obvious

> when indicating that something was wrong with it.

i do not care what is sent.  all these payload issues are a layer above
pdu size.  but i will admit to great amusement at wanting to transport a
full error frame of an erroneous full error frame reporting a full error
frame and so on ad infinitum.

randy

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 13 Apr 2011 23:34:17 -0400

In message <[email-masked]>
Keyur Patel writes:
> packet, it can always send original packet with size less than
> 65K. You don't need capability to be *bi-directional*. There are cases
> where asymmetric capability may be needed.

And what are those cases?

---

From: Keyur Patel <[email-masked]>
Date: Wed, 13 Apr 2011 20:44:06 -0700

> And what are those cases?

SIDR requirement being the prime one. Any other applications that require
BGP to send longer length messages.

Regards,
Keyur

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 14 Apr 2011 00:59:06 -0400

> BGP to send longer length messages.

You still haven't indicated why any usage of BGP would need the long
message capability to be uni-directional.

Please explain why the other direction cannot send extended messages.

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Apr 2011 14:04:54 +0900

Allow simplex edge site bgpsec router not to need large receive buffers 
-- 
sent from android, so terse

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Apr 2011 08:28:52 +0200

> Allow simplex edge site bgpsec router not to need large receive buffers

So the "simplex edge site bgpsec router" will be capable of sending 
large messages and not be able to receive them ?

IMHO it is always a bad idea to maintain different defaults for routing 
protocols on a per platform basis.

R.

---

From: Claudio Jeker <[email-masked]>
Date: Thu, 14 Apr 2011 08:36:52 +0200

> BGP to send longer length messages.

BGP MPLS VPN in some setups hit limits if the list of export-targets
become huge. In the end a 4k message can only hold around 500 ext.
communities. Now if this is a valid reason for larger messages is beyond
me.  Plus I'm waiting for the first person to prepend his AS 990 times :) 

-- 
:wq Claudio

---

From: john heasley <[email-masked]>
Date: Thu, 14 Apr 2011 06:56:48 +0000

> large messages and not be able to receive them ?

yes, but if cares to receive "full errors", it restricts itself to the
peer's limit.

> IMHO it is always a bad idea to maintain different defaults for
> routing protocols on a per platform basis.

yes, this is silly.  it appears to be engineering around limitations of a
specific piece of hardware.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Apr 2011 09:13:00 +0200

Hi John,

> yes, but if cares to receive "full errors", it restricts itself to the

Let's put the "full errors" case aside.

The assumed topology here is:

simplex edge CE ----- ebgp -----  PE of ISP

So CE can send messages > 4K and can not receive such from PE. My 
question was what would be in this simplex CE generated BGP message to 
require it to be > 4K ?

This seems to be a reason to keep asymmetry in place which just like 
Curtis I am trying to better understand.

> specific piece of hardware.

As a matter of fact those are imaginary limitations to start with. I am 
yet to see a platform which can run BGP and which would have any issue 
to transiently handle 65K vs 4K messages. RP receive buffers are carved 
out of RP DRAM and those today are measured in gigabytes for BGP systems.

Cheers,
R.

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Apr 2011 17:15:07 +0900

> large messages and not be able to receive them ?

yep.  which is why it is referred to as a simplex relationship

> routing protocols on a per platform basis.

who said per-platform?

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Apr 2011 17:16:38 +0900

> yes, this is silly.  it appears to be engineering around limitations
> of a specific piece of hardware.

no.  it is telling the edge site, your paying customer, that they can
secure their prefix without upgrading hardware.

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Apr 2011 17:17:56 +0900

> RP receive buffers are carved out of RP DRAM and those today are
> measured in gigabytes for BGP systems.

perhaps you should look into what the edge enterprises have bought from
your company.

randy

---

From: Claudio Jeker <[email-masked]>
Date: Thu, 14 Apr 2011 10:37:00 +0200

> perhaps you should look into what the edge enterprises have bought from

If they run BGP on the edge and don't have even 1M of memory left to
handle 10 65k buffers then they should reconsider their HW decision.
Sorry but a BGP router with so little free memory where using bigger
receive and send buffer causes memory starvation will not survive the
winter and most probably not even the next few weeks.

If edge customers want to secure their prefix they need to update the
route OS and with that they can get fat BGP messages for free. I see no
reason to make something that could be so simple complex just for the heck
of it.

-- 
:wq Claudio

---

From: Randy Bush <[email-masked]>
Date: Thu, 14 Apr 2011 17:47:35 +0900

>reason to make something that could be so simple complex just for the

As I said, I am not strongly against symmetric. I am trying to explain one place where it would be useful. As an operator, I do not like to tell the customer they need to buy more hardware to use my service. Money goes to wrong place :)
-- 
sent from android, so terse

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Apr 2011 13:13:12 +0200

> secure their prefix without upgrading hardware.

Can anyone in IDR or SIDR demystify for us here what securing BGP really 
requires (certificates, signatures, attestations you name it)  if to 
secure a single prefix originated by customer site requires more then 4K 
of BGP message size ?

We already know that update packing is gone and that there is going to 
be single NLRI per update. OK or NOK but a different debate for 
different time.

But if securing 1 prefix really requires more then 4K of data attached 
to it I think we should question the entire approach rather then waist 
time to argue about draft-ymbk-bgp-extended-messages. Well unless this 
secure BGP is the only reason for this draft ;-).

Thx,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 14 Apr 2011 14:07:30 +0200

I would not reply if I wouldn't verify first.

And all routers even those which have no BGP support do support RFC1323. 
That means that their receive buffers not only support 65,535 bytes 
messages, but with described in this RFC new TCP option the max message 
size can be increased to 1,073,741,823 bytes

If you want to read more or see a list of platforms please google for 
"tcpwslfn.html"

Example from little 1801 in my home office:

SIMPLEX_EDGE(config)#ip tcp window-size ?
   <68-1073741823>  Window size

Conclusion: Edge enterprises do not need to purchase any new hardware to 
support draft-ymbk-bgp-extended-messages if only new software will be 
available for their platform.

R.

---

From: Keyur Patel <[email-masked]>
Date: Thu, 14 Apr 2011 07:23:45 -0700

Curtis,

> Please explain why the other direction cannot send extended messages.

Typically, most implementations terminate BGP sessions upon *receiving*
messages longer than current 4096 octets. This is a receiving side behavior
(Sending side speaker would automatically build another message beyond 4096
octets). The draft aims to relax this part and hence the capability being
asymmetric (specifically targeting the receiving side behavior).

Its not that other direction "cannot send extended messages". Its more like
other direction does NOT need to enable this capability unless required.

Hope that clarifies.

Regards,
Keyur

---

From: Danny McPherson <[email-masked]>
Date: Thu, 14 Apr 2011 14:16:40 -0400

> secure their prefix without upgrading hardware.

Hrmmm...  Can someone explain to me what an "edge site" is in the 
context of BGP and protocol interactions,?

danny@pork% grep -i "edge" rfc4271.txt 
   2. Acknowledgements ................................................6
2.  Acknowledgements
   We would like to specially acknowledge numerous contributions by
   We would like to specially acknowledge Andrew Lange for his help in
   acknowledgement, and sequencing.  BGP listens on TCP port 179.  The
         because once an IP packet arrives at the edge of a group of
Acknowledgement

And for those not paying attention, the new SIDR Charter essentially gives them the authority to "enhance" BGP, so long as it's co-WG last called in IDR (even though the new charter doesn't even provide that guidance).  So if you're not subscribed (and care)....

-danny

---

From: john heasley <[email-masked]>
Date: Fri, 15 Apr 2011 05:50:00 +0000

> > routing protocols on a per platform basis.

i believe he is implying that a platform [limitation] is the most likely
reason for the non-congruent message size capability.

besides a major design flaw, why would any bgp speaker not be capable of
handling a 65k msg?

---

From: Randy Bush <[email-masked]>
Date: Fri, 15 Apr 2011 15:03:52 +0900

> besides a major design flaw, why would any bgp speaker not be capable
> of handling a 65k msg?

when ram is one of the most frequent causes of having to spend more
money to upgrade routers, i tend toward conservative.

randy

---

From: Claudio Jeker <[email-masked]>
Date: Fri, 15 Apr 2011 09:24:33 +0200

> money to upgrade routers, i tend toward conservative.

The long term cost of a bad RFC are magnitudes bigger than the cost of
upgrading those routers that you try to save (and which will be
replaced soon because of some other limit they hit).

-- 
:wq Claudio

---

From: Jeff Wheeler <[email-masked]>
Date: Fri, 15 Apr 2011 09:50:15 -0400

> money to upgrade routers, i tend toward conservative.

Frankly, Randy, this is either an unbelievably ignorant statement, or
an empty argument posted to waste readers' time.  If a router doesn't
have 64KB of memory to buffer an incoming message, its resource
constraints are certainly such that it has no memory for additional
code required to implement new features, or to store new state
information, resulting from any extensions to BGP.

I would love to see the practical example of a BGP router which
operates correctly today, can be extended to support new extensions,
yet will malfunction if its message buffer must be increased from 4KB
to 64KB.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 15 Apr 2011 17:58:49 +0200

Hi Stephen,

This is really great explanation. I did try to find this in some papers
and your slides but I could not.

So it confirms that even if we use "worse case" BGP update advertised by
customer to provider (1 hop only case) it will fit within 4K limit
easily. Of course for any subsequent advertisement we need increase the
message size and this is where the draft-ymbk-bgp-extended-messages
addresses the issue.

One sentence in your reply is still not clear:

> If on users RSA, each signature would be at least 1K bytes, and might
> be 2K bytes, depending on the key length size chosen. So, a 20-hop
> path could yield a 20-40K set of sigs, independent of the other path
> security data.

- What is the maximum key length possible and how big would be the RSA 
signature with such key length ?

- What are the other path security data and what their size might be 
min-max.

Many thx again,
R.

> At 1:13 PM +0200 4/14/11, Robert Raszuk wrote:
>>> no. it is telling the edge site, your paying customer, that they
>>> can secure their prefix without upgrading hardware.
>>
>> Can anyone in IDR or SIDR demystify for us here what securing BGP
>> really requires (certificates, signatures, attestations you name
>> it) if to secure a single prefix originated by customer site
>> requires more then 4K of BGP message size ?
>
> BGPSEC calls for each AS hop along a path to sign AS path info.
> Although the average path length is a bit less than 4 hops, there are
> some very long paths that appear in FIBs, e.g., over 20 hops. The
> desire to increase the max UPADTE size (from the current 4K limit) is
> intended to accommodate very long paths. The principle contributor to
> the size increase is the digital signature. If on users RSA, each
> signature would be at least 1K bytes, and might be 2K bytes,
> depending on the key length size chosen. So, a 20-hop path could
> yield a 20-40K set of sigs, independent of the other path security
> data.
>
> The good news is that many long paths contain repeated AS#s, which
> could be collapsed into a single signature. But, that optimization
> has not yet been explored. Also, if we were to use DSA or ECDSA as a
> signature algorithm, instead of RSA, the signature size could drop to
> 128 or 256 bytes, from 1-2K, a savings of a factor of 4-8. Still, a
> 20-hop path, with no repeats, might not fit in a 4K UPDATE, with all
> of the other secruity data.
>
> Hope that explanation helps.
>
> Steve
>

---

From: Stephen Kent <[email-masked]>
Date: Fri, 15 Apr 2011 11:46:24 -0400

At 1:13 PM +0200 4/14/11, Robert Raszuk wrote:
>really requires (certificates, signatures, attestations you name it) 
>if to secure a single prefix originated by customer site requires 
>more then 4K of BGP message size ?

The good news is that many long paths contain repeated AS#s, which 
could be collapsed into a single signature. But, that optimization 
has not yet been
explored. Also, if we were to use DSA or ECDSA as a signature 
algorithm, instead
of RSA, the signature size could drop to 128 or 256 bytes, from 1-2K, 
a savings of a factor of 4-8. Still, a 20-hop path, with no repeats, 
might not fit in a 4K UPDATE, with all of the other secruity data.

Steve

---

From: John Scudder <[email-masked]>
Date: Fri, 15 Apr 2011 10:11:27 -0700

May I suggest we wind down this thread and move conversation to the -02 thread?  (Hint: -02 seems to have addressed the issues being currently discussed.)

--John

---

From: Stephen Kent <[email-masked]>
Date: Fri, 15 Apr 2011 15:12:12 -0400

Robert,

First let me apologize as I accidentally put in "bytes" where I meant 
"bits" in the examples I provided. Whoops!  RSA 2K keys are 2K bits, 
i.e., 256 bytes (not 2K bytes). And, 2K RSA sig is the default, as 
this matches the key sizes we have already agreed upon for the RPKI 
certs. Still, a 20-hop path with RSA 2K (bit) sigs exceeds the 4K 
(byte) max UPDATE size, without any other overhead.  Sorry for the 
confusion.

>
>- What is the maximum key length possible and how big would be the 
>RSA signature with such key length ?

There is no max key size, but it does not make sense to push for bigger RSA
key sizes, instead of moving to more efficient (in space and 
computation) sig algorithms. The sig size for RSA is the same size as 
the key.

The likely sucessor algs are DSA or EC-DSA, which use smaller keys, 
but offer secruity equivalent to the larger RSA key sizes. The key 
sizes for those algorithms yield 128 or 256-bit sigs, under current 
hash algs.

>- What are the other path security data and what their size might be min-max.

I defer to Matt Lepinski for the details of the other data, as he is 
the author of the BGPSEC protocol doc.

Steve

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 16:01:31 -0400

> Allow simplex edge site bgpsec router not to need large receive buffers 

You can receive 64KB BGP packets over TCP with a 4KB or 8KB TCP
receive window but it has to be reassembled in user space.
Prioritzing sockets for which some segments have been received would
be an enabler to keep the number of active 64KB BGP buffers low.
Presumably these really small routers have very few BGP sessions,
usuaully just one and one times 64KB is not a big number even for
implementations with very tiny amounts of RAM.

I could see this argument maybe if we still had 20 year old IGS
routers in the network, but I don't think they will be running the
latest code.

Curtis

---

From: Geoff Huston <[email-masked]>
Date: Sat, 16 Apr 2011 06:09:10 +1000

On 16/04/2011, at 5:12 AM, Stephen Kent wrote:

> Robert,
> 
> First let me apologize as I accidentally put in "bytes" where I meant "bits" in the examples I provided. Whoops!  RSA 2K keys are 2K bits, i.e., 256 bytes (not 2K bytes). And, 2K RSA sig is the default, as this matches the key sizes we have already agreed upon for the RPKI certs. Still, a 20-hop path with RSA 2K (bit) sigs exceeds the 4K (byte) max UPDATE size, without any other overhead.  Sorry for the confusion.
> 
>> 
>> - What is the maximum key length possible and how big would be the RSA signature with such key length ?
> 
> key sizes, instead of moving to more efficient (in space and computation) sig algorithms. The sig size for RSA is the same size as the key.
> 
> The likely sucessor algs are DSA or EC-DSA, which use smaller keys, but offer secruity equivalent to the larger RSA key sizes. The key sizes for those algorithms yield 128 or 256-bit sigs, under current hash algs.
> 
> I defer to Matt Lepinski for the details of the other data, as he is the author of the BGPSEC protocol doc.

I was doing a similar mental sum in my head and 256 bytes per sig is not a lot. As far as I am aware the only other added per-AS part of the attribute is the SKI of the public key. Presumably this is also 256 bytes.

SO thats 500 bytes per AS, and for a 20 AS path thats 10K in additional data.

Is my back of the envelope anywhere near in the right ball park?

Geoff

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 17:04:19 -0400

> other direction does NOT need to enable this capability unless required.

Actually no that doesn't help.  It sounds like it can be assymetrical.
It is defined to be assymetrical.  But is has no solid reason that it
needs to be assymetrical.

It is also perfectly valid to define the capability such that both
ends have to be known to be capable before either end sends a very
long BGP packet.

Curtis

---

From: Randy Bush <[email-masked]>
Date: Sat, 16 Apr 2011 06:11:49 +0900

> It is defined to be assymetrical.

you may want to read the current draft

randy

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 18:44:30 -0400

In message <[email-masked]>
> secure their prefix without upgrading hardware.

Randy,

A router that can't support a 64KB buffer must have been purchased at
WallyWorld and can't be much of an investment to toss.

Wasn't the 64Kbit DRAM chip commercialized in about 1978?  Even the
IGS and AGS routers circa early 1990s had whole 4 MB of DRAM.

Are we talking about DSL modems with only on-chip SRAM running BGP?

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 18:47:16 -0400

Randy,

I'll get back to you after I check with grandma as to whether she is
running BGP and feels the need to secure her announced routes.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 19:03:21 -0400

In message <[email-masked]>
> money to upgrade routers, i tend toward conservative.

I always thought that had more to do with 300,000+ global BGP routes,
not the socket buffer size.  [Is grandma running full routes on her
linksys dsl modem?  I'll have to check with grandma again.]

I also mentioned a few times that you can support 64K application
messages over TCP with socket buffers of less than 64K.  OTOH, right
now 56K and 64K are the most common defaults so I seriously doubt
there is any valid memory concern.

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 19:12:19 -0400

> Hope that explanation helps.

Steve,

I would venture to guess that almost everyone on this thread knows of
this motivation for extended-message.

The question being debated is whether having extended-message in one
direction but not the other direction serves any legitimate purpose.

Curtis

BTW- SIDR dropped from the Cc so as not to cross post and also since I
dropped off SIDR quite a long time ago.

---

From: Randy Bush <[email-masked]>
Date: Sat, 16 Apr 2011 08:47:26 +0900

> direction but not the other direction serves any legitimate purpose.

no.  only those who have not read the current draft are debating that.

randy

---

From: Curtis Villamizar <[email-masked]>
Date: Fri, 15 Apr 2011 21:32:00 -0400

In message <[email-masked]>
> you may want to read the current draft

OK.  thanks.  I stand corrected.  the unidirectionalness is out.  (is
that a word?)

Curtis

---

From: John Leslie <[email-masked]>
Date: Sat, 16 Apr 2011 08:31:55 -0400

> you may want to read the current draft

   Presumably Randy refers to:
] 
] To advertise BGP Extended Message Capability to a peer, a BGP speaker	
] uses BGP Capadbilities Advertisement [RFC3392].  By advertising the	
] BGP Extended message Capability to a peer, a BGP speaker conveys to	
] that peer that the speaker may send, and is capable of receiving and	
] properly handling, BGP Extended Messages.

   I really dislike this new "speaker may send" language. (I have no
problem requiring the capability be symmetric; it's the "may" I dislike.)

   The capability being advertised is receiving and handling extended
length. If I understand correctly, we mean to require that a speaker
advertising this capability MUST also be able to send extended length
(though I'm less than clear how we could tell).

   "may send" doesn't convey that, and puts us in the MUSTard-land of
"what does lower-case 'may' mean?"

   Also, it's too easy to misinterpret to suggest this speaker might
send extended-length without waiting for receipt of the same advertisement
from its peer. (Yes, one would have to be clue-challenged to think that;
but I keep meeting folks that clue-challenged: why invite the problem?)

   I suggest language closer to "capable of both receiving and sending
BGP Extended Messages".

--
John Leslie <[email-masked]>

---

From: Jeffrey Haas <[email-masked]>
Date: Sun, 17 Apr 2011 13:23:45 +0000

Jakob,

> To preserve the value of the marker, I propose the maximum message length to be 65279 (0xfeff).

While I strongly sympathize with the usefulness of the marker field for
purposes of re-detecting lost framing in BGP PDUs in a TCP stream, I think
the property you're really looking for is predictability of value.  As such,
I don't think we want to preclude any particular value for the message
length.

As an observation, the message type is a well constrained value that is
unlikely to ever jump over 128 for hopefully most of our lifetimes.  This
should assist in the necessary bit pattern matches you might be looking for.

As another observation, the marker field is an appropriate place for any
UPDATE sequence numbering should we ever decide to use such a mechanism for
one of our recovery mechanisms for bad BGP UPDATEs.  (This has been mostly
IETF hallway conversation rather than any formal proposal at this time.)

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Sun, 17 Apr 2011 22:56:15 +0000

Followup to my own comment.

> I don't think we want to preclude any particular value for the message

Another observation is that if the trailing NLRI contains 0xff in the byte
stream, we still have a variant of this issue.

-- Jeff

---

From: Randy Bush <[email-masked]>
Date: Mon, 18 Apr 2011 08:26:35 +0900

> stream, we still have a variant of this issue.

all the lessons of SLIP twenty years later.  nancy reagan was right.

randy

---

From: Randy Bush <[email-masked]>
Date: Mon, 18 Apr 2011 12:15:10 +0900

> Another observation is that if the trailing NLRI contains 0xff in the
> byte stream, we still have a variant of this issue.

cosmic ray hits length but misses next 2 megabytes!  news at 11.

randy