Subject: [Idr] A slightly different perspective ....

From: Robert Raszuk <[email-masked]>
Date: Sun, 17 Apr 2011 16:26:50 +0200

Hi,

Jakob and Jeff's emails triggered me to think if the overall approach 
here to extending all BGP messages size is the right one.

Let's realize that:

- To send  4096 octets msg on 10 Mbit/s wire takes  3.28 sec
- To send 65535 octets msg on 10 Mbit/s wire takes 52.43 sec !

No congestion, no delays ... pure best case numbers.

Now add to this time for acks, some router CPU delay, some I/O 
processing or worse some congestion or slow peer case and we are off the 
cliff.

I like the summary Jeff sent indicating a real consequences for implicit 
bound on current BGP attributes. Do we want to redefine BGP spec and 
subsequent specs so fundamentally now to try only one proposal of 
available number of others in the BGP security area ?

The answer I think is pretty obvious.

So let me propose that all elements related to securing BGP would be 
contain in a new message all together. (Perhaps new SAFI perhaps not .. 
I am not sure yet).

That would allow zero impact to Internet IPv4 and IPv6 routing as we 
know it today, gradual deployment etc ...

Here one also needs to keep in mind that all additional AS signatures as 
proposed today would be duplicated or N-plicated for each AFI/SAFI 
individually.

You would use the same space amount of information to sign IPv4 updates 
as IPv6 updates. Leave alone the encapsulation part .. but storing and 
processing the same thing many times seems to me like really waist.

The open issues are how to couple the secure BGP message with normal BGP 
message if such coupling at all is needed. Even if we would duplicate 
MP_REACH_NLRI into such Secure messages it still seems worth.

Another advantage is processing ... If we combine the secure BGP 
overhead with current BGP UPDATE message you like it or not it will have 
to be handled by today's old fashion general BGP CPU processing. But if 
this is defined as a separate message .. maybe even on separate session 
(by enforcing multisession on it) it could be all together easily 
processed by some other CPU/CPU core/programmable FPGA or ASIC without 
causing any burden and delay to current BGP processing. That would very 
nicely allow to decouple the functions and only pass final decision to 
main core BGP as prefix X - valid .. prefix Y - invalid.

Many thx,
R.

---

From: Jeffrey Haas <[email-masked]>
Date: Sun, 17 Apr 2011 15:08:43 +0000

Robert,

> No congestion, no delays ... pure best case numbers.

I suspect you'll send your own correction soon, but I think you're off a few
orders of magnitude.

4096 octets with a simplistic encoding of 10 bits to one octet on the wire
(for simplicity) means we have 40960 (40kb).  At a rate of 10Mbps, that's 
0.0041 seconds to send the packet.

I think it's fair to say that while BGP is TCP-bound with regard to some of
its poor behavior, it's not link-bound unless we're still talking T1. :-)

> processing or worse some congestion or slow peer case and we are off the  

Those are the sticky points.  I've spawned a new thread specifically for TCP
interactions of extended messages.

> The answer I think is pretty obvious.

While I think I sympathize with your thought about "let's go for the path of
least pain", I also think that we have a good general problem to solve with
at least two known applications to drive it.

IDR readers seeing my comments on extended messages should take those
comments not as attacks on the proposal but as constructive criticisms of
things we have to deal with as part of standardization.

I think we can get there from here.

> contain in a new message all together. (Perhaps new SAFI perhaps not ..  

As long as the proposal is backward compatible with 4k messages, I don't
know that we need a new code point for the reachability.  As I half alluded
to at the SIDR session on Friday at IETF 80, I think we'll eventually want
to move on to securing other reachability and perhaps other path attributes.
Requiring new AFI/SAFI code points would give us a full cross-product of
things to worry about.  (Perhaps fewer if we're clever with our bits.)

> processing the same thing many times seems to me like really waist.

I think what you're talking about here is a form of detached signature.  If
you can make the crypto for it work, I'd suggest proposing it in SIDR. 

> main core BGP as prefix X - valid .. prefix Y - invalid.

I think these are implementation discussions that will be very interesting
to have in SIDR. :-)

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 17 Apr 2011 19:58:07 +0200

Jeff,

> 0.0041 seconds to send the packet.

Yes .. sorry eaten 3 zeros - thx for correction ! But the main point was 
to show that it takes 16 times longer to send 65K then 4K message.

> its poor behavior, it's not link-bound unless we're still talking T1. :-)

Ack.

> interactions of extended messages.

Indeed. And this thread was sort of saying .. Perhaps we look at the 
overall problem rather then start 12 threads fixing details which 
perhaps do not need to be fixed. At least not now.

> you can make the crypto for it work, I'd suggest proposing it in SIDR.

Assuming that securing BGP brings value, everything which brings value 
costs money. And if now ISP have to invest and will not be able to 
differentiate to their customers with raise of service fee I am not sure 
how it will get deployed.

So some form of "detached signatures" or "semi-detached signatures" 
would just create basis of secured internet which could cost more from 
this best effort one - the one we know today - which would cost less.

If Grandma is using net to check email once a day she really does not 
need to bear any costs associated with "securing the Internet".

Many thx,
R.

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 19 Apr 2011 16:33:21 -0400

In message <[email-masked]>

Robert,

65535 * 8 = 524,280 bits  at 56kb/sec its under 10 sec, but grandma is
now using DSL so her BGP is going a bit faster than that.

You need to use faster-light.

The real problem is the TCP default 3 sec after losing some packets.
Then assuming a recent TCP that starts with 4 segments, it may take a
quite a few RTT to get a packet across.

With no loss, start with 4 segments and double each RTT.  For a 100
msec RTT it is well under a second.  So worst case is probably on the
order of 5 seconds if loss is moderate, maybe 10 seconds or more if
loss is really high.

Curtis

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>

---

From: Jakob Heitz <[email-masked]>
Date: Wed, 20 Apr 2011 03:32:44 -0400

Even this is uncomfortably close to the minimum 3 second hold time.
suggestions:
1. If using extended message length, minimum hold time is 30 seconds.
2. reset hold timer when BGP receives any data, even if it's only part of a message.

any objections to (2) ?

--
Jakob Heitz.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Curtis Villamizar
> Sent: Tuesday, April 19, 2011 1:33 PM
> To: [email-masked]
> Cc: [email-masked] List
> Subject: Re: [Idr] A slightly different perspective ....
> 
>

---

From: Pradosh Mohapatra <[email-masked]>
Date: Wed, 20 Apr 2011 08:01:28 -0700

> 2. reset hold timer when BGP receives any data, even if it's only part of a message.

No. Most of the BGP implementations already do this.

---

From: Rob Shakir <[email-masked]>
Date: Thu, 21 Apr 2011 05:58:35 +0100

> 2. reset hold timer when BGP receives any data, even if it's only part of a message.

Hi Jakob.

I would very strongly advise against taking the course of action described in point 1.

There are numerous scenarios in which (although we would prefer not to) we need to detect the failure of a BGP speaker in as short amount of time as possible for reasons that the service over the top _needs_ to converge in this time [0]. Where these might be in a private (e.g. VPN) context, I do not think that we can discount the fact that one day they might need to run extended message length (whilst we have a set of applications for which we are extending the message length today, this set is not bounded, and additional applications will surely come to light).

In the SIDR case, there are still BGP speakers, receiving Internet routes (which would be signed) that require detection of a peer failure within a short period of time. In numerous cases, 30s will break other applications, and whilst the statement "there is no SLA on the Internet" might be convenient, I don't think it is universally true. So, even for the known application of extended messages, I think 30s is unacceptably long for a hold-time _in some scenarios_.

In addition, where we deploy single session with multiple <AFI,SAFI> pairs down it, we might need to support extended message length for IPv4, but also run VPNv4 down the same session. In this case, we have to endure 30s hold-time for both even though we might have a requirement for failure detection (at the BGP daemon level) quicker than this for one of the AFIs (where sensitive services may be running in the VPNv4 one for instance)). Running a single session is of benefit to scaling and manageability.

Whilst I see the intention here, and the suggestion appears to be motivated out of ensuring robustness - is this really such a risk? Going by Curtis' figures (100msec RTT, 4 segments, no loss being <1sec) whilst this may be 'close' to the 3 second minimum, I would suggest that few sessions configured with such low timers run over links with 50msec one-way latency.

I think changing this minimum is a removing functionality that some deployments which cannot be discounted from requiring extended messages rely on.

Just my £0.02,
r.

[0]: There are really two scenarios here, one is suboptimal due to constraints of existing devices/deployments, but the other is more general.

1. If we cannot run OAM/BFD etc end-to-end for a session then I have seen numerous operators 'tune' BGP timers to provide some liveliness detection mechanism on the session. Platform support for OAM is not universal, and numerous hardware architectures do not support BFD in a reliable manner - hence, I can't see this going away in the long term. Whilst it's not "design intent" of the protocol, it is a current use, and one that it is difficult to see any alternative to without re-procuring hardware in some cases.

2. In the more general case, if my BGP daemon has failed or is failing, and my application cannot tolerate being broken for N seconds - I want to determine that this daemon is broken and hence routing might be stale in <<N seconds. If the minimum hold time is increased, this increases the bound on which this stale information can sit around (because the broken BGP speaker didn't react to an UPDATE that told it the current path is not valid), Where it's not the forwarding plane that breaks, and is just the BGP daemon, then we cannot detect this without using a message that goes straight to BGP, and hence KEEPALIVE/hold time is the general solution to this problem.

---

From: Jakob Heitz <[email-masked]>
Date: Thu, 21 Apr 2011 08:48:01 -0400

> From: Pradosh Mohapatra [mailto:[email-masked]] 
> Sent: Wednesday, April 20, 2011 8:01 AM
> 

Thanks Pradosh. Makes sense. So let's make it a rule.

However...

Now that we can not check a message length for sanity,
if the parser accidentally hits a 0xFFFF for a length,
it could take over 2 days to detect the error.
If the sending speaker sends very little other than a keepalive
every minute, it will take 65535/19 minutes to fill
the supposed message before it gets parsed.

Another possibility to restore the lost sanity checking of
the length:

Change the marker field to a different pattern.
It just needs not to look the same if it's shifted.
Use it like this:
If the message length is 4096 or less, either the old
or the new marker pattern is allowed before that length
field.
If the message length is greater than 4096, only the
new marker pattern is allowed before that length field.

--
Jakob Heitz.

---

From: Pradosh Mohapatra <[email-masked]>
Date: Thu, 21 Apr 2011 13:52:29 -0700

Hi Jakob,

> if the parser accidentally hits a 0xFFFF for a length,

and the type field is <= 5...

> new marker pattern is allowed before that length field.

What would you suggest as the new pattern? Wouldn't
that also have the same issue? The unfortunate side-effect
is that there is another variability now to the debugging
process ;-(

- Pradosh

---

From: Curtis Villamizar <[email-masked]>
Date: Sun, 24 Apr 2011 17:59:05 -0400

In message <[email-masked]>
> is that there is another variability now to the debugging

Changing the marker is a waste of effort.

4K/19 time 1 minute is already a big number yet there are no problems.

When no completed packet arrives the keepalive timer fires.  The 19
bytes every minutes (or less) doesn't count against the timer, only
complete packets.

Curtis

---

From: Jeff Wheeler <[email-masked]>
Date: Sun, 24 Apr 2011 18:50:33 -0400

> Changing the marker is a waste of effort.

I agree that limiting the message length to 0xFEFF is a much better
solution, but I don't think I am against the idea of changing the
marker.  This has plenty of precedent in RFC1771 as well -- the marker
was envisioned to contain authentication data.  Later revisions took
that (unnecessary, and AFAIK never implemented) capability away.

If the marker octets were all 0x00 instead of 0xFF, what would be the
impact, aside from requiring some changes to header validation code
which may be driven by the extended message capability flag?  Is it
any more likely that future BGP message payload would contain 16 0x00
octets in a row, than 16 0xFF octets?  I think it's easy to say "yes"
to that, but the likelihood of either of these circumstances arising
along with the necessary serializer/parser bugs needed to trigger bugs
is quite low.  On the other hand, the likelihood of
off-by-one/off-by-two bugs causing a BGP session to (apparently)
"stall" for a long period of time is IMO higher, and changing the
marker octets from 0xFF to 0x00 would reduce the negative impact of
that.

Still, this is a fix requiring more code which would be totally
unnecessary if the draft simply extended the maximum message length
from 4096 octets to 65,279 octets instead of 65,535 octets.

> 4K/19 time 1 minute is already a big number yet there are no problems.

Again, there is virtually zero chance, given the current
specifications, that you will ever think an incoming message will be
4K, or any other size than what the remote neighbor actually says it
will be.  Off-by-one/two errors which could affect the perceived
length of the following message are currently guaranteed to be
detectable.

If the current draft is adopted, an off-by-one or off-by-two bug,
combined with a previous message that has one or two trailing 0xFF
octets, will definitely trick the parser into thinking a >= 65280
octet message should be expected.  It does not require the remote
neighbor to make a huge error in length calculation -- the common "off
by one" can and will do it, and this bug will no longer manifest
itself as an invalid message header, but will instead appear to the
operator to be a "stalled BGP session" which cannot be diagnosed.

In the case of this time-out issue, these problems become directly
related to each-other, if the current header check mechanism is
removed from BGP.  It is not simple.

-- 
Jeff S Wheeler <[email-masked]>
Sr Network Operator  /  Innovative Network Concepts

---

From: Jakob Heitz <[email-masked]>
Date: Mon, 25 Apr 2011 09:20:47 -0400

What I had in mind was a marker that would look
different if it was shifted.
today, it is 16 of 0xFF. If the last byte of the previous
message is also 0xFF, then there are 17 of 0xFF and an
off-by-one error becomes possible.
Same thing with 17 of 0x00.
However, change just one byte and such a shifting can
be detected.

Having said that, I do acknowledge that the undetected
error is rare, but the solution is also simple.

I'm not going to push this anymore. It was a small
thing to begin with anyway and in my view the solution
was also small. I am astounded at the volume of controversy
it caused.

--
Jakob Heitz.

> Behalf Of Jeff Wheeler
> Sent: Sunday, April 24, 2011 3:51 PM
> To: [email-masked]