Subject: RE: INFORM proposal and MP-BGP (RFC 2858)

From: "Natale, Jonathan" <[email-masked]>
Date: Tue, 9 Jul 2002 12:35:19 -0400

Thank you very much for the prompt reply.  I agree completely.

My question is this: If you receive a route refresh when you did not
advertise that capability at all (and/or your code does not support it at
all) then you should reset the peer with a 1/3 (Message Header Error/Bad
Message Type) notification. Right?

Also:
"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker MAY send a
   NOTIFICATION message to the peer, and terminate peering (see Section
   "Extensions to Error Handling" for more details). The Error Subcode
   in the message is set to Unsupported Capability. The message SHOULD
   contain the capability (capabilities) that causes the speaker to send
   the message.  The decision to send the message and terminate peering
   is local to the speaker. If terminated, such peering SHOULD NOT be
   re-established automatically. --draft-ietf-idr-rfc2842bis-02.txt,
"Capabilities Advertisement with BGP-4"

Seems harsh, I would think that the preferred behavior is to re-open w/o the
offending capability.  A major vendor does not and they have a back door cmd
to work around it.  They also have a bug on it but based on history
(AS_CONFED_SET <-> AS_CONFED_SEQUENCE, deterministic med, etc...) the
rfc/draft seem to follow them NOT the other way around. Any thoughts???

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Tuesday, July 09, 2002 12:26 PM
To: Natale, Jonathan
Cc: [email-masked]'
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)

On Tue, Jul 09, 2002 at 12:19:23PM -0400, Natale, Jonathan wrote:
> : "If a BGP speaker receives from its peer a ROUTE-REFRESH message with
>    the <AFI, SAFI> that the speaker didn't advertise to the peer at the
>    session establishment time via capability advertisement, the speaker
>    shall ignore such a message." - RFC2918, "Route Refresh Capability for
> BGP-4"
> 
> shouldn't this be an error/notification???

The operational community continues to demand resiliency from the BGP
protocol.  If we can avoid sending a NOTIFICATION and gracefully
deal with someone elses buggy code, thats a Good Thing.

Sending an INFORM would be nice here, if such a thing existed.

-- 
Jeff Haas 
NextHop Technologies

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Tue, 9 Jul 2002 12:54:32 -0400

> rfc/draft seem to follow them NOT the other way around. Any 

For network deployment phasing reasons and backward compatibility reasons.
We should not drop a peer just because we recieved a message that contained
unrecognized capability(s). I vote the spec be changed to accomodate an
evolving network. The lesser capable BGP router should silently ignore the
unrecognized capability(s) or the entire message if there is nothing else
recognizable in it. The more advanced BGP capable router upon detecting a
peer that does not understand certain advanced capability(s) should merely
not send any more advanced capability(s) to this peer. The two routers
should also log the event for their operators to review.

Also, if we drop peers do to unrecognized messages or capabilities, we make
it easier for hacker attacks causing network outages, or the topology to get
unstable.  

Right?

Ben

---

From: "Bruno Rijsman" <[email-masked]>
Date: Wed, 10 Jul 2002 11:47:17 -0400

Re> Just because you receive an Update message with capabilities that
Re> were not negotiated at the OPEN message, should not require the
Re> peering session to terminate. The affect of all that peer routes
Re> going down will destabilize the network.

Also, keep in mind that in the case of dynamic capability negotiation
is possible and perfectly legal to receive a message which relies on
a capability which was recently withdrawn. Consider for example the
case where a route-refresh message and a capability message which
withdraws the route-refresh capability cross each other.

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Wed, 10 Jul 2002 15:03:43 -0400

I think we are getting very philosophical here, but as we all know no
protocol
that is heavily used stands still. BGP is one that is constantly in a state
of 
change because of the various users it has and will continue to add on.
To say one cannot change it and improve it, is to say we dont make progress.

We know there are many issues with the way it works and I am sure many
experts
will tell you what they are. So I wont get into it. All I and others here
are 
trying to do is add a little elasticity/gracefullness/resiliency, you name
it what you want.

Ben

> From: Pedro Roque Marques [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 2:55 PM
> To: Abarbanel, Benjamin
> Cc: [email-masked]
> Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Abarbanel, Benjamin writes:
> 
> > Correct. The protocol has to be elastic, since it carries so much
> > weight in controlling stability in the Internet.
> 
> <meta>
> 
> If the stability of the Internet is the concern, then i believe that
> one should avoid any changes to the protocol unless they are
> absolutely necessary.
> 
> If we where talking about a protocol in early development then adding
> extra 'bells and whistles' would probably be ok (although i can think
> of quite a large number of new protocols which deployment has been
> very negativly impacted by constant churn).
> 
> But since this an essential protocol to the internet, the burden of
> proof required to make a modification to the protocol should be much
> higher and should be on the side of the proponents.
> 
> By default the answer should be 'no'. Don't change the protocol, add
> flexibility that you desire, and is fact required operationally, in
> the implementations. There is plenty of space to compete there...
> 
> </meta>
> 
>   Pedro.
>

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Wed, 10 Jul 2002 15:12:45 -0400

> Sent: Wednesday, July 10, 2002 3:08 PM
> To: Pedro Roque Marques
> Cc: Abarbanel, Benjamin; [email-masked]
> I want to post something more thoughtful on this thread, but have
> to do it later due to time constraints.
> 
> I couldn't let this pass though.
> 
> On Wed, Jul 10, 2002 at 11:54:41AM -0700, Pedro Roque Marques wrote:
> This made me laugh extremely loudly - but not at you.
> 
> I count 9 RFC extensions past RFC1771 base functionality.
> I count at least 10 protocol extensions in ID form in just
> my personal watch list of stuff that we may implement or have
> implemented.  This list is by no means complete.
> 
> When talking to people, I often refer to BGP as the "Border 
> Glue Protocol".
> People keep adding stuff.
> 
> The trick is to do it in an interoperable way.

I totally agree, I also think we should add things in a way that is
phaseable
into existing networks. Thus causing little if any network disburbances
between 
the old routers and the new.

Ben

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Wed, 10 Jul 2002 17:39:17 -0400

Thanks for the clarification on the original intent.

Clearly I think there are many cases, were we want to send potential
operational affecting information to a neighboring peer without reseting
the session. INFORM type message could provide that communication.

I can see the following type of data it would convey:
  1. CPU Utilization levels in an effort to slow down the rate of update.
  2. Memory utilization in an effort to control the amount of routing
     information sent, possibly forbid any optional capabilities while
     router is in a memory depleted state.
  3. Reaching maximum limits on configured or defaulted resources. Number
     of prefixes, attributes, capabilities, etc.
  4. Protocol related WARNING conditions that do not cause a session 
     reset. Repair race conditions, identify FSM state, etc.  

Thanks,
Ben 

> From: Gargi Nalawade [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 5:12 PM
> To: [email-masked]
> Hi All,
> A few clarifications about the INFORM message & its 
> motivations. This is
> supposed to be purely an informational message. Does not 
> propose to affect
> the functionality of the protocol. Rather, it proposes to 
> give flexibility
> to implementations in that; they dont need to reset the 
> sessions, just so
> they can inform the peer of a specific noteworthy event happening. 
> 
> Take the example of max-prefix limit. If the limit is hit, 
> the receiving
> peer would just generate a local warning (if so configured) 
> and continue
> with the session. There is no mechanism of informing the 
> other peer that
> it has exceeded the prefix-limit value, other than churning 
> through the log,
> noticing the warnin & then picking up the telephone (or oops 
> - writing an
> email) & informing the peer's administrator.
> 
> This is just one such hypothetical example where the INFORM 
> would be useful.
> But the INFORM message gives us the capability to do just 
> that. There are 
> several other such scenarios seen in deployment, that necessitate the
> existence of just such a mechanism.
> 
> -Gargi
>

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Wed, 10 Jul 2002 18:03:53 -0400

John:

What if the dynamic capabilities was controlled by an operator's script
that was dynamic and automatic and configured from NMS outside the router. 
The script would add/delete capabilities based on the current state of the
router. Lets say this situation occured continuosly.

Obviously, it would not be sensible to run a live network this way. But this
is one hypthetical condition in which we could re-enter the race condition
after
the session restarted. I guess maybe, one has to judge how relient you want
the router to be. 

In practice, if you can create it with a script, there is a (maybe small)
chance it would happen in a live network.

Ben

> From: John G. Scudder [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 5:53 PM
> To: Abarbanel, Benjamin
> Cc: 'Pedro Roque Marques'; Abarbanel, Benjamin; [email-masked]'
> At 5:46 PM -0400 7/10/02, Abarbanel, Benjamin wrote:
> >I was not saying that two different TCP sessions will cross 
> each other and
> >convey each other messages in a mixed up way. I was saying that the
> >potential
> >for the "ships crossing in the night" that caused the first 
> race condition
> >with
> >the capabilities could be there again, after the new session 
> is established.
> >If some combination of routes or capabilities or just router 
> execution
> >timing
> >caused it, it could happen again with similar environment 
> and thus cause
> >the condition. Murphy's law always strikes the same way if 
> the conditions
> >are right. Simply reseting a session I would believe could replay the
> >condition
> >that cause the first race condition and thus throw the two 
> routers into a
> >perpetual session restart loop.
> 
> Since the hypothetical problem relates to dynamic capability 
> advertisement, why would it recur?  Surely any sensible 
> implementation will only dynamically advertise a capability as a 
> result of a configuration change.  The changed configuration will be 
> used to produce the capabilities in the initial OPEN of the new 
> session.  Thus, no need to use dynamic capability on the new session 
> (until and unless the configuration changes again).
> 
> --John
>

---

From: "Natale, Jonathan" <[email-masked]>
Date: Wed, 10 Jul 2002 18:31:36 -0400

Good example.

I had also suggested a new NOTIFICATION for the case when it IS configured
to kill the con:

-----Original Message-----
From: Natale, Jonathan 
Sent: Monday, April 29, 2002 4:13 PM
To: 'Russ White'; Pedro Roque Marques
Cc: [email-masked]
Subject: RE: BGP-17 Draft Comments...

I would like to suggest another UPDATE Message Error subcode be introduced
~= "Maximum Number of Allowed Prefixes Exceeded" (if this has been done,
then please direct me to where it is documented).  Currently ( 12.1(5)T4 ),
a "Malformed Attribute List" notification is sent when the user configured
maximum number of allowed prefixes is exceeded.  Alternatively, this current
(or the "correct" behavior) should be specified in the draft-ietf-idr-bgp4
draft which recently added:

   "A BGP speaker may support the ability to impose an (locally
   configured) upper bound on the number of address prefixes the speaker
   is willing to accept from a neighbor. When the upper bound is
   reached, the speaker (under control of local configuration) may
   either (a) discard new address prefixes from the neighbor, or (b)
   terminate the BGP peering with the neighbor. If the BGP speaker
   decides to terminate its peering with a neighbor because the number
   of address prefixes received from the neighbor exceeds the locally
   configured upper bound, then the speaker must send to the neighbor a
   NOTIFICATION message with the Error Code Cease."

-$0.02

Hi All,

A few clarifications about the INFORM message & its motivations. This is
supposed to be purely an informational message. Does not propose to affect
the functionality of the protocol. Rather, it proposes to give flexibility
to implementations in that; they dont need to reset the sessions, just so
they can inform the peer of a specific noteworthy event happening. 

Take the example of max-prefix limit. If the limit is hit, the receiving
peer would just generate a local warning (if so configured) and continue
with the session. There is no mechanism of informing the other peer that
it has exceeded the prefix-limit value, other than churning through the log,
noticing the warnin & then picking up the telephone (or oops - writing an
email) & informing the peer's administrator.

This is just one such hypothetical example where the INFORM would be useful.
But the INFORM message gives us the capability to do just that. There are 
several other such scenarios seen in deployment, that necessitate the
existence of just such a mechanism.

-Gargi

---

From: "Natale, Jonathan" <[email-masked]>
Date: Wed, 10 Jul 2002 18:34:06 -0400

Rxing mcast nlri, when only doing unicast
The draft added this clarification--"ignore them" (seems like a job for the
INFORM???)

-$0.03

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]] 
Sent: Wednesday, July 10, 2002 5:39 PM
To: 'Gargi Nalawade'; [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)

Thanks,
Ben

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Thu, 11 Jul 2002 10:28:42 -0400

> 
> You can be tolerant in your implementation today if you choose
> and if your customers will accept it. I do not see how you hope
> that a new INFORM message will let an implementation correct itself 
> when the implementation failed to do the right thing in the first 
> place.
> 

The case I am refering to is the "ships in the night" condition where the
messages cross on the wire, thus causing improper processing. With the use
of the INFORM like message, the two routers can get synchronized on what 
capabilities to honor without a session restart. Thus preserving the state
of all their routes.

Ben

> -
> Chandra A.
> 
> 
> On Wed, Jul 10, 2002 at 02:49:12PM -0400, Abarbanel, Benjamin wrote:
> > I am in favour of providing a tolerance counter that will 
> drop the session
> > only after a max number of bad update messages are 
> received. Thus if the
> > offending
> > peer does not correct or adjust itself in terms of 
> capabilities, after a
> > number
> > of INFORM responses are sent to it, then and only then 
> should the receiving
> > router drop the session. This will cover the case, where 
> the INFORM is also
> > mis-processed in the "ships in the night" scenario.
> > 
> > Ben
> > 
> > 
> > > From: Chandrashekhar Appanna [mailto:[email-masked]]
> > > Sent: Wednesday, July 10, 2002 2:09 PM
> > > To: Abarbanel, Benjamin
> > > Cc: 'Bruno Rijsman'; [email-masked]
> > > But that and the ability to INFORM is not the same. In the
> > > example you provide, the INFORM is not providing any extra 
> > > advantage. If the peer did not process/rcv the withdraw 
> > > why do you hope that it will rcv/process the INFORM?
> > > 
> > > I think the behavior of the withdraw itself is the problem here,
> > > It has to handle the case of a message cross that you describe, 
> > > in a well defined manner. 
> > > 
> > > -
> > > Chandra A.
> > > 
> > > On Wed, Jul 10, 2002 at 11:50:32AM -0400, Abarbanel, 
> Benjamin wrote:
> > > > Correct. The protocol has to be elastic, since it carries 
> > > so much weight in 
> > > > controlling stability in the Internet.
> > > > 
> > > > Ben
> > > > 
> > > > > From: Bruno Rijsman [mailto:[email-masked]]
> > > > > Sent: Wednesday, July 10, 2002 11:47 AM
> > > > > To: [email-masked]
> > > > > Cc: [email-masked]

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Mon, 15 Jul 2002 17:58:14 -0400

John:
 I dont think people who suggest to use INFORM message will argue with 
you on the merits of not keeping the route. I do think the tendency in
BGP is, if you see something that looks strange drop the session. This
religious approach leads to much more serious consequences, which 
sometimes could be remedied by not dropping the session but by providing 
a mechanism for resynchronizing the corrrect information.

$.02 Worth,
Ben

> Sent: Monday, July 15, 2002 5:40 PM
> To: Jeffrey Haas
> Cc: Pedro Roque Marques; [email-masked]
> At 4:20 PM -0400 7/12/02, Jeffrey Haas wrote:
> >On Fri, Jul 12, 2002 at 12:14:38PM -0700, Pedro Roque Marques wrote:
> >>  >From both quotes above it apears to me that you are proposing to
> >>  discard an UPDATE and leave the session up. Since BGP state is
> >>  incremental, that would mean the two peers would now 
> potentially be
> >>  out of sync.
> >
> >And no less intentionally than if I discarded everything via policy.
> >In this case, my policy is "discard bad (for some value) packets".
> 
> No, it's worse than that.  Here's an example of harm using your 
> "let's introduce a new origin code of 4" case.
> 
> - Router A sends router B an announcement of prefix foo with origin 1.
> - Router A now sends router B a replacement for prefix foo, but with 
> newfangled origin 4.  The replacement has a different next hop from 
> the previous announcement.
> - Router B ignores the replacement because of the posited "discard 
> updates that contain unrecognized data" rule.
> 
> Result?  Router B retains router A's old route, whose next hop is no 
> longer valid.  This goes on forever, or until router A sends an 
> update with a recognized origin, or until router A withdraws the 
> route.  The bad next hop may result in nothing serious, but it may 
> equally well result in a black hole or a persistent forwarding loop. 
> This seems to a cure which is much worse than the disease.
> 
> Of course, refusing routes by policy doesn't have this problem at all.
> 
> Let me grant you up front that you can fix this specific case by 
> being more careful in stating your rule -- instead of "discard 
> updates that contain unrecognized data" make it "treat updates that 
> contain unrecognized data as if they were withdraws".  But whenever 
> (and if ever) you want to gloss over a protocol error like this IMO 
> you need to analyze the consequences quite carefully.
> 
> --John
>

---

From: "Natale, Jonathan" <[email-masked]>
Date: Tue, 16 Jul 2002 10:28:34 -0400

Actually, that is not what I proposed; I was mearly reporting the
behavior I observed which was to ignore the attribute, but not the
route--it got installed.  Also, this was an UNEXPECTED/mis-configured
CONFED AS_PATH (sorry for not making this clear).  Whether this is a
bug, or "correct" could certainly be debated.  But I thought it was
related to:

    "As you infer, there are situations where we should just assume
    that the buffer contains cruft and that the implementation needs
    a strong reset.

    However, there are situations where we could get stuff that we
    can deal with and just ignore the "Bad" things.  This was the
    case when someone's implementation started leaking non-prefixed
    confederation segments into the internet.  The packets were well
    formed, but had cruft in them."

    -- Jeff, Fri 7/12/2002 2:29 PM

Seems to me that there are certainly cases where it is better to
ignore something then flap the whole session.

Another case where this happens is RXing a route to yourself (if
a neighbor runs PPP and redistributes connected)--route gets dropped.
Juniper sends the routes back so the neighbor would have a better
chance of knowing what got dropped, Cisco does not do this, the
RFC/Draft is unclear on this.  Both have
advantages/diaadvantages--my vote is to make it user configurable
(split-horrizon), this is already an option for other protocols.

What to do is not [publicly] defined, and should be.  Or maybe user
configurable would be good (I think so), but that could be
[publicly] defined as well.  There are many ways things could go
wrong.  The two basic categories are syntactical and semantically.
It seems that generally the RFCs/Drafts indicate that for
syntactical errors you should send a NOTIFICATION and close the
session (for how long and how many times?), and for semantical
errors you should ignore [the route or the attribute].  In cases
where things get ignored due to the fact that they are not
appropriate for the particular scenario, then an INFORM would not
hurt and may help troubleshoot the misconfig/bug.  The first step
would be to categorize all (lol) the scenarios, and then
have a vote/decree and come to a final complete protocol (lol).

-----Original Message-----
From: Pedro Roque Marques [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 3:15 PM
To: Jeffrey Haas
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)

Jeffrey Haas writes:

> On Fri, Jul 12, 2002 at 02:33:05PM -0400, Natale, Jonathan wrote:
>> This is already done--if CONFED path is received the peer stays up
>> (and the local pref gets ignored).  This is good, but not in an rfc
>> nor a draft.  My vote is for an INFORM.

> IMO, this is a case where the route's local pref shouldn't be just
> ignored, but the entire packet should be discarded.

From both quotes above it apears to me that you are proposing to
discard an UPDATE and leave the session up. Since BGP state is
incremental, that would mean the two peers would now potentially be
out of sync.

  Pedro.