Subject: RE: TCP state machine doubts

From: "Natale, Jonathan" <[email-masked]>
Date: Tue, 9 Jul 2002 10:42:33 -0400

Increasing the time on each try is not a good idea and is not currently
implemented.  Correct me if I am wrong...

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]] 
Sent: Friday, July 05, 2002 6:22 AM
To: BGP mailing list
Cc: Gilles Geerts
Subject: TCP state machine doubts

all,

i have a few questions related to the new state machine proposed in the
latest BGP draft.

1. related to terminology related to the connect retry counter.
I find back in the state descriptions following statement:
- "clears the ConnectRetry timer" (e.g. page 35, state connect)
- "Set connect retry timer to zero" (e.g. page 37, state Open sent)
- "Connect retry timer = 0," (e.g. page 38, state Open sent)
 I assume that the meaning in all of the above cases is : "the
ConnectRetry timer is stopped".  If so, would this not be a better
wording for the action?

2. related to the terminology for the holdtimer.
In state Connect on page 34 I find the following text: "set Hold timer
to a large value". I expect that this means "The Hold timer value is set
to a large value, and the Hold timer is started".

3. related to the ConnectRetryCnt.
- Should the value of this counter not be initialsed in the Idle state
as well?

4. General doubt  related to the TCP backoff strategy
Suppose I have a BGP peer that goes down say once every 2hours. With the
current sheme the ConnectRetryCnt keeps on increasing, and as a
consequence the connection takes increasinly more time to be
re-establsihed, even if actually it only fails like every 2 hours.

The mechanism only gives like 'bad points' , but never 'good points'. In
this way a peer never gets rid of his flapping history.
Is this really the intented behavior?

Thanks,
  Hans De Vleeschouwer.

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Mon, 15 Jul 2002 10:51:19 -0400

Hi Sue:
  I was looking for "ietf-hares-bgp-statemt-02.txt" in the draft depository
and could not find it. I did find verion ietf-hares-bgp-statemt-01.txt.
Should that one suffice for review?

Regards,
Ben

> From: Susan Hares [mailto:[email-masked]]
> Sent: Saturday, July 13, 2002 7:15 AM
> To: Kunihiro Ishiguro
> Cc: [email-masked]; BGP mailing list; Gilles Geerts
> Subject: Re: TCP state machine doubts
> 
> 
> Kunihiro:
> 
> I've got the FSM text for draft-18 revised.  I'm looking
> for a group of people to review 5th round of the FSM
> text along with the associated documents:
> 
>          1) BGP state machine (ietf-hares-statemt-02.txt)
>          2) BGP backoff draft (ietf-hares-backoff-01.txt)
>          3) Passive and Call Collision input
>          4) BGP update draft (ietf-hares-update-00.txt]
> 
>   Thanks for volunteering to review the documents prior to
>   me flooding the information to the mail list.  I welcome
>   any other early reviewers.
> 
>   I submitted the first two as Internet drafts, but I found a few
>   typos.
> 
>   I've answered your questions below:
> 
> At 08:49 PM 7/5/2002 -0700, Kunihiro Ishiguro wrote:
> >Current draft-17 has some known problems.  I think Sue 
> aleady prepared
> >draft of draft-18.  So best way is updating the draft to 
> draft-18 then
> >process IDR working group review to it.
> >
> I've changed the text to be common.  Let me know if you like the
> new wording.  Otherwise, I can add a little more text.
> 
> 
> > >
> Could you look at the current text
> 
>          - Set the Hold timer to a large value (4 minutes),
> 
> 
> > >
> >These are right.  I believes all of these are fixed in the draft of
> >draft-18.
> 
> 
> See the FSM words and state machine.  I could use another
> pair of eyes to look at the FSM.
> 
> The backoff mechanisms can actually change that.  I would
> suggest that is another mechanism we could add in the
> backoff section.   We could add a timer at that point or
> agree to add another state to the FSM's timer.
> 
> Please comment on which one you'd like to see.  Please
> suggest text for the backoff draft.
> 
> 
> >My understanding is once BGP peer is establieshed, the 
> ConnectRetryCnt
> >is cleared.
> 
> This mechanism does not fix the persistent peer oscillation.
> Again, I think we can change this in the backoff draft, but
> maybe we need another timer to reduce the connect retry count.
> 
> Could you suggest some text based on the backoff draft?
> 
> 
> Sue Hares
>

---

From: "Abarbanel, Benjamin" <[email-masked]>
Date: Tue, 30 Jul 2002 17:50:29 -0400

Jeff:
  The FSM state machine actions taken are internal sequenced procedures.
These
actions could get messy and possibly execute out of sequence when there is a
multi-threads implemenatation. Although the FSM is preoccupied mostly with
the setting up and tearing down the peering sessions, in the processing of
these states certain multi-threaded sensitive processing rules should be
defined.

For Example:
============
Define locks around the various RIB synchronization points and mentioned
when to take them and when to release them if a state transition has occured
via the various action statements. This will gaurantee that the RIB data is
protected from parallel thread execution. It really does not matter which
thread takes the lock or which thread releases it.  If RIB resource is
locked, what happens to the state machine when state changing events are
seen. (E.G. Session disconnected by TCP or Peer. What happens when an
operator configuration command shuts down a peer session or the whole BGP
engine? 

I know this is getting very detailed, but the FSM is a gray area between the
internal and external world.

Ben

> From: Jeffrey Haas [mailto:[email-masked]]
> Sent: Tuesday, July 30, 2002 4:52 PM
> To: Abarbanel, Benjamin
> Cc: 'Susan Hares'; Kunihiro Ishiguro; BGP mailing list
> On Thu, Jul 25, 2002 at 11:19:11AM -0400, Abarbanel, Benjamin wrote:
> > There are various phases that BGP goes through that one
> > might want to use multiple threads. The inter-locking or 
> inter-working
> > between the threads could affect the state transitions. For 
> example, Say you
> > are in Establish state and something happens to the session 
> and it is
> > disconnected. In a single threaded environment it is easy, 
> first come, first
> > served. In a multi-threaded case with thread priorities and 
> interruptions,
> > it might get tricky at times. Some assumptions or 
> expectations by the state
> > machine on actions it should take with regards to semaphore 
> locks (RIB
> > Table, etc.), thread prioritization, processing, 
> pre-emption and so forth
> > might be helpfull. Maybe these actions should only be 
> suggestions not
> > required ways for implementations.
> 
> There are certain situations where the FSM suggests that things be
> done in certain events where an implementation *could* do things
> in a slightly different order.
> 
> For example, after the TCP connection has completed, you are supposed
> to send your messages *before* reading the socket.  In a 
> multi-threaded
> environment, you may obviously not do this.  Indeed, for passive or
> unconfigured peers, you may not even send your message immediately.
> 
> The FSM would suggest where your synchronization points need to go.
> Happily, most of this work seems to need to be done during 
> peering setup.
> 
> For the most part, like the 3-rib abstraction, you should be able
> to do whatever you want inside your black box so long as it looks
> right on the wire.  The FSM gives guidance on what probably is
> happening in the box to get things out as you expect them to.
> 
> More specific examples might be helpful.
> 
> > Ben
> 
> -- 
> Jeff Haas 
> NextHop Technologies
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Jeffrey Haas'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Susan Hares'" <[email-masked]>, Kunihiro Ishiguro <[email-masked]>, BGP mailing list <[email-masked]>
Subject: RE: TCP state machine doubts
Date: Tue, 30 Jul 2002 17:50:29 -0400
Sender: [email-masked]

Jeff:
  The FSM state machine actions taken are internal sequenced procedures.
These
actions could get messy and possibly execute out of sequence when there is a
multi-threads implemenatation. Although the FSM is preoccupied mostly with
the setting up and tearing down the peering sessions, in the processing of
these states certain multi-threaded sensitive processing rules should be
defined.

For Example:
============
Define locks around the various RIB synchronization points and mentioned
when to take them and when to release them if a state transition has occured
via the various action statements. This will gaurantee that the RIB data is
protected from parallel thread execution. It really does not matter which
thread takes the lock or which thread releases it.  If RIB resource is
locked, what happens to the state machine when state changing events are
seen. (E.G. Session disconnected by TCP or Peer. What happens when an
operator configuration command shuts down a peer session or the whole BGP
engine? 

I know this is getting very detailed, but the FSM is a gray area between the
internal and external world.

Ben

> From: Jeffrey Haas [mailto:[email-masked]]
> Sent: Tuesday, July 30, 2002 4:52 PM
> To: Abarbanel, Benjamin
> Cc: 'Susan Hares'; Kunihiro Ishiguro; BGP mailing list
> On Thu, Jul 25, 2002 at 11:19:11AM -0400, Abarbanel, Benjamin wrote:
> > There are various phases that BGP goes through that one
> > might want to use multiple threads. The inter-locking or 
> inter-working
> > between the threads could affect the state transitions. For 
> example, Say you
> > are in Establish state and something happens to the session 
> and it is
> > disconnected. In a single threaded environment it is easy, 
> first come, first
> > served. In a multi-threaded case with thread priorities and 
> interruptions,
> > it might get tricky at times. Some assumptions or 
> expectations by the state
> > machine on actions it should take with regards to semaphore 
> locks (RIB
> > Table, etc.), thread prioritization, processing, 
> pre-emption and so forth
> > might be helpfull. Maybe these actions should only be 
> suggestions not
> > required ways for implementations.
> 
> There are certain situations where the FSM suggests that things be
> done in certain events where an implementation *could* do things
> in a slightly different order.
> 
> For example, after the TCP connection has completed, you are supposed
> to send your messages *before* reading the socket.  In a 
> multi-threaded
> environment, you may obviously not do this.  Indeed, for passive or
> unconfigured peers, you may not even send your message immediately.
> 
> The FSM would suggest where your synchronization points need to go.
> Happily, most of this work seems to need to be done during 
> peering setup.
> 
> For the most part, like the 3-rib abstraction, you should be able
> to do whatever you want inside your black box so long as it looks
> right on the wire.  The FSM gives guidance on what probably is
> happening in the box to get things out as you expect them to.
> 
> More specific examples might be helpful.
> 
> > Ben
> 
> -- 
> Jeff Haas 
> NextHop Technologies
> 

From: "Turk, Doughan A." <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>, "Turk, Doughan A." <[email-masked]>, PamSri <[email-masked]>
Cc: [email-masked]
Subject: RE: Need your opinion of this.
Date: Tue, 30 Jul 2002 17:47:46 -0400
Sender: [email-masked]

Comments inline

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]] 
Sent: Monday, July 29, 2002 10:06 AM
To: 'Turk, Doughan A.'; PamSri
Cc: [email-masked]
Subject: RE: Need your opinion of this.

Doughan:

Since some of the traffic that is trapped with the DoS traffic is going to
be redirected to the sinkhole router. would that cause those poor cutomers
to loose thier communications? 
*** Not at all, I'm not tampering with routing tables here***

What if the sinkhole router was smart enough to filter through the good
packets from the bad, could it then redirect the good packets back to their
intended destination? 
*** this is exactly what the draft is all about. Since the BGP community
only triggers a local routing change to the router that has the hooks for
this comm. The traffic slides through an MPLS tunnel for example. After your
smart sinkhole device cleans the traffic, you could have a default statement
on the sinkhole pointing to any interface to connected to your network, even
the same physical interface the tunnel was built on. This will guarantee
that traffic goes back to your customer. Since out side of the edge router
with the community hooks, no change to the legitimate next hop is done, it
is not an issue.***

If so, that implies the sinkhole routers have alot more cycles to handle all
the load that will be imposed on them by this service. 

*** choosing a powerful interface on the sinkhole router is completely your
choice, if you know what are the limits of that interface, you could limit
your ACLs or Policing statement not to compromise the sinkhole router.
Investing in a good sinkhole router can save you money of other parts of
your network.***

Is source and destination address the only things needed to filter good
packets from bad ones?

*** This is out of the scope of this draft and totally depends on the
circumstance. But to comment on this, I would say what if the attacker is
doing a great job in randomizing his source/ports how can one filter the
traffic. You need to find a distinguishing signature for the attack traffic.
This might or might not be an easy or possible task.***

My instinct tells me it is better to stop DoS traffic before it enters the
cloud on the border router, then try to manage it at some redirected
sinkhole router. 

*** See there are too parts to this draft, one that does exactly what you
are suggesting and this is at the beginning of the draft, and the other take
traffic through a tunnel for further analysis or treatment.***

Communities attribute although they tend to converge on some Next Hop would
tend to propagate the traffic through the cloud and thereby cause more
loading conditions on the intermediary routers.

*** Communities are used in many places and for different reasons, I can't
see a network outage cause by adding an extra comm. value to a prefix.***

Appreciate you input :-)
DT

Ben

> From: Turk, Doughan A. [mailto:[email-masked]]
> Sent: Friday, July 26, 2002 9:24 PM
> To: PamSri
> Cc: [email-masked]
> Subject: RE: Need your opinion of this.
> 
> 
> I am using BGP to help stopping DoS and DDoS traffic. BGP can
> send updates
> quickly and route-maps can translate communities to stop 
> traffic destined to
> a certain destination. TCP is not the issue here. Attackers 
> usually relay on
> the fact that most routers out there do destination based 
> routing without
> checking the validity of the source address in case it is spoofed.
> 
> DT
> 
> From: PamSri [mailto:[email-masked]]
> Sent: Friday, July 26, 2002 9:08 PM
> To: Turk, Doughan A.
> Subject: RE: Need your opinion of this.
> 
> 
> Hi Turk ,
>          Just started to read through your doc. I have
> a question :
> BGP is an application from the perspective of TCP. Why
> do you see the need for provding such security at the
> application level than
> TCP ? Do you think the security features in TCP as inadequate 
> ? It would be
> really a good idea to provide security at the underlying 
> layers than at this
> layer. 
> 
> sri
> 
> --- "Turk, Doughan A." <[email-masked]> wrote:
> > Benjamin,
> >  
> > BGP will be used to alter next-hop address for a
> > particular destination
> > address (representing a customer under attack) on
> > selected Routers. Instead
> > of using BGP to announce a different next-hop for
> > the destination address
> > everywhere in your network. we can use communities
> > to make it selective.
> > once that is achieved, your next hop can be a null interface or a 
> > Tunnel interface where the traffic is then pulled to the
> > sinkhole router.
> >  
> > Based on Attack signature you could do various
> > packet lever filtering ( with
> > hardware as a constrain),  simply rate limit the
> > flow or send it out through
> > a shared media (Ethernet) for sniffing after you
> > pull it to a sinkhole
> > router.
> >  
> > As for overheads on routers in the network, there is
> > hardly any since you're
> > only altering next-hop on border routers or edge
> > routers. MPLS tunnel are
> > what I used in my lab. as for the sinkhole router,
> > one can invest in a high
> > performance line card. The sinkhole router should be powerful enough 
> > to handle whatever filtering or policing you deem
> > required to massage the flow.
> > 
> >  
> > Scalability is high in my opinion. the route maps on
> > the borders can be a
> > part of your template. announcing the destination
> > address with the special
> > community would have to be done on only one router (sinkhole is a 
> > good idea).
> >  
> > As for where to place the route maps or policy
> > statements, IMHO, this should
> > be applied anywhere a large attack can enter from.
> > (I.e Transit links or to
> > Peers with large number of subscribers) I wouldn't
> > apply it on a customer
> > interface since a customer is in my reach and can be contacted
> > immediately in case of an attack initiated by him. Another place
> > to apply it on is at
> > peering interfaces where an access-list or policing
> > statement might bring
> > down interface performance. at this point traffic
> > can be pulled in the
> > sinkhole router for the policies to be applied.
> >  
> > This technique can be used in high volume attacks
> > where source based
> > filtering is not possible because of spoofing or
> > high number of source IP
> > addresses. In many cases ISP's are forced to kill
> > traffic destined to a
> > single /32 to relief the rest of the customer
> > network from suffering from
> > the high volume of traffic. I would also use this in
> > cases of attacks
> > targeting infrastructure.
> >  
> > As for the last question, yes I have built this in a
> > large lab and got the
> > result I expected.
> >  
> > Hope this answers your questions.
> >  
> > Regards
> > DT
> >  
> > 
> > From: Abarbanel, Benjamin [mailto:[email-masked]]
> > Sent: Friday, July 26, 2002 4:55 PM
> > To: 'Turk, Doughan A.'; [email-masked]
> > Subject: RE: Need your opinion of this.
> > 
> > 
> > Doughan:
> >   Let me see if I understand the fundamental point
> > of your draft. You want
> > BGP via communities to be the big MAGNET that
> > catches all the denial of
> > service attack routes and shoves them through some
> > sink hole router that
> > will keep statistics and knowledge of the source of
> > the attack and be able
> > to report it to the operator?
> >  
> > Did I summarize it properly?
> >  
> > My first reaction, is what kind of overhead will
> > these techniques impose on
> > the routers and the sink hole one?
> >  
> > Will it be scallable?
> >  
> > Can this be done only on border routers, Router
> > Reflectors? To minimize the
> > processing load needed?
> >  
> > What real life cases have you seen where this
> > mechanism was needed?
> >  
> > Have you prototyped this idea and gotten some
> > empirical data?
> >  
> > Ben
> > 
> > From: Turk, Doughan A. [mailto:[email-masked]]
> > Sent: Friday, July 26, 2002 4:09 PM
> > To: [email-masked]
> > Cc: Turk, Doughan A.
> > Subject: Need your opinion of this.
> > 
> > 
> > Folks,
> >  
> > I would like to get some discussion started on a
> > draft I wrote. It was
> > suggested to me to have the draft worked on under
> > this group. I appreciate
> > any constructive comments and pointers. In summary,
> > the draft could be
> > classified as an operational one. It is an
> > enhancement of a technique that
> > already exists but relying more on BGP as a protocol
> > to help in network
> > security issues.
> >  
> > A copy of the draft could be found at
> >
> http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
> >
> <http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt>
> > 
> >  
> > Regards
> > Doughan Turk
> >  
> >   _____
> > 
> > 
> >  <http://www.bell.ca/> Bell Canada Logo
> > 
> > Doughan Turk, M.Eng., CCIE#7274
> > Bell Canada, IP Engineering
> > Flr 3, 100 Wynford Dr.
> > Toronto M3C 1K4
> > 
> > Voice: (416) 353-0368
> > Fax: (416) 445-9893
> > Pager: (416) 685-0590
> >  <http://www.bell.ca/> Bell Canada
> > 
> >  Cisco Certified Internetwork Expert 
> > <http://www3.sympatico.ca/doughan/ccie_large.gif>
> > 
> >          "To Engineer and Integrate world class
> >                               IP Services and
> > Technologies for BCE" 	
> >  
> > 
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Health - Feel better, live better http://health.yahoo.com
> 
> 

From: "Turk, Doughan A." <[email-masked]>
To: [email-masked], "Turk, Doughan A." <[email-masked]>
Cc: [email-masked]
Subject: RE: Need your opinion of this.
Date: Tue, 30 Jul 2002 17:17:19 -0400
Sender: [email-masked]

Andrew,

1) Abuse is not a possibility. While this technique adds no more risk to the
one you get from a fat finger. In the route map match statements you would
add a match for AS-path ^$ this will guarantee that no eBGP peer can
compromise your set up. If you are worried about some one in operations to
miss use it, then the technique adds no more risk.

2) We need a route that is statically sent to a null interface on your edge
routers, You can surely your any of your public IP space is they are of
abandons but logic says RFC 1918 should not be routed out side of your AS
and therefore many ISPs have already in place a set of static routes to drop
those routes. Again, Juniper does that be default in what they call Martian
I believe. ** Juniper folks, correct me if I'm wrong**

3) I'm using routing to effectively acts as an basic ACL to drop traffic. I
can't comment on whether or not this should be on this WG.

Thanks for your comments
DT

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]

Sent: Tuesday, July 30, 2002 1:20 PM
To: Turk, Doughan A.
Cc: [email-masked]
Subject: Re: Need your opinion of this.

Hi Doughan,

Going through the draft, a couple of thoughts came up:

1) Why use BGP communities?  It seems like that just exposes the network to
the possibility of abuse of those communities.  The NOC or Security group
can just log into the router in question, and apply the NH statement.

2) Why use RFC 1918?  One could also use public space.

3) This probably belongs in ops, not routing, but that is an admin detail.

Andrew

On Fri, Jul 26, 2002 at 04:09:19PM -0400, Turk, Doughan A. wrote:
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> X-Server-Uuid: b85f21a3-cfd1-11d3-8401-00104bf46ab7
> From: "Turk, Doughan A." <[email-masked]>
> To: [email-masked]
> Cc: "Turk, Doughan A." <[email-masked]>
> Subject: Need your opinion of this.
> Date: Fri, 26 Jul 2002 16:09:19 -0400
> X-Mailer: Internet Mail Service (5.5.2653.19)
> X-WSS-ID: 115F73FE134713-01-01
> Precedence: bulk
> X-OriginalArrivalTime: 26 Jul 2002 20:10:58.0281 (UTC) 
> FILETIME=[8E32D590:01C234E0]
> 
> Folks,
>  
> I would like to get some discussion started on a draft I wrote. It was 
> suggested to me to have the draft worked on under this group. I 
> appreciate any constructive comments and pointers. In summary, the 
> draft could be classified as an operational one. It is an enhancement 
> of a technique that already exists but relying more on BGP as a 
> protocol to help in network security issues.
>  
> A copy of the draft could be found at 
> http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
> <http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt>
>  
> Regards
> Doughan Turk
>  
>   _____
> 
> 
>  <http://www.bell.ca/> Bell Canada Logo
> 
> Doughan Turk, M.Eng., CCIE#7274
> Bell Canada, IP Engineering
> Flr 3, 100 Wynford Dr.
> Toronto M3C 1K4
> 
> Voice: (416) 353-0368
> Fax: (416) 445-9893
> Pager: (416) 685-0590
>  <http://www.bell.ca/> Bell Canada
> 
>  Cisco Certified Internetwork Expert 
> <http://www3.sympatico.ca/doughan/ccie_large.gif>
> 
>          "To Engineer and Integrate world class
>                               IP Services and Technologies for BCE" 	
>  

From: "Turk, Doughan A." <[email-masked]>
To: "Jeffrey Haas" <[email-masked]>, "Turk, Doughan A." <[email-masked]>
Cc: [email-masked]
Subject: RE: Need your opinion of this.
Date: Tue, 30 Jul 2002 17:05:15 -0400
Sender: [email-masked]

Jeff, comments inline

> I would like to get some discussion started on a draft I wrote. It was 
> suggested to me to have the draft worked on under this group.

Although this is BGP, this seems more of something along the lines of
operations.  But I'm not going to debate that strongly.

*** As far as I know this technique could not be done using other routing
protocols. It is totally related to BGP ***

> I appreciate
> any constructive comments and pointers. In summary, the draft could be 
> classified as an operational one. It is an enhancement of a technique 
> that already exists but relying more on BGP as a protocol to help in 
> network security issues.

Did I see this as a presentation at a NANOG, or was that a UU-Net/Worldcom
presentation?  This sounds very familiar.

*** As explained in the draft, this is an enhancement to a technique which
in this case is something you saw in Nanog. The new technique builds on the
existing one. The technique adds a way to pull traffic into a tunnel, please
refer to the draft for more information.***

In any case, the technique seems to be pretty solid.  My biggest
observations about the draft:

1. Your sink addresses are always mentioned as RFC 1918.  Sure, you can do
this, but why not some prefix within your own network? 2. A little more talk
about filtering the internal control communities at the edges may be
warranted in the security considerations section.

*** In an ISP environment you would most likely see RFC 1918 routes
statically sent to a null interface on many routers especially edge routers.
On a Juniper box I believe this is the default behavior and you need not to
add any static routes. Therefore riding on the back on an existing common
practice make sense. 

Thanks for the comments
DT

Date: Tue, 30 Jul 2002 16:51:58 -0400
From: Jeffrey Haas <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Susan Hares'" <[email-masked]>, Kunihiro Ishiguro <[email-masked]>, BGP mailing list <[email-masked]>
Subject: Re: TCP state machine doubts
Sender: [email-masked]

On Thu, Jul 25, 2002 at 11:19:11AM -0400, Abarbanel, Benjamin wrote:
> There are various phases that BGP goes through that one
> might want to use multiple threads. The inter-locking or inter-working
> between the threads could affect the state transitions. For example, Say you
> are in Establish state and something happens to the session and it is
> disconnected. In a single threaded environment it is easy, first come, first
> served. In a multi-threaded case with thread priorities and interruptions,
> it might get tricky at times. Some assumptions or expectations by the state
> machine on actions it should take with regards to semaphore locks (RIB
> Table, etc.), thread prioritization, processing, pre-emption and so forth
> might be helpfull. Maybe these actions should only be suggestions not
> required ways for implementations.

There are certain situations where the FSM suggests that things be
done in certain events where an implementation *could* do things
in a slightly different order.

For example, after the TCP connection has completed, you are supposed
to send your messages *before* reading the socket.  In a multi-threaded
environment, you may obviously not do this.  Indeed, for passive or
unconfigured peers, you may not even send your message immediately.

The FSM would suggest where your synchronization points need to go.
Happily, most of this work seems to need to be done during peering setup.

For the most part, like the 3-rib abstraction, you should be able
to do whatever you want inside your black box so long as it looks
right on the wire.  The FSM gives guidance on what probably is
happening in the box to get things out as you expect them to.

More specific examples might be helpful.

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 30 Jul 2002 16:44:36 -0400
From: Jeffrey Haas <[email-masked]>
To: "Pusz, MateuszX" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: CEASE problems
Sender: [email-masked]

On Fri, Jul 26, 2002 at 09:07:42AM +0200, Pusz, MateuszX wrote:
> I have one problem with CEASE message. 'draft-ietf-idr-bgp4-17' says in 6.7:
>    In absence of any fatal errors (that are indicated in this section),
>    a BGP peer may choose at any given time to close its BGP connection
>    by sending the NOTIFICATION message with Error Code Cease. However,
>    the Cease NOTIFICATION message must not be used when a fatal error
>    indicated by this section does exist.
> 
> but in section 8-Established state:
>    In response to a stop event initiated by an operator:
>       - release all resources (including deleting all routes),
>       - set ConnectRetryCnt to zero (0),
>       - set connect retry timer to zero (0), and
>       - transition to the Idle.
> 
> so how shuold it be? Should BGP router in such a situation send that
> notification message or not?

You are correct, the NOTIFY should be sent.

This is correctly documented in:
http://www.ietf.org/internet-drafts/draft-hares-bgp-statemt-01.txt

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 30 Jul 2002 16:37:12 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: "Turk, Doughan A." <[email-masked]>, [email-masked]
Subject: Re: Need your opinion of this.
Sender: [email-masked]

On Tue, Jul 30, 2002 at 10:19:53AM -0700, [email-masked] wrote:
> 1) Why use BGP communities?  It seems like that just exposes the network
> to the possibility of abuse of those communities.  The NOC or Security
> group can just log into the router in question, and apply the NH statement.

The trick is to distribute it across your network.
Dropping something like this as a host route into your IGP could do
the trick - you could even tag the routes to let you know they're
"special".
However, what if you need to cross igp domains?

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 30 Jul 2002 10:19:53 -0700
From: [email-masked]
To: "Turk, Doughan A." <[email-masked]>
Cc: [email-masked]
Subject: Re: Need your opinion of this.
Sender: [email-masked]

Hi Doughan,

Going through the draft, a couple of thoughts came up:

1) Why use BGP communities?  It seems like that just exposes the network
to the possibility of abuse of those communities.  The NOC or Security
group can just log into the router in question, and apply the NH statement.

2) Why use RFC 1918?  One could also use public space.

3) This probably belongs in ops, not routing, but that is an admin detail.

Andrew

On Fri, Jul 26, 2002 at 04:09:19PM -0400, Turk, Doughan A. wrote:
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> X-Server-Uuid: b85f21a3-cfd1-11d3-8401-00104bf46ab7
> From: "Turk, Doughan A." <[email-masked]>
> To: [email-masked]
> Cc: "Turk, Doughan A." <[email-masked]>
> Subject: Need your opinion of this.
> Date: Fri, 26 Jul 2002 16:09:19 -0400
> X-Mailer: Internet Mail Service (5.5.2653.19)
> X-WSS-ID: 115F73FE134713-01-01
> Precedence: bulk
> X-OriginalArrivalTime: 26 Jul 2002 20:10:58.0281 (UTC) FILETIME=[8E32D590:01C234E0]
> 
> Folks, 
>  
> I would like to get some discussion started on a draft I wrote. It was
> suggested to me to have the draft worked on under this group. I appreciate
> any constructive comments and pointers. In summary, the draft could be
> classified as an operational one. It is an enhancement of a technique that
> already exists but relying more on BGP as a protocol to help in network
> security issues.
>  
> A copy of the draft could be found at
> http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
> <http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt> 
>  
> Regards
> Doughan Turk
>  
>   _____  
> 
> 
>  <http://www.bell.ca/> Bell Canada Logo
> 
> Doughan Turk, M.Eng., CCIE#7274
> Bell Canada, IP Engineering
> Flr 3, 100 Wynford Dr. 
> Toronto M3C 1K4
> 
> Voice: (416) 353-0368
> Fax: (416) 445-9893
> Pager: (416) 685-0590
>  <http://www.bell.ca/> Bell Canada
> 
>  Cisco Certified Internetwork Expert
> <http://www3.sympatico.ca/doughan/ccie_large.gif> 
> 
>          "To Engineer and Integrate world class
>                               IP Services and Technologies for BCE" 	
>  

Date: Tue, 30 Jul 2002 11:27:01 -0400
From: Jeffrey Haas <[email-masked]>
To: "Turk, Doughan A." <[email-masked]>
Cc: [email-masked]
Subject: Re: Need your opinion of this.
Sender: [email-masked]

On Fri, Jul 26, 2002 at 04:09:19PM -0400, Turk, Doughan A. wrote:
> I would like to get some discussion started on a draft I wrote. It was
> suggested to me to have the draft worked on under this group.

Although this is BGP, this seems more of something along the
lines of operations.  But I'm not going to debate that strongly.

> I appreciate
> any constructive comments and pointers. In summary, the draft could be
> classified as an operational one. It is an enhancement of a technique that
> already exists but relying more on BGP as a protocol to help in network
> security issues.

Did I see this as a presentation at a NANOG, or was that a UU-Net/Worldcom
presentation?  This sounds very familiar.

In any case, the technique seems to be pretty solid.  My biggest
observations about the draft:

1. Your sink addresses are always mentioned as RFC 1918.  Sure, you
can do this, but why not some prefix within your own network?
2. A little more talk about filtering the internal control communities
at the edges may be warranted in the security considerations section.

> Doughan Turk

-- 
Jeff Haas 
NextHop Technologies

Date: Thu, 11 Jul 2002 09:40:14 +0800
From: lidefeng <[email-masked]>
Subject: about:draft-wilfong-ibgp-oscillations-00.txt
To: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Cc: [email-masked]
Sender: [email-masked]

Hi,

    In Section 3 An Extension to BGP of  draft-wilfong-ibgp-oscillations-00.txt,there is an example as to the relevant figure as follows:

 "  As an example, consider again the configuration shown in Figure 1.
   The modified BGP process will have router RR1 always announcing
   the route through C1 even though it will choose the route
   through C2 as its best route.
   Similarly, RR2 will choose the route through C3 and RR3 will
   choose the route through C1."

IMO, what you said  that RR1 will choose the route through C2,RR2 will choose the route through C3 and RR3 will
   choose the route through C1 are referred to IGP route , as to BGP,  in  the light of RFC 1771,the RR1  will reflect the route learned from C1 to
RR2 and RR3,the RR2  will reflect the route learned from C2 to RR1 and RR3, the RR3  will reflect the route learned from C3 to RR1 and RR2,
at last, every BGP route advertised by C1,C2 or C3 will reside in RR1,RR2,RR3,C1,C2,C3,and every  prefix will have three nexthops.

As to IGP, there are another scheme to compute the IGP route, if there aren't  route redistribution between IGP and BGP, the IGP route 
pattern and BGP route pattern are independent respectively, and there is little chance of route oscillation.

Li Defeng

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Turk, Doughan A.'" <[email-masked]>, PamSri <[email-masked]>
Cc: [email-masked]
Subject: RE: Need your opinion of this.
Date: Mon, 29 Jul 2002 10:06:14 -0400
Sender: [email-masked]

Doughan:

Since some of the traffic that is trapped with the DoS traffic is going to
be
redirected to the sinkhole router. would that cause those poor cutomers to
loose thier communications?

What if the sinkhole router was smart enough to filter through the good
packets from the bad, could it then redirect the good packets back to their
intended destination? 

If so, that implies the sinkhole routers have alot more cycles to handle all
the load that will be imposed on them by this service.

Is source and destination address the only things needed to filter good
packets from bad ones?

My instinct tells me it is better to stop DoS traffic before it enters the
cloud on the border router, then try to manage it at some redirected
sinkhole router. 

Communities attribute although they tend to converge on some Next Hop would
tend to propagate the traffic through the cloud and thereby cause more
loading conditions on the intermediary routers.

Ben

> From: Turk, Doughan A. [mailto:[email-masked]]
> Sent: Friday, July 26, 2002 9:24 PM
> To: PamSri
> Cc: [email-masked]
> Subject: RE: Need your opinion of this.
> 
> 
> I am using BGP to help stopping DoS and DDoS traffic. BGP can 
> send updates
> quickly and route-maps can translate communities to stop 
> traffic destined to
> a certain destination. TCP is not the issue here. Attackers 
> usually relay on
> the fact that most routers out there do destination based 
> routing without
> checking the validity of the source address in case it is spoofed.
> 
> DT
> 
> From: PamSri [mailto:[email-masked]] 
> Sent: Friday, July 26, 2002 9:08 PM
> To: Turk, Doughan A.
> Subject: RE: Need your opinion of this.
> 
> 
> Hi Turk ,
>          Just started to read through your doc. I have
> a question :
> BGP is an application from the perspective of TCP. Why
> do you see the need for provding such security at the 
> application level than
> TCP ? Do you think the security features in TCP as inadequate 
> ? It would be
> really a good idea to provide security at the underlying 
> layers than at this
> layer. 
> 
> sri
> 
> --- "Turk, Doughan A." <[email-masked]> wrote:
> > Benjamin,
> >  
> > BGP will be used to alter next-hop address for a
> > particular destination
> > address (representing a customer under attack) on
> > selected Routers. Instead
> > of using BGP to announce a different next-hop for
> > the destination address
> > everywhere in your network. we can use communities
> > to make it selective.
> > once that is achieved, your next hop can be a null
> > interface or a Tunnel
> > interface where the traffic is then pulled to the
> > sinkhole router.
> >  
> > Based on Attack signature you could do various
> > packet lever filtering ( with
> > hardware as a constrain),  simply rate limit the
> > flow or send it out through
> > a shared media (Ethernet) for sniffing after you
> > pull it to a sinkhole
> > router.
> >  
> > As for overheads on routers in the network, there is
> > hardly any since you're
> > only altering next-hop on border routers or edge
> > routers. MPLS tunnel are
> > what I used in my lab. as for the sinkhole router,
> > one can invest in a high
> > performance line card. The sinkhole router should be
> > powerful enough to
> > handle whatever filtering or policing you deem
> > required to massage the flow.
> > 
> >  
> > Scalability is high in my opinion. the route maps on
> > the borders can be a
> > part of your template. announcing the destination
> > address with the special
> > community would have to be done on only one router
> > (sinkhole is a good
> > idea).
> >  
> > As for where to place the route maps or policy
> > statements, IMHO, this should
> > be applied anywhere a large attack can enter from.
> > (I.e Transit links or to
> > Peers with large number of subscribers) I wouldn't
> > apply it on a customer
> > interface since a customer is in my reach and can be contacted 
> > immediately in case of an attack initiated by him. Another place
> > to apply it on is at
> > peering interfaces where an access-list or policing
> > statement might bring
> > down interface performance. at this point traffic
> > can be pulled in the
> > sinkhole router for the policies to be applied.
> >  
> > This technique can be used in high volume attacks
> > where source based
> > filtering is not possible because of spoofing or
> > high number of source IP
> > addresses. In many cases ISP's are forced to kill
> > traffic destined to a
> > single /32 to relief the rest of the customer
> > network from suffering from
> > the high volume of traffic. I would also use this in
> > cases of attacks
> > targeting infrastructure.
> >  
> > As for the last question, yes I have built this in a
> > large lab and got the
> > result I expected.
> >  
> > Hope this answers your questions.
> >  
> > Regards
> > DT
> >  
> > 
> > From: Abarbanel, Benjamin [mailto:[email-masked]]
> > Sent: Friday, July 26, 2002 4:55 PM
> > To: 'Turk, Doughan A.'; [email-masked]
> > Subject: RE: Need your opinion of this.
> > 
> > 
> > Doughan:
> >   Let me see if I understand the fundamental point
> > of your draft. You want
> > BGP via communities to be the big MAGNET that
> > catches all the denial of
> > service attack routes and shoves them through some
> > sink hole router that
> > will keep statistics and knowledge of the source of
> > the attack and be able
> > to report it to the operator?
> >  
> > Did I summarize it properly?
> >  
> > My first reaction, is what kind of overhead will
> > these techniques impose on
> > the routers and the sink hole one? 
> >  
> > Will it be scallable? 
> >  
> > Can this be done only on border routers, Router
> > Reflectors? To minimize the
> > processing load needed?
> >  
> > What real life cases have you seen where this
> > mechanism was needed?
> >  
> > Have you prototyped this idea and gotten some
> > empirical data?
> >  
> > Ben
> > 
> > From: Turk, Doughan A. [mailto:[email-masked]]
> > Sent: Friday, July 26, 2002 4:09 PM
> > To: [email-masked]
> > Cc: Turk, Doughan A.
> > Subject: Need your opinion of this.
> > 
> > 
> > Folks, 
> >  
> > I would like to get some discussion started on a
> > draft I wrote. It was
> > suggested to me to have the draft worked on under
> > this group. I appreciate
> > any constructive comments and pointers. In summary,
> > the draft could be
> > classified as an operational one. It is an
> > enhancement of a technique that
> > already exists but relying more on BGP as a protocol
> > to help in network
> > security issues.
> >  
> > A copy of the draft could be found at
> >
> http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
> >
> <http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt>
> > 
> >  
> > Regards
> > Doughan Turk
> >  
> >   _____  
> > 
> > 
> >  <http://www.bell.ca/> Bell Canada Logo
> > 
> > Doughan Turk, M.Eng., CCIE#7274
> > Bell Canada, IP Engineering
> > Flr 3, 100 Wynford Dr. 
> > Toronto M3C 1K4
> > 
> > Voice: (416) 353-0368
> > Fax: (416) 445-9893
> > Pager: (416) 685-0590
> >  <http://www.bell.ca/> Bell Canada
> > 
> >  Cisco Certified Internetwork Expert
> > <http://www3.sympatico.ca/doughan/ccie_large.gif> 
> > 
> >          "To Engineer and Integrate world class
> >                               IP Services and
> > Technologies for BCE" 	
> >  
> > 
> > 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Health - Feel better, live better
> http://health.yahoo.com
> 
> 

Date: Wed, 10 Jul 2002 14:21:27 +0800
From: lidefeng <[email-masked]>
Subject: about: draft-ietf-idr-route-oscillation-01.txt
To: [email-masked], [email-masked], [email-masked], [email-masked]
Cc: [email-masked]
Sender: [email-masked]

Hi,

    In "Section 4.1. Route Reflection and Type I Churn" of draft-ietf-idr-route-oscillation-01.txt, you stated as follows: 

    1) Ra has the following installed in its BGP table with
        the path learned via AS2 marked best:

                          NEXT_HOP
           AS_PATH  MED   IGP Cost
           -----------------------
             6 100    1          4
          * 10 100   10       5

I have some questiion:
(1)  In figure 1, I can't find any AS numbered as AS2,
(2) I wonder why Ra learned the wrong route information?

Because I was confused from the beginning, so I can't grope along the draft, I hope you can clarity the above questions
so that I can advance along.

Li Defeng

Date: Mon, 29 Jul 2002 08:35:19 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: draft-ietf-idr-bgp4-18.txt
To: lidefeng <[email-masked]>, [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

umm .. what i meant was that the suggested changes should go into the draft
mentioned in the subject i.e 18th.
apologies for the confusion .. i was referring to the draft 17 only!

----- Original Message -----
From: "lidefeng" <[email-masked]>
To: "Manav Bhatia" <[email-masked]>; <[email-masked]>
Sent: Saturday, July 27, 2002 5:51 PM
Subject: Re: draft-ietf-idr-bgp4-18.txt

| hi,
|
|     I can only get the draft-ietf-idr-bgp4-17.txt from the
"Internet-Drafts Index ", and where can I get the newest
version:Internet-Drafts Index , which you are discussing.And I wonder why
"Internet-Drafts Index" doesn't list the latest version of draft. I am
always downloading the draft from there, and I am afraid I will drag behind
the discuss.
|
| Regards
|
| Li Defeng
|
|
| ----- Original Message -----
| From: "Manav Bhatia" <[email-masked]>
| To: <[email-masked]>
| Sent: Friday, July 26, 2002 3:12 PM
| Subject: draft-ietf-idr-bgp4-18.txt
|
|
| > sec 6.5 of the latest draft states - " If  system does not receive
| > successive KEEPALIVE and/or UPDATE and/or NOTIFICATION messages within
the
| > period specified in the Hold Time field of the OPEN message, then the
| > NOTIFICATION message with   Hold Timer Expired Error Code must be sent
and
| > the BGP connection closed"
| >
| > IMHO we must update the draft to also include other BGP messages (Route
| > Refresh, Dynamic Capability, INFORM) which when received by the local
| > system act as surrogate for the KEEPALIVE messages it would expect from
the
| > remote end. The local system upon receiving such messages must restart
its
| > Hold Timer
| >
| > Similarly the draft should mention that each time the local system
sends
| > such BGP messages it should restart its KeepAlive Timer, unless the
| > negotiated HoldTime value is zero.
| >
| > These changes can be reflected in the BGP draft or can be done in the
| > drafts introducing such BGP messages.
| >
| > Regards,
| > Manav
| >
| > ----
| > "When you are courting a nice girl an hour seems like a second. When
you
| > sit on a red-hot cinder a second seems like an hour. That's
relativity."
| >
| > -Albert Einstein, on relativity
| >
| >

To: lidefeng <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: about draft-ietf-idr-as4bytes-05.txt 
Date: Sat, 27 Jul 2002 23:34:24 -0700
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Hi, 

> Date: Sat, 27 Jul 2002 20:58:22 +0800
> From: lidefeng <[email-masked]>
> Subject: about draft-ietf-idr-as4bytes-05.txt
> To: [email-masked], [email-masked]
> Cc: [email-masked]
> Message-id: <[email-masked]>
> 
> hi,
> 
>    In "Section7. IANA Consideration" of  "draft-ietf-idr-as4bytes-05.txt" ,it
> states that "The AS number for AS_TRANS has been assigned by the IANA.", I want
> to know what is the AS_TRANS number assigned by the IANA,

The AS_TRAN is listed in http://www.iana.org/assignments/as-numbers:

    Specially designated Autonomous System Numbers:

    23456     AS_TRAN             [draft-ietf-idr-as4bytes-04.txt]

> where I can find the number assignment scheme,

Do not see any reason why the assignment scheme would be different
from the current policy.

> and all the NEW BGP SPEAKER user the only one number ?

No. (Use AS_TRAN only when a 4byte-ASN speaker talks to an OLD peer
that does not understand 4-byte AS.)

-- Enke

> if so, how to handle the situation when they update across several AS? 
> 
> Regards
> 
> Li Defeng 

From: "Turk, Doughan A." <[email-masked]>
To: "TANDEL Sebastien" <[email-masked]>, [email-masked]
Subject: RE: Need your opinion of this.
Date: Sat, 27 Jul 2002 15:16:36 -0400
Sender: [email-masked]

Tandel,

no-advertise is used to avoid the advertisement of a prefix to all peers
including iBGP. The one that stops a prefix from going out to eBGP neighbors
is the no-export or no-export-subconfed. To keep things in prospective, a
router should not crash because of attack traffic transiting through it.
Routing protocols take precedence over regular traffic in my understanding.
Even if an attack is so big in volume it will never exceed your link
capacity and therefore your upstream provider will be dropping all excess
traffic. 

To clarify one thing, When you identify a destination address as a target
you do not stamp the entire network with the special community. Lets for
example take a case of a customer with a /24 of which one host /32 is under
attack at this point you announce the /32 in addition to the /24 where the
/32 has the special community but the /24 gets advertised normally.

Regards
DT

-----Original Message-----
From: TANDEL Sebastien [mailto:[email-masked]] 
Sent: Saturday, July 27, 2002 10:08 AM
To: Turk, Doughan A.
Subject: Re: Need your opinion of this.

Hello,

   It's just a question about the use of the community...
   If I have understood your method, when you overwrite the community with
the well-known community no-advertise, it's to avoid the advertisement of
this community in the others eBGP sessions.

   So why don't you use the extended communities ?
   (IMHO) There are two reasons to use the extended communities and not the
community. With the extended community, you can set the non transitive bit.
This has two consequences:

   	1) this community is not advertised to other eBGP sessions
   	2) While there is an attack and if there is a drop session with
           the BGP router (by which the attack traffic goes through) in
           session with the border router (i.e. R1) which recognizes the
           regular community, the prefix under attack will not be
           advertised anymore by R1.
           So there is two cases :
   		a) this prefix is not reachable anymore. In this cases
                   there is a black hole (of course this is the case for
                   an attack routed to a network for an analysis, but
                   there will be no more analysis).
   		b) this prefix can be reach but by another route. And in
                   this case, you have to restart an other operation like
                   described in your draft on another border router.
   	   But if you are using the extended community with the non
           transitive bit set, this prefix will always be advertised in
           any cases and without the community value.

----------------------------
Tandel Sébastien
2ème licence en informatique
F.U.N.D.P.
[email-masked]
----------------------------

On Fri, 26 Jul 2002, Turk, Doughan A. wrote:

> Folks,
>
> I would like to get some discussion started on a draft I wrote. It was 
> suggested to me to have the draft worked on under this group. I 
> appreciate any constructive comments and pointers. In summary, the 
> draft could be classified as an operational one. It is an enhancement 
> of a technique that already exists but relying more on BGP as a 
> protocol to help in network security issues.
>
> A copy of the draft could be found at 
> http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
> <http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt>
>
> Regards
> Doughan Turk
>
>   _____
>
>
>  <http://www.bell.ca/> Bell Canada Logo
>
> Doughan Turk, M.Eng., CCIE#7274
> Bell Canada, IP Engineering
> Flr 3, 100 Wynford Dr.
> Toronto M3C 1K4
>
> Voice: (416) 353-0368
> Fax: (416) 445-9893
> Pager: (416) 685-0590
>  <http://www.bell.ca/> Bell Canada
>
>  Cisco Certified Internetwork Expert 
> <http://www3.sympatico.ca/doughan/ccie_large.gif>
>
>          "To Engineer and Integrate world class
>                               IP Services and Technologies for BCE"
>
>

From: "Turk, Doughan A." <[email-masked]>
To: PamSri <[email-masked]>
Cc: [email-masked]
Subject: RE: Need your opinion of this.
Date: Sat, 27 Jul 2002 14:59:46 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C2359F.C6AF3AC0
Content-Type: text/plain; 
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Sri,
 
The draft is not addressing a box or a protocol security. I am using BGP
capabilities to assist in fighting DoS attacks. The reason why BGP is a
protocol suitable for this job is it's many attributes like "communities".
Since routing is the driver of traffic why not develop ways to use routing
protocols to assist in fighting attacks.
 
Regards
DT

-----Original Message-----
From: PamSri [mailto:[email-masked]] 
Sent: Saturday, July 27, 2002 11:19 AM
To: Turk, Doughan A.
Cc: [email-masked]
Subject: RE: Need your opinion of this.

Hi Turk , 

            Here are a few more questions : 

1. Is DoS a problem specific to BGP or the box ? 

2. So do we provide this kind of solution to all the routing protocols.

3. Here's what i think : DoS, Anti-Spoofing, Firewalls etc.. can be
catagorized as IP Services. Do we want to bring these concepts into IP
routing ? 

4. There are vendors who provide these IP Services along with routing by
clearly making a distinction between these two concepts. 

Regards, 

sri 

 "Turk, Doughan A." wrote: 

The draft does not suggest a way of verifying reverse path or validating
source address. My comment was just an answer to Sri's questions. BGP is the
only routing protocol I can think of with the capabilities to do what the
draft suggests. More information about the draft can be found at
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt

DT

-----Original Message-----
From: Justin Fletcher [mailto:[email-masked]] 
Sent: Friday, July 26, 2002 9:11 PM
To: Turk, Doughan A.
Cc: [email-masked]
Subject: RE: Need your opinion of this.

On Fri, 2002-07-26 at 18:24, Turk, Doughan A. wrote:
> I am using BGP to help stopping DoS and DDoS traffic. BGP can send 
> updates quickly and route-maps can translate communities to stop 
> traffic destined to a certain destination. TCP is not the issue here. 
> Attackers usually relay on the fact that most routers out there do 
> destination based routing without checking the validity of the source 
> address in case it is spoofed.
> 
> DT
> 

Is BGP the place to do this? Note this section from RFC1812:

5.3.8 Source Address Validation

A router SHOULD IMPLEMENT the ability to filter traffic based on a
comparison of the source address of a packet and the forwarding table
for a logical interface on which the packet was received. If this
filtering is enabled, the router MUST silently discard a packet if
the interface on which the packet was received is not the interface
on which a packet would be forwarded to reach the address contained
in the source address. In simpler terms, if a router wouldn't route
a packet containing this address through a particular interface, it
shouldn't believe the address if it appears as a source address in a
packet read from this interface.

--
Justin Fletcher
Proficient Networks, Inc.

  _____  

Do You Yahoo!?
Yahoo! Health <http://health.yahoo.com/>  - Feel better, live better

------_=_NextPart_001_01C2359F.C6AF3AC0
Content-Type: text/html; 
 charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<TITLE>Message</TITLE>

<META content="MSHTML 6.00.2716.2200" name=GENERATOR></HEAD>
<BODY>
<DIV><SPAN class=993394918-27072002><FONT face=Arial color=#0000ff 
size=2>Sri,</FONT></SPAN></DIV>
<DIV><SPAN class=993394918-27072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=993394918-27072002><FONT face=Arial color=#0000ff size=2>The 
draft is not addressing a box or a protocol security. I am using BGP 
capabilities to assist in fighting DoS attacks. The reason why BGP is a protocol 
suitable&nbsp;for this job is it's many attributes like "communities". Since 
routing is the driver of traffic why not develop ways to use routing protocols 
to assist in fighting attacks.</FONT></SPAN></DIV>
<DIV><SPAN class=993394918-27072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=993394918-27072002><FONT face=Arial color=#0000ff 
size=2>Regards</FONT></SPAN></DIV>
<DIV><SPAN class=993394918-27072002><FONT face=Arial color=#0000ff 
size=2>DT</FONT></SPAN></DIV>
<BLOCKQUOTE style="MARGIN-RIGHT: 0px">
  <DIV></DIV>
  <DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left><FONT 
  face=Tahoma size=2>-----Original Message-----<BR><B>From:</B> PamSri 
  [mailto:[email-masked]] <BR><B>Sent:</B> Saturday, July 27, 2002 11:19 
  AM<BR><B>To:</B> Turk, Doughan A.<BR><B>Cc:</B> 
  [email-masked]<BR><B>Subject:</B> RE: Need your opinion of 
  this.<BR><BR></FONT></DIV>
  <P>Hi Turk , 
  <P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Here 
  are a few more questions : 
  <P>1. Is DoS a&nbsp;problem specific to BGP or the box ? 
  <P>2. So do we provide this kind of solution to all the routing 
  protocols.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
  <P>3. Here's what i think : DoS, Anti-Spoofing, Firewalls etc.. can be 
  catagorized as IP Services. Do we want to bring these concepts into IP routing 
  ? 
  <P>4. There are vendors who provide these IP Services along with routing by 
  clearly making a distinction between these two concepts. 
  <P>Regards, 
  <P>sri 
  <P>&nbsp;<B><I>"Turk, Doughan A." <[email-masked]></I></B>wrote: 
  <BLOCKQUOTE 
  style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">The 
    draft does not suggest a way of verifying reverse path or 
    validating<BR>source address. My comment was just an answer to Sri's 
    questions. BGP is the<BR>only routing protocol I can think of with the 
    capabilities to do what the<BR>draft suggests. More information about the 
    draft can be found 
    at<BR>http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt<BR><BR>DT<BR><BR>-----Original 
    Message-----<BR>From: Justin Fletcher [mailto:[email-masked]] 
    <BR>Sent: Friday, July 26, 2002 9:11 PM<BR>To: Turk, Doughan A.<BR>Cc: 
    [email-masked]<BR>Subject: RE: Need your opinion of this.<BR><BR><BR>On Fri, 
    2002-07-26 at 18:24, Turk, Doughan A. wrote:<BR>&gt; I am using BGP to help 
    stopping DoS and DDoS traffic. BGP can send <BR>&gt; updates quickly and 
    route-maps can translate communities to stop <BR>&gt; traffic destined to a 
    certain destination. TCP is not the issue here. <BR>&gt; Attackers usually 
    relay on the fact that most routers out there do <BR>&gt; destination based 
    routing without checking the validity of the source <BR>&gt; address in case 
    it is spoofed.<BR>&gt; <BR>&gt; DT<BR>&gt; <BR><BR>Is BGP the place to do 
    this? Note this section from RFC1812:<BR><BR>5.3.8 Source Address 
    Validation<BR><BR>A router SHOULD IMPLEMENT the ability to filter traffic 
    based on a<BR>comparison of the source address of a packet and the 
    forwarding table<BR>for a logical interface on which the packet was 
    received. If this<BR>filtering is enabled, the router MUST silently discard 
    a packet if<BR>the interface on which the packet was received is not the 
    interface<BR>on which a packet would be forwarded to reach the address 
    contained<BR>in the source address. In simpler terms, if a router wouldn't 
    route<BR>a packet containing this address through a particular interface, 
    it<BR>shouldn't believe the address if it appears as a source address in 
    a<BR>packet read from this interface.<BR><BR>--<BR>Justin 
    Fletcher<BR>Proficient Networks, Inc.<BR><BR><BR></BLOCKQUOTE>
  <P><BR>
  <HR SIZE=1>
  <B>Do You Yahoo!?</B><BR><A href="http://health.yahoo.com/">Yahoo! Health</A> 
  - Feel better, live better</BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C2359F.C6AF3AC0--

Date: Sat, 27 Jul 2002 08:18:47 -0700 (PDT)
From: PamSri <[email-masked]>
Subject: RE: Need your opinion of this.
To: "Turk, Doughan A." <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

--0-738754811-1027783127=:80858
Content-Type: text/plain; charset=us-ascii

Hi Turk ,
            Here are a few more questions :
1. Is DoS a problem specific to BGP or the box ? 
2. So do we provide this kind of solution to all the routing protocols.         
3. Here's what i think : DoS, Anti-Spoofing, Firewalls etc.. can be catagorized as IP Services. Do we want to bring these concepts into IP routing ?
4. There are vendors who provide these IP Services along with routing by clearly making a distinction between these two concepts.
Regards,
sri
 "Turk, Doughan A." wrote:The draft does not suggest a way of verifying reverse path or validating
source address. My comment was just an answer to Sri's questions. BGP is the
only routing protocol I can think of with the capabilities to do what the
draft suggests. More information about the draft can be found at
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt

DT

-----Original Message-----
From: Justin Fletcher [mailto:[email-masked]] 
Sent: Friday, July 26, 2002 9:11 PM
To: Turk, Doughan A.
Cc: [email-masked]
Subject: RE: Need your opinion of this.

On Fri, 2002-07-26 at 18:24, Turk, Doughan A. wrote:
> I am using BGP to help stopping DoS and DDoS traffic. BGP can send 
> updates quickly and route-maps can translate communities to stop 
> traffic destined to a certain destination. TCP is not the issue here. 
> Attackers usually relay on the fact that most routers out there do 
> destination based routing without checking the validity of the source 
> address in case it is spoofed.
> 
> DT
> 

Is BGP the place to do this? Note this section from RFC1812:

5.3.8 Source Address Validation

A router SHOULD IMPLEMENT the ability to filter traffic based on a
comparison of the source address of a packet and the forwarding table
for a logical interface on which the packet was received. If this
filtering is enabled, the router MUST silently discard a packet if
the interface on which the packet was received is not the interface
on which a packet would be forwarded to reach the address contained
in the source address. In simpler terms, if a router wouldn't route
a packet containing this address through a particular interface, it
shouldn't believe the address if it appears as a source address in a
packet read from this interface.

--
Justin Fletcher
Proficient Networks, Inc.

---------------------------------
Do You Yahoo!?
Yahoo! Health - Feel better, live better
--0-738754811-1027783127=:80858
Content-Type: text/html; charset=us-ascii

<P>Hi Turk ,
<P>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Here are a few more questions :
<P>1. Is DoS a&nbsp;problem specific to BGP or the box ? 
<P>2. So do we provide this kind of solution to all the routing protocols.&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
<P>3. Here's what i think : DoS, Anti-Spoofing, Firewalls etc.. can be catagorized as IP Services. Do we want to bring these concepts into IP routing ?
<P>4. There are vendors who provide these IP Services along with routing by clearly making a distinction between these two concepts.
<P>Regards,
<P>sri
<P>&nbsp;<B><I>"Turk, Doughan A." <[email-masked]></I></B>wrote:
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">The draft does not suggest a way of verifying reverse path or validating<BR>source address. My comment was just an answer to Sri's questions. BGP is the<BR>only routing protocol I can think of with the capabilities to do what the<BR>draft suggests. More information about the draft can be found at<BR>http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt<BR><BR>DT<BR><BR>-----Original Message-----<BR>From: Justin Fletcher [mailto:[email-masked]] <BR>Sent: Friday, July 26, 2002 9:11 PM<BR>To: Turk, Doughan A.<BR>Cc: [email-masked]<BR>Subject: RE: Need your opinion of this.<BR><BR><BR>On Fri, 2002-07-26 at 18:24, Turk, Doughan A. wrote:<BR>&gt; I am using BGP to help stopping DoS and DDoS traffic. BGP can send <BR>&gt; updates quickly and route-maps can translate communities to stop <BR>&gt; traffic destined to a certain destination. TCP is not the issue here. <BR>&gt; Attackers usually relay on the fact that most routers out there do <BR>&gt; destination based routing without checking the validity of the source <BR>&gt; address in case it is spoofed.<BR>&gt; <BR>&gt; DT<BR>&gt; <BR><BR>Is BGP the place to do this? Note this section from RFC1812:<BR><BR>5.3.8 Source Address Validation<BR><BR>A router SHOULD IMPLEMENT the ability to filter traffic based on a<BR>comparison of the source address of a packet and the forwarding table<BR>for a logical interface on which the packet was received. If this<BR>filtering is enabled, the router MUST silently discard a packet if<BR>the interface on which the packet was received is not the interface<BR>on which a packet would be forwarded to reach the address contained<BR>in the source address. In simpler terms, if a router wouldn't route<BR>a packet containing this address through a particular interface, it<BR>shouldn't believe the address if it appears as a source address in a<BR>packet read from this interface.<BR><BR>--<BR>Justin Fletcher<BR>Proficient Networks, Inc.<BR><BR><BR></BLOCKQUOTE><p><br><hr size=1><b>Do You Yahoo!?</b><br>
<a href="http://health.yahoo.com/">Yahoo! Health</a> - Feel better, live better
--0-738754811-1027783127=:80858--

Date: Sat, 27 Jul 2002 20:58:22 +0800
From: lidefeng <[email-masked]>
Subject: about draft-ietf-idr-as4bytes-05.txt
To: [email-masked], [email-masked]
Cc: [email-masked]
Sender: [email-masked]

hi,

   In "Section7. IANA Consideration" of  "draft-ietf-idr-as4bytes-05.txt" ,it states that "The AS number for AS_TRANS has been assigned by the IANA.", I want to know what is the AS_TRANS number assigned by the IANA, where I can find the number assignment scheme, and all the NEW BGP SPEAKER user the only one number ? if so, how to handle the situation when they update across several AS? 

Regards

Li Defeng 

Date: Sat, 27 Jul 2002 20:21:20 +0800
From: lidefeng <[email-masked]>
Subject: Re: draft-ietf-idr-bgp4-18.txt
To: Manav Bhatia <[email-masked]>, [email-masked]
Sender: [email-masked]

hi,

    I can only get the draft-ietf-idr-bgp4-17.txt from the "Internet-Drafts Index ", and where can I get the newest version:Internet-Drafts Index , which you are discussing.And I wonder why "Internet-Drafts Index" doesn't list the latest version of draft. I am always downloading the draft from there, and I am afraid I will drag behind the discuss.

Regards

Li Defeng

----- Original Message ----- 
From: "Manav Bhatia" <[email-masked]>
To: <[email-masked]>
Sent: Friday, July 26, 2002 3:12 PM
Subject: draft-ietf-idr-bgp4-18.txt

> sec 6.5 of the latest draft states - " If  system does not receive
> successive KEEPALIVE and/or UPDATE and/or NOTIFICATION messages within the
> period specified in the Hold Time field of the OPEN message, then the
> NOTIFICATION message with   Hold Timer Expired Error Code must be sent and
> the BGP connection closed"
> 
> IMHO we must update the draft to also include other BGP messages (Route
> Refresh, Dynamic Capability, INFORM) which when received by the local
> system act as surrogate for the KEEPALIVE messages it would expect from the
> remote end. The local system upon receiving such messages must restart its
> Hold Timer
> 
> Similarly the draft should mention that each time the local system sends
> such BGP messages it should restart its KeepAlive Timer, unless the
> negotiated HoldTime value is zero.
> 
> These changes can be reflected in the BGP draft or can be done in the
> drafts introducing such BGP messages.
> 
> Regards,
> Manav
> 
> ----
> "When you are courting a nice girl an hour seems like a second. When you
> sit on a red-hot cinder a second seems like an hour. That's relativity."
> 
> -Albert Einstein, on relativity
> 
> 

From: "Turk, Doughan A." <[email-masked]>
To: [email-masked]
Subject: RE: Need your opinion of this.
Date: Fri, 26 Jul 2002 22:18:48 -0400
Sender: [email-masked]

The draft does not suggest a way of verifying reverse path or validating
source address. My comment was just an answer to Sri's questions. BGP is the
only routing protocol I can think of with the capabilities to do what the
draft suggests. More information about the draft can be found at
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt

DT

-----Original Message-----
From: Justin Fletcher [mailto:[email-masked]] 
Sent: Friday, July 26, 2002 9:11 PM
To: Turk, Doughan A.
Cc: [email-masked]
Subject: RE: Need your opinion of this.

On Fri, 2002-07-26 at 18:24, Turk, Doughan A. wrote:
> I am using BGP to help stopping DoS and DDoS traffic. BGP can send 
> updates quickly and route-maps can translate communities to stop 
> traffic destined to a certain destination. TCP is not the issue here. 
> Attackers usually relay on the fact that most routers out there do 
> destination based routing without checking the validity of the source 
> address in case it is spoofed.
> 
> DT
> 

Is BGP the place to do this?  Note this section from RFC1812:

5.3.8 Source Address Validation
   
   A router SHOULD IMPLEMENT the ability to filter traffic based on a
   comparison of the source address of a packet and the forwarding table
   for a logical interface on which the packet was received.  If this
   filtering is enabled, the router MUST silently discard a packet if
   the interface on which the packet was received is not the interface
   on which a packet would be forwarded to reach the address contained
   in the source address.  In simpler terms, if a router wouldn't route
   a packet containing this address through a particular interface, it
   shouldn't believe the address if it appears as a source address in a
   packet read from this interface.

--
Justin Fletcher
Proficient Networks, Inc.

Subject: RE: Need your opinion of this.
From: Justin Fletcher <[email-masked]>
To: "Turk, Doughan A." <[email-masked]>
Cc: [email-masked]
Date: 26 Jul 2002 18:10:55 -0700
Sender: [email-masked]

On Fri, 2002-07-26 at 18:24, Turk, Doughan A. wrote:
> I am using BGP to help stopping DoS and DDoS traffic. BGP can send updates
> quickly and route-maps can translate communities to stop traffic destined to
> a certain destination. TCP is not the issue here. Attackers usually relay on
> the fact that most routers out there do destination based routing without
> checking the validity of the source address in case it is spoofed.
> 
> DT
> 

Is BGP the place to do this?  Note this section from RFC1812:

5.3.8 Source Address Validation
   
   A router SHOULD IMPLEMENT the ability to filter traffic based on a
   comparison of the source address of a packet and the forwarding table
   for a logical interface on which the packet was received.  If this
   filtering is enabled, the router MUST silently discard a packet if
   the interface on which the packet was received is not the interface
   on which a packet would be forwarded to reach the address contained
   in the source address.  In simpler terms, if a router wouldn't route
   a packet containing this address through a particular interface, it
   shouldn't believe the address if it appears as a source address in a
   packet read from this interface.

--
Justin Fletcher
Proficient Networks, Inc.

From: "Turk, Doughan A." <[email-masked]>
To: PamSri <[email-masked]>
Cc: [email-masked]
Subject: RE: Need your opinion of this.
Date: Fri, 26 Jul 2002 21:24:20 -0400
Sender: [email-masked]

I am using BGP to help stopping DoS and DDoS traffic. BGP can send updates
quickly and route-maps can translate communities to stop traffic destined to
a certain destination. TCP is not the issue here. Attackers usually relay on
the fact that most routers out there do destination based routing without
checking the validity of the source address in case it is spoofed.

DT

-----Original Message-----
From: PamSri [mailto:[email-masked]] 
Sent: Friday, July 26, 2002 9:08 PM
To: Turk, Doughan A.
Subject: RE: Need your opinion of this.

Hi Turk ,
         Just started to read through your doc. I have
a question :
BGP is an application from the perspective of TCP. Why
do you see the need for provding such security at the application level than
TCP ? Do you think the security features in TCP as inadequate ? It would be
really a good idea to provide security at the underlying layers than at this
layer. 

sri

--- "Turk, Doughan A." <[email-masked]> wrote:
> Benjamin,
>  
> BGP will be used to alter next-hop address for a
> particular destination
> address (representing a customer under attack) on
> selected Routers. Instead
> of using BGP to announce a different next-hop for
> the destination address
> everywhere in your network. we can use communities
> to make it selective.
> once that is achieved, your next hop can be a null
> interface or a Tunnel
> interface where the traffic is then pulled to the
> sinkhole router.
>  
> Based on Attack signature you could do various
> packet lever filtering ( with
> hardware as a constrain),  simply rate limit the
> flow or send it out through
> a shared media (Ethernet) for sniffing after you
> pull it to a sinkhole
> router.
>  
> As for overheads on routers in the network, there is
> hardly any since you're
> only altering next-hop on border routers or edge
> routers. MPLS tunnel are
> what I used in my lab. as for the sinkhole router,
> one can invest in a high
> performance line card. The sinkhole router should be
> powerful enough to
> handle whatever filtering or policing you deem
> required to massage the flow.
> 
>  
> Scalability is high in my opinion. the route maps on
> the borders can be a
> part of your template. announcing the destination
> address with the special
> community would have to be done on only one router
> (sinkhole is a good
> idea).
>  
> As for where to place the route maps or policy
> statements, IMHO, this should
> be applied anywhere a large attack can enter from.
> (I.e Transit links or to
> Peers with large number of subscribers) I wouldn't
> apply it on a customer
> interface since a customer is in my reach and can be contacted 
> immediately in case of an attack initiated by him. Another place
> to apply it on is at
> peering interfaces where an access-list or policing
> statement might bring
> down interface performance. at this point traffic
> can be pulled in the
> sinkhole router for the policies to be applied.
>  
> This technique can be used in high volume attacks
> where source based
> filtering is not possible because of spoofing or
> high number of source IP
> addresses. In many cases ISP's are forced to kill
> traffic destined to a
> single /32 to relief the rest of the customer
> network from suffering from
> the high volume of traffic. I would also use this in
> cases of attacks
> targeting infrastructure.
>  
> As for the last question, yes I have built this in a
> large lab and got the
> result I expected.
>  
> Hope this answers your questions.
>  
> Regards
> DT
>  
> 
> From: Abarbanel, Benjamin [mailto:[email-masked]]
> Sent: Friday, July 26, 2002 4:55 PM
> To: 'Turk, Doughan A.'; [email-masked]
> Subject: RE: Need your opinion of this.
> 
> 
> Doughan:
>   Let me see if I understand the fundamental point
> of your draft. You want
> BGP via communities to be the big MAGNET that
> catches all the denial of
> service attack routes and shoves them through some
> sink hole router that
> will keep statistics and knowledge of the source of
> the attack and be able
> to report it to the operator?
>  
> Did I summarize it properly?
>  
> My first reaction, is what kind of overhead will
> these techniques impose on
> the routers and the sink hole one? 
>  
> Will it be scallable? 
>  
> Can this be done only on border routers, Router
> Reflectors? To minimize the
> processing load needed?
>  
> What real life cases have you seen where this
> mechanism was needed?
>  
> Have you prototyped this idea and gotten some
> empirical data?
>  
> Ben
> 
> From: Turk, Doughan A. [mailto:[email-masked]]
> Sent: Friday, July 26, 2002 4:09 PM
> To: [email-masked]
> Cc: Turk, Doughan A.
> Subject: Need your opinion of this.
> 
> 
> Folks, 
>  
> I would like to get some discussion started on a
> draft I wrote. It was
> suggested to me to have the draft worked on under
> this group. I appreciate
> any constructive comments and pointers. In summary,
> the draft could be
> classified as an operational one. It is an
> enhancement of a technique that
> already exists but relying more on BGP as a protocol
> to help in network
> security issues.
>  
> A copy of the draft could be found at
>
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
<http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt>
> 
>  
> Regards
> Doughan Turk
>  
>   _____  
> 
> 
>  <http://www.bell.ca/> Bell Canada Logo
> 
> Doughan Turk, M.Eng., CCIE#7274
> Bell Canada, IP Engineering
> Flr 3, 100 Wynford Dr. 
> Toronto M3C 1K4
> 
> Voice: (416) 353-0368
> Fax: (416) 445-9893
> Pager: (416) 685-0590
>  <http://www.bell.ca/> Bell Canada
> 
>  Cisco Certified Internetwork Expert
> <http://www3.sympatico.ca/doughan/ccie_large.gif> 
> 
>          "To Engineer and Integrate world class
>                               IP Services and
> Technologies for BCE" 	
>  
> 
> 

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

From: "Turk, Doughan A." <[email-masked]>
To: [email-masked]
Subject: RE: Need your opinion of this.
Date: Fri, 26 Jul 2002 20:27:31 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C23504.65669C40
Content-Type: text/plain; 
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Benjamin,
 
BGP will be used to alter next-hop address for a particular destination
address (representing a customer under attack) on selected Routers. Instead
of using BGP to announce a different next-hop for the destination address
everywhere in your network. we can use communities to make it selective.
once that is achieved, your next hop can be a null interface or a Tunnel
interface where the traffic is then pulled to the sinkhole router. 
 
Based on Attack signature you could do various packet lever filtering ( with
hardware as a constrain),  simply rate limit the flow or send it out through
a shared media (Ethernet) for sniffing after you pull it to a sinkhole
router.
 
As for overheads on routers in the network, there is hardly any since you're
only altering next-hop on border routers or edge routers. MPLS tunnel are
what I used in my lab. as for the sinkhole router, one can invest in a high
performance line card. The sinkhole router should be powerful enough to
handle whatever filtering or policing you deem required to massage the flow.

Scalability is high in my opinion. the route maps on the borders can be a
part of your template. announcing the destination address with the special
community would have to be done on only one router (sinkhole is a good
idea).
 
As for where to place the route maps or policy statements, IMHO, this should
be applied anywhere a large attack can enter from. (I.e Transit links or to
Peers with large number of subscribers) I wouldn't apply it on a customer
interface since a customer is in my reach and can be contacted immediately
in case of an attack initiated by him. Another place to apply it on is at
peering interfaces where an access-list or policing statement might bring
down interface performance. at this point traffic can be pulled in the
sinkhole router for the policies to be applied.
 
This technique can be used in high volume attacks where source based
filtering is not possible because of spoofing or high number of source IP
addresses. In many cases ISP's are forced to kill traffic destined to a
single /32 to relief the rest of the customer network from suffering from
the high volume of traffic. I would also use this in cases of attacks
targeting infrastructure.
 
As for the last question, yes I have built this in a large lab and got the
result I expected.
 
Hope this answers your questions.
 
Regards
DT

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]] 
Sent: Friday, July 26, 2002 4:55 PM
To: 'Turk, Doughan A.'; [email-masked]
Subject: RE: Need your opinion of this.

Doughan:
  Let me see if I understand the fundamental point of your draft. You want
BGP via communities to be the big MAGNET that catches all the denial of
service attack routes and shoves them through some sink hole router that
will keep statistics and knowledge of the source of the attack and be able
to report it to the operator?
 
Did I summarize it properly?
 
My first reaction, is what kind of overhead will these techniques impose on
the routers and the sink hole one? 
 
Will it be scallable? 
 
Can this be done only on border routers, Router Reflectors? To minimize the
processing load needed?
 
What real life cases have you seen where this mechanism was needed?
 
Have you prototyped this idea and gotten some empirical data?
 
Ben

-----Original Message-----
From: Turk, Doughan A. [mailto:[email-masked]]
Sent: Friday, July 26, 2002 4:09 PM
To: [email-masked]
Cc: Turk, Doughan A.
Subject: Need your opinion of this.

Folks, 
 
I would like to get some discussion started on a draft I wrote. It was
suggested to me to have the draft worked on under this group. I appreciate
any constructive comments and pointers. In summary, the draft could be
classified as an operational one. It is an enhancement of a technique that
already exists but relying more on BGP as a protocol to help in network
security issues.
 
A copy of the draft could be found at
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
<http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt> 
 
Regards
Doughan Turk
 
  _____  

 <http://www.bell.ca/> Bell Canada Logo

Doughan Turk, M.Eng., CCIE#7274
Bell Canada, IP Engineering
Flr 3, 100 Wynford Dr. 
Toronto M3C 1K4

Voice: (416) 353-0368
Fax: (416) 445-9893
Pager: (416) 685-0590
 <http://www.bell.ca/> Bell Canada

 Cisco Certified Internetwork Expert
<http://www3.sympatico.ca/doughan/ccie_large.gif> 

         "To Engineer and Integrate world class
                              IP Services and Technologies for BCE" 	

------_=_NextPart_001_01C23504.65669C40
Content-Type: text/html; 
 charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<TITLE>Message</TITLE>

<META content="MSHTML 6.00.2716.2200" name=GENERATOR></HEAD>
<BODY>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2>Benjamin,</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>BGP 
will be used to alter next-hop address for a particular destination address 
(representing a customer under attack)&nbsp;on selected Routers. Instead of 
using BGP to announce a different next-hop for the destination address 
everywhere in your network. we can use communities to make it selective. once 
that is achieved, your next hop can be a null interface or a Tunnel interface 
where the traffic is then pulled to the sinkhole router. </FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>Based 
on Attack signature you could do various packet lever filtering ( with hardware 
as a constrain),&nbsp; simply rate limit the flow or send it out through a 
shared media (Ethernet) for sniffing after you pull it to a sinkhole 
router.</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>As for 
overheads on routers in the network, there is hardly any since you're only 
altering next-hop on border routers or edge routers. MPLS tunnel are what I used 
in my lab. as for the sinkhole router, one can invest in a high performance line 
card. The sinkhole router should be powerful enough to handle whatever filtering 
or policing you deem required to massage the flow. </FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2>Scalability is high in my opinion. the route maps on the borders can be a 
part of your template. announcing the destination address with the special 
community would have to be done on only one router (sinkhole is a good 
idea).</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>As for 
where to place the route maps or policy statements, IMHO, this should be applied 
anywhere a large attack can enter from. (I.e Transit links or to Peers with 
large number of subscribers) I wouldn't apply it on a customer interface since a 
customer is in my reach and can be contacted immediately in case of an attack 
initiated by him. Another place to apply it on is at peering interfaces where an 
access-list or policing statement might bring down interface performance. at 
this point traffic can be pulled in the sinkhole router for the policies to be 
applied.</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>This 
technique can be used in high volume attacks where source based filtering is not 
possible because of spoofing or high number of source IP addresses. In many 
cases ISP's are forced to kill traffic destined to a single /32 to relief the 
rest of the customer network from suffering from the high volume of traffic. I 
would also use this in cases of attacks targeting 
infrastructure.</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>As for 
the last question, yes I have built this in a large lab and got the result I 
expected.</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff size=2>Hope 
this answers your questions.</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2></FONT></SPAN>&nbsp;</DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2>Regards</FONT></SPAN></DIV>
<DIV><SPAN class=957321621-26072002><FONT face=Arial color=#0000ff 
size=2>DT</FONT></SPAN></DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT>&nbsp;</DIV>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
  <DIV></DIV>
  <DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left><FONT 
  face=Tahoma size=2>-----Original Message-----<BR><B>From:</B> Abarbanel, 
  Benjamin [mailto:[email-masked]] <BR><B>Sent:</B> Friday, July 
  26, 2002 4:55 PM<BR><B>To:</B> 'Turk, Doughan A.'; 
  [email-masked]<BR><B>Subject:</B> RE: Need your opinion of 
  this.<BR><BR></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002>Doughan:</SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002>&nbsp; Let me see if I understand the fundamental 
  point of your draft. You want BGP via communities to be the big MAGNET that 
  catches all the denial of service attack routes and shoves them through some 
  sink hole router that will keep statistics and knowledge of the source of the 
  attack and be able to report it to the operator?</SPAN></FONT></DIV>
  <DIV>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN class=283324920-26072002>Did 
  I summarize it properly?</SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN class=283324920-26072002>My 
  first reaction, is what kind of overhead will these techniques impose on the 
  routers and the sink hole one? </SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN class=283324920-26072002>Will 
  it be scallable? </SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN class=283324920-26072002>Can 
  this be done only on border routers, Router Reflectors? To minimize the 
  processing load needed?</SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN class=283324920-26072002>What 
  real life cases have you seen where this mechanism was 
  needed?</SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN class=283324920-26072002>Have 
  you prototyped this idea and gotten some empirical data?</SPAN></FONT></DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial color=#0000ff size=2><SPAN 
  class=283324920-26072002>Ben</SPAN></FONT></DIV>
  <BLOCKQUOTE 
  style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
    <DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma 
    size=2>-----Original Message-----<BR><B>From:</B> Turk, Doughan A. 
    [mailto:[email-masked]]<BR><B>Sent:</B> Friday, July 26, 2002 4:09 
    PM<BR><B>To:</B> [email-masked]<BR><B>Cc:</B> Turk, Doughan 
    A.<BR><B>Subject:</B> Need your opinion of this.<BR><BR></DIV></FONT>
    <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>Folks, 
    </SPAN></FONT></DIV>
    <DIV><FONT face=Arial size=2><SPAN 
    class=247400120-26072002></SPAN></FONT>&nbsp;</DIV>
    <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>I would like to 
    get some discussion started on a draft I wrote. It was suggested to me to 
    have the draft worked on under this group. I appreciate any constructive 
    comments and pointers. In summary, the draft could be classified as an 
    operational one. It is an enhancement of a technique that already exists but 
    relying more on BGP as a protocol to help in network security 
    issues.</SPAN></FONT></DIV>
    <DIV><FONT face=Arial size=2><SPAN 
    class=247400120-26072002></SPAN></FONT>&nbsp;</DIV>
    <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>A copy of the 
    draft could be found at <A 
    href="http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt">http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt</A></SPAN></FONT></DIV>
    <DIV><FONT face=Arial size=2><SPAN 
    class=247400120-26072002></SPAN></FONT>&nbsp;</DIV>
    <DIV><FONT face=Arial size=2><SPAN 
    class=247400120-26072002>Regards</SPAN></FONT></DIV>
    <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>Doughan 
    Turk</SPAN></FONT></DIV>
    <DIV>&nbsp;</DIV>
    <DIV class=Section1>
    <DIV class=MsoNormal style="TEXT-ALIGN: center" align=center>
    <HR align=center width="100%" color=#cccccc noShade SIZE=1>
    </DIV>
    <TABLE class=MsoNormalTable 
    style="BORDER-COLLAPSE: collapse; mso-padding-alt: 0in 5.4pt 0in 5.4pt" 
    cellSpacing=0 cellPadding=0 border=0>
      <TBODY>
      <TR style="mso-yfti-irow: 0">
        <TD 
        style="PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: 1.5pt; WIDTH: 93.75pt; PADDING-TOP: 1.5pt" 
        vAlign=top width=125>
          <P class=MsoNormal><SPAN 
          style="FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: Arial"><A 
          href="http://www.bell.ca/"><SPAN 
          style="TEXT-DECORATION: none; text-underline: none"><IMG 
          id=_x0000_i1026 height=46 alt="Bell Canada Logo" 
          src="http://www3.sympatico.ca/doughan/small_bell_logo.gif" width=102 
          border=0 NOSEND="1"></SPAN></A></SPAN></P></TD>
        <TD 
        style="PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: 1.5pt; WIDTH: 135pt; PADDING-TOP: 1.5pt" 
        vAlign=top width=180>
          <P class=MsoNormal><SPAN 
          style="FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: Arial">Doughan 
          Turk, M.Eng., CCIE#7274<BR>Bell Canada, IP Engineering<BR>Flr 3, 100 
          Wynford Dr. <BR>Toronto M3C 1K4</SPAN></P></TD>
        <TD 
        style="PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: 1.5pt; PADDING-TOP: 1.5pt" 
        vAlign=top noWrap>
          <P class=MsoNormal><SPAN 
          style="FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: Arial">Voice: 
          (416) 353-0368<BR>Fax: (416) 445-9893<BR>Pager: (416) 685-0590<BR><A 
          href="http://www.bell.ca/"><FONT color=#ffb900><SPAN 
          style="COLOR: #ffb900">Bell 
        Canada</SPAN></FONT></A></SPAN></FONT></P></TD>
        <TD 
        style="PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: 1.5pt; WIDTH: 52.15pt; PADDING-TOP: 1.5pt" 
        vAlign=top width=70>
          <P class=MsoNormal style="TEXT-ALIGN: right" align=right><SPAN 
          style="FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: Arial"><IMG 
          id=_x0000_i1027 height=60 alt="Cisco Certified Internetwork Expert" 
          src="http://www3.sympatico.ca/doughan/ccie_large.gif" width=61 
          border=0 NOSEND="1"></SPAN></P></TD></TR>
      <TR vAlign=top>
        <TD colSpan=3><B><I><FONT face=Georgia color=#55549a 
          size=1>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;</FONT><FONT 
          style="FONT-SIZE: 8.5pt" face=Georgia color=#55549a>"To Engineer and 
          Integrate world 
          class<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
          &nbsp;&nbsp;&nbsp;IP Services and Technologies for BCE" 
        </FONT></I></B></TD></TR></TBODY></TABLE></DIV>
    <DIV>&nbsp;</DIV></BLOCKQUOTE></BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C23504.65669C40--

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Turk, Doughan A.'" <[email-masked]>, [email-masked]
Subject: RE: Need your opinion of this.
Date: Fri, 26 Jul 2002 16:54:56 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C234E6.B2B1E2C0
Content-Type: text/plain;
	charset="iso-8859-1"

Doughan:
  Let me see if I understand the fundamental point of your draft. You want
BGP via communities to be the big MAGNET that catches all the denial of
service attack routes and shoves them through some sink hole router that
will keep statistics and knowledge of the source of the attack and be able
to report it to the operator?
 
Did I summarize it properly?
 
My first reaction, is what kind of overhead will these techniques impose on
the routers and the sink hole one? 
 
Will it be scallable? 
 
Can this be done only on border routers, Router Reflectors? To minimize the
processing load needed?
 
What real life cases have you seen where this mechanism was needed?
 
Have you prototyped this idea and gotten some empirical data?
 
Ben

-----Original Message-----
From: Turk, Doughan A. [mailto:[email-masked]]
Sent: Friday, July 26, 2002 4:09 PM
To: [email-masked]
Cc: Turk, Doughan A.
Subject: Need your opinion of this.

Folks, 
 
I would like to get some discussion started on a draft I wrote. It was
suggested to me to have the draft worked on under this group. I appreciate
any constructive comments and pointers. In summary, the draft could be
classified as an operational one. It is an enhancement of a technique that
already exists but relying more on BGP as a protocol to help in network
security issues.
 
A copy of the draft could be found at
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
<http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt> 
 
Regards
Doughan Turk
 
   _____  

 <http://www.bell.ca/> Bell Canada Logo

Doughan Turk, M.Eng., CCIE#7274
Bell Canada, IP Engineering
Flr 3, 100 Wynford Dr. 
Toronto M3C 1K4

Voice: (416) 353-0368
Fax: (416) 445-9893
Pager: (416) 685-0590
 <http://www.bell.ca/> Bell Canada

 Cisco Certified Internetwork Expert
<http://www3.sympatico.ca/doughan/ccie_large.gif> 

         "To Engineer and Integrate world class
                              IP Services and Technologies for BCE" 	

------_=_NextPart_001_01C234E6.B2B1E2C0
Content-Type: text/html;
	charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<TITLE>Message</TITLE>

<META content="MSHTML 5.00.2920.0" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002>Doughan:</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>&nbsp; 
Let me see if I understand the fundamental point of your draft. You want BGP via 
communities to be the big MAGNET that catches all the denial of service attack 
routes and shoves them through some sink hole router that will keep statistics 
and knowledge of the source of the attack and be able to report it to the 
operator?</SPAN></FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>Did I 
summarize it properly?</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>My 
first reaction, is what kind of overhead will these techniques impose on the 
routers and the sink hole one? </SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>Will 
it be scallable? </SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>Can 
this be done only on border routers, Router Reflectors? To minimize the 
processing load needed?</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>What 
real life cases have you seen where this mechanism was 
needed?</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=283324920-26072002>Have 
you prototyped this idea and gotten some empirical data?</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=283324920-26072002>Ben</SPAN></FONT></DIV>
<BLOCKQUOTE 
style="BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px; PADDING-LEFT: 5px">
  <DIV align=left class=OutlookMessageHeader dir=ltr><FONT face=Tahoma 
  size=2>-----Original Message-----<BR><B>From:</B> Turk, Doughan A. 
  [mailto:[email-masked]]<BR><B>Sent:</B> Friday, July 26, 2002 4:09 
  PM<BR><B>To:</B> [email-masked]<BR><B>Cc:</B> Turk, Doughan 
  A.<BR><B>Subject:</B> Need your opinion of this.<BR><BR></DIV></FONT>
  <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>Folks, 
  </SPAN></FONT></DIV>
  <DIV><FONT face=Arial size=2><SPAN 
  class=247400120-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>I would like to 
  get some discussion started on a draft I wrote. It was suggested to me to have 
  the draft worked on under this group. I appreciate any constructive comments 
  and pointers. In summary, the draft could be classified as an operational one. 
  It is an enhancement of a technique that already exists but relying more on 
  BGP as a protocol to help in network security issues.</SPAN></FONT></DIV>
  <DIV><FONT face=Arial size=2><SPAN 
  class=247400120-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>A copy of the 
  draft could be found at <A 
  href="http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt">http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt</A></SPAN></FONT></DIV>
  <DIV><FONT face=Arial size=2><SPAN 
  class=247400120-26072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face=Arial size=2><SPAN 
  class=247400120-26072002>Regards</SPAN></FONT></DIV>
  <DIV><FONT face=Arial size=2><SPAN class=247400120-26072002>Doughan 
  Turk</SPAN></FONT></DIV>
  <DIV>&nbsp;</DIV>
  <DIV class=Section1>
  <DIV align=center class=MsoNormal style="TEXT-ALIGN: center">
  <HR align=center color=#cccccc noShade SIZE=1 width="100%">
  </DIV>
  <TABLE border=0 cellPadding=0 cellSpacing=0 class=MsoNormalTable 
  style="BORDER-COLLAPSE: collapse; mso-padding-alt: 0in 5.4pt 0in 5.4pt">
    <TBODY>
    <TR style="mso-yfti-irow: 0">
      <TD 
      style="PADDING-BOTTOM: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-RIGHT: 1.5pt; PADDING-TOP: 1.5pt; WIDTH: 93.75pt" 
      vAlign=top width=125>
        <P class=MsoNormal><SPAN 
        style="COLOR: #55549a; FONT-FAMILY: Arial; FONT-SIZE: 8.5pt"><A 
        href="http://www.bell.ca/"><SPAN 
        style="TEXT-DECORATION: none; text-underline: none"><IMG 
        alt="Bell Canada Logo" border=0 height=46 id=_x0000_i1026 
        src="http://www3.sympatico.ca/doughan/small_bell_logo.gif" width=102 
        NOSEND="1"></SPAN></A></SPAN></P></TD>
      <TD 
      style="PADDING-BOTTOM: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-RIGHT: 1.5pt; PADDING-TOP: 1.5pt; WIDTH: 135pt" 
      vAlign=top width=180>
        <P class=MsoNormal><SPAN 
        style="COLOR: #55549a; FONT-FAMILY: Arial; FONT-SIZE: 8.5pt">Doughan 
        Turk, M.Eng., CCIE#7274<BR>Bell Canada, IP Engineering<BR>Flr 3, 100 
        Wynford Dr. <BR>Toronto M3C 1K4</SPAN></P></TD>
      <TD noWrap 
      style="PADDING-BOTTOM: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-RIGHT: 1.5pt; PADDING-TOP: 1.5pt" 
      vAlign=top>
        <P class=MsoNormal><SPAN 
        style="COLOR: #55549a; FONT-FAMILY: Arial; FONT-SIZE: 8.5pt">Voice: 
        (416) 353-0368<BR>Fax: (416) 445-9893<BR>Pager: (416) 685-0590<BR><A 
        href="http://www.bell.ca/"><FONT color=#ffb900><SPAN 
        style="COLOR: #ffb900">Bell 
Canada</SPAN></FONT></A></SPAN></FONT></P></TD>
      <TD 
      style="PADDING-BOTTOM: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-RIGHT: 1.5pt; PADDING-TOP: 1.5pt; WIDTH: 52.15pt" 
      vAlign=top width=70>
        <P align=right class=MsoNormal style="TEXT-ALIGN: right"><SPAN 
        style="COLOR: #55549a; FONT-FAMILY: Arial; FONT-SIZE: 8.5pt"><IMG 
        alt="Cisco Certified Internetwork Expert" border=0 height=60 
        id=_x0000_i1027 src="http://www3.sympatico.ca/doughan/ccie_large.gif" 
        width=61 NOSEND="1"></SPAN></P></TD></TR>
    <TR vAlign=top>
      <TD colSpan=3><B><I><FONT color=#55549a face=Georgia 
        size=1>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;</FONT><FONT 
        color=#55549a face=Georgia style="FONT-SIZE: 8.5pt">"To Engineer and 
        Integrate world 
        class<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 
        &nbsp;&nbsp;&nbsp;IP Services and Technologies for BCE" 
      </FONT></I></B></TD></TR></TBODY></TABLE></DIV>
  <DIV>&nbsp;</DIV></BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C234E6.B2B1E2C0--

From: "Turk, Doughan A." <[email-masked]>
To: [email-masked]
Cc: "Turk, Doughan A." <[email-masked]>
Subject: Need your opinion of this.
Date: Fri, 26 Jul 2002 16:09:19 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C234E0.53791CC0
Content-Type: text/plain; 
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Folks, 
 
I would like to get some discussion started on a draft I wrote. It was
suggested to me to have the draft worked on under this group. I appreciate
any constructive comments and pointers. In summary, the draft could be
classified as an operational one. It is an enhancement of a technique that
already exists but relying more on BGP as a protocol to help in network
security issues.
 
A copy of the draft could be found at
http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt
<http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt> 
 
Regards
Doughan Turk
 
  _____  

 <http://www.bell.ca/> Bell Canada Logo

Doughan Turk, M.Eng., CCIE#7274
Bell Canada, IP Engineering
Flr 3, 100 Wynford Dr. 
Toronto M3C 1K4

Voice: (416) 353-0368
Fax: (416) 445-9893
Pager: (416) 685-0590
 <http://www.bell.ca/> Bell Canada

 Cisco Certified Internetwork Expert
<http://www3.sympatico.ca/doughan/ccie_large.gif> 

         "To Engineer and Integrate world class
                              IP Services and Technologies for BCE" 	

------_=_NextPart_001_01C234E0.53791CC0
Content-Type: text/html; 
 charset=us-ascii
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">
<TITLE>Message</TITLE>

<META content=3D"MSHTML 6.00.2716.2200" name=3DGENERATOR></HEAD>
<BODY>
<DIV><FONT face=3DArial size=3D2><SPAN =
class=3D247400120-26072002>Folks,=20
</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D247400120-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D247400120-26072002>I =
would like to get=20
some discussion started on a draft I wrote. It was suggested to me to =
have the=20
draft worked on under this group. I appreciate any constructive =
comments and=20
pointers. In summary, the draft could be classified as an operational =
one. It is=20
an enhancement of a technique that already exists but relying more on =
BGP as a=20
protocol to help in network security issues.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D247400120-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D247400120-26072002>A =
copy of the draft=20
could be found at <A=20
href=3D"http://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt">http=
://www.ietf.org/internet-drafts/draft-turk-ertb-00.txt</A></SPAN></FONT>=
</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D247400120-26072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D247400120-26072002>Regards</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN =
class=3D247400120-26072002>Doughan=20
Turk</SPAN></FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV class=3DSection1>
<DIV class=3DMsoNormal style=3D"TEXT-ALIGN: center" align=3Dcenter>
<HR align=3Dcenter width=3D"100%" color=3D#cccccc noShade SIZE=3D1>
</DIV>
<TABLE class=3DMsoNormalTable=20
style=3D"BORDER-COLLAPSE: collapse; mso-padding-alt: 0in 5.4pt 0in =
5.4pt"=20
cellSpacing=3D0 cellPadding=3D0 border=3D0>
  <TBODY>
  <TR style=3D"mso-yfti-irow: 0">
    <TD=20
    style=3D"PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: =
1.5pt; WIDTH: 93.75pt; PADDING-TOP: 1.5pt"=20
    vAlign=3Dtop width=3D125>
      <P class=3DMsoNormal><SPAN=20
      style=3D"FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: Arial"><A =

      href=3D"http://www.bell.ca/"><SPAN=20
      style=3D"TEXT-DECORATION: none; text-underline: none"><IMG =
id=3D_x0000_i1026=20
      height=3D46 alt=3D"Bell Canada Logo"=20
      src=3D"http://www3.sympatico.ca/doughan/small_bell_logo.gif" =
width=3D102=20
      border=3D0></SPAN></A></SPAN></P></TD>
    <TD=20
    style=3D"PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: =
1.5pt; WIDTH: 135pt; PADDING-TOP: 1.5pt"=20
    vAlign=3Dtop width=3D180>
      <P class=3DMsoNormal><SPAN=20
      style=3D"FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: =
Arial">Doughan Turk,=20
      M.Eng., CCIE#7274<BR>Bell Canada, IP Engineering<BR>Flr 3, 100 =
Wynford Dr.=20
      <BR>Toronto M3C 1K4</SPAN></P></TD>
    <TD=20
    style=3D"PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: =
1.5pt; PADDING-TOP: 1.5pt"=20
    vAlign=3Dtop noWrap>
      <P class=3DMsoNormal><SPAN=20
      style=3D"FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: =
Arial">Voice: (416)=20
      353-0368<BR>Fax: (416) 445-9893<BR>Pager: (416) 685-0590<BR><A=20
      href=3D"http://www.bell.ca/"><FONT color=3D#ffb900><SPAN=20
      style=3D"COLOR: #ffb900">Bell =
Canada</SPAN></FONT></A></SPAN></FONT></P></TD>
    <TD=20
    style=3D"PADDING-RIGHT: 1.5pt; PADDING-LEFT: 1.5pt; PADDING-BOTTOM: =
1.5pt; WIDTH: 52.15pt; PADDING-TOP: 1.5pt"=20
    vAlign=3Dtop width=3D70>
      <P class=3DMsoNormal style=3D"TEXT-ALIGN: right" =
align=3Dright><SPAN=20
      style=3D"FONT-SIZE: 8.5pt; COLOR: #55549a; FONT-FAMILY: =
Arial"><IMG=20
      id=3D_x0000_i1027 height=3D60 alt=3D"Cisco Certified Internetwork =
Expert"=20
      src=3D"http://www3.sympatico.ca/doughan/ccie_large.gif" =
width=3D61=20
      border=3D0></SPAN></P></TD></TR>
  <TR vAlign=3Dtop>
    <TD colSpan=3D3><B><I><FONT face=3DGeorgia color=3D#55549a=20
      size=3D1>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; =
&nbsp;&nbsp;&nbsp;</FONT><FONT=20
      style=3D"FONT-SIZE: 8.5pt" face=3DGeorgia color=3D#55549a>"To =
Engineer and=20
      Integrate world=20
      =
class<BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;=20
      &nbsp;&nbsp;&nbsp;IP Services and Technologies for BCE"=20
  </FONT></I></B></TD></TR></TBODY></TABLE></DIV>
<DIV>&nbsp;</DIV></BODY></HTML>

------_=_NextPart_001_01C234E0.53791CC0--

Date: Fri, 26 Jul 2002 14:00:14 -0400
From: Jeffrey Haas <[email-masked]>
To: "Natale, Jonathan" <[email-masked]>
Cc: "'Sprzaczkowski, Krzysztof'" <[email-masked]>, [email-masked]
Subject: Re: "third party" next-hop
Sender: [email-masked]

On Fri, Jul 26, 2002 at 12:49:04PM -0400, Natale, Jonathan wrote:
> Of course you could have another shared subnet that you are not peering
> across--I don't think this case was ever considered/implemented.

This is fine so long as you don't receive a third-party nexthop
for an subnets that you both have configured, but aren't actually
the same subnet.

(Think separate rfc 1918 subnets that are similarly numbered.)

If you are concerned about receiving third party announcements,
force the nexthop to be first party to your peering session.  That
will result in sub-optimal forwarding, but it should still work.

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 26 Jul 2002 13:58:03 -0400
From: Jeffrey Haas <[email-masked]>
To: "Sprzaczkowski, Krzysztof" <[email-masked]>
Cc: [email-masked]
Subject: Re: "third party" next-hop
Sender: [email-masked]

On Fri, Jul 26, 2002 at 06:31:21PM +0200, Sprzaczkowski, Krzysztof wrote:
> According to this, should we compare next-hop
> address subnet with peer X address subnet (not with interface's address
> subnet where a given peer is established)?

One could know by configuration what the subnet of the remote side is
for interface/nexthop comparisons.

> Krzysztof Sprzaczkowski

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 26 Jul 2002 10:40:01 -0700
From: Chandrashekhar Appanna <[email-masked]>
To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked]
Subject: Re: inform (was: IDR WG minutes IETF 54)
Sender: [email-masked]

In addition I think it needs to be discussed how one can
mandate that actions should not be automated/taken on rcv'ing
an INFORM - I think it cannot be mandated. And a compliance 
check will also not be possible since not all implementations 
are open source :)

-
Chandra A.

On Thu, Jul 18, 2002 at 06:47:36PM -0700, Pedro Roque Marques wrote:
> David Ward writes:
> 
> > INFORM message I. Gargi Nalawade presenter draft-nalawade-bgp-inform-00.txt 1.
> > Goals a. no way to tell remote
> > peer that there has been an error or event w/o use of NOTIFICATION
> > i. this resets peering session b. no change to existing
> > implementations c. limit sharing of messages between peers
> > 2. Encoding overview a. Packet format b. Event codes c. Data types
> > 3. Rx'ing the INFORM a. log it b. inform the administrator
> > (implementation specific) Question: How to use w/ NOTIFICATION?
> > Answer: Use it to inform that you are going to reset. Use
> > immediately before the NOTIFICATION but, now can add
> > PDU/ATTR/whatever to help debug.
> 
> > Question: Will this cause a change to the FSM?  Answser: No
> 
> > Questons: What is "too many routes option?" How to define?  Answer:
> > It is an implementation specific configuration
> 
> > Question: Jeff Pickering Should a peer take any specific action upon
> > reception of the INFORM?  The authors should explicitly prohibit any
> > automated action
> 
> > Statement: Andrew Lange Move forward draft
> 
> > Outcome: Sue Hares Accepted as WG item pending taking it to list
> 
> I object to making this a WG document.
> 
> I believe this mechanism is unnecessary and that no sufficient
> justification has been presented to add this to the spec. There should
> be a very compeling reason to be making changes so basic such as
> adding new messages.
> 
> A couple of comments on the details above:
> 
> > Question: How to use w/ NOTIFICATION?
> > Answer: Use it to inform that you are going to reset. Use
> > immediately before the NOTIFICATION but, now can add
> > PDU/ATTR/whatever to help debug.
> 
> How can that be of extra value given that notification allows you to
> carry variable lenght data ? Adding this message is just going to
> cause a debugging mess with all the deployed systems out there that do
> not implement it.
> 
> > Question: Jeff Pickering Should a peer take any specific action upon
> > reception of the INFORM?
> 
> Yes, reset the session, at least in the case that has been vastly
> discussed in the list which is receicept of invalid nlri. Furtunatly
> no code needs to be written to achieve that since it is the default
> action on a receipt of an invalid message code.
> 
>   Pedro.

From: "Natale, Jonathan" <[email-masked]>
To: "'Sprzaczkowski, Krzysztof'" <[email-masked]>, [email-masked]
Subject: RE: "third party" next-hop 
Date: Fri, 26 Jul 2002 12:49:04 -0400
Sender: [email-masked]

I think it boils down to:
   IF (nexthop is on subnet of interface used to reach peer) OR (ebgp
multi-hop enabled) OR (IBGP) THEN
    Accept it
ELSE
    Toss it

Older IOS (less than 12.0.?) did accept any next-hop.

Of course you could have another shared subnet that you are not peering
across--I don't think this case was ever considered/implemented.  So you
have a point (there are many things like this in the RFC).  I don't think
implementing this to the letter would be practical (and it would be
difficult to write, if possible at all).

-$0.02

-----Original Message-----
From: Sprzaczkowski, Krzysztof [mailto:[email-masked]] 
Sent: Friday, July 26, 2002 12:31 PM
To: [email-masked]
Subject: "third party" next-hop 

I have a question about 'third party' next-hop in the context of IPv6 routes
sending over IPv4 session. Does 'third party' next-hop make sense in such
situation? Both documents draft-ietf-idr-bgp4-17 (chapter 5.1.3 NEXT_HOP)
and RFC 2858 (chapter 2 Multiprotocol Reachable NLRI - MP_REACH_NLRI) say
that for 'third party' next-hop peer X should share a common subnet with
received next-hop address. According to this, should we compare next-hop
address subnet with peer X address subnet (not with interface's address
subnet where a given peer is established)?

Krzysztof Sprzaczkowski

From: "Sprzaczkowski, Krzysztof" <[email-masked]>
To: [email-masked]
Subject: "third party" next-hop 
Date: Fri, 26 Jul 2002 18:31:21 +0200
Sender: [email-masked]

I have a question about 'third party' next-hop in the context of IPv6 routes
sending over IPv4 session. Does 'third party' next-hop make sense in such
situation? Both documents draft-ietf-idr-bgp4-17 (chapter 5.1.3 NEXT_HOP)
and RFC 2858 (chapter 2 Multiprotocol Reachable NLRI - MP_REACH_NLRI) say
that for 'third party' next-hop peer X should share a common subnet with
received next-hop address. According to this, should we compare next-hop
address subnet with peer X address subnet (not with interface's address
subnet where a given peer is established)?

Krzysztof Sprzaczkowski

Date: Fri, 26 Jul 2002 13:40:40 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: CEASE problems
To: "Pusz, MateuszX" <[email-masked]>, [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Hi Pusz,
If a BGP speaker decides to administratively reset the peering with a
neighbor, then the speaker should send a NOTIFICATION message with the
Error Code Cease, and the Error Subcode "Administratively Reset".

Regards,
Manav
----- Original Message -----
From: "Pusz, MateuszX" <[email-masked]>
To: <[email-masked]>
Sent: Friday, July 26, 2002 12:37 PM
Subject: CEASE problems

| Hello
|
| I have one problem with CEASE message. 'draft-ietf-idr-bgp4-17' says in
6.7:
|    In absence of any fatal errors (that are indicated in this section),
|    a BGP peer may choose at any given time to close its BGP connection
|    by sending the NOTIFICATION message with Error Code Cease. However,
|    the Cease NOTIFICATION message must not be used when a fatal error
|    indicated by this section does exist.
|
| but in section 8-Established state:
|    In response to a stop event initiated by an operator:
|       - release all resources (including deleting all routes),
|       - set ConnectRetryCnt to zero (0),
|       - set connect retry timer to zero (0), and
|       - transition to the Idle.
|
| so how shuold it be? Should BGP router in such a situation send that
| notification message or not?
|
| Please help me
|
| Thoughts from IDR community welcome.
|
| Best regards
|
|
| Mateusz Pusz

Date: Fri, 26 Jul 2002 12:42:58 +0530
From: Manav Bhatia <[email-masked]>
Subject: draft-ietf-idr-bgp4-18.txt
To: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

sec 6.5 of the latest draft states - " If  system does not receive
successive KEEPALIVE and/or UPDATE and/or NOTIFICATION messages within the
period specified in the Hold Time field of the OPEN message, then the
NOTIFICATION message with   Hold Timer Expired Error Code must be sent and
the BGP connection closed"

IMHO we must update the draft to also include other BGP messages (Route
Refresh, Dynamic Capability, INFORM) which when received by the local
system act as surrogate for the KEEPALIVE messages it would expect from the
remote end. The local system upon receiving such messages must restart its
Hold Timer

Similarly the draft should mention that each time the local system sends
such BGP messages it should restart its KeepAlive Timer, unless the
negotiated HoldTime value is zero.

These changes can be reflected in the BGP draft or can be done in the
drafts introducing such BGP messages.

Regards,
Manav

----
"When you are courting a nice girl an hour seems like a second. When you
sit on a red-hot cinder a second seems like an hour. That's relativity."

-Albert Einstein, on relativity

From: "Pusz, MateuszX" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: CEASE problems
Date: Fri, 26 Jul 2002 09:07:42 +0200
Sender: [email-masked]

Hello

I have one problem with CEASE message. 'draft-ietf-idr-bgp4-17' says in 6.7:
   In absence of any fatal errors (that are indicated in this section),
   a BGP peer may choose at any given time to close its BGP connection
   by sending the NOTIFICATION message with Error Code Cease. However,
   the Cease NOTIFICATION message must not be used when a fatal error
   indicated by this section does exist.

but in section 8-Established state:
   In response to a stop event initiated by an operator:
      - release all resources (including deleting all routes),
      - set ConnectRetryCnt to zero (0),
      - set connect retry timer to zero (0), and
      - transition to the Idle.

so how shuold it be? Should BGP router in such a situation send that
notification message or not?

Please help me

Thoughts from IDR community welcome.

Best regards

Mateusz Pusz

Date: Thu, 25 Jul 2002 12:53:56 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked], [email-masked], [email-masked]
Cc: [email-masked]
Subject: Thought on non-transitive extended communities
Sender: [email-masked]

It occurs to me that, for non-transitive extended communities,
we may want to discard them on INGRESS if the extended community
path attribute is marked with the PARTIAL bit.  This can mean
that we have received something that should never had made it to
us in the first place.

Admittedly, this also would hit the case where a given border
router in your AS doesn't understand it, marks it as partial
and sends to an internal peer that does understand it.

-- 
Jeff Haas 
NextHop Technologies

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Susan Hares'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: Kunihiro Ishiguro <[email-masked]>, BGP mailing list <[email-masked]>
Subject: RE: TCP state machine doubts
Date: Thu, 25 Jul 2002 11:19:11 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C233EE.A0D609D0
Content-Type: text/plain;
	charset="iso-8859-1"

Sue: 

   General Comments:

   2.      Do you have any assumptions on Multi-thread processing impacts to
the FSM?

If not, can you mention any operational assumptions?

The multi-thread processing issues would focus around the choice between 1
or 2 FSMs.   The state machine's event 22 and the call collision allows you
to do both.

No assumptions on the multi-threaded processing impacts to the FSM were
made. 

A two state versus 1 state model was allowed.  The single state model is
indicated in the places were TCP and con

Event 22 is merely a an open collision case. I think there is more to it
than just that. There are various phases that BGP goes through that one
might want to use multiple threads. The inter-locking or inter-working
between the threads could affect the state transitions. For example, Say you
are in Establish state and something happens to the session and it is
disconnected. In a single threaded environment it is easy, first come, first
served. In a multi-threaded case with thread priorities and interruptions,
it might get tricky at times. Some assumptions or expectations by the state
machine on actions it should take with regards to semaphore locks (RIB
Table, etc.), thread prioritization, processing, pre-emption and so forth
might be helpfull. Maybe these actions should only be suggestions not
required ways for implementations.

    I hope I have not opened a can of worms here?

Regards,
Ben

------_=_NextPart_001_01C233EE.A0D609D0
Content-Type: text/html;
	charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<META content="MSHTML 5.00.2920.0" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=475025414-25072002>Sue:</SPAN></FONT><FONT color=#0000ff face=Arial 
size=2><SPAN class=475025414-25072002> </SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=475025414-25072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=475025414-25072002>&nbsp;</SPAN></FONT></DIV>
<BLOCKQUOTE 
style="BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px; PADDING-LEFT: 5px">
  <DIV align=left class=OutlookMessageHeader dir=ltr><FONT color=#0000ff 
  face=Arial size=2><SPAN class=475025414-25072002>&nbsp; 
  &nbsp;</SPAN></FONT><FONT face="Courier New, Courier">General 
  Comments:<BR>&nbsp;</FONT> </DIV>
  <DL><FONT size=4><FONT color=#0000ff><FONT face=Arial>
    <DD></FONT><FONT size=2><SPAN class=475025414-25072002>&nbsp; 
    &nbsp;</SPAN></FONT></FONT></FONT><FONT face="Times New Roman, Times" 
    size=4>2.<X-TAB>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</X-TAB></FONT><FONT 
    face="Courier New, Courier">Do you have any assumptions on Multi-thread 
    processing impacts to the FSM?<BR><BR></DD></DL>If not, can you mention any 
  operational assumptions?<BR>
  <DL>
    <DD>The multi-thread processing issues would focus around the choice between 
    1 or 2 FSMs.&nbsp;&nbsp; The state machine's event 22 and the call collision 
    allows you to do both.<BR><BR>
    <DD>No assumptions on the multi-threaded processing impacts to the FSM were 
    made. 
    <DD>A two state versus 1 state model was allowed.&nbsp; The single state 
    model is indicated in the places were TCP and con<BR></FONT><FONT 
    size=2><FONT color=#0000ff><FONT face=Arial><SPAN 
    class=475025414-25072002>&nbsp;</SPAN></FONT></FONT></FONT>
    <DD><FONT size=2><FONT color=#0000ff><FONT face=Arial><SPAN 
    class=475025414-25072002><FONT color=#000000><FONT size=3><FONT 
    face="Courier New">Event 22 is merely a an open collision case. I think 
    there is more to it than just that. There are various phases that BGP goes 
    through that&nbsp;one might want to use multiple threads. The inter-locking 
    or inter-working between the threads could affect the state transitions. For 
    example, Say you are in Establish state and something happens to the session 
    and it is disconnected. In a single threaded environment it is easy, first 
    come, first served. In a multi-threaded case with thread priorities and 
    interruptions, it might get tricky at times. Some assumptions or 
    expectations by the state machine on actions it should take with regards to 
    semaphore locks (RIB Table, etc.), thread prioritization, processing, 
    pre-emption and so forth might be helpfull. Maybe these actions should only 
    be suggestions not required ways&nbsp;for 
    implementations.</FONT></FONT></FONT></SPAN></FONT></FONT></FONT></DD></DL>
  <DIV><FONT face="Courier New"><SPAN 
  class=475025414-25072002>&nbsp;&nbsp;&nbsp; I hope I have not opened a can of 
  worms here?</SPAN></FONT></DIV>
  <DIV><FONT face="Courier New"><SPAN 
  class=475025414-25072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face="Courier New"><SPAN 
  class=475025414-25072002></SPAN></FONT>&nbsp;</DIV>
  <DIV><FONT face="Courier New"><SPAN 
  class=475025414-25072002>Regards,</SPAN></FONT></DIV>
  <DIV><FONT face="Courier New"><SPAN 
  class=475025414-25072002>Ben</SPAN></FONT></DIV></BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C233EE.A0D609D0--

Date: Wed, 24 Jul 2002 12:38:08 -0700
From: [email-masked]
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: BGP Extended Communities
Sender: [email-masked]

Jeff,

Thanks for the input.  At the meeting in Yokohama, a number of other
people expressed your same concerns about deployment of the existing
2547-oriented extended communities draft.  The consensus was for me to
write up the proposal as a seperate BGP attribute.  I hope to get a 
draft out in the next couple weeks for the idr community to beat on.

Andrew

On Wed, Jul 24, 2002 at 12:18:03PM -0400, Jeffrey Haas wrote:
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Date: Wed, 24 Jul 2002 12:18:03 -0400
> From: Jeffrey Haas <[email-masked]>
> To: [email-masked]
> Cc: [email-masked], [email-masked]
> Subject: Re: BGP Extended Communities
> User-Agent: Mutt/1.2.5i
> In-Reply-To: <[email-masked]>; from [email-masked] on Tue, Jul 16, 2002 at 10:14:14PM -0700
> X-Virus-Scanned: by AMaViS perl-11
> Precedence: bulk
> X-OriginalArrivalTime: 24 Jul 2002 16:19:49.0182 (UTC) FILETIME=[EEBEC5E0:01C2332D]
> 
> Andrew,
> 
> I've given this some time for others to respond.  Since I'm
> mostly hearing resounding silence, here's my take on the issue:
> 
> On Tue, Jul 16, 2002 at 10:14:14PM -0700, [email-masked] wrote:
> > I was going over the draft-ietf-idr-bgp-ext-communities-05.txt.  I
> > think we can make significant enhancements to the functionality,
> > efficency and flexibility of this draft.  If we step back and look
> > at defining a more generic solution which can address both
> > current and future needs we will be better served.
> 
> IMO:
> 1. The authors *could* have done a much better job at providing
>    a generic coloring mechnism for the routes.
> 2. Extended communities were originally deployed to solve a
>    specific problem - RFC 2547 vpns.  These don't deal with IPv6
>    either.
> 3. This extension is already widely deployed for the above.
> 
> While it would be nice to go back and redo things, IMO this is
> too widely deployed by too many big router vendors to make this
> change.
> 
> > - Support for IPv6.  If we are going to move to IPv6 at some point,
> > we should include support for it in the protocol extentions we define.
> > - More efficient packing of values in the data field.  A good example
> > of where this enhancement would be applicable is in the
> > draft-ietf-ptomaine-bgp-redistribution-00.txt document.
> > - Cleaner support for a broad range of future data field structures,
> > and interpretations.
> > - Support for locally defined community structures (types).
> > - Support for locally defined community sub-types.
> 
> What I would recommend is that IDR spend a little time enumerating
> what properties a generic route coloring mechanism should have.
> (The above is a good start.)
> Even if this has nothing done with it over the short term, it should
> be there for anyone who decides to do the "next greatest thing".
> 
> The thing that makes me queasy is that we can very easily go
> beyond TLVs and sub-TLVs into a mess that looks like BER.
> For all their deficiencies, (extended) communities are easy to
> parse in your code.  BER isn't.
> 
> 
> -- 
> Jeff Haas 
> NextHop Technologies

Date: Wed, 24 Jul 2002 12:18:03 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: BGP Extended Communities
Sender: [email-masked]

Andrew,

I've given this some time for others to respond.  Since I'm
mostly hearing resounding silence, here's my take on the issue:

On Tue, Jul 16, 2002 at 10:14:14PM -0700, [email-masked] wrote:
> I was going over the draft-ietf-idr-bgp-ext-communities-05.txt.  I
> think we can make significant enhancements to the functionality,
> efficency and flexibility of this draft.  If we step back and look
> at defining a more generic solution which can address both
> current and future needs we will be better served.

IMO:
1. The authors *could* have done a much better job at providing
   a generic coloring mechnism for the routes.
2. Extended communities were originally deployed to solve a
   specific problem - RFC 2547 vpns.  These don't deal with IPv6
   either.
3. This extension is already widely deployed for the above.

While it would be nice to go back and redo things, IMO this is
too widely deployed by too many big router vendors to make this
change.

> - Support for IPv6.  If we are going to move to IPv6 at some point,
> we should include support for it in the protocol extentions we define.
> - More efficient packing of values in the data field.  A good example
> of where this enhancement would be applicable is in the
> draft-ietf-ptomaine-bgp-redistribution-00.txt document.
> - Cleaner support for a broad range of future data field structures,
> and interpretations.
> - Support for locally defined community structures (types).
> - Support for locally defined community sub-types.

What I would recommend is that IDR spend a little time enumerating
what properties a generic route coloring mechanism should have.
(The above is a good start.)
Even if this has nothing done with it over the short term, it should
be there for anyone who decides to do the "next greatest thing".

The thing that makes me queasy is that we can very easily go
beyond TLVs and sub-TLVs into a mess that looks like BER.
For all their deficiencies, (extended) communities are easy to
parse in your code.  BER isn't.

-- 
Jeff Haas 
NextHop Technologies

Date: Wed, 24 Jul 2002 12:08:24 -0400
From: Jeffrey Haas <[email-masked]>
To: vrishab sikand <[email-masked]>
Cc: lidefeng <[email-masked]>, [email-masked]
Subject: Re: consult :  What is the FTP User ID and Password
Sender: [email-masked]

On Tue, Jul 23, 2002 at 05:24:31AM -0700, vrishab sikand wrote:
> The archive has not been acessable for quite some time. Either this should be fixed or the link removed from IDR home page.

http://www.merit.edu/mail.archives/idr/
ftp://ftp.merit.edu/mail.archives/idr

Admittedly, their archiver has a y2k issue... :-)

-- 
Jeff Haas 
NextHop Technologies

Date: Wed, 24 Jul 2002 11:53:16 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: Pedro Roque Marques <[email-masked]>, [email-masked]
Subject: Re: inform (was: IDR WG minutes IETF 54)
Sender: [email-masked]

On Mon, Jul 22, 2002 at 07:38:52AM -0700, [email-masked] wrote:
> As an operator, I think that there are a number of useful places for an 
> INFORM message, as a mecahanism to pass information between BGP speakers
> that does not result in a session reset.

Pedro has convinced me that INFORM is a bad idea.  However, I still
have great sympathies for the operational issues.

In this light, I propose the following:
The bgp-mib-v2 is still only set in wet concrete (and is due for
the collected comments to be published).

How about if we add a counter/trap on a per-peer basis that reflects
conditions where the implementation is specifically ignoring routes
rather than dropping the peering session?  This will have to be 
worded carefully.

This will at least allow the router being "victimized" by the bad routes
to bring it to an operators attention.  The pushback mechanism
is no longer an INFORM message, rather its back to operators
giving the other operators a call.

> Andrew

-- 
Jeff Haas 
NextHop Technologies

Date: Wed, 24 Jul 2002 10:55:17 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: limit the No. of routes received from a neighbor
Sender: [email-masked]

On Tue, Jul 23, 2002 at 03:43:06PM -0700, [email-masked] wrote:
> If max-prefix is exceeded, tear down the session.  In this case there is
> no way to know which routes you should listen to and which ones you 
> shouldn't.

One *could* make an argument that one could discard any more-specific
routes from your adj-ribs-in.  This will maintain *some* reachability
for that peer.  However, one might want to flag the remaining
routes as being required and if they go away, *then* tear down
the peering session.

This is a lot more work and probably not worth it. :-)

> Andrew

-- 
Jeff Haas 
NextHop Technologies

Date: Wed, 24 Jul 2002 20:13:30 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: "Tsillas, Jim" <[email-masked]>
Cc: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Jim,
All that i'm trying to say is that we don't need to put in any extra
features into BGP (atleast not as of now!) to achieve what all you
mentioned down below. Encompassing a slow peer will be automatically taken
care of by TCP.

- Manav
|
| the real challenge for an implementation is to be both efficient
| in terms of building updates and policy processing but at the same
| time to be accomodating of slow peers. If all peers ran at the
| same speed, an implementation could theoretically build a single
| update and retransmit this update to all the peers (I'm ignoring
| alot of detail here). With the above (efficient approach) when a
| single peer slows down enough to make TCP flow control, all update
| sending must wait on that slow peer. The alternative approach is
| to independently update all peers at their given rate: this has
| the drawback that potentially for every peer the update must be
| rebuilt for sending. A hybrid approach could use some local buffering
| (above the TCP buffering) to accomodate slow peers to some extent
| and (most of the time) allowing reuse of that update buffer.
| Another complicating aspect is that peers can often be grouped
| so that their out-policy is reused - it is beneficial to reuse
| updates for these groups.
|
| Seems like a topic ripe for some modelling.
|
| -Jim.
|
| -----Original Message-----
| From: Dennis Ferguson [mailto:[email-masked]]
| Sent: Tuesday, July 23, 2002 6:36 PM
| To: Manav Bhatia
| Cc: Mareline Sheldon; [email-masked]
| Subject: Re: My BGP Route Update Pacing Draft
|
|
| Hello,
|
| > If you have some UPDATEs to announce and if you are holding them back
| > because of the congestion you perceive in the network and then if you
| > recieve a WITHDRAW for one of those same prefixes then you have to
first
| > advertise the UPDATEs because it is invalid to announce the same
address
| > prefix in the WITHDRAWN routes and the NLRI field of an UPDATE message.
| You
| > thus end up flushing all the BGP UPDATEs you had piled up waiting for
the
| > congestion to clear out at your remote end.
|
| Why would you do this?  If you receive a withdrawal for a route for which
| you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE
| and just send the withdrawal (or, if you've never advertised the route
| to the peer, just send nothing at all)?  If a route is flapping and
| goes up and down 62 times while your peer has you flow controlled, would
| you really send the 62 flaps when the peer allows you to send again?  Why
| wouldn't you just send the final result?
|
| You need to do routing protocol implementations in a way which avoids
| unbounded state, and doing this means that you shouldn't be recording
| all route transitions to play back to a peer which isn't listening at
| the moment.  If the only state you keep is what you advertised to the
| peer before and what the state of the route is now, then the amount
| of state you need to keep remains proportional to the number of routes
| you have and you only need to advertise the current state (if it doesn't
| match what the peer already knows, otherwise you send nothing) when
| the peer starts taking updates again.
|
| > The better solution is to let
| > TCP take care of the congestion internally as it has always been
|
| I don't think this has much to do with TCP, TCP flow control works
| the same way.
|
| Dennis Ferguson
|

From: "Tsillas, Jim" <[email-masked]>
To: Manav Bhatia <[email-masked]>
Cc: Mareline Sheldon <[email-masked]>, [email-masked]
Subject: RE: My BGP Route Update Pacing Draft 
Date: Wed, 24 Jul 2002 10:37:03 -0400
Sender: [email-masked]

Manav,

the real challenge for an implementation is to be both efficient
in terms of building updates and policy processing but at the same
time to be accomodating of slow peers. If all peers ran at the
same speed, an implementation could theoretically build a single
update and retransmit this update to all the peers (I'm ignoring
alot of detail here). With the above (efficient approach) when a
single peer slows down enough to make TCP flow control, all update
sending must wait on that slow peer. The alternative approach is
to independently update all peers at their given rate: this has
the drawback that potentially for every peer the update must be
rebuilt for sending. A hybrid approach could use some local buffering
(above the TCP buffering) to accomodate slow peers to some extent
and (most of the time) allowing reuse of that update buffer.
Another complicating aspect is that peers can often be grouped
so that their out-policy is reused - it is beneficial to reuse
updates for these groups.

Seems like a topic ripe for some modelling.

-Jim.

-----Original Message-----
From: Dennis Ferguson [mailto:[email-masked]]
Sent: Tuesday, July 23, 2002 6:36 PM
To: Manav Bhatia
Cc: Mareline Sheldon; [email-masked]
Subject: Re: My BGP Route Update Pacing Draft 

Hello,

> If you have some UPDATEs to announce and if you are holding them back
> because of the congestion you perceive in the network and then if you
> recieve a WITHDRAW for one of those same prefixes then you have to first
> advertise the UPDATEs because it is invalid to announce the same address
> prefix in the WITHDRAWN routes and the NLRI field of an UPDATE message.
You
> thus end up flushing all the BGP UPDATEs you had piled up waiting for the
> congestion to clear out at your remote end.

Why would you do this?  If you receive a withdrawal for a route for which
you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE
and just send the withdrawal (or, if you've never advertised the route
to the peer, just send nothing at all)?  If a route is flapping and
goes up and down 62 times while your peer has you flow controlled, would
you really send the 62 flaps when the peer allows you to send again?  Why
wouldn't you just send the final result?

You need to do routing protocol implementations in a way which avoids
unbounded state, and doing this means that you shouldn't be recording
all route transitions to play back to a peer which isn't listening at
the moment.  If the only state you keep is what you advertised to the
peer before and what the state of the route is now, then the amount
of state you need to keep remains proportional to the number of routes
you have and you only need to advertise the current state (if it doesn't
match what the peer already knows, otherwise you send nothing) when
the peer starts taking updates again.

> The better solution is to let
> TCP take care of the congestion internally as it has always been

I don't think this has much to do with TCP, TCP flow control works
the same way.

Dennis Ferguson

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Manav Bhatia'" <[email-masked]>, Dennis Ferguson <[email-masked]>
Cc: [email-masked]
Subject: RE: My BGP Route Update Pacing Draft
Date: Wed, 24 Jul 2002 10:28:50 -0400
Sender: [email-masked]

I will agree the issue with keeping updates current might become stale when
flow control is activated, because each implementation might be different
and
thus may or may not clear up the updates before they leave the box. So the 
effect of historical replay will take place at the destination router. Even
if its physically wrong it will adjust and resolve the state to the current
levels. The problem comes in when the flow control is constantly engaged and
the sending and receiving routers start to drift out of phase from reality.
That is when we will see forwarding problems, due to inaccurate routes.

I will say again, it is the responsiblity of the sending router to send the
best and most current route information available to it, even if it means 
it has to go through some session queue and cleanup the updates, which I
am sure most implementations dont do. As for the TCP queue it is as good as
gone, but as other's have said that is only 8k bytes.

Ben

> From: Manav Bhatia [mailto:[email-masked]]
> Sent: Wednesday, July 24, 2002 12:07 AM
> To: Dennis Ferguson
> Cc: [email-masked]
> Subject: Re: My BGP Route Update Pacing Draft
> 
> 
> Hi Dennis,
> 
> | Why would you do this?  If you receive a withdrawal for a 
> route for which
> | you haven't sent an UPDATE yet, why wouldn't you forget 
> about the UPDATE
> | and just send the withdrawal (or, if you've never 
> advertised the route
> | to the peer, just send nothing at all)?  If a route is flapping and
> 
> Depends on whether your BGP UPDATE packets waiting to be 
> announced are in
> some transit queue above the TCP or lying below it. If you 
> have already
> given the UPDATE to the TCP then there isnt much that you can do upon
> receiving the WITHDRAW. If its the former case (above TCP) 
> then i have yet
> to see any implementation which scans through all the pending 
> BGP UPDATEs
> it has in its transmit queue upon receiving the withdraw to 
> remove the NLRI
> if it exists in the withdraw also OR scanning through all the WITHDRAW
> messages when it recieves any new NLRI to advertise.
> 
> Manav
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'vrishab sikand'" <[email-masked]>, Manav Bhatia <[email-masked]>, Dennis Ferguson <[email-masked]>
Cc: [email-masked]
Subject: RE: My BGP Route Update Pacing Draft
Date: Wed, 24 Jul 2002 10:23:00 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C2331D.9D12D340
Content-Type: text/plain;
	charset="iso-8859-1"

Its 16K per session correct. If I have 500 sessions that is 8 million bytes.
 
Ben 

-----Original Message-----
From: vrishab sikand [mailto:[email-masked]]
Sent: Wednesday, July 24, 2002 12:59 AM
To: Manav Bhatia; Dennis Ferguson
Cc: [email-masked]
Subject: Re: My BGP Route Update Pacing Draft

  Manav Bhatia <[email-masked]> wrote: 

Hi Dennis,

| Why would you do this? If you receive a withdrawal for a route for which
| you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE
| and just send the withdrawal (or, if you've never advertised the route
| to the peer, just send nothing at all)? If a route is flapping and

Depends on whether your BGP UPDATE packets waiting to be announced are in
some transit queue above the TCP or lying below it. If you have already
given the UPDATE to the TCP then there isnt much that you can do upon
receiving the WITHDRAW. If its the former case (above TCP) then i have yet
to see any implementation which scans through all the pending BGP UPDATEs
it has in its transmit queue upon receiving the withdraw to remove the NLRI
if it exists in the withdraw also OR scanning through all the WITHDRAW
messages when it recieves any new NLRI to advertise.

Such  implemenations exist. and it does not require scanning any update
list. The route entry itself can have enough information in it to indicate
to which neighors it has been sent and to which neighors it has been queued
to be sent to. Off course once it is given to TCP it is as good as sent but
thats not a long list as TCP buffers are not VERY large @16K in most
implemenatations.

Manav

   _____  

Do You Yahoo!?
Yahoo! Health <http://health.yahoo.com/>  - Feel better, live better

------_=_NextPart_001_01C2331D.9D12D340
Content-Type: text/html;
	charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<META content="MSHTML 5.00.2920.0" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN class=302432314-24072002>Its 
16K per session correct. If I have 500 sessions that is&nbsp;8 million 
bytes.</SPAN></FONT></DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=302432314-24072002></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=302432314-24072002>Ben&nbsp;</SPAN></FONT></DIV>
<BLOCKQUOTE 
style="BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: 5px">
  <DIV align=left class=OutlookMessageHeader dir=ltr><FONT face=Tahoma 
  size=2>-----Original Message-----<BR><B>From:</B> vrishab sikand 
  [mailto:[email-masked]]<BR><B>Sent:</B> Wednesday, July 24, 2002 12:59 
  AM<BR><B>To:</B> Manav Bhatia; Dennis Ferguson<BR><B>Cc:</B> 
  [email-masked]<BR><B>Subject:</B> Re: My BGP Route Update Pacing 
  Draft<BR><BR></DIV></FONT>
  <P>
  <P>&nbsp; <B><I>Manav Bhatia &lt;[email-masked]&gt;</I></B> wrote: 
  <BLOCKQUOTE 
  style="BORDER-LEFT: #1010ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: 5px">
    <P>Hi Dennis,<BR><BR>| Why would you do this? If you receive a withdrawal 
    for a route for which<BR>| you haven't sent an UPDATE yet, why wouldn't you 
    forget about the UPDATE<BR>| and just send the withdrawal (or, if you've 
    never advertised the route<BR>| to the peer, just send nothing at all)? If a 
    route is flapping and<BR><BR>Depends on whether your BGP UPDATE packets 
    waiting to be announced are in<BR>some transit queue above the TCP or lying 
    below it. If you have already<BR>given the UPDATE to the TCP then there isnt 
    much that you can do upon<BR>receiving the WITHDRAW. If its the former case 
    (above TCP) then i have yet<BR>to see any implementation which scans through 
    all the pending BGP UPDATEs<BR>it has in its transmit queue upon receiving 
    the withdraw to remove the NLRI<BR>if it exists in the withdraw also OR 
    scanning through all the WITHDRAW<BR>messages when it recieves any new NLRI 
    to advertise.</P>
    <P>Such&nbsp; implemenations exist. and it does not require scanning any 
    update list. The route entry itself can have enough information in it to 
    indicate to which neighors it has been sent and to which neighors it has 
    been queued to be sent to. Off course once it is given to TCP it is as good 
    as sent but thats not a long list as TCP buffers are not VERY large @16K in 
    most implemenatations.<BR><BR>Manav<BR></P></BLOCKQUOTE>
  <P><BR>
  <HR SIZE=1>
  <B>Do You Yahoo!?</B><BR><A href="http://health.yahoo.com/">Yahoo! Health</A> 
  - Feel better, live better</BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C2331D.9D12D340--

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Dennis Ferguson'" <[email-masked]>, Manav Bhatia <[email-masked]>
Cc: Mareline Sheldon <[email-masked]>, [email-masked]
Subject: RE: My BGP Route Update Pacing Draft 
Date: Wed, 24 Jul 2002 10:16:18 -0400
Sender: [email-masked]

Dennis:
  I agree we should not send updates that are flow controlled that have 
new changes imposed on them by new events in the network. Only send the 
current state of all routes. It is the responsibility of the sending
router to report current and accurate information to its peers.

As we can see flow control conditions are a messy subject to deal with 
and does often ripple back to the application to take new (current) 
actions.

Ben

> From: Dennis Ferguson [mailto:[email-masked]]
> Sent: Tuesday, July 23, 2002 6:36 PM
> To: Manav Bhatia
> Cc: Mareline Sheldon; [email-masked]
> Subject: Re: My BGP Route Update Pacing Draft 
> 
> 
> Hello,
> 
> > If you have some UPDATEs to announce and if you are holding 
> them back
> > because of the congestion you perceive in the network and 
> then if you
> > recieve a WITHDRAW for one of those same prefixes then you 
> have to first
> > advertise the UPDATEs because it is invalid to announce the 
> same address
> > prefix in the WITHDRAWN routes and the NLRI field of an 
> UPDATE message. You
> > thus end up flushing all the BGP UPDATEs you had piled up 
> waiting for the
> > congestion to clear out at your remote end.
> 
> Why would you do this?  If you receive a withdrawal for a 
> route for which
> you haven't sent an UPDATE yet, why wouldn't you forget about 
> the UPDATE
> and just send the withdrawal (or, if you've never advertised the route
> to the peer, just send nothing at all)?  If a route is flapping and
> goes up and down 62 times while your peer has you flow 
> controlled, would
> you really send the 62 flaps when the peer allows you to send 
> again?  Why
> wouldn't you just send the final result?
> 
> You need to do routing protocol implementations in a way which avoids
> unbounded state, and doing this means that you shouldn't be recording
> all route transitions to play back to a peer which isn't listening at
> the moment.  If the only state you keep is what you advertised to the
> peer before and what the state of the route is now, then the amount
> of state you need to keep remains proportional to the number of routes
> you have and you only need to advertise the current state (if 
> it doesn't
> match what the peer already knows, otherwise you send nothing) when
> the peer starts taking updates again.
> 
> > The better solution is to let
> > TCP take care of the congestion internally as it has always been
> 
> I don't think this has much to do with TCP, TCP flow control works
> the same way.
> 
> Dennis Ferguson
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Dennis Ferguson'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: FW: My BGP Route Update Pacing Draft 
Date: Wed, 24 Jul 2002 10:05:56 -0400
Sender: [email-masked]

Dennis:
  I respect your knowledge and have read your books. You have said a
mouthfull
here. I have no ground to respond so I wont. Thanks for the long
explanation.
I hope you are right and I am wrong and we are not loosing a good idea.
Anyway
I have my thoughts and doubts about things, but that is a matter of opinion.

Thanks again,
Ben

> From: Dennis Ferguson [mailto:[email-masked]]
> Sent: Tuesday, July 23, 2002 5:42 PM
> To: Abarbanel, Benjamin
> Cc: [email-masked]'
> Subject: Re: FW: My BGP Route Update Pacing Draft 
> 
> 
> Hello,
> 
> > > 
> > > Why not relly on the transport. Flow control is after all
> > > traditionally the function of the transport. What makes 
> you think it
> > > needs to be reinvented... and how is your version of flow control
> > > supperior to the current one ?
> > >
> [...]
> > The other end that is being flow controlled does not know why 
> > or by how much he
> > will be flow controlled off. Remember on a long session with 
> > many hops it takes significant time to communicate the flow 
> > control condition by ACKing or not ACKing messages. For me 
> > this is not a direct but indirect mechanism.
> > In the interim, the pipe will be full of data.
> 
> I think you are confused about this.  TCP flow control does 
> not work by
> withholding ACKs. TCP always ACKs messages that are received, and flow
> control is instead implemented by directly telling the other end
> not to send more.  The sender is informed of this condition as soon as
> it occurs, and will be informed that the condition has cleared in the
> time it takes for a packet to transit the path between the 
> peers (which
> is why it is probably good to have a TCP window of outstanding data
> at the receiver that it can process until the sender gets the
> message to send more, and noting that the receiver is in full control
> of just how much or little data that is).  It doesn't get much better
> than that.
> 
> Now a sender who has had all outstanding packets ACK'd but which
> is flow controlled off knows precisely why this has occurred; it
> has been sending updates faster than the receiver was willing to
> process them.  There is no ambiguity, this is the only way this
> situation could occur.
> 
> Therefore, out of everything you said above, the only thing I can
> see which is perfectly accurate is that the flow-controlled sender
> won't know how long it will be flow-controlled off.  This will depend
> on what other things the receiving router has to do which it finds
> more important than processing BGP messages from this particular
> peer.  It is hard to see how the receiver could know how long that
> is going to take until it actually does it, however, so it seems
> that telling the sender this would require the receiver to be able
> to predict the future (the sender knows all about the past already
> so there isn't much use having the receiver tell the sender about
> that).  I can't dispute that both prescient routers and 
> fortune tellers
> might be useful, but I'm not sure that either is a practical
> possibility in real life.
> 
> While I doubt the implementability of the future-prediction you
> seem to be looking for here, however, I think there is another
> more fundamental issue which says your approach is incorrect.  That
> is, TCP flow control already puts the congested receiver in full
> control of its load.  No one can send data faster than the congested
> receiver is willing to process it.  The congested receiver directly
> controls how much data is outstanding when flow control is asserted.
> The sender can't do anything to the receiver that the receiver doesn't
> explicitly permit.  With TCP flow control the congested receiver
> has full control of its resources.  If there is any problem at all
> here, then, it is that the sender has no control at all over the
> receiver's resources and hence may be inconvenienced by a congested
> receiver which chooses to take its time responding.  And, in fact,
> if we're talking about Capability messages in particular, it is of
> no advantage in general at all to the busy receiver to get this
> message any earlier than updates or anything else; it doesn't 
> even know
> the message is coming until it sees it.  Only the uncongested sender
> knows there is an outstanding capability message.
> 
> So if there is a problem here at all it is a problem for the
> idle sender, which might have to wait for a response to a Capability
> message, rather than the busy receiver.  Yet your proposed solution
> for this is to have the busy receiver do yet more work, continually
> predicting the future and sending Inform messages about it to all of
> its (possibly many) peers, on the off chance that a sender which
> might someday have a Capability message to send, and which has CPU
> cycles to burn, isn't inconvenienced.  Since it is the guy who has
> CPU cycles available which has the problem in the first place it seems
> utterly backwards to make the guy who has no CPU cycles free do more
> work to fix it.  The guy with the CPU cycles should be the one to fix
> this, i.e. the sender should spend his free cycles doing whatever it
> takes to wait until the congested guy gets around to responding.  Then
> everything works fine with no protocol changes.
> 
> > > In BGP a receiver can choose to accept a anyrate of updates 
> > it wants.
> > 
> > Yes, it can by reading or not reading the TCP socket queue. 
> > Which gets us
> > back to the original point I mentioned earlier. TCP FLOW 
> > Control is not
> > perfect.
> 
> Your original point seems to have been based on a faulty understanding
> of how TCP flow control works.
> 
> > > Please do offer more to substantiate this other than your 
> claim that
> > > the problem exists.
> > >
> >  
> > By the fact that the pacing control is more direct and implicit to a
> > particular
> > traffic type (in this case update messages). The pacing mechanism is
> > deterministic, whereas TCP flow control is not. You should 
> talk to people
> > who build IPC mechanisms to replace TCP about this issue. 
> 
> Or the guys that built deterministic FDDI or Token Ring or Token Bus
> or ATM LANs to replace non-deterministic Ethernets.  We have networks
> where non-deterministic stuff just happens despite our best efforts to
> avoid it, so you might be better off to write code which deals with
> that rather than depending on some other box to deterministically
> predict the future so you aren't inconvenienced.  While you 
> are correct
> that this would be easier for your box if other routers could predict
> their future for you with 100% reliability, I don't think this is
> likely to be possible in any case.
> 
> Dennis Ferguson
> 

Date: Wed, 24 Jul 2002 07:04:20 -0700 (PDT)
From: PamSri <[email-masked]>
Subject: Re:
To: Manav Bhatia <[email-masked]>, [email-masked], [email-masked]
Sender: [email-masked]

Hi Manav,
        Per peer prefix limit has a direct impact on
the BGP RIBs. Regarding what happens when the limit is
hit is not what we're discussing. I dont think there
is a shortcoming/flaw in the protocol that could lead
to this situation but is merely a case of the
limitation of the router/system/implementation. If
reasonable peers and the routes handled by them are
configured (based on what the system/router can
handle)then it will not lead to the situation
described by Ben.

Regards,

sri   

--- Manav Bhatia <[email-masked]> wrote:
> Padma,
> As the name suggests peer prefix limit is per peer
> basis and is not
> applicable to the BGP RIB in whole. The peer can get
> congested even if it
> receives routes < prefix limit from multiple peers.
> What peer prefix limit
> will merely do is to tear down the session with the
> offending peer if the
> nos. of prefixes we receive from it goes beyond our
> default/configured
> acceptable value.
> 
> configuring and using the peer prefix limit is *not*
> at all a solution for
> peer congestion.
> 
> Manav
> 
> ----- Original Message -----
> From: "Padmaja kotamarthiC" <[email-masked]>
> To: <[email-masked]>;
> <[email-masked]>; <[email-masked]>
> Sent: Tuesday, July 23, 2002 8:05 PM
> 
> 
> | Hi Manav,
> |         If your peer is congested for whatever
> reason
> | the simple solution is to: set the peer prefix
> limit
> | to the appropriate value to avoid the problem.
> |
> | sri
> |
> | Hi Ben,
> | I have some thoughts on the draft.
> |
> | The crux of your draft is that if a peer gets so
> | congested that it is not
> | able to send any UPDATE/KEEPALIVE message to the
> | remote uncongested peer
> | then the session can break down because of the
> Hold
> | Timer expiring and it
> | can be prevented by sending some pacing
> information
> | which will inform the
> | uncongested peer to limit its route advertisement
> | rate. The PACING
> | information will too be carried in the TCP
> packets, so
> | if the TCP is
> | implementing flow control and is not sending
> packets
> | to avoid congestion
> | then how will the PACING information be sent? And
> if
> | TCP can send the
> | PACING information then it can as well send the
> | KEEPALIVE packet which will
> | keep the HoldTimer from expiring at the other end.
> |
> | Secondly, if you are aware that your router is old
> and
> | might come in a
> | situation when it might not be able to handle the
> full
> | feed of BGP Updates
> | from the remote end then it can negotiate a
> HoldTime
> | of zero and keep the
> | session running even when it is in the congested
> | state.
> |
> | Thirdly, the very idea of artificially impeding
> | reachability information
> | (or sending of BGP UPDATE messages) doesn't really
> | appeal to me. I was
> | initially against the idea of using
> | MinRouteAdvertisementInternal etc
> | timers in my implementation arguing that a router
> | should pack off any
> | reachability information it has as soon as it
> comes to
> | know of it. But this
> | created a problem when i got say some 80K routes.
> I
> | would then be sending
> | 80K packets to my other peers! Finally , better
> sense
> | prevailed on me and i
> | started using the timers for efficiently packing
> my
> | NLRIs in the UPDATE
> | messages. This is one point where i hold my
> | reachability information from
> | my peers & I discourage it doing any other time.
> IMHO
> | i feel we should let
> | TCP take care of withholding any data it wants
> with
> | itself till the
> | congestion state rides by rather than us
> interfering
> | with it.
> |
> | What happens say when an uncongested router
> receives
> | pacing information
> | telling it to reduce the advertisement rate by
> some
> | 70%. In that case there
> | will be many UPDATEs lying with it remaining to be
> yet
> | announced to the
> | remote peer which is experiencing congestion. Now
> what
> | if the uncongested
> | network receives withdrawals (both implicit and
> | explicit) for certain
> | prefixes which have already been announced or
> remain
> | yet to be announced?
> | You open up a can of worms now !!
> |
> | You have to now flush all your output buffers
> | regardless of the congestion
> | state your remote peer lies in!
> |
> | Thus you end up doing what you always wanted to
> avoid.
> |
> | I think a cleaner and a much simpler approach
> would be
> | to dynamically
> | negotiate the value of the HoldTimer during the
> | session. If you feel you
> | are getting congested then increase the value and
> once
> | the red flags are
> | all down you can revert back to your original
> value or
> | decrease it back
> | again.
> |
> | Regards,
> | Manav
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> |
> | ----- Original Message -----
> | From: "Abarbanel, Benjamin"
> | <[email-masked]>
> | To: <[email-masked]>
> | Sent: Friday, July 19, 2002 1:01 AM
> | Subject: My BGP Route Update Pacing Draft
> |
> |
> | | Hi all:
> | |
> | |    Our recent discussions on this list and my
> recent
> | work
> | |  experiences have led me to write this draft and
> | offer it
> | |  to the IETF community. I would appreciate any
> | comments
> | |  anyone has to make.
> | |
> | |  Thanks in advance,
> | |  Ben
> | |
> | |
> | |
> | |
> |
> |
> |
> |
> |
> | __________________________________________________
> | Do You Yahoo!?
> | Yahoo! Health - Feel better, live better
> | http://health.yahoo.com
> 

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Date: Tue, 23 Jul 2002 21:59:14 -0700 (PDT)
From: vrishab sikand <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: Manav Bhatia <[email-masked]>, Dennis Ferguson <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

--0-1930548566-1027486754=:6384
Content-Type: text/plain; charset=us-ascii

  Manav Bhatia <[email-masked]> wrote: 
Hi Dennis,

| Why would you do this? If you receive a withdrawal for a route for which
| you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE
| and just send the withdrawal (or, if you've never advertised the route
| to the peer, just send nothing at all)? If a route is flapping and

Depends on whether your BGP UPDATE packets waiting to be announced are in
some transit queue above the TCP or lying below it. If you have already
given the UPDATE to the TCP then there isnt much that you can do upon
receiving the WITHDRAW. If its the former case (above TCP) then i have yet
to see any implementation which scans through all the pending BGP UPDATEs
it has in its transmit queue upon receiving the withdraw to remove the NLRI
if it exists in the withdraw also OR scanning through all the WITHDRAW
messages when it recieves any new NLRI to advertise.

Such  implemenations exist. and it does not require scanning any update list. The route entry itself can have enough information in it to indicate to which neighors it has been sent and to which neighors it has been queued to be sent to. Off course once it is given to TCP it is as good as sent but thats not a long list as TCP buffers are not VERY large @16K in most implemenatations.

Manav

---------------------------------
Do You Yahoo!?
Yahoo! Health - Feel better, live better
--0-1930548566-1027486754=:6384
Content-Type: text/html; charset=us-ascii

<P> 
<P>&nbsp; <B><I>Manav Bhatia &lt;[email-masked]&gt;</I></B> wrote: 
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">
<P>Hi Dennis,<BR><BR>| Why would you do this? If you receive a withdrawal for a route for which<BR>| you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE<BR>| and just send the withdrawal (or, if you've never advertised the route<BR>| to the peer, just send nothing at all)? If a route is flapping and<BR><BR>Depends on whether your BGP UPDATE packets waiting to be announced are in<BR>some transit queue above the TCP or lying below it. If you have already<BR>given the UPDATE to the TCP then there isnt much that you can do upon<BR>receiving the WITHDRAW. If its the former case (above TCP) then i have yet<BR>to see any implementation which scans through all the pending BGP UPDATEs<BR>it has in its transmit queue upon receiving the withdraw to remove the NLRI<BR>if it exists in the withdraw also OR scanning through all the WITHDRAW<BR>messages when it recieves any new NLRI to advertise.</P>
<P>Such&nbsp; implemenations exist. and it does not require scanning any update list. The route entry itself can have enough information in it to indicate to which neighors it has been sent and to which neighors it has been queued to be sent to. Off course once it is given to TCP it is as good as sent but thats not a long list as TCP buffers are not VERY large @16K in most implemenatations.<BR><BR>Manav<BR></P></BLOCKQUOTE><p><br><hr size=1><b>Do You Yahoo!?</b><br>
<a href="http://health.yahoo.com/">Yahoo! Health</a> - Feel better, live better
--0-1930548566-1027486754=:6384--

Date: Wed, 24 Jul 2002 09:37:11 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: Dennis Ferguson <[email-masked]>
Cc: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Hi Dennis,

| Why would you do this?  If you receive a withdrawal for a route for which
| you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE
| and just send the withdrawal (or, if you've never advertised the route
| to the peer, just send nothing at all)?  If a route is flapping and

Depends on whether your BGP UPDATE packets waiting to be announced are in
some transit queue above the TCP or lying below it. If you have already
given the UPDATE to the TCP then there isnt much that you can do upon
receiving the WITHDRAW. If its the former case (above TCP) then i have yet
to see any implementation which scans through all the pending BGP UPDATEs
it has in its transmit queue upon receiving the withdraw to remove the NLRI
if it exists in the withdraw also OR scanning through all the WITHDRAW
messages when it recieves any new NLRI to advertise.

Manav

Date: Wed, 24 Jul 2002 09:23:16 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re:
To: Padmaja kotamarthiC <[email-masked]>, [email-masked], [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Padma,
As the name suggests peer prefix limit is per peer basis and is not
applicable to the BGP RIB in whole. The peer can get congested even if it
receives routes < prefix limit from multiple peers. What peer prefix limit
will merely do is to tear down the session with the offending peer if the
nos. of prefixes we receive from it goes beyond our default/configured
acceptable value.

configuring and using the peer prefix limit is *not* at all a solution for
peer congestion.

Manav

----- Original Message -----
From: "Padmaja kotamarthiC" <[email-masked]>
To: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Tuesday, July 23, 2002 8:05 PM

| Hi Manav,
|         If your peer is congested for whatever reason
| the simple solution is to: set the peer prefix limit
| to the appropriate value to avoid the problem.
|
| sri
|
| Hi Ben,
| I have some thoughts on the draft.
|
| The crux of your draft is that if a peer gets so
| congested that it is not
| able to send any UPDATE/KEEPALIVE message to the
| remote uncongested peer
| then the session can break down because of the Hold
| Timer expiring and it
| can be prevented by sending some pacing information
| which will inform the
| uncongested peer to limit its route advertisement
| rate. The PACING
| information will too be carried in the TCP packets, so
| if the TCP is
| implementing flow control and is not sending packets
| to avoid congestion
| then how will the PACING information be sent? And if
| TCP can send the
| PACING information then it can as well send the
| KEEPALIVE packet which will
| keep the HoldTimer from expiring at the other end.
|
| Secondly, if you are aware that your router is old and
| might come in a
| situation when it might not be able to handle the full
| feed of BGP Updates
| from the remote end then it can negotiate a HoldTime
| of zero and keep the
| session running even when it is in the congested
| state.
|
| Thirdly, the very idea of artificially impeding
| reachability information
| (or sending of BGP UPDATE messages) doesn't really
| appeal to me. I was
| initially against the idea of using
| MinRouteAdvertisementInternal etc
| timers in my implementation arguing that a router
| should pack off any
| reachability information it has as soon as it comes to
| know of it. But this
| created a problem when i got say some 80K routes. I
| would then be sending
| 80K packets to my other peers! Finally , better sense
| prevailed on me and i
| started using the timers for efficiently packing my
| NLRIs in the UPDATE
| messages. This is one point where i hold my
| reachability information from
| my peers & I discourage it doing any other time. IMHO
| i feel we should let
| TCP take care of withholding any data it wants with
| itself till the
| congestion state rides by rather than us interfering
| with it.
|
| What happens say when an uncongested router receives
| pacing information
| telling it to reduce the advertisement rate by some
| 70%. In that case there
| will be many UPDATEs lying with it remaining to be yet
| announced to the
| remote peer which is experiencing congestion. Now what
| if the uncongested
| network receives withdrawals (both implicit and
| explicit) for certain
| prefixes which have already been announced or remain
| yet to be announced?
| You open up a can of worms now !!
|
| You have to now flush all your output buffers
| regardless of the congestion
| state your remote peer lies in!
|
| Thus you end up doing what you always wanted to avoid.
|
| I think a cleaner and a much simpler approach would be
| to dynamically
| negotiate the value of the HoldTimer during the
| session. If you feel you
| are getting congested then increase the value and once
| the red flags are
| all down you can revert back to your original value or
| decrease it back
| again.
|
| Regards,
| Manav
|
|
|
|
|
|
|
|
|
|
|
| ----- Original Message -----
| From: "Abarbanel, Benjamin"
| <[email-masked]>
| To: <[email-masked]>
| Sent: Friday, July 19, 2002 1:01 AM
| Subject: My BGP Route Update Pacing Draft
|
|
| | Hi all:
| |
| |    Our recent discussions on this list and my recent
| work
| |  experiences have led me to write this draft and
| offer it
| |  to the IETF community. I would appreciate any
| comments
| |  anyone has to make.
| |
| |  Thanks in advance,
| |  Ben
| |
| |
| |
| |
|
|
|
|
|
| __________________________________________________
| Do You Yahoo!?
| Yahoo! Health - Feel better, live better
| http://health.yahoo.com

Date: Wed, 24 Jul 2002 09:14:47 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: "M. ELK" <[email-masked]>
Cc: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Elk,

| for case 1 :
|
| Could the graceful restart mechanism be a solution to this pblm .
|
| Say R1 is peering with N peer . R1 is under stress
|
| R1 select to close 1..m  TCP session with peer 1 to m , deal with the
remaining  session (m+1 ...N) and clear his congestion (now R1 have
|
| enough HP ) . R1 initiae a session to peer 1 ...m with restart bit set
and f bit set  .

Few comments follow:

- How will R1 know in advance the time it may take to come out of the
congested state? Because R1 has to advertise some "Restart time" as the
worst case time during the startup. And since the Restart Time < HoldTime
carried in the OPEN message you are not left with much time to clear your
congestion !
- On what basis does R1 choose peers 1 .. m to close its TCP session with?
Why not m+1 .. N OR m+1 .. p where p<=N for that matter ?
- Closing the session with 1 .. m because R1 cant take all that load isnt
really a bright idea. Moreover what if the load doesnt go away and the
congestion doesnt clear with time?

|  Here we assume that R1 could sort his stress within a restart time
value.

on what basis? do you have any statistics/reports/studies which state that?

Manav

Date: Wed, 24 Jul 2002 08:42:34 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Ben,

| > I think a cleaner and a much simpler approach would be to dynamically
| > negotiate the value of the HoldTimer during the session. If
| > you feel you
| > are getting congested then increase the value and once the
| > red flags are
| > all down you can revert back to your original value or
| > decrease it back
| > again.
| >
| Changing the hold timer does not reduce the congestion imposed on the
| current
| router by its peers. It only extends the time it takes for the session to
| disconnect.

which is exactly what i wanted to say .. you mentioned that the uncongested
peer is unable to receive KEEPALIVEs and UPDATEs from the remote end and it
brings down the session .. "The mistake is made by the uncongested peer
since it does not see any Keepalive/Update messages before its Hold timer
expires and drops the session thereby dropping all its associated routes."

I was merely suggesting an alternative to avoid this "mistake" by
dynamically suggesting a way to increase the time it takes to disconnect
the session.

Manav

Date: Tue, 23 Jul 2002 15:43:06 -0700
From: [email-masked]
To: Tian Fang <[email-masked]>
Cc: [email-masked]
Subject: Re: limit the No. of routes received from a neighbor
Sender: [email-masked]

If max-prefix is exceeded, tear down the session.  In this case there is
no way to know which routes you should listen to and which ones you 
shouldn't.  Tearing down the session will force human intervention who
can either fix the problem, if max-prefix was exceeded accidently, or
request that the limit be raised if max-prefix was triggered by normal
growth.

Andrew

On Tue, Jul 23, 2002 at 06:11:47PM -0400, Tian Fang wrote:
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> From: Tian Fang <[email-masked]>
> To: [email-masked]
> Subject: limit the No. of routes received from a neighbor
> Date: Tue, 23 Jul 2002 18:11:47 -0400
> X-Mailer: Internet Mail Service (5.5.2653.19)
> Precedence: bulk
> X-OriginalArrivalTime: 23 Jul 2002 22:14:42.0869 (UTC) FILETIME=[585BCE50:01C23296]
> 
> Hi all,
> 
> When the No. of routes received from a neighbor exceeds the configured
> limit, is it better to tear down the session or drop the routes received
> later on? If dropping routes is a option, is there any preference that we
> can use to choose the routes to be dropped? Thx.
> 
> Regards,
> Tian Fang

To: Manav Bhatia <[email-masked]>
Cc: Mareline Sheldon <[email-masked]>, [email-masked]
Subject: Re: My BGP Route Update Pacing Draft 
Date: Tue, 23 Jul 2002 15:35:42 -0700
From: Dennis Ferguson <[email-masked]>
Sender: [email-masked]

Hello,

> If you have some UPDATEs to announce and if you are holding them back
> because of the congestion you perceive in the network and then if you
> recieve a WITHDRAW for one of those same prefixes then you have to first
> advertise the UPDATEs because it is invalid to announce the same address
> prefix in the WITHDRAWN routes and the NLRI field of an UPDATE message. You
> thus end up flushing all the BGP UPDATEs you had piled up waiting for the
> congestion to clear out at your remote end.

Why would you do this?  If you receive a withdrawal for a route for which
you haven't sent an UPDATE yet, why wouldn't you forget about the UPDATE
and just send the withdrawal (or, if you've never advertised the route
to the peer, just send nothing at all)?  If a route is flapping and
goes up and down 62 times while your peer has you flow controlled, would
you really send the 62 flaps when the peer allows you to send again?  Why
wouldn't you just send the final result?

You need to do routing protocol implementations in a way which avoids
unbounded state, and doing this means that you shouldn't be recording
all route transitions to play back to a peer which isn't listening at
the moment.  If the only state you keep is what you advertised to the
peer before and what the state of the route is now, then the amount
of state you need to keep remains proportional to the number of routes
you have and you only need to advertise the current state (if it doesn't
match what the peer already knows, otherwise you send nothing) when
the peer starts taking updates again.

> The better solution is to let
> TCP take care of the congestion internally as it has always been

I don't think this has much to do with TCP, TCP flow control works
the same way.

Dennis Ferguson

From: Tian Fang <[email-masked]>
To: [email-masked]
Subject: limit the No. of routes received from a neighbor
Date: Tue, 23 Jul 2002 18:11:47 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C23295.EF95C100
Content-Type: text/plain;
	charset="iso-8859-1"

Hi all,

When the No. of routes received from a neighbor exceeds the configured
limit, is it better to tear down the session or drop the routes received
later on? If dropping routes is a option, is there any preference that we
can use to choose the routes to be dropped? Thx.

Regards,
Tian Fang

------_=_NextPart_001_01C23295.EF95C100
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Diso-8859-1">
<META NAME=3D"Generator" CONTENT=3D"MS Exchange Server version =
5.5.2653.12">
<TITLE>limit the No. of routes received from a neighbor</TITLE>
</HEAD>
<BODY>

<P><FONT SIZE=3D2>Hi all,</FONT>
</P>

<P><FONT SIZE=3D2>When the No. of routes received from a neighbor =
exceeds the configured limit, is it better to tear down the session or =
drop the routes received later on? If dropping routes is a option, is =
there any preference that we can use to choose the routes to be =
dropped? Thx.</FONT></P>

<P><FONT SIZE=3D2>Regards,</FONT>
<BR><FONT SIZE=3D2>Tian Fang</FONT>
</P>

</BODY>
</HTML>
------_=_NextPart_001_01C23295.EF95C100--

From: "Natale, Jonathan" <[email-masked]>
To: "'Dennis Ferguson'" <[email-masked]>, Manav Bhatia <[email-masked]>
Cc: "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: RE: My BGP Route Update Pacing Draft 
Date: Tue, 23 Jul 2002 18:06:37 -0400
Sender: [email-masked]

TCP URG ???
2nd session ???

-----Original Message-----
From: Dennis Ferguson [mailto:[email-masked]] 
Sent: Tuesday, July 23, 2002 5:48 PM
To: Manav Bhatia
Cc: Abarbanel, Benjamin; [email-masked]
Subject: Re: My BGP Route Update Pacing Draft 

Hello,

> The crux of your draft is that if a peer gets so congested that it is not
> able to send any UPDATE/KEEPALIVE message to the remote uncongested peer
> then the session can break down because of the Hold Timer expiring and it
> can be prevented by sending some pacing information which will inform the
> uncongested peer to limit its route advertisement rate.

Note that, right now, it is impossible to send BGP updates to a peer any
faster than the peer is willing to process them.  Period.  TCP flow control
ensures this.  It hence isn't possible for update senders to put any more
load on a peer than that peer itself chooses to accept.  If a BGP peer
wants to ensure that its CPU is never more that 50% busy it can do
that already by not spending more than 50% of its CPU processing BGP
updates,
there is no way its neighbours can make it work harder than it wants to.
And the peer is free to stop processing update messages whenever it needs
to, like when it needs to send KEEPALIVEs, so a peer which voluntarily keeps
processing incoming updates which it should have been sending KEEPALIVEs is
just broken and needs to be fixed.

I hence don't think this draft has anything to do with fixing busy
BGP speakers, they are already in full control of their load.  The
draft seems to in fact have something to do with the inconvenience
an idle remote peer might suffer if the busy guy takes a long time
to respond to, say, a Capability message, but since the guys who are
inconvenienced are the guys who have available CPU cycles it seems to me
to be appropriate for those guys to spend the cycles dealing with this
themselves.

Dennis Ferguson

Date: Tue, 23 Jul 2002 14:56:12 -0700 (PDT)
From: Padmaja kotamarthiC <[email-masked]>
Subject: Re: FW: My BGP Route Update Pacing Draft 
To: Dennis Ferguson <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Sender: [email-masked]

Hi ,
    Here's a transcript of my (offline) discussion
with Ben.

sri 

Ben wrote :
> OK,

> From: Padmaja kotamarthiC
[mailto:[email-masked]]
> Sent: Tuesday, July 23, 2002 12:48 PM
> To: Abarbanel, Benjamin
> Subject: RE: Route Update Pacing
> > > Hi Ben ,
> In a *practical* situation with all the
> knobs/configurations provided by the protocol i am
not
> sure if this situation can be encountered. If in a
> *practical* situation this situation is encountered
i
> would consider the onus is on the system/router than
> the protocol. 
> > Like i said for the hypothetical case you mentiond
> limiting the number of routes/prefix per peer based
on
> the system/router specs should sufficiently handle
the
> congestion if not the onus is on the system and not
> the protocol. 
> > sri
> > --- "Abarbanel, Benjamin"
> <[email-masked]> wrote:
> > 300 peers with 100k routes each stressing the
router
> > continuously by
> > adding and withdrawing routes.
> > > > Although this is hypothetical, the router
should be
> > able to handle
> > it without session drops or data loss. And a
> > reasonable convergence
> > time under 1 minute.
> > > > Ben
> > > From: Padmaja kotamarthiC
> > [mailto:[email-masked]]
> > > Sent: Tuesday, July 23, 2002 12:24 PM
> > > To: Abarbanel, Benjamin
> > > Subject: RE: Route Update Pacing
> > > > > > > > > Hi Ben ,
> > > The thing to understand is what conditions
> > > this situation arises. How many peers and routes
> > are
> > > being processed when this situation (congestion)
> > is
> > > encountered by you? I am just trying to
understand
> > if
> > > there's a:  system/router bottleneck or protocol
> > > issue.
> > > > > > sri
> > > > > > --- "Abarbanel, Benjamin"
> > > <[email-masked]> wrote:
> > > > > From: Padmaja kotamarthiC
> > > > [mailto:[email-masked]]
> > > > > Sent: Tuesday, July 23, 2002 11:38 AM
> > > > > To: Abarbanel, Benjamin; [email-masked]
> > > > > Cc: [email-masked]
> > > > > Subject: Route Update Pacing
> > > > > > > > > > > > > > > Hi Ben ,
> > > > > What is the affect of this solution on
> > > > > convergence ? Your solution seems to be
> > similar to
> > > > > minimumrouteadvertisementinterval with a
> > > > modification
> > > > > that this timer comes into picture only
during
> > > > > congestion rather than all times. If we can
> > modify
> > > > the
> > > > > role of minimumrouteadvertisementinterval by
> > > > > implementing as stated above it can solve
the
> > > > issue
> > > > > too rather than introducing new messages.
> > > > > > > > > > sri
> > > > > > > > > Sri:
> > > > The minimumrouteadvertisementinterval for IBGP
> > > > routes works 
> > > > on a preconfigured timeout value. This value
> > does
> > > > not change 
> > > > dynamically to load/stress conditions
therefore
> > it
> > > > is suboptimal
> > > > to handle the congestion problem. There has
been
> > a
> > > > study done
> > > > that shows its partner timer
> > > > MinRouteAdvertisementInternal for EBGP
> > > > routes to take to long and cause convergence
> > > > problems.
> > > > > > > > check this out:
> > > > > > > >
> > >
> >
>
http://www.ripe.net/ripe/meetings/archive/ripe-42/presentation
> > s/ripe42-eof-b
> > > gp/sld038.html

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

To: Manav Bhatia <[email-masked]>
Cc: "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: Re: My BGP Route Update Pacing Draft 
Date: Tue, 23 Jul 2002 14:48:23 -0700
From: Dennis Ferguson <[email-masked]>
Sender: [email-masked]

Hello,

> The crux of your draft is that if a peer gets so congested that it is not
> able to send any UPDATE/KEEPALIVE message to the remote uncongested peer
> then the session can break down because of the Hold Timer expiring and it
> can be prevented by sending some pacing information which will inform the
> uncongested peer to limit its route advertisement rate.

Note that, right now, it is impossible to send BGP updates to a peer any
faster than the peer is willing to process them.  Period.  TCP flow control
ensures this.  It hence isn't possible for update senders to put any more
load on a peer than that peer itself chooses to accept.  If a BGP peer
wants to ensure that its CPU is never more that 50% busy it can do
that already by not spending more than 50% of its CPU processing BGP updates,
there is no way its neighbours can make it work harder than it wants to.
And the peer is free to stop processing update messages whenever it needs
to, like when it needs to send KEEPALIVEs, so a peer which voluntarily keeps
processing incoming updates which it should have been sending KEEPALIVEs is
just broken and needs to be fixed.

I hence don't think this draft has anything to do with fixing busy
BGP speakers, they are already in full control of their load.  The
draft seems to in fact have something to do with the inconvenience
an idle remote peer might suffer if the busy guy takes a long time
to respond to, say, a Capability message, but since the guys who are
inconvenienced are the guys who have available CPU cycles it seems to me
to be appropriate for those guys to spend the cycles dealing with this
themselves.

Dennis Ferguson

To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: FW: My BGP Route Update Pacing Draft 
Date: Tue, 23 Jul 2002 14:42:27 -0700
From: Dennis Ferguson <[email-masked]>
Sender: [email-masked]

Hello,

> > 
> > Why not relly on the transport. Flow control is after all
> > traditionally the function of the transport. What makes you think it
> > needs to be reinvented... and how is your version of flow control
> > supperior to the current one ?
> >
[...]
> The other end that is being flow controlled does not know why 
> or by how much he
> will be flow controlled off. Remember on a long session with 
> many hops it takes significant time to communicate the flow 
> control condition by ACKing or not ACKing messages. For me 
> this is not a direct but indirect mechanism.
> In the interim, the pipe will be full of data.

I think you are confused about this.  TCP flow control does not work by
withholding ACKs. TCP always ACKs messages that are received, and flow
control is instead implemented by directly telling the other end
not to send more.  The sender is informed of this condition as soon as
it occurs, and will be informed that the condition has cleared in the
time it takes for a packet to transit the path between the peers (which
is why it is probably good to have a TCP window of outstanding data
at the receiver that it can process until the sender gets the
message to send more, and noting that the receiver is in full control
of just how much or little data that is).  It doesn't get much better
than that.

Now a sender who has had all outstanding packets ACK'd but which
is flow controlled off knows precisely why this has occurred; it
has been sending updates faster than the receiver was willing to
process them.  There is no ambiguity, this is the only way this
situation could occur.

Therefore, out of everything you said above, the only thing I can
see which is perfectly accurate is that the flow-controlled sender
won't know how long it will be flow-controlled off.  This will depend
on what other things the receiving router has to do which it finds
more important than processing BGP messages from this particular
peer.  It is hard to see how the receiver could know how long that
is going to take until it actually does it, however, so it seems
that telling the sender this would require the receiver to be able
to predict the future (the sender knows all about the past already
so there isn't much use having the receiver tell the sender about
that).  I can't dispute that both prescient routers and fortune tellers
might be useful, but I'm not sure that either is a practical
possibility in real life.

While I doubt the implementability of the future-prediction you
seem to be looking for here, however, I think there is another
more fundamental issue which says your approach is incorrect.  That
is, TCP flow control already puts the congested receiver in full
control of its load.  No one can send data faster than the congested
receiver is willing to process it.  The congested receiver directly
controls how much data is outstanding when flow control is asserted.
The sender can't do anything to the receiver that the receiver doesn't
explicitly permit.  With TCP flow control the congested receiver
has full control of its resources.  If there is any problem at all
here, then, it is that the sender has no control at all over the
receiver's resources and hence may be inconvenienced by a congested
receiver which chooses to take its time responding.  And, in fact,
if we're talking about Capability messages in particular, it is of
no advantage in general at all to the busy receiver to get this
message any earlier than updates or anything else; it doesn't even know
the message is coming until it sees it.  Only the uncongested sender
knows there is an outstanding capability message.

So if there is a problem here at all it is a problem for the
idle sender, which might have to wait for a response to a Capability
message, rather than the busy receiver.  Yet your proposed solution
for this is to have the busy receiver do yet more work, continually
predicting the future and sending Inform messages about it to all of
its (possibly many) peers, on the off chance that a sender which
might someday have a Capability message to send, and which has CPU
cycles to burn, isn't inconvenienced.  Since it is the guy who has
CPU cycles available which has the problem in the first place it seems
utterly backwards to make the guy who has no CPU cycles free do more
work to fix it.  The guy with the CPU cycles should be the one to fix
this, i.e. the sender should spend his free cycles doing whatever it
takes to wait until the congested guy gets around to responding.  Then
everything works fine with no protocol changes.

> > In BGP a receiver can choose to accept a anyrate of updates 
> it wants.
> 
> Yes, it can by reading or not reading the TCP socket queue. 
> Which gets us
> back to the original point I mentioned earlier. TCP FLOW 
> Control is not
> perfect.

Your original point seems to have been based on a faulty understanding
of how TCP flow control works.

> > Please do offer more to substantiate this other than your claim that
> > the problem exists.
> >
>  
> By the fact that the pacing control is more direct and implicit to a
> particular
> traffic type (in this case update messages). The pacing mechanism is
> deterministic, whereas TCP flow control is not. You should talk to people
> who build IPC mechanisms to replace TCP about this issue. 

Or the guys that built deterministic FDDI or Token Ring or Token Bus
or ATM LANs to replace non-deterministic Ethernets.  We have networks
where non-deterministic stuff just happens despite our best efforts to
avoid it, so you might be better off to write code which deals with
that rather than depending on some other box to deterministically
predict the future so you aren't inconvenienced.  While you are correct
that this would be easier for your box if other routers could predict
their future for you with 100% reliability, I don't think this is
likely to be possible in any case.

Dennis Ferguson

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'M. ELK'" <[email-masked]>, [email-masked], [email-masked]
Cc: [email-masked]
Subject: RE: My BGP Route Update Pacing Draft
Date: Tue, 23 Jul 2002 11:02:14 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C23259.EE331B00
Content-Type: text/plain;
	charset="iso-8859-1"

Brgds:

   for case 1 : 

Could the graceful restart mechanism be a solution to this pblm . 

Say R1 is peering with N peer . R1 is under stress 

R1 select to close 1..m  TCP session with peer 1 to m , deal with the
remaining  session (m+1 ...N) and clear his congestion (now R1 have 

enough HP ) . R1 initiae a session to peer 1 ...m with restart bit set and f
bit set  .  

Closing down a peer session just because you cannot handle its load is a bad
idea.  And then using some

gracefull session restart mechanism to control the load. I dont see how that
will work efficiently and keep routes up to date?

 Here we assume that R1 could sort his stress within a restart time value.

  What if the stress does not go away after a time period?

Ben

   _____  

Chat with friends online, try MSN Messenger: Click
<http://g.msn.com/1HM1ENXX/c144??PS=47575> Here

------_=_NextPart_001_01C23259.EE331B00
Content-Type: text/html;
	charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<META content="MSHTML 5.00.2920.0" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=111275814-23072002>Brgds:</SPAN></FONT></DIV>
<BLOCKQUOTE 
style="BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: 5px">
  <DIV align=left class=OutlookMessageHeader dir=ltr><FONT color=#0000ff 
  face=Arial size=2><SPAN class=111275814-23072002>&nbsp; 
  &nbsp;</SPAN></FONT>for case 1 : </DIV>
  <DIV>
  <DIV>
  <P>Could&nbsp;the graceful restart&nbsp;mechanism be a solution to this pblm . 
  </P>
  <P>Say R1 is peering with N peer . R1 is under stress </P>
  <P>R1 select to close&nbsp;1..m&nbsp; TCP session with peer 1 to m&nbsp;, deal 
  with the remaining &nbsp;session (m+1 ...N)&nbsp;and&nbsp;clear his congestion 
  (now R1 have </P>
  <P>enough HP ) . R1 initiae a session to peer&nbsp;1 ...m with restart bit set 
  and f bit set&nbsp; .&nbsp;<FONT color=#0000ff face=Arial size=2><SPAN 
  class=111275814-23072002>&nbsp;</SPAN></FONT><FONT color=#0000ff face=Arial 
  size=2><SPAN class=111275814-23072002></P>
  <P><FONT color=#0000ff face=Arial size=2><SPAN 
  class=111275814-23072002>Closing down a peer session just because you cannot 
  handle its load is a bad idea.&nbsp; And then using some</SPAN></FONT></P>
  <P><FONT color=#0000ff face=Arial size=2><SPAN 
  class=111275814-23072002>gracefull session restart mechanism to control the 
  load. I dont see how that will work efficiently and keep routes up to 
  date?</SPAN></FONT></P></SPAN></FONT>
  <P>&nbsp;Here we assume that R1 could sort his stress within a restart time 
  value.</P>
  <P><FONT color=#0000ff face=Arial size=2><SPAN 
  class=111275814-23072002>&nbsp;&nbsp;What if the stress does not go away after 
  a time period?</SPAN></FONT></P>
  <P><FONT color=#0000ff face=Arial size=2><SPAN 
  class=111275814-23072002>Ben</SPAN></FONT></P></DIV></DIV><BR clear=all>
  <HR>
  Chat with friends online, try MSN Messenger: <A 
  href="http://g.msn.com/1HM1ENXX/c144??PS=47575">Click 
Here</A><BR></BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C23259.EE331B00--

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Padmaja kotamarthiC'" <[email-masked]>, [email-masked], "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: RE: 
Date: Tue, 23 Jul 2002 10:47:36 -0400
Sender: [email-masked]

Sri:
  How would you handle many adds and withdrawls of same routes. Would that
not cause
the effect of alot more update messages?

Say we get into route oscillations related problems, would we see alot of
updates in a given period of time?

Ben

> From: Padmaja kotamarthiC [mailto:[email-masked]]
> Sent: Tuesday, July 23, 2002 10:35 AM
> To: [email-masked]; [email-masked]; [email-masked]
> Subject: 
> 
> 
> Hi Manav,
>         If your peer is congested for whatever reason
> the simple solution is to: set the peer prefix limit
> to the appropriate value to avoid the problem. 
> 
> sri
> 
> Hi Ben,
> I have some thoughts on the draft.
> 
> The crux of your draft is that if a peer gets so
> congested that it is not
> able to send any UPDATE/KEEPALIVE message to the
> remote uncongested peer
> then the session can break down because of the Hold
> Timer expiring and it
> can be prevented by sending some pacing information
> which will inform the
> uncongested peer to limit its route advertisement
> rate. The PACING
> information will too be carried in the TCP packets, so
> if the TCP is
> implementing flow control and is not sending packets
> to avoid congestion
> then how will the PACING information be sent? And if 
> TCP can send the
> PACING information then it can as well send the
> KEEPALIVE packet which will
> keep the HoldTimer from expiring at the other end.
> 
> Secondly, if you are aware that your router is old and
> might come in a
> situation when it might not be able to handle the full
> feed of BGP Updates
> from the remote end then it can negotiate a HoldTime
> of zero and keep the
> session running even when it is in the congested
> state.
> 
> Thirdly, the very idea of artificially impeding
> reachability information
> (or sending of BGP UPDATE messages) doesn't really
> appeal to me. I was
> initially against the idea of using
> MinRouteAdvertisementInternal etc
> timers in my implementation arguing that a router
> should pack off any
> reachability information it has as soon as it comes to
> know of it. But this
> created a problem when i got say some 80K routes. I
> would then be sending
> 80K packets to my other peers! Finally , better sense
> prevailed on me and i
> started using the timers for efficiently packing my
> NLRIs in the UPDATE
> messages. This is one point where i hold my
> reachability information from
> my peers & I discourage it doing any other time. IMHO
> i feel we should let
> TCP take care of withholding any data it wants with
> itself till the
> congestion state rides by rather than us interfering
> with it.
> 
> What happens say when an uncongested router receives
> pacing information
> telling it to reduce the advertisement rate by some
> 70%. In that case there
> will be many UPDATEs lying with it remaining to be yet
> announced to the
> remote peer which is experiencing congestion. Now what
> if the uncongested
> network receives withdrawals (both implicit and
> explicit) for certain
> prefixes which have already been announced or remain
> yet to be announced?
> You open up a can of worms now !!
> 
> You have to now flush all your output buffers
> regardless of the congestion
> state your remote peer lies in!
> 
> Thus you end up doing what you always wanted to avoid.
> 
> I think a cleaner and a much simpler approach would be
> to dynamically
> negotiate the value of the HoldTimer during the
> session. If you feel you
> are getting congested then increase the value and once
> the red flags are
> all down you can revert back to your original value or
> decrease it back
> again.
> 
> Regards,
> Manav
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ----- Original Message -----
> From: "Abarbanel, Benjamin"
> <[email-masked]>
> To: <[email-masked]>
> Sent: Friday, July 19, 2002 1:01 AM
> Subject: My BGP Route Update Pacing Draft
> 
> 
> | Hi all:
> |
> |    Our recent discussions on this list and my recent
> work
> |  experiences have led me to write this draft and
> offer it
> |  to the IETF community. I would appreciate any
> comments
> |  anyone has to make.
> |
> |  Thanks in advance,
> |  Ben
> |
> |
> |
> |
> 
> 
> 
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Health - Feel better, live better
> http://health.yahoo.com
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Manav Bhatia'" <[email-masked]>, Mareline Sheldon <[email-masked]>
Cc: [email-masked]
Subject: RE: My BGP Route Update Pacing Draft
Date: Tue, 23 Jul 2002 10:44:50 -0400
Sender: [email-masked]

Manav:

Response below.

> From: Manav Bhatia [mailto:[email-masked]]
> Sent: Tuesday, July 23, 2002 8:02 AM
> To: Mareline Sheldon
> Cc: [email-masked]
> Subject: Re: My BGP Route Update Pacing Draft
> 
> 
> Mareline,
> 
> | | What happens say when an uncongested router receives 
> pacing information
> | | telling it to reduce the advertisement rate by some 70%. 
> In that case
> there
> | | will be many UPDATEs lying with it remaining to be yet 
> announced to the
> | | remote peer which is experiencing congestion. Now what if the
> uncongested
> | | network receives withdrawals (both implicit and explicit) 
> for certain
> | | prefixes which have already been announced or remain yet to be
> announced?
> | | You open up a can of worms now !!
> | |
> | | You have to now flush all your output buffers regardless of the
> congestion
> | | state your remote peer lies in!
> |
> | what exactly do you mean by this?
> 
> If you have some UPDATEs to announce and if you are holding them back
> because of the congestion you perceive in the network and then if you
> recieve a WITHDRAW for one of those same prefixes then you 
> have to first
> advertise the UPDATEs because it is invalid to announce the 
> same address
> prefix in the WITHDRAWN routes and the NLRI field of an 
> UPDATE message. You
> thus end up flushing all the BGP UPDATEs you had piled up 
> waiting for the
> congestion to clear out at your remote end. The better 
> solution is to let
> TCP take care of the congestion internally as it has always 
> been OR if you
> want to dynamically adjust to the congestion then you may 
> re-negotiate the
> HoldTimer using the Dynamic Capabilities.
> 
> I hope i make myself clear.
> 
Manav:

Whether you use the pacing method or TCP flow control the updates will be
roadblocked in the source router and the end result will be the same. 
Remember TCP in the source router will only hold a window full of packets
and the rest of the UPDATE routes will be left in the RIB-OUT area of BGP.
Unless you
have implementation that moves 100,000+ routes into some transmit queue that
sits
above TCP. In which case, I doubt you would update this queue of update
messages
when new withdrawls come in affecting those routes.

So if your implementation leaves it in the RIB-out, the new Route withdrawls
coming in the source router could cause updates that have not made it to the
destination router to be affected. 

So there is no difference between the two mechanism in this case.

Ben

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Manav Bhatia'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: RE: My BGP Route Update Pacing Draft
Date: Tue, 23 Jul 2002 10:43:55 -0400
Sender: [email-masked]

Manav:

> From: Manav Bhatia [mailto:[email-masked]]
> Sent: Tuesday, July 23, 2002 12:12 AM
> To: Abarbanel, Benjamin; [email-masked]
> Subject: Re: My BGP Route Update Pacing Draft
> 
> 
> Hi Ben,
> I have some thoughts on the draft.
> 
> The crux of your draft is that if a peer gets so congested 
> that it is not
> able to send any UPDATE/KEEPALIVE message to the remote 
> uncongested peer
> then the session can break down because of the Hold Timer 
> expiring and it
> can be prevented by sending some pacing information which 
> will inform the
> uncongested peer to limit its route advertisement rate. The PACING
> information will too be carried in the TCP packets, so if the TCP is
> implementing flow control and is not sending packets to avoid 
> congestion
> then how will the PACING information be sent? And if  TCP can send the
> PACING information then it can as well send the KEEPALIVE 
> packet which will
> keep the HoldTimer from expiring at the other end.
> 
The assumption is that pacing is keeping the congestion low enough by
throttling the rate at which the source peer is transmitting to the local
peer to be
able to send more pacing messages back to the source router. When the router
is totally congested, its too late and even pacing messages will fail to
reach back to the source router. You would say you can achieve the same
control of the incoming stream by artificially limiting the rate at which
you
read the BGP TCP session socket. The only difference between the two
mechanism is
one is conveyed end to end between the BGP peers (via pacing) the other is
done
indirectly via the TCP end points being controlled via the socket reads. I
think
the Pacing method allows one router to send more detail information about
its 
congestion state then a simple flow on/off condition that is provided via
TCP.
The pacing mechanism is more deterministic than the simple TCP flow control
mechanism
which consumes alot more memory resources when flow control is activated.
 
Maybe now we dont need such fancy capabilities, but tomorrow is another day.

> Secondly, if you are aware that your router is old and might come in a
> situation when it might not be able to handle the full feed 
> of BGP Updates
> from the remote end then it can negotiate a HoldTime of zero 
> and keep the
> session running even when it is in the congested state.
>

If you have a zero hold timer, you would have extended period of time where
routes are not current with topological states. The reason we use the hold 
timer is in the case that the remote peer is not talking without a session
disconnect.
This time period is the most a given timer will tolerate before recomputing
alternate
paths for that router's routes. I dont believe that setting this timer off
will
help an old congested peer. It will make the topology highly unstable and
cause
possible black holes via the unresponsive router.
 
> Thirdly, the very idea of artificially impeding reachability 
> information
> (or sending of BGP UPDATE messages) doesn't really appeal to me. I was
> initially against the idea of using MinRouteAdvertisementInternal etc
> timers in my implementation arguing that a router should pack off any
> reachability information it has as soon as it comes to know 
> of it. But this
> created a problem when i got say some 80K routes. I would 
> then be sending
> 80K packets to my other peers! Finally , better sense 
> prevailed on me and i
> started using the timers for efficiently packing my NLRIs in 
> the UPDATE
> messages. This is one point where i hold my reachability 
> information from
> my peers & I discourage it doing any other time. IMHO i feel 
> we should let
> TCP take care of withholding any data it wants with itself till the
> congestion state rides by rather than us interfering with it.
> 
> What happens say when an uncongested router receives pacing 
> information
> telling it to reduce the advertisement rate by some 70%. In 
> that case there
> will be many UPDATEs lying with it remaining to be yet 
> announced to the
> remote peer which is experiencing congestion. Now what if the 
> uncongested
> network receives withdrawals (both implicit and explicit) for certain
> prefixes which have already been announced or remain yet to 
> be announced?
> You open up a can of worms now !!
> 
> You have to now flush all your output buffers regardless of 
> the congestion
> state your remote peer lies in!
> 
> Thus you end up doing what you always wanted to avoid.
> 
> I think a cleaner and a much simpler approach would be to dynamically
> negotiate the value of the HoldTimer during the session. If 
> you feel you
> are getting congested then increase the value and once the 
> red flags are
> all down you can revert back to your original value or 
> decrease it back
> again.
> 
Changing the hold timer does not reduce the congestion imposed on the
current
router by its peers. It only extends the time it takes for the session to
disconnect.

My draft was not attempting to treat session disconnects as the main issue
but reduce congestion level on a given router.  

Ben

P.S. Since there is no interest in this draft I have withdrawn it from IETF.

> 
> 
> 
> 
> 
> ----- Original Message -----
> From: "Abarbanel, Benjamin" <[email-masked]>
> To: <[email-masked]>
> Sent: Friday, July 19, 2002 1:01 AM
> Subject: My BGP Route Update Pacing Draft
> 
> 
> | Hi all:
> |
> |    Our recent discussions on this list and my recent work
> |  experiences have led me to write this draft and offer it
> |  to the IETF community. I would appreciate any comments
> |  anyone has to make.
> |
> |  Thanks in advance,
> |  Ben
> |
> |
> |
> |
> 

From: "M. ELK" <[email-masked]>
To: [email-masked], [email-masked]
Cc: [email-masked]
Subject: Re: My BGP Route Update Pacing Draft
Date: Tue, 23 Jul 2002 13:59:37 +0000
Sender: [email-masked]

<html><div style='background-color:'><DIV>
<P><BR>Manav / Marelines </P>
<P>May be it will be clear if we define what pbm this draft is trying to</P>
<P>solve . </P>
<P>1) R1 could be&nbsp;under stress occasionaly </P>
<P>or </P>
<P>2)R1 have a low HP (relative to his location in the netw ) and R1 will be </P>
<P>in this situation very often .</P>
<P>&nbsp;</P>
<P>i guess solving&nbsp;case 2 is not appropriate&nbsp;by a protocol change ,also we</P>
<P>are pushing the purden of&nbsp; R1 weakness to his peers . </P>
<P>&nbsp;</P>
<P>for case 1 : </P>
<P>Could&nbsp;the graceful restart&nbsp;mechanism be a solution to this pblm . </P>
<P>Say R1 is peering with N peer . R1 is under stress </P>
<P>R1 select to close&nbsp;1..m&nbsp; TCP session with peer 1 to m&nbsp;, deal with the remaining &nbsp;session (m+1 ...N)&nbsp;and&nbsp;clear his congestion (now R1 have </P>
<P>enough HP ) . R1 initiae a session to peer&nbsp;1 ...m with restart bit set and f bit set&nbsp; . </P>
<P>&nbsp;Here we assume that R1 could sort his stress within a restart time value.</P>
<P>Brgds </P>
<P>&nbsp;</P></DIV></div><br clear=all><hr>Chat with friends online, try MSN Messenger: <a href='http://g.msn.com/1HM1ENXX/c144??PS=47575'>Click Here</a><br></html>

Date: Tue, 23 Jul 2002 05:24:31 -0700 (PDT)
From: vrishab sikand <[email-masked]>
Subject: Re: consult :  What is the FTP User ID and Password
To: lidefeng <[email-masked]>, [email-masked]
Sender: [email-masked]

--0-1183507891-1027427071=:79155
Content-Type: text/plain; charset=us-ascii

The archive has not been acessable for quite some time. Either this should be fixed or the link removed from IDR home page.
  lidefeng <[email-masked]> wrote: 
----- Original Message ----- 
From: "lidefeng" 

To: 
Sent: Tuesday, July 23, 2002 11:22 AM
Subject: consult : What is the FTP User ID and Password

> Dear Sir,
> 
> In the Mailing List of Work Group, listed following FTP directory for us to get the relevant archives:
> 
> Archive: ftp://ftp.merit.edu/mail.archives/idr
> 
> But I don't know the FTP User ID and the Paddword,so I can't get the document. Could you please tell me ?
> Thank you very much!
> 
> Li Defeng
> 

---------------------------------
Do You Yahoo!?
Yahoo! Health - Feel better, live better
--0-1183507891-1027427071=:79155
Content-Type: text/html; charset=us-ascii

<P>The archive has not been acessable for quite some time. Either this should be fixed or the link removed from IDR home page.
<P>&nbsp; <B><I>lidefeng &lt;[email-masked]&gt;</I></B> wrote: 
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid"><BR>----- Original Message ----- <BR>From: "lidefeng" <[email-masked]><BR>To: <[email-masked]><BR>Sent: Tuesday, July 23, 2002 11:22 AM<BR>Subject: consult : What is the FTP User ID and Password<BR><BR><BR>&gt; Dear Sir,<BR>&gt; <BR>&gt; In the Mailing List of Work Group, listed following FTP directory for us to get the relevant archives:<BR>&gt; <BR>&gt; Archive: ftp://ftp.merit.edu/mail.archives/idr<BR>&gt; <BR>&gt; But I don't know the FTP User ID and the Paddword,so I can't get the document. Could you please tell me ?<BR>&gt; Thank you very much!<BR>&gt; <BR>&gt; Li Defeng<BR>&gt; </BLOCKQUOTE><p><br><hr size=1><b>Do You Yahoo!?</b><br>
<a href="http://health.yahoo.com/">Yahoo! Health</a> - Feel better, live better
--0-1183507891-1027427071=:79155--

Date: Tue, 23 Jul 2002 17:31:55 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: Mareline Sheldon <[email-masked]>
Cc: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Mareline,

| | What happens say when an uncongested router receives pacing information
| | telling it to reduce the advertisement rate by some 70%. In that case
there
| | will be many UPDATEs lying with it remaining to be yet announced to the
| | remote peer which is experiencing congestion. Now what if the
uncongested
| | network receives withdrawals (both implicit and explicit) for certain
| | prefixes which have already been announced or remain yet to be
announced?
| | You open up a can of worms now !!
| |
| | You have to now flush all your output buffers regardless of the
congestion
| | state your remote peer lies in!
|
| what exactly do you mean by this?

If you have some UPDATEs to announce and if you are holding them back
because of the congestion you perceive in the network and then if you
recieve a WITHDRAW for one of those same prefixes then you have to first
advertise the UPDATEs because it is invalid to announce the same address
prefix in the WITHDRAWN routes and the NLRI field of an UPDATE message. You
thus end up flushing all the BGP UPDATEs you had piled up waiting for the
congestion to clear out at your remote end. The better solution is to let
TCP take care of the congestion internally as it has always been OR if you
want to dynamically adjust to the congestion then you may re-negotiate the
HoldTimer using the Dynamic Capabilities.

I hope i make myself clear.

Regards,
Manav

Date: Tue, 23 Jul 2002 04:52:05 -0700 (PDT)
From: Mareline Sheldon <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: [email-masked], [email-masked]
Sender: [email-masked]

Hi Manav,
| 
| What happens say when an uncongested router receives pacing information
| telling it to reduce the advertisement rate by some 70%. In that case there
| will be many UPDATEs lying with it remaining to be yet announced to the
| remote peer which is experiencing congestion. Now what if the uncongested
| network receives withdrawals (both implicit and explicit) for certain
| prefixes which have already been announced or remain yet to be announced?
| You open up a can of worms now !!
| 
| You have to now flush all your output buffers regardless of the congestion
| state your remote peer lies in!

what exactly do you mean by this?

Regards,
mareline s.

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Date: Tue, 23 Jul 2002 09:42:10 +0530
From: Manav Bhatia <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Hi Ben,
I have some thoughts on the draft.

The crux of your draft is that if a peer gets so congested that it is not
able to send any UPDATE/KEEPALIVE message to the remote uncongested peer
then the session can break down because of the Hold Timer expiring and it
can be prevented by sending some pacing information which will inform the
uncongested peer to limit its route advertisement rate. The PACING
information will too be carried in the TCP packets, so if the TCP is
implementing flow control and is not sending packets to avoid congestion
then how will the PACING information be sent? And if  TCP can send the
PACING information then it can as well send the KEEPALIVE packet which will
keep the HoldTimer from expiring at the other end.

Secondly, if you are aware that your router is old and might come in a
situation when it might not be able to handle the full feed of BGP Updates
from the remote end then it can negotiate a HoldTime of zero and keep the
session running even when it is in the congested state.

Thirdly, the very idea of artificially impeding reachability information
(or sending of BGP UPDATE messages) doesn't really appeal to me. I was
initially against the idea of using MinRouteAdvertisementInternal etc
timers in my implementation arguing that a router should pack off any
reachability information it has as soon as it comes to know of it. But this
created a problem when i got say some 80K routes. I would then be sending
80K packets to my other peers! Finally , better sense prevailed on me and i
started using the timers for efficiently packing my NLRIs in the UPDATE
messages. This is one point where i hold my reachability information from
my peers & I discourage it doing any other time. IMHO i feel we should let
TCP take care of withholding any data it wants with itself till the
congestion state rides by rather than us interfering with it.

What happens say when an uncongested router receives pacing information
telling it to reduce the advertisement rate by some 70%. In that case there
will be many UPDATEs lying with it remaining to be yet announced to the
remote peer which is experiencing congestion. Now what if the uncongested
network receives withdrawals (both implicit and explicit) for certain
prefixes which have already been announced or remain yet to be announced?
You open up a can of worms now !!

You have to now flush all your output buffers regardless of the congestion
state your remote peer lies in!

Thus you end up doing what you always wanted to avoid.

I think a cleaner and a much simpler approach would be to dynamically
negotiate the value of the HoldTimer during the session. If you feel you
are getting congested then increase the value and once the red flags are
all down you can revert back to your original value or decrease it back
again.

Regards,
Manav

----- Original Message -----
From: "Abarbanel, Benjamin" <[email-masked]>
To: <[email-masked]>
Sent: Friday, July 19, 2002 1:01 AM
Subject: My BGP Route Update Pacing Draft

| Hi all:
|
|    Our recent discussions on this list and my recent work
|  experiences have led me to write this draft and offer it
|  to the IETF community. I would appreciate any comments
|  anyone has to make.
|
|  Thanks in advance,
|  Ben
|
|
|
|

Date: Tue, 23 Jul 2002 11:25:08 +0800
From: lidefeng <[email-masked]>
Subject: Re: consult :  What is the FTP User ID and Password
To: [email-masked]
Sender: [email-masked]

----- Original Message ----- 
From: "lidefeng" <[email-masked]>
To: <[email-masked]>
Sent: Tuesday, July 23, 2002 11:22 AM
Subject: consult : What is the FTP User ID and Password

> Dear Sir,
> 
>     In the Mailing List of  Work Group, listed following FTP directory for us to get the relevant archives:
> 
> Archive: ftp://ftp.merit.edu/mail.archives/idr
> 
> But I don't know the FTP User ID and the Paddword,so I can't get the document. Could you please tell me ?
> Thank you very much!
> 
>                 Li Defeng
> 

Date: Tue, 23 Jul 2002 11:22:00 +0800
From: lidefeng <[email-masked]>
Subject: consult :  What is the FTP User ID and Password
To: [email-masked]
Sender: [email-masked]

Dear Sir,

    In the Mailing List of  Work Group, listed following FTP directory for us to get the relevant archives:

Archive: ftp://ftp.merit.edu/mail.archives/idr

But I don't know the FTP User ID and the Paddword,so I can't get the document. Could you please tell me ?
Thank you very much!

                Li Defeng

Date: Mon, 22 Jul 2002 18:13:42 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
To: David Ward <[email-masked]>
Cc: [email-masked]
Subject: Re: IDR WG minutes IETF 54
Sender: [email-masked]

David,

  Please see some corrections below:

> II. Alvaro Retana presenter:
...
> QUESTION: Will the number of oscillating routes increase in the future?
> ANSWER: Potentially

> QUESTION (Alex Zinin): What about detection mechanism?
> ANSWER: YTBD

I think it was probably someone else. I asked the presenters
to please summarize the discussion on the list, explaining
what issues have been raised, how they have been addressed and
what the remaining ones are.

>         draft-walton-bgp-add-paths-00.txt
...
>         4. problems in 3107
>                 a. how to send multipath and encoding doesn't match text
>                 b. proposal is to change 3107 encoding to use new proposal
>                 NOTE: PPVPN owns 3107

Add please:

Alex: If we have a problem in 3107, maybe we should fix it
Sue : 3107 is not in IDR

> III. Sue Hares presenter
...
>         4. How to proceed?
>                 a. vote for adoption

I would change "vote" here to something like "show of hands to get the
feeling of the room" :)

>                         i. no wilfong
>                         ii. some walton
>                         iii. few Chen

Might be useful to note the approximate number of hands
for ii and iii. If I recall correctly it was not more than
half a dozen for each.

Add here:

Sue : Need to take this to the list, as there's no clear
      consensus.

Thanks.
Alex

To: David Ward <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: IDR WG minutes IETF 54 
Date: Mon, 22 Jul 2002 15:45:33 -0700
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Hi, David:
Some comments are in-lined.

> Message-Id: <p05111a00b95d0a37ad7b@[133.93.77.124]>
> Date: Thu, 18 Jul 2002 19:45:20 -0500
> To: [email-masked]
> From: David Ward <[email-masked]>
> Subject: IDR WG minutes IETF 54
> Cc: [email-masked]
> Content-Type: text/plain; charset="us-ascii" ; format="flowed"
> Sender: [email-masked]
> Precedence: bulk
> 
> NOTES: IDR WG meeting IETF 54
> Scribe: David Ward
> 
> Agenda
> 	Route Oscillation
> 	Inform message type
> 	Extended Communities
> 	Document Status
> 
> Route Oscillation
> I.	Anindya Basu Lucent presenter
> 	draft-wilfong-ibgp-oscillations-00.txt
> 
> 	1. Brief Overview of Problem (Route Reflection example)
> 	2. Overview of Transient Route Oscillations
> 		a. proposal converges to stable situation
> 			i. potential problem in scenario where there 
> could be an
> 			non-deterministic outcome
> 	3. Solution
> 		a. send xtra info
> 		b. if same aspath length, same loc_pref, diff AS: 
> send both prefixes
> 	4. Correctness
> 		a. always converges
> 		b. loop free
> 		c. works for multi level RR and confeds
> 	5. Scaling
> 		a. don't need to always send xtra routes
> 		b. only advert when detect oscillations
> 			i. propose using low-pass filter to weed out 
> restarts and detect
> 			high frequency osc.
> 	6. Issues seen in other drafts
> 		a. Walton
> 			i. doesn't solve transient and not all 
> persistent oscillations

> 		b. Chen
> 			i. believes all clients must be fully meshed

Actually only clients at the same level need to be fully meshed to
guaratee route selection consistency. (We will make this clearer in the
next revision.)

Is this a real issue considering that is how many networks are
configured?

> 
> QUESTION:
> 	How to detect oscillation?
> ANSWER:
> 	Add a counter to each entry in FIB and a timer and use 
> low-pass filter to determine if osc.
> NOTE from author: If want to scale must fine some measure to detect oscillation
> 
> 
> II. Alvaro Retana presenter:
> 	draft-walton-bgp-route-oscillation-stop-00.txt
> 
> 
> 	1. Brief Overview of Problem
> 		a. Type 1: single level RR/Confed
> 			i. solved in RFC 2796, 3065
> 		b. Type 2: multitier/hierarchical RR/Config
> 			i. must send multiple paths
> 			ii. always-compare-med considered unacceptable solution
> 		c. Oscillation stats from empirical analysis (2001 data)
> 			112086	 total routes
> 			9011	 candidates for oscillation (~10%)
> 			899           actually oscillating		(< 1%)
> 	2. Solution
> 		a. If RR or ASBR of confed
> 			i. send best path
> 			ii. send 'neighbor AS best path'
> 	3. Comparison
> 		a. Walton or Wilfong - requires oscillation detection

> 		b. Chen - not multi-level

Actually the mechanism does work with multi-level route reflection.
I did not finish this part before taking the vaction. This portion
will be revised in the next version.

> 	4. Next Steps
> 		a. determine oscillation detection mechanism - may be 
> implementation specific
> QUESTION: Will the number of oscillating routes increase in the future?
> ANSWER: Potentially
> 
> QUESTION (Alex Zinin): What about detection mechanism?
> ANSWER: YTBD
> 
> 	draft-walton-bgp-add-paths-00.txt
> 	1. This extension is required for oscillation solution
> 		a. could be adopted as an additional solution above 
> and beyond oscillation
> 	2. Proposal
> 		a. change NLRI encoding in 2858
> 			i. add bestpath bit - represents that path 
> has been selected by speaker
> 			and put into FIB
> 		b. same change to 3107 for MPLs
> 			i. additional clarifications necessary
> 		c. add new capability advertisement
> 			i. multipath capability
> 			NOTE: must not use encoding in 3107
> 	3. Soon to be released new proposal
> 		a. remove multiprotocol linkage
> 		b. change encoding to flags (1 octet) in all specs:
> 			i. 1771, 2858, 3107
> 		c. can also solve route server functionality in 
> addition to oscillation
> 	4. problems in 3107
> 		a. how to send multipath and encoding doesn't match text
> 		b. proposal is to change 3107 encoding to use new proposal
> 		NOTE: PPVPN owns 3107
> 
> III. Sue Hares presenter
> 	1. Mixed Issues w/ draft 17 -> 18 and FSM seen on list in 
> dicussion of osc. drafts
> 		a. tie breaking rules
> 			i. e.g. cluster_id
> 	2. Oscillation drafts comparison and issues
> 		a. always-compare-med not adequate sol'n
> 			i. solution today is to restrict topology
> 			i. too difficult to coordinate MED values 
> across multiple domains (asp)
> 	3. Presentation of Enke Chen's proposal
> 	draft-chen-route-oscillation-avoid-00.txt
> 		a. restrictions
> 			i. solves one level RR/Confed hierarchy
> 			ii. requires full mesh
> 		b. solutions
> 			i. announe best path rx'ed from clients
> 			ii. best path from all
> 			iii. calc IBEST and EBEST
> 			iv. announce according to rules in draft
> Statement: Naiming Sheng: Enke Chen's proposal has fewer protocol and 
> implementation changes
> 	4. How to proceed?
> 		a. vote for adoption
> 			i. no wilfong
> 			ii. some walton
> 			iii. few Chen
> Statement: Alvaro Retana
> 	What is problem we are trying to solve?
> 	Enke's doesn't solve any problem
> 	Wilfong's solves problems that are not known to exist
> 	There is a real problem as there are multilevel topologies deployed
> Statement: David Ward
> 	We know that Enke's proposal does not solve the problem in 
> the protocol specification
> 		and his proposed solution is already covered in 
> exisiting RFCs. Therefore, it should
> 		be removed from consideration.

Do not follow the comments by Alvaro and David on "does not solve
any/the problem". The essence of draft-chen-route-oscillation-avoid-00.txt
is to suggest:

    o advertising the best external path by a non-RR to IBGP (as
      in RFC 1771).

    o revising the RR behavior to remove advertisemnet dependence
      on IBGP updates.

The combination does solve the problem of route oscillation.

Admittedly the draft has some overlap with the "oscillation-reduce"
drafts, and we will consolidate the drafts in the next revision.

-- Enke

Date: Mon, 22 Jul 2002 14:34:15 -0700
From: [email-masked]
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
Sender: [email-masked]

Ben,

One thing we discussed at the meeting in Yokohama was that we MUST NOT
associate protocol actions with the INFORM message.  It is Informational
ONLY.  An INFORM may be followed by a protocol action (such as a NOTIFY),
but, in and of itself, it is not to be used to trigger protocol actions.

Andrew

On Thu, Jul 18, 2002 at 03:31:54PM -0400, Abarbanel, Benjamin wrote:
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> From: "Abarbanel, Benjamin" <[email-masked]>
> To: "[email-masked]'" <[email-masked]>
> Subject: My BGP Route Update Pacing Draft
> Date: Thu, 18 Jul 2002 15:31:54 -0400
> X-Mailer: Internet Mail Service (5.5.2650.21)
> Precedence: bulk
> X-OriginalArrivalTime: 18 Jul 2002 19:34:50.0014 (UTC) FILETIME=[2E818BE0:01C22E92]
> 
> Hi all:
> 
>    Our recent discussions on this list and my recent work 
>  experiences have led me to write this draft and offer it 
>  to the IETF community. I would appreciate any comments 
>  anyone has to make.
>  
>  Thanks in advance,
>  Ben
> 
>   
> 

> 
> 
> 
> 
> Network Working Group                                     Ben Abarbanel
> Internet Draft                                            Marconi Communicatons
> Expiration Date: December 2002                          
> 
> 
>           				
> 					BGP Route Update Pacing
> 
>                     draft-abarbanel-bgp-route-update-pacing-00.txt
> 
> 
> 1. Status of this Memo
> 
>    This document is an Internet-Draft and is in full conformance with
>    all provisions of Section 10 of RFC2026 except that the right to
>    produce derivative works is not granted.
> 
>    Internet-Drafts are working documents of the Internet Engineering
>    Task Force (IETF), its areas, and its working groups.  Note that
>    other groups may also distribute working documents as Internet-
>    Drafts.
> 
>    Internet-Drafts are draft documents valid for a maximum of six months
>    and may be updated, replaced, or obsoleted by other documents at any
>    time.  It is inappropriate to use Internet-Drafts as reference
>    material or to cite them other than as ``work in progress.''
> 
>    The list of current Internet-Drafts can be accessed at
>    http://www.ietf.org/ietf/1id-abstracts.txt
> 
>    The list of Internet-Draft Shadow Directories can be accessed at
>    http://www.ietf.org/shadow.html.
> 
> 
> 2. Abstract
> 
> This document defines a mechanism for controlling or limiting the rate at 
> which BGP update messages are sent from one peer to the next when a BGP peer 
> experiences internal congestion. With the introduction of new dynamic BGP 
> protocol capabilities [CAP] message or other none BGP session destructive 
> messages, it is necessary to limit the rate at which the BGP update messages 
> are sent without affecting the entire BGP session and without relying on the 
> transport (TCP) layer to do so as a normal reaction to data congestion of the 
> TCP session across the network.
> 
> 3. Specification of Requirements
> 
> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", 
> "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be 
> interpreted as described in RFC 2119 [RFC2119].
> 
>  
> 
> 
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 2]
> 
>  
> 
> 4. Introduction
> 
> This document defines a mechanism for controlling or limiting the rate at 
> which BGP update messages are sent from one peer to the next when a BGP peer 
> experiences internal congestion. With the introduction of new dynamic BGP 
> protocol capabilities [CAP] message or other none BGP session destructive 
> messages, it is necessary to limit the rate at which the BGP update messages 
> are sent without affecting the entire BGP session and without relying on the 
> transport (TCP) layer to do so as a normal reaction to data congestion of the 
> TCP session across the network.
> 
> When a router enters a state where either its CPU utilization is maximized 
> (reaches close to 100%) or its memory is nearly depleted (less than 10% of 
> memory left), it cannot handle new or heavy streams of updates from its peers 
> and at times unable to send messages to its peer in a timely manner (within 
> 30 seconds). As a consequence it cannot keep current with the topological 
> state. In some scenarios, a non-congested peer might want to negotiate new 
> capabilities with a congested peer. The congested peer is so degraded that 
> its TCP session goes into significant flow control off conditions and is 
> unable to see the new BGP messages. Peer to peer communication is severely 
> hampered and as a result the uncongested peer will take corrective action 
> when its hold timer expires and drops the session. The uncongested peer 
> computes alternate routing paths that are suboptimal in distance or attribute 
> and thus affect the forwarding decisions of all routers in this network. It 
> is possible that the congested peer's routing (control) plane is badly 
> degraded but its forwarding plane is at normal working level. The mistake is 
> made by the uncongested peer since it does not see any Keepalive/Update 
> messages before its Hold timer expires and drops the session thereby dropping 
> all its associated routes. After routes are dropped, network instability 
> occurs and suboptimal paths are used by the remaining peers.
> 
> The MinRouteAdvertisementInternal and MinAsOriginationInterval timers are 
> inadequate since they are mostly implemented on a peer session basis and 
> studies have shown when they are used they severely degrade route convergence 
> time. The problem with statically defined timers (initialized during system 
> load or session establishment phase) is that they do not adjust to peer 
> internal dynamically changing congestion conditions. The problem with the 
> congested peer using TCP flow control to reduce its congestion condition, is 
> that it completely stops all incoming session traffic and thus preventing any 
> messages of high priority nature from being seen.
> 
> TCP Out of Band Data or Urgent Message is one way to bypass the TCP flow 
> control condition and allow high priority BGP messages to get to the 
> congested peer, assuming it is able to read the session socket queue. This 
> solution has its drawbacks as described in [UNIX-NET] chapter 21, p. 568.  
> 
> 
> 
> 
> 
> 
> 
> 
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 3]
> 
>  
> 
> This document presents a mechanism by which the BGP session of a congested 
> router need not be degraded to the level that communication is broken with 
> its peers. By using a peer to peer pacing mechanism which allows one peer to 
> rate limit the number of update messages per second received from another 
> peer, it can avoid the severe congestion conditions and process these 
> messages at a manageable level. In addition, the uncongested peer has the 
> knowledge to send high priority messages ahead of or in place of the normal 
> high volume of update messages.
> 
> Usually, topological disturbances are spiky in nature and once they subside, 
> the network returns to its optimum path oriented level. Whatever caused the 
> network to become unstable, such as routers handling too much data in a small 
> period of time or routers loosing their sessions or their links going up and 
> down, occurs in most live network on an infrequent basis. By using the pacing 
> mechanism as outlined in this draft, a BGP peer can prevent the serious 
> congestion long before it is in trouble and thus ride through topological 
> disturbances and still regain its stability without causing its sessions to 
> drop or its routes/paths to be discarded and recomputed.
> 
> The assumption in this spec is that the source of most or all of the internal 
> BGP router congestion is due to the heavy reception of update messages from 
> neighboring peers containing large number of routes.
> 
> 
> 5. BGP Update Pacing Mechanism
> 
> The BGP Update Message Pacing Mechanism is used to slow down the rate at 
> which a peer sends update messages to another. This extension to the BGP 
> protocol is simplified to use session non-destructive messages such as INFORM 
> as described in [INFORM]. The pacing is performed dynamically upon congestion 
> detection and subsidence and thus needs to use the new [INFORM] message that 
> will not infringe on the underlying BGP protocol or all its semantics and 
> rules.
>  
> 
> 5.1 BGP Update Message Pacing (PACE) Dynamic Capability
> 
> The BGP Update Message Pacing capability is dynamically negotiated with all 
> BGP speakers as type code=PACE (where, PACE=TBD) per TLV structures as 
> defined in [BGP-CAP] of the OPEN message or anytime after session is 
> established using the Dynamic Capability Message as defined [DYN-CAP] 
> specification. All those peers that accept the PACE capability will be 
> expected to support the new INFORM message as defined in [INFORM] 
> specification to carry the pacing TLV structure. 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 4]
> 
>  
> 
> All PACE capable routers will provide a configuration option to their 
> operator to enable the BGP Update Message Pacing mechanism on a per peer 
> basis. 
> 
> Any peer wishing to withdraw the PACE capability can do so dynamically using 
> the Dynamic Capability message as outlined in [DYN-CAP] specification. Once 
> withdrawn, affected peering session will remain intact but will not benefit 
> from the performance improvements offered by the pacing mechanism.
> 
> 
> 5.2 Use of INFORM Message
> 
> The INFORM message as described in [INFORM] is used to carry the rate 
> limiting (pacing TLV) control structure to neighboring peers. This 
> information informs the peer that the current BGP router has entered a 
> congestion state and it is to rate limit its transmission to the level 
> specified. 
> 
> The INFORM message contains the following PACE TLV structure:
>  
>    Type = Pacing Information, type=TBD 
>    Length = 2
>    1st Byte of Value = Cmd as shown below 
>    2nd Byte of Value = Level as shown below
> 
>  Cmd       Description 
> -------    -----------
>   11       Request to Pace update messages
>   12       ACK Response to Pace Update messages
>   13       NACK Response to Pace Update Messages
> 
> A. Request to Pace Update Messages Level Indication (code=11)
> 
> The rate is divided into sub levels in term of categories (Gray, Yellow, 
> Orange, Red, and Green) to denote the level of pacing which is directly 
> related to the level of congestion experienced within the congested peer. The 
> colors are used as simple handles used for flagging the severity of the 
> condition. The colors are also used as indicators for display by the NMS to 
> identify the rate limiting levels from any neighboring peer to the operator. 
> Associated Level Management MIBs are defined in section 6. 
> 
> The actual pacing control is done via the sub levels within these colors. 
>   
>     Level     Description
>    ------     -----------
>     Gray      (Entering minor congestion state)
>          1       Reduce update message traffic by 10% your normal rate.
>          2       Reduce update message traffic by 20% your normal rate.
>          3       Reduce update message traffic by 30% your normal rate.
> 
> 
> 
>                                                     
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 5]
> 
> 
> 
>       Yellow     (Entering medium congestion state)
>          4       Reduce update message traffic by 40% your normal rate. 
>          5       Reduce update message traffic by 50% your normal rate.
>          6       Reduce update message traffic by 60% your normal rate.
> 
>       Orange     (Entering major congestion state)
>          7       Reduce update message traffic by 70% your normal rate. 
>          8       Reduce update message traffic by 80% your normal rate.
>          9       Reduce update message traffic by 90% your normal rate
> 
>         Red      (Entering critical congestion state)
>         10       Complete cessation of all update message traffic (flow off). 
>                  At this Level the receiving peer might decide to bypass the 
>                  Congested Router and pick another less optimal router for the
>                  affected Routes.
> 
>     Green     (Exiting any congestion state)
>          0       Restore update message traffic to your normal level (flow on).
>               Routes redirected can be resumed to the original peer.
> 
> The receiving peer will send an INFORM message with an ACK or NACK Indicating 
> it has received and understood the pacing request and will either comply with 
> the it or forbid it. If the congested peer receives a NACK, it should remove 
> that peer from its list of PACE capable peers but still maintaining its 
> session without the use of Pacing.  If the NACKing peer decides at a future 
> date to re-enable the Pacing with the local peer, it will renegotiate the 
> PACE capability with the local peer at that time.
> 
> B. Response Message Error Indication:
> 
>     Level     Description
>    ------     -----------
> 0	 ACK indication. The peer will comply with the 
>      		     Pacing level request.
> 1	 NACK indication. The peer is unable to perform Pacing or comply  
>         with the pacing level requested.
> 
> 
> 5.3 INFORM Message Response Timer
>    
> When a congested peer sends an INFORM message with a "Request to Pace Update 
> Messages (code=11)" to all peers that support the pacing feature, it will 
> also start an associated INFORM Message Response Timer. Any peer that does 
> not respond within a 30 second timeout period with either an ACK/NACK INFORM 
> Response message, its associated session will be dropped. This is done to 
> clear any PACE capable peers that are also congested to the point where their 
> communication with the local congested peer is severed.
> 
> 
>  
> 
> 
> 
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 6]
> 
>  
> 
> 5.4 Congestion Detection Within the Router
> 
> There are at least two ways congestion could be detected and measured by a 
> BGP router.
> 
> - A high CPU utilization condition 
> 
> - A Lack of available memory to accept incoming BGP messages or inability 
>   to get memory to successfully complete BGP processing. 
> 
> 5.4.1 CPU Utilization Based Level Computation
> 
> It is recommended that when the congested router detects its inability to 
> perform route calculations or accept new BGP session messages at a normal 
> rate meaning the current router's CPU utilization is more than 49%, it should 
> inform all its peers of a rate limiting level and slow them down accordingly. 
> 
> The recommended way for computing the Level, based on percent CPU 
> Utilization, is done using the following:
> 
>    Level = (((%CPU Utilization – 50) x 2) + 5) / 10).
>    Where, % CPU Utilization is a whole integer number. Use Integer math  
>    here to remove any fractional value.
> 
>   Note: If computed Level is negative, go to Green and set Level = 0. If   
>          Level = 0 send INFORM message with pacing Level = 0, implying return 
>          to normal (100%) update rate.
> 
> Level implies, whatever number of messages you transmitted per second 
> before, transmit (100 – (Level x 10))% of that now. 
> 
>   e.g. If you transmitted 100 messages/second before. A level 2 (20%) 
>        reduction will cause you to transmit 80 messages/second now.
>           Assumption is that these messages have an average size (1500 bytes).
> 
> 
> 5.4.2 Memory Allocation Based Level Computation
> 
> It is recommended that when the congested router detects its inability to 
> perform route calculations or accept new BGP session messages at a normal 
> rate meaning the current router's memory allocation is more than 69%, it 
> should inform all its peers of a rate limiting level and slow them down 
> accordingly. 
> 
> The recommended way for computing the Level, based on percent Memory 
> Allocation, is done using the following:
> 
>    Level = (((%Memory Allocated – 70) x 2) + 5) / 10).
>     Where, % Memory Allocated is a whole integer number. Use Integer math  
>     here to remove any fractional value.
> 
> 
> 
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 7]
> 
> 
> 
>    Note: If computed Level is negative, go to Green sub-group and set 
>          Level = 0. If Level = 0 send INFORM message with pacing Level = 0, 
>          implying return to normal (100%) update rate.
>  
> 
> 5.5 INFORM Message Throttling
> 
> Once the congested peer receives acknowledgement from another peer, it will 
> send a modification INFORM message with a new Level to that peer after the 
> computed pacing Level changes by at least 1 value. This will amount to no 
> more than one INFORM modification message every 5 seconds. This is done to 
> debounce any spiky bursts of INFORM messages to all PACE negotiated peers 
> each time the computed pacing Level changes. Depending on vendor 
> implementations, the internal utilization levels could change at the 
> Microsecond or Millisecond rate.
>   
> 
> 6. Implementation Specific Mechanisms
> 
> The Memory Allocation and CPU Utilization Level detection algorithms 
> discussed in section 5.4.1 and 5.4.2 are suggested ways one can implement 
> these solutions. However, each vendor can implement a unique Memory 
> Allocation and CPU Utilization Level detection algorithms that best suits 
> his/her needs and will not negatively impact the overall BGP Route Update 
> Pacing mechanism described in this spec. Any issues relating to internal 
> implementation algorithms are outside the scope of this document.
> 
>  
> 7. Level Indicators Management MIBs
> 
>    TBD
> 
> 
> 8. Security Considerations
> 
>    This extension to BGP does not change the underlying security issues.
> 
> 
> 9. References
>    
>    [BGP-CAP] Chandra, R., Scudder, J., "Capabilities Advertisement with 
>              BGP-4", draft-ietf-idr-rfc2842bis-02.txt
> 
>        [DYN-CAP]         Chen E., Sangli S.,  "Dynamic Capability for BGP-4", 
>             draft-ietf-idr-dynamic-cap-02.txt, October 2002.
> 
>    [INFORM] Nalawade G., Scudder J., "BGPv4 INFORM Message",
>             draft-nalawade-bgp-inform-00.txt, December 2002
> 
>    [BGP-4]  Rekhter, Y., and T. Li, "A Border Gateway Protocol 4 
>                               (BGP-4)", RFC 1771, March 1995.
> 
> 
> Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    [Page 8]
> 
> 
> 
>    [BGP-4-DRAFT] Rekhter, Y. and T. Li (editors), "A Border Gateway Protocol 4
>                           (BGP-4)", Internet Draft draft-ietf-idr-bgp4-18.txt, 
>                            January 2002.
> 
>    [UNIX-NET]  Stevens, R., "UNIX Network Programming, Vol 1", Second Edition
>                             1998 Prentice Hall, Inc.
>    
> 
> 10. Author Information
> 
>    Ben Abarbanel
>    Marconi Communications
>    1595 Spring Hill Road, 5th Floor
>    Vienna, VA 22182
>    Email: [email-masked]
> 
> 

To: "Ankur Goyal" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: RFC-2842 Encoding BGP Capabilities 
Date: Mon, 22 Jul 2002 11:28:11 -0700
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Hi, Ankur:

> Subject: RFC-2842 Encoding  BGP Capabilities
> Date: Thu, 18 Jul 2002 15:20:57 -0700
> Message-ID: <[email-masked]>
> From: "Ankur Goyal" <[email-masked]>
> To: <[email-masked]>
> Sender: [email-masked]
> Precedence: bulk
> 
> Hi,
> 
> Section 2 of RFC 2842 states that:
> 
>   This is an Optional Parameter that is used by a BGP speaker to convey
>   to its BGP peer the list of capabilities supported by the speaker.
> 
>   The parameter contains one or more triples <Capability Code,
>   Capability Length, Capability Value>
>   ...
>   <snip>
>   ...
>    A particular capability, as identified by its Capability Code, may
>    occur more than once within the Optional Parameter.
> 
> I wonder which or both of the following encoding schemes are valid for =
> advertising supported capabilities to a peer.
> 
> OP# - is the Optional Parameter number in the OPEN message
> C   - represents the optional parameter type Capability
> A   - represents the optional parameter for Authentication
> c#  - represents individual Capabilities in the form code+length+value
> 
> (1) OP1-C:c1.c2.c3.c1 + OP2-A
> (2) OP1-C:c1 + OP2-C:c2 + OP3-C:c3 + OP4-C:c1 + OP5-A
> 
> any clarification on this will be greatly appreciated.

I am not aware of the use of the "Authentication" field. Other than that,
both encoding types (1) and (2) are in use, although (1) is clearly more
more compact than (2).

-- Enke

To: "Stephen Waters" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Cooperative Route Filtering Capability 
Date: Mon, 22 Jul 2002 11:05:15 -0700
From: Enke Chen <[email-masked]>
Sender: [email-masked]

Hi, Steve:

> Subject: Cooperative Route Filtering Capability 
> Date: Sat, 20 Jul 2002 09:58:36 +0100
> Message-ID: <[email-masked]>
> From: "Stephen Waters" <[email-masked]>
> To: <[email-masked]>
> Sender: [email-masked]
> Precedence: bulk
> 
> 
> What is the correct interpretation on filtering if I receive a =
> REMOVE_ALL/IMMEDIATE?  I delete all the received filters for that =
> AFI/SAFI, but is the default filter action still DENY?

REMOVE_ALL would remove all the entries of a particular ORF type.
As a result the filter no longer exists. As there is no filter to
be applied, the action should be PERMIT instead of DENY.

This is different from having implicit DENY for an existing filter.

-- Enke

> 
> 
> (my view is that the default action should be PERMIT at all times - or =
> at least settable).
> Regards, Steve.=20

Date: Mon, 22 Jul 2002 07:38:52 -0700
From: [email-masked]
To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked]
Subject: Re: inform (was: IDR WG minutes IETF 54)
Sender: [email-masked]

Pedro,

As an operator, I think that there are a number of useful places for an 
INFORM message, as a mecahanism to pass information between BGP speakers
that does not result in a session reset.  Two that come to mind are:

1) Max-prefix.  Currently we only have the option of a hard max-prefix
limit.  With an INFORM we could create a hard and a soft limit.  When
a peer hits the soft limit, among other things, it sends them an INFORM
telling them so.  The peer could then look at the issue, and if the
route growth is legitimate, call us to ask that the limit be increased.
This would limit unnecessary peering session resets that do occur due
to normal route growth.

2) As-confed seq/set propogation.  Although, I do think the default
behavior upon receipt of a type 3/4 from a non-confed peer should be
a session reset, the fact remains that doing so is not a current option
due to the installed base (at least not as a default option).  Under 
the current behavior of existing code, routes with this inapproprate
information are ignored.  With an INFORM, we can tell a peer that
route x is being ignored because it contains an as-confed set/seq.
At least that way they know there is a problem, and there exists
an automated mechanism above and beyond trying to track down their
NOC.

I'm sure others can think of additional uses.  So long as we specify
that no protocol action is to be taken upon receipt of INFORMs, I believe
that the utility justifys implementing the functionality.

Andrew

On Thu, Jul 18, 2002 at 06:47:36PM -0700, Pedro Roque Marques wrote:
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Delivered-To: [email-masked]
> Date: Thu, 18 Jul 2002 18:47:36 -0700 (PDT)
> From: Pedro Roque Marques <[email-masked]>
> To: [email-masked]
> Subject: inform (was: IDR WG minutes IETF 54)
> In-Reply-To: <p05111a00b95d0a37ad7b@[133.93.77.124]>
> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid
> Precedence: bulk
> X-OriginalArrivalTime: 19 Jul 2002 01:49:37.0068 (UTC) FILETIME=[89D5AEC0:01C22EC6]
> 
> David Ward writes:
> 
> > INFORM message I. Gargi Nalawade presenter draft-nalawade-bgp-inform-00.txt 1.
> > Goals a. no way to tell remote
> > peer that there has been an error or event w/o use of NOTIFICATION
> > i. this resets peering session b. no change to existing
> > implementations c. limit sharing of messages between peers
> > 2. Encoding overview a. Packet format b. Event codes c. Data types
> > 3. Rx'ing the INFORM a. log it b. inform the administrator
> > (implementation specific) Question: How to use w/ NOTIFICATION?
> > Answer: Use it to inform that you are going to reset. Use
> > immediately before the NOTIFICATION but, now can add
> > PDU/ATTR/whatever to help debug.
> 
> > Question: Will this cause a change to the FSM?  Answser: No
> 
> > Questons: What is "too many routes option?" How to define?  Answer:
> > It is an implementation specific configuration
> 
> > Question: Jeff Pickering Should a peer take any specific action upon
> > reception of the INFORM?  The authors should explicitly prohibit any
> > automated action
> 
> > Statement: Andrew Lange Move forward draft
> 
> > Outcome: Sue Hares Accepted as WG item pending taking it to list
> 
> I object to making this a WG document.
> 
> I believe this mechanism is unnecessary and that no sufficient
> justification has been presented to add this to the spec. There should
> be a very compeling reason to be making changes so basic such as
> adding new messages.
> 
> A couple of comments on the details above:
> 
> > Question: How to use w/ NOTIFICATION?
> > Answer: Use it to inform that you are going to reset. Use
> > immediately before the NOTIFICATION but, now can add
> > PDU/ATTR/whatever to help debug.
> 
> How can that be of extra value given that notification allows you to
> carry variable lenght data ? Adding this message is just going to
> cause a debugging mess with all the deployed systems out there that do
> not implement it.
> 
> > Question: Jeff Pickering Should a peer take any specific action upon
> > reception of the INFORM?
> 
> Yes, reset the session, at least in the case that has been vastly
> discussed in the list which is receicept of invalid nlri. Furtunatly
> no code needs to be written to achieve that since it is the default
> action on a receipt of an invalid message code.
> 
>   Pedro.

content-class: urn:content-classes:message
Subject: Cooperative Route Filtering Capability 
Date: Sat, 20 Jul 2002 09:58:36 +0100
From: "Stephen Waters" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

What is the correct interpretation on filtering if I receive a REMOVE_ALL/IMMEDIATE?  I delete all the received filters for that AFI/SAFI, but is the default filter action still DENY?

(my view is that the default action should be PERMIT at all times - or at least settable).
Regards, Steve. 

Date: Fri, 19 Jul 2002 10:39:10 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: RFC 2545 (Link-Local)
Sender: [email-masked]

andrey movshin writes:

> Dear Sirs/Madam,

> I'm working with BGP protocol for IPv6 routing and I'm a bit
> confused with sect.3. Constructing the Next Hop field of RFC 2545.

> Could you be so kind to clarify me the following:

> The RFC 2545 "Use of BGP-4 Multiprotocol Extensions for IPv6
> Inter-Domain Routing" says:

> "The link-local address shall be included in the Next Hop field if
> and only if the BGP speaker shares a common subnet with the entity
> identified by the global IPv6 address carried in the Network Address
> of Next Hop field and the peer the route is being advertised to.

You can look at it in another way...
whenever 2 BGP speakers share a link, and the advertising router uses
the global address on the link as a nexthop address it should also
include a link local address.

> In all other cases a BGP speaker shall advertise to its peer in the
> Network Address field only the global IPv6 address of the next hop
> (the value of the Length of Network Address of Next Hop field shall
> be set to 16)".

> If I have a configuration
> 
>              Router A (AS1)            Router B (AS1)               Router C (AS2)
>              3FFE:0:0:12::2  ------>   3FFE:0:0:22::1 --------->    3FFE:0:0:22::2                 
>                                        3FFE:0:0:12::1  __               
> 						                     |
> 						                     V
>                                            Router D (AS3)
>                                            3FFE:0:0:12::3
> 
> where router A and router B belong to AS1, router C belongs to AS2
> and router D belongs to AS3.

> Router A sends UPDATE to Router B with MP_REACH_NLRI containing
> NEXT_HOP 3FFE:0:0:12::2. I expect that router C should receive
> UPDATE with MP_REACH_NLRI containing NEXT_HOP 3FFE:0:0:22:1 (only
> Global) and router D should receive UPDATE with MP_REACH_NLRI
> containing NEXT_HOP 3FFE:0:0:12::21 (Global) and Link-Local. Am I
> correct?

It seems to me that in all your examples the nexthop address
advertised is the link address... if that is true the link-local
should be included always... but i'm probably not understanding your
diagram.

  Pedro.

From: "Khanna, Vinay" <[email-masked]>
To: "'John G. Scudder'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: "BGP graceful restart" clarifications
Date: Fri, 19 Jul 2002 09:35:03 -0400
Sender: [email-masked]

At 2:16 AM -0400 7/19/02, Khanna, Vinay wrote:
>  >2) Does this draft address only planned BGP outages?
>
>It may apply to any outage.
>[Vinay]: Agreed. But in unplanned outages, ONLY the restart mechanism for
>the Restarting Speaker is applicable. The Receiving Speaker shall NOT
retain
>the routes received from the Restarting Speaker on detecting a connection
>closure; since the Forwarding State bit is 0. Thoughts?

No, the FS bit can be set, it depends on the restarting speaker.  The 
receiving speaker just takes action as given in the spec according to 
the FS bit, no additional cleverness is required. :-)
[Vinay]: Got it. Last para in section 6.1 misled me to believe so:)

>  >3) What happens when both the speakers advertise themselves as
'receiving
>>speakers' (by setting the BGP Restart State bit to FALSE)? Should usual
BGP
>>procedures as specified in draft-17 be followed in this scenario?
>
>They should both follow the procedures for "receiving speaker".
>[Vinay]: On revisiting the draft, it makes sense. The Receiving Speaker
>shall either flush the peer routes (draft-17 behaviour) OR shall mark them
>as stale routes (graceful restart behaviour) depending on the value of the
>Forwarding State bit. Thoughts?

Right.

--John

Date: Fri, 19 Jul 2002 09:16:21 -0400 (EDT)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked]
Subject: Re: inform (was: IDR WG minutes IETF 54)
Sender: [email-masked]

I think the inform message originally came from a desire to find
ways of _not_ resetting the session on simple, can be fixed
inline errors. It would provide you, as an implementor, and the
operators, based on your implementation, the option of doing
something _other_ than a session reset. What is done with the
information is clearly up to the operator and implementor....
Simple logging seems like a valuable enough thing to start with.

But simply adding the information should not be a big deal either
way.

:-)

Russ

On Thu, 18 Jul 2002, Pedro Roque Marques wrote:

> David Ward writes:
> 
> > INFORM message I. Gargi Nalawade presenter draft-nalawade-bgp-inform-00.txt 1.
> > Goals a. no way to tell remote
> > peer that there has been an error or event w/o use of NOTIFICATION
> > i. this resets peering session b. no change to existing
> > implementations c. limit sharing of messages between peers
> > 2. Encoding overview a. Packet format b. Event codes c. Data types
> > 3. Rx'ing the INFORM a. log it b. inform the administrator
> > (implementation specific) Question: How to use w/ NOTIFICATION?
> > Answer: Use it to inform that you are going to reset. Use
> > immediately before the NOTIFICATION but, now can add
> > PDU/ATTR/whatever to help debug.
> 
> > Question: Will this cause a change to the FSM?  Answser: No
> 
> > Questons: What is "too many routes option?" How to define?  Answer:
> > It is an implementation specific configuration
> 
> > Question: Jeff Pickering Should a peer take any specific action upon
> > reception of the INFORM?  The authors should explicitly prohibit any
> > automated action
> 
> > Statement: Andrew Lange Move forward draft
> 
> > Outcome: Sue Hares Accepted as WG item pending taking it to list
> 
> I object to making this a WG document.
> 
> I believe this mechanism is unnecessary and that no sufficient
> justification has been presented to add this to the spec. There should
> be a very compeling reason to be making changes so basic such as
> adding new messages.
> 
> A couple of comments on the details above:
> 
> > Question: How to use w/ NOTIFICATION?
> > Answer: Use it to inform that you are going to reset. Use
> > immediately before the NOTIFICATION but, now can add
> > PDU/ATTR/whatever to help debug.
> 
> How can that be of extra value given that notification allows you to
> carry variable lenght data ? Adding this message is just going to
> cause a debugging mess with all the deployed systems out there that do
> not implement it.
> 
> > Question: Jeff Pickering Should a peer take any specific action upon
> > reception of the INFORM?
> 
> Yes, reset the session, at least in the case that has been vastly
> discussed in the list which is receicept of invalid nlri. Furtunatly
> no code needs to be written to achieve that since it is the default
> action on a receipt of an invalid message code.
> 
>   Pedro.
> 

__________________________________
[email-masked] CCIE <>< Grace Alone

From: [email-masked]
To: [email-masked]
Subject:  RFC 2545 (Link-Local)
Date: Fri, 19 Jul 2002 07:00:06 -0600
Sender: [email-masked]

Dear Sirs/Madam,

I'm working with BGP protocol for IPv6 routing and I'm a bit confused with sect.3. Constructing the Next Hop field of RFC 2545.

Could you be so kind to clarify me the following:

The RFC 2545 "Use of BGP-4 Multiprotocol Extensions for IPv6 Inter-Domain Routing" says:

"The link-local address shall be included in the Next Hop field if and
only if the BGP speaker shares a common subnet with the entity identified by the global IPv6 address carried in the Network Address
of Next Hop field and the peer the route is being advertised to.

In all other cases a BGP speaker shall advertise to its peer in the  Network Address field only the global IPv6 address of the next hop   (the value of the Length of Network Address of Next Hop field shall be set to 16)".

If I have a configuration

             Router A (AS1)            Router B (AS1)               Router C (AS2)
             3FFE:0:0:12::2  ------>   3FFE:0:0:22::1 --------->    3FFE:0:0:22::2                 
                                       3FFE:0:0:12::1  __               
						                     |
						                     V
                                           Router D (AS3)
                                           3FFE:0:0:12::3

where router A and router B belong to AS1, router C belongs to AS2 and router D belongs to AS3.

Router A sends UPDATE to Router B with MP_REACH_NLRI containing NEXT_HOP 3FFE:0:0:12::2. I expect that router C should receive UPDATE with MP_REACH_NLRI containing NEXT_HOP 3FFE:0:0:22:1 (only Global) and router D should receive UPDATE with MP_REACH_NLRI containing NEXT_HOP  3FFE:0:0:12::21 (Global) and Link-Local. Am I correct?                

Sincerely,

Andrey Movshin
Software Development Engineer
Qosnetics Product Operation
Agilent Technologies Inc.
75 Rochester Avenue, Unit 1
Portsmouth, NH 03801
[email-masked]
603-559-5769

Date: Fri, 19 Jul 2002 08:11:23 -0400
To: "Khanna, Vinay" <[email-masked]>
From: "John G. Scudder" <[email-masked]>
Subject: RE: "BGP graceful restart" clarifications
Cc: "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "'Alex Zinin'" <[email-masked]>, "Khanna, Vinay" <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

At 2:16 AM -0400 7/19/02, Khanna, Vinay wrote:
>  >2) Does this draft address only planned BGP outages?
>
>It may apply to any outage.
>[Vinay]: Agreed. But in unplanned outages, ONLY the restart mechanism for
>the Restarting Speaker is applicable. The Receiving Speaker shall NOT retain
>the routes received from the Restarting Speaker on detecting a connection
>closure; since the Forwarding State bit is 0. Thoughts?

No, the FS bit can be set, it depends on the restarting speaker.  The 
receiving speaker just takes action as given in the spec according to 
the FS bit, no additional cleverness is required. :-)

>  >3) What happens when both the speakers advertise themselves as 'receiving
>>speakers' (by setting the BGP Restart State bit to FALSE)? Should usual BGP
>>procedures as specified in draft-17 be followed in this scenario?
>
>They should both follow the procedures for "receiving speaker".
>[Vinay]: On revisiting the draft, it makes sense. The Receiving Speaker
>shall either flush the peer routes (draft-17 behaviour) OR shall mark them
>as stale routes (graceful restart behaviour) depending on the value of the
>Forwarding State bit. Thoughts?

Right.

--John

From: "Khanna, Vinay" <[email-masked]>
To: "'Stephen Waters'" <[email-masked]>
Cc: [email-masked], "Khanna, Vinay" <[email-masked]>
Subject: RE: "BGP graceful restart" clarifications
Date: Fri, 19 Jul 2002 05:50:42 -0400
Sender: [email-masked]

Steve,

Thanks for the comments. Please find my responses inlined with [Vinay].

Regards.
Vinay:)

-----Original Message-----
From: Stephen Waters [mailto:[email-masked]]
Sent: Friday, July 19, 2002 2:38 PM
To: Khanna, Vinay
Cc: [email-masked]
Subject: RE: "BGP graceful restart" clarifications

> [Vinay]: Agreed. But in unplanned outages, ONLY the restart 
> mechanism for
> the Restarting Speaker is applicable. The Receiving Speaker 
> shall NOT retain
> the routes received from the Restarting Speaker on detecting 
> a connection
> closure; since the Forwarding State bit is 0. Thoughts?
> 

The receiving speaker should take any failure of the peer of a 
signal to enter GR processing if the peer sent the GR option.
The restarting router then has 'restart-time' to reconnect to the 
receiving speaker before the stale routes on the the receiving speaker
are deleted.
[Vinay] Agreed.

Forwarding from the receiving speaker continues. 

Note :

"It is noted that the normal BGP procedures must be followed when the
   TCP session terminates due to the sending or receiving of a BGP
   NOTIFICATION message."
[Vinay] True.

So, the 'end' of the restarting peer was messy to some degree. Whether
the cpu was planned to bounce or not should not make any difference to 
the outcome -ideally.  
[Vinay]: I dont agree here. The Receiving Speaker behaviour is different in
these scenarios. During unplanned outages, the Restarting Speaker sets the
Forwarding State bit to 0. This forces the Receiving Speaker to delete the
stale routes after the session re-establishment. This will happen even if
the session is re-established within 'Restart Time'.
>From Section 6.2
   "Once the session is re-established, if the "Forwarding State" bit for
   a specific address family is not set in the newly received Graceful
   Restart Capability, or if a specific address family is not included
   in the newly received Graceful Restart Capability, or if the Graceful
   Restart Capability isn't received in the re-established session at
   all, then Receiving Speaker shall immediately remove all the stale
   routes from the peer that it is retaining for that address family."

Regards, Steve.

content-class: urn:content-classes:message
Subject: RE: "BGP graceful restart" clarifications
Date: Fri, 19 Jul 2002 10:07:38 +0100
From: "Stephen Waters" <[email-masked]>
To: "Khanna, Vinay" <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

> [Vinay]: Agreed. But in unplanned outages, ONLY the restart 
> mechanism for
> the Restarting Speaker is applicable. The Receiving Speaker 
> shall NOT retain
> the routes received from the Restarting Speaker on detecting 
> a connection
> closure; since the Forwarding State bit is 0. Thoughts?
> 

The receiving speaker should take any failure of the peer of a 
signal to enter GR processing if the peer sent the GR option.
The restarting router then has 'restart-time' to reconnect to the 
receiving speaker before the stale routes on the the receiving speaker
are deleted.

Forwarding from the receiving speaker continues. 

Note :

"It is noted that the normal BGP procedures must be followed when the
   TCP session terminates due to the sending or receiving of a BGP
   NOTIFICATION message."

So, the 'end' of the restarting peer was messy to some degree. Whether
the cpu was planned to bounce or not should not make any difference to 
the outcome -ideally.  

Regards, Steve.

From: "Khanna, Vinay" <[email-masked]>
To: "'John G. Scudder'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "'Alex Zinin'" <[email-masked]>, "Khanna, Vinay" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: "BGP graceful restart" clarifications
Date: Fri, 19 Jul 2002 02:16:23 -0400
Sender: [email-masked]

John,

Thanks for your clarifications. Please find my responses inlined with
[Vinay].

Regards.
Vinay:)

-----Original Message-----
From: John G. Scudder [mailto:[email-masked]]
Sent: Thursday, July 18, 2002 5:20 PM
To: Khanna, Vinay
Cc: [email-masked]'; [email-masked]'; [email-masked]';
[email-masked]'; 'Alex Zinin'; Khanna, Vinay
Subject: Re: "BGP graceful restart" clarifications

At 2:15 AM -0400 7/18/02, Khanna, Vinay wrote:
>Hello Everybody,
>
>I've some doubts with respect to the graceful restart mechanisms for BGP.
>
>1) Section 6.1 says
>"When the Restarting Speaker restarts, if possible it shall retain the
>forwarding state for the BGP routes in the Loc-RIB, and shall mark them as
>stale."
>This sentence gives an impression as if the Loc-RIB is retained in certain
>cases. Why is it so? Isn't it enough if ONLY forwarding state is preserved
>on the restarting speaker? Although the stale route marking mechanism is
>necessary on the receiving side (to avoid flapping), I dont feel it's
>necessity on the restarting side. It 's enough if the route-selection
>procedure is deferred on the restarting speaker to avoid unnecessary route
>advertisements as and when routes are received from receiving speakers. Am
I
>missing something? Please clarify.

As with any routing protocol spec, the spec describes behavior.  The 
implementation is up to you.  Is it possible to produce the described 
implementation using retained FIB information only?  I would say yes.
[Vinay]: That answers my concern.

(Many implementations don't physically implement distince Loc-RIB, 
Adj-RIB-In, etc, but they're still used for description in the base 
spec.)
[Vinay]: Section 3.2, draft-17 explicitely states
   "Although the conceptual model distinguishes between Adj-RIBs-In, Loc-
   RIB, and Adj-RIBs-Out, this neither implies nor requires that an
   implementation must maintain three separate copies of the routing
   information. The choice of implementation (for example, 3 copies of
   the information vs 1 copy with pointers) is not constrained by the
   protocol."

Shouldn't a similar statement be added in the GR draft?

>2) Does this draft address only planned BGP outages?

It may apply to any outage.
[Vinay]: Agreed. But in unplanned outages, ONLY the restart mechanism for
the Restarting Speaker is applicable. The Receiving Speaker shall NOT retain
the routes received from the Restarting Speaker on detecting a connection
closure; since the Forwarding State bit is 0. Thoughts?

>Can these mechanisms be
>applied to a subset of peers too?

Sure, although one might want to be judicious about exactly how one 
did such a mixed deployment.

>3) What happens when both the speakers advertise themselves as 'receiving
>speakers' (by setting the BGP Restart State bit to FALSE)? Should usual BGP
>procedures as specified in draft-17 be followed in this scenario?

They should both follow the procedures for "receiving speaker".
[Vinay]: On revisiting the draft, it makes sense. The Receiving Speaker
shall either flush the peer routes (draft-17 behaviour) OR shall mark them
as stale routes (graceful restart behaviour) depending on the value of the
Forwarding State bit. Thoughts?

>Thanks in advance,
>Vinay:)

Regards,

--John

Date: Thu, 18 Jul 2002 18:47:36 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: [email-masked]
Subject: inform (was: IDR WG minutes IETF 54)
Sender: [email-masked]

David Ward writes:

> INFORM message I. Gargi Nalawade presenter draft-nalawade-bgp-inform-00.txt 1.
> Goals a. no way to tell remote
> peer that there has been an error or event w/o use of NOTIFICATION
> i. this resets peering session b. no change to existing
> implementations c. limit sharing of messages between peers
> 2. Encoding overview a. Packet format b. Event codes c. Data types
> 3. Rx'ing the INFORM a. log it b. inform the administrator
> (implementation specific) Question: How to use w/ NOTIFICATION?
> Answer: Use it to inform that you are going to reset. Use
> immediately before the NOTIFICATION but, now can add
> PDU/ATTR/whatever to help debug.

> Question: Will this cause a change to the FSM?  Answser: No

> Questons: What is "too many routes option?" How to define?  Answer:
> It is an implementation specific configuration

> Question: Jeff Pickering Should a peer take any specific action upon
> reception of the INFORM?  The authors should explicitly prohibit any
> automated action

> Statement: Andrew Lange Move forward draft

> Outcome: Sue Hares Accepted as WG item pending taking it to list

I object to making this a WG document.

I believe this mechanism is unnecessary and that no sufficient
justification has been presented to add this to the spec. There should
be a very compeling reason to be making changes so basic such as
adding new messages.

A couple of comments on the details above:

> Question: How to use w/ NOTIFICATION?
> Answer: Use it to inform that you are going to reset. Use
> immediately before the NOTIFICATION but, now can add
> PDU/ATTR/whatever to help debug.

How can that be of extra value given that notification allows you to
carry variable lenght data ? Adding this message is just going to
cause a debugging mess with all the deployed systems out there that do
not implement it.

> Question: Jeff Pickering Should a peer take any specific action upon
> reception of the INFORM?

Yes, reset the session, at least in the case that has been vastly
discussed in the list which is receicept of invalid nlri. Furtunatly
no code needs to be written to achieve that since it is the default
action on a receipt of an invalid message code.

  Pedro.

Date: Thu, 18 Jul 2002 19:45:20 -0500
To: [email-masked]
From: David Ward <[email-masked]>
Subject: IDR WG minutes IETF 54
Cc: [email-masked]
Sender: [email-masked]

NOTES: IDR WG meeting IETF 54
Scribe: David Ward

Agenda
	Route Oscillation
	Inform message type
	Extended Communities
	Document Status

Route Oscillation
I.	Anindya Basu Lucent presenter
	draft-wilfong-ibgp-oscillations-00.txt

	1. Brief Overview of Problem (Route Reflection example)
	2. Overview of Transient Route Oscillations
		a. proposal converges to stable situation
			i. potential problem in scenario where there 
could be an
			non-deterministic outcome
	3. Solution
		a. send xtra info
		b. if same aspath length, same loc_pref, diff AS: 
send both prefixes
	4. Correctness
		a. always converges
		b. loop free
		c. works for multi level RR and confeds
	5. Scaling
		a. don't need to always send xtra routes
		b. only advert when detect oscillations
			i. propose using low-pass filter to weed out 
restarts and detect
			high frequency osc.
	6. Issues seen in other drafts
		a. Walton
			i. doesn't solve transient and not all 
persistent oscillations
		b. Chen
			i. believes all clients must be fully meshed

QUESTION:
	How to detect oscillation?
ANSWER:
	Add a counter to each entry in FIB and a timer and use 
low-pass filter to determine if osc.
NOTE from author: If want to scale must fine some measure to detect oscillation

II. Alvaro Retana presenter:
	draft-walton-bgp-route-oscillation-stop-00.txt

	1. Brief Overview of Problem
		a. Type 1: single level RR/Confed
			i. solved in RFC 2796, 3065
		b. Type 2: multitier/hierarchical RR/Config
			i. must send multiple paths
			ii. always-compare-med considered unacceptable solution
		c. Oscillation stats from empirical analysis (2001 data)
			112086	 total routes
			9011	 candidates for oscillation (~10%)
			899           actually oscillating		(< 1%)
	2. Solution
		a. If RR or ASBR of confed
			i. send best path
			ii. send 'neighbor AS best path'
	3. Comparison
		a. Walton or Wilfong - requires oscillation detection
		b. Chen - not multi-level
	4. Next Steps
		a. determine oscillation detection mechanism - may be 
implementation specific
QUESTION: Will the number of oscillating routes increase in the future?
ANSWER: Potentially

QUESTION (Alex Zinin): What about detection mechanism?
ANSWER: YTBD

	draft-walton-bgp-add-paths-00.txt
	1. This extension is required for oscillation solution
		a. could be adopted as an additional solution above 
and beyond oscillation
	2. Proposal
		a. change NLRI encoding in 2858
			i. add bestpath bit - represents that path 
has been selected by speaker
			and put into FIB
		b. same change to 3107 for MPLs
			i. additional clarifications necessary
		c. add new capability advertisement
			i. multipath capability
			NOTE: must not use encoding in 3107
	3. Soon to be released new proposal
		a. remove multiprotocol linkage
		b. change encoding to flags (1 octet) in all specs:
			i. 1771, 2858, 3107
		c. can also solve route server functionality in 
addition to oscillation
	4. problems in 3107
		a. how to send multipath and encoding doesn't match text
		b. proposal is to change 3107 encoding to use new proposal
		NOTE: PPVPN owns 3107

III. Sue Hares presenter
	1. Mixed Issues w/ draft 17 -> 18 and FSM seen on list in 
dicussion of osc. drafts
		a. tie breaking rules
			i. e.g. cluster_id
	2. Oscillation drafts comparison and issues
		a. always-compare-med not adequate sol'n
			i. solution today is to restrict topology
			i. too difficult to coordinate MED values 
across multiple domains (asp)
	3. Presentation of Enke Chen's proposal
	draft-chen-route-oscillation-avoid-00.txt
		a. restrictions
			i. solves one level RR/Confed hierarchy
			ii. requires full mesh
		b. solutions
			i. announe best path rx'ed from clients
			ii. best path from all
			iii. calc IBEST and EBEST
			iv. announce according to rules in draft
Statement: Naiming Sheng: Enke Chen's proposal has fewer protocol and 
implementation changes
	4. How to proceed?
		a. vote for adoption
			i. no wilfong
			ii. some walton
			iii. few Chen
Statement: Alvaro Retana
	What is problem we are trying to solve?
	Enke's doesn't solve any problem
	Wilfong's solves problems that are not known to exist
	There is a real problem as there are multilevel topologies deployed
Statement: David Ward
	We know that Enke's proposal does not solve the problem in 
the protocol specification
		and his proposed solution is already covered in 
exisiting RFCs. Therefore, it should
		be removed from consideration.
Statement: Basu
	Must solve problem w/o toplogy restriction
	All possible topologies must be under consideration
	Protocol changes required
Statement: Robert Rasuk:
	Should separate add_path from oscillation solution as it 
solves other problems
Question: Sue Hares
	Should we solve the problem of only one level or multilevel 
topologies? Should we ask
	operators?
Question: Kireeti Kompella
	Is a survey of operators available? Where they forced to 
restrict topology?
Answer: Sue Hares
	In an informal survey Sue has found multilevel topologies 
deployed. Other operators
	restricted their topology but, didn't want to
Question: Amir
	If we are trying to solve a known problem why is there an issue?
Statement: David Ward
	We should move forward in working group today to adopt a 
draft or limit the numbers of
	drafts under consideration. IDR has identified and been 
working on this problem for
	over three years and still have no agreed upon solution 
leaving a large hole in the
	protocol specification. We should solve the problem that is 
known today and solve it
	fully and take on future problems as they arise. Only 
Walton's draft fits that requirement.
Statement: Basu
	Our draft does solve problems that do occur today and may 
occur. We cannot discount the
	ability to solve potential problems.
Statement: Alex Zinin
	Discussion must continue on list as there is no agreement. No 
working group action to be
	taken.

INFORM message
I. Gargi Nalawade presenter
draft-nalawade-bgp-inform-00.txt
	1. Goals
		a. no way to tell remote peer that there has been an 
error or event w/o use of
		NOTIFICATION
			i. this resets peering session
		b. no change to existing implementations
		c. limit sharing of messages between peers
	2. Encoding overview
		a. Packet format
		b. Event codes
		c. Data types
	3. Rx'ing the INFORM
		a. log it
		b. inform the administrator (implementation specific)
Question: How to use w/ NOTIFICATION?
Answer: Use it to inform that you are going to reset. Use immediately 
before the NOTIFICATION
but, now can add PDU/ATTR/whatever to help debug.

Question: Will this cause a change to the FSM?
Answser: No

Questons: What is "too many routes option?" How to define?
Answer: It is an implementation specific configuration

Question: Jeff Pickering
	Should a peer take any specific action upon reception of the INFORM?
	The authors should explicitly prohibit any automated action

Statement: Andrew Lange
	Move forward draft

Outcome: Sue Hares
	Accepted as WG item pending taking it to list

Extended Communities
1. Andrew Lange presenter
discussion on list is source of material
	1. Problem
		a. need to support IPV6
		b. could have more efficient packeing of values in data field
			i. see ptomain-bgp-redist
		c. clean for future data types
		d. support for locally defined community structures 
(types and subtypes)
	2. Improvements
		a. regular typed encoding over extended encoding
		b. value field becomes variable length
		c. type field and subtype field added
	3. Packet format
		a. see archive on list
		b. adds new base type
		c. type
		d. subtype
		e. locally defined types/subtypes
	4. Issues
		a. backward compatibility
		b. there is installed base (2547 deployments) of 
extended communities
Statement: Barry Friedman
	Make new attribute type
Question: Barry
	Does transitive mean across AS boundary?
Answer: Andrew
	Transitive means ti will cross boundary but, perhaps go no further
Queston: Geoff H.
	What is proposal to deal w/ changes in length? See historical 
issues w/ SNMP
	Recommend that the draft specifically states what to do.
		c. there are three ways to deal w/ backward compatibility
			i. there is a hack
				Only have new encoding beyond the 
first 4 codepoints (in ext.
				comm now)
				and then switch to new encoding
			ii. new attribute type
				Move forward existing ext comm work
				perhaps deprecate over time
Statement: Kireeti
	2 specs w/ new attr type is fine but, don't use word 
deprecate of orig ext. comm.
Statement: Gargi
	Don't use word deprecate as there are many installed bases

		d. Should we duplicate all ext comm into new 
attribute for IPV6?

Statement: Kireeti
	It wouuld be nice to do everything under new atttr but, have 
to create transition plan
	Need feedback from operators
Question: What happens if you have Route Target in old and new 
format? Perhaps dont' define in new attr
Answer: Andrew
	That is a transition and implementation issue but, we can add 
a statement on how to
	deal w/ duplicate information

Statement: Sue Hares
	WG action: Andrew to write new draft as new attribute and submit

Document status (Sue went too fast for me to get it all)
	1. draft 17 -> 18 based on FSM work
		a. skh wrote two new drafts
			i. statemt - state machine diagram
			ii. backoff - optional (in draft opt means optional)
Question: should there be a state machine and FSM table in draft?
Answer: Yes, there will be (concensus vote yes)
	2. V1 MIB working on security section
	3. V2 MIB going to LC
	4. various things in queue for RFC editor
	5. Various docs under discussion
	6. 2545bis
		a. MIB ready
		b. site local issues
		c. BGP ID issues
Statement: Alex Zinin
	What are consideratinos for site local addressing? Need to define:
		If needed
		How needed
		Where needed
	Alex requests WG to provide some clue on how to resolve issue in BGP.
Question: Sue Hares to Alex
	Can the IPV6 WG define site boundaries and scoping architecture?
	When will this be resolved?
Answer: Bill Fenner
	Perhaps soon. It would be good if IDR could help w/ 
architectural question if a site
	should be limited to within an AS, cross AS',etc

	7. Dynamic capability draft? Is it adequate?

Statement: Bill Fenner
	Cannot go to standards track w/ IPR statement as is. Must be 
removed to be standard

	8. Want implemenation report for Graceful Restart

	9. ORF to be discussed on mailing list

	10. Murphy's vulnerability draft to be IDR WG item

	11. Murphy's protection mechanisms to be RPSec WG item

Statement: Alex Zinin
Please help skh finish base spec. This needs to be completed for 
interoperable implementations.
Skh should not have to carry burden alone. Please help.

content-class: urn:content-classes:message
Subject: RFC-2842 Encoding  BGP Capabilities
Date: Thu, 18 Jul 2002 15:20:57 -0700
From: "Ankur Goyal" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

Hi,

Section 2 of RFC 2842 states that:

  This is an Optional Parameter that is used by a BGP speaker to convey
  to its BGP peer the list of capabilities supported by the speaker.

  The parameter contains one or more triples <Capability Code,
  Capability Length, Capability Value>
  ...
  <snip>
  ...
   A particular capability, as identified by its Capability Code, may
   occur more than once within the Optional Parameter.

I wonder which or both of the following encoding schemes are valid for advertising supported capabilities to a peer.

OP# - is the Optional Parameter number in the OPEN message
C   - represents the optional parameter type Capability
A   - represents the optional parameter for Authentication
c#  - represents individual Capabilities in the form code+length+value

(1) OP1-C:c1.c2.c3.c1 + OP2-A
(2) OP1-C:c1 + OP2-C:c2 + OP3-C:c3 + OP4-C:c1 + OP5-A

any clarification on this will be greatly appreciated.

Ankur

From: "Abarbanel, Benjamin" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: FW: My BGP Route Update Pacing Draft
Date: Thu, 18 Jul 2002 18:14:33 -0400
Sender: [email-masked]

> From: Abarbanel, Benjamin 
> Sent: Thursday, July 18, 2002 6:10 PM
> To: 'Pedro Roque Marques'; Abarbanel, Benjamin
> Subject: RE: My BGP Route Update Pacing Draft
> 
> 
> Pedro:
> 
> > From: Pedro Roque Marques [mailto:[email-masked]]
> > Sent: Thursday, July 18, 2002 5:28 PM
> > To: Abarbanel, Benjamin
> > Cc: [email-masked]'
> > Subject: My BGP Route Update Pacing Draft
> > 
> > 
> > Abarbanel, Benjamin writes:
> > 
> > 
> > Ben,
> > I'm my opinion your draft offers a much worse alternative 
> solution to
> > a problem that is already solved in a much more elegant way 
> using TCP
> > flow control mechanism.
> > 
>   I would agree only if you flow control only update packets. 
> But now we
>   introduce other packets like Capability Message, Inform 
> Message, etc.
>   they too will be flow controlled, right?
> 
>   If so, would it be a problem?
> 
> > To test the assertion above that the problem has been solved, take a
> > mature BGP implementation and congest the resources of receiver. You
> > will see that the sender will automatically throttle it's sending
> > proccess. Contrary to what you state, sessions are not dropped.
> > 
> > BGP when correctly implemented is a very resilient protocol 
> and it has
> > been widly deployed in very demanding environments.
> > 
> > I'm also including some comments inline...
> > 
> > > 2. Abstract
> > 
> > > This document defines a mechanism for controlling or limiting the
> > > rate at which BGP update messages are sent from one peer 
> to the next
> > > when a BGP peer experiences internal congestion. With the
> > > introduction of new dynamic BGP protocol capabilities 
> [CAP] message
> > > or other none BGP session destructive messages, it is necessary to
> > > limit the rate at which the BGP update messages are sent 
> > 
> > What difference does the introduction of capabilities do ?
> > 
>  Same as above.
> 
> > > without
> > > affecting the entire BGP session and without relying on the
> > > transport (TCP) layer
> > 
> > Why not relly on the transport. Flow control is after all
> > traditionally the function of the transport. What makes you think it
> > needs to be reinvented... and how is your version of flow control
> > supperior to the current one ?
> >
> My way allows the two peer to exchange more detail 
> information on the proper
> course of action during congestion, like how fast or how slow 
> to go. The 
> TCP flow control is a stop, start mechanism. Either the pipe 
> is open or closed.
> The other end that is being flow controlled does not know why 
> or by how much he
> will be flow controlled off. Remember on a long session with 
> many hops it takes significant time to communicate the flow 
> control condition by ACKing or not ACKing messages. For me 
> this is not a direct but indirect mechanism.
> In the interim, the pipe will be full of data.
> 
> With a pacing message you can add more intelligence to the 
> communication and convey directly whatever you want without 
> affecting the underlying TCP protocol.
>  
> > > to do so as a normal reaction to data
> > > congestion of the TCP session across the network.
> > 
> > [...]
> > 
> > > 4. Introduction
> > 
> > > This document defines a mechanism for controlling or limiting the
> > > rate at which BGP update messages are sent from one peer 
> to the next
> > > when a BGP peer experiences internal congestion. With the
> > > introduction of new dynamic BGP protocol capabilities 
> [CAP] message
> > > or other none BGP session destructive messages, it is necessary to
> > > limit the rate at which the BGP update messages are sent without
> > > affecting the entire BGP session and without relying on the
> > > transport (TCP) layer to do so as a normal reaction to data
> > > congestion of the TCP session across the network.
> > 
> > > When a router enters a state where either its CPU utilization is
> > > maximized (reaches close to 100%) or its memory is nearly depleted
> > > (less than 10% of memory left), it cannot handle new or heavy
> > > streams of updates from its peers and at times unable to send
> > > messages to its peer in a timely manner (within 30 seconds).
> > 
> > If a router cannot send a message for 30 seconds there is something
> > very wrong with it. It is not clear to me how your 
> mechanism prevents
> > that.
> 
> By starting the pacing early in the game you avoid the 
> implosion that might
> occur. Now, maybe you might say its an internal 
> implementation bug. I bet
> I can put the top vendor's routers under alot of update 
> stress and they will 
> implode. Especially with several hundred sessions.
> 
> 
> > 
> > > As a
> > > consequence it cannot keep current with the topological state. In
> > > some scenarios, a non-congested peer might want to negotiate new
> > > capabilities with a congested peer. The congested peer is so
> > > degraded that its TCP session goes into significant flow 
> control off
> > > conditions and is unable to see the new BGP messages. Peer to peer
> > > communication is severely hampered and as a result the uncongested
> > > peer will take corrective action when its hold timer expires and
> > > drops the session. The uncongested peer computes alternate routing
> > > paths that are suboptimal in distance or attribute and thus affect
> > > the forwarding decisions of all routers in this network. It is
> > > possible that the congested peer's routing (control) 
> plane is badly
> > > degraded but its forwarding plane is at normal working level. The
> > > mistake
> > 
> > What mistake ?
> 
> Recomputing the routes because control plane is not working 
> right. Yet, 
> forwarding plane is working perfect.
> 
> > 
> > > is made by the uncongested peer since it does not see any
> > > Keepalive/Update messages before its Hold timer expires and drops
> > > the session thereby dropping all its associated routes. 
> After routes
> > > are dropped, network instability occurs and suboptimal paths are
> > > used by the remaining peers.
> > 
> > The scenario that you present above is one where you 
> basically have a
> > peer that is totally broken. What does it matter ? A decent
> > implementation will choose to degrade it is service levels
> > gradually. It may achieve that with the current mechanisms.
> > 
> > > The MinRouteAdvertisementInternal and MinAsOriginationInterval
> > 
> > What does min advert interval have to do with the topic in
> > discussion... ? These control only the rate at which the sender may
> > attempt to generate updates, not the rate at which the 
> receiver needs
> > to accept it.
> >
> 
> True, but these timers cause the sender to slow down the rate 
> at which it
> sends the updates on an artificial basis with no regards to 
> current processing
> congestions or such. That is why I mentioned it, in case 
> someone would say
> we have these timers to control the sender why do we need 
> your solution.
>  
> > In BGP a receiver can choose to accept a anyrate of updates 
> it wants.
> >
> 
> Yes, it can by reading or not reading the TCP socket queue. 
> Which gets us
> back to the original point I mentioned earlier. TCP FLOW 
> Control is not
> perfect.
>  
> > 
> > > timers are inadequate since they are mostly implemented on a peer
> > > session basis and studies have shown when they are used they
> > > severely degrade route convergence time. The problem with 
> statically
> > > defined timers (initialized during system load or session
> > > establishment phase) is that they do not adjust to peer internal
> > > dynamically changing congestion conditions.
> > 
> > You are completly missing the point... 
> 
> Please explain what I missed. I found very little background 
> information on these
> timers.
> 
> I have a report done, that shows when the 
> MinRouteAdvertisemetInterval is used
> and set to 30 seconds it degrades the convergence time. When 
> set to 0 or 1, things work much better.
> 
> Check this out:
> http://www.ripe.net/ripe/meetings/archive/ripe-42/presentation
s/ripe42-eof-bgp/sld038.html

> > The problem with the
> > congested peer using TCP flow control to reduce its congestion
> > condition, is that it completely stops all incoming session traffic
> > and thus preventing any messages of high priority nature from being
> > seen.
> 
> What would be high priority in a BGP session... ?
> To my understanding there is no need to introduce priorities 
> in BGP messages.

I think by having dynamic messages that change the nature of the session
traffic, they need to have a sense of priority. You dont want to change
the type of NLRI (say from IPv4 to IPv6) and wait 10 minutes to see if it
happened. Since BGP is piggy backing so much other stuff in its update
messages
it only seems reasonable to put priority on these messages.

> 
> > This document presents a mechanism by which the BGP session of a
> > congested router need not be degraded to the level that
> > communication is broken with its peers.
> 
> Please do offer more to substantiate this other than your claim that
> the problem exists.
>
 
By the fact that the pacing control is more direct and implicit to a
particular
traffic type (in this case update messages). The pacing mechanism is
deterministic, whereas TCP flow control is not. You should talk to people
who build IPC mechanisms to replace TCP about this issue. 

whether the session breaks or badly degraded is an implementation specific
case. 
But I feel there is enough there to warrant the improvements.

The rest needs to be proving empiracly.

Ben

Date: Thu, 18 Jul 2002 14:27:47 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: My BGP Route Update Pacing Draft
Sender: [email-masked]

Abarbanel, Benjamin writes:

Ben,
I'm my opinion your draft offers a much worse alternative solution to
a problem that is already solved in a much more elegant way using TCP
flow control mechanism.

To test the assertion above that the problem has been solved, take a
mature BGP implementation and congest the resources of receiver. You
will see that the sender will automatically throttle it's sending
proccess. Contrary to what you state, sessions are not dropped.

BGP when correctly implemented is a very resilient protocol and it has
been widly deployed in very demanding environments.

I'm also including some comments inline...

> 2. Abstract

> This document defines a mechanism for controlling or limiting the
> rate at which BGP update messages are sent from one peer to the next
> when a BGP peer experiences internal congestion. With the
> introduction of new dynamic BGP protocol capabilities [CAP] message
> or other none BGP session destructive messages, it is necessary to
> limit the rate at which the BGP update messages are sent 

What difference does the introduction of capabilities do ?

> without
> affecting the entire BGP session and without relying on the
> transport (TCP) layer

Why not relly on the transport. Flow control is after all
traditionally the function of the transport. What makes you think it
needs to be reinvented... and how is your version of flow control
supperior to the current one ?

> to do so as a normal reaction to data
> congestion of the TCP session across the network.

[...]

> 4. Introduction

> This document defines a mechanism for controlling or limiting the
> rate at which BGP update messages are sent from one peer to the next
> when a BGP peer experiences internal congestion. With the
> introduction of new dynamic BGP protocol capabilities [CAP] message
> or other none BGP session destructive messages, it is necessary to
> limit the rate at which the BGP update messages are sent without
> affecting the entire BGP session and without relying on the
> transport (TCP) layer to do so as a normal reaction to data
> congestion of the TCP session across the network.

> When a router enters a state where either its CPU utilization is
> maximized (reaches close to 100%) or its memory is nearly depleted
> (less than 10% of memory left), it cannot handle new or heavy
> streams of updates from its peers and at times unable to send
> messages to its peer in a timely manner (within 30 seconds).

If a router cannot send a message for 30 seconds there is something
very wrong with it. It is not clear to me how your mechanism prevents
that.

> As a
> consequence it cannot keep current with the topological state. In
> some scenarios, a non-congested peer might want to negotiate new
> capabilities with a congested peer. The congested peer is so
> degraded that its TCP session goes into significant flow control off
> conditions and is unable to see the new BGP messages. Peer to peer
> communication is severely hampered and as a result the uncongested
> peer will take corrective action when its hold timer expires and
> drops the session. The uncongested peer computes alternate routing
> paths that are suboptimal in distance or attribute and thus affect
> the forwarding decisions of all routers in this network. It is
> possible that the congested peer's routing (control) plane is badly
> degraded but its forwarding plane is at normal working level. The
> mistake

What mistake ?

> is made by the uncongested peer since it does not see any
> Keepalive/Update messages before its Hold timer expires and drops
> the session thereby dropping all its associated routes. After routes
> are dropped, network instability occurs and suboptimal paths are
> used by the remaining peers.

The scenario that you present above is one where you basically have a
peer that is totally broken. What does it matter ? A decent
implementation will choose to degrade it is service levels
gradually. It may achieve that with the current mechanisms.

> The MinRouteAdvertisementInternal and MinAsOriginationInterval

What does min advert interval have to do with the topic in
discussion... ? These control only the rate at which the sender may
attempt to generate updates, not the rate at which the receiver needs
to accept it.

In BGP a receiver can choose to accept a anyrate of updates it wants.

> timers are inadequate since they are mostly implemented on a peer
> session basis and studies have shown when they are used they
> severely degrade route convergence time. The problem with statically
> defined timers (initialized during system load or session
> establishment phase) is that they do not adjust to peer internal
> dynamically changing congestion conditions.

You are completly missing the point... 

> The problem with the
> congested peer using TCP flow control to reduce its congestion
> condition, is that it completely stops all incoming session traffic
> and thus preventing any messages of high priority nature from being
> seen.

What would be high priority in a BGP session... ?
To my understanding there is no need to introduce priorities in BGP messages.

> This document presents a mechanism by which the BGP session of a
> congested router need not be degraded to the level that
> communication is broken with its peers.

Please do offer more to substantiate this other than your claim that
the problem exists.

  Pedro.

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'vrishab sikand'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: My BGP Route Update Pacing Draft
Date: Thu, 18 Jul 2002 16:47:59 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_001_01C22E9C.66EA1D30
Content-Type: text/plain;
	charset="iso-8859-1"

Vrishab:

-----Original Message-----
From: vrishab sikand [mailto:[email-masked]]
Sent: Thursday, July 18, 2002 4:37 PM
To: Abarbanel, Benjamin; [email-masked]'
Subject: Re: My BGP Route Update Pacing Draft

Specific comment: 

Unless the rate of pacing is defined in absolute terms (as opposed to the a
subjectively for e.g 

yellow 4: reduce update message traffic by 40% of your normal rate). It will
be very difficult to verify compliance. 

Also in my experience, the "normal rate" varies drastically from vendor to
vendor. 

 What I meant here, whatever the message/second or route/second transmission
was as an average, reduce it by 40%. So if source router sent 100
messages/second it should now send 60 messages/second. That implies that the
sending router is keeping average statistics of its transmission rate to the
given peer on a message or route per second basis. I dont think this will be
too costly to do. 

General comment: 

 It appears to me that here is an attempt to solve an internal scheduling or
horse power problem with the help of external protocol.   

Yes in way it does, but it could also help scale older routers when they are
placed in a new environment where the loads exceed their previous tuned
expectations. You can think of this as a gracefull way to deal with abnormal
loading conditions without emploding internally.  

This mechanism may put undue burden on the transmitting router, which is in
the best interest of quick convergence in mind is sending prefixes as fast
as possible.   

I am not sure this is the case, it all depends on how people implement the
solution.  

Ben 

  "Abarbanel, Benjamin" <[email-masked]> wrote: 

Hi all:

Our recent discussions on this list and my recent work 
experiences have led me to write this draft and offer it 
to the IETF community. I would appreciate any comments 
anyone has to make.

Thanks in advance,
Ben

Network Working Group Ben Abarbanel
Internet Draft Marconi Communicatons
Expiration Date: December 2002 

BGP Route Update Pacing

draft-abarbanel-bgp-route-update-pacing-00.txt

1. Status of this Memo

This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026 except that the right to
produce derivative works is not granted.

Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as ``work in progress.''

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.

2. Abstract

This document defines a mechanism for controlling or limiting the rate at 
which BGP update messages are sent from one peer to the next when a BGP peer

experiences internal congestion. With the introduction of new dynamic BGP 
protocol capabilities [CAP] message or other none BGP session destructive 
messages, it is necessary to limit the rate at which the BGP update messages

are sent without affecting the entire BGP session and without relying on the

transport (TCP) layer to do so as a normal reaction to data congestion of
the 
TCP session across the network.

3. Specification of Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", 
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to
be 
interpreted as described in RFC 2119 [RFC2119].

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 2]

4. Introduction

This document defines a mechanism for controlling or limiting the rate at 
which BGP update messages are sent from one peer to the next when a BGP peer

experiences internal congestion. With the introduction of new dynamic BGP 
protocol capabilities [CAP] message or other none BGP session destructive 
messages, it is necessary to limit the rate at which the BGP update messages

are sent without affecting the entire BGP session and without relying on the

transport (TCP) layer to do so as a normal reaction to data congestion of
the 
TCP session across the network.

When a router enters a state where either its CPU utilization is maximized 
(reaches close to 100%) or its memory is nearly depleted (less than 10% of 
memory left), it cannot handle new or heavy streams of updates from its
peers 
and at times unable to send messages to its peer in a timely manner (within 
30 seconds). As a consequence it cannot keep current with the topological 
state. In some scenarios, a non-congested peer might want to negotiate new 
capabilities with a congested peer. The congested peer is so degraded that 
its TCP session goes into significant flow control off conditions and is 
unable to see the new BGP messages. Peer to peer communication is severely 
hampered and as a result the uncongested peer will take corrective action 
when its hold timer expires and drops the session. The uncongested peer 
computes alternate routing paths that are suboptimal in distance or
attribute 
and thus affect the forwarding decisions of all routers in this network. It 
is possible that the congested peer's routing (control) plane is badly 
degraded but its forwarding plane is at normal working level. The mistake is

made by the uncongested peer since it does not see any Keepalive/Update 
messages before its Hold timer expires and drops the session thereby
dropping 
all its associated routes. After routes are dropped, network instability 
occurs and suboptimal paths are used by the remaining peers.

The MinRouteAdvertisementInternal and MinAsOriginationInterval timers are 
inadequate since they are mostly implemented on a peer session basis and 
studies have shown when they are used they severely degrade route
convergence 
time. The problem with statically defined timers (initialized during system 
load or session establishment phase) is that they do not adjust to peer 
internal dynamically changing congestion conditions. The problem with the 
congested peer using TCP flow control to reduce its congestion condition, is

that it completely stops all incoming session traffic and thus preventing
any 
messages of high priority nature from being seen.

TCP Out of Band Data or Urgent Message is one way to bypass the TCP flow 
control condition and allow high priority BGP messages to get to the 
congested peer, assuming it is able to read the session socket queue. This 
solution has its drawbacks as described in [UNIX-NET] chapter 21, p. 568. 

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 3]

This document presents a mechanism by which the BGP session of a congested 
router need not be degraded to the level that communication is broken with 
its peers. By using a peer to peer pacing mechanism which allows one peer to

rate limit the number of update messages per second received from another 
peer, it can avoid the severe congestion conditions and process these 
messages at a manageable level. In addition, the uncongested peer has the 
knowledge to send high priority messages ahead of or in place of the normal 
high volume of update messages.

Usually, topological disturbances are spiky in nature and once they subside,

the network returns to its optimum path oriented level. Whatever caused the 
network to become unstable, such as routers handling too much data in a
small 
period of time or routers loosing their sessions or their links going up and

down, occurs in most live network on an infrequent basis. By using the
pacing 
mechanism as outlined in this draft, a BGP peer can prevent the serious 
congestion long before it is in trouble and thus ride through topological 
disturbances and still regain its stability without causing its sessions to 
drop or its routes/paths to be discarded and recomputed.

The assumption in this spec is that the source of most or all of the
internal 
BGP router congestion is due to the heavy reception of update messages from 
neighboring peers containing large number of routes.

5. BGP Update Pacing Mechanism

The BGP Update Message Pacing Mechanism is used to slow down the rate at 
which a peer sends update messages to another. This extension to the BGP 
protocol is simplified to use session non-destructive messages such as
INFORM 
as described in [INFORM]. The pacing is performed dynamically upon
congestion 
detection and subsidence and thus needs to use the new [INFORM] message that

will not infringe on the underlying BGP protocol or all its semantics and 
rules.

5.1 BGP Update Message Pacing (PACE) Dynamic Capability

The BGP Update Message Pacing capability is dynamically negotiated with all 
BGP speakers as type code=PACE (where, PACE=TBD) per TLV structures as 
defined in [BGP-CAP] of the OPEN message or anytime after session is 
established using the Dynamic Capability Message as defined [DYN-CAP] 
specification. All those peers that accept the PACE capability will be 
expected to support the new INFORM message as defined in [INFORM] 
specification to carry the pacing TLV structure. 

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 4]

All PACE capable routers will provide a configuration option to their 
operator to enable the BGP Update Message Pacing mechanism on a per peer 
basis. 

Any peer wishing to withdraw the PACE capability can do so dynamically using

the Dynamic Capability message as outlined in [DYN-CAP] specification. Once 
withdrawn, affected peering session will remain intact but will not benefit 
from the performance improvements offered by the pacing mechanism.

5.2 Use of INFORM Message

The INFORM message as described in [INFORM] is used to carry the rate 
limiting (pacing TLV) control structure to neighboring peers. This 
information informs the peer that the current BGP router has entered a 
congestion state and it is to rate limit its transmission to the level 
specified. 

The INFORM message contains the following PACE TLV structure:

Type = Pacing Information, type=TBD 
Length = 2
1st Byte of Value = Cmd as shown below 
2nd Byte of Value = Level as shown below

Cmd Description 
------- -----------
11 Request to Pace update messages
12 ACK Response to Pace Update messages
13 NACK Response to Paace Update Messages

A. Request to Pace Update Messages Level Indication (code=11)

The rate is divided into sub levels in term of categories (Gray, Yellow, 
Orange, Red, and Green) to denote the level of pacing which is directly 
related to the level of congestion experienced within the congested peer.
The 
colors are used as simple handles used for flagging the severity of the 
condition. The colors are also used as indicators for display by the NMS to 
identify the rate limiting levels from any neighboring peer to the operator.

Associated Level Management MIBs are defined in section 6. 

The actual pacing control is done via the sub levels within these colors. 

Level Description
------ -----------
Gray (Entering minor congestion state)
1 Reduce update message traffic by 10% your normal rate.
2 Reduce update message traffic by 20% your normal rate.
3 Reduce update message traffic by 30% your normal rate.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 5]

Yellow (Entering medium congestion state)
4 Reduce update message traffic by 40% your normal rate. 
5 Reduce update message traffic by 50% your normal rate.
6 Reduce update message traffic by 60% your normal rate.

Orange (Entering major congestion state)
7 Reduce update message traffic by 70% your normal rate. 
8 Reduce update message traffic by 80% your normal rate.
9 Reduce update message traffic by 90% your normal rate

Red (Entering critical congestion state)
10 Complete cessation of all update message traffic (flow off). 
At this Level the receiving peer might decide to bypass the 
Congested Router and pick another less optimal router for the
affected Routes.

Green (Exiting any congestion state)
0 Restore update message traffic to your normal level (flow on).
Routes redirected can be resumed to the original peer.

The receiving peer will send an INFORM message with an ACK or NACK
Indicating 
it has received and understood the pacing request and will either comply
with 
the it or forbid it. If the congested peer receives a NACK, it should remove

that peer from its list of PACE capable peers but still maintaining its 
session without the use of Pacing. If the NACKing peer decides at a future 
date to re-enable the Pacing with the local peer, it will renegotiate the 
PACE capability with the local peer at that time.

B. Response Message Error Indication:

Level Description
------ -----------
0 ACK indication. The peer will comply with the 
Pacing level request.
1 NACK indication. The peer is unable to perform Pacing or comply 
with the pacing level requested.

5.3 INFORM Message Response Timer

When a congested peer sends an INFORM message with a "Request to Pace Update

Messages (code=11)" to all peers that support the pacing feature, it will 
also start an associated INFORM Message Response Timer. Any peer that does 
not respond within a 30 second timeout period with either an ACK/NACK INFORM

Response message, its associated session will be dropped. This is done to 
clear any PACE capable peers that are also congested to the point where
their 
communication with the local congested peer is severed.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 6]

5.4 Congestion Detection Within the Router

There are at least two ways congestion could be detected and measured by a 
BGP router.

- A high CPU utilization condition 

- A Lack of available memory to accept incoming BGP messages or inability 
to get memory to successfully complete BGP processing. 

5.4.1 CPU Utilization Based Level Computation

It is recommended that when the congested router detects its inability to 
perform route calculations or accept new BGP session messages at a normal 
rate meaning the current router's CPU utilization is more than 49%, it
should 
inform all its peers of a rate limiting level and slow them down
accordingly. 

The recommended way for computing the Level, based on percent CPU 
Utilization, is done using the following:

Level = (((%CPU Utilization - 50) x 2) + 5) / 10).
Where, % CPU Utilization is a whole integer number. Use Integer math 
here to remove any fractional value.

Note: If computed Level is negative, go to Green and set Level = 0. If 
Level = 0 send INFORM message with pacing Level = 0, implying return 
to normal (100%) update rate.

Level implies, whatever number of messages you transmitted per second 
before, transmit (100 - (Level x 10))% of that now. 

e.g. If you transmitted 100 messages/second before. A level 2 (20%) 
reduction will cause you to transmit 80 messages/second now.
Assumption is that these messages have an average size (1500 bytes).

5.4.2 Memory Allocation Based Level Computation

It is recommended that when the congested router detects its inability to 
perform route calculations or accept new BGP session messages at a normal 
rate meaning the current router's memory allocation is more than 69%, it 
should inform all its peers of a rate limiting level and slow them down 
accordingly. 

The recommended way for computing the Level, based on percent Memory 
Allocation, is done using the following:

Level = (((%Memory Allocated - 70) x 2) + 5) / 10).
Where, % Memory Allocated is a whole integer number. Use Integer math 
here to remove any fractional value.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 7]

Note: If computed Level is negative, go to Green sub-group and set 
Level = 0. If Level = 0 send INFORM message with pacing Level = 0, 
implying return to normal (100%) update rate.

5.5 INFORM Message Throttling

Once the congested peer receives acknowledgement from another peer, it will 
send a modification INFORM message with a new Level to that peer after the 
computed pacing Level changes by at least 1 value. This will amount to no 
more than one INFORM modification message every 5 seconds. This is done to 
debounce any spiky bursts of INFORM messages to all PACE negotiated peers 
each time the computed pacing Level changes. Depending on vendor 
implementations, the internal utilization levels could change at the 
Microsecond or Millisecond rate.

6. Implementation Specific Mechanisms

The Memory Allocation and CPU Utilization Level detection algorithms 
discussed in section 5.4.1 and 5.4.2 are suggested ways one can implement 
these solutions. However, each vendor can implement a unique Memory 
Allocation and CPU Utilization Level detection algorithms that best suits 
his/her needs and will not negatively impact the overall BGP Route Update 
Pacing mechanism described in this spec. Any issues relating to internal 
implementation algorithms are outside the scope of this document.

7. Level Indicators Management MIBs

TBD

8. Security Considerations

This extension to BGP does not change the underlying security issues.

9. References

[BGP-CAP] Chandra, R., Scudder, J., "Capabilities Advertisement with 
BGP-4", draft-ietf-idr-rfc2842bis-02.txt

[DYN-CAP] Chen E., Sangli S., "Dynamic Capability for BGP-4", 
draft-ietf-idr-dynamic-cap-02.txt, October 2002.

[INFORM] Nalawade G., Scudder J., "BGPv4 INFORM Message",
draft-nalawade-bgp-inform-00.txt, December 2002

[BGP-4] Rekhter, Y., and T. Li, "A Border Gateway Protocol 4 
(BGP-4)", RFC 1771, March 1995.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 8]

[BGP-4-DRAFT] Rekhter, Y. and T. Li (editors), "A Border Gateway Protocol 4
(BGP-4)", Internet Draft draft-ietf-idr-bgp4-18.txt, 
January 2002.

[UNIX-NET] Stevens, R., "UNIX Network Programming, Vol 1", Second Edition
1998 Prentice Hall, Inc.

10. Author Information

Ben Abarbanel
Marconi Communications
1595 Spring Hill Road, 5th Floor
Vienna, VA 22182
Email: [email-masked]

   _____  

Do You Yahoo!?
Yahoo! Autos <http://autos.yahoo.com/>  - Get free new car price quotes

------_=_NextPart_001_01C22E9C.66EA1D30
Content-Type: text/html;
	charset="iso-8859-1"

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">

<META content="MSHTML 5.00.2920.0" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT color=#0000ff face=Arial size=2><SPAN 
class=316534120-18072002>Vrishab:</SPAN></FONT></DIV>
<BLOCKQUOTE 
style="BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: 5px">
  <DIV align=left class=OutlookMessageHeader dir=ltr><FONT face=Tahoma 
  size=2>-----Original Message-----<BR><B>From:</B> vrishab sikand 
  [mailto:[email-masked]]<BR><B>Sent:</B> Thursday, July 18, 2002 4:37 
  PM<BR><B>To:</B> Abarbanel, Benjamin; [email-masked]'<BR><B>Subject:</B> Re: 
  My BGP Route Update Pacing Draft<BR><BR></DIV></FONT>
  <P>Specific comment: 
  <P>Unless the rate of pacing is defined in absolute terms (as opposed to the a 
  subjectively for e.g 
  <P>yellow 4: reduce update message traffic by 40% of your normal rate). It 
  will be very difficult to verify compliance. 
  <P>Also in my experience, the "normal rate" varies drastically from vendor to 
  vendor. 
  <P>&nbsp;<FONT color=#0000ff face=Arial size=2><SPAN 
  class=316534120-18072002>What I meant here, whatever the message/second or 
  route/second transmission was as an average, reduce it by 40%. So if source 
  router sent 100 messages/second&nbsp;it should now send 60 messages/second. 
  That implies that the sending router is keeping average statistics of its 
  transmission rate to the given peer on a message or route per second basis. I 
  dont think this will be too costly to do.</SPAN></FONT>
  <P>General comment: 
  <P>&nbsp;It appears to me that here is an attempt to solve an internal 
  scheduling or horse power problem with the help of external 
  protocol.&nbsp;<FONT color=#0000ff face=Arial size=2><SPAN 
  class=316534120-18072002>&nbsp;</SPAN></FONT>
  <P><FONT color=#0000ff face=Arial size=2><SPAN class=316534120-18072002>Yes in 
  way it does, but it could also help scale older routers when they are placed 
  in a new environment where the loads exceed their previous tuned 
  expectations.&nbsp;You can think of this as a gracefull way to deal with 
  abnormal loading conditions without emploding internally.&nbsp;</SPAN></FONT>
  <P>This mechanism may put undue burden on the transmitting router, which is in 
  the best interest of quick convergence in mind is sending prefixes as fast as 
  possible.&nbsp;<FONT color=#0000ff face=Arial size=2><SPAN 
  class=316534120-18072002>&nbsp;</SPAN></FONT>
  <P><FONT color=#0000ff face=Arial size=2><SPAN class=316534120-18072002>I am 
  not sure this is the case, it all depends on how people implement the 
  solution.&nbsp;</SPAN></FONT>
  <P><FONT color=#0000ff face=Arial size=2><SPAN 
  class=316534120-18072002></SPAN></FONT>&nbsp;
  <P><FONT color=#0000ff face=Arial size=2><SPAN 
  class=316534120-18072002>Ben</SPAN></FONT>
  <P>&nbsp; <B><I>"Abarbanel, Benjamin" 
  &lt;[email-masked]&gt;</I></B> wrote: 
  <BLOCKQUOTE 
  style="BORDER-LEFT: #1010ff 2px solid; MARGIN-LEFT: 5px; PADDING-LEFT: 5px">Hi 
    all:<BR><BR>Our recent discussions on this list and my recent work 
    <BR>experiences have led me to write this draft and offer it <BR>to the IETF 
    community. I would appreciate any comments <BR>anyone has to 
    make.<BR><BR>Thanks in 
    advance,<BR>Ben<BR><BR><BR><BR><BR><BR><BR><BR>Network Working Group Ben 
    Abarbanel<BR>Internet Draft Marconi Communicatons<BR>Expiration Date: 
    December 2002 <BR><BR><BR><BR>BGP Route Update 
    Pacing<BR><BR>draft-abarbanel-bgp-route-update-pacing-00.txt<BR><BR><BR>1. 
    Status of this Memo<BR><BR>This document is an Internet-Draft and is in full 
    conformance with<BR>all provisions of Section 10 of RFC2026 except that the 
    right to<BR>produce derivative works is not granted.<BR><BR>Internet-Drafts 
    are working documents of the Internet Engineering<BR>Task Force (IETF), its 
    areas, and its working groups. Note that<BR>other groups may also distribute 
    working documents as Internet-<BR>Drafts.<BR><BR>Internet-Drafts are draft 
    documents valid for a maximum of six months<BR>and may be updated, replaced, 
    or obsoleted by other documents at any<BR>time. It is inappropriate to use 
    Internet-Drafts as reference<BR>material or to cite them other than as 
    ``work in progress.''<BR><BR>The list of current Internet-Drafts can be 
    accessed at<BR>http://www.ietf.org/ietf/1id-abstracts.txt<BR><BR>The list of 
    Internet-Draft Shadow Directories can be accessed 
    at<BR>http://www.ietf.org/shadow.html.<BR><BR><BR>2. Abstract<BR><BR>This 
    document defines a mechanism for controlling or limiting the rate at 
    <BR>which BGP update messages are sent from one peer to the next when a BGP 
    peer <BR>experiences internal congestion. With the introduction of new 
    dynamic BGP <BR>protocol capabilities [CAP] message or other none BGP 
    session destructive <BR>messages, it is necessary to limit the rate at which 
    the BGP update messages <BR>are sent without affecting the entire BGP 
    session and without relying on the <BR>transport (TCP) layer to do so as a 
    normal reaction to data congestion of the <BR>TCP session across the 
    network.<BR><BR>3. Specification of Requirements<BR><BR>The key words 
    "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", <BR>"SHOULD 
    NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be 
    <BR>interpreted as described in RFC 2119 
    [RFC2119].<BR><BR><BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 2]<BR><BR><BR><BR>4. 
    Introduction<BR><BR>This document defines a mechanism for controlling or 
    limiting the rate at <BR>which BGP update messages are sent from one peer to 
    the next when a BGP peer <BR>experiences internal congestion. With the 
    introduction of new dynamic BGP <BR>protocol capabilities [CAP] message or 
    other none BGP session destructive <BR>messages, it is necessary to limit 
    the rate at which the BGP update messages <BR>are sent without affecting the 
    entire BGP session and without relying on the <BR>transport (TCP) layer to 
    do so as a normal reaction to data congestion of the <BR>TCP session across 
    the network.<BR><BR>When a router enters a state where either its CPU 
    utilization is maximized <BR>(reaches close to 100%) or its memory is nearly 
    depleted (less than 10% of <BR>memory left), it cannot handle new or heavy 
    streams of updates from its peers <BR>and at times unable to send messages 
    to its peer in a timely manner (within <BR>30 seconds). As a consequence it 
    cannot keep current with the topological <BR>state. In some scenarios, a 
    non-congested peer might want to negotiate new <BR>capabilities with a 
    congested peer. The congested peer is so degraded that <BR>its TCP session 
    goes into significant flow control off conditions and is <BR>unable to see 
    the new BGP messages. Peer to peer communication is severely <BR>hampered 
    and as a result the uncongested peer will take corrective action <BR>when 
    its hold timer expires and drops the session. The uncongested peer 
    <BR>computes alternate routing paths that are suboptimal in distance or 
    attribute <BR>and thus affect the forwarding decisions of all routers in 
    this network. It <BR>is possible that the congested peer's routing (control) 
    plane is badly <BR>degraded but its forwarding plane is at normal working 
    level. The mistake is <BR>made by the uncongested peer since it does not see 
    any Keepalive/Update <BR>messages before its Hold timer expires and drops 
    the session thereby dropping <BR>all its associated routes. After routes are 
    dropped, network instability <BR>occurs and suboptimal paths are used by the 
    remaining peers.<BR><BR>The MinRouteAdvertisementInternal and 
    MinAsOriginationInterval timers are <BR>inadequate since they are mostly 
    implemented on a peer session basis and <BR>studies have shown when they are 
    used they severely degrade route convergence <BR>time. The problem with 
    statically defined timers (initialized during system <BR>load or session 
    establishment phase) is that they do not adjust to peer <BR>internal 
    dynamically changing congestion conditions. The problem with the 
    <BR>congested peer using TCP flow control to reduce its congestion 
    condition, is <BR>that it completely stops all incoming session traffic and 
    thus preventing any <BR>messages of high priority nature from being 
    seen.<BR><BR>TCP Out of Band Data or Urgent Message is one way to bypass the 
    TCP flow <BR>control condition and allow high priority BGP messages to get 
    to the <BR>congested peer, assuming it is able to read the session socket 
    queue. This <BR>solution has its drawbacks as described in [UNIX-NET] 
    chapter 21, p. 568. <BR><BR><BR><BR><BR><BR><BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 3]<BR><BR><BR><BR>This 
    document presents a mechanism by which the BGP session of a congested 
    <BR>router need not be degraded to the level that communication is broken 
    with <BR>its peers. By using a peer to peer pacing mechanism which allows 
    one peer to <BR>rate limit the number of update messages per second received 
    from another <BR>peer, it can avoid the severe congestion conditions and 
    process these <BR>messages at a manageable level. In addition, the 
    uncongested peer has the <BR>knowledge to send high priority messages ahead 
    of or in place of the normal <BR>high volume of update 
    messages.<BR><BR>Usually, topological disturbances are spiky in nature and 
    once they subside, <BR>the network returns to its optimum path oriented 
    level. Whatever caused the <BR>network to become unstable, such as routers 
    handling too much data in a small <BR>period of time or routers loosing 
    their sessions or their links going up and <BR>down, occurs in most live 
    network on an infrequent basis. By using the pacing <BR>mechanism as 
    outlined in this draft, a BGP peer can prevent the serious <BR>congestion 
    long before it is in trouble and thus ride through topological 
    <BR>disturbances and still regain its stability without causing its sessions 
    to <BR>drop or its routes/paths to be discarded and recomputed.<BR><BR>The 
    assumption in this spec is that the source of most or all of the internal 
    <BR>BGP router congestion is due to the heavy reception of update messages 
    from <BR>neighboring peers containing large number of routes.<BR><BR><BR>5. 
    BGP Update Pacing Mechanism<BR><BR>The BGP Update Message Pacing Mechanism 
    is used to slow down the rate at <BR>which a peer sends update messages to 
    another. This extension to the BGP <BR>protocol is simplified to use session 
    non-destructive messages such as INFORM <BR>as described in [INFORM]. The 
    pacing is performed dynamically upon congestion <BR>detection and subsidence 
    and thus needs to use the new [INFORM] message that <BR>will not infringe on 
    the underlying BGP protocol or all its semantics and 
    <BR>rules.<BR><BR><BR>5.1 BGP Update Message Pacing (PACE) Dynamic 
    Capability<BR><BR>The BGP Update Message Pacing capability is dynamically 
    negotiated with all <BR>BGP speakers as type code=PACE (where, PACE=TBD) per 
    TLV structures as <BR>defined in [BGP-CAP] of the OPEN message or anytime 
    after session is <BR>established using the Dynamic Capability Message as 
    defined [DYN-CAP] <BR>specification. All those peers that accept the PACE 
    capability will be <BR>expected to support the new INFORM message as defined 
    in [INFORM] <BR>specification to carry the pacing TLV structure. 
    <BR><BR><BR><BR><BR><BR><BR><BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 4]<BR><BR><BR><BR>All 
    PACE capable routers will provide a configuration option to their 
    <BR>operator to enable the BGP Update Message Pacing mechanism on a per peer 
    <BR>basis. <BR><BR>Any peer wishing to withdraw the PACE capability can do 
    so dynamically using <BR>the Dynamic Capability message as outlined in 
    [DYN-CAP] specification. Once <BR>withdrawn, affected peering session will 
    remain intact but will not benefit <BR>from the performance improvements 
    offered by the pacing mechanism.<BR><BR><BR>5.2 Use of INFORM 
    Message<BR><BR>The INFORM message as described in [INFORM] is used to carry 
    the rate <BR>limiting (pacing TLV) control structure to neighboring peers. 
    This <BR>information informs the peer that the current BGP router has 
    entered a <BR>congestion state and it is to rate limit its transmission to 
    the level <BR>specified. <BR><BR>The INFORM message contains the following 
    PACE TLV structure:<BR><BR>Type = Pacing Information, type=TBD <BR>Length = 
    2<BR>1st Byte of Value = Cmd as shown below <BR>2nd Byte of Value = Level as 
    shown below<BR><BR>Cmd Description <BR>------- -----------<BR>11 Request to 
    Pace update messages<BR>12 ACK Response to Pace Update messages<BR>13 NACK 
    Response to Paace Update Messages<BR><BR>A. Request to Pace Update Messages 
    Level Indication (code=11)<BR><BR>The rate is divided into sub levels in 
    term of categories (Gray, Yellow, <BR>Orange, Red, and Green) to denote the 
    level of pacing which is directly <BR>related to the level of congestion 
    experienced within the congested peer. The <BR>colors are used as simple 
    handles used for flagging the severity of the <BR>condition. The colors are 
    also used as indicators for display by the NMS to <BR>identify the rate 
    limiting levels from any neighboring peer to the operator. <BR>Associated 
    Level Management MIBs are defined in section 6. <BR><BR>The actual pacing 
    control is done via the sub levels within these colors. <BR><BR>Level 
    Description<BR>------ -----------<BR>Gray (Entering minor congestion 
    state)<BR>1 Reduce update message traffic by 10% your normal rate.<BR>2 
    Reduce update message traffic by 20% your normal rate.<BR>3 Reduce update 
    message traffic by 30% your normal rate.<BR><BR><BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 
    5]<BR><BR><BR><BR>Yellow (Entering medium congestion state)<BR>4 Reduce 
    update message traffic by 40% your normal rate. <BR>5 Reduce update message 
    traffic by 50% your normal rate.<BR>6 Reduce update message traffic by 60% 
    your normal rate.<BR><BR>Orange (Entering major congestion state)<BR>7 
    Reduce update message traffic by 70% your normal rate. <BR>8 Reduce update 
    message traffic by 80% your normal rate.<BR>9 Reduce update message traffic 
    by 90% your normal rate<BR><BR>Red (Entering critical congestion 
    state)<BR>10 Complete cessation of all update message traffic (flow off). 
    <BR>At this Level the receiving peer might decide to bypass the 
    <BR>Congested Router and pick another less optimal router for 
    the<BR>affected Routes.<BR><BR>Green (Exiting any congestion state)<BR>0 
    Restore update message traffic to your normal level (flow on).<BR>Routes 
    redirected can be resumed to the original peer.<BR><BR>The receiving peer 
    will send an INFORM message with an ACK or NACK Indicating <BR>it has 
    received and understood the pacing request and will either comply with 
    <BR>the it or forbid it. If the congested peer receives a NACK, it should 
    remove <BR>that peer from its list of PACE capable peers but still 
    maintaining its <BR>session without the use of Pacing. If the NACKing peer 
    decides at a future <BR>date to re-enable the Pacing with the local peer, it 
    will renegotiate the <BR>PACE capability with the local peer at that 
    time.<BR><BR>B. Response Message Error Indication:<BR><BR>Level 
    Description<BR>------ -----------<BR>0 ACK indication. The peer will comply 
    with the <BR>Pacing level request.<BR>1 NACK indication. The peer is unable 
    to perform Pacing or comply <BR>with the pacing level 
    requested.<BR><BR><BR>5.3 INFORM Message Response Timer<BR><BR>When a 
    congested peer sends an INFORM message with a "Request to Pace Update 
    <BR>Messages (code=11)" to all peers that support the pacing feature, it 
    will <BR>also start an associated INFORM Message Response Timer. Any peer 
    that does <BR>not respond within a 30 second timeout period with either an 
    ACK/NACK INFORM <BR>Response message, its associated session will be 
    dropped. This is done to <BR>clear any PACE capable peers that are also 
    congested to the point where their <BR>communication with the local 
    congested peer is severed.<BR><BR><BR><BR><BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 6]<BR><BR><BR><BR>5.4 
    Congestion Detection Within the Router<BR><BR>There are at least two ways 
    congestion could be detected and measured by a <BR>BGP router.<BR><BR>- A 
    high CPU utilization condition <BR><BR>- A Lack of available memory to 
    accept incoming BGP messages or inability <BR>to get memory to successfully 
    complete BGP processing. <BR><BR>5.4.1 CPU Utilization Based Level 
    Computation<BR><BR>It is recommended that when the congested router detects 
    its inability to <BR>perform route calculations or accept new BGP session 
    messages at a normal <BR>rate meaning the current router's CPU utilization 
    is more than 49%, it should <BR>inform all its peers of a rate limiting 
    level and slow them down accordingly. <BR><BR>The recommended way for 
    computing the Level, based on percent CPU <BR>Utilization, is done using the 
    following:<BR><BR>Level = (((%CPU Utilization - 50) x 2) + 5) / 
    10).<BR>Where, % CPU Utilization is a whole integer number. Use Integer math 
    <BR>here to remove any fractional value.<BR><BR>Note: If computed Level is 
    negative, go to Green and set Level = 0. If <BR>Level = 0 send INFORM 
    message with pacing Level = 0, implying return <BR>to normal (100%) update 
    rate.<BR><BR>Level implies, whatever number of messages you transmitted per 
    second <BR>before, transmit (100 - (Level x 10))% of that now. <BR><BR>e.g. 
    If you transmitted 100 messages/second before. A level 2 (20%) <BR>reduction 
    will cause you to transmit 80 messages/second now.<BR>Assumption is that 
    these messages have an average size (1500 bytes).<BR><BR><BR>5.4.2 Memory 
    Allocation Based Level Computation<BR><BR>It is recommended that when the 
    congested router detects its inability to <BR>perform route calculations or 
    accept new BGP session messages at a normal <BR>rate meaning the current 
    router's memory allocation is more than 69%, it <BR>should inform all its 
    peers of a rate limiting level and slow them down <BR>accordingly. 
    <BR><BR>The recommended way for computing the Level, based on percent Memory 
    <BR>Allocation, is done using the following:<BR><BR>Level = (((%Memory 
    Allocated - 70) x 2) + 5) / 10).<BR>Where, % Memory Allocated is a whole 
    integer number. Use Integer math <BR>here to remove any fractional 
    value.<BR><BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 7]<BR><BR><BR><BR>Note: 
    If computed Level is negative, go to Green sub-group and set <BR>Level = 0. 
    If Level = 0 send INFORM message with pacing Level = 0, <BR>implying return 
    to normal (100%) update rate.<BR><BR><BR>5.5 INFORM Message 
    Throttling<BR><BR>Once the congested peer receives acknowledgement from 
    another peer, it will <BR>send a modification INFORM message with a new 
    Level to that peer after the <BR>computed pacing Level changes by at least 1 
    value. This will amount to no <BR>more than one INFORM modification message 
    every 5 seconds. This is done to <BR>debounce any spiky bursts of INFORM 
    messages to all PACE negotiated peers <BR>each time the computed pacing 
    Level changes. Depending on vendor <BR>implementations, the internal 
    utilization levels could change at the <BR>Microsecond or Millisecond 
    rate.<BR><BR><BR>6. Implementation Specific Mechanisms<BR><BR>The Memory 
    Allocation and CPU Utilization Level detection algorithms <BR>discussed in 
    section 5.4.1 and 5.4.2 are suggested ways one can implement <BR>these 
    solutions. However, each vendor can implement a unique Memory <BR>Allocation 
    and CPU Utilization Level detection algorithms that best suits <BR>his/her 
    needs and will not negatively impact the overall BGP Route Update <BR>Pacing 
    mechanism described in this spec. Any issues relating to internal 
    <BR>implementation algorithms are outside the scope of this 
    document.<BR><BR><BR>7. Level Indicators Management 
    MIBs<BR><BR>TBD<BR><BR><BR>8. Security Considerations<BR><BR>This extension 
    to BGP does not change the underlying security issues.<BR><BR><BR>9. 
    References<BR><BR>[BGP-CAP] Chandra, R., Scudder, J., "Capabilities 
    Advertisement with <BR>BGP-4", 
    draft-ietf-idr-rfc2842bis-02.txt<BR><BR>[DYN-CAP] Chen E., Sangli S., 
    "Dynamic Capability for BGP-4", <BR>draft-ietf-idr-dynamic-cap-02.txt, 
    October 2002.<BR><BR>[INFORM] Nalawade G., Scudder J., "BGPv4 INFORM 
    Message",<BR>draft-nalawade-bgp-inform-00.txt, December 2002<BR><BR>[BGP-4] 
    Rekhter, Y., and T. Li, "A Border Gateway Protocol 4 <BR>(BGP-4)", RFC 1771, 
    March 1995.<BR><BR><BR>Internet Draft 
    draft-abarbanel-bgp-route-update-pacing-00.txt [Page 
    8]<BR><BR><BR><BR>[BGP-4-DRAFT] Rekhter, Y. and T. Li (editors), "A Border 
    Gateway Protocol 4<BR>(BGP-4)", Internet Draft draft-ietf-idr-bgp4-18.txt, 
    <BR>January 2002.<BR><BR>[UNIX-NET] Stevens, R., "UNIX Network Programming, 
    Vol 1", Second Edition<BR>1998 Prentice Hall, Inc.<BR><BR><BR>10. Author 
    Information<BR><BR>Ben Abarbanel<BR>Marconi Communications<BR>1595 Spring 
    Hill Road, 5th Floor<BR>Vienna, VA 22182<BR>Email: 
    [email-masked]<BR><BR><BR></BLOCKQUOTE>
  <P><BR>
  <HR SIZE=1>
  <B>Do You Yahoo!?</B><BR><A href="http://autos.yahoo.com/">Yahoo! Autos</A> - 
  Get free new car price quotes</BLOCKQUOTE></BODY></HTML>

------_=_NextPart_001_01C22E9C.66EA1D30--

Date: Thu, 18 Jul 2002 13:37:20 -0700 (PDT)
From: vrishab sikand <[email-masked]>
Subject: Re: My BGP Route Update Pacing Draft
To: "Abarbanel, Benjamin" <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

--0-504351559-1027024640=:78832
Content-Type: text/plain; charset=us-ascii

Specific comment:
Unless the rate of pacing is defined in absolute terms (as opposed to the a subjectively for e.g
yellow 4: reduce update message traffic by 40% of your normal rate). It will be very difficult to verify compliance.
Also in my experience, the "normal rate" varies drastically from vendor to vendor.
 
General comment:
 It appears to me that here is an attempt to solve an internal scheduling or horse power problem with the help of external protocol. 
This mechanism may put undue burden on the transmitting router, which is in the best interest of quick convergence in mind is sending prefixes as fast as possible. 
  "Abarbanel, Benjamin" <[email-masked]> wrote: Hi all:

Our recent discussions on this list and my recent work 
experiences have led me to write this draft and offer it 
to the IETF community. I would appreciate any comments 
anyone has to make.

Thanks in advance,
Ben

Network Working Group Ben Abarbanel
Internet Draft Marconi Communicatons
Expiration Date: December 2002 

BGP Route Update Pacing

draft-abarbanel-bgp-route-update-pacing-00.txt

1. Status of this Memo

This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026 except that the right to
produce derivative works is not granted.

Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as ``work in progress.''

The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt

The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.

2. Abstract

This document defines a mechanism for controlling or limiting the rate at 
which BGP update messages are sent from one peer to the next when a BGP peer 
experiences internal congestion. With the introduction of new dynamic BGP 
protocol capabilities [CAP] message or other none BGP session destructive 
messages, it is necessary to limit the rate at which the BGP update messages 
are sent without affecting the entire BGP session and without relying on the 
transport (TCP) layer to do so as a normal reaction to data congestion of the 
TCP session across the network.

3. Specification of Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", 
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be 
interpreted as described in RFC 2119 [RFC2119].

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 2]

4. Introduction

This document defines a mechanism for controlling or limiting the rate at 
which BGP update messages are sent from one peer to the next when a BGP peer 
experiences internal congestion. With the introduction of new dynamic BGP 
protocol capabilities [CAP] message or other none BGP session destructive 
messages, it is necessary to limit the rate at which the BGP update messages 
are sent without affecting the entire BGP session and without relying on the 
transport (TCP) layer to do so as a normal reaction to data congestion of the 
TCP session across the network.

When a router enters a state where either its CPU utilization is maximized 
(reaches close to 100%) or its memory is nearly depleted (less than 10% of 
memory left), it cannot handle new or heavy streams of updates from its peers 
and at times unable to send messages to its peer in a timely manner (within 
30 seconds). As a consequence it cannot keep current with the topological 
state. In some scenarios, a non-congested peer might want to negotiate new 
capabilities with a congested peer. The congested peer is so degraded that 
its TCP session goes into significant flow control off conditions and is 
unable to see the new BGP messages. Peer to peer communication is severely 
hampered and as a result the uncongested peer will take corrective action 
when its hold timer expires and drops the session. The uncongested peer 
computes alternate routing paths that are suboptimal in distance or attribute 
and thus affect the forwarding decisions of all routers in this network. It 
is possible that the congested peer's routing (control) plane is badly 
degraded but its forwarding plane is at normal working level. The mistake is 
made by the uncongested peer since it does not see any Keepalive/Update 
messages before its Hold timer expires and drops the session thereby dropping 
all its associated routes. After routes are dropped, network instability 
occurs and suboptimal paths are used by the remaining peers.

The MinRouteAdvertisementInternal and MinAsOriginationInterval timers are 
inadequate since they are mostly implemented on a peer session basis and 
studies have shown when they are used they severely degrade route convergence 
time. The problem with statically defined timers (initialized during system 
load or session establishment phase) is that they do not adjust to peer 
internal dynamically changing congestion conditions. The problem with the 
congested peer using TCP flow control to reduce its congestion condition, is 
that it completely stops all incoming session traffic and thus preventing any 
messages of high priority nature from being seen.

TCP Out of Band Data or Urgent Message is one way to bypass the TCP flow 
control condition and allow high priority BGP messages to get to the 
congested peer, assuming it is able to read the session socket queue. This 
solution has its drawbacks as described in [UNIX-NET] chapter 21, p. 568. 

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 3]

This document presents a mechanism by which the BGP session of a congested 
router need not be degraded to the level that communication is broken with 
its peers. By using a peer to peer pacing mechanism which allows one peer to 
rate limit the number of update messages per second received from another 
peer, it can avoid the severe congestion conditions and process these 
messages at a manageable level. In addition, the uncongested peer has the 
knowledge to send high priority messages ahead of or in place of the normal 
high volume of update messages.

Usually, topological disturbances are spiky in nature and once they subside, 
the network returns to its optimum path oriented level. Whatever caused the 
network to become unstable, such as routers handling too much data in a small 
period of time or routers loosing their sessions or their links going up and 
down, occurs in most live network on an infrequent basis. By using the pacing 
mechanism as outlined in this draft, a BGP peer can prevent the serious 
congestion long before it is in trouble and thus ride through topological 
disturbances and still regain its stability without causing its sessions to 
drop or its routes/paths to be discarded and recomputed.

The assumption in this spec is that the source of most or all of the internal 
BGP router congestion is due to the heavy reception of update messages from 
neighboring peers containing large number of routes.

5. BGP Update Pacing Mechanism

The BGP Update Message Pacing Mechanism is used to slow down the rate at 
which a peer sends update messages to another. This extension to the BGP 
protocol is simplified to use session non-destructive messages such as INFORM 
as described in [INFORM]. The pacing is performed dynamically upon congestion 
detection and subsidence and thus needs to use the new [INFORM] message that 
will not infringe on the underlying BGP protocol or all its semantics and 
rules.

5.1 BGP Update Message Pacing (PACE) Dynamic Capability

The BGP Update Message Pacing capability is dynamically negotiated with all 
BGP speakers as type code=PACE (where, PACE=TBD) per TLV structures as 
defined in [BGP-CAP] of the OPEN message or anytime after session is 
established using the Dynamic Capability Message as defined [DYN-CAP] 
specification. All those peers that accept the PACE capability will be 
expected to support the new INFORM message as defined in [INFORM] 
specification to carry the pacing TLV structure. 

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 4]

All PACE capable routers will provide a configuration option to their 
operator to enable the BGP Update Message Pacing mechanism on a per peer 
basis. 

Any peer wishing to withdraw the PACE capability can do so dynamically using 
the Dynamic Capability message as outlined in [DYN-CAP] specification. Once 
withdrawn, affected peering session will remain intact but will not benefit 
from the performance improvements offered by the pacing mechanism.

5.2 Use of INFORM Message

The INFORM message as described in [INFORM] is used to carry the rate 
limiting (pacing TLV) control structure to neighboring peers. This 
information informs the peer that the current BGP router has entered a 
congestion state and it is to rate limit its transmission to the level 
specified. 

The INFORM message contains the following PACE TLV structure:

Type = Pacing Information, type=TBD 
Length = 2
1st Byte of Value = Cmd as shown below 
2nd Byte of Value = Level as shown below

Cmd Description 
------- -----------
11 Request to Pace update messages
12 ACK Response to Pace Update messages
13 NACK Response to Pace Update Messages

A. Request to Pace Update Messages Level Indication (code=11)

The rate is divided into sub levels in term of categories (Gray, Yellow, 
Orange, Red, and Green) to denote the level of pacing which is directly 
related to the level of congestion experienced within the congested peer. The 
colors are used as simple handles used for flagging the severity of the 
condition. The colors are also used as indicators for display by the NMS to 
identify the rate limiting levels from any neighboring peer to the operator. 
Associated Level Management MIBs are defined in section 6. 

The actual pacing control is done via the sub levels within these colors. 

Level Description
------ -----------
Gray (Entering minor congestion state)
1 Reduce update message traffic by 10% your normal rate.
2 Reduce update message traffic by 20% your normal rate.
3 Reduce update message traffic by 30% your normal rate.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 5]

Yellow (Entering medium congestion state)
4 Reduce update message traffic by 40% your normal rate. 
5 Reduce update message traffic by 50% your normal rate.
6 Reduce update message traffic by 60% your normal rate.

Orange (Entering major congestion state)
7 Reduce update message traffic by 70% your normal rate. 
8 Reduce update message traffic by 80% your normal rate.
9 Reduce update message traffic by 90% your normal rate

Red (Entering critical congestion state)
10 Complete cessation of all update message traffic (flow off). 
At this Level the receiving peer might decide to bypass the 
Congested Router and pick another less optimal router for the
affected Routes.

Green (Exiting any congestion state)
0 Restore update message traffic to your normal level (flow on).
Routes redirected can be resumed to the original peer.

The receiving peer will send an INFORM message with an ACK or NACK Indicating 
it has received and understood the pacing request and will either comply with 
the it or forbid it. If the congested peer receives a NACK, it should remove 
that peer from its list of PACE capable peers but still maintaining its 
session without the use of Pacing. If the NACKing peer decides at a future 
date to re-enable the Pacing with the local peer, it will renegotiate the 
PACE capability with the local peer at that time.

B. Response Message Error Indication:

Level Description
------ -----------
0 ACK indication. The peer will comply with the 
Pacing level request.
1 NACK indication. The peer is unable to perform Pacing or comply 
with the pacing level requested.

5.3 INFORM Message Response Timer

When a congested peer sends an INFORM message with a "Request to Pace Update 
Messages (code=11)" to all peers that support the pacing feature, it will 
also start an associated INFORM Message Response Timer. Any peer that does 
not respond within a 30 second timeout period with either an ACK/NACK INFORM 
Response message, its associated session will be dropped. This is done to 
clear any PACE capable peers that are also congested to the point where their 
communication with the local congested peer is severed.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 6]

5.4 Congestion Detection Within the Router

There are at least two ways congestion could be detected and measured by a 
BGP router.

- A high CPU utilization condition 

- A Lack of available memory to accept incoming BGP messages or inability 
to get memory to successfully complete BGP processing. 

5.4.1 CPU Utilization Based Level Computation

It is recommended that when the congested router detects its inability to 
perform route calculations or accept new BGP session messages at a normal 
rate meaning the current router's CPU utilization is more than 49%, it should 
inform all its peers of a rate limiting level and slow them down accordingly. 

The recommended way for computing the Level, based on percent CPU 
Utilization, is done using the following:

Level = (((%CPU Utilization – 50) x 2) + 5) / 10).
Where, % CPU Utilization is a whole integer number. Use Integer math 
here to remove any fractional value.

Note: If computed Level is negative, go to Green and set Level = 0. If 
Level = 0 send INFORM message with pacing Level = 0, implying return 
to normal (100%) update rate.

Level implies, whatever number of messages you transmitted per second 
before, transmit (100 – (Level x 10))% of that now. 

e.g. If you transmitted 100 messages/second before. A level 2 (20%) 
reduction will cause you to transmit 80 messages/second now.
Assumption is that these messages have an average size (1500 bytes).

5.4.2 Memory Allocation Based Level Computation

It is recommended that when the congested router detects its inability to 
perform route calculations or accept new BGP session messages at a normal 
rate meaning the current router's memory allocation is more than 69%, it 
should inform all its peers of a rate limiting level and slow them down 
accordingly. 

The recommended way for computing the Level, based on percent Memory 
Allocation, is done using the following:

Level = (((%Memory Allocated – 70) x 2) + 5) / 10).
Where, % Memory Allocated is a whole integer number. Use Integer math 
here to remove any fractional value.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 7]

Note: If computed Level is negative, go to Green sub-group and set 
Level = 0. If Level = 0 send INFORM message with pacing Level = 0, 
implying return to normal (100%) update rate.

5.5 INFORM Message Throttling

Once the congested peer receives acknowledgement from another peer, it will 
send a modification INFORM message with a new Level to that peer after the 
computed pacing Level changes by at least 1 value. This will amount to no 
more than one INFORM modification message every 5 seconds. This is done to 
debounce any spiky bursts of INFORM messages to all PACE negotiated peers 
each time the computed pacing Level changes. Depending on vendor 
implementations, the internal utilization levels could change at the 
Microsecond or Millisecond rate.

6. Implementation Specific Mechanisms

The Memory Allocation and CPU Utilization Level detection algorithms 
discussed in section 5.4.1 and 5.4.2 are suggested ways one can implement 
these solutions. However, each vendor can implement a unique Memory 
Allocation and CPU Utilization Level detection algorithms that best suits 
his/her needs and will not negatively impact the overall BGP Route Update 
Pacing mechanism described in this spec. Any issues relating to internal 
implementation algorithms are outside the scope of this document.

7. Level Indicators Management MIBs

TBD

8. Security Considerations

This extension to BGP does not change the underlying security issues.

9. References

[BGP-CAP] Chandra, R., Scudder, J., "Capabilities Advertisement with 
BGP-4", draft-ietf-idr-rfc2842bis-02.txt

[DYN-CAP] Chen E., Sangli S., "Dynamic Capability for BGP-4", 
draft-ietf-idr-dynamic-cap-02.txt, October 2002.

[INFORM] Nalawade G., Scudder J., "BGPv4 INFORM Message",
draft-nalawade-bgp-inform-00.txt, December 2002

[BGP-4] Rekhter, Y., and T. Li, "A Border Gateway Protocol 4 
(BGP-4)", RFC 1771, March 1995.

Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 8]

[BGP-4-DRAFT] Rekhter, Y. and T. Li (editors), "A Border Gateway Protocol 4
(BGP-4)", Internet Draft draft-ietf-idr-bgp4-18.txt, 
January 2002.

[UNIX-NET] Stevens, R., "UNIX Network Programming, Vol 1", Second Edition
1998 Prentice Hall, Inc.

10. Author Information

Ben Abarbanel
Marconi Communications
1595 Spring Hill Road, 5th Floor
Vienna, VA 22182
Email: [email-masked]

---------------------------------
Do You Yahoo!?
Yahoo! Autos - Get free new car price quotes
--0-504351559-1027024640=:78832
Content-Type: text/html; charset=us-ascii

<P>Specific comment:
<P>Unless the rate of pacing is defined in absolute terms (as opposed to the a subjectively for e.g
<P>yellow 4: reduce update message traffic by 40% of your normal rate). It will be very difficult to verify compliance.
<P>Also in my experience, the "normal rate" varies drastically from vendor to vendor.
<P>&nbsp;
<P>General comment:
<P>&nbsp;It appears to me that here is an attempt to solve an internal scheduling or horse power problem with the help of external protocol. 
<P>This mechanism may put undue burden on the transmitting router, which is in the best interest of quick convergence in mind is sending prefixes as fast as possible. 
<P>&nbsp; <B><I>"Abarbanel, Benjamin" &lt;[email-masked]&gt;</I></B> wrote: 
<BLOCKQUOTE style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #1010ff 2px solid">Hi all:<BR><BR>Our recent discussions on this list and my recent work <BR>experiences have led me to write this draft and offer it <BR>to the IETF community. I would appreciate any comments <BR>anyone has to make.<BR><BR>Thanks in advance,<BR>Ben<BR><BR><BR><BR><BR><BR><BR><BR>Network Working Group Ben Abarbanel<BR>Internet Draft Marconi Communicatons<BR>Expiration Date: December 2002 <BR><BR><BR><BR>BGP Route Update Pacing<BR><BR>draft-abarbanel-bgp-route-update-pacing-00.txt<BR><BR><BR>1. Status of this Memo<BR><BR>This document is an Internet-Draft and is in full conformance with<BR>all provisions of Section 10 of RFC2026 except that the right to<BR>produce derivative works is not granted.<BR><BR>Internet-Drafts are working documents of the Internet Engineering<BR>Task Force (IETF), its areas, and its working groups. Note that<BR>other groups may also distribute working documents as Internet-<BR>Drafts.<BR><BR>Internet-Drafts are draft documents valid for a maximum of six months<BR>and may be updated, replaced, or obsoleted by other documents at any<BR>time. It is inappropriate to use Internet-Drafts as reference<BR>material or to cite them other than as ``work in progress.''<BR><BR>The list of current Internet-Drafts can be accessed at<BR>http://www.ietf.org/ietf/1id-abstracts.txt<BR><BR>The list of Internet-Draft Shadow Directories can be accessed at<BR>http://www.ietf.org/shadow.html.<BR><BR><BR>2. Abstract<BR><BR>This document defines a mechanism for controlling or limiting the rate at <BR>which BGP update messages are sent from one peer to the next when a BGP peer <BR>experiences internal congestion. With the introduction of new dynamic BGP <BR>protocol capabilities [CAP] message or other none BGP session destructive <BR>messages, it is necessary to limit the rate at which the BGP update messages <BR>are sent without affecting the entire BGP session and without relying on the <BR>transport (TCP) layer to do so as a normal reaction to data congestion of the <BR>TCP session across the network.<BR><BR>3. Specification of Requirements<BR><BR>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", <BR>"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be <BR>interpreted as described in RFC 2119 [RFC2119].<BR><BR><BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 2]<BR><BR><BR><BR>4. Introduction<BR><BR>This document defines a mechanism for controlling or limiting the rate at <BR>which BGP update messages are sent from one peer to the next when a BGP peer <BR>experiences internal congestion. With the introduction of new dynamic BGP <BR>protocol capabilities [CAP] message or other none BGP session destructive <BR>messages, it is necessary to limit the rate at which the BGP update messages <BR>are sent without affecting the entire BGP session and without relying on the <BR>transport (TCP) layer to do so as a normal reaction to data congestion of the <BR>TCP session across the network.<BR><BR>When a router enters a state where either its CPU utilization is maximized <BR>(reaches close to 100%) or its memory is nearly depleted (less than 10% of <BR>memory left), it cannot handle new or heavy streams of updates from its peers <BR>and at times unable to send messages to its peer in a timely manner (within <BR>30 seconds). As a consequence it cannot keep current with the topological <BR>state. In some scenarios, a non-congested peer might want to negotiate new <BR>capabilities with a congested peer. The congested peer is so degraded that <BR>its TCP session goes into significant flow control off conditions and is <BR>unable to see the new BGP messages. Peer to peer communication is severely <BR>hampered and as a result the uncongested peer will take corrective action <BR>when its hold timer expires and drops the session. The uncongested peer <BR>computes alternate routing paths that are suboptimal in distance or attribute <BR>and thus affect the forwarding decisions of all routers in this network. It <BR>is possible that the congested peer's routing (control) plane is badly <BR>degraded but its forwarding plane is at normal working level. The mistake is <BR>made by the uncongested peer since it does not see any Keepalive/Update <BR>messages before its Hold timer expires and drops the session thereby dropping <BR>all its associated routes. After routes are dropped, network instability <BR>occurs and suboptimal paths are used by the remaining peers.<BR><BR>The MinRouteAdvertisementInternal and MinAsOriginationInterval timers are <BR>inadequate since they are mostly implemented on a peer session basis and <BR>studies have shown when they are used they severely degrade route convergence <BR>time. The problem with statically defined timers (initialized during system <BR>load or session establishment phase) is that they do not adjust to peer <BR>internal dynamically changing congestion conditions. The problem with the <BR>congested peer using TCP flow control to reduce its congestion condition, is <BR>that it completely stops all incoming session traffic and thus preventing any <BR>messages of high priority nature from being seen.<BR><BR>TCP Out of Band Data or Urgent Message is one way to bypass the TCP flow <BR>control condition and allow high priority BGP messages to get to the <BR>congested peer, assuming it is able to read the session socket queue. This <BR>solution has its drawbacks as described in [UNIX-NET] chapter 21, p. 568. <BR><BR><BR><BR><BR><BR><BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 3]<BR><BR><BR><BR>This document presents a mechanism by which the BGP session of a congested <BR>router need not be degraded to the level that communication is broken with <BR>its peers. By using a peer to peer pacing mechanism which allows one peer to <BR>rate limit the number of update messages per second received from another <BR>peer, it can avoid the severe congestion conditions and process these <BR>messages at a manageable level. In addition, the uncongested peer has the <BR>knowledge to send high priority messages ahead of or in place of the normal <BR>high volume of update messages.<BR><BR>Usually, topological disturbances are spiky in nature and once they subside, <BR>the network returns to its optimum path oriented level. Whatever caused the <BR>network to become unstable, such as routers handling too much data in a small <BR>period of time or routers loosing their sessions or their links going up and <BR>down, occurs in most live network on an infrequent basis. By using the pacing <BR>mechanism as outlined in this draft, a BGP peer can prevent the serious <BR>congestion long before it is in trouble and thus ride through topological <BR>disturbances and still regain its stability without causing its sessions to <BR>drop or its routes/paths to be discarded and recomputed.<BR><BR>The assumption in this spec is that the source of most or all of the internal <BR>BGP router congestion is due to the heavy reception of update messages from <BR>neighboring peers containing large number of routes.<BR><BR><BR>5. BGP Update Pacing Mechanism<BR><BR>The BGP Update Message Pacing Mechanism is used to slow down the rate at <BR>which a peer sends update messages to another. This extension to the BGP <BR>protocol is simplified to use session non-destructive messages such as INFORM <BR>as described in [INFORM]. The pacing is performed dynamically upon congestion <BR>detection and subsidence and thus needs to use the new [INFORM] message that <BR>will not infringe on the underlying BGP protocol or all its semantics and <BR>rules.<BR><BR><BR>5.1 BGP Update Message Pacing (PACE) Dynamic Capability<BR><BR>The BGP Update Message Pacing capability is dynamically negotiated with all <BR>BGP speakers as type code=PACE (where, PACE=TBD) per TLV structures as <BR>defined in [BGP-CAP] of the OPEN message or anytime after session is <BR>established using the Dynamic Capability Message as defined [DYN-CAP] <BR>specification. All those peers that accept the PACE capability will be <BR>expected to support the new INFORM message as defined in [INFORM] <BR>specification to carry the pacing TLV structure. <BR><BR><BR><BR><BR><BR><BR><BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 4]<BR><BR><BR><BR>All PACE capable routers will provide a configuration option to their <BR>operator to enable the BGP Update Message Pacing mechanism on a per peer <BR>basis. <BR><BR>Any peer wishing to withdraw the PACE capability can do so dynamically using <BR>the Dynamic Capability message as outlined in [DYN-CAP] specification. Once <BR>withdrawn, affected peering session will remain intact but will not benefit <BR>from the performance improvements offered by the pacing mechanism.<BR><BR><BR>5.2 Use of INFORM Message<BR><BR>The INFORM message as described in [INFORM] is used to carry the rate <BR>limiting (pacing TLV) control structure to neighboring peers. This <BR>information informs the peer that the current BGP router has entered a <BR>congestion state and it is to rate limit its transmission to the level <BR>specified. <BR><BR>The INFORM message contains the following PACE TLV structure:<BR><BR>Type = Pacing Information, type=TBD <BR>Length = 2<BR>1st Byte of Value = Cmd as shown below <BR>2nd Byte of Value = Level as shown below<BR><BR>Cmd Description <BR>------- -----------<BR>11 Request to Pace update messages<BR>12 ACK Response to Pace Update messages<BR>13 NACK Response to Pace Update Messages<BR><BR>A. Request to Pace Update Messages Level Indication (code=11)<BR><BR>The rate is divided into sub levels in term of categories (Gray, Yellow, <BR>Orange, Red, and Green) to denote the level of pacing which is directly <BR>related to the level of congestion experienced within the congested peer. The <BR>colors are used as simple handles used for flagging the severity of the <BR>condition. The colors are also used as indicators for display by the NMS to <BR>identify the rate limiting levels from any neighboring peer to the operator. <BR>Associated Level Management MIBs are defined in section 6. <BR><BR>The actual pacing control is done via the sub levels within these colors. <BR><BR>Level Description<BR>------ -----------<BR>Gray (Entering minor congestion state)<BR>1 Reduce update message traffic by 10% your normal rate.<BR>2 Reduce update message traffic by 20% your normal rate.<BR>3 Reduce update message traffic by 30% your normal rate.<BR><BR><BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 5]<BR><BR><BR><BR>Yellow (Entering medium congestion state)<BR>4 Reduce update message traffic by 40% your normal rate. <BR>5 Reduce update message traffic by 50% your normal rate.<BR>6 Reduce update message traffic by 60% your normal rate.<BR><BR>Orange (Entering major congestion state)<BR>7 Reduce update message traffic by 70% your normal rate. <BR>8 Reduce update message traffic by 80% your normal rate.<BR>9 Reduce update message traffic by 90% your normal rate<BR><BR>Red (Entering critical congestion state)<BR>10 Complete cessation of all update message traffic (flow off). <BR>At this Level the receiving peer might decide to bypass the <BR>Congested Router and pick another less optimal router for the<BR>affected Routes.<BR><BR>Green (Exiting any congestion state)<BR>0 Restore update message traffic to your normal level (flow on).<BR>Routes redirected can be resumed to the original peer.<BR><BR>The receiving peer will send an INFORM message with an ACK or NACK Indicating <BR>it has received and understood the pacing request and will either comply with <BR>the it or forbid it. If the congested peer receives a NACK, it should remove <BR>that peer from its list of PACE capable peers but still maintaining its <BR>session without the use of Pacing. If the NACKing peer decides at a future <BR>date to re-enable the Pacing with the local peer, it will renegotiate the <BR>PACE capability with the local peer at that time.<BR><BR>B. Response Message Error Indication:<BR><BR>Level Description<BR>------ -----------<BR>0 ACK indication. The peer will comply with the <BR>Pacing level request.<BR>1 NACK indication. The peer is unable to perform Pacing or comply <BR>with the pacing level requested.<BR><BR><BR>5.3 INFORM Message Response Timer<BR><BR>When a congested peer sends an INFORM message with a "Request to Pace Update <BR>Messages (code=11)" to all peers that support the pacing feature, it will <BR>also start an associated INFORM Message Response Timer. Any peer that does <BR>not respond within a 30 second timeout period with either an ACK/NACK INFORM <BR>Response message, its associated session will be dropped. This is done to <BR>clear any PACE capable peers that are also congested to the point where their <BR>communication with the local congested peer is severed.<BR><BR><BR><BR><BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 6]<BR><BR><BR><BR>5.4 Congestion Detection Within the Router<BR><BR>There are at least two ways congestion could be detected and measured by a <BR>BGP router.<BR><BR>- A high CPU utilization condition <BR><BR>- A Lack of available memory to accept incoming BGP messages or inability <BR>to get memory to successfully complete BGP processing. <BR><BR>5.4.1 CPU Utilization Based Level Computation<BR><BR>It is recommended that when the congested router detects its inability to <BR>perform route calculations or accept new BGP session messages at a normal <BR>rate meaning the current router's CPU utilization is more than 49%, it should <BR>inform all its peers of a rate limiting level and slow them down accordingly. <BR><BR>The recommended way for computing the Level, based on percent CPU <BR>Utilization, is done using the following:<BR><BR>Level = (((%CPU Utilization – 50) x 2) + 5) / 10).<BR>Where, % CPU Utilization is a whole integer number. Use Integer math <BR>here to remove any fractional value.<BR><BR>Note: If computed Level is negative, go to Green and set Level = 0. If <BR>Level = 0 send INFORM message with pacing Level = 0, implying return <BR>to normal (100%) update rate.<BR><BR>Level implies, whatever number of messages you transmitted per second <BR>before, transmit (100 – (Level x 10))% of that now. <BR><BR>e.g. If you transmitted 100 messages/second before. A level 2 (20%) <BR>reduction will cause you to transmit 80 messages/second now.<BR>Assumption is that these messages have an average size (1500 bytes).<BR><BR><BR>5.4.2 Memory Allocation Based Level Computation<BR><BR>It is recommended that when the congested router detects its inability to <BR>perform route calculations or accept new BGP session messages at a normal <BR>rate meaning the current router's memory allocation is more than 69%, it <BR>should inform all its peers of a rate limiting level and slow them down <BR>accordingly. <BR><BR>The recommended way for computing the Level, based on percent Memory <BR>Allocation, is done using the following:<BR><BR>Level = (((%Memory Allocated – 70) x 2) + 5) / 10).<BR>Where, % Memory Allocated is a whole integer number. Use Integer math <BR>here to remove any fractional value.<BR><BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 7]<BR><BR><BR><BR>Note: If computed Level is negative, go to Green sub-group and set <BR>Level = 0. If Level = 0 send INFORM message with pacing Level = 0, <BR>implying return to normal (100%) update rate.<BR><BR><BR>5.5 INFORM Message Throttling<BR><BR>Once the congested peer receives acknowledgement from another peer, it will <BR>send a modification INFORM message with a new Level to that peer after the <BR>computed pacing Level changes by at least 1 value. This will amount to no <BR>more than one INFORM modification message every 5 seconds. This is done to <BR>debounce any spiky bursts of INFORM messages to all PACE negotiated peers <BR>each time the computed pacing Level changes. Depending on vendor <BR>implementations, the internal utilization levels could change at the <BR>Microsecond or Millisecond rate.<BR><BR><BR>6. Implementation Specific Mechanisms<BR><BR>The Memory Allocation and CPU Utilization Level detection algorithms <BR>discussed in section 5.4.1 and 5.4.2 are suggested ways one can implement <BR>these solutions. However, each vendor can implement a unique Memory <BR>Allocation and CPU Utilization Level detection algorithms that best suits <BR>his/her needs and will not negatively impact the overall BGP Route Update <BR>Pacing mechanism described in this spec. Any issues relating to internal <BR>implementation algorithms are outside the scope of this document.<BR><BR><BR>7. Level Indicators Management MIBs<BR><BR>TBD<BR><BR><BR>8. Security Considerations<BR><BR>This extension to BGP does not change the underlying security issues.<BR><BR><BR>9. References<BR><BR>[BGP-CAP] Chandra, R., Scudder, J., "Capabilities Advertisement with <BR>BGP-4", draft-ietf-idr-rfc2842bis-02.txt<BR><BR>[DYN-CAP] Chen E., Sangli S., "Dynamic Capability for BGP-4", <BR>draft-ietf-idr-dynamic-cap-02.txt, October 2002.<BR><BR>[INFORM] Nalawade G., Scudder J., "BGPv4 INFORM Message",<BR>draft-nalawade-bgp-inform-00.txt, December 2002<BR><BR>[BGP-4] Rekhter, Y., and T. Li, "A Border Gateway Protocol 4 <BR>(BGP-4)", RFC 1771, March 1995.<BR><BR><BR>Internet Draft draft-abarbanel-bgp-route-update-pacing-00.txt [Page 8]<BR><BR><BR><BR>[BGP-4-DRAFT] Rekhter, Y. and T. Li (editors), "A Border Gateway Protocol 4<BR>(BGP-4)", Internet Draft draft-ietf-idr-bgp4-18.txt, <BR>January 2002.<BR><BR>[UNIX-NET] Stevens, R., "UNIX Network Programming, Vol 1", Second Edition<BR>1998 Prentice Hall, Inc.<BR><BR><BR>10. Author Information<BR><BR>Ben Abarbanel<BR>Marconi Communications<BR>1595 Spring Hill Road, 5th Floor<BR>Vienna, VA 22182<BR>Email: [email-masked]<BR><BR><BR></BLOCKQUOTE><p><br><hr size=1><b>Do You Yahoo!?</b><br>
<a href="http://autos.yahoo.com/">Yahoo! Autos</a> - Get free new car price quotes
--0-504351559-1027024640=:78832--

From: "Abarbanel, Benjamin" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: My BGP Route Update Pacing Draft
Date: Thu, 18 Jul 2002 15:31:54 -0400
Sender: [email-masked]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_000_01C22E91.C60E3270
Content-Type: text/plain;
	charset="iso-8859-1"

Hi all:

   Our recent discussions on this list and my recent work 
 experiences have led me to write this draft and offer it 
 to the IETF community. I would appreciate any comments 
 anyone has to make.
 
 Thanks in advance,
 Ben

------_=_NextPart_000_01C22E91.C60E3270
Content-Type: text/plain;
	name="draft-abarbanel-bgp-update-pacing-txt.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="draft-abarbanel-bgp-update-pacing-txt.txt"

Network Working Group                                     Ben Abarbanel
Internet Draft                                            Marconi =
Communicatons
Expiration Date: December 2002                         =20

          			=09
					BGP Route Update Pacing

                    draft-abarbanel-bgp-route-update-pacing-00.txt

1. Status of this Memo

   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026 except that the right to
   produce derivative works is not granted.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six =
months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as ``work in progress.''

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

2. Abstract

This document defines a mechanism for controlling or limiting the rate =
at=20
which BGP update messages are sent from one peer to the next when a BGP =
peer=20
experiences internal congestion. With the introduction of new dynamic =
BGP=20
protocol capabilities [CAP] message or other none BGP session =
destructive=20
messages, it is necessary to limit the rate at which the BGP update =
messages=20
are sent without affecting the entire BGP session and without relying =
on the=20
transport (TCP) layer to do so as a normal reaction to data congestion =
of the=20
TCP session across the network.

3. Specification of Requirements

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", =
"SHOULD",=20
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are =
to be=20
interpreted as described in RFC 2119 [RFC2119].

=20

Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 2]

=20

4. Introduction

This document defines a mechanism for controlling or limiting the rate =
at=20
which BGP update messages are sent from one peer to the next when a BGP =
peer=20
experiences internal congestion. With the introduction of new dynamic =
BGP=20
protocol capabilities [CAP] message or other none BGP session =
destructive=20
messages, it is necessary to limit the rate at which the BGP update =
messages=20
are sent without affecting the entire BGP session and without relying =
on the=20
transport (TCP) layer to do so as a normal reaction to data congestion =
of the=20
TCP session across the network.

When a router enters a state where either its CPU utilization is =
maximized=20
(reaches close to 100%) or its memory is nearly depleted (less than 10% =
of=20
memory left), it cannot handle new or heavy streams of updates from its =
peers=20
and at times unable to send messages to its peer in a timely manner =
(within=20
30 seconds). As a consequence it cannot keep current with the =
topological=20
state. In some scenarios, a non-congested peer might want to negotiate =
new=20
capabilities with a congested peer. The congested peer is so degraded =
that=20
its TCP session goes into significant flow control off conditions and =
is=20
unable to see the new BGP messages. Peer to peer communication is =
severely=20
hampered and as a result the uncongested peer will take corrective =
action=20
when its hold timer expires and drops the session. The uncongested peer =

computes alternate routing paths that are suboptimal in distance or =
attribute=20
and thus affect the forwarding decisions of all routers in this =
network. It=20
is possible that the congested peer's routing (control) plane is badly=20
degraded but its forwarding plane is at normal working level. The =
mistake is=20
made by the uncongested peer since it does not see any Keepalive/Update =

messages before its Hold timer expires and drops the session thereby =
dropping=20
all its associated routes. After routes are dropped, network =
instability=20
occurs and suboptimal paths are used by the remaining peers.

The MinRouteAdvertisementInternal and MinAsOriginationInterval timers =
are=20
inadequate since they are mostly implemented on a peer session basis =
and=20
studies have shown when they are used they severely degrade route =
convergence=20
time. The problem with statically defined timers (initialized during =
system=20
load or session establishment phase) is that they do not adjust to peer =

internal dynamically changing congestion conditions. The problem with =
the=20
congested peer using TCP flow control to reduce its congestion =
condition, is=20
that it completely stops all incoming session traffic and thus =
preventing any=20
messages of high priority nature from being seen.

TCP Out of Band Data or Urgent Message is one way to bypass the TCP =
flow=20
control condition and allow high priority BGP messages to get to the=20
congested peer, assuming it is able to read the session socket queue. =
This=20
solution has its drawbacks as described in [UNIX-NET] chapter 21, p. =
568. =20

Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 3]

=20

This document presents a mechanism by which the BGP session of a =
congested=20
router need not be degraded to the level that communication is broken =
with=20
its peers. By using a peer to peer pacing mechanism which allows one =
peer to=20
rate limit the number of update messages per second received from =
another=20
peer, it can avoid the severe congestion conditions and process these=20
messages at a manageable level. In addition, the uncongested peer has =
the=20
knowledge to send high priority messages ahead of or in place of the =
normal=20
high volume of update messages.

Usually, topological disturbances are spiky in nature and once they =
subside,=20
the network returns to its optimum path oriented level. Whatever caused =
the=20
network to become unstable, such as routers handling too much data in a =
small=20
period of time or routers loosing their sessions or their links going =
up and=20
down, occurs in most live network on an infrequent basis. By using the =
pacing=20
mechanism as outlined in this draft, a BGP peer can prevent the serious =

congestion long before it is in trouble and thus ride through =
topological=20
disturbances and still regain its stability without causing its =
sessions to=20
drop or its routes/paths to be discarded and recomputed.

The assumption in this spec is that the source of most or all of the =
internal=20
BGP router congestion is due to the heavy reception of update messages =
from=20
neighboring peers containing large number of routes.

5. BGP Update Pacing Mechanism

The BGP Update Message Pacing Mechanism is used to slow down the rate =
at=20
which a peer sends update messages to another. This extension to the =
BGP=20
protocol is simplified to use session non-destructive messages such as =
INFORM=20
as described in [INFORM]. The pacing is performed dynamically upon =
congestion=20
detection and subsidence and thus needs to use the new [INFORM] message =
that=20
will not infringe on the underlying BGP protocol or all its semantics =
and=20
rules.
=20

5.1 BGP Update Message Pacing (PACE) Dynamic Capability

The BGP Update Message Pacing capability is dynamically negotiated with =
all=20
BGP speakers as type code=3DPACE (where, PACE=3DTBD) per TLV structures =
as=20
defined in [BGP-CAP] of the OPEN message or anytime after session is=20
established using the Dynamic Capability Message as defined [DYN-CAP]=20
specification. All those peers that accept the PACE capability will be=20
expected to support the new INFORM message as defined in [INFORM]=20
specification to carry the pacing TLV structure.=20

Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 4]

=20

All PACE capable routers will provide a configuration option to their=20
operator to enable the BGP Update Message Pacing mechanism on a per =
peer=20
basis.=20

Any peer wishing to withdraw the PACE capability can do so dynamically =
using=20
the Dynamic Capability message as outlined in [DYN-CAP] specification. =
Once=20
withdrawn, affected peering session will remain intact but will not =
benefit=20
from the performance improvements offered by the pacing mechanism.

5.2 Use of INFORM Message

The INFORM message as described in [INFORM] is used to carry the rate=20
limiting (pacing TLV) control structure to neighboring peers. This=20
information informs the peer that the current BGP router has entered a=20
congestion state and it is to rate limit its transmission to the level=20
specified.=20

The INFORM message contains the following PACE TLV structure:
=20
   Type =3D Pacing Information, type=3DTBD=20
   Length =3D 2
   1st Byte of Value =3D Cmd as shown below=20
   2nd Byte of Value =3D Level as shown below

 Cmd       Description=20
-------    -----------
  11       Request to Pace update messages
  12       ACK Response to Pace Update messages
  13       NACK Response to Pace Update Messages

A. Request to Pace Update Messages Level Indication (code=3D11)

The rate is divided into sub levels in term of categories (Gray, =
Yellow,=20
Orange, Red, and Green) to denote the level of pacing which is directly =

related to the level of congestion experienced within the congested =
peer. The=20
colors are used as simple handles used for flagging the severity of the =

condition. The colors are also used as indicators for display by the =
NMS to=20
identify the rate limiting levels from any neighboring peer to the =
operator.=20
Associated Level Management MIBs are defined in section 6.=20

The actual pacing control is done via the sub levels within these =
colors.=20
 =20
    Level     Description
   ------     -----------
    Gray      (Entering minor congestion state)
         1       Reduce update message traffic by 10% your normal rate.
         2       Reduce update message traffic by 20% your normal rate.
         3       Reduce update message traffic by 30% your normal rate.

                                                   =20
Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 5]

      Yellow     (Entering medium congestion state)
         4       Reduce update message traffic by 40% your normal rate. =

         5       Reduce update message traffic by 50% your normal rate.
         6       Reduce update message traffic by 60% your normal rate.

      Orange     (Entering major congestion state)
         7       Reduce update message traffic by 70% your normal rate. =

         8       Reduce update message traffic by 80% your normal rate.
         9       Reduce update message traffic by 90% your normal rate

        Red      (Entering critical congestion state)
        10       Complete cessation of all update message traffic (flow =
off).=20
                 At this Level the receiving peer might decide to =
bypass the=20
                 Congested Router and pick another less optimal router =
for the
                 affected Routes.

    Green     (Exiting any congestion state)
         0       Restore update message traffic to your normal level =
(flow on).
              Routes redirected can be resumed to the original peer.

The receiving peer will send an INFORM message with an ACK or NACK =
Indicating=20
it has received and understood the pacing request and will either =
comply with=20
the it or forbid it. If the congested peer receives a NACK, it should =
remove=20
that peer from its list of PACE capable peers but still maintaining its =

session without the use of Pacing.  If the NACKing peer decides at a =
future=20
date to re-enable the Pacing with the local peer, it will renegotiate =
the=20
PACE capability with the local peer at that time.

B. Response Message Error Indication:

    Level     Description
   ------     -----------
0	 ACK indication. The peer will comply with the=20
     		     Pacing level request.
1	 NACK indication. The peer is unable to perform Pacing or comply =20
        with the pacing level requested.

5.3 INFORM Message Response Timer
  =20
When a congested peer sends an INFORM message with a "Request to Pace =
Update=20
Messages (code=3D11)" to all peers that support the pacing feature, it =
will=20
also start an associated INFORM Message Response Timer. Any peer that =
does=20
not respond within a 30 second timeout period with either an ACK/NACK =
INFORM=20
Response message, its associated session will be dropped. This is done =
to=20
clear any PACE capable peers that are also congested to the point where =
their=20
communication with the local congested peer is severed.

=20

Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 6]

=20

5.4 Congestion Detection Within the Router

There are at least two ways congestion could be detected and measured =
by a=20
BGP router.

- A high CPU utilization condition=20

- A Lack of available memory to accept incoming BGP messages or =
inability=20
  to get memory to successfully complete BGP processing.=20

5.4.1 CPU Utilization Based Level Computation

It is recommended that when the congested router detects its inability =
to=20
perform route calculations or accept new BGP session messages at a =
normal=20
rate meaning the current router's CPU utilization is more than 49%, it =
should=20
inform all its peers of a rate limiting level and slow them down =
accordingly.=20

The recommended way for computing the Level, based on percent CPU=20
Utilization, is done using the following:

   Level =3D (((%CPU Utilization =96 50) x 2) + 5) / 10).
   Where, % CPU Utilization is a whole integer number. Use Integer math =
=20
   here to remove any fractional value.

  Note: If computed Level is negative, go to Green and set Level =3D 0. =
If  =20
         Level =3D 0 send INFORM message with pacing Level =3D 0, =
implying return=20
         to normal (100%) update rate.

Level implies, whatever number of messages you transmitted per second=20
before, transmit (100 =96 (Level x 10))% of that now.=20

  e.g. If you transmitted 100 messages/second before. A level 2 (20%)=20
       reduction will cause you to transmit 80 messages/second now.
          Assumption is that these messages have an average size (1500 =
bytes).

5.4.2 Memory Allocation Based Level Computation

It is recommended that when the congested router detects its inability =
to=20
perform route calculations or accept new BGP session messages at a =
normal=20
rate meaning the current router's memory allocation is more than 69%, =
it=20
should inform all its peers of a rate limiting level and slow them down =

accordingly.=20

The recommended way for computing the Level, based on percent Memory=20
Allocation, is done using the following:

   Level =3D (((%Memory Allocated =96 70) x 2) + 5) / 10).
    Where, % Memory Allocated is a whole integer number. Use Integer =
math =20
    here to remove any fractional value.

Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 7]

   Note: If computed Level is negative, go to Green sub-group and set=20
         Level =3D 0. If Level =3D 0 send INFORM message with pacing =
Level =3D 0,=20
         implying return to normal (100%) update rate.
=20

5.5 INFORM Message Throttling

Once the congested peer receives acknowledgement from another peer, it =
will=20
send a modification INFORM message with a new Level to that peer after =
the=20
computed pacing Level changes by at least 1 value. This will amount to =
no=20
more than one INFORM modification message every 5 seconds. This is done =
to=20
debounce any spiky bursts of INFORM messages to all PACE negotiated =
peers=20
each time the computed pacing Level changes. Depending on vendor=20
implementations, the internal utilization levels could change at the=20
Microsecond or Millisecond rate.
 =20

6. Implementation Specific Mechanisms

The Memory Allocation and CPU Utilization Level detection algorithms=20
discussed in section 5.4.1 and 5.4.2 are suggested ways one can =
implement=20
these solutions. However, each vendor can implement a unique Memory=20
Allocation and CPU Utilization Level detection algorithms that best =
suits=20
his/her needs and will not negatively impact the overall BGP Route =
Update=20
Pacing mechanism described in this spec. Any issues relating to =
internal=20
implementation algorithms are outside the scope of this document.

=20
7. Level Indicators Management MIBs

   TBD

8. Security Considerations

   This extension to BGP does not change the underlying security =
issues.

9. References
  =20
   [BGP-CAP] Chandra, R., Scudder, J., "Capabilities Advertisement with =

             BGP-4", draft-ietf-idr-rfc2842bis-02.txt

       [DYN-CAP]         Chen E., Sangli S.,  "Dynamic Capability for =
BGP-4",=20
            draft-ietf-idr-dynamic-cap-02.txt, October 2002.

   [INFORM] Nalawade G., Scudder J., "BGPv4 INFORM Message",
            draft-nalawade-bgp-inform-00.txt, December 2002

   [BGP-4]  Rekhter, Y., and T. Li, "A Border Gateway Protocol 4=20
                              (BGP-4)", RFC 1771, March 1995.

Internet Draft      draft-abarbanel-bgp-route-update-pacing-00.txt    =
[Page 8]

   [BGP-4-DRAFT] Rekhter, Y. and T. Li (editors), "A Border Gateway =
Protocol 4
                          (BGP-4)", Internet Draft =
draft-ietf-idr-bgp4-18.txt,=20
                           January 2002.

   [UNIX-NET]  Stevens, R., "UNIX Network Programming, Vol 1", Second =
Edition
                            1998 Prentice Hall, Inc.
  =20

10. Author Information

   Ben Abarbanel
   Marconi Communications
   1595 Spring Hill Road, 5th Floor
   Vienna, VA 22182
   Email: [email-masked]

------_=_NextPart_000_01C22E91.C60E3270--

Date: Thu, 18 Jul 2002 16:04:40 +0200
From: [email-masked]
To: Bill Fenner <[email-masked]>
Cc: [email-masked]
Subject: Re: FSM words for the draft-18 
Sender: [email-masked]

This is a multi-part message in MIME format.
--------------149A43A8660ACEFC11F84974
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii

All,

I have reviewed the FSM document below, upto the Active state. I have added my
comments inline in the text.
If it felt that it makes sense to continue the review in this way, please let
me know.

Kind regards,
  Hans De Vleeschouwer.

Bill Fenner wrote:

> Sue asked me to forward this since it didn't appear on the list.
> Sorry if this causes duplicates.
>
>   Bill
>
> ----- Begin forwarded message:
>
> From: Susan Hares <[email-masked]>
> Subject: FSM words for the draft-18
> Date: Wed, 17 Jul 2002 23:24:40 -0400
> To: [email-masked]
> Cc: yakov Rekhter <[email-masked]>, [email-masked],
>     [email-masked]
>
> I would like to propose that section 8 be replaced with the
> attached words.
>
> Sue Hares
>
>   ------------------------------------------------------------------------
>
>    FSM-words-05b.txtName: FSM-words-05b.txt
>                     Type: Plain Text (text/plain)
>
>   ------------------------------------------------------------------------

--------------149A43A8660ACEFC11F84974
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=us-ascii;
 name="FSM-words-05b.txt"
Content-Disposition: inline;
 filename="FSM-words-05b.txt"

> General questions/doubts/remarks: 
>
> - Several versions of chapter 8 are floating around.
>   The version below was posted by Bill Fenner on July 17th.   
>    
> - I have placed comments inline in the text. The comment
>   lines start with a >. 
>
> - For timers I noticed that several syntaxes are being used.
>   e.g. 
>     Connect Retry Timer
>     ConnectRetry Timer
>     ConnectRetryTimer
>
>   As the second notation seems most used, I assumed that this was    
>   the correct notation.
>
> - for timers I suggest in the text to use consistently:
>   'start timer' and 'stop timer'. 
>    
> - Section 8.1 decribes all possible events. Would it not be good
>   Also to have a section describing the purpose of eah timer.
>
>

Note: (this is version 5 of the changes to
       the text.) 

8.0	BGP Finite State Machine

This section specifies the BGP operation in terms of a 
Finite State Machine (FSM).  The section falls into 2 
parts:

    1)	Description of Events for the State machine (section 
        8.1
    2)	Description of the FSM (section 8.2)

Session Attributes required for each connection are; 

   1)	state
   2)	Connect Retry timer

>>
> Naming Consistency:  ConnectRetry timer
>>

   3)	Hold timer
   4)	Hold time
   5)	Keepalive timer

>>
> Naming Consistency: KeepAlive timer
>>

8.1 Events for the BGP FSM

8.1.1   Administrative Events (events 1-5)

Please note that only Event 1 (manual start) and Event 2 
(manual stop) are mandatory administrative events. All 
other administrative events are optional.

 Event1: Manual start 

	   Definition: Administrator manually starts peer
                       connection
>>
> Syntax: Administrator -> Local system administrator
>         For consistency with Event2 below. 
>>

	   Status: 	Mandatory

 Event2: Manual stop

	   Definition: Local system administrator manually 
                       stops the peer connection.

	   Status: 	mandatory

 Event3: Automatic Start

 	   Definition: Local system automatically starts the 
                       BGP peer session

	   Status:     Optional depending on local system

 Event4: Manual start with passive Transport Establishment

	   Definition: Administrator manually start the peer
>>
> Syntax: Administrator maually start->
>          Local system administrator manually starts
>
>>
                       Connection, but has the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing the connection.

	   Status:     Optional depending on local system

 Event5: Automatic start with passive Transport establishment

	  Definition:  Local system automatically starts the
                       BGP session with the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing a connection.

	  Status:      Optional depending on local system use
		       of a passive connection.

 Event6: Automatic start with bgp_stop_flap option set

	   Definition: Local system automatically starts the 
                       BGP peer session with persistent peer 
                       oscillation damping enabled.  The exact 
                       method of damping persistent peer 
                       oscillations is left up to the 
                       implementation.   These methods of 
                       damping persistent BGP adjacency 
                       flapping are outside the scope of this 
                       document.

           Status: 	Optional, used only if the bgp peer has 
	  	        Enabled a method of damping persistent
		        BGP peer flapping.

 Event7: Auto stop

	   Definition: Local system automatically stops the
                       BGP peer session.  
	
	   Status:     Optional depending on local system

8.1.2 Timer Events (events 8-11)

 Event8:  idle Hold timer expires 

>>
> Naming Consistency: idle Hold timer --> IdleHold timer
>>

           Definition: Idle Hold timer expires.  The Idle 
                       Hold Timer is only used when persistent  
>>
> Naming Consistency: idle Hold timer --> IdleHold timer
>>

                       BGP oscillation damping functions are 
                       enabled.  

           Status: 	Optional.  Used when persistent
			BGP peer oscillation damping functions
			are enabled. 

 Event9: connect retry timer expires

>>
> Naming Consistency: connect retry timer --> ConnectRetry timer
>>
           Definition: An event triggered by the expiration of 
                       the ConnectRetry timer.
  
	   Status:     Mandatory

 Event10: hold time expires

	   Definition: An event generated when the HoldTimer 
                       expires.

>>
> Naming COnsistency: HoldTimer --> Hold timer
>>

	   Status:     Mandatory

 Event11: keepalive timer expires

>>
> Naming Consistency: Keepalive --> KeepAlive
>>

	   Definition: A periodic event generated due to the 
                       expiration of the KeepAlive Timer. 
 
	   Status:     Mandatory

 Event12: DelayBGP Open timer expires [changed]

>>
> NamingConsistency: DelayBGP Open -> DelayBGPOpen 
>>
	
	   Definition: A timer that delays sending of the BGP
	               Open message for n seconds after the
                       Transport connection has been completed. 

	   status:     Optional

8.2.3) Tranport Message based (13-16) 

 Event13: Transport Connection Indication & valid remote peer[changed]

	   Definition: Event indicating that transport
                       Request valid source IP address and TCP 
                       port, and valid destination IP address 
                       and TCP Port.  The definition of 
                       invalid Source, and invalid Destination 
                       IP address is left to the implementation.
                       BGP's destination port should be port 
                       179 as defined by IANA.

>>
> Comment:
>   If it is the intention of the text to allow BGP to use other
>   transport protocol besides TCP, then this text will probably 
>   need to be modified a bit in the lines of:
>
>           Event indicating that a transport connection is 
>           initiated by a peer with a valid source and 
>           destination address
>                      
>           The definition of valid source, and valid destination 
>           address is left to the implementation.
>                                              
>           If BGP is using TCP as transport protocol, the event 
>           is denoted by the local system receiving a TCP SYN 
>           on port  179 as defined by IANA from a peer, with a valid 
>           source and destination IP address, and a valid source
>            TCP source port.
>
>>

	   Status:     Mandatory

 Event14: RCV Transport Connection Indication and Invalid Source or 
          Destination [changed]

           Definition: Transport request received with either
 		       an invalid source address or port
		       number or an invalid destination 
                       address or port number. BGP destination 
                       port  number should be 179 as defined 
                       by IANA.

	    Status: 	 Mandatory
	
>>
> Same comment as for Event13. Here the text could become something like:
>
>                       Event indicating that a transport connection is 
>                       initiated by a peer with an invalid source and/or
>                       destination address
>                       
>                       The definition of invalid source, and invalid
>                       destination address is left to the implementation.
>                                              
>                       If BGP is using TCP a transport protocol, the event 
>                       is denoted by the local system receiving a TCP SYN 
>                       on port 179 with an invalid source IP address, 
>                       source TCP port, or destination IP address.
>>

 Event15: Transport Connection request sent received an ACK.

           Definition: Local system's request to establish a transport
		       Connection to the remote side received
		       an ACK.
>>
> Typo:  remote side received an ACK -> remote side, and received an ACK.
>>

	    Status:    Mandatory

 Event16: Transport Connection Confirmed [changed]

	   Definition: The local system has received a Confirmation that
                       the transport connection has been established by
		       the remote site. 

	  Status:      Mandatory

 Event17: Transport connection fails [changed]

           Definition: This BGP peer receives a transport
		       connection failure notice.  This
		       connection notice could be caused by a
		       Transport disconnect message or a
		       Timeout in the transport session.

                       If this connection is using TCP, the
		       remote BGP peer's TCP machine could
		       have sent a FIN.  The local peer would
		       respond with a FIN-ACK. Another
		       alternative is that the local peer
		       indicated a timeout in the TCP session
		       and downed the connection.
>>
> Comment: what about the TCP signal RST. Should this not also be
>          considered as an error?
>>

	    Status:    Mandatory

8.1.4 BGP Messages (18-27)

 Event18: BGPOpen

           Definition:  An event indicating that a valid Open 
>>
> Naming Consistency:  Open -> BGP Open
>>
			message has been received.

	   Status: 	Mandatory

 Event19: BGPOpen with BGP Delay Open Timer running [changed]

           Definition: An event indicating that a valid Open 
                       Message has been successful 
                       established for a peer that is 
                       currently delaying the sending of an 
                       BGP Open message. 
>>
> 
>"...for a peer that is currently delaying the sending of an 
> BGP Open message." 
>
> In fact I guess it is not the peer who is delaying the sending of 
> the BGP Open, but the local system. proposed rephrase"
>
>"...for a peer for which the local peer is currently delaying the
>    sending of a BGP Open message." 
>
>>

	    Status:    Optional

 Event20: BGPHeaderErr

	   Definition: BGP message header is not valid.  

	   Status:     Mandatory

 Event21: BGPOpenMsgErr 
	   Definition: An BGP Open message has been received 
                        with errors. 
 
	   Status: 	Mandatory

 Event22: Open collision discard

           Definition: An event generated administratively 
                       when a connection Collision has been 
                       detected while processing an incoming  
                       Open message. This connection has been 
>>
> Naming Consystency: Open message -> BGP Open message
>>
                       selected to disconnected.  See section 

>>
> Typo: selected to disconnected -> selected to be disconnected.
>>

                       6.8 for more information on collision
	               detection.  

                       Event 22 is an administrative could 
                       occur if FSM is implemented as two 
                       linked state machines.  

>>
> Typo:  Event 22 is an administrative could occur if FSM -> 
>        Event22 is an administrative event that could occur if the FSM
>>

	  Status:      Optional

 Event23: NotifMsgVerErr

	    Definition: An event is generated when a 
                        NOTIFICIATION message with "version   
                        error" is received.

	    Status:     Mandatory

 Event24: NotifMsg

	    Definition: An event is generated when a 
                        NOTIFICATION messages is received and
                        the error code is anything but 	 
			"version error".

	     Status:	Mandatory
		        
 Event25: KeepAliveMsg

	    Definition: An event is generated when a KEEPALIVE
			Message is received.

		Status: Mandatory

 Event26: UpdateMsg

	    Definition: An event is generated when a valid 
			Update Message is received.

		Status: Mandatory

 Event27: UpdateMsgErr

	    Definition: An event is generated when an invalid
			Update message is received.

		Status:	Mandatory

8.2) Description of FSM

Idle state

   Initially BGP is in the Idle state.   

>>
>
> In fact BGP itself is not in the Idle state, but the FSM for a given 
> peer. Therefore I would suggest a rephrase to:
>
>   Initially the BGP FSM for a peer is in the Idle state. 
>
>
>>

   In this state BGP refuses all incoming BGP connections.  No 
   resources are allocated to the peer.    In response to a 
   manual start event(Event1) or an automatic start 
   event(Event3), the local system 
      -	initializes all BGP resources, 

>>
>
> The text says that "No resources are allocated to the peer. "
> Therefore i would suggest to rephrase this bullet as:
>
>     - allocates and initializes all BGP resources,
>
>  This also matches better with the text furtheron where 
>  sometimes I read " release all BGP resources and return to Idle state"
>>

      -	sets the Connect retry counter to zero
      -	starts the ConnectRetry timer with initial value, 
      -	initiates a transport connection to the other BGP 
        peer, 
      -	listens for a connection that may be initiated by 
        the remote BGP peer, and
        [Action A in the FSM table]
      -	changes its state to connect.

>>
>1. Naming Consistency:
>      -	sets the ConnectRetryCnt to zero
>      -	changes its state to Connect.
>
>
>2. I see nowhere a place where the value of the ConnectRetry timer
>   is changed. Is then not enough to say
>     - start the ConnectRetry timer.
>>

  An manual stop event (Event2) is ignored in the Idle state.

  In response to a manual start event with the passive Transport
  flag (Event 4) or automatic start with the passive Transport flag 
  (Event 5), the local system:
      -	initializes all BGP resources,
>>
> Same remark:  suggested new text:
>  
>     - allocates and initializes all BGP resources,
>>

      -	sets the connect retry count to zero,
      -	start the Connect Retry timer with initial value, 
>>
> Naming Consistency: 
>      -	sets the ConnectRetryCnt to zero,
>      -	start the ConnectRetry timer with initial value, 
>>

      -	listens for a connection that may be initiated by 
        the remote peer
        [Action B in the FSM table] 
      -	changes its state to Active.

  The exact value of the ConnectRetry timer is a local 
  matter, but it should be sufficiently large to allow TCP 
  initialization.   

  If a persistent BGP peer oscillation damping function is 
  enabled, two additional events may occur within Idle state:  
      -	Automatic start with bgp_stop_flap set [Event6],
      -	Idle Hold Timer expired [Event 8].

  The method of preventing persistent BGP peer oscillation is 
  outside the scope of this document.

  Any other events [Events 9-27] received in the Idle state, 
  are noted by the MIB processing as FSM Errors [action V]
  and the local peer stays in the Idle State.

Connect State:

  In this state, BGP is waiting for the transport protocol 
  connection to be completed.

  If the transport connection succeeds [Event 15 or 
  Event 16], the local system checks the "Delay Open 
  Flag".  If the Delay Open flag is set, the local system: 
     -	clears the ConnectRetry timer, 
     -	set the BGP Open Delay timer to the initial 
        value.

>>
> Naming Consistency:
>    
>   - stop the ConnectRetry timer, 
>   - start the BGPOpenDelay timer with its initial value
>
>
>>

        [Action ZZ in FSM table]

  If the Delay Open flag is not set, the local system: 
     - clears the ConnectRetry timer, 
     - completes BGP initialization
     - send an Open message to its peer,
     - sets Hold timer to a large value,
>>
> Naming Consistency:
>
>    - stop the ConnectRetry timer,
>    - send a BGP Open message to its peer,
>    - start the Hold timer with a large value
>
>>

     - Change the state to Open Sent 
   [Action H in the FSM table]

>>
> In all other cases the state change is not part of the action itself.
> Should therefore the order not be reversed to:
>
>  [Action H in the FSM table], and
>  - Change the state to Open Sent 
>   
>>

  A hold timer value of 4 minutes is suggested.

>>
> Question:
>The text above assumes the same behavior for both event 15 and 16. In 
>reality I  wonder whether most implementations would not use 2 linked
> FSM machines for this. In this case a scenario could be:
>- if 15 arrives: continue as described in the text;
>- if 16 arrives: As this is in fact a new connection that comes in 
>orginated by the remote peer, create a new FSM.
>  The new FSM then eithers starts the delay timer and goes into connect 
>state, or sends an OPEN message and goes into OPEN sent state.
>
>This behavior is it seems also more in line with section 6.8 of the BGP RFC 
>on collisions.  
>
>Would it make sense to work this out any further?
>
>>

  If the Open Delay timer expires [Event 12] in the connect 
  state,
>>
> Naming consistency BGPOpenDelay
>>

     - send an Open message to its peer,
     - set the Hold timer to a large value,
>>
> Naming consistency:
> - start the Hold timer with a large value
>>

	[Action H in the FSM Table], and
     - change the state to Open Sent. 

  If the BGP port receives a Transport connection indication 
  [Event 13], the Transport connection is processed (actions AA or
  AB in the FSM table) and the connection remains
  in the connected state. 

>>
> If it is the intention to make this text independent of the Transport
> protocol, it might be better to remove the term port:
>
>    If a Transport connection indication [Event13] is received, the event 
>    is processed (actions AA orAB in the FSM table) and the connection
>    remains in the Connect state. 
>>

  If the transport connection receives an Transport indication
  that is invalid or unconfigured. [Event 14]:
     - the transport connection is rejected.
	[Action L in the FSM table]
>>
> This seems to me like a strange case. We are in the Connect state for
> a session. The session is identified sofar with a source and destination 
> IP address and (in case of TCP the correct TCP port). 
>
> If a Transport indication would come in that either has invalid info
> or which is not configured, I can not really see how this event could be
> 'dropped' inside this FSM instance. I would assume that this request
> would be treated outside the scope of any peer FSM, in the
> 'BGP-main-session'
>
>>

  If the transport connection fails (timeout or transport
  disconnect) [Event17], the local system:
      - restarts the ConnectRetry timer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
     [Action G in the FSM table]
      - changes its state to Active. 

  If an Open is received with the BGP Delay Open timer is 
  running [Event 19], the local system:
	- clears the connect retry timer (cleared to zero),
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message

>>
>
> 1. I think an action is missing here: The setting of the Hold timer.
>
> 2. For naming consistency: 
>
>       - stops the ConnectRetry timer
>	- completes the BGP initialization, 
>	- Stops the BGPOpenDelay timer
>	- Sends an Open message
>       
>     ? - set the hold timer to a large value?
>>

	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 The start events [Event 1, 3-6] are ignored in connect 
 state.

 A manual stop event[Event2], the local system:
       - drops the transport connection, 
       - releases all BGP resources,
       - sets connect retry count to zero
       - resets the connect retry timer (sets to zero)
      [Action Z in the FSM table]
       - goes to Idle state.

>>
>  Would it not be enough to say:
>
>  If a manual stop event[Event2] is received, the local system:
>    - drops the transport connection, 
>    - stops all running timers,
>    - releases all BGP resources,    
>      [Action Z in the FSM table], and
>    - goes to Idle state.
>
>   Note that it is indicated in the idle state that
>   - the connct retry count is set to 0; and
>   - the ConnectRetry timer is started with its inital value.
>
>   Also note that I moved the clearing of the resources to the end
>   as, I guess , resources are needed to clear the timers.
>
>>

  In response to the ConnectRetry timer expired event(Event 
  9), the local system: 
      - Sets the MIB FSM error information with ConnectRetry 
        expired, 
      - drops the transport connection 
      - restarts the ConnectRetry timer
      - initiates a transport connection to the other BGP 
        peer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
      [Action O in the FSM table]
      - stays in Connect state.

 In response to any other events [Events 7-8, 10-11, 18, 20-
 27] the local system:

     - resets the connect retry timer (sets to zero),
     - drops the Transport connection,
     - release all BGP resources, 
     - increments the ConnectRetryCnt by 1,
     - [optionally] performs bgp peer oscillation damping, and
     [Action D in the FSM table],
     - goes to Idle state.

>>
> 
> Note that in the Idle state 'all resources' for this peer will
> be re-allocated and initialised.
> 
> So, with the current definition of the Idle state you will not be
> able to increment the connectRetryCnt if you move into Idle state.
> A solution might be to create a new init-state where resources are
> allocated an intialised, and which is then braught into idle state.
> In this way the resources intialisation can be removed from the Idle
> state and hence the ConnectRetryCnt can be kept.
>
>>

Active State:

 In this state BGP is trying to acquire a peer by listening 
 for and accepting a transport protocol connection.  

 A transport connection succeeds [Event 15 or Event 16], the 
 local system: process the transport connection flags
	- If the BGP delay open flag is set:
          o clears the ConnectRetry timer, 
          o completes the BGP initialization,
          o sets the BGP delay Open timer
         [Action ZZ]
>>
>
>Naming consistency:
>          o stops the ConnectRetry timer, 
>          o completes the BGP initialization,
>          o starts the BGPOpenDelay timer
>
>> 

        - If the BGP delay open flag is not set:
          o clears the ConnectRetry timer,
          o completes the BGP initialization, 
          o sends the Open message to it's peer, 
          o sets its Hold timer to a large value, 
         [Action H in the FSM table]
          and changes its state to OpenSent.  

>>
>Naming consistency
>
>          o stops the ConnectRetry timer,
>          o sends the BGP Open message to it's peer, 
>          o start the Hold timer with a large value, 
>>

 A Hold timer value of 4 minutes is suggested.  

 If the local system receives a valid Transport Indication
 [Event 13], the local system processes the Transport flags
  (actions aa or ab in FSM section 2.3.4).  

>>
>
> This way of describing actions, means that the complete sepcification
> for basic BGP is not longer complete without the FSM document.
>
> Is this the way the RFC is intended to evolve?
>>

 If the local system receives a Transport indication
 that is invalid for this connection [Event 14]:
      - the transport connection is rejected
        [Action L in the FSM table]

>>
> Same comment as for [event 14] in the previous state.
>>

 If the local system receives a Transport connection
 failed [Event 17] (timeout or receives Transport
 disconnect), the local system will:
	- set Transport disconnect in the MIB reason code,
	- restart ConnectRetry timer (with initial value)
	- release all BGP resources
	- Acknowledge the drop of Transport connection if
          transport disconnect (If TCP, send a FIN ACK),
	- Increment ConnectRetryCnt by 1,
	- perform the BGP peer oscillation damping process [2]
	[Action Y in the FSM table]

>>
>1.  To what state is the FSM going here? As all BGP resources are released
>    it seems to me that we can do little more then to go to Idle state?
>
>2. In case of TCP a RST signal is also possible I guess. In this case
>   no FIN ACK is needed.
>
>> 

 If the local system has the Delay Open timer expired [event 
 12] local system: 
        - clears the Connect Retry timer (set to zero),
	- stops and clears the Delay Open timer (set to zero) 
        - completes the BGP initialization,
        - sends the Open message to it's remote peer,
        - sets it's Hold timer to a large value,
        [Action H in the FSM table]
        - and set the state to Open Confirm.

>>
> 1. Naming consistency (not repeated in detail hereafter.
>
> 2. Should we not move to the state Open Sent? In the State Open Confirm
>    the OPEN message from the peer will not longer be accepted.
>>

 A Hold timer value of 4 minutes is also suggested for this 
 state transition.

 If an Open is received with the BGP Delay Open timer is 
 running [Event 19], the local system 
	- clears the connect retry timer (cleared to zero),
	- Stops and clears the BGP Open Delay timer  
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message
	- Set the Hold timer to a large value (4 minutes),
	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 In response the ConnectRetry timer expired event[Event9], 
 the local system:
        - restarts the ConnectRetry timer (with initial value),
        - initiates a transport connection to the other BGP 
          peer,
        - Continues to listen for transport connection that may be 
          initiated by remote BGP peer, 
        [Action F in the FSM table]
	- and changes its state to Connect.  

 The start events [Event1, 3-6] are ignored in the Active 
 state.

 A manual stop event[Event2], the local system:
        - Sets the administrative down in the MIB reason code,
	- Sends a notification with a Cease,
	- If any BGP routes exist, delete the routes
	- release all BGP resources, 
	- drops the Transport connection, 
        - sets connect retry count to zero
        - resets the connect retry timer (sets to zero)
        [Action S in the FSM table]
        - goes to Idle state.

 In response to any other event (Events 7-8, 10-11,18, 20-
 27), the local system:
        - stores the MIB information to indicate appropriate 
          error [FSM for Events 7-8, 10-11, 18, 20-27]
        - reset the connect retry timer (sets to zero),
        - release all BGP resources, 
	- drops the transport connection, 
        - increments the ConnectRetryCnt by one,
        - optionally performs BGP peer oscillation damping, 
  	  [Action D in FSM table],
	- and goes to the idle state

--------------149A43A8660ACEFC11F84974--

From: Bill Fenner <[email-masked]>
To: [email-masked]
Subject: FSM words for the draft-18
Date: Wed, 17 Jul 2002 23:13:59 -0700
Versions: dmail (solaris) 2.4c/makemail 2.9d
Sender: [email-masked]

--19701020
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline

Sue asked me to forward this since it didn't appear on the list.
Sorry if this causes duplicates.

  Bill

----- Begin forwarded message:

From: Susan Hares <[email-masked]>
Subject: FSM words for the draft-18
Date: Wed, 17 Jul 2002 23:24:40 -0400
To: [email-masked]
Cc: yakov Rekhter <[email-masked]>, [email-masked],
    [email-masked]

I would like to propose that section 8 be replaced with the
attached words.

Sue Hares

--19701020
Content-Type: text/plain; name="FSM-words-05b.txt"; x-unix-mode=0644
Content-Disposition: attachment; filename="FSM-words-05b.txt"

Note: (this is version 5 of the changes to
       the text.) 

8.0	BGP Finite State Machine

This section specifies the BGP operation in terms of a 
Finite State Machine (FSM).  The section falls into 2 
parts:

    1)	Description of Events for the State machine (section 
        8.1
    2)	Description of the FSM (section 8.2)

Session Attributes required for each connection are; 

   1)	state
   2)	Connect Retry timer
   3)	Hold timer
   4)	Hold time
   5)	Keepalive timer

8.1 Events for the BGP FSM

8.1.1   Administrative Events (events 1-5)

Please note that only Event 1 (manual start) and Event 2 
(manual stop) are mandatory administrative events. All 
other administrative events are optional.

 Event1: Manual start 

	   Definition: Administrator manually starts peer
                       connection
	   Status: 	Mandatory

 Event2: Manual stop

	   Definition: Local system administrator manually 
                       stops the peer connection.

	   Status: 	mandatory

 Event3: Automatic Start

 	   Definition: Local system automatically starts the 
                       BGP peer session

	   Status:     Optional depending on local system

 Event4: Manual start with passive Transport Establishment

	   Definition: Administrator manually start the peer
                       Connection, but has the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing the connection.

	   Status:     Optional depending on local system

 Event5: Automatic start with passive Transport establishment

	  Definition:  Local system automatically starts the
                       BGP session with the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing a connection.

	  Status:      Optional depending on local system use
		       of a passive connection.

 Event6: Automatic start with bgp_stop_flap option set

	   Definition: Local system automatically starts the 
                       BGP peer session with persistent peer 
                       oscillation damping enabled.  The exact 
                       method of damping persistent peer 
                       oscillations is left up to the 
                       implementation.   These methods of 
                       damping persistent BGP adjacency 
                       flapping are outside the scope of this 
                       document.

           Status: 	Optional, used only if the bgp peer has 
	  	        Enabled a method of damping persistent
		        BGP peer flapping.

 Event7: Auto stop

	   Definition: Local system automatically stops the
                       BGP peer session.  
	
	   Status:     Optional depending on local system

8.1.2 Timer Events (events 8-11)

 Event8:  idle Hold timer expires 

           Definition: Idle Hold timer expires.  The Idle 
                       Hold Timer is only used when persistent  
                       BGP oscillation damping functions are 
                       enabled.  

           Status: 	Optional.  Used when persistent
			BGP peer oscillation damping functions
			are enabled. 

 Event9: connect retry timer expires

           Definition: An event triggered by the expiration of 
                       the ConnectRetry timer.
  
	   Status:     Mandatory

 Event10: hold time expires

	   Definition: An event generated when the HoldTimer 
                       expires.

	   Status:     Mandatory

 Event11: keepalive timer expires

	   Definition: A periodic event generated due to the 
                       expiration of the KeepAlive Timer. 
 
	   Status:     Mandatory

 Event12: DelayBGP Open timer expires [changed]
	
	   Definition: A timer that delays sending of the BGP
	               Open message for n seconds after the
                       Transport connection has been completed. 

	   status:     Optional

8.2.3) Tranport Message based (13-16) 

 Event13: Transport Connection Indication & valid remote peer[changed]

	   Definition: Event indicating that transport
                       Request valid source IP address and TCP 
                       port, and valid destination IP address 
                       and TCP Port.  The definition of 
                       invalid Source, and invalid Destination 
                       IP address is left to the implementation.
                       BGP's destination port should be port 
                       179 as defined by IANA.

	   Status:     Mandatory

 Event14: RCV Transport Connection Indication and Invalid Source or 
          Destination [changed]

           Definition: Transport request received with either
 		       an invalid source address or port
		       number or an invalid destination 
                       address or port number. BGP destination 
                       port  number should be 179 as defined 
                       by IANA.

	    Status: 	 Mandatory
	
 Event15: Transport Connection request sent received an ACK.

           Definition: Local system's request to establish a transport
		       Connection to the remote side received
		       an ACK.

	    Status:    Mandatory

 Event16: Transport Connection Confirmed [changed]

	   Definition: The local system has received a Confirmation that
                       the transport connection has been established by
		       the remote site. 

	  Status:      Mandatory

 Event17: Transport connection fails [changed]

           Definition: This BGP peer receives a transport
		       connection failure notice.  This
		       connection notice could be caused by a
		       Transport disconnect message or a
		       Timeout in the transport session.

                       If this connection is using TCP, the
		       remote BGP peer's TCP machine could
		       have sent a FIN.  The local peer would
		       respond with a FIN-ACK. Another
		       alternative is that the local peer
		       indicated a timeout in the TCP session
		       and downed the connection.

	    Status:    Mandatory

8.1.4 BGP Messages (18-27)

 Event18: BGPOpen

           Definition:  An event indicating that a valid Open 
			message has been received.

	   Status: 	Mandatory

 Event19: BGPOpen with BGP Delay Open Timer running [changed]

           Definition: An event indicating that a valid Open 
                       Message has been successful 
                       established for a peer that is 
                       currently delaying the sending of an 
                       BGP Open message. 

	    Status:    Optional

 Event20: BGPHeaderErr

	   Definition: BGP message header is not valid.  

	   Status:     Mandatory

 Event21: BGPOpenMsgErr 
	   Definition: An BGP Open message has been received 
                        with errors. 
 
	   Status: 	Mandatory

 Event22: Open collision discard

           Definition: An event generated administratively 
                       when a connection Collision has been 
                       detected while processing an incoming  
                       Open message. This connection has been 
                       selected to disconnected.  See section 
                       6.8 for more information on collision
	               detection.  

                       Event 22 is an administrative could 
                       occur if FSM is implemented as two 
                       linked state machines.  

	  Status:      Optional

 Event23: NotifMsgVerErr

	    Definition: An event is generated when a 
                        NOTIFICIATION message with "version   
                        error" is received.

	    Status:     Mandatory

 Event24: NotifMsg

	    Definition: An event is generated when a 
                        NOTIFICATION messages is received and
                        the error code is anything but 	 
			"version error".

	     Status:	Mandatory
		        
 Event25: KeepAliveMsg

	    Definition: An event is generated when a KEEPALIVE
			Message is received.

		Status: Mandatory

 Event26: UpdateMsg

	    Definition: An event is generated when a valid 
			Update Message is received.

		Status: Mandatory

 Event27: UpdateMsgErr

	    Definition: An event is generated when an invalid
			Update message is received.

		Status:	Mandatory

8.2) Description of FSM

Idle state

   Initially BGP is in the Idle state.   

   In this state BGP refuses all incoming BGP connections.  No 
   resources are allocated to the peer.    In response to a 
   manual start event(Event1) or an automatic start 
   event(Event3), the local system 
      -	initializes all BGP resources, 
      -	sets the Connect retry counter to zero
      -	starts the ConnectRetry timer with initial value, 
      -	initiates a transport connection to the other BGP 
        peer, 
      -	listens for a connection that may be initiated by 
        the remote BGP peer, and
        [Action A in the FSM table]
      -	changes its state to connect.

  An manual stop event (Event2) is ignored in the Idle state.

  In response to a manual start event with the passive Transport
  flag (Event 4) or automatic start with the passive Transport flag 
  (Event 5), the local system:
      -	initializes all BGP resources,
      -	sets the connect retry count to zero,
      -	start the Connect Retry timer with initial value, 
      -	listens for a connection that may be initiated by 
        the remote peer
        [Action B in the FSM table] 
      -	changes its state to Active.

  The exact value of the ConnectRetry timer is a local 
  matter, but it should be sufficiently large to allow TCP 
  initialization.   

  If a persistent BGP peer oscillation damping function is 
  enabled, two additional events may occur within Idle state:  
      -	Automatic start with bgp_stop_flap set [Event6],
      -	Idle Hold Timer expired [Event 8].

  The method of preventing persistent BGP peer oscillation is 
  outside the scope of this document.

  Any other events [Events 9-27] received in the Idle state, 
  are noted by the MIB processing as FSM Errors [action V]
  and the local peer stays in the Idle State.

Connect State:

  In this state, BGP is waiting for the transport protocol 
  connection to be completed.

  If the transport connection succeeds [Event 15 or 
  Event 16], the local system checks the "Delay Open 
  Flag".  If the Delay Open flag is set, the local system: 
     -	clears the ConnectRetry timer, 
     -	set the BGP Open Delay timer to the initial 
        value.
        [Action ZZ in FSM table]

  If the Delay Open flag is not set, the local system: 
     - clears the ConnectRetry timer, 
     - completes BGP initialization
     - send an Open message to its peer,
     - sets Hold timer to a large value,
     - Change the state to Open Sent 
   [Action H in the FSM table]

  A hold timer value of 4 minutes is suggested.

  If the Open Delay timer expires [Event 12] in the connect 
  state,
     - send an Open message to its peer,
     - set the Hold timer to a large value,
	[Action H in the FSM Table], and
     - change the state to Open Sent. 

  If the BGP port receives a Transport connection indication 
  [Event 13], the Transport connection is processed (actions AA or
  AB in the FSM table) and the connection remains
  in the connected state. 

  If the transport connection receives an Transport indication
  that is invalid or unconfigured. [Event 14]:
     - the transport connection is rejected.
	[Action L in the FSM table]

  If the transport connection fails (timeout or transport
  disconnect) [Event17], the local system:
      - restarts the ConnectRetry timer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
     [Action G in the FSM table]
      - changes its state to Active. 

  If an Open is received with the BGP Delay Open timer is 
  running [Event 19], the local system:
	- clears the connect retry timer (cleared to zero),
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message

	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 The start events [Event 1, 3-6] are ignored in connect 
 state.

 A manual stop event[Event2], the local system:
       - drops the transport connection, 
       - releases all BGP resources,
       - sets connect retry count to zero
       - resets the connect retry timer (sets to zero)
      [Action Z in the FSM table]
       - goes to Idle state.

  In response to the ConnectRetry timer expired event(Event 
  9), the local system: 
      - Sets the MIB FSM error information with ConnectRetry 
        expired, 
      - drops the transport connection 
      - restarts the ConnectRetry timer
      - initiates a transport connection to the other BGP 
        peer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
      [Action O in the FSM table]
      - stays in Connect state.

 In response to any other events [Events 7-8, 10-11, 18, 20-
 27] the local system:
     - resets the connect retry timer (sets to zero),
     - drops the Transport connection,
     - release all BGP resources, 
     - increments the ConnectRetryCnt by 1,
     - [optionally] performs bgp peer oscillation damping, and
     [Action D in the FSM table],
     - goes to Idle state.

Active State:

 In this state BGP is trying to acquire a peer by listening 
 for and accepting a transport protocol connection.  

 A transport connection succeeds [Event 15 or Event 16], the 
 local system: process the transport connection flags
	- If the BGP delay open flag is set:
          o clears the ConnectRetry timer, 
          o completes the BGP initialization,
          o sets the BGP delay Open timer
         [Action ZZ]

        - If the BGP delay open flag is not set:
          o clears the ConnectRetry timer,
          o completes the BGP initialization, 
          o sends the Open message to it's peer, 
          o sets its Hold timer to a large value, 
         [Action H in the FSM table]
          and changes its state to OpenSent.  

 A Hold timer value of 4 minutes is suggested.  

 If the local system receives a valid Transport Indication
 [Event 13], the local system processes the Transport flags
  (actions aa or ab in FSM section 2.3.4).  

 If the local system receives a Transport indication
 that is invalid for this connection [Event 14]:
      - the transport connection is rejected
        [Action L in the FSM table]

 If the local system receives a Transport connection
 failed [Event 17] (timeout or receives Transport
 disconnect), the local system will:
	- set Transport disconnect in the MIB reason code,
	- restart ConnectRetry timer (with initial value)
	- release all BGP resources
	- Acknowledge the drop of Transport connection if
          transport disconnect (If TCP, send a FIN ACK),
	- Increment ConnectRetryCnt by 1,
	- perform the BGP peer oscillation damping process [2]
	[Action Y in the FSM table]

 If the local system has the Delay Open timer expired [event 
 12] local system: 
        - clears the Connect Retry timer (set to zero),
	- stops and clears the Delay Open timer (set to zero) 
        - completes the BGP initialization,
        - sends the Open message to it's remote peer,
        - sets it's Hold timer to a large value,
        [Action H in the FSM table]
        - and set the state to Open Confirm.

 A Hold timer value of 4 minutes is also suggested for this 
 state transition.

 If an Open is received with the BGP Delay Open timer is 
 running [Event 19], the local system 
	- clears the connect retry timer (cleared to zero),
	- Stops and clears the BGP Open Delay timer  
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message
	- Set the Hold timer to a large value (4 minutes),
	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 In response the ConnectRetry timer expired event[Event9], 
 the local system:
        - restarts the ConnectRetry timer (with initial value),
        - initiates a transport connection to the other BGP 
          peer,
        - Continues to listen for transport connection that may be 
          initiated by remote BGP peer, 
        [Action F in the FSM table]
	- and changes its state to Connect.  

 The start events [Event1, 3-6] are ignored in the Active 
 state.

 A manual stop event[Event2], the local system:
        - Sets the administrative down in the MIB reason code,
	- Sends a notification with a Cease,
	- If any BGP routes exist, delete the routes
	- release all BGP resources, 
	- drops the Transport connection, 
        - sets connect retry count to zero
        - resets the connect retry timer (sets to zero)
        [Action S in the FSM table]
        - goes to Idle state.

 In response to any other event (Events 7-8, 10-11,18, 20-
 27), the local system:
        - stores the MIB information to indicate appropriate 
          error [FSM for Events 7-8, 10-11, 18, 20-27]
        - reset the connect retry timer (sets to zero),
        - release all BGP resources, 
	- drops the transport connection, 
        - increments the ConnectRetryCnt by one,
        - optionally performs BGP peer oscillation damping, 
  	  [Action D in FSM table],
	- and goes to the idle state

Open Sent:

 In this state BGP waits for an Open Message from its peer.  

 When an OPEN message is received, all fields are checked 
 for correctness.  If there are no errors in the OPEN message 
 [Event 18] the local system:
       - resets the BGP Delay timer to zero,
       - reset BGP Connect Timer to zero, 
       - sends a KEEPALIVE message and
       - sets a KeepAlive timer (via the text below)
       - sets the Hold timer according to the negotiated value 
         (see section 4.2), and 
         [Action N in the FSM table]
       - sets the state to Open Confirm.

 If the negotiated Hold time value is zero, then the Hold 
 Time timer and KeepAlive timers are not started.   If the 
 value of the Autonomous System field is the same as the 
 local Autonomous System number, then the connection is an 
 "internal" connection; otherwise, it is an "external" 
 connection.   (This will impact UPDATE processing as 
 described below.)  

 If the BGP message header checking [Event20] or OPEN message
 check detects an error (see Section 6.2)[Event21], the local system:
       - sends a NOTIFICATION message with appropriate error 
         code, 
       - reset the connect retry timer (sets to zero),
       - if there are any routes associated with the BGP session,
	 delete these routes
       - release all BGP resources,
       - drop the transport session 
       - increments the ConnectRetryCnt by 1, 
       - bgp peer oscillation damping process,  
       [Actions I, J in FSM table, depending on error],
       - and goes to the Idle state.

 Collision detection mechanisms (section 6.8) need to be 
 applied when a valid BGP Open is received [Event 18 or 
 Event 19].  Please refer to section 6.8 for the details of 
 the comparison. An administrative collision detect is when 
 BGP implementation determines my means outside the scope of 
 this document that a connection collision has occurred.   

 If a connection in Open Sent is determined to be the 
 connection that must be closed, an administrative collision 
 detect [Event 22] is signaled to the state machine. If such 
 an administrative collision detect dump [Event 22] is 
 received in Open Sent, the local system:  
       - sets MIB state information to 
         collision detect closure, 
       - send a NOTIFICATION with a CEASE
       - resets the connect retry timer,
       - release all BGP resources,
       - drop the transport connection,
       - increments ConnectRetryCnt by 1,
       - performs any BGP peer oscillation damp process, and
       [Action R in the FSM],
       - enters Idle state.

  If a NOTIFICATION message is received with a version 
  error[Event23], Notification message without version number 
  [Event 24], the local system: 
       - resets the connect retry timer (sets to zero)
       - drops the Transport connection,
       - releases all BGP resources, 
       - increments the ConnectRetryCnt by 1
       - process any BGP peer oscillation damping,
	[Action Y]  
       - and sets the state to Idle.

 The Start events [Event1, 3-6] are ignored in the OpenSent 
 state.

  If a manual stop event [Event 2] is issued in Open sent 
 state, the local system:
	- Sets administrative down reason in MIB reason,
	- sends the Notification with a cease,
	- if BGP routes exists, delete the routes,
	- Release all BGP resources, 
	- Drops the Transport connection, 
	- set ConnectRetryCnt to zero, 
	- resets the Connect Retry timer (set to zero),
	[Action S in the FSM table], and
	- transitions to the Idle state.

  If an automatic stop event [Event 7] is issued in Open sent 
  state, the local system:
	- Sets administrative down reason in MIB reason,
	- sends the Notification with a cease,
	- if any routes are associated with te BGP session,
	  delete the routes, 
	- release all the BGP resources
	- Drops the Transport connection, 
	- increments the ConnectRetryCnt by 1, 
	- BGP peer oscillation process [2]
	[Action C in the FSM table], and
	- transitions to the Idle state.

 If the Hold Timer expires[Event 10], the local system:
       - set Hold timer expired in MIB Error reason code, 
       - send a NOTIFICATION message with error code Hold 
         Timer Expired,
       - reset the connect retry timer (sets to zero),
       - releases all BGP resources,
       - drops the Transport connection, 
       - increments the ConnectRetryCnt by 1
         [Action K in the FSM table], and
	- transitions to the Idle state.

 If a transport indication is received for valid connection 
 [Event 13] or transport Request Acknowledgement [Event 15]
 is received, or a transport connect confirm [Event 16] is
 received a second TCP session may be in progress.  This
 second TCP session is tracked per the Call Collision 
 processing (section 6.8) until an OPEN message is received.

 A TCP connection for an invalid port [Event 14] is ignored. 

 If a Transport Failure [Event17], is received from the 
 underlying transport protocol, the local system:
       - closes the BGP connection, 
       - restarts the Connect Retry timer, 
       - and continues to listen for a connection that may be 
         initiated by the remote BGP peer,
       [Action O in the FSM table]
       - and goes into Active state.

 In response to any other event [Events 8-9, 11-12, 19, 25-27], 
  the local system:
	- sends the NOTIFICATION with the Error Code Finite 
          state machine error,
        - resets the connect retry timer (sets to zero),
	- releases all BGP resources
        - drops the Transport connection, 
        - increments the ConnectRetryCnt by 1, 
        - process any bgp peer oscillation damping[2] 
        [Action E in the FSM table],
        - and sets the state to idle.

Open Confirm State

 In this state BGP waits for a KEEPALIVE or NOTIFICATION 
 message.

 If the local system receives a KEEPALIVE message[Event 25], 
        - restarts the Hold timer, 
          [Action P in the FSM table], and
        - changes its state to Established. 

 If the local system receives a NOTIFICATION message [event 
 23-24] or receives a TCP Disconnect [event 17] from the 
 underlying transport protocol, the local system:  
        - sets the appropriate MIB information for FSM error, 
        - resets the connect retry timer (sets the timer to 
          zero),
	- releases all BGP resources, 
        - drops the TCP connection,
        - increments the ConnectRetryCnt by 1, 
        [Action Y in the FSM table],
        - and sets the state to idle.

 Any start event [Event1, 3-6] is ignored in the OpenConfirm 
 state.

 In response to a manual stop event[Event 2] initiated by 
 the operator, the local system: 
	- set Administrative down in MIB Reason code,
        - sends the NOTIFICATION message with Cease, 
	- if any BGP routes, dete the routes
        - releases all BGP resources, 
	- drop the transport connection, 
        - sets the ConnectRetryCnt to zero
        - sets the connect retry timer to zero
        [Action S in the FSM table]
        - transitions to Idle state.

 In response to the Automatic stop event initiated by the 
 system[event 7], the local system:
        - sets the MIB entry for this peer to administratively 
          down, 
        - sends the NOTIFICATION message with Cease,
	- connect retry timer reset (set to zero)
	- If any BGP routes exist, delete the routes,
	- release all BGP resources,
        - drops the transport connection,
        - increments the ConnectRetryCnt by 1, 
        [Action C in the FSM table], and
        - transitions to the Idle State.

 If the Hold Timer expires before a KEEPALIVE message is 
 received [event10], the local system:
        - set the MIB reason to Hold time expired, 
        - send the NOTIFICATION message with the error code 
          set to Hold Time Expired, 
        - resets the connect retry timer (sets the timer to to 
          zero),
	- releases all BGP resources, 
        - drops the transport connection, 
        - increments the ConnectRetryCnt by 1 
          [Action K in the FSM table], 
        - and sets the state to Idle.

 If the local system receives a KEEPALIVE timer expires 
 event [Event 11], the system:
        - sends a KEEPALIVE message,
        - restarts the Keepalive timer 
        [Action Q the in FSM table],and 
        - remains in Open Confirmed state.

 In the event of TCP establishment [Event 13], or TCP 
 connection succeeding [Event 15 or Event 16] while in Open 
 Confirm, the local system needs to track the 2nd 
 connection.  

 If a TCP connection is attempted to an invalid port [Event 
 14], the local system will ignore the second connection 
 attempt.  

 If an OPEN message is received, all fields are check for 
 correctness.  If the BGP message header checking [Event20] 
 or OPEN message check detects an error (see Section 
 6.2)[Event21], [the local system:
        - sends a NOTIFICATION message with appropriate error 
          code, 
        - resets the connect retry timer (sets the timer to 
          zero),
        - releases all BGP resources, 
        - drops the TCP connection,
        - increments the ConnectRetryCnt by 1, 
        - runs the BGP peer oscillation damping process [2]  
       [Actions I, J, in the FSM table depending on the error]
        - and goes to the Idle state.

 If the Open messages is valid [Event 18], the collision 
 detect function is processed per section 6.8.  If this 
 connection is to be dropped due to call collision, the 
 local system:
        - sets the Call Collision cease in the MIB reason 
         code,
        - sends a Notification with a Cease 
        - resets the Connect timer (set to zero),
	- releases all BGP resources,
	- Drops the TCP connection (send TCP FIN),
	- increments the ConnectRetryCnt by 1, and
	- performs any BGP peer oscillation damping process [2].
	[action 

 If during the processing of another Open message, the BGP 
 implementation determines my means outside the scope of 
 this document that a connection collision has occurred and 
 this connection is to be closed, the local system will 
 issue a call collision dump [Event 22].  When the local 
 system receives a call collision dump event [Event 22], the 
 local system:
        - Sets the MIB FSM variable to indicate collision 
          detected and dump connection.
        - send a NOTIFICATION with a CEASE
        - deletes all routes associated with connection,
        - resets the connect retry timer,
        - releases all BGP resources
        - drops all TCP connection,
        - increments the ConnectRetryCnt by 1, 
        - and performs any BGP peer oscillation damping,
        [Action R in the FSM table],
        - enters Idle state.

 In response to any other event [Events 8-9, 12, 19, 26-27], 
 the local system:
        - sends a NOTIFICATION with a code of Finite State 
          Machine Error,
        - resets the connect retry timer (sets to zero)
        - drops the TCP connection,
        - releases all BGP resources, 
        - increments the ConnectRetryCnt by 1, 
        - performs any BGP peer oscillation damping,
        [Action E in the FSM table], and 
        - transitions to Idle state.

Established State:

 In the Established state BGP can exchange UPDATE, 
 NOTFICATION, and KEEPALIVE messages with its peer.

 If the local system receives an UPDATE message [Event26], 
 the local system will:
	- process the update packet
	- restarts its Hold timer, if the negotiated Hold Time 
          value is non-zero.
	[Action W in the FSM table], and
	- remain in the Established state.

 If the local system receives a NOTIFICATION message 
 [Event23 or Event24] or a disconnect [Event17] from the 
 underlying transport protocol, it:
	- sets the appropriate error code in MIB reason code,
	- if any BGP routes exist, delete all BGP routes,  
        - resets the connect retry timer (sets to zero), 
        - releases all the BGP resources, 
	- drops the TCP connection,
	- increments the ConnectRetryCnt by 1, and 
       [Action T in the FSM table]   
	- Goes to the Idle state.

 If the local system receives a Keepalive message
 [Event 25], the local system will:
        - restarts its Hold Timer, if the negotiated Hold Time 
          value is non-zero.
       [Action P in the FSM table], and
	- remain in the Established state.

 If the local system receives an UPDATE message, and the 
 Update message error handling procedure (see Section 6.3) 
 detects an error [Event27], the local system:
       - sends a NOTIFICATION message with Update error,
       - resets the connect retry timer (sets to zero),
       - drops the TCP connection,
       - releases all BGP resources, 
       - increments the ConnectRetryCnt by 1 
       - performs any BGP peer oscillation damping
      [Action U in FSM table], 
       - and goes to Idle state.

 Any start event (Event 1, 3-6) is ignored in the 
 Established state.

 In response to a manual stop event (initiated by an 
 operator)[Event2]:
        - sets the Administrative stop in MIB reason code, 
	- sends the NOTIFICATION message with Cease, 
        - if BGP routes exist, delete the BGP routes, 
        - release BGP resources,
	- drop TCP connection, 
        - set ConnectRetryCnt to zero (0),
        - reset connect retry timer to zero (0), and
        [Action S in FSM table]
        - transition to the Idle.

 In response to an automatic stop event initiated by the 
 system (automatic) [Event7], the local system:
     - sets Administrative Stop in MIB Reason code, 
     - sends a NOTIFICATION with Cease,
     - resets the connect retry timer (sets to zero)
     - deletes all routes associated with bgp peer session,
     - releases all BGP resources, 
     - drops the transport connection,   
     - increments the ConnectRetryCnt by 1, 
     - performs any BGP peer oscillation damping, and 
     [Action C in FSM table]
     - transitions to the idle state.

 An example automatic stop event is exceeding the number of 
 prefixes for a given peer and the local system 
 automatically disconnecting the peer.

 If the Hold timer expires [Event10], the local system:
      - sends a NOTIFICATION message with Error Code Hold 
        Timer Expired,
      - resets the connect retry timer (sets to zero),
      - releases all BGP resources, 
      - drops the TCP connection, 
      - increments the ConnectRetryCnt by 1,  
      -	performs any BGP peer oscillation damping,
      [Action M in FSM table] 
      - and goes to Idle state.

 If the KeepAlive timer expires [Event11], the local system 
 sends a KEEPALIVE message, it restarts its KeepAlive timer, 
 unless the negotiated Hold Time value is zero [Action Q].

 Each time time the local system sends a KEEPALIVE or UPDATE 
 message, it restarts its KeepAlive timer, unless the 
 negotiated Hold Time value is zero.

 A transport connection indication [Event 13] received
 for a valid port will cause the 2nd connection to be
 tracked.  A transport connection indications for 
 invalid port [Event 14], will be ignored. 
 
 In response to a Transport connection succeeds [Event 15
 or Event 16], the 2nd connection shall be tracked until
 it sends an OPEN message.  

 If a valid Open message [Event 18] is received, it will be 
 checked to see if it collides (section 6.8) with any other 
 session. If the BGP implementation determines that this 
 connection needs to be terminated, it will process an Call 
 Collision dump event[Event 22].  If this session needs to be 
 terminated, the connection will be terminated by:

     - send a NOTIFICATION with a CEASE
     - deletes all routes associated with connection,
     - resets the connect retry timer,
     - if any BGP routes, delete the routes,
     - release all BGP resources, 
     - drops the transport connection,
     - increments ConnectRetryCnt by 1,
     - and performs any BGP peer oscillation damping,
    [Action R in the FSM table],
     - and enters the Idle state

 In response to any other event [Events 8-9,12, 19-21] the 
 local system:
     - sends a NOTIFICATION message with Error Code Finite 
       State Machine Error,
     - deletes all routes associated with BGP peer session, 
     - resets the connect retry timer (sets to zero)
     - releases all BGP resources,
     - drops the TCP connection,
     - increments the ConnectRetryCnt by 1, 
     - performs any BGP peer oscillation damping, and
     [Action E in FSM table],
     - transitions to Idle.

--19701020
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline

--19701020
Content-Type: text/plain; name="FSM-words-05b.txt"; x-unix-mode=0644
Content-Disposition: attachment; filename="FSM-words-05b.txt"

Note: (this is version 5 of the changes to
       the text.) 

8.0	BGP Finite State Machine

This section specifies the BGP operation in terms of a 
Finite State Machine (FSM).  The section falls into 2 
parts:

    1)	Description of Events for the State machine (section 
        8.1
    2)	Description of the FSM (section 8.2)

Session Attributes required for each connection are; 

   1)	state
   2)	Connect Retry timer
   3)	Hold timer
   4)	Hold time
   5)	Keepalive timer

8.1 Events for the BGP FSM

8.1.1   Administrative Events (events 1-5)

Please note that only Event 1 (manual start) and Event 2 
(manual stop) are mandatory administrative events. All 
other administrative events are optional.

 Event1: Manual start 

	   Definition: Administrator manually starts peer
                       connection
	   Status: 	Mandatory

 Event2: Manual stop

	   Definition: Local system administrator manually 
                       stops the peer connection.

	   Status: 	mandatory

 Event3: Automatic Start

 	   Definition: Local system automatically starts the 
                       BGP peer session

	   Status:     Optional depending on local system

 Event4: Manual start with passive Transport Establishment

	   Definition: Administrator manually start the peer
                       Connection, but has the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing the connection.

	   Status:     Optional depending on local system

 Event5: Automatic start with passive Transport establishment

	  Definition:  Local system automatically starts the
                       BGP session with the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing a connection.

	  Status:      Optional depending on local system use
		       of a passive connection.

 Event6: Automatic start with bgp_stop_flap option set

	   Definition: Local system automatically starts the 
                       BGP peer session with persistent peer 
                       oscillation damping enabled.  The exact 
                       method of damping persistent peer 
                       oscillations is left up to the 
                       implementation.   These methods of 
                       damping persistent BGP adjacency 
                       flapping are outside the scope of this 
                       document.

           Status: 	Optional, used only if the bgp peer has 
	  	        Enabled a method of damping persistent
		        BGP peer flapping.

 Event7: Auto stop

	   Definition: Local system automatically stops the
                       BGP peer session.  
	
	   Status:     Optional depending on local system

8.1.2 Timer Events (events 8-11)

 Event8:  idle Hold timer expires 

           Definition: Idle Hold timer expires.  The Idle 
                       Hold Timer is only used when persistent  
                       BGP oscillation damping functions are 
                       enabled.  

           Status: 	Optional.  Used when persistent
			BGP peer oscillation damping functions
			are enabled. 

 Event9: connect retry timer expires

           Definition: An event triggered by the expiration of 
                       the ConnectRetry timer.
  
	   Status:     Mandatory

 Event10: hold time expires

	   Definition: An event generated when the HoldTimer 
                       expires.

	   Status:     Mandatory

 Event11: keepalive timer expires

	   Definition: A periodic event generated due to the 
                       expiration of the KeepAlive Timer. 
 
	   Status:     Mandatory

 Event12: DelayBGP Open timer expires [changed]
	
	   Definition: A timer that delays sending of the BGP
	               Open message for n seconds after the
                       Transport connection has been completed. 

	   status:     Optional

8.2.3) Tranport Message based (13-16) 

 Event13: Transport Connection Indication & valid remote peer[changed]

	   Definition: Event indicating that transport
                       Request valid source IP address and TCP 
                       port, and valid destination IP address 
                       and TCP Port.  The definition of 
                       invalid Source, and invalid Destination 
                       IP address is left to the implementation.
                       BGP's destination port should be port 
                       179 as defined by IANA.

	   Status:     Mandatory

 Event14: RCV Transport Connection Indication and Invalid Source or 
          Destination [changed]

           Definition: Transport request received with either
 		       an invalid source address or port
		       number or an invalid destination 
                       address or port number. BGP destination 
                       port  number should be 179 as defined 
                       by IANA.

	    Status: 	 Mandatory
	
 Event15: Transport Connection request sent received an ACK.

           Definition: Local system's request to establish a transport
		       Connection to the remote side received
		       an ACK.

	    Status:    Mandatory

 Event16: Transport Connection Confirmed [changed]

	   Definition: The local system has received a Confirmation that
                       the transport connection has been established by
		       the remote site. 

	  Status:      Mandatory

 Event17: Transport connection fails [changed]

           Definition: This BGP peer receives a transport
		       connection failure notice.  This
		       connection notice could be caused by a
		       Transport disconnect message or a
		       Timeout in the transport session.

                       If this connection is using TCP, the
		       remote BGP peer's TCP machine could
		       have sent a FIN.  The local peer would
		       respond with a FIN-ACK. Another
		       alternative is that the local peer
		       indicated a timeout in the TCP session
		       and downed the connection.

	    Status:    Mandatory

8.1.4 BGP Messages (18-27)

 Event18: BGPOpen

           Definition:  An event indicating that a valid Open 
			message has been received.

	   Status: 	Mandatory

 Event19: BGPOpen with BGP Delay Open Timer running [changed]

           Definition: An event indicating that a valid Open 
                       Message has been successful 
                       established for a peer that is 
                       currently delaying the sending of an 
                       BGP Open message. 

	    Status:    Optional

 Event20: BGPHeaderErr

	   Definition: BGP message header is not valid.  

	   Status:     Mandatory

 Event21: BGPOpenMsgErr 
	   Definition: An BGP Open message has been received 
                        with errors. 
 
	   Status: 	Mandatory

 Event22: Open collision discard

           Definition: An event generated administratively 
                       when a connection Collision has been 
                       detected while processing an incoming  
                       Open message. This connection has been 
                       selected to disconnected.  See section 
                       6.8 for more information on collision
	               detection.  

                       Event 22 is an administrative could 
                       occur if FSM is implemented as two 
                       linked state machines.  

	  Status:      Optional

 Event23: NotifMsgVerErr

	    Definition: An event is generated when a 
                        NOTIFICIATION message with "version   
                        error" is received.

	    Status:     Mandatory

 Event24: NotifMsg

	    Definition: An event is generated when a 
                        NOTIFICATION messages is received and
                        the error code is anything but 	 
			"version error".

	     Status:	Mandatory
		        
 Event25: KeepAliveMsg

	    Definition: An event is generated when a KEEPALIVE
			Message is received.

		Status: Mandatory

 Event26: UpdateMsg

	    Definition: An event is generated when a valid 
			Update Message is received.

		Status: Mandatory

 Event27: UpdateMsgErr

	    Definition: An event is generated when an invalid
			Update message is received.

		Status:	Mandatory

8.2) Description of FSM

Idle state

   Initially BGP is in the Idle state.   

   In this state BGP refuses all incoming BGP connections.  No 
   resources are allocated to the peer.    In response to a 
   manual start event(Event1) or an automatic start 
   event(Event3), the local system 
      -	initializes all BGP resources, 
      -	sets the Connect retry counter to zero
      -	starts the ConnectRetry timer with initial value, 
      -	initiates a transport connection to the other BGP 
        peer, 
      -	listens for a connection that may be initiated by 
        the remote BGP peer, and
        [Action A in the FSM table]
      -	changes its state to connect.

  An manual stop event (Event2) is ignored in the Idle state.

  In response to a manual start event with the passive Transport
  flag (Event 4) or automatic start with the passive Transport flag 
  (Event 5), the local system:
      -	initializes all BGP resources,
      -	sets the connect retry count to zero,
      -	start the Connect Retry timer with initial value, 
      -	listens for a connection that may be initiated by 
        the remote peer
        [Action B in the FSM table] 
      -	changes its state to Active.

  The exact value of the ConnectRetry timer is a local 
  matter, but it should be sufficiently large to allow TCP 
  initialization.   

  If a persistent BGP peer oscillation damping function is 
  enabled, two additional events may occur within Idle state:  
      -	Automatic start with bgp_stop_flap set [Event6],
      -	Idle Hold Timer expired [Event 8].

  The method of preventing persistent BGP peer oscillation is 
  outside the scope of this document.

  Any other events [Events 9-27] received in the Idle state, 
  are noted by the MIB processing as FSM Errors [action V]
  and the local peer stays in the Idle State.

Connect State:

  In this state, BGP is waiting for the transport protocol 
  connection to be completed.

  If the transport connection succeeds [Event 15 or 
  Event 16], the local system checks the "Delay Open 
  Flag".  If the Delay Open flag is set, the local system: 
     -	clears the ConnectRetry timer, 
     -	set the BGP Open Delay timer to the initial 
        value.
        [Action ZZ in FSM table]

  If the Delay Open flag is not set, the local system: 
     - clears the ConnectRetry timer, 
     - completes BGP initialization
     - send an Open message to its peer,
     - sets Hold timer to a large value,
     - Change the state to Open Sent 
   [Action H in the FSM table]

  A hold timer value of 4 minutes is suggested.

  If the Open Delay timer expires [Event 12] in the connect 
  state,
     - send an Open message to its peer,
     - set the Hold timer to a large value,
	[Action H in the FSM Table], and
     - change the state to Open Sent. 

  If the BGP port receives a Transport connection indication 
  [Event 13], the Transport connection is processed (actions AA or
  AB in the FSM table) and the connection remains
  in the connected state. 

  If the transport connection receives an Transport indication
  that is invalid or unconfigured. [Event 14]:
     - the transport connection is rejected.
	[Action L in the FSM table]

  If the transport connection fails (timeout or transport
  disconnect) [Event17], the local system:
      - restarts the ConnectRetry timer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
     [Action G in the FSM table]
      - changes its state to Active. 

  If an Open is received with the BGP Delay Open timer is 
  running [Event 19], the local system:
	- clears the connect retry timer (cleared to zero),
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message

	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 The start events [Event 1, 3-6] are ignored in connect 
 state.

 A manual stop event[Event2], the local system:
       - drops the transport connection, 
       - releases all BGP resources,
       - sets connect retry count to zero
       - resets the connect retry timer (sets to zero)
      [Action Z in the FSM table]
       - goes to Idle state.

  In response to the ConnectRetry timer expired event(Event 
  9), the local system: 
      - Sets the MIB FSM error information with ConnectRetry 
        expired, 
      - drops the transport connection 
      - restarts the ConnectRetry timer
      - initiates a transport connection to the other BGP 
        peer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
      [Action O in the FSM table]
      - stays in Connect state.

 In response to any other events [Events 7-8, 10-11, 18, 20-
 27] the local system:
     - resets the connect retry timer (sets to zero),
     - drops the Transport connection,
     - release all BGP resources, 
     - increments the ConnectRetryCnt by 1,
     - [optionally] performs bgp peer oscillation damping, and
     [Action D in the FSM table],
     - goes to Idle state.

Active State:

 In this state BGP is trying to acquire a peer by listening 
 for and accepting a transport protocol connection.  

 A transport connection succeeds [Event 15 or Event 16], the 
 local system: process the transport connection flags
	- If the BGP delay open flag is set:
          o clears the ConnectRetry timer, 
          o completes the BGP initialization,
          o sets the BGP delay Open timer
         [Action ZZ]

        - If the BGP delay open flag is not set:
          o clears the ConnectRetry timer,
          o completes the BGP initialization, 
          o sends the Open message to it's peer, 
          o sets its Hold timer to a large value, 
         [Action H in the FSM table]
          and changes its state to OpenSent.  

 A Hold timer value of 4 minutes is suggested.  

 If the local system receives a valid Transport Indication
 [Event 13], the local system processes the Transport flags
  (actions aa or ab in FSM section 2.3.4).  

 If the local system receives a Transport indication
 that is invalid for this connection [Event 14]:
      - the transport connection is rejected
        [Action L in the FSM table]

 If the local system receives a Transport connection
 failed [Event 17] (timeout or receives Transport
 disconnect), the local system will:
	- set Transport disconnect in the MIB reason code,
	- restart ConnectRetry timer (with initial value)
	- release all BGP resources
	- Acknowledge the drop of Transport connection if
          transport disconnect (If TCP, send a FIN ACK),
	- Increment ConnectRetryCnt by 1,
	- perform the BGP peer oscillation damping process [2]
	[Action Y in the FSM table]

 If the local system has the Delay Open timer expired [event 
 12] local system: 
        - clears the Connect Retry timer (set to zero),
	- stops and clears the Delay Open timer (set to zero) 
        - completes the BGP initialization,
        - sends the Open message to it's remote peer,
        - sets it's Hold timer to a large value,
        [Action H in the FSM table]
        - and set the state to Open Confirm.

 A Hold timer value of 4 minutes is also suggested for this 
 state transition.

 If an Open is received with the BGP Delay Open timer is 
 running [Event 19], the local system 
	- clears the connect retry timer (cleared to zero),
	- Stops and clears the BGP Open Delay timer  
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message
	- Set the Hold timer to a large value (4 minutes),
	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 In response the ConnectRetry timer expired event[Event9], 
 the local system:
        - restarts the ConnectRetry timer (with initial value),
        - initiates a transport connection to the other BGP 
          peer,
        - Continues to listen for transport connection that may be 
          initiated by remote BGP peer, 
        [Action F in the FSM table]
	- and changes its state to Connect.  

 The start events [Event1, 3-6] are ignored in the Active 
 state.

 A manual stop event[Event2], the local system:
        - Sets the administrative down in the MIB reason code,
	- Sends a notification with a Cease,
	- If any BGP routes exist, delete the routes
	- release all BGP resources, 
	- drops the Transport connection, 
        - sets connect retry count to zero
        - resets the connect retry timer (sets to zero)
        [Action S in the FSM table]
        - goes to Idle state.

 In response to any other event (Events 7-8, 10-11,18, 20-
 27), the local system:
        - stores the MIB information to indicate appropriate 
          error [FSM for Events 7-8, 10-11, 18, 20-27]
        - reset the connect retry timer (sets to zero),
        - release all BGP resources, 
	- drops the transport connection, 
        - increments the ConnectRetryCnt by one,
        - optionally performs BGP peer oscillation damping, 
  	  [Action D in FSM table],
	- and goes to the idle state

Open Sent:

 In this state BGP waits for an Open Message from its peer.  

 When an OPEN message is received, all fields are checked 
 for correctness.  If there are no errors in the OPEN message 
 [Event 18] the local system:
       - resets the BGP Delay timer to zero,
       - reset BGP Connect Timer to zero, 
       - sends a KEEPALIVE message and
       - sets a KeepAlive timer (via the text below)
       - sets the Hold timer according to the negotiated value 
         (see section 4.2), and 
         [Action N in the FSM table]
       - sets the state to Open Confirm.

 If the negotiated Hold time value is zero, then the Hold 
 Time timer and KeepAlive timers are not started.   If the 
 value of the Autonomous System field is the same as the 
 local Autonomous System number, then the connection is an 
 "internal" connection; otherwise, it is an "external" 
 connection.   (This will impact UPDATE processing as 
 described below.)  

 If the BGP message header checking [Event20] or OPEN message
 check detects an error (see Section 6.2)[Event21], the local system:
       - sends a NOTIFICATION message with appropriate error 
         code, 
       - reset the connect retry timer (sets to zero),
       - if there are any routes associated with the BGP session,
	 delete these routes
       - release all BGP resources,
       - drop the transport session 
       - increments the ConnectRetryCnt by 1, 
       - bgp peer oscillation damping process,  
       [Actions I, J in FSM table, depending on error],
       - and goes to the Idle state.

 Collision detection mechanisms (section 6.8) need to be 
 applied when a valid BGP Open is received [Event 18 or 
 Event 19].  Please refer to section 6.8 for the details of 
 the comparison. An administrative collision detect is when 
 BGP implementation determines my means outside the scope of 
 this document that a connection collision has occurred.   

 If a connection in Open Sent is determined to be the 
 connection that must be closed, an administrative collision 
 detect [Event 22] is signaled to the state machine. If such 
 an administrative collision detect dump [Event 22] is 
 received in Open Sent, the local system:  
       - sets MIB state information to 
         collision detect closure, 
       - send a NOTIFICATION with a CEASE
       - resets the connect retry timer,
       - release all BGP resources,
       - drop the transport connection,
       - increments ConnectRetryCnt by 1,
       - performs any BGP peer oscillation damp process, and
       [Action R in the FSM],
       - enters Idle state.

  If a NOTIFICATION message is received with a version 
  error[Event23], Notification message without version number 
  [Event 24], the local system: 
       - resets the connect retry timer (sets to zero)
       - drops the Transport connection,
       - releases all BGP resources, 
       - increments the ConnectRetryCnt by 1
       - process any BGP peer oscillation damping,
	[Action Y]  
       - and sets the state to Idle.

 The Start events [Event1, 3-6] are ignored in the OpenSent 
 state.

  If a manual stop event [Event 2] is issued in Open sent 
 state, the local system:
	- Sets administrative down reason in MIB reason,
	- sends the Notification with a cease,
	- if BGP routes exists, delete the routes,
	- Release all BGP resources, 
	- Drops the Transport connection, 
	- set ConnectRetryCnt to zero, 
	- resets the Connect Retry timer (set to zero),
	[Action S in the FSM table], and
	- transitions to the Idle state.

  If an automatic stop event [Event 7] is issued in Open sent 
  state, the local system:
	- Sets administrative down reason in MIB reason,
	- sends the Notification with a cease,
	- if any routes are associated with te BGP session,
	  delete the routes, 
	- release all the BGP resources
	- Drops the Transport connection, 
	- increments the ConnectRetryCnt by 1, 
	- BGP peer oscillation process [2]
	[Action C in the FSM table], and
	- transitions to the Idle state.

 If the Hold Timer expires[Event 10], the local system:
       - set Hold timer expired in MIB Error reason code, 
       - send a NOTIFICATION message with error code Hold 
         Timer Expired,
       - reset the connect retry timer (sets to zero),
       - releases all BGP resources,
       - drops the Transport connection, 
       - increments the ConnectRetryCnt by 1
         [Action K in the FSM table], and
	- transitions to the Idle state.

 If a transport indication is received for valid connection 
 [Event 13] or transport Request Acknowledgement [Event 15]
 is received, or a transport connect confirm [Event 16] is
 received a second TCP session may be in progress.  This
 second TCP session is tracked per the Call Collision 
 processing (section 6.8) until an OPEN message is received.

 A TCP connection for an invalid port [Event 14] is ignored. 

 If a Transport Failure [Event17], is received from the 
 underlying transport protocol, the local system:
       - closes the BGP connection, 
       - restarts the Connect Retry timer, 
       - and continues to listen for a connection that may be 
         initiated by the remote BGP peer,
       [Action O in the FSM table]
       - and goes into Active state.

 In response to any other event [Events 8-9, 11-12, 19, 25-27], 
  the local system:
	- sends the NOTIFICATION with the Error Code Finite 
          state machine error,
        - resets the connect retry timer (sets to zero),
	- releases all BGP resources
        - drops the Transport connection, 
        - increments the ConnectRetryCnt by 1, 
        - process any bgp peer oscillation damping[2] 
        [Action E in the FSM table],
        - and sets the state to idle.

Open Confirm State

 In this state BGP waits for a KEEPALIVE or NOTIFICATION 
 message.

 If the local system receives a KEEPALIVE message[Event 25], 
        - restarts the Hold timer, 
          [Action P in the FSM table], and
        - changes its state to Established. 

 If the local system receives a NOTIFICATION message [event 
 23-24] or receives a TCP Disconnect [event 17] from the 
 underlying transport protocol, the local system:  
        - sets the appropriate MIB information for FSM error, 
        - resets the connect retry timer (sets the timer to 
          zero),
	- releases all BGP resources, 
        - drops the TCP connection,
        - increments the ConnectRetryCnt by 1, 
        [Action Y in the FSM table],
        - and sets the state to idle.

 Any start event [Event1, 3-6] is ignored in the OpenConfirm 
 state.

 In response to a manual stop event[Event 2] initiated by 
 the operator, the local system: 
	- set Administrative down in MIB Reason code,
        - sends the NOTIFICATION message with Cease, 
	- if any BGP routes, dete the routes
        - releases all BGP resources, 
	- drop the transport connection, 
        - sets the ConnectRetryCnt to zero
        - sets the connect retry timer to zero
        [Action S in the FSM table]
        - transitions to Idle state.

 In response to the Automatic stop event initiated by the 
 system[event 7], the local system:
        - sets the MIB entry for this peer to administratively 
          down, 
        - sends the NOTIFICATION message with Cease,
	- connect retry timer reset (set to zero)
	- If any BGP routes exist, delete the routes,
	- release all BGP resources,
        - drops the transport connection,
        - increments the ConnectRetryCnt by 1, 
        [Action C in the FSM table], and
        - transitions to the Idle State.

 If the Hold Timer expires before a KEEPALIVE message is 
 received [event10], the local system:
        - set the MIB reason to Hold time expired, 
        - send the NOTIFICATION message with the error code 
          set to Hold Time Expired, 
        - resets the connect retry timer (sets the timer to to 
          zero),
	- releases all BGP resources, 
        - drops the transport connection, 
        - increments the ConnectRetryCnt by 1 
          [Action K in the FSM table], 
        - and sets the state to Idle.

 If the local system receives a KEEPALIVE timer expires 
 event [Event 11], the system:
        - sends a KEEPALIVE message,
        - restarts the Keepalive timer 
        [Action Q the in FSM table],and 
        - remains in Open Confirmed state.

 In the event of TCP establishment [Event 13], or TCP 
 connection succeeding [Event 15 or Event 16] while in Open 
 Confirm, the local system needs to track the 2nd 
 connection.  

 If a TCP connection is attempted to an invalid port [Event 
 14], the local system will ignore the second connection 
 attempt.  

 If an OPEN message is received, all fields are check for 
 correctness.  If the BGP message header checking [Event20] 
 or OPEN message check detects an error (see Section 
 6.2)[Event21], [the local system:
        - sends a NOTIFICATION message with appropriate error 
          code, 
        - resets the connect retry timer (sets the timer to 
          zero),
        - releases all BGP resources, 
        - drops the TCP connection,
        - increments the ConnectRetryCnt by 1, 
        - runs the BGP peer oscillation damping process [2]  
       [Actions I, J, in the FSM table depending on the error]
        - and goes to the Idle state.

 If the Open messages is valid [Event 18], the collision 
 detect function is processed per section 6.8.  If this 
 connection is to be dropped due to call collision, the 
 local system:
        - sets the Call Collision cease in the MIB reason 
         code,
        - sends a Notification with a Cease 
        - resets the Connect timer (set to zero),
	- releases all BGP resources,
	- Drops the TCP connection (send TCP FIN),
	- increments the ConnectRetryCnt by 1, and
	- performs any BGP peer oscillation damping process [2].
	[action 

 If during the processing of another Open message, the BGP 
 implementation determines my means outside the scope of 
 this document that a connection collision has occurred and 
 this connection is to be closed, the local system will 
 issue a call collision dump [Event 22].  When the local 
 system receives a call collision dump event [Event 22], the 
 local system:
        - Sets the MIB FSM variable to indicate collision 
          detected and dump connection.
        - send a NOTIFICATION with a CEASE
        - deletes all routes associated with connection,
        - resets the connect retry timer,
        - releases all BGP resources
        - drops all TCP connection,
        - increments the ConnectRetryCnt by 1, 
        - and performs any BGP peer oscillation damping,
        [Action R in the FSM table],
        - enters Idle state.

 In response to any other event [Events 8-9, 12, 19, 26-27], 
 the local system:
        - sends a NOTIFICATION with a code of Finite State 
          Machine Error,
        - resets the connect retry timer (sets to zero)
        - drops the TCP connection,
        - releases all BGP resources, 
        - increments the ConnectRetryCnt by 1, 
        - performs any BGP peer oscillation damping,
        [Action E in the FSM table], and 
        - transitions to Idle state.

Established State:

 In the Established state BGP can exchange UPDATE, 
 NOTFICATION, and KEEPALIVE messages with its peer.

 If the local system receives an UPDATE message [Event26], 
 the local system will:
	- process the update packet
	- restarts its Hold timer, if the negotiated Hold Time 
          value is non-zero.
	[Action W in the FSM table], and
	- remain in the Established state.

 If the local system receives a NOTIFICATION message 
 [Event23 or Event24] or a disconnect [Event17] from the 
 underlying transport protocol, it:
	- sets the appropriate error code in MIB reason code,
	- if any BGP routes exist, delete all BGP routes,  
        - resets the connect retry timer (sets to zero), 
        - releases all the BGP resources, 
	- drops the TCP connection,
	- increments the ConnectRetryCnt by 1, and 
       [Action T in the FSM table]   
	- Goes to the Idle state.

 If the local system receives a Keepalive message
 [Event 25], the local system will:
        - restarts its Hold Timer, if the negotiated Hold Time 
          value is non-zero.
       [Action P in the FSM table], and
	- remain in the Established state.

 If the local system receives an UPDATE message, and the 
 Update message error handling procedure (see Section 6.3) 
 detects an error [Event27], the local system:
       - sends a NOTIFICATION message with Update error,
       - resets the connect retry timer (sets to zero),
       - drops the TCP connection,
       - releases all BGP resources, 
       - increments the ConnectRetryCnt by 1 
       - performs any BGP peer oscillation damping
      [Action U in FSM table], 
       - and goes to Idle state.

 Any start event (Event 1, 3-6) is ignored in the 
 Established state.

 In response to a manual stop event (initiated by an 
 operator)[Event2]:
        - sets the Administrative stop in MIB reason code, 
	- sends the NOTIFICATION message with Cease, 
        - if BGP routes exist, delete the BGP routes, 
        - release BGP resources,
	- drop TCP connection, 
        - set ConnectRetryCnt to zero (0),
        - reset connect retry timer to zero (0), and
        [Action S in FSM table]
        - transition to the Idle.

 In response to an automatic stop event initiated by the 
 system (automatic) [Event7], the local system:
     - sets Administrative Stop in MIB Reason code, 
     - sends a NOTIFICATION with Cease,
     - resets the connect retry timer (sets to zero)
     - deletes all routes associated with bgp peer session,
     - releases all BGP resources, 
     - drops the transport connection,   
     - increments the ConnectRetryCnt by 1, 
     - performs any BGP peer oscillation damping, and 
     [Action C in FSM table]
     - transitions to the idle state.

 An example automatic stop event is exceeding the number of 
 prefixes for a given peer and the local system 
 automatically disconnecting the peer.

 If the Hold timer expires [Event10], the local system:
      - sends a NOTIFICATION message with Error Code Hold 
        Timer Expired,
      - resets the connect retry timer (sets to zero),
      - releases all BGP resources, 
      - drops the TCP connection, 
      - increments the ConnectRetryCnt by 1,  
      -	performs any BGP peer oscillation damping,
      [Action M in FSM table] 
      - and goes to Idle state.

 If the KeepAlive timer expires [Event11], the local system 
 sends a KEEPALIVE message, it restarts its KeepAlive timer, 
 unless the negotiated Hold Time value is zero [Action Q].

 Each time time the local system sends a KEEPALIVE or UPDATE 
 message, it restarts its KeepAlive timer, unless the 
 negotiated Hold Time value is zero.

 A transport connection indication [Event 13] received
 for a valid port will cause the 2nd connection to be
 tracked.  A transport connection indications for 
 invalid port [Event 14], will be ignored. 
 
 In response to a Transport connection succeeds [Event 15
 or Event 16], the 2nd connection shall be tracked until
 it sends an OPEN message.  

 If a valid Open message [Event 18] is received, it will be 
 checked to see if it collides (section 6.8) with any other 
 session. If the BGP implementation determines that this 
 connection needs to be terminated, it will process an Call 
 Collision dump event[Event 22].  If this session needs to be 
 terminated, the connection will be terminated by:

     - send a NOTIFICATION with a CEASE
     - deletes all routes associated with connection,
     - resets the connect retry timer,
     - if any BGP routes, delete the routes,
     - release all BGP resources, 
     - drops the transport connection,
     - increments ConnectRetryCnt by 1,
     - and performs any BGP peer oscillation damping,
    [Action R in the FSM table],
     - and enters the Idle state

 In response to any other event [Events 8-9,12, 19-21] the 
 local system:
     - sends a NOTIFICATION message with Error Code Finite 
       State Machine Error,
     - deletes all routes associated with BGP peer session, 
     - resets the connect retry timer (sets to zero)
     - releases all BGP resources,
     - drops the TCP connection,
     - increments the ConnectRetryCnt by 1, 
     - performs any BGP peer oscillation damping, and
     [Action E in FSM table],
     - transitions to Idle.

--19701020
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline

--19701020
Content-Type: text/plain; name="ietf-hares-backoff-01.txt"; x-unix-mode=0644
Content-Disposition: attachment; filename="ietf-hares-backoff-01.txt"

   Internet Draft                                              S. Hares 
   Document: draft-hares-bgp-backoff-01.txt                     NextHop 
                                                          Technologies, 
                                                                   Inc. 
   Expires: December 2002                                     July 2002 

                   BGP Peer Restart Backoff Mechanisms 

 Status of this Memo 
 
   This document is an Internet-Draft and is in full conformance 
   with all provisions of Section 10 of RFC2026. 
    
   Internet-Drafts are working documents of the Internet Engineering 
   Task Force (IETF), its areas, and its working groups.  Note that      
   other groups may also distribute working documents as Internet-
   Drafts. 
    
   Internet-Drafts are draft documents valid for a maximum of six 
   months and may be updated, replaced, or obsoleted by other documents 
   at any time.  It is inappropriate to use Internet-Drafts as 
   reference material or to cite them other than as "work in progress." 
    
   The list of current Internet-Drafts can be accessed at 
        http://www.ietf.org/ietf/1id-abstracts.txt 
  
   The list of Internet-Draft Shadow Directories can be accessed at 
        http://www.ietf.org/shadow.html. 

 Abstract 
    
   This document describes mechanisms for methods for damping the 
   oscillations of BGP-4 [1] peers by increasing the time between 
   automatic start functions of the BGP peers.  The increase of time 
   between automatic start functions will be denoted as a backoff of 
   the automatic start functions by this draft. 
    
   This draft is an informational RFC which provides a description of 
   mechanisms for back-off of the automatic start function in bgp-4 
   peer reconnections. This Informational RFC includes descriptions 
   about the back-off mechanisms and how this information is recorded 
   in the BGP-4 MIB version 2 [2]. 
    
   The expontential back-off mechanism contained in this draft was 
   first recorded in a draft of the bgp-4 specification.  The author 
   requests that anyone with information regarding the original authors 
   of that work contact the author.  The lack of credit for those
   authors is based on the author's lack of knowledge the originators
   of the expotential back-off.   

 Hares             Informational Expires December 2002                 1 

Internet Draft      ietf-skh-bgp-backoff-01.txt           June 20, 2002
    
   Table of Contents 

   Status of this Memo...........................................1
 
   Abstract     .................................................1 

   1.0 Overview    ..............................................2 

   2.0 Conventions used in this document.........................3 

   3.0 Types of Back-off.........................................4 

   4.0 Exponential back-off......................................4 

   5.0 Step-wise Back-off .......................................5 

   6.0 Recording the Back-off information in the MIB ............5 

   7.0 Back-off Mechanisms in FSM ...............................6

      7.1 Variables to support Back-off functions................6
      7.2 Existing MIB objects used .............................8 
      7.3 Events only utilized by the back-off mechanisms .......8 
      7.4 IdleHold substate in Idle FSM description .............8 
      7.5 Action upon entering the Idle state ...................9
      7.6 Substate processing...................................10
 
   8.0 Security Considerations..................................13

   9.0 References...............................................13

   10. Author's Addresses.......................................13

 1.0 Overview 

   If a BGP speaker detects an error, it shuts down the connection 
   and changes its connection state to Idle. Getting out of the Idle
   state requires generation of the Start event.  If such an event is 
   generated automatically, then persistent BGP errors may result in 
   persistent flapping of the speaker.  To avoid such a condition it  
   is recommended that automatic start events should not be  
   generated immediately for a peer that was previously transitioned  
   to Idle due to an error. 

   For a peer that was previously transitioned to Idle due to an  
   error, the time between consecutive generation of start events,  
   if such events are generated automatically, shall increase.  This  
   increase of time between automatic start events will be called in  
   backoff of automatic BGP peer start events.   

   The automatic start events listed in the BGP-4 specification are:  
     
         - Event3: Automatic start 
         - Event5: Automatic start with passive flag on 
         - Event6: Automatic start with bgp_flap_stop flag on 

  Event 3 is issued when an automatic stop occurs when no additional
  flags impact the starting of the bgp peer session.  Event 5 is 
  generated when the passive flag is set to allow the local peer to
  listen prior to starting a TCP session. 

   Hares         Informational - Expires December 2002                2 

Internet Draft      ietf-skh-bgp-backoff-01.txt        June 20, 2002

  Event 6 is issued when an automatic start occurs with the
  bgp peer oscillation damping features turned on.  The 
  "bgp_flap_stop" flag engages the damping of bgp peer oscillation.

   Event 7: Automatic stop is the only automatic stop event for the 
   BGP finite state machine.   Automatic stop followed by automatic 
   start can also accelerate the BGP peer oscillation. 

   BGP-4 implementations today have a variety of back-off  
   mechanisms.  This document is intended to provide information on  
   these back-off mechanisms and how to record this information in a  
   MIB.  Two mechanisms are recorded in this Internet Draft:  

	1) Exponential back-off (section 2) and 
        2) Step-wise back-off (section 3).  

   The author welcomes input on additional back-off mechanisms.   
   Please send input to the author or the idr list ([email-masked]). 
 
   A BGP peer oscillation backoff mechanism requires additional
   processing  in the bgp-4[1] Finite State Machine (FSM).
   The BGP-4 draft contains an  indication of where the FSM would
   process the peer oscillation damping.  The implementation of 
   backoff mechanisms in a BGP-4 is  optional, but wide spread
   in implementations.  The exact  mechanisms vary between
   implementations.   

   Section 6.0 contains information on how the BGP-4 MIB encodes the  
   time between establishing BGP-4 peer sessions.  Section 7.0  
   contains the additional mechanisms needed in a BGP-4 FSM for  
   back-off implementations.  

2.0 Conventions used in this document 
    
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL  
   NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED",  "MAY", and  
   "OPTIONAL" in this document are to be interpreted as described in  
   RFC-2119 [1]. 
    
   As an informational draft there are no mandatory functions.  This  
   draft hopes to recommend healthy life styles for back-off  
   mechanisms in BGP peers.  As in all recommendations for healthy  
   life styles the user of BGP-4 is not required to do anything,  
   except expire the BGP-4 connection according to the letter of the  
   BGP-4 standard.  

   Hares         Informational - Expires December 2002             3  

Internet Draft      ietf-skh-bgp-backoff-01.txt           June 20,2002 

3.0 Types of Back-off  

   The author knows of three types of back-off at this time: 
    
        - Exponential back-off, and 
        - Step-wise back-off,  
        - no back-off. 
    
   The author welcomes any comments on additional back-offs and will 
   hasten to include this information in the next revision of this  
   draft. 

4.0 Exponential back-off  

   The original description occurred in RFC 1654 and was included in
   a version of bgp-4 [1] draft. 

   If a BGP speaker detects an error, it shuts down the connection  
   and changes its state to Idle. Getting out of the Idle state  
   requires generation of the Start event.  If such an event is  
   generated automatically, then persistent BGP errors may result in  
   persistent flapping of the speaker.  To avoid such a condition it  
   is recommended that automatic Start events should not be  
   generated immediately for a peer that was previously transitioned  
   to Idle due to an error.  
    
   For a peer that was previously transitioned to Idle due to an  
   error, the time between consecutive generation of Start events,  
   if such events are generated automatically, shall exponentially  
   increase.   The IdleHoldTimer will be the timer is utilized for  
   the count down of this feature.    The value of the initial timer  
   shall be 60 seconds. The time shall be doubled for each  
   consecutive retry.    The equation for the setting of the Idle  
   time is as follows: 
    
        Idle Hold timer = 2**(ConnectRetryCnt)*60 
    
   If the Idle Hold timer exceeds a local maximum value, the  
   connection shall be held down until a manual start event occurs. 
   
   The Idle Hold timer is initialized to zero.     

   Hares         Informational - Expires December 2002            4 

Internet Draft      ietf-skh-bgp-backoff-01.txt        June 20,2002

5.0 Step-wise Back-off 

   The Step-wise back-off uses the following back-off function. The  
   step-wise back-off functions provide a step function based on the  
   number of retries.   
    
   An example of the Step-wise code, is as follows:  
    
     - For 1-8 retries, the amount of time between retries is 8 
       seconds. 
    
     - For 9-24 retries, the amount of time between retries, the  
       amount time between retries is 64 seconds, 
    
     - For larger than 25 retries, the amount of time between retries 
       is 128 seconds, 
    
     - Maximum number of retries is 35. 

6.0 Recording the Back-off information in the BGP-4 MIB [3]

   The BGP-4 MIB version 2[3] will have two variables for the hold time  
   Maximums: 
    
     BGP-4 Maximum retry count value 
     BGP-4 Maximum hold time value 

   The BGP-4 MIB version 2 will have scalar flag indicating that the
   a backoff implementation is present in a BGP peer implementaiton
 
   The BGP-4 MIB version 2 will have the following table for BGP-4  
   Restart Peer Back-off: 

   BGP-4 Peer Restart Back-off table 
    
   retry cnt  hold time value (seconds) 
   --------   --------------- 
   0             value-1
   1             Value-1  
   2             Value-2  
   3             Value-3 
   4             Value-4 
   5             Value-5 
   6             Value-6 
   … 
   max-cnt    max-cnt-value                           

   For our example of the step-wise back-off, the table would be  
   filled in as 
    
   Retry cnt    hold time value 
   ---------    ---------------
   1              8 seconds 
   2              8 seconds 
   3              8 seconds 
   4              8 seconds 
   5              8 seconds 

   Hares         Informational - Expires December 2002                5 

 Internet Draft      ietf-skh-bgp-backoff-01.txt        June 20,2002

   6              8 seconds 
   7              8 seconds 
   8              8 seconds 
   9             64 seconds 
   10            64 seconds 
   … 
   24            64 seconds 
   25           128 seconds 
   26           128 seconds 
   .. 
   35           128 seconds 
    
   BGP-max retry cnt =35 
   BGP-max hold time = 128 seconds 

7.0) Back-off Mechanisms in FSM 
    
 7.1) Per BGP Peer Variables to support Back-off functions
    
   1) Idle Hold timer 
    
        Definition: Timer that keeps track of exponential back-    
                    Off.  This timer must expire before the bgp peer
                    generates an automatic start event.    
    
        Units:      Seconds 
        Status:     Needs to be added to BGP-4 MIB version 2[3]
                    In running timers. [new category] 

   2) bgp-4 Last Hold Timer value 
    
        Definition: Time last set in the Idle Hold timer 
    
        Units:      seconds 
        Status:     Needs to be added to BGP-4 MIB version 2 [3] 
                    In running timers [new category] 

   3) Idle Hold Time Maximum  
    
        Definition: Maximum value for Idle Hold timer 
        Units:      Seconds 
        Status:     Needs to be added to BGP-4 MIB version 2 [3]
                    In configured timers.   

    
   Hares         Informational - Expires December 2002            6 

Internet Draft      ietf-skh-bgp-backoff-01.txt        June 20,2002

   
   4) Keep Idle Flag  
    
        Definition: Flag set if the idle Hold timer value 
                    exceeds the Idle hold maximum value. 
         
        Units:      true/false 
        Status:     Needs to be added to BGP-4 MIB version 2 in 
                    Running status. 

  5) bgp_stop_flap flag  
    
        Definition: Flag to enable the back off mechanisms 
                    on automatic restarts.  These mechanism 
                    prevent persistent bgp peer oscillation. 
    
        Units:      True/False 
        Status:     Needs to be added to BGP-4 MIB Version 2 
                    In configured options.   
                           
   6) BGP-4 max retry count 
      
       Definition:  maximum number of times a bgp-4 peer will 
                    Automatically try to restart the bgp-4 connection. 
    
       Units:       non-zero, positive integer 
       Status:      Needs to be added to BGP-4 MIB in configured 
                    Parameters  

   7) BGP retry count 
      Definition:   Count of times the bgp-4 peer automatically 
                    Attempts to restart the connection. 
    
      Units:        non-zero, positive integer 
      Status:       Needs to be added to BGP-4 MIB in running 
                    Running parameters.  

   8) BGP-Backoff-RetryTimeTable 
 
      Definition:   Table that specifies the exact ConnectRetryInterval 
                    Per each time the bgp-4 peer attempts to  
                     automatically retart. 
 
      Units:         Table-index = cnt 
                     Table-value = timer in seconds 
 
   9) Idle Substate  
 
      Definition:    Idle Substate value (defined in section 7.3) 
      Units:         0 = Idle hold Null 
                     1 = Idle Hold wait  
                     2 = Idle Hold down 
                     3 = Idle Hold ticking 

   Hares         Informational - Expires August 2002               7 

Internet Draft      ietf-skh-bgp-backoff-01.txt     February 28,2002

 7.2 Existing MIB objects used 
    
   The back-off functions are implemented as a processing within the 
   Idle state in the BGP-4 FSM.  The FSM processing for the BGP-4 also 
   uses the following variables (found in BGP-4 MIB version 2). 

   1)   bgpPeerState  
         
        Definition: From the BGP-4 MIB version 2 [2] "The BGP Peer's  
                    FSM state". 
    
        status:     Exists in BGP-4 MIB version 2. 
    
   2)   bgpPeerConnectRetryInterval  

        Definition: Time interval in seconds for the ConnectRetry  
                    Timer.  The suggested value for this timer is 120 
                    Seconds if no back-off mechanism is engaged.  If a 
                    Back-off mechanism is engaged, the bgpPeerBackoff 
                    Table entry is used for the value of  
                     ConnectRetryCnt. 
    
        status:     Exists in BGP-4 MIB version 2. 

    
7.3 Events only utilized by the back-off mechanisms 

   The following optional events are defined in section 8.1 of the BGP-4 Draft [1]. 
    
        Event3: automatic start with bgp_stop_flap option 
        Event6: Idle Hold timer expires 

 7.4 IdleHold substate in Idle FSM description 

   The Idle Hold has the following four substates: 
    
   IdleHoldWait 
   IdleHoldDown 
   IdleHoldTicking 
   IdleHoldNull
    
    The definitions of these sub-states are:  

   Hares         Informational - Expires December 2002               8 

Internet Draft      ietf-skh-bgp-backoff-01.txt            June 20, 2002

   Substate1:    Idle Hold Wait  
    
    Definition:  Idle Hold Down timer has expired. Local system is  
                 waiting for auto-start request on bgp peer session. 

     Sub-state 
     Status  
     flags:      Idle Hold Down timer is zero. 
                 Keep Idle flag is clear. 

   Substate2:    Idle Hold down: 
                  
     Definition: Keep Idle flag is set, and Automatic start is  
                 disabled for peer session.   
    
     Sub-state 
     status 
     flags:      Keep Idle flag is set.

   Substate3:    Idle Hold timer ticking: 
    
    Definition:  Idle Hold Timer is ticking (running) 
                 and local system will not automatically 
                 restart the BGP session. 
    
    Variable:    Idle Hold Timer is non-zero. 

  Substate4:     Idle Hold Null

    Definition:   No Idle Hold functions are being performed.

    Sub-state 
     status 
      flags:	"Keep Idle flag" is clear,
	         Idle Hold timer is zero
		 Idle Hold off  

   How these sub-states are represented internally is an implementation 
   matter.  The important part is that these sub-states be available to 
   the BGP MIB. 

7.5 Action upon entering the Idle state

   Processing upon entering the Idle state: 
    
   If bgp_stop_flap flag on, the Idle Hold sub state is entered: 
    
   1)Idle Hold time = connect-retry-table[connect-retry-count] 
    
   2)Idle Hold Timer = Idle Hold time 
    
   If (Idle hold time > Idle Hold time maximum), then: 
         
        1) bgp peer sets the keep-idle-flag 
        2) set substate to Idle Hold down 
        3) clear Idle Hold timer 
          
   else 
        1) increment ConnectRetryCnt, 
        2) start Idle Hold timer with value set above 
        3) set Idle-hold substate to "Idle Timer ticking" 

   If bgp_stop_flag is off, this sub-state action should not occur.  

   Hares         Informational - Expires November 2002            9 

Internet Draft      ietf-skh-bgp-backoff-01.txt          June 20,2002

7.6 Substate processing  

   The events are defined from the BGP-4 draft in section 8 on 
   BGP-4 FSM [2].  This processing occurs within the IDLE state if the 
   Bgp peer oscillation back-off mechanism are used. 

   The form of the table is state/sub-state/action. 

                Idle State      Idle State  Idle State    Idle state    
                Substate 1      Substate2   Substate3     Substate4 
   # Event      Idle Hold       Idle Hold   Idle Hold     null
                Wait            Down        timer         
                                            ticking 
    
   ------------------------------------------------------------     
   1 Manual    Connect/         Connect /    Connect/      Connect 
      start    null/            null/        null/         null/ 
		   A1           A2               A3        A1
   ------------------------------------------------------------ 
   2  Manual   Idle/           Idle/         Idle/         Idle/    
        stop   null/           null/         null/         null/
                Ignore          A4            A5           Ignore 
   ------------------------------------------------------------ 
   3  Auto     Connect         Connect/      Connect      Connect
       start   null/           null/         null/         null/ 
                  F1            F2                F3       F1 
   ------------------------------------------------------------- 
   4  Manual   Active /       Active /      Active/       Active/ 
      start    null/          null/         null/         null/ 
      with     B1             B2            B3            B1
      passive              
   ------------------------------------------------------------- 
   5  Auto      Active /      Active /      Active /      Active 
      start     null/         null/         null/         null/
      with      B1            B2            B3            B1
      passive              
   ------------------------------------------------------------ 
   6  Automatic Connect/     Idle/          Idle/       Connect/
      start &   null/        Idle hold      Idle hold   null/ 
      bgp_stop_              down/          ticking/  
      _flap on  B1           ignore          ignore      B1
   -------------------------------------------------------------- 
   7  Automatic Idle/        Idle/          Idle/        Idle/       
      Stop      Idle Hold    Idle Hold      Idle Hold          
                Wait/        down/          tick/   
                Ignore       Ignore          Ignore 
   --------------------------------------------------------------- 
   8  Idle Hold Idle/        Idle/          Idle/        Idle/ 
      timer     Idle Hold    Idle Hold      Idle Hold    null/
      expires   wait/        down/          wait/        null/
                V            Ignore         A6           V   
   ---------------------------------------------------------------- 

   Hares         Informational - Expires August 2002               10 

Internet Draft      ietf-skh-bgp-backoff-01.txt     February 20,2002

   Action A1:  
     
   1.   Initialize all BGP resources
   2.   Set ConnectRetryCnt to zero
   3.   Start Connect retry timer with initial value 
   4.   Initiate transport connection Request to the remote BGP peer
   5.   Listen for connection indication from remote BGP peer 
    
   Action A2:  
    
   1.   Stop and Clear Idle Hold timer 
   2.   Clear Keep Idle flag 
   3.   initialize all BGP resources 
   4.   ConnectRetryCnt set to zero 
   5.   Start Connect retry counter with initial value  
   6.   Initiate transport connection request to remote BGP peer
   7.   Listen for connection indication from remote BGP peer 
    
	Note: items 3-7 are the same actions as Action A of the 
	      BGP FSM.

   Action A3: 
 
   1.   Stop and Clear Idle Hold Timer 
   2.   initialize all BGP resources 
   3.   ConnectRetryCnt set to zero 
   4.   Start Connect retry counter with initial value  
   5.   Initiate transport connection to BGP peer by
        send a TCP syn
   6.   Listen for connection (TCP syn, ack) from remote Peer
    
 	Note: items 2-6 are the same actions as action A of the BGP
	      FSM.

   Action A4:    
   1.   clear Keep Idle flag 

   Action A5: 
   1.   Clear Idle Hold Timer 

   Action A6:  
    
   1.  Idle Hold substate = Idle Hold Wait 
   2.  Clear Idle Hold Timer 

   Hares         Informational - Expires December 2002               11 

  Internet Draft      ietf-skh-bgp-backoff-01.txt     June 20, 2002

   Action B1: 
    
   1.   Initialize all BGP resources 
   2.   ConnectRetryCnt set to 0 
   3.   Start Connect Retry timer with initial value 
   4.   Listen for connection set-up by the remote BGP peer 
	
	Note: B1 are the same actions as action B
    
   Action B2: 
    
   1.   Clear Keep Idle Flag 
   2.   Clear Idle Hold Timer 
   3.   Initialize all BGP resources 
   4.   ConnectRetryCnt set to 0 
   5.   Start Connect Retry timer with initial value 
   6.   Listen for connection set-up from the remote BGP peer 
	[TCP Syn]
	
	Note: items 3-6 are the same actions as action B. 
    
   Action B3: 
     
   1.   Clear Idle Hold Timer
   2.   Initialize all BGP resources 
   3.   ConnectRetryCnt set to 0 
   4.   Start Connect Retry timer with initial value 
   5.   Listen for transport connection indication from the remote BGP peer 

	Note: items 2-5 are the same acttions as action B. 

   Action F1:   
   1.   Restart ConnectRetry timer (with initial value)  
   2.   Initiates a transport connection request to the other bgp peer  
           [if TCP send a TCP SYN] 
   3.   Listen for remote transport connection indication 
             may be initiated from the remote BGP peer (TCP Syn)

	Note: items 1-3 are the same actions as action F	
 
   Action F2:  
   1.   Clear Idle Hold Timer 
   2.   Clear Keep Idle Flag  
   3.   Restart ConnectRetry timer (with initial value)  
   4.   Initiates a transport connection request to the remote bgp peer  
   5.   Listen for a remote transport connection indication 
	     from the remote BGP peer 

	Note: items 3-6 are the same as action F.  
		
   Action F3 
   1.   Clear Idle Hold Timer 
   2.   Restart ConnectRetry timer (with initial value)  
   3.   Initiates a transport connection request to the
        remote bgp peer  
   4.   Listen for remote transport connection that 
             may be initiated by the remote BGP peer

	Note: items 2-4 are the same as action F.

   Action v:
    
   1.   Set FSM error in MIB reason code.  

   Hares         Informational - Expires August 2002               12 

Internet Draft      ietf-skh-bgp-backoff-01.txt     June 20, 2002

8.0 Security Considerations 
   Security concerns for BGP-4 are addressed in the BGP-4 
   specification, and accompanying specifications on TCP MD5 [3] and 
   IP Security[4].  No additional considerations need to be made for 
   the BGP-4 state machine description. 
    
9.0 References 
    
   [1] RFC 2119 Bradner, S., "Key words for use in RFCs to Indicate 
       Requirement Levels", BCP 14, RFC 2119, March 1997 
    
   [2] "A Border Gateway Protocol 4 (BGP-4)" Y. Rekhter, T. Li Editors 
        http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp4-17.txt 

   [3] "Definitions of Managed Objects for the Fourth Version of Border  
       Gateway Protocol (BGP-4),Second Version",  
      http://www.ietf.org/internet-drafts/draft-ietf-idr-bgp4-mibv2- 
         01.txt 
    
   [5] "Protection of BGP Sessions via the TCP MD5 Signature Option" 
        A. Heffernan, rfc2385.txt 
    
   [6] Securing BGPv4 using IPSec , D. Ward,  
        draft-ward-bgp-ipsec-00.txt 

10. Author's Addresses 
     Susan Hares 
   NextHop Technologies, Inc 
   825 Victors Way              Phone:  1-734-222-1610 
   Ann Arbor, MI USA            Email:  [email-masked] 
     
   Hares         Informational - Expires December 2002             13

--19701020
Content-Type: text/plain; charset=US-ASCII
Content-Disposition: inline

----- End forwarded message:

--19701020--

From: "M. ELK" <[email-masked]>
To: [email-masked]
Subject: Route oscillation 
Date: Wed, 17 Jul 2002 16:19:06 +0000
Sender: [email-masked]

<html><div style='background-color:'><DIV><FONT face=Helv size=1>
<P>Hi </P>
<P>Just an&nbsp;idea from non-expert ,</P>
<P>&nbsp;</P>
<P>2 choice :</P>
<P>Choice1- </P>
<P>Assume R1 and R2 in AS1 </P>
<P>R3 in AS3 , R4 in AS4</P>
<P>R1/R3 have eBGP session , R2/R4 have eBGP session . R1/R1 have iBGP session .</P>
<P>R3 advertise path to 10.0.0.0/8 to R1 . </P>
<P>R1 advertise 10.0.0.0/8 from AS3 to R2 . </P>
<P>R4 advertise 10.0.0.0/8 to R2 . </P>
<P>R2 select the path from AS4 as the best . </P>
<P>R2 advertise the path from AS4 to R1 . </P>
<P>R1 also select the path from As4 as the best ,but instead of sending to R2 a withdraw of the path from AS3 it re-advertise the path from AS3 but this time with Community=0xFFFFFFFF </P>
<P>Community 0xFFFFFFFF = The path is "HOLD/Poison" advertise because U provide me a better path ,pls assign a distance (CISCO terminology) of 255 to</P>
<P>this path . </P>
<P>R1 associate a flag "F" with the path from As3 in the ADJ-RIB-OUT </P>
<P>The distance=255 will prevent this path to be inserted in the routing table even if it is the best path based on BGP selection . </P>
<P>Case1: R3 withdraw the path to 10.0.0.0/8 . </P>
<P>R1 will send normal withdrwal to R2 . </P>
<P>R2 delete the path from AS3</P>
<P>Case 2: R4 withdraw the path to 10.0.0.0/8 . </P>
<P>R2 now is left with the path from As3 as the best (it is the only one) but since it can not be installed in the routing table R3 just delete the entry </P>
<P>of 10.0.0.0/8 from the routing table . . </P>
<P>R2 advertise the withdraw of 10/8 from AS4 to R1 . </P>
<P>R1 select the path from AS3 as the best (the only one) . </P>
<P>Since this path from AS3 is associated with flag "F" in ADJ-RIB-OUT so R1 need to consider as if this path was not advertised to R2 before and </P>
<P>as such to be advertise (this time as normal ie: without 0xFFFFFFFF community) to R2 . </P>
<P>R2 insert the new route in the BGP ,and it will be inserted as normal in the routing table . </P>
<P>&nbsp;</P>
<P>The use of community 0xFFFFFF is exactly as if R1 is sending to R2 "distance 255 10/8" . </P>
<P>&nbsp;</P>
<P>&nbsp;</P>
<P>Choice2- </P>
<P>Assume R1 and R2 in AS1 </P>
<P>R3 in AS3 , R4 in AS4</P>
<P>R1/R3 have eBGP session , R2/R4 have eBGP session . </P>
<P>R3 advertise path to 10.0.0.0/8 to R1 . </P>
<P>R1 advertise 10.0.0.0/8 from AS3 to R2 . </P>
<P>R4 advertise 10.0.0.0/8 to R2 . </P>
<P>R2 select the path from AS4 as the best . </P>
<P>R2 advertise the path from AS4 to R1 . </P>
<P>R1 also select the path from As4 as the best , R1 send to R2 the withdraw of the path from AS3 but with Community=0xFFFFFFFF </P>
<P>Community 0xFFFFFFFF = The path is withdrwan because U provide me a better path ,pls do not delete from ADJ-RIB-IN just assign a distance (CISCO terminology) of 255 to this path . </P>
<P>R2 upon recieveing the withdraw of the path from AS3 with community 0xFFFFFFFF ,it will not remove it from the ADJ-RIB-IN but simply will </P>
<P>assign a distance of 255 and start a timer "T" . say Timer T= 60 min . </P>
<P>on expiry of timer T , the path is deleted from ADJ-RIB-IN . </P>
<P>The distance=255 will prevent this path to be inserted in the routing table even if it is the best path based on BGP selection . </P>
<P>Case1: R3 withdraw the path to 10.0.0.0/8 .</P>
<P>since R1 is already withdraw the path from As3 toward R2 so it will do nothing . </P>
<P>Providing that R3 will not advertise back again the path for a period &gt; T . The path will be stored in R2 for a max duration of 60 min . </P>
<P>ie: this residual path will be clean after a max period of 60 min . </P>
<P></P>
<P>Case 2: R4 withdraw the path to 10.0.0.0/8 . </P>
<P>R2 now is left with the path from As3 as the best (it is the only one) but since it can not be installed in the routing table R3 just delete the entry </P>
<P>of 10.0.0.0/8 from the routing table . . </P>
<P>R2 advertise the withdraw of 10/8 from AS4 to R1 . </P>
<P>R1 select the path from AS3 as the best (the only one) . </P>
<P>Since this path from AS3 is associated with flag "F" in ADJ-RIB-OUT so R1 need to consider as if this path was not advertised to R2 before and </P>
<P>as such to be advertise (this time normally without 0xFFFFFFFF community) to R2 . </P>
<P>R2 insert the new route in the BGP ,and it will be inserted as normal in the routing table . </P>
<P>&nbsp;</P>
<P>The use of community 0xFFFFFF is exactly as if R1 is sending to R2 "distance 255 10/8" . </P>
<P>&nbsp;</P>
<P>The idea of choice "1" or "2" is just to differentiate betweeen 1)a path is withdrawn because it is no longer valid , 2) the path is withdrawn because the router just got a better route from his peer . </P>
<P>Now for the RR : </P>
<P>using the&nbsp;doc avail on ESnet&nbsp; &nbsp;<A href="http://www.es.net/hypertext/welcome/pr/BGP/BGP_Route_Oscillation_Type1a_v1.0.pdf">http://www.es.net/hypertext/welcome/pr/BGP/BGP_Route_Oscillation_Type1a_v1.0.pdf</A></P>
<P></P>
<P>On page 7 of BGP_Route_Oscillation_Type1a_v1.0[1].pdf :</P>
<P>instead of RD(RR2) withdraw route from AS6b :</P>
<P>1)it re-advertise the path from AS6b but with community 0xFFFFFFFF (choice 1 above) </P>
<P>OR </P>
<P>2)it withdraw but with community 0xFFFFFFFF . (choice 2 above) </P>
<P>I guess this could block the oscillation .When using choice 2 (due to the use of timer T) the oscillation will start again when timer T expire but within few exchange it will be stoped . ie: 1 cycle of oscillation each T . </P>
<P>For scalability :</P>
<P>This feature to be deployed only on the RR .</P>
<P>The RR (say RR1)&nbsp; send such community only to the&nbsp; IBGP perr&nbsp;&nbsp;which advertised the best route providing that peer is not one of RR1&nbsp;clients . </P>
<P>If the above is workable we have avoided to change the BGP standard such as </P>
<P>advertising multi path . </P>
<P>&nbsp;</P>
<P>Brgds EL-Koussy </P>
<P>&nbsp;</P></FONT></DIV></div><br clear=all><hr>MSN Photos is the easiest way to share and print your photos: <a href='http://g.msn.com/1HM1ENXX/c156??PI=31901'>Click Here</a><br></html>

Date: Wed, 17 Jul 2002 11:25:19 -0400
From: Jeffrey Haas <[email-masked]>
To: Bruno Rijsman <[email-masked]>
Cc: [email-masked]
Subject: Re: Questions about the "Authentication Information" optional parameter
Sender: [email-masked]

On Wed, Jul 17, 2002 at 11:18:48AM -0400, Bruno Rijsman wrote:
> If no "Authentication Information" parameter was exchanged, it seems that 
> the 16 marker bytes are a complete waste of space.

Note that in bgp 1-3, this field was used for synchronizing the start
of packet in the data stream.  It was in bgp 4 that this was changed
to be capable of being used for authentication.

There was also a change at some point of the size of the marker field.

Just following the rfc's in the evolution of bgp and reading the IDRP
field, the whole "authentication" bit has a very ISO feel to it.
The authors may be able to shed more light on this. :-)

In any case, you can't get rid of it without breaking backwards 
compatability.

> Would it make sense to 
> define a capability (or maybe an authentication code) which means 
> "subsequent messages after the OPEN message will not contain the marker 
> bytes".

Why bother?  Routing data doesn't consume that much bandwidth.

-- 
Jeff Haas 
NextHop Technologies

From: "Bruno Rijsman" <[email-masked]>
To: [email-masked]
Subject: Questions about the "Authentication Information" optional parameter
Date: Wed, 17 Jul 2002 11:18:48 -0400
Sender: [email-masked]

I have some questions about the "Authentication Information" optional 
parameter in the BGP OPEN message.

Have any authentication codes been defined anywhere?

Does any implementation actually support this method of authentication (as 
far as I know everyone uses RFC 2385 which uses a TCP MD5 signature 
instead)?

If it is not used in real life should we remove the "Authentication 
Information" optional parameter from the draft (or at least deprecate it)?

If no "Authentication Information" parameter was exchanged, it seems that 
the 16 marker bytes are a complete waste of space. Would it make sense to 
define a capability (or maybe an authentication code) which means 
"subsequent messages after the OPEN message will not contain the marker 
bytes".

_________________________________________________________________
Join the world’s largest e-mail service with MSN Hotmail. 
http://www.hotmail.com

Date: Wed, 17 Jul 2002 04:34:38 -0400
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: New agenda - addition
Cc: yakov Rekhter <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

IDR Agenda for 7/18/02

Time of meeting: 13:00 - 15:00
Room: 501

    Agenda Bashing  13:00 - 13:10

    Route Oscillation Fixes [13:10-14:00]

       1) Wilfong,BAsu, Shepherd, Ong, RAsala
	
	  presentor: Anindya Basu
	  time: 13:10 - 13:25
	
	 draft: "Eliminating IBGP oscillations"
                  draft-wilfong-ibgp-oscillations-00.txt 	
	
      2) Walton, Retano, Scudder, Cook drafts

	presentor: Alvaro Retano
	time: 13:25 - 13:40
          Advertisement of Multiple Paths in BGP
             draft-walton-bgp-add-paths-00.txt

          BGP Persistent Route Oscillation Solution
              draft-walton-bgp-route-oscillation-stop-00.txt

      3) Chen drafts overview
	
	presentor: Sue Hares
	time: 13:40 - 14:00
	
	a) BGP Route Oscillation Reduction with Confederation
             draft-chen-confed-oscillation-reduce-01.txt

	b) BGP Route Oscillation Avoidance for Route Reflection and Confederation
             draft-chen-route-oscillation-avoid-00.txt

	c) BGP Route Oscillation Reduction with Route Reflection
            draft-chen-rr-oscillation-reduce-01.txt

      4) Discussion time
	
	time:14:00 - 14:15

    Inform proposal [14:00 - 14:15]

      1) "BGPv4 INFORM message"

	draft-nalawade-bgp-inform-00.txt
         presentor: Gargi Nalawade
	time: 14:00 - 14:15

        Extended Community Issues [14:15 - 14:30]
	presentor: Andrew Lange

    Status of BGP documents [14:30 - 14:45]
	
	1) BGP-4 base document status
	2) Status of all BGP drafts

Date: Wed, 17 Jul 2002 03:52:55 -0400
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: Re: BGP Extended Communities
Cc: [email-masked], [email-masked]
Sender: [email-masked]

Andrew:

You are on the agenda.  Could you send me your last name as well?

Sue

At 10:14 PM 7/16/2002 -0700, [email-masked] wrote:
>Greetings,
>
>I was going over the draft-ietf-idr-bgp-ext-communities-05.txt.  I
>think we can make significant enhancements to the functionality,
>efficency and flexibility of this draft.  If we step back and look
>at defining a more generic solution which can address both
>current and future needs we will be better served.
>
>In order to do that, I'd like to propose a couple of changes to the
>current draft.  Here are the enhancements these changes will provide:
>
>- Support for IPv6.  If we are going to move to IPv6 at some point,
>we should include support for it in the protocol extentions we define.
>- More efficient packing of values in the data field.  A good example
>of where this enhancement would be applicable is in the
>draft-ietf-ptomaine-bgp-redistribution-00.txt document.
>- Cleaner support for a broad range of future data field structures,
>and interpretations.
>- Support for locally defined community structures (types).
>- Support for locally defined community sub-types.
>
>Below I have proposed some modifications to the text of the draft to
>support these enhancements.
>
>(Section 5)
>
>Each Extended Community is encoded as a variable length quantity, as
>follows:
>
>   - Type Field: 1 octet
>   - Subtype Field: 1 octet
>   - Length Field: 1 octet
>   - Value Field: 0-63 octets
>
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |     Type      |    Subtype    |    Length     |               |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Value     |
>       |                                               (0-63 octets)   |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>**  1) This eliminates the "regular" and the "extended" variations in favor
>**  of a single definition.  Since all of the current proposals are or can be
>**  represented as extended types, and the addition of a variable length
>**  value field allows as many bytes are needed for data, the "regular" type
>**  can be removed.
>**  2) The Value field is made variable length.  This key enhancement allows
>**  us to efficently encode lists of ASN's, IPv4 and IPv6 addresses.
>
>Type Field:
>
>   The Type field is one octet.  This octect is split into three sections:
>
>              0 1 2 3 4 5 6 7
>             +-+-+-+-+-+-+-+-+
>             |I|T|           |
>             +-+-+-+-+-+-+-+-+
><< all the same as the existing draft until: >>
>
>   Remaining 6 bits:  Indicates the structure of the community.  Specifically
>    the type defined by these bits indicates how the data in the Value field
>    is structured.  For example, a type 1 (000001) might represent a 2 part
>    value field where the first part is expected to be a 2-octet ASN and
>    the second part would be a variable-length value.  A basic set of
>    useful types are defined later in this document.
>
>Subtype Field:
>
>   The Subtype field is one octet.  This contents of this octet are used
>   to clarify the interpretation of the type field.  For example, a
>   subtype 2, which would apply to both the IP-addres types defined later
>   in this document, might indicate that the IP addresses contained in
>   part one of the value field indicate that the BGP route originated
>   at the router with that IP address.
>
>(Section 6)
>
>Base BGP Extended Community Type
>
>   This section defines the base BGP community specification.  Since the
>   root value of communities is the ability to tag a route with arbitrary
>   information, and then create a system to give that information meaning,
>   the base extended community type (type 0), is very simple.  This
>   base type could also be described as the "opaque type."
>
>   Like all extended commnuities, this type can be transitive or
>   non-transitive.
>
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |     Type      |      0x00     |    Length     |               |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Value     |
>       |                                               (0-63 octets)   |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>** The differentiation of the base type from the other types defined in
>** this document is important.  The base type is what the rest are built
>** on.  Similar functionality to all the other types can be recreated
>** with sufficient manual configuration and this base type.
>
>(Section 7)
>
>Extended Community Type Summary:
>
>   Not including the base type, there are four community types defined in
>   this document.  Each represents a different structure a community can
>   take.
>
>   Type 1:
>     2 part Value Field.  The first part is 2 octets.  The second part
>     is variable.  If this type is transitive, the first part
>     MUST contain the 2-octet ASN of the originating network.
>   Type 2:
>     2 part Value Field.  The first part is 4 octets.  The second part
>     is variable.  If the type is transitive, the first part
>     MUST contain the 4-octet ASN of the originating network.
>   Type 3:
>     2 part Value Field.  The first part is 4 octets.  The second part
>     is variable.  The first part contains an IPv4 address.
>   Type 4:
>     2 part Value Field.  The first part is 16 octets. The second part
>     is variable.  The first part contains an IPv6 address.
>
>(Section 8)
>
>Extended Community Subtype Summary:
>
>   Type 0:
>     No Subtype Defined.
>   Type 1:
>     This is the Route Target Subtype.  This subtype applies to the
>     IP address type communities (types 3 & 4). The first part of a community
>     with this subtype represents the loopback of the router where this
>     route is to be announced.  Please see RFC2547 for more details.
>
>** Correct me if my interperetation of the Route Target Community is
>** incorrect.
>
>   Type 2:
>     This is the Route Origin Subtype.  This subtype applies to the
>     IP address type communities (types 3 & 4). The first part of a community
>     with this subtype represents the loopback of the router where this
>     route is announced.
>   Type 3:
>     This is the Link Bandwidth Subtype.  This subtype applies to the
>     ASN type communities (types 1 & 2). The first part od a community
>     with this subtype represents the ASN of the eBGP neighbor whose
>     link bandwidth you wish to distribute.  The second part of a
>     community with this subtype represents the bandwidth of the link
>     in bits-per-second, encoded in IEEE floating point format.
>
>(Section 9)
>
>Locally Defined Commnuity Types
>
>   This section reserves a subgroup of the Vendor-Specific/Experimental
>   type values (I bit = 1), for locally defined types.  This allows
>   operators to define their own community structures.  To prevent type
>   collision, these types MUST be non-transitive.
>
>   Experimental Non-Transitive Type 32-63 (0xE0-0xFF)
>
>   A locally defined type will have a syntax for interpretation defined
>   on the routers that need to interpret it.  If a router receives a
>   community type associated with this range, and does not have a value
>   defined, it should pass it along normally, and ignore the contents.
>
>(Section 10)
>
>Locally Defined Community Subtypes
>
>   This section reserves a subgroup of the subtype space for local
>   definition.  This allows operator to define varriant interpertations
>   for exiting types (locally defined or otherwise).  These subtypes
>   MUST be attached to non-transitive community values to prevent
>   subtype collision.
>
>   Non-Transitive Subtypes 192-255 (0xC0-0xFF)
>
>   A locally defined subtype will have an syntax for interpertation
>   defined on the routers that need to interpret it.  If a router
>   receives a community subtype associated with this range, and does
>   not have a value defined, it should pass the it along normally, and
>   ignore the contents.

Date: Tue, 16 Jul 2002 22:14:14 -0700
From: [email-masked]
To: [email-masked]
Cc: [email-masked]
Subject: BGP Extended Communities
Sender: [email-masked]

Greetings,

I was going over the draft-ietf-idr-bgp-ext-communities-05.txt.  I
think we can make significant enhancements to the functionality,
efficency and flexibility of this draft.  If we step back and look
at defining a more generic solution which can address both
current and future needs we will be better served.

In order to do that, I'd like to propose a couple of changes to the
current draft.  Here are the enhancements these changes will provide:

- Support for IPv6.  If we are going to move to IPv6 at some point,
we should include support for it in the protocol extentions we define.
- More efficient packing of values in the data field.  A good example
of where this enhancement would be applicable is in the
draft-ietf-ptomaine-bgp-redistribution-00.txt document.
- Cleaner support for a broad range of future data field structures,
and interpretations.
- Support for locally defined community structures (types).
- Support for locally defined community sub-types.

Below I have proposed some modifications to the text of the draft to
support these enhancements.

(Section 5)

Each Extended Community is encoded as a variable length quantity, as
follows:

  - Type Field: 1 octet
  - Subtype Field: 1 octet
  - Length Field: 1 octet
  - Value Field: 0-63 octets

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     Type      |    Subtype    |    Length     |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Value     |
      |                                               (0-63 octets)   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

**  1) This eliminates the "regular" and the "extended" variations in favor
**  of a single definition.  Since all of the current proposals are or can be
**  represented as extended types, and the addition of a variable length
**  value field allows as many bytes are needed for data, the "regular" type
**  can be removed.
**  2) The Value field is made variable length.  This key enhancement allows
**  us to efficently encode lists of ASN's, IPv4 and IPv6 addresses.

Type Field:

  The Type field is one octet.  This octect is split into three sections:

             0 1 2 3 4 5 6 7
            +-+-+-+-+-+-+-+-+
            |I|T|           |
            +-+-+-+-+-+-+-+-+
<< all the same as the existing draft until: >>

  Remaining 6 bits:  Indicates the structure of the community.  Specifically
   the type defined by these bits indicates how the data in the Value field
   is structured.  For example, a type 1 (000001) might represent a 2 part
   value field where the first part is expected to be a 2-octet ASN and
   the second part would be a variable-length value.  A basic set of
   useful types are defined later in this document.

Subtype Field:

  The Subtype field is one octet.  This contents of this octet are used
  to clarify the interpretation of the type field.  For example, a
  subtype 2, which would apply to both the IP-addres types defined later
  in this document, might indicate that the IP addresses contained in
  part one of the value field indicate that the BGP route originated
  at the router with that IP address.

(Section 6)

Base BGP Extended Community Type

  This section defines the base BGP community specification.  Since the
  root value of communities is the ability to tag a route with arbitrary
  information, and then create a system to give that information meaning,
  the base extended community type (type 0), is very simple.  This
  base type could also be described as the "opaque type."

  Like all extended commnuities, this type can be transitive or
  non-transitive.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     Type      |      0x00     |    Length     |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Value     |
      |                                               (0-63 octets)   |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

** The differentiation of the base type from the other types defined in
** this document is important.  The base type is what the rest are built
** on.  Similar functionality to all the other types can be recreated
** with sufficient manual configuration and this base type.

(Section 7)

Extended Community Type Summary:

  Not including the base type, there are four community types defined in
  this document.  Each represents a different structure a community can
  take.

  Type 1:
    2 part Value Field.  The first part is 2 octets.  The second part
    is variable.  If this type is transitive, the first part
    MUST contain the 2-octet ASN of the originating network.
  Type 2:
    2 part Value Field.  The first part is 4 octets.  The second part
    is variable.  If the type is transitive, the first part
    MUST contain the 4-octet ASN of the originating network.
  Type 3:
    2 part Value Field.  The first part is 4 octets.  The second part
    is variable.  The first part contains an IPv4 address.
  Type 4:
    2 part Value Field.  The first part is 16 octets. The second part
    is variable.  The first part contains an IPv6 address.

(Section 8)

Extended Community Subtype Summary:

  Type 0:
    No Subtype Defined.
  Type 1:
    This is the Route Target Subtype.  This subtype applies to the
    IP address type communities (types 3 & 4). The first part of a community
    with this subtype represents the loopback of the router where this
    route is to be announced.  Please see RFC2547 for more details.

** Correct me if my interperetation of the Route Target Community is
** incorrect.

  Type 2:
    This is the Route Origin Subtype.  This subtype applies to the
    IP address type communities (types 3 & 4). The first part of a community
    with this subtype represents the loopback of the router where this
    route is announced.
  Type 3:
    This is the Link Bandwidth Subtype.  This subtype applies to the
    ASN type communities (types 1 & 2). The first part od a community
    with this subtype represents the ASN of the eBGP neighbor whose
    link bandwidth you wish to distribute.  The second part of a
    community with this subtype represents the bandwidth of the link
    in bits-per-second, encoded in IEEE floating point format.

(Section 9)

Locally Defined Commnuity Types

  This section reserves a subgroup of the Vendor-Specific/Experimental
  type values (I bit = 1), for locally defined types.  This allows
  operators to define their own community structures.  To prevent type
  collision, these types MUST be non-transitive.

  Experimental Non-Transitive Type 32-63 (0xE0-0xFF)

  A locally defined type will have a syntax for interpretation defined
  on the routers that need to interpret it.  If a router receives a
  community type associated with this range, and does not have a value
  defined, it should pass it along normally, and ignore the contents.

(Section 10)

Locally Defined Community Subtypes

  This section reserves a subgroup of the subtype space for local
  definition.  This allows operator to define varriant interpertations
  for exiting types (locally defined or otherwise).  These subtypes
  MUST be attached to non-transitive community values to prevent
  subtype collision.

  Non-Transitive Subtypes 192-255 (0xC0-0xFF)

  A locally defined subtype will have an syntax for interpertation
  defined on the routers that need to interpret it.  If a router
  receives a community subtype associated with this range, and does
  not have a value defined, it should pass the it along normally, and
  ignore the contents.

Date: Tue, 16 Jul 2002 11:39:33 -0400
From: Jeffrey Haas <[email-masked]>
To: "Fryczke, Pawel" <[email-masked]>
Cc: [email-masked]
Subject: Re: Aggregation of the overlapping routes and ATOMIC_AGGREGATE
Sender: [email-masked]

On Thu, Jul 11, 2002 at 12:32:14PM +0200, Fryczke, Pawel wrote:
> What does happen if BGP router installs both routes in RIB and next
> aggregates them ?
> Should ATOMIC_AGGREGATE be added to aggregate or not ?
> 
> Consider scenario:
> BGP Router is configured to aggregate routes 192.0.0.0/8
> This router receives 2 NLRIs: 192.0.0.0/8 and 192.1.0.0/16
> Received NLRIs are installed in RIB, but are suppressed by aggregation and
> are not advertised.

Assuming you mean "suppresed by export policy because they
are contributors to an aggregate"...

> As a result of aggregation the router sends update that contains only one
> NLRI: 192.0.0.0/8
> Does this update should contain ATOMIC_AGGREGATE attached ?

According to the spec?  No.
Logically, yes.
An attempt was made to address this in -16, but it was determined
that the rules of whether or not AA was to be attached to make
proper semantic (rather than spec-defined) sense imposed the necessity
of keeping too much state.

My recommendation - treat AA as a piece of fluff.  Add it like
everyone else does when suppressing as's from an aggregated as-path.
Also add it when you suppress more specifics in your adj-ribs-in if
you want to be compliant, but its work that is unappreciated by
the Internet at large.

> Why does RFC assume that aggregation is done during decision process?

Logically, because routes are installed in LocRib/Adj-Ribs-Out during
this phase and Update-Send just sends Adj-Ribs-Out.

> Pawel

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 16 Jul 2002 11:34:09 -0400
From: Jeffrey Haas <[email-masked]>
To: Bruno Rijsman <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Aggregation of the overlapping routes and ATOMIC_AGGREGATE
Sender: [email-masked]

On Thu, Jul 11, 2002 at 06:47:26AM -0400, Bruno Rijsman wrote:
> 1) If the aggregate is an "as-set" aggregate (i.e. the as-path and
>    other path attributes of the aggregate are constructed by
>    "summarizing" the attributes of the covered routes) then the
>    ATOMIC_AGGREGATE attribute is not attached.

Is this actually the case if one of the contributing routes already
ATOMIC_AGGREGATE?

> 2) If the aggregate is not an "as-set" aggregate (i.e. the as-path
>    and other path attributes are constructed as if the aggregate
>    route is a locally originated route) then the ATOMIC_AGGREGATE
>    attribute is attached.
> 
> I have seen nothing in RFC1771 or the current draft that describes
> this behavior but -as far as I know- that is what happens in real
> life.

>From draft -17:
: 5.1.6 ATOMIC_AGGREGATE
: 
: 
:    ATOMIC_AGGREGATE is a well-known discretionary attribute.
: 
:    When a router aggregates several routes for the purpose of
:    advertisement to a particular peer, and the AS_PATH of the aggregated
:    route excludes at least some of the AS numbers present in the AS_PATH
:    of the routes that are aggregated, the aggregated route, when
:    advertised to the peer, MUST include the ATOMIC_AGGREGATE attribute.

What was only really mentioned in the more recent drafts was the
option of not generating as-sets in the first place.  Prior to
this, implementations that truncated paths were non-compliant.

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 16 Jul 2002 11:24:11 -0400
From: Jeffrey Haas <[email-masked]>
To: "John G. Scudder" <[email-masked]>
Cc: Pedro Roque Marques <[email-masked]>, [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

On Mon, Jul 15, 2002 at 05:39:45PM -0400, John G. Scudder wrote:
> - Router B ignores the replacement because of the posited "discard 
> updates that contain unrecognized data" rule.

You've just given the strongest example of why not NOTIFYing can be
a bad idea.

As you mention later:

> Let me grant you up front that you can fix this specific case by 
> being more careful in stating your rule 

This leaves me asking again, "Is there some way we can gracefully
deal with some errors?"  In the case of dealing with the non-prefixed
confederation segments, the answer was yes.  The example you give
could also have worked had it been worded more carefully to only
apply to path attributes and unrecognized/semantically incorrect
attributes yet still use the NLRI for purposes of discarding those
prefixes from your adj-ribs-in.

My general concern is that we already have vendors implementing
various forms of graceful error handling that aren't strictly
conformant with the spec.  If this is the norm, we should codify it.
If we don't, we risk interoperability issues due to one vendor's
bugs killing other vendor's peering sessions "far away".

-- 
Jeff Haas 
NextHop Technologies

From: "Shah, VJ" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: FW:  dynamic capabilities and ORF
Date: Tue, 16 Jul 2002 10:29:16 -0400
Sender: [email-masked]

I had earlier posted some queries regarding use of dynamic capability
negotiation for ORF capability.  I hope these will be addressed in the new
version of the draft.

thanks

VJ

With reference to drafts "draft-ietf-idr-dynamic-cap-01.txt and
"draft-ietf-idr-route-filter-05.txt", I have the following question:

Is dynamic capability supported for Cooperative Route Filters(ORF)?

If yes, It is not clear how to advertise or withdraw ORF capability for a
particular ORF Type in an address family using Dynamic capability. 

Consider the following example:

1) When BGP session is established, we advertise 

ORF capability 
	address family IPV4-Unicast 
		ORF Type: Prefix-List, Send/Receive: Both
            ORF Type: Extended-Community, Send/Receive: Both

2) User turns on ORF capability for Communities on the local BGP system

In this case, using dynamic capability, do we need to advertise all three
Types 

OR 

we advertise only the new type i.e Community Type.

3) Now, the user turns off ORF capability on the local BGP system for ORF
Type: Extended-Community

In this case, do we send a dynamic capability message withdrawing all the
ORF capabilities and then send another capability message advertising the
new set of ORF Types being supported.

OR

Send a Dynamic capability message removing only the Extended Community ORF
Type.  In this case, does the Send/Receive field in Capability data indicate
the mode being withdrawn.

Thanks,

VJ Shah

>>From: Srihari R. Sangli [mailto:[email-masked]]
>>Sent: Thursday, July 11, 2002 4:36 PM
>>To: Abarbanel, Benjamin
>>Cc: 'Tsillas, Jim'; [email-masked]'; [email-masked]'
>>Subject: RE: Re>Are dynamic capabilities really necessary if peers
>>support gracef ul restart?
>>
>>
>>
>>Ben,
>>
>>> Srihari:
>>>   I looked over the draft again and I have to apologize I 
>>misunderstood you
>>> responses below. I would recommend rewording the two 
>>paragraph so that
>>> others
>>> do not get confused like me.
>>
>>Ok. Will be added in the next revision. Thanks for the feedback.
>>
>>srihari...
>>

From: "Natale, Jonathan" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>, Jeffrey Haas <[email-masked]>
Cc: [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Tue, 16 Jul 2002 10:28:34 -0400
Sender: [email-masked]

Actually, that is not what I proposed; I was mearly reporting the
behavior I observed which was to ignore the attribute, but not the
route--it got installed.  Also, this was an UNEXPECTED/mis-configured
CONFED AS_PATH (sorry for not making this clear).  Whether this is a
bug, or "correct" could certainly be debated.  But I thought it was
related to:

    "As you infer, there are situations where we should just assume
    that the buffer contains cruft and that the implementation needs
    a strong reset.

    However, there are situations where we could get stuff that we
    can deal with and just ignore the "Bad" things.  This was the
    case when someone's implementation started leaking non-prefixed
    confederation segments into the internet.  The packets were well
    formed, but had cruft in them."

    -- Jeff, Fri 7/12/2002 2:29 PM

Seems to me that there are certainly cases where it is better to
ignore something then flap the whole session.

Another case where this happens is RXing a route to yourself (if
a neighbor runs PPP and redistributes connected)--route gets dropped.
Juniper sends the routes back so the neighbor would have a better
chance of knowing what got dropped, Cisco does not do this, the
RFC/Draft is unclear on this.  Both have
advantages/diaadvantages--my vote is to make it user configurable
(split-horrizon), this is already an option for other protocols.

What to do is not [publicly] defined, and should be.  Or maybe user
configurable would be good (I think so), but that could be
[publicly] defined as well.  There are many ways things could go
wrong.  The two basic categories are syntactical and semantically.
It seems that generally the RFCs/Drafts indicate that for
syntactical errors you should send a NOTIFICATION and close the
session (for how long and how many times?), and for semantical
errors you should ignore [the route or the attribute].  In cases
where things get ignored due to the fact that they are not
appropriate for the particular scenario, then an INFORM would not
hurt and may help troubleshoot the misconfig/bug.  The first step
would be to categorize all (lol) the scenarios, and then
have a vote/decree and come to a final complete protocol (lol).

-----Original Message-----
From: Pedro Roque Marques [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 3:15 PM
To: Jeffrey Haas
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)

Jeffrey Haas writes:

> On Fri, Jul 12, 2002 at 02:33:05PM -0400, Natale, Jonathan wrote:
>> This is already done--if CONFED path is received the peer stays up
>> (and the local pref gets ignored).  This is good, but not in an rfc
>> nor a draft.  My vote is for an INFORM.

> IMO, this is a case where the route's local pref shouldn't be just
> ignored, but the entire packet should be discarded.

>From both quotes above it apears to me that you are proposing to
discard an UPDATE and leave the session up. Since BGP state is
incremental, that would mean the two peers would now potentially be
out of sync.

  Pedro.

content-class: urn:content-classes:message
Subject: ORF implementation  issue #2
Date: Tue, 16 Jul 2002 09:53:09 +0100
From: "Stephen Waters" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

Here's another issue that came up when implementing ORF:

The filters can be Normal, or Exact matching.  For the Normal case the draft says:

"NORMAL scope
   indicates that the remote peer should consider only those routes
   whose Communities attribute either is equal to the Communities list
   specified in the ORF, or exhibit a subset relation with the
   Communities list specified in the ORF."

This gives the manager a support issue with the defined ORFs. Since community IDs will come and go, and the local manager has no control over what community IDs a route will collect, it is not feasible to set up a filter (as defined in the draft) that will be a stable superset of the IDs carried by a route.

It is more likely that a core set of IDs carried by a route will remain constant, so allowing the Normal filter community-list to be a subset of the route's community-list is more useful/stable.

Alternatively, a third filter class is added:  Exact/Subset/Superset. All of these match 'exact',
subset => filter can be a subset of the route's IDs, superset is the old 'Normal'.

At the moment, I've take the approach to allow Normal filters to be a subset of the route, but unlike issue #1, this does need to be agreed. So, I should at least have a control to allow a peer to work in subset or superset mode when using ORFs to filter.

Steve.

content-class: urn:content-classes:message
Subject: FW: ORF implementation issue
Date: Tue, 16 Jul 2002 09:35:30 +0100
From: "Stephen Waters" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

Thanks for the responses to the mail below.

I just thought I summarize the views expressed in the mail I received:

1) don't use ORF if it doesn't suit your filtering needs.

This translates to:  If want to accept most routes, and filter the odd annoying one, you probably don't want ORF since this could involve entering a large number of permit filters.  

Regardless of your policy model, you also have to live with the fact that if routes you like start picking up new community IDs along their merry way, you will lose routes until you fix the filters.

2) we need a 'permit-all' alternative to the 'deny-all' default.

My view on this is that the default could be changed to 'permit-all'. This is not a 'firewall'; it's a feature to help save processing bandwidth where back-stop local policy is there to make the final call.

Steve.

> From: Stephen Waters 
> Sent: Monday, July 15, 2002 3:36 PM
> To: [email-masked]
> Subject: ORF implementation issue
> 
> 
> 
> Having implemented ORF, I have a few issues with the draft. 
> Here is the first one...
> 
> 
> "If a particular route maintained by a BGP speaker doesn't match any
>    of the ORF entries of any of the (non-empty) ORFs associated with a
>    particular peer, then this route should not be advertised to the
>    peer."
> 
> I think this makes things very difficult for the manager.  
> The idea with ORF (I think) is to 
> reduce the local processing of routes by warning your peer 
> about routes you are definitely not interested in.
> 
> Since there is no support for wild cards in spec., this text 
> would require the manager to set up ORF entries (to be sent 
> to the peer) for every conceivable community-mix that may be 
> of interest. If he misses some, or a new community appears on 
> the block, useful routes may be lost.
> 
> For example, I am will to accept routes with any one of 300 
> different communities, and am not interested in 20 other 
> community groups.  I would need to send 20 DENY filters, and 
> 300 PERMIT filters.
> 
> Now the problem:
> 
> A route can carry a number of communities, and new community 
> numbers will come and go.  This would cause we to have to 
> update my filters constantly.
> 
> I have taken the view that blocking useful routes would be 
> very bad, and if the ORF filtering is in any doubt, the route 
> should be sent and the local filtering on the remote peer can 
> make the final decision.
> 
> So, in my implementation, if there is no matching ORF filter, 
> the route is passed. Taking this approach also means that I 
> can no reduce my ORF to a list of explicit DENY filters (with 
> the occasional imbedded PERMIT).  This approach also means 
> that if community lists change, the result is much less 
> likely to result in lost useful routes, and unwanted routes 
> carrying strange new community lists can be filtered as and 
> when the manager has time.
> 
> Comments?
> 
> Regards, Steve.
> 

Date: Mon, 15 Jul 2002 15:14:43 -0700
From: john heasley <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: john heasley <[email-masked]>, "Srihari R. Sangli" <[email-masked]>, Stephen Waters <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: Re: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

Fri, Jul 12, 2002 at 01:28:55PM -0400, Jeffrey Haas:
> On Thu, Jul 11, 2002 at 11:44:09AM -0700, john heasley wrote:
> > 	"it must be possible to turn such options off and force
> > 	 values/capabilities that will be required."
> 
> Considered from a slightly different angle, do you perhaps also mean:
> 1. If I said I supported some AFI/SAFI
> 2. I decide that your implementation of said AFI/SAFI is buggy
> 3. I send you a dynamic capability update withdrawing that AFI/SAFI
> 4. If you keep sending it after I sent you the capability update, what
>    should I do?
> 
> To 4, answers are:
> 1. Wait a certain amount of time for some queues to clear.  If we
>    still receive the unwanted messages, we should then NOTIFY.
> 2. Grin and bear it.

s/bear/ignore/ ?

> Other observations?

there really are two cases, right?  1) its safe to ignore continued msgs
for a capability (eg: SAFI multicast prefixes) and 2) it isnt.  though
some capability may exist that i am unaware of, i dont believe there are
any cases where 2 applies at the moment, though some such capability may
be added in the future.

so, in most cases it can just be ignored with the little cost of extra
messages/process time.  but, for example the cisco knob

router bgp N
 neighbor x.x.x.x version 4

will cause it to not establish peering with a version 3 speaker or to
"force version 4"; the ability to force that option was/is desirable.

i understand that such a knob would be an implementation option, but the
framework has to exist to support it.  some way for the peer to say NAK
(and ACK, which could be used to for your suggested timing answer #1) to
a dynamic change where the initiator can either accept that or close the
session.  correct me if i am wrong, but that is not part of the current
draft.

so, in context; if i determine that a particular peer's router is sending
botched filters (due to a bug),

router bgp N
 no neighbor x.x.x.x rx-ORF

might disallow ORF updates from the peer; but there is no reason to reset
the session if the peer refuses or continues to send filter updates.  or
maybe i think the customer is using route-refresh in an abusive manner....

make sense?

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'John G. Scudder'" <[email-masked]>, Jeffrey Haas <[email-masked]>
Cc: Pedro Roque Marques <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Mon, 15 Jul 2002 17:58:14 -0400
Sender: [email-masked]

John:
 I dont think people who suggest to use INFORM message will argue with 
you on the merits of not keeping the route. I do think the tendency in
BGP is, if you see something that looks strange drop the session. This
religious approach leads to much more serious consequences, which 
sometimes could be remedied by not dropping the session but by providing 
a mechanism for resynchronizing the corrrect information.

$.02 Worth,
Ben

> From: John G. Scudder [mailto:[email-masked]]
> Sent: Monday, July 15, 2002 5:40 PM
> To: Jeffrey Haas
> Cc: Pedro Roque Marques; [email-masked]
> Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> At 4:20 PM -0400 7/12/02, Jeffrey Haas wrote:
> >On Fri, Jul 12, 2002 at 12:14:38PM -0700, Pedro Roque Marques wrote:
> >>  >From both quotes above it apears to me that you are proposing to
> >>  discard an UPDATE and leave the session up. Since BGP state is
> >>  incremental, that would mean the two peers would now 
> potentially be
> >>  out of sync.
> >
> >And no less intentionally than if I discarded everything via policy.
> >In this case, my policy is "discard bad (for some value) packets".
> 
> No, it's worse than that.  Here's an example of harm using your 
> "let's introduce a new origin code of 4" case.
> 
> - Router A sends router B an announcement of prefix foo with origin 1.
> - Router A now sends router B a replacement for prefix foo, but with 
> newfangled origin 4.  The replacement has a different next hop from 
> the previous announcement.
> - Router B ignores the replacement because of the posited "discard 
> updates that contain unrecognized data" rule.
> 
> Result?  Router B retains router A's old route, whose next hop is no 
> longer valid.  This goes on forever, or until router A sends an 
> update with a recognized origin, or until router A withdraws the 
> route.  The bad next hop may result in nothing serious, but it may 
> equally well result in a black hole or a persistent forwarding loop. 
> This seems to a cure which is much worse than the disease.
> 
> Of course, refusing routes by policy doesn't have this problem at all.
> 
> Let me grant you up front that you can fix this specific case by 
> being more careful in stating your rule -- instead of "discard 
> updates that contain unrecognized data" make it "treat updates that 
> contain unrecognized data as if they were withdraws".  But whenever 
> (and if ever) you want to gloss over a protocol error like this IMO 
> you need to analyze the consequences quite carefully.
> 
> --John
> 

Date: Mon, 15 Jul 2002 17:39:45 -0400
To: Jeffrey Haas <[email-masked]>
From: "John G. Scudder" <[email-masked]>
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Cc: Pedro Roque Marques <[email-masked]>, [email-masked]
Sender: [email-masked]

At 4:20 PM -0400 7/12/02, Jeffrey Haas wrote:
>On Fri, Jul 12, 2002 at 12:14:38PM -0700, Pedro Roque Marques wrote:
>>  >From both quotes above it apears to me that you are proposing to
>>  discard an UPDATE and leave the session up. Since BGP state is
>>  incremental, that would mean the two peers would now potentially be
>>  out of sync.
>
>And no less intentionally than if I discarded everything via policy.
>In this case, my policy is "discard bad (for some value) packets".

No, it's worse than that.  Here's an example of harm using your 
"let's introduce a new origin code of 4" case.

- Router A sends router B an announcement of prefix foo with origin 1.
- Router A now sends router B a replacement for prefix foo, but with 
newfangled origin 4.  The replacement has a different next hop from 
the previous announcement.
- Router B ignores the replacement because of the posited "discard 
updates that contain unrecognized data" rule.

Result?  Router B retains router A's old route, whose next hop is no 
longer valid.  This goes on forever, or until router A sends an 
update with a recognized origin, or until router A withdraws the 
route.  The bad next hop may result in nothing serious, but it may 
equally well result in a black hole or a persistent forwarding loop. 
This seems to a cure which is much worse than the disease.

Of course, refusing routes by policy doesn't have this problem at all.

Let me grant you up front that you can fix this specific case by 
being more careful in stating your rule -- instead of "discard 
updates that contain unrecognized data" make it "treat updates that 
contain unrecognized data as if they were withdraws".  But whenever 
(and if ever) you want to gloss over a protocol error like this IMO 
you need to analyze the consequences quite carefully.

--John

From: "Natale, Jonathan" <[email-masked]>
To: "'Stephen Waters'" <[email-masked]>, [email-masked]
Subject: RE: ORF implementation issue
Date: Mon, 15 Jul 2002 13:03:48 -0400
Sender: [email-masked]

I see this as equivalent to the "implied deny all" rule.  I am assuming
there is a way to replace/preempt this with a "permit all" in ORF.  If not,
then I would agree that this makes things very difficult for the manager.

-----Original Message-----
From: Stephen Waters [mailto:[email-masked]] 
Sent: Monday, July 15, 2002 10:36 AM
To: [email-masked]
Subject: ORF implementation issue

Having implemented ORF, I have a few issues with the draft. Here is the
first one...

"If a particular route maintained by a BGP speaker doesn't match any
   of the ORF entries of any of the (non-empty) ORFs associated with a
   particular peer, then this route should not be advertised to the
   peer."

I think this makes things very difficult for the manager.  The idea with ORF
(I think) is to 
reduce the local processing of routes by warning your peer about routes you
are definitely not interested in.

Since there is no support for wild cards in spec., this text would require
the manager to set up ORF entries (to be sent to the peer) for every
conceivable community-mix that may be of interest. If he misses some, or a
new community appears on the block, useful routes may be lost.

For example, I am will to accept routes with any one of 300 different
communities, and am not interested in 20 other community groups.  I would
need to send 20 DENY filters, and 300 PERMIT filters.

Now the problem:

A route can carry a number of communities, and new community numbers will
come and go.  This would cause we to have to update my filters constantly.

I have taken the view that blocking useful routes would be very bad, and if
the ORF filtering is in any doubt, the route should be sent and the local
filtering on the remote peer can make the final decision.

So, in my implementation, if there is no matching ORF filter, the route is
passed. Taking this approach also means that I can no reduce my ORF to a
list of explicit DENY filters (with the occasional imbedded PERMIT).  This
approach also means that if community lists change, the result is much less
likely to result in lost useful routes, and unwanted routes carrying strange
new community lists can be filtered as and when the manager has time.

Comments?

Regards, Steve.

From: "Bradley Dunn" <[email-masked]>
To: "Parag Deshpande" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: BGP-Figures
Date: Mon, 15 Jul 2002 11:03:30 -0400
Sender: [email-masked]

> I would appreciate if someone could provide the current figures for the
> following:
> 1. No. of Networks in the Internet =
> 2. Avg No. of BGP peers in an AS =
> 3. No of Autonomous Systems in the Internet =
> 4. Typical (No. of) BGP Routing table entries =

3 and 4 can be answered by browsing http://bgp.potaroo.net/.

I'm not sure what you mean by 1 and how it would differ from 3 or 4.

2 is nearly impossible to measure, but I'd guess the average is relatively
low (<10), weighed down by lots of multi-homed edge networks with only a few
routers and upstreams. Service provider networks might push into the low
100s on border routers, but even for large networks the number will be
moderated by deployment of route reflectors and/or confederations.

Bradley

Date: Mon, 15 Jul 2002 10:53:05 -0400
From: Jeffrey Haas <[email-masked]>
To: Stephen Waters <[email-masked]>
Cc: [email-masked]
Subject: Re: ORF implementation issue
Sender: [email-masked]

On Mon, Jul 15, 2002 at 03:35:45PM +0100, Stephen Waters wrote:
> I think this makes things very difficult for the manager.  The
> idea with ORF (I think) is to reduce the local processing of routes
> by warning your peer about routes you are definitely not interested in.

I may be reading too much into things, however this is my opinion:

I think that ORFs were only ever intended to be a very coarse grained
method of sending around policy.  The lack of a useful policy algebra
means that all you can ever send is one coarse level of a given
policy element (communities, prefix lists, etc.) and see if it goes
beyond there.  This makes in the responsibility of the entity
constructing the ORFs to make them perhaps more permissive that one would
want them to be.

Now given that they are only good for coarse grained policy, what are
they good for?  You cite the lack of wild cards as an example of
why they are coarse.

Making rfc 2547 scale is probably the most obvious.  The route target
communities provide exactly the coarse filter element that is needed.
One could probably construct other examples.

> Since there is no support for wild cards in spec., this text
> would require the manager to set up ORF entries (to be sent to the
> peer) for every conceivable community-mix that may be of interest.
> If he misses some, or a new community appears on the block, useful
> routes may be lost.

Given the coarseness of the tool, it is perhaps better to allow the
user to manually construct the list rather than to derive the list
from policy.

Note that given a particular regular expression, you *could* construct
the list of all possibile matches.  Its just combinatorically messy.

> So, in my implementation, if there is no matching ORF filter,
> the route is passed. Taking this approach also means that I can no
> reduce my ORF to a list of explicit DENY filters (with the occasional
> imbedded PERMIT).  This approach also means that if community lists
> change, the result is much less likely to result in lost useful
> routes, and unwanted routes carrying strange new community lists
> can be filtered as and when the manager has time.

It would be useful to have default policies for each ORF type
(or not) on a configurable basis along with a default policy when
running ORFs.  Does this adequately restate the above?

-- 
Jeff Haas 
NextHop Technologies

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Susan Hares'" <[email-masked]>, Kunihiro Ishiguro <[email-masked]>
Cc: [email-masked], BGP mailing list <[email-masked]>, Gilles Geerts <[email-masked]>
Subject: RE: TCP state machine doubts
Date: Mon, 15 Jul 2002 10:51:19 -0400
Sender: [email-masked]

Hi Sue:

Regards,
Ben

> Could you suggest some text based on the backoff draft?

Date: Mon, 15 Jul 2002 09:55:56 -0500
From: "Parag Deshpande" <[email-masked]>
To: [email-masked]
Subject: BGP-Figures
Sender: [email-masked]

This is a multi-part message in MIME format.

------=_NextPart_000_0120_01C22BE5.D0718180
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,

I would appreciate if someone could provide the current figures for the =
following:
1. No. of Networks in the Internet =3D=20
2. Avg No. of BGP peers in an AS =3D=20
3. No of Autonomous Systems in the Internet =3D=20
4. Typical (No. of) BGP Routing table entries =3D=20

Thanks
Parag

------=_NextPart_000_0120_01C22BE5.D0718180
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2614.3500" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT size=3D2>Hi,</FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV><FONT size=3D2>I would appreciate if someone could&nbsp;provide the =
current=20
figures for the following:</FONT></DIV>
<DIV><FONT size=3D2>1. No. of Networks in the Internet =3D </FONT></DIV>
<DIV><FONT size=3D2>2. Avg No. of BGP peers in an AS =3D </FONT></DIV>
<DIV><FONT size=3D2>3. No of Autonomous Systems in the Internet =3D =
</FONT></DIV>
<DIV><FONT size=3D2>4. Typical (No. of) BGP Routing table entries =3D =
</FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV><FONT size=3D2>Thanks</FONT></DIV>
<DIV><FONT size=3D2>Parag</FONT></DIV></BODY></HTML>

------=_NextPart_000_0120_01C22BE5.D0718180--

content-class: urn:content-classes:message
Subject: ORF implementation issue
Date: Mon, 15 Jul 2002 15:35:45 +0100
From: "Stephen Waters" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

Having implemented ORF, I have a few issues with the draft. Here is the first one...

"If a particular route maintained by a BGP speaker doesn't match any
   of the ORF entries of any of the (non-empty) ORFs associated with a
   particular peer, then this route should not be advertised to the
   peer."

I think this makes things very difficult for the manager.  The idea with ORF (I think) is to 
reduce the local processing of routes by warning your peer about routes you are definitely not interested in.

Since there is no support for wild cards in spec., this text would require the manager to set up ORF entries (to be sent to the peer) for every conceivable community-mix that may be of interest. If he misses some, or a new community appears on the block, useful routes may be lost.

For example, I am will to accept routes with any one of 300 different communities, and am not interested in 20 other community groups.  I would need to send 20 DENY filters, and 300 PERMIT filters.

Now the problem:

A route can carry a number of communities, and new community numbers will come and go.  This would cause we to have to update my filters constantly.

I have taken the view that blocking useful routes would be very bad, and if the ORF filtering is in any doubt, the route should be sent and the local filtering on the remote peer can make the final decision.

So, in my implementation, if there is no matching ORF filter, the route is passed. Taking this approach also means that I can no reduce my ORF to a list of explicit DENY filters (with the occasional imbedded PERMIT).  This approach also means that if community lists change, the result is much less likely to result in lost useful routes, and unwanted routes carrying strange new community lists can be filtered as and when the manager has time.

Comments?

Regards, Steve.

From: "Natale, Jonathan" <[email-masked]>
To: "'Stephen Waters'" <[email-masked]>, [email-masked]
Subject: RE: ORF implementations?
Date: Mon, 15 Jul 2002 08:42:38 -0400
Sender: [email-masked]

Yes:
Cisco-12000-RtrB(config-router)#nei 1.1.1.1 capability ?
  orf  Advertise ORF capability to the peer

Cisco-12000-RtrB#sho ver
Cisco Internetwork Operating System Software
IOS (tm) GS Software (GSR-P-M), Version 12.0(18.6)ST1,

...this is ED s/w but it should be in 12.0(19)ST1 and/or other IOS.

Also:
  
http://www.cisco.com/warp/public/cc/pd/iosw/iore/iomjv122/prodlit/1363_pp.ht
m

http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122newft/122
t/122t4/ftbgporf.htm

...

http://www.cisco.com/ (search for "orf"--lot's of hits)

-----Original Message-----
From: Stephen Waters [mailto:[email-masked]] 
Sent: Monday, July 15, 2002 5:39 AM
To: [email-masked]
Subject: ORF implementations?

Hi,

Are there any ORF implementations out there? I have a few 'problems'
following the 
draft that I'd like to discuss with any fellow ORFers.

Cheers, Steve.

From: "Natale, Jonathan" <[email-masked]>
To: "'Susan Hares'" <[email-masked]>, "Natale, Jonathan" <[email-masked]>
Cc: "'Fryczke, Pawel'" <[email-masked]>, [email-masked]
Subject: RE: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Date: Mon, 15 Jul 2002 08:18:29 -0400
Sender: [email-masked]

Ok

-----Original Message-----
From: Susan Hares [mailto:[email-masked]] 
Sent: Sunday, July 14, 2002 4:27 AM
To: Natale, Jonathan
Cc: 'Fryczke, Pawel'; [email-masked]
Subject: RE: Aggregation and MULTI_EXIT_DISC, NEXT_HOP

Natale:

Draft-18 of the BGP specification is awaiting the FSM work.
Watch for the FSM wording within a week.

Please send the laundry list in two forms:

    1) Where the current draft-17 + FSM words does not
         match currently implemented code.

    2) what changes you like to see

Sue Hares

At 10:48 AM 7/9/2002 -0400, Natale, Jonathan wrote:
>I raised this issue more than two years ago.  What is the story on bgp?
Why
>is the draft/rfc so far off base?  I would be happy to provide a laundry
>list of things, if that will help.
>
>I think the MED was fixed in a major vendor's code with the ~=
deterministic
>med cmd (or so I was told then, but now the story on that cmd seem
>different).
>
>The next hop was never "fixed".  The only answer for why not to aggregate
if
>nxhp not the same was that it has something to do w/ mcast RPF... but I
>never fully digested this.
>
>-$0.02
>
>From: Fryczke, Pawel [mailto:[email-masked]]
>Sent: Thursday, July 04, 2002 8:08 AM
>To: [email-masked]
>Subject: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
>
>The latest draft, draft-ietf-idr-bgp4-17.txt states:
>Routes that have the following attributes shall not be aggregated
>unless the corresponding attributes of each route are identical:
>MULTI_EXIT_DISC, NEXT_HOP.
>
>Many vendors (e.g. Cisco/Juniper) does not follow this requirement. Why?
>What is the purpose of this requirement?
>Pawel

content-class: urn:content-classes:message
Subject: ORF implementations?
Date: Mon, 15 Jul 2002 10:38:55 +0100
From: "Stephen Waters" <[email-masked]>
To: <[email-masked]>
Sender: [email-masked]

Hi,

Are there any ORF implementations out there? I have a few 'problems' following the 
draft that I'd like to discuss with any fellow ORFers.

Cheers, Steve.

Date: Mon, 15 Jul 2002 01:01:04 -0400
To: [email-masked], [email-masked]
From: Susan Hares <[email-masked]>
Subject: Agenda for IDR meeting on 
Cc: [email-masked], [email-masked]
Sender: [email-masked]

Below is the agenda for the Thursday IDR meeting.
Please send any requests for additions to me.

================
IDR Agenda for 7/18/02

Time of meeting: 13:00 - 15:00
Room: 501

    Agenda Bashing  13:00 - 13:10

    Route Oscillation Fixes [13:10-14:00]

       1) Wilfong,BAsu, Shepherd, Ong, RAsala
	
	  presentor: Anindya Basu
	  time: 13:10 - 13:25
	
	 draft: "Eliminating IBGP oscillations"
                  draft-wilfong-ibgp-oscillations-00.txt 	
	
      2) Walton, Retano, Scudder, Cook drafts

	presentor: Alvaro Retano
	time: 13:25 - 13:40
          Advertisement of Multiple Paths in BGP
             draft-walton-bgp-add-paths-00.txt

          BGP Persistent Route Oscillation Solution
              draft-walton-bgp-route-oscillation-stop-00.txt

      3) Chen drafts overview
	
	presentor: Sue Hares
	time: 13:40 - 13:50
	
	a) BGP Route Oscillation Reduction with Confederation
             draft-chen-confed-oscillation-reduce-01.txt

	b) BGP Route Oscillation Avoidance for Route Reflection and Confederation
             draft-chen-route-oscillation-avoid-00.txt

	c) BGP Route Oscillation Reduction with Route Reflection
            draft-chen-rr-oscillation-reduce-01.txt

      4) Discussion time
	
	time: 13:50 - 14:00

    Inform proposal [14:00 - 14:15]

      1) "BGPv4 INFORM message"

	draft-nalawade-bgp-inform-00.txt
         presentor: Gargi Nalawade
	time: 14:00 - 14:15

    Status of BGP documents [14:15 - 14:30]
	
	1) BGP-4 base document status
	2) Status of all BGP drafts

Date: Sun, 14 Jul 2002 04:27:25 -0400
To: "Natale, Jonathan" <[email-masked]>
From: Susan Hares <[email-masked]>
Subject: RE: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Cc: "'Fryczke, Pawel'" <[email-masked]>, [email-masked]
Sender: [email-masked]

Natale:

Draft-18 of the BGP specification is awaiting the FSM work.
Watch for the FSM wording within a week.

Please send the laundry list in two forms:

    1) Where the current draft-17 + FSM words does not
         match currently implemented code.

    2) what changes you like to see

Sue Hares

At 10:48 AM 7/9/2002 -0400, Natale, Jonathan wrote:
>I raised this issue more than two years ago.  What is the story on bgp?  Why
>is the draft/rfc so far off base?  I would be happy to provide a laundry
>list of things, if that will help.
>
>I think the MED was fixed in a major vendor's code with the ~= deterministic
>med cmd (or so I was told then, but now the story on that cmd seem
>different).
>
>The next hop was never "fixed".  The only answer for why not to aggregate if
>nxhp not the same was that it has something to do w/ mcast RPF... but I
>never fully digested this.
>
>-$0.02
>
>From: Fryczke, Pawel [mailto:[email-masked]]
>Sent: Thursday, July 04, 2002 8:08 AM
>To: [email-masked]
>Subject: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
>
>The latest draft, draft-ietf-idr-bgp4-17.txt states:
>Routes that have the following attributes shall not be aggregated
>unless the corresponding attributes of each route are identical:
>MULTI_EXIT_DISC, NEXT_HOP.
>
>Many vendors (e.g. Cisco/Juniper) does not follow this requirement. Why?
>What is the purpose of this requirement?
>Pawel

Date: Sun, 14 Jul 2002 00:07:55 -0400
To: Anindya Basu <[email-masked]>
From: Susan Hares <[email-masked]>
Subject: Re: IDR agenda
Cc: Susan Hares <[email-masked]>, [email-masked], [email-masked], yakov Rekhter <[email-masked]>
Sender: [email-masked]

Anindya:

We'll reserve a 10-15 slot for your paper.

Sue

At 10:08 AM 7/9/2002 -0400, Anindya Basu wrote:
>Dear WG Chairs,
>
>I would like a 10-15 minute slot for presenting the work in
>draft-wilfong-ibgp-oscillations-00.txt that addresses the route
>oscillation problem identified in
>draft-ietf-idr-route-oscillation-00.txt.
>
>Thanks for your consideration,
>-Anindya Basu
>  MTS, Bell Laboratories
>  Lucent Technologies
>
>Susan Hares wrote:
> >
> > Please send any requests for the agenda to the list.
> > The final agenda will be sent on Sunday.
> > (I am traveling to Japan until that date.)
> >
> > Sue Hares

Date: Sat, 13 Jul 2002 07:15:17 -0400
To: Kunihiro Ishiguro <[email-masked]>
From: Susan Hares <[email-masked]>
Subject: Re: TCP state machine doubts
Cc: [email-masked], BGP mailing list <[email-masked]>, Gilles Geerts <[email-masked]>
Sender: [email-masked]

Kunihiro:

At 08:49 PM 7/5/2002 -0700, Kunihiro Ishiguro wrote:
>Current draft-17 has some known problems.  I think Sue aleady prepared
>draft of draft-18.  So best way is updating the draft to draft-18 then

>My understanding is once BGP peer is establieshed, the ConnectRetryCnt
>is cleared.

Sue Hares

Date: Fri, 12 Jul 2002 16:20:14 -0400
From: Jeffrey Haas <[email-masked]>
To: Pedro Roque Marques <[email-masked]>
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

On Fri, Jul 12, 2002 at 12:14:38PM -0700, Pedro Roque Marques wrote:
> >From both quotes above it apears to me that you are proposing to
> discard an UPDATE and leave the session up. Since BGP state is
> incremental, that would mean the two peers would now potentially be
> out of sync.

And no less intentionally than if I discarded everything via policy.
In this case, my policy is "discard bad (for some value) packets".

Currently, we can only discard and move along.
The proposal with INFORM at least lets us tell our upstream that
they did something we didn't like, if they even care.

>   Pedro.

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 12 Jul 2002 12:14:38 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Jeffrey Haas writes:

> On Fri, Jul 12, 2002 at 02:33:05PM -0400, Natale, Jonathan wrote:
>> This is already done--if CONFED path is received the peer stays up
>> (and the local pref gets ignored).  This is good, but not in an rfc
>> nor a draft.  My vote is for an INFORM.

> IMO, this is a case where the route's local pref shouldn't be just
> ignored, but the entire packet should be discarded.

>From both quotes above it apears to me that you are proposing to
discard an UPDATE and leave the session up. Since BGP state is
incremental, that would mean the two peers would now potentially be
out of sync.

  Pedro.

Date: Fri, 12 Jul 2002 14:56:41 -0400
From: Jeffrey Haas <[email-masked]>
To: "Natale, Jonathan" <[email-masked]>
Cc: Alex Zinin <[email-masked]>, [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

On Fri, Jul 12, 2002 at 02:33:05PM -0400, Natale, Jonathan wrote:
> This is already done--if CONFED path is received the peer stays up (and the
> local pref gets ignored).  This is good, but not in an rfc nor a draft.  My
> vote is for an INFORM.

IMO, this is a case where the route's local pref shouldn't be just ignored,
but the entire packet should be discarded.

-- 
Jeff Haas 
NextHop Technologies

From: "Natale, Jonathan" <[email-masked]>
To: "'Jeffrey Haas'" <[email-masked]>, Alex Zinin <[email-masked]>
Cc: [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Fri, 12 Jul 2002 14:33:05 -0400
Sender: [email-masked]

This is already done--if CONFED path is received the peer stays up (and the
local pref gets ignored).  This is good, but not in an rfc nor a draft.  My
vote is for an INFORM.

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 2:29 PM
To: Alex Zinin
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)

Alex,

There are some circumstances under which we currently send notifications
where some weaker mechanism may be sufficient.  Almost every one of
these situations fall under the category of "well formed packet,
well-formed recognized attributes, weird/bogus data inside them".

As you infer, there are situations where we should just assume that
the buffer contains cruft and that the implementation needs a 
strong reset.

However, there are situations where we could get stuff that we can
deal with and just ignore the "Bad" things.  This was the case when
someone's implementation started leaking non-prefixed confederation
segments into the internet.  The packets were well formed, but had
cruft in them.

Similarly, if we ever add an origin type of 4, we'll have migration
issues.  If implementations just discarded updates that contained 
unrecognized data....

It may be worth our while to spend some time analyzing the protocol
for places where we can gracefully deal with failures and move along.

Or...
We could just say that NOTIFY is the right answer to deal with
confused implementations and trust to graceful restart to minimize
the impact on the Internet as a whole.  (But note that this wouldn't
have dealt with the non-prefixed confeds issue.)

On Mon, Jul 08, 2002 at 03:22:59PM -0700, Alex Zinin wrote:
>  BTW, a more generic question: if we can't trust an attribute
>  as a whole (since it's invalid), how can we trust specific fields
>  (AFI/SAFI) inside, i.e., if an MP_* attribute is malformed, it is
>  unsafe to assume that AFI/SAFI fields are correct... so, we may
>  very well end up deleting and ignoring a wrong (but valid) AFI/SAFI.

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 12 Jul 2002 14:28:33 -0400
From: Jeffrey Haas <[email-masked]>
To: Alex Zinin <[email-masked]>
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Alex,

There are some circumstances under which we currently send notifications
where some weaker mechanism may be sufficient.  Almost every one of
these situations fall under the category of "well formed packet,
well-formed recognized attributes, weird/bogus data inside them".

As you infer, there are situations where we should just assume that
the buffer contains cruft and that the implementation needs a 
strong reset.

However, there are situations where we could get stuff that we can
deal with and just ignore the "Bad" things.  This was the case when
someone's implementation started leaking non-prefixed confederation
segments into the internet.  The packets were well formed, but had
cruft in them.

Similarly, if we ever add an origin type of 4, we'll have migration
issues.  If implementations just discarded updates that contained 
unrecognized data....

It may be worth our while to spend some time analyzing the protocol
for places where we can gracefully deal with failures and move along.

Or...
We could just say that NOTIFY is the right answer to deal with
confused implementations and trust to graceful restart to minimize
the impact on the Internet as a whole.  (But note that this wouldn't
have dealt with the non-prefixed confeds issue.)

On Mon, Jul 08, 2002 at 03:22:59PM -0700, Alex Zinin wrote:
>  BTW, a more generic question: if we can't trust an attribute
>  as a whole (since it's invalid), how can we trust specific fields
>  (AFI/SAFI) inside, i.e., if an MP_* attribute is malformed, it is
>  unsafe to assume that AFI/SAFI fields are correct... so, we may
>  very well end up deleting and ignoring a wrong (but valid) AFI/SAFI.

-- 
Jeff Haas 
NextHop Technologies

Date: Fri, 12 Jul 2002 13:28:55 -0400
From: Jeffrey Haas <[email-masked]>
To: john heasley <[email-masked]>
Cc: "Srihari R. Sangli" <[email-masked]>, Stephen Waters <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: Re: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

On Thu, Jul 11, 2002 at 11:44:09AM -0700, john heasley wrote:
> 	"it must be possible to turn such options off and force
> 	 values/capabilities that will be required."

Considered from a slightly different angle, do you perhaps also mean:
1. If I said I supported some AFI/SAFI
2. I decide that your implementation of said AFI/SAFI is buggy
3. I send you a dynamic capability update withdrawing that AFI/SAFI
4. If you keep sending it after I sent you the capability update, what
   should I do?

To 4, answers are:
1. Wait a certain amount of time for some queues to clear.  If we
   still receive the unwanted messages, we should then NOTIFY.
2. Grin and bear it.

Other observations?

-- 
Jeff Haas 
NextHop Technologies

From: "Tsillas, Jim" <[email-masked]>
To: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Subject: RE: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 11:01:37 -0400
Sender: [email-masked]

Bruno,

how do most vendors handle the case where the MP-BGP
capabilities are disjoint? One could make a case for
just letting the session stay alive and not sending
the NOTIFY. After all a peer may be interested in receiving
a certain MP-BGP AF but not sending it.

I agree that in most cases sending the "unsupported
capability" notification is the wrong things to do if
you receive an unsupported capability. The text in the
draft is clear on this point, but a concrete example
would help.

-Jim.

-----Original Message-----
From: Bruno Rijsman [mailto:[email-masked]]
Sent: Friday, July 12, 2002 9:38 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re> Unsupported capability notification

The usage of the unsupported capability notification is widely
misunderstood and this is causing real problems in real networks.

If you receive a capability code X which you do not recognize,
you MUST NOT send back an unsupported capability notification.
It is sufficient for you to simply not advertise capability X
and as a result the peer will known you do not support capability X.

If you receive a capability code Y which you do recognize but
for some reason you do not want to use capability Y on that session
you SHOULD NOT send back an unsupported capability notification. It is
sufficient for you to simply not advertise capability Y and as a result
the peer will knwon you do not support capability Y.

I known of only one case where the sending of an "unsupported
capability" notification is justified: if both sides propose the
multi-protocol capability but the proposed set of AFI/SAFIs is
completely disjunct.

There are many implementations out there that send "unsupported
capability" notifications when they should not. For example,
there is one widely deployed implementation that sends an
"unsupported capability" notification when it receives the
four-octet-as-numbers capability. Even worse: this implementation
sends the notification without any data (i.e. we don't know which
is the offending capability) and often sends the notification before
the open (i.e. we can't guess which capabilities they support by
looking at their open).

--- Original message ---

"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker may send a
   NOTIFICATION message to the peer, and terminate peering. The Error
   Subcode in the message is set to Unsupported Capability. The message
   should contain the capability (capabilities) that causes the speaker
   to send the message.  The decision to send the message and terminate
   peering is local to the speaker.  Such peering should not be re-
   established automatically"
--RFC2842, "Capabilities Advertisement with BGP-4"

This sounds like:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then rtrA will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to send UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open with the offending capability

It seems to me that it should be:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open without the offending capability

Thoughts?

Thank you very much for your time.

_________________________________________________________________
MSN Photos is the easiest way to share and print your photos: 
http://photos.msn.com/support/worldwide.aspx

From: "Natale, Jonathan" <[email-masked]>
To: "'Tsillas, Jim'" <[email-masked]>, "'Mathew Richardson'" <[email-masked]>, [email-masked]
Subject: RE: Unsupported capability notification
Date: Fri, 12 Jul 2002 11:05:18 -0400
Sender: [email-masked]

Yes, I think it should

-----Original Message-----
From: Tsillas, Jim [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 10:40 AM
To: 'Mathew Richardson'; [email-masked]
Subject: RE: Unsupported capability notification

Maybe the draft should be updated to make this clear. I know
it took me a while to understand the intent and then only after
I observed other implementations.

-Jim.

-----Original Message-----
From: Mathew Richardson [mailto:[email-masked]]
Sent: Friday, July 12, 2002 10:41 AM
To: [email-masked]
Subject: Re: Unsupported capability notification

> Natale, Jonathan <[email-masked]> [Fri, Jul 12, 2002 at
09:50:43AM -0400]:
>Thanks.
>
>My question is:
>Does TXing unsupported capability=C mean:
>A) I want to RX open with cap. C
>OR
>B) I want to RX open without cap. C

<snip>

A.

>Before you answer B), which seems to be the obvious answer,

<snip>

It may seem obvious, but it is incorrect.  The router sending unsupported
capability is saying "I won't peer with you because you don't support
this capability..."

mrr

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Cc: [email-masked]
Subject: RE: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 10:59:24 -0400
Sender: [email-masked]

Agreed, if either router has no common ground to communicate on a basic
level
say IPv4 routes, then the session should not be established. But if there
are many
capabilities that are not agreed but one that does, the session should be
allowed
to be established. Again, rejecting router should send some sort of
indication to
offending router of the problem so there is no confusion on what the issue
is. This
notification should NOT cause a session drop.

If they simply state that in the spec, all confusion will be eliminated.

Ben

> From: Bruno Rijsman [mailto:[email-masked]]
> Sent: Friday, July 12, 2002 10:11 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: Re> Unsupported capability notification
> 
> 
> Re> My question is:
> Re> Does TXing unsupported capability=C mean:
> Re> A) I want to RX open with cap. C
> Re> OR
> Re> B) I want to RX open without cap. C
> 
> Sending an "unsupported capability notification containing
> capability C" means: "The fact that you support capability
> C means I don't want to have a session with you" or to put
> it differently "I want you to stop advertising capability
> C to me".
> 
> There are very few situations where this is really what you want.
> 
> If I simply don't support capability C or I don't want to use
> capability C on that session, I could simply refrain from advertising
> capability C in my open.
> 
> The one case that I know where sending a notification is justified is
> in multi-protocol negotiation. Say router A proposes multi-protocol
> ipv4 unicast. Router B proposes multi-protocol ipv6 multicast. When B
> receives the open from A (or vice versa) it may send an "unsupported
> capability" notification because they have no address families in
> common to exchange ("well if you are only interested in talking about
> ipv4 I don't want to talk to you").
> 
> 
> _________________________________________________________________
> Send and receive Hotmail on your mobile device: http://mobile.msn.com
> 

From: "Natale, Jonathan" <[email-masked]>
To: "'Abarbanel, Benjamin'" <[email-masked]>, "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Cc: "Natale, Jonathan" <[email-masked]>
Subject: RE: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 10:50:45 -0400
Sender: [email-masked]

True.

But in the case of the capability, the other side will not need to guess, it
knows what you support by you OPEN:
" A BGP speaker determines the capabilities supported by its peer by
   examining the list of capabilities present in the Capabilities
   Optional Parameter carried by the OPEN message that the speaker
   receives from the peer."

...This part of the spec seems very clear and very sensible.

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 10:40 AM
To: 'Bruno Rijsman'; [email-masked]
Cc: [email-masked]
Subject: RE: Re> Unsupported capability notification

Guys:
  I really think the NOTIFY message without clearing the session can take 
care of all of these conflicts and still let both routers know what they 
can or cannot do. Just ignoring something and hoping the other side will
figure it out by deduction, usually leads to assumptions that are wrong.

Ben

> From: Bruno Rijsman [mailto:[email-masked]]
> Sent: Friday, July 12, 2002 9:38 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re> Unsupported capability notification
> 
> 
> The usage of the unsupported capability notification is widely
> misunderstood and this is causing real problems in real networks.
> 
> If you receive a capability code X which you do not recognize,
> you MUST NOT send back an unsupported capability notification.
> It is sufficient for you to simply not advertise capability X
> and as a result the peer will known you do not support capability X.
> 
> If you receive a capability code Y which you do recognize but
> for some reason you do not want to use capability Y on that session
> you SHOULD NOT send back an unsupported capability notification. It is
> sufficient for you to simply not advertise capability Y and 
> as a result
> the peer will knwon you do not support capability Y.
> 
> I known of only one case where the sending of an "unsupported
> capability" notification is justified: if both sides propose the
> multi-protocol capability but the proposed set of AFI/SAFIs is
> completely disjunct.
> 
> There are many implementations out there that send "unsupported
> capability" notifications when they should not. For example,
> there is one widely deployed implementation that sends an
> "unsupported capability" notification when it receives the
> four-octet-as-numbers capability. Even worse: this implementation
> sends the notification without any data (i.e. we don't know which
> is the offending capability) and often sends the notification before
> the open (i.e. we can't guess which capabilities they support by
> looking at their open).
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --- Original message ---
> 
> "If a BGP speaker that supports a certain capability determines that
>    its peer doesn't support this capability, the speaker may send a
>    NOTIFICATION message to the peer, and terminate peering. The Error
>    Subcode in the message is set to Unsupported Capability. 
> The message
>    should contain the capability (capabilities) that causes 
> the speaker
>    to send the message.  The decision to send the message and 
> terminate
>    peering is local to the speaker.  Such peering should not be re-
>    established automatically"
> --RFC2842, "Capabilities Advertisement with BGP-4"
> 
> 
> This sounds like:
>     if rtrA is doing capC   and rtrB is not doing CapC,
>         then rtrA will send an UnsupCapNotifErrorSubcode=capC
> 
>     ...so if you were to send UnsupCapNotifErrorSubcode=capC
>         then you would/could/should re-open with the 
> offending capability
> 
> It seems to me that it should be:
>     if rtrA is doing capC   and rtrB is not doing CapC,
>         then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC
> 
>     ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
>         then you would/could/should re-open without the 
> offending capability
> 
> Thoughts?
> 
> Thank you very much for your time.
> 
> 
> _________________________________________________________________
> MSN Photos is the easiest way to share and print your photos: 
> http://photos.msn.com/support/worldwide.aspx
> 

From: "Natale, Jonathan" <[email-masked]>
To: "'Abarbanel, Benjamin'" <[email-masked]>, "Natale, Jonathan" <[email-masked]>, "'Bgp ([email-masked])'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Cc: "'Bruno Rijsman'" <[email-masked]>
Subject: RE: BGP Capability Negotiation
Date: Fri, 12 Jul 2002 10:45:51 -0400
Sender: [email-masked]

Thanks,  I agree with Bruno:
" The one case that I know where sending a notification is justified is in
multi-protocol negotiation. Say router A proposes multi-protocol ipv4
unicast. Router B proposes multi-protocol ipv6 multicast. When B receives
the open from A (or vice versa) it may send an "unsupported capability"
notification because they have no address families in common to exchange
("well if you are only interested in talking about ipv4 I don't want to talk
to you")."

--this seems to be the only case.

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 10:36 AM
To: 'Natale, Jonathan'; 'Bgp ([email-masked])'; [email-masked]';
[email-masked]'
Subject: RE: BGP Capability Negotiation

Jonathan:

as the original spec says. 
> 
> "If a BGP speaker that supports a certain capability determines that
>    its peer doesn't support this capability, the speaker may send a
>    NOTIFICATION message to the peer, and terminate peering. The Error
>    Subcode in the message is set to Unsupported Capability. 
> The message
>    should contain the capability (capabilities) that causes 
> the speaker
>    to send the message.  The decision to send the message and 
> terminate
>    peering is local to the speaker.  Such peering should not be re-
>    established automatically"
> --RFC2842, "Capabilities Advertisement with BGP-4"
> 
> 

It should say, "such peering with the offending capability should not be 
re-established automatically. Thus if the offending router drops the
capability
from the OPEN message, it can retry to establish the session."

If I were a network operator, I rather have a session up with less
capabilities, 
than not have one at all and loose reachability through a perfectly good
router.
Or till the operator manually unconfigured the capability from the offending
router.

Ben

From: "Tsillas, Jim" <[email-masked]>
To: "'Mathew Richardson'" <[email-masked]>, [email-masked]
Subject: RE: Unsupported capability notification
Date: Fri, 12 Jul 2002 10:40:21 -0400
Sender: [email-masked]

Maybe the draft should be updated to make this clear. I know
it took me a while to understand the intent and then only after
I observed other implementations.

-Jim.

-----Original Message-----
From: Mathew Richardson [mailto:[email-masked]]
Sent: Friday, July 12, 2002 10:41 AM
To: [email-masked]
Subject: Re: Unsupported capability notification

> Natale, Jonathan <[email-masked]> [Fri, Jul 12, 2002 at
09:50:43AM -0400]:
>Thanks.
>
>My question is:
>Does TXing unsupported capability=C mean:
>A) I want to RX open with cap. C
>OR
>B) I want to RX open without cap. C

<snip>

A.

>Before you answer B), which seems to be the obvious answer,

<snip>

It may seem obvious, but it is incorrect.  The router sending unsupported
capability is saying "I won't peer with you because you don't support
this capability..."

mrr

Date: Fri, 12 Jul 2002 10:40:55 -0400
From: Mathew Richardson <[email-masked]>
To: [email-masked]
Subject: Re: Unsupported capability notification
Sender: [email-masked]

> Natale, Jonathan <[email-masked]> [Fri, Jul 12, 2002 at 09:50:43AM -0400]:
>Thanks.
>
>My question is:
>Does TXing unsupported capability=C mean:
>A) I want to RX open with cap. C
>OR
>B) I want to RX open without cap. C

<snip>

A.

>Before you answer B), which seems to be the obvious answer,

<snip>

It may seem obvious, but it is incorrect.  The router sending unsupported
capability is saying "I won't peer with you because you don't support
this capability..."

mrr

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Cc: [email-masked]
Subject: RE: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 10:39:37 -0400
Sender: [email-masked]

Guys:
  I really think the NOTIFY message without clearing the session can take 
care of all of these conflicts and still let both routers know what they 
can or cannot do. Just ignoring something and hoping the other side will
figure it out by deduction, usually leads to assumptions that are wrong.

Ben

> From: Bruno Rijsman [mailto:[email-masked]]
> Sent: Friday, July 12, 2002 9:38 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re> Unsupported capability notification
> 
> 
> The usage of the unsupported capability notification is widely
> misunderstood and this is causing real problems in real networks.
> 
> If you receive a capability code X which you do not recognize,
> you MUST NOT send back an unsupported capability notification.
> It is sufficient for you to simply not advertise capability X
> and as a result the peer will known you do not support capability X.
> 
> If you receive a capability code Y which you do recognize but
> for some reason you do not want to use capability Y on that session
> you SHOULD NOT send back an unsupported capability notification. It is
> sufficient for you to simply not advertise capability Y and 
> as a result
> the peer will knwon you do not support capability Y.
> 
> I known of only one case where the sending of an "unsupported
> capability" notification is justified: if both sides propose the
> multi-protocol capability but the proposed set of AFI/SAFIs is
> completely disjunct.
> 
> There are many implementations out there that send "unsupported
> capability" notifications when they should not. For example,
> there is one widely deployed implementation that sends an
> "unsupported capability" notification when it receives the
> four-octet-as-numbers capability. Even worse: this implementation
> sends the notification without any data (i.e. we don't know which
> is the offending capability) and often sends the notification before
> the open (i.e. we can't guess which capabilities they support by
> looking at their open).
> 
> 
> 
> 
> 
> 
> 
> 
> 
> --- Original message ---
> 
> "If a BGP speaker that supports a certain capability determines that
>    its peer doesn't support this capability, the speaker may send a
>    NOTIFICATION message to the peer, and terminate peering. The Error
>    Subcode in the message is set to Unsupported Capability. 
> The message
>    should contain the capability (capabilities) that causes 
> the speaker
>    to send the message.  The decision to send the message and 
> terminate
>    peering is local to the speaker.  Such peering should not be re-
>    established automatically"
> --RFC2842, "Capabilities Advertisement with BGP-4"
> 
> 
> This sounds like:
>     if rtrA is doing capC   and rtrB is not doing CapC,
>         then rtrA will send an UnsupCapNotifErrorSubcode=capC
> 
>     ...so if you were to send UnsupCapNotifErrorSubcode=capC
>         then you would/could/should re-open with the 
> offending capability
> 
> It seems to me that it should be:
>     if rtrA is doing capC   and rtrB is not doing CapC,
>         then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC
> 
>     ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
>         then you would/could/should re-open without the 
> offending capability
> 
> Thoughts?
> 
> Thank you very much for your time.
> 
> 
> _________________________________________________________________
> MSN Photos is the easiest way to share and print your photos: 
> http://photos.msn.com/support/worldwide.aspx
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Natale, Jonathan'" <[email-masked]>, "'Bgp ([email-masked])'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: BGP Capability Negotiation
Date: Fri, 12 Jul 2002 10:35:57 -0400
Sender: [email-masked]

Jonathan:

as the original spec says. 
> 
> "If a BGP speaker that supports a certain capability determines that
>    its peer doesn't support this capability, the speaker may send a
>    NOTIFICATION message to the peer, and terminate peering. The Error
>    Subcode in the message is set to Unsupported Capability. 
> The message
>    should contain the capability (capabilities) that causes 
> the speaker
>    to send the message.  The decision to send the message and 
> terminate
>    peering is local to the speaker.  Such peering should not be re-
>    established automatically"
> --RFC2842, "Capabilities Advertisement with BGP-4"
> 
> 

It should say, "such peering with the offending capability should not be 
re-established automatically. Thus if the offending router drops the
capability
from the OPEN message, it can retry to establish the session."

If I were a network operator, I rather have a session up with less
capabilities, 
than not have one at all and loose reachability through a perfectly good
router.
Or till the operator manually unconfigured the capability from the offending
router.

Ben

From: "Bruno Rijsman" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: RE: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 10:10:43 -0400
Sender: [email-masked]

Re> My question is:
Re> Does TXing unsupported capability=C mean:
Re> A) I want to RX open with cap. C
Re> OR
Re> B) I want to RX open without cap. C

Sending an "unsupported capability notification containing
capability C" means: "The fact that you support capability
C means I don't want to have a session with you" or to put
it differently "I want you to stop advertising capability
C to me".

There are very few situations where this is really what you want.

If I simply don't support capability C or I don't want to use
capability C on that session, I could simply refrain from advertising
capability C in my open.

The one case that I know where sending a notification is justified is
in multi-protocol negotiation. Say router A proposes multi-protocol
ipv4 unicast. Router B proposes multi-protocol ipv6 multicast. When B
receives the open from A (or vice versa) it may send an "unsupported
capability" notification because they have no address families in
common to exchange ("well if you are only interested in talking about
ipv4 I don't want to talk to you").

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com

From: "Natale, Jonathan" <[email-masked]>
To: "'Bgp ([email-masked])'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: recursion flap
Date: Fri, 12 Jul 2002 09:52:35 -0400
Sender: [email-masked]

Here is another twist on the same issue:

This seems to be an outgrowth of <another vendor> practice of assigning
<another vendor> point-to-point interface IP addresses with a /32 mask and
then redistributing direct/connected.  I believe the /32 for yourself is
incorrect, but:
"A third option is to <another vendor> e host addresses on both ends of a
point-to-
   point link.  This option provides the same address space savings as
   <another vendor> ing a 31-bit subnet mask, but may only be <another
vendor> ed in links <another vendor> ing PPP
   encapsulation [RFC1332].  The <another vendor> e of host addresses allows
for the
   assignment of IP addresses belonging to different networks at each
   side of the link, causing link and network management not to be
   straight forward."
    --RFC3021, "Using 31-Bit Prefixes on IPv4 Point-to-Point Links"
(PROPOSED STANDARD)

Using BGP, <another vendor> told the <a major vendor> that to get to
50.1.1.2/32 it should use the next hop of 50.1.1.2.  A host route with a
nexthop equal to itself is not mentioned in the bgp RFC nor in the draft.
At least I did not find it [yet], but I did not really look that hard
because it is not that important.  Intuitively, it should not be sent and if
received it should be ignored.  In this regard it IS a <a major vendor> bug
(but that may not help--<another vendor> should not advertise such a route,
and/or <another vendor> e the "work-around" I propose below).

RFC:
    ...blah blah blah...
    "Mutually recursive routes (routes
    resolving each other or themselves), also fail the resolvability
    check"
    ...blah blah blah...

--draft-ietf-idr-bgp4-17.txt

I suspect that, if you watch it, you will see the bgp 50.1.1.2/32 route get
removed due to the recursiveness.  Then it will get re-installed based on
the 1.1.0/24 directly connected route.  This will repeat based on the
["nettable walker" or "scan" ???] <a major vendor> timer.  The interval
(half the period) is 60 seconds and is not configurable (again, not in any
RFCs). 

Also:
    ...blah blah blah...
    "implementations must
    take care that before any packets are forwarded along a BGP route,
    its associated NEXT_HOP address is resolved to the immediate
    (directly connected) next-hop address and this address"
    ...blah blah blah...

--draft-ietf-idr-bgp4-17.txt

And at one angle, the 50.1.1.2 address cannot be directly connected to
anybody but <another vendor>  since it has a /32 mask and <another vendor>
are its 1 and only 1 host.  So <another vendor> should not send it since it
will never be resolved.  There is some mention of the 32  I suppose it would
be ok to send the bgp 50.1.1.2/32 route to peers that are not on the same
point-to-point link, since in that case the next hop and prefix would not be
recursive; but then <another vendor> might break
the: "care must be taken to ensure a consistent view of routing within the
AS." (RFC1771) rule.  My vote is against allowing the bgp 50.1.1.2/32 route
to be sent to peers that are not on the same point-to-point link (or any
other link).  Also, I think maybe bgp should try a next longest bit match to
resolve the next hop.

The Work around is to add:
ip route 50.1.1.2 255.255.255.255 atm 4/0.1 on <major vendor>

Thoughts?

-----Original Message-----
From: Natale, Jonathan 
Sent: Thursday, July 11, 2002 7:47 PM
To: Bgp ([email-masked]); [email-masked]; [email-masked]
Subject: recursion flap

if the peer is reachable via OPSF and EBGP then the route tab will have the
EBGP route (since the admin dist of EBGP is 20; OSPF is 110) so it will be
flagged as a recursion error and removed.  Then, when a 1 minute timer
triggers, the BGP route will be removed and the OPSF route will be installed
due to the recursion.  Then, when the 1 minute timer triggers again, the
OPSF route will be removed and the BGP route will be installed due to the
lower admin dist. ...Etc... (2 minute period)  Not sure how they resolve
this, how <another vendor> should, and/or how "real" this problem is

???:
timer in rfc?
A "persistent flap"?
Notification/inform?
Juniper?

From: "Natale, Jonathan" <[email-masked]>
To: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Cc: "Natale, Jonathan" <[email-masked]>
Subject: RE: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 09:50:43 -0400
Sender: [email-masked]

Thanks.

My question is:
Does TXing unsupported capability=C mean:
A) I want to RX open with cap. C
OR
B) I want to RX open without cap. C

Before you answer B), which seems to be the obvious answer, please read:
""If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker MAY send a
   NOTIFICATION message to the peer, and terminate peering (see Section
   "Extensions to Error Handling" for more details). The Error Subcode
   in the message is set to Unsupported Capability. The message SHOULD
   contain the capability (capabilities) that causes the speaker to send
   the message.  The decision to send the message and terminate peering
   is local to the speaker. If terminated, such peering SHOULD NOT be
   re-established automatically." -- draft-ietf-idr-rfc2842bis-02.txt

-----Original Message-----
From: Bruno Rijsman [mailto:[email-masked]] 
Sent: Friday, July 12, 2002 9:38 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re> Unsupported capability notification

The usage of the unsupported capability notification is widely
misunderstood and this is causing real problems in real networks.

If you receive a capability code X which you do not recognize,
you MUST NOT send back an unsupported capability notification.
It is sufficient for you to simply not advertise capability X
and as a result the peer will known you do not support capability X.

If you receive a capability code Y which you do recognize but
for some reason you do not want to use capability Y on that session
you SHOULD NOT send back an unsupported capability notification. It is
sufficient for you to simply not advertise capability Y and as a result
the peer will knwon you do not support capability Y.

I known of only one case where the sending of an "unsupported
capability" notification is justified: if both sides propose the
multi-protocol capability but the proposed set of AFI/SAFIs is
completely disjunct.

There are many implementations out there that send "unsupported
capability" notifications when they should not. For example,
there is one widely deployed implementation that sends an
"unsupported capability" notification when it receives the
four-octet-as-numbers capability. Even worse: this implementation
sends the notification without any data (i.e. we don't know which
is the offending capability) and often sends the notification before
the open (i.e. we can't guess which capabilities they support by
looking at their open).

--- Original message ---

"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker may send a
   NOTIFICATION message to the peer, and terminate peering. The Error
   Subcode in the message is set to Unsupported Capability. The message
   should contain the capability (capabilities) that causes the speaker
   to send the message.  The decision to send the message and terminate
   peering is local to the speaker.  Such peering should not be re-
   established automatically"
--RFC2842, "Capabilities Advertisement with BGP-4"

This sounds like:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then rtrA will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to send UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open with the offending capability

It seems to me that it should be:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open without the offending capability

Thoughts?

Thank you very much for your time.

_________________________________________________________________
MSN Photos is the easiest way to share and print your photos: 
http://photos.msn.com/support/worldwide.aspx

From: "Bruno Rijsman" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re> Unsupported capability notification
Date: Fri, 12 Jul 2002 09:37:50 -0400
Sender: [email-masked]

The usage of the unsupported capability notification is widely
misunderstood and this is causing real problems in real networks.

If you receive a capability code X which you do not recognize,
you MUST NOT send back an unsupported capability notification.
It is sufficient for you to simply not advertise capability X
and as a result the peer will known you do not support capability X.

If you receive a capability code Y which you do recognize but
for some reason you do not want to use capability Y on that session
you SHOULD NOT send back an unsupported capability notification. It is
sufficient for you to simply not advertise capability Y and as a result
the peer will knwon you do not support capability Y.

I known of only one case where the sending of an "unsupported
capability" notification is justified: if both sides propose the
multi-protocol capability but the proposed set of AFI/SAFIs is
completely disjunct.

There are many implementations out there that send "unsupported
capability" notifications when they should not. For example,
there is one widely deployed implementation that sends an
"unsupported capability" notification when it receives the
four-octet-as-numbers capability. Even worse: this implementation
sends the notification without any data (i.e. we don't know which
is the offending capability) and often sends the notification before
the open (i.e. we can't guess which capabilities they support by
looking at their open).

--- Original message ---

"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker may send a
   NOTIFICATION message to the peer, and terminate peering. The Error
   Subcode in the message is set to Unsupported Capability. The message
   should contain the capability (capabilities) that causes the speaker
   to send the message.  The decision to send the message and terminate
   peering is local to the speaker.  Such peering should not be re-
   established automatically"
--RFC2842, "Capabilities Advertisement with BGP-4"

This sounds like:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then rtrA will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to send UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open with the offending capability

It seems to me that it should be:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open without the offending capability

Thoughts?

Thank you very much for your time.

_________________________________________________________________
MSN Photos is the easiest way to share and print your photos: 
http://photos.msn.com/support/worldwide.aspx

From: "Tsillas, Jim" <[email-masked]>
To: "'Natale, Jonathan'" <[email-masked]>, "'Bgp ([email-masked])'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: BGP Capability Negotiation
Date: Fri, 12 Jul 2002 09:14:15 -0400
Sender: [email-masked]

Jon,

I think the draft is OK on this point. An example:
rtrA supports a route-refresh capability and tells rtrB.
rtrB doesn't support this capability. rtrB will not
notify rtrA because it doesn't care to use rtrA's advertised
capability (it ignores the capability). rtrA may notify
rtrB because it may require that rtrB support route-refresh
too. On the other hand, rtrA may not care to notify rtrB
since the capability may be "one way": in the case of
route-refresh rtrB doesn't need to support the capability
in order to use rtrA's refresh capability. In the case
of an MP-BGP capability, rtrA will probably want to
notify rtrB since the capability will probably need to
be bidirectional (what is the common practice in this case??)

-Jim.

-----Original Message-----
From: Natale, Jonathan [mailto:[email-masked]]
Sent: Friday, July 12, 2002 9:05 AM
To: 'Bgp ([email-masked])'; [email-masked]';
[email-masked]'
Subject: BGP Capability Negotiation

"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker may send a
   NOTIFICATION message to the peer, and terminate peering. The Error
   Subcode in the message is set to Unsupported Capability. The message
   should contain the capability (capabilities) that causes the speaker
   to send the message.  The decision to send the message and terminate
   peering is local to the speaker.  Such peering should not be re-
   established automatically"
--RFC2842, "Capabilities Advertisement with BGP-4"

This sounds like:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then rtrA will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to send UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open with the offending capability

It seems to me that it should be:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open without the offending capability

Thoughts?

Thank you very much for your time.

From: "Natale, Jonathan" <[email-masked]>
To: "'Bgp ([email-masked])'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: BGP Capability Negotiation
Date: Fri, 12 Jul 2002 09:05:06 -0400
Sender: [email-masked]

"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker may send a
   NOTIFICATION message to the peer, and terminate peering. The Error
   Subcode in the message is set to Unsupported Capability. The message
   should contain the capability (capabilities) that causes the speaker
   to send the message.  The decision to send the message and terminate
   peering is local to the speaker.  Such peering should not be re-
   established automatically"
--RFC2842, "Capabilities Advertisement with BGP-4"

This sounds like:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then rtrA will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to send UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open with the offending capability

It seems to me that it should be:
    if rtrA is doing capC   and rtrB is not doing CapC,
        then ***rtrB*** will send an UnsupCapNotifErrorSubcode=capC

    ...so if you were to ***receive*** UnsupCapNotifErrorSubcode=capC
        then you would/could/should re-open without the offending capability

Thoughts?

Thank you very much for your time.

From: "Natale, Jonathan" <[email-masked]>
To: "Bgp ([email-masked])" <[email-masked]>, [email-masked], [email-masked]
Subject: recursion flap
Date: Thu, 11 Jul 2002 19:47:28 -0400
Sender: [email-masked]

if the peer is reachable via OPSF and EBGP then the route tab will have the
EBGP route (since the admin dist of EBGP is 20; OSPF is 110) so it will be
flagged as a recursion error and removed.  Then, when a 1 minute timer
triggers, the BGP route will be removed and the OPSF route will be installed
due to the recursion.  Then, when the 1 minute timer triggers again, the
OPSF route will be removed and the BGP route will be installed due to the
lower admin dist. ...Etc... (2 minute period)  Not sure how they resolve
this, how we should, and/or how "real" this problem is

???:
timer in rfc?
A "persistent flap"?
Notification/inform?
Juniper?

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Tsillas, Jim'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: graceful restart TCP reset discovery (was Re>Are dynamic capa bili ties really necessary ...)
Date: Thu, 11 Jul 2002 16:53:20 -0400
Sender: [email-masked]

Jim:
  Understood, you make good points.

Regards,
Ben

> From: Tsillas, Jim [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 3:21 PM
> To: 'Abarbanel, Benjamin'
> Cc: [email-masked]'
> Subject: graceful restart TCP reset discovery (was Re>Are dynamic
> capabili ties really necessary ...)
> 
> 
> Ben,
> 
> You are correct. I was thinking about the case where TCP is
> reset due to "fast external fallover". In this case, the
> forwarding plane is down and if it stays down for more than
> the graceful restart timeout, the receiving speaker resorts to
> normal restart.
> 
> Sections 6.1 and 6.2 go into good detail on this. The length
> of time it takes to determine if TCP has reset is implementation
> dependent. One way to detect TCP reset is to look for a new
> session for the same peer (since the neighbor will try to
> intiate a new session when it restarts). Another way is to rely
> on "fast-external-fallover" being turned on. Otherwise you just
> have to wait for either the keepalive expiration or the TCP timeout.
> 
> There is also an obscure case with some (probably most) TCPs where
> the restarting speaker has not yet attempted to open the session but
> the receiving speaker attempts to send data on a non-existing session.
> This causes the restarting speaker to send a FIN back. There are
> probably other equally obscure cases which are implementation 
> dependent.
> 
> Once TCP reset is detected, we start the graceful restart timeout
> period. This means that in some cases, the total timeout can
> also include the amount of time it took to discover the TCP
> session has been reset. The draft makes no distinction as far
> as how the session was reset, as far as timeout is concerned.
> 
> -Jim.
> 
> From: Abarbanel, Benjamin [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 2:43 PM
> To: 'Tsillas, Jim'; Abarbanel, Benjamin; 'Yakov Rekhter'; Srihari R.
> Sangli
> Cc: [email-masked]'
> Subject: RE: Are dynamic capabilities really necessary if 
> peers support
> gr acef ul restart? 
> 
> 
> 
> 
> > graceful restart can also be useful when the control plane
> > is implemented separately from the forwarding plane. When
> > this is true, the control place can reset (or fail-over)
> > while the forwarding plane is unaffected. The reset, can
> > be made transparent to the forwarding plane. Graceful
> > restart can also serve to limit the scope of the reset to
> > just the given speaker and its neighbors. In other words,
> > the reset state doesn't propagate through the network.
> >
> 
> Thanks for the clarification. It would be nice if applicable uses for
> gracefull restart were added to the draft. 
> 
> > The case which you point out (where TCP is broken) is actually
> > not covered since a broken TCP session implies a broken
> > forwarding plane.
> > 
> 
> I disagree, this is taken out of draft-ietf-idr-restart-05.txt, 
> Abstract section 2.
>                                     
>  "Finally,procedures are outlined for temporarily retaining routing
> information
>   across a TCP transport reset." 
> 
> I assume it means we hold routing information after the 
> forwarding plane
> is broken (BGP peering session dropped), right?
> 
> 
> > -Jim.
> > 
> > From: Abarbanel, Benjamin [mailto:[email-masked]]
> > Sent: Thursday, July 11, 2002 1:05 PM
> > To: 'Yakov Rekhter'; Srihari R. Sangli
> > Cc: Tsillas, Jim; [email-masked]'
> > Subject: RE: Are dynamic capabilities really necessary if 
> > peers support
> > gr acef ul restart? 
> > 
> > 
> > Yakov:
> >   The only time I think gracefull restart is usefull is if 
> > two peers are
> > operating
> > fine, but the TCP connection is broken by the network. In the 
> > case that
> > either peer drops the session or either peer reboot/crash, I 
> > dont see much
> > value in gracefull restart.
> > 
> > Did I miss something?
> > 
> > Ben
> > 
> > > From: Yakov Rekhter [mailto:[email-masked]]
> > > Sent: Thursday, July 11, 2002 12:55 PM
> > > To: Srihari R. Sangli
> > > Cc: Tsillas, Jim; [email-masked]'
> > > Subject: Re: Are dynamic capabilities really necessary if 
> > > peers support
> > > gracef ul restart? 
> > > 
> > > 
> > > Srihari,
> > > 
> > > > > It seems we are entering major discussion on dynamic
> > > > > capabilities and the can of worms is open. Can anyone
> > > > > think of a scenario where a dynamic capabilities
> > > > > negotiation is still needed if we have graceful restart?
> > > > 
> > > > Talking about address-families exchanged between BGP 
> speakers, the
> > > > dynamic capability is useful for enabling a new 
> address-family or
> > > > disabling a address-family w/o disrupting the session 
> and/or other
> > > > address-families. It is also useful for turning ON/OFF 
> > > other capabilities
> > > > (like ORF, etc)  without much disruption. 
> > > 
> > > Just to clarify, in the presence of graceful restart the 
> disruption
> > > you refer to above is just the disruption in the routing peering
> > > between the two speakers. There is no disruption in the 
> > > data/forwarding
> > > plane and no disruption in any other routing peering. Correct ?
> > > 
> > > Yakov.
> > > 
> > 
> 

Date: Thu, 11 Jul 2002 13:35:50 -0700 (PDT)
From: "Srihari R. Sangli" <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support  gracef ul restart?
Sender: [email-masked]

Ben,

> Srihari:
>   I looked over the draft again and I have to apologize I misunderstood you
> responses below. I would recommend rewording the two paragraph so that
> others
> do not get confused like me.

Ok. Will be added in the next revision. Thanks for the feedback.

srihari...

Date: Thu, 11 Jul 2002 13:32:14 -0700 (PDT)
From: "Srihari R. Sangli" <[email-masked]>
To: john heasley <[email-masked]>
Cc: Stephen Waters <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, <[email-masked]>
Subject: Re: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

Hi John,

> >
> > > A network manager wanting to change the state or extent of a
> > > capability would appreciate doing that within a bgp session, wouldn't
> > > they?
> > >
> >
> > The network manager can turn off other features (or capabilities) and thus
> > communicate such an event to the remote peers via this message. To keep it
> > simple, the draft does not allow turning OFF "Dynamic-capability" via the
> > capability message.
> >
> > srihari...
> > p.s. If I've misinterpreted your message, I apologise.
>
> these functions run chills up my spine.  i fear not being able to disable
> various capabilities and auto-negotiation when faced with a business policy
> or buggy or abusive peer/customer (or bugs in my own equipment, requiring
> s/w upgrades - requiring a maintenance window to be scheduled).
>
> so, not a complaint, but a comment on design and to vendors from an
> operator perspective:
>
> 	"it must be possible to turn such options off and force
> 	 values/capabilities that will be required."
>
> eg: if i do not want to accept filters from a peer, then i ought to be able
> disable that functionality.  or, i want nlri unicast and multicast, then i
> ought to be able to force that negotiation, else the session is reset or the
> dynamic re-negotiation is denied.
>
> the more flexible the configuration knobs the better, imiho.

Agreed. What you are suggesting is still implementable given the current
state of the draft.

srihari...

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Mathew Richardson'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Srihari R. Sangli'" <[email-masked]>, "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 15:48:46 -0400
Sender: [email-masked]

Hi Mathew:
 Did not mean to sound like I am sniping. My second mail 
to srihari should cover it all.

Regards,
Ben

> From: Mathew Richardson [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 3:35 PM
> To: Abarbanel, Benjamin
> Cc: 'Srihari R. Sangli'; 'Tsillas, Jim'; [email-masked]';
> [email-masked]'
> Subject: Re: Re>Are dynamic capabilities really necessary if peers
> support gracef ul restart?
> 
> 
> > Abarbanel, Benjamin <[email-masked]> [Thu, 
> Jul 11, 2002 at 02:52:19PM -0400]:
> >> > I have a question on the current 
> >> draft-ietf-idr-dynamic-cap-02.txt, section
> >> > 7 - Operation, 3rd and 4th paragraph:
> >> >
> >> >     "The Dynamic Capability itself can not be revised 
> >> dynamically via a
> 
> May I suggest that this sentence be changed to read:
> 
> "The Dynamic Capability Capability can not be revised
> dynamically via a ..."
> 
> ?
> 
> Would this change address your concerns, Mr. Abarbanel?
> 
> >> >      CAPABILITY message. The lifetime of the Dynamic 
> >> Capability is the
> >> >      duration of the BGP session in which the capability is 
> >> advertised.
> >> 
> >> This is referring to the "Dynamic-Capability" itself.  
> This capability
> >> can ONLY appear in the OPEN message.
> >> 
> >
> >I thought this spec was talking about dynamic capabilities 
> not static 
> >ones, so why are we assuming its the one in the OPEN message?
> 
> I don't think that was an assumption... it was an assertion.  He's
> saying that the Capability that advertises the ability to perform
> Dynamic-Capability negotiations MUST NOT be negotiated dynamically;
> it is only legal to advertise the Dynamic-Capability Capability in
> OPEN messages.
> 
> >What purpose does it serve to talk about static capabilities in this
> >draft, other than confuse the reader?
> 
> This is the draft in which the Dynamic-Capability Capability is
> introduced.  Hence, it is the logical place to talk about the
> limitations of the Dynamic-Capability Capability.
> 
> <snip>
> 
> mrr
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Srihari R. Sangli'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 15:43:02 -0400
Sender: [email-masked]

Srihari:
  I looked over the draft again and I have to apologize I misunderstood you 
responses below. I would recommend rewording the two paragraph so that
others
do not get confused like me.

I have added the implied changes in the paragraph to fix the issue. Below.

Ben

> >
> > I have a question on the current 
> draft-ietf-idr-dynamic-cap-02.txt, section
> > 7 - Operation, 3rd and 4th paragraph:
> >
> >     "The Dynamic Capability itself can not be revised 
> dynamically via a
> >      CAPABILITY message. The lifetime of the Dynamic 
> Capability is the
> >      duration of the BGP session in which the capability is 
> advertised.
> 

I would reword it as follows:

"The Dynamic Capability feature defined in the OPEN message can not
 be revised dynamically via a CAPABILITY message. The lifetime of the
 Dynamic Capabaility feature is the duration of the BGP session."

> This is referring to the "Dynamic-Capability" itself.  This capability
> can ONLY appear in the OPEN message.
> 
> >      Upon receiving a CAPABILITY message from its peer, the 
> BGP speaker
> >      shall update the capabilities previously received from 
> that peer
> >      based on the "Action" in the message, and then 
> function in accordance
> >      with the revised capabilities for the peer. Any unrecognized
> >      capabilities in the message should be ignored."
> 

"Upon receiving a CAPABILITY message with specific capabilities from
 its peer, the BGP speaker shall update those specific capabilities 
 previously received from that peer based on the "Action" in the 
 message, and then function in accordance with the revised capabilities
 for the peer. 

 Any unrecognized capabilities found in the CAPABALITY message should be 
 ignored".

Regards,
Ben

Date: Thu, 11 Jul 2002 15:35:03 -0400
From: Mathew Richardson <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Srihari R. Sangli'" <[email-masked]>, "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: Re: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

> Abarbanel, Benjamin <[email-masked]> [Thu, Jul 11, 2002 at 02:52:19PM -0400]:
>> > I have a question on the current 
>> draft-ietf-idr-dynamic-cap-02.txt, section
>> > 7 - Operation, 3rd and 4th paragraph:
>> >
>> >     "The Dynamic Capability itself can not be revised 
>> dynamically via a

May I suggest that this sentence be changed to read:

"The Dynamic Capability Capability can not be revised
dynamically via a ..."

?

Would this change address your concerns, Mr. Abarbanel?

>> >      CAPABILITY message. The lifetime of the Dynamic 
>> Capability is the
>> >      duration of the BGP session in which the capability is 
>> advertised.
>> 
>> This is referring to the "Dynamic-Capability" itself.  This capability
>> can ONLY appear in the OPEN message.
>> 
>
>I thought this spec was talking about dynamic capabilities not static 
>ones, so why are we assuming its the one in the OPEN message?

I don't think that was an assumption... it was an assertion.  He's
saying that the Capability that advertises the ability to perform
Dynamic-Capability negotiations MUST NOT be negotiated dynamically;
it is only legal to advertise the Dynamic-Capability Capability in
OPEN messages.

>What purpose does it serve to talk about static capabilities in this
>draft, other than confuse the reader?

This is the draft in which the Dynamic-Capability Capability is
introduced.  Hence, it is the logical place to talk about the
limitations of the Dynamic-Capability Capability.

<snip>

mrr

From: "Tsillas, Jim" <[email-masked]>
To: "'Abarbanel, Benjamin'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: graceful restart TCP reset discovery (was Re>Are dynamic capabili ties really necessary ...)
Date: Thu, 11 Jul 2002 15:20:34 -0400
Sender: [email-masked]

Ben,

You are correct. I was thinking about the case where TCP is
reset due to "fast external fallover". In this case, the
forwarding plane is down and if it stays down for more than
the graceful restart timeout, the receiving speaker resorts to
normal restart.

Sections 6.1 and 6.2 go into good detail on this. The length
of time it takes to determine if TCP has reset is implementation
dependent. One way to detect TCP reset is to look for a new
session for the same peer (since the neighbor will try to
intiate a new session when it restarts). Another way is to rely
on "fast-external-fallover" being turned on. Otherwise you just
have to wait for either the keepalive expiration or the TCP timeout.

There is also an obscure case with some (probably most) TCPs where
the restarting speaker has not yet attempted to open the session but
the receiving speaker attempts to send data on a non-existing session.
This causes the restarting speaker to send a FIN back. There are
probably other equally obscure cases which are implementation dependent.

Once TCP reset is detected, we start the graceful restart timeout
period. This means that in some cases, the total timeout can
also include the amount of time it took to discover the TCP
session has been reset. The draft makes no distinction as far
as how the session was reset, as far as timeout is concerned.

-Jim.

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]]
Sent: Thursday, July 11, 2002 2:43 PM
To: 'Tsillas, Jim'; Abarbanel, Benjamin; 'Yakov Rekhter'; Srihari R.
Sangli
Cc: [email-masked]'
Subject: RE: Are dynamic capabilities really necessary if peers support
gr acef ul restart? 

> graceful restart can also be useful when the control plane
> is implemented separately from the forwarding plane. When
> this is true, the control place can reset (or fail-over)
> while the forwarding plane is unaffected. The reset, can
> be made transparent to the forwarding plane. Graceful
> restart can also serve to limit the scope of the reset to
> just the given speaker and its neighbors. In other words,
> the reset state doesn't propagate through the network.
>

Thanks for the clarification. It would be nice if applicable uses for
gracefull restart were added to the draft. 

> The case which you point out (where TCP is broken) is actually
> not covered since a broken TCP session implies a broken
> forwarding plane.
> 

I disagree, this is taken out of draft-ietf-idr-restart-05.txt, 
Abstract section 2.
                                    
 "Finally,procedures are outlined for temporarily retaining routing
information
  across a TCP transport reset." 

I assume it means we hold routing information after the forwarding plane
is broken (BGP peering session dropped), right?

> -Jim.
> From: Abarbanel, Benjamin [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 1:05 PM
> To: 'Yakov Rekhter'; Srihari R. Sangli
> Cc: Tsillas, Jim; [email-masked]'
> Subject: RE: Are dynamic capabilities really necessary if 
> peers support
> gr acef ul restart? 
> 
> 
> Yakov:
>   The only time I think gracefull restart is usefull is if 
> two peers are
> operating
> fine, but the TCP connection is broken by the network. In the 
> case that
> either peer drops the session or either peer reboot/crash, I 
> dont see much
> value in gracefull restart.
> 
> Did I miss something?
> 
> Ben
> 
> > From: Yakov Rekhter [mailto:[email-masked]]
> > Sent: Thursday, July 11, 2002 12:55 PM
> > To: Srihari R. Sangli
> > Cc: Tsillas, Jim; [email-masked]'
> > Subject: Re: Are dynamic capabilities really necessary if 
> > peers support
> > gracef ul restart? 
> > 
> > 
> > Srihari,
> > 
> > > > It seems we are entering major discussion on dynamic
> > > > capabilities and the can of worms is open. Can anyone
> > > > think of a scenario where a dynamic capabilities
> > > > negotiation is still needed if we have graceful restart?
> > > 
> > > Talking about address-families exchanged between BGP speakers, the
> > > dynamic capability is useful for enabling a new address-family or
> > > disabling a address-family w/o disrupting the session and/or other
> > > address-families. It is also useful for turning ON/OFF 
> > other capabilities
> > > (like ORF, etc)  without much disruption. 
> > 
> > Just to clarify, in the presence of graceful restart the disruption
> > you refer to above is just the disruption in the routing peering
> > between the two speakers. There is no disruption in the 
> > data/forwarding
> > plane and no disruption in any other routing peering. Correct ?
> > 
> > Yakov.
> > 
> 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Srihari R. Sangli'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 14:52:19 -0400
Sender: [email-masked]

Srihari:

see below
 
> 
> 
> Handling the race condition will be clarified in the draft in the next
> revision.
> 

Thanks for this.

> >
> > I have a question on the current 
> draft-ietf-idr-dynamic-cap-02.txt, section
> > 7 - Operation, 3rd and 4th paragraph:
> >
> >     "The Dynamic Capability itself can not be revised 
> dynamically via a
> >      CAPABILITY message. The lifetime of the Dynamic 
> Capability is the
> >      duration of the BGP session in which the capability is 
> advertised.
> 
> This is referring to the "Dynamic-Capability" itself.  This capability
> can ONLY appear in the OPEN message.
> 

I thought this spec was talking about dynamic capabilities not static 
ones, so why are we assuming its the one in the OPEN message?

What purpose does it serve to talk about static capabilities in this
draft, other than confuse the reader?

> >      Upon receiving a CAPABILITY message from its peer, the 
> BGP speaker
> >      shall update the capabilities previously received from 
> that peer
> >      based on the "Action" in the message, and then 
> function in accordance
> >      with the revised capabilities for the peer. Any unrecognized
> >      capabilities in the message should be ignored."
> 
> This is referring to capabilities other than "Dynamic-Capability".
> 
> >
> > Sounds like the second paragraph contradicts the first one.
> 
> No, it does not.

Same comment as state above.

Ben

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'john heasley'" <[email-masked]>, "Srihari R. Sangli" <[email-masked]>
Cc: Stephen Waters <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 14:46:28 -0400
Sender: [email-masked]

Excellent Points, Stephen

Ben

> From: john heasley [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 2:44 PM
> To: Srihari R. Sangli
> Cc: Stephen Waters; Abarbanel, Benjamin; [email-masked]
> Subject: Re: Re>Are dynamic capabilities really necessary if peers
> support gracef ul restart?
> 
> 
> Thu, Jul 11, 2002 at 11:27:44AM -0700, Srihari R. Sangli:
> > 
> > > A network manager wanting to change the state or extent of a
> > > capability would appreciate doing that within a bgp 
> session, wouldn't
> > > they?
> > >
> > 
> > The network manager can turn off other features (or 
> capabilities) and thus
> > communicate such an event to the remote peers via this 
> message. To keep it
> > simple, the draft does not allow turning OFF 
> "Dynamic-capability" via the
> > capability message.
> > 
> > srihari...
> > p.s. If I've misinterpreted your message, I apologise.
> 
> these functions run chills up my spine.  i fear not being 
> able to disable
> various capabilities and auto-negotiation when faced with a 
> business policy
> or buggy or abusive peer/customer (or bugs in my own 
> equipment, requiring
> s/w upgrades - requiring a maintenance window to be scheduled).
> 
> so, not a complaint, but a comment on design and to vendors from an
> operator perspective:
> 
> 	"it must be possible to turn such options off and force
> 	 values/capabilities that will be required."
> 
> eg: if i do not want to accept filters from a peer, then i 
> ought to be able
> disable that functionality.  or, i want nlri unicast and 
> multicast, then i
> ought to be able to force that negotiation, else the session 
> is reset or the
> dynamic re-negotiation is denied.
> 
> the more flexible the configuration knobs the better, imiho.
> 

Date: Thu, 11 Jul 2002 11:44:09 -0700
From: john heasley <[email-masked]>
To: "Srihari R. Sangli" <[email-masked]>
Cc: Stephen Waters <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: Re: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

Thu, Jul 11, 2002 at 11:27:44AM -0700, Srihari R. Sangli:
> 
> > A network manager wanting to change the state or extent of a
> > capability would appreciate doing that within a bgp session, wouldn't
> > they?
> >
> 
> The network manager can turn off other features (or capabilities) and thus
> communicate such an event to the remote peers via this message. To keep it
> simple, the draft does not allow turning OFF "Dynamic-capability" via the
> capability message.
> 
> srihari...
> p.s. If I've misinterpreted your message, I apologise.

these functions run chills up my spine.  i fear not being able to disable
various capabilities and auto-negotiation when faced with a business policy
or buggy or abusive peer/customer (or bugs in my own equipment, requiring
s/w upgrades - requiring a maintenance window to be scheduled).

so, not a complaint, but a comment on design and to vendors from an
operator perspective:

	"it must be possible to turn such options off and force
	 values/capabilities that will be required."

eg: if i do not want to accept filters from a peer, then i ought to be able
disable that functionality.  or, i want nlri unicast and multicast, then i
ought to be able to force that negotiation, else the session is reset or the
dynamic re-negotiation is denied.

the more flexible the configuration knobs the better, imiho.

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Tsillas, Jim'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, "'Yakov Rekhter'" <[email-masked]>, "Srihari R. Sangli" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: Are dynamic capabilities really necessary if peers support gr acef ul restart? 
Date: Thu, 11 Jul 2002 14:43:11 -0400
Sender: [email-masked]

> graceful restart can also be useful when the control plane
> is implemented separately from the forwarding plane. When
> this is true, the control place can reset (or fail-over)
> while the forwarding plane is unaffected. The reset, can
> be made transparent to the forwarding plane. Graceful
> restart can also serve to limit the scope of the reset to
> just the given speaker and its neighbors. In other words,
> the reset state doesn't propagate through the network.
>

Thanks for the clarification. It would be nice if applicable uses for
gracefull restart were added to the draft. 

> The case which you point out (where TCP is broken) is actually
> not covered since a broken TCP session implies a broken
> forwarding plane.
> 

I disagree, this is taken out of draft-ietf-idr-restart-05.txt, 
Abstract section 2.
                                    
 "Finally,procedures are outlined for temporarily retaining routing
information
  across a TCP transport reset." 

I assume it means we hold routing information after the forwarding plane
is broken (BGP peering session dropped), right?

> -Jim.
> From: Abarbanel, Benjamin [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 1:05 PM
> To: 'Yakov Rekhter'; Srihari R. Sangli
> Cc: Tsillas, Jim; [email-masked]'
> Subject: RE: Are dynamic capabilities really necessary if 
> peers support
> gr acef ul restart? 
> 
> 
> Yakov:
>   The only time I think gracefull restart is usefull is if 
> two peers are
> operating
> fine, but the TCP connection is broken by the network. In the 
> case that
> either peer drops the session or either peer reboot/crash, I 
> dont see much
> value in gracefull restart.
> 
> Did I miss something?
> 
> Ben
> 
> > From: Yakov Rekhter [mailto:[email-masked]]
> > Sent: Thursday, July 11, 2002 12:55 PM
> > To: Srihari R. Sangli
> > Cc: Tsillas, Jim; [email-masked]'
> > Subject: Re: Are dynamic capabilities really necessary if 
> > peers support
> > gracef ul restart? 
> > 
> > 
> > Srihari,
> > 
> > > > It seems we are entering major discussion on dynamic
> > > > capabilities and the can of worms is open. Can anyone
> > > > think of a scenario where a dynamic capabilities
> > > > negotiation is still needed if we have graceful restart?
> > > 
> > > Talking about address-families exchanged between BGP speakers, the
> > > dynamic capability is useful for enabling a new address-family or
> > > disabling a address-family w/o disrupting the session and/or other
> > > address-families. It is also useful for turning ON/OFF 
> > other capabilities
> > > (like ORF, etc)  without much disruption. 
> > 
> > Just to clarify, in the presence of graceful restart the disruption
> > you refer to above is just the disruption in the routing peering
> > between the two speakers. There is no disruption in the 
> > data/forwarding
> > plane and no disruption in any other routing peering. Correct ?
> > 
> > Yakov.
> > 
> 

Date: Thu, 11 Jul 2002 11:27:44 -0700 (PDT)
From: "Srihari R. Sangli" <[email-masked]>
To: Stephen Waters <[email-masked]>
Cc: "Abarbanel, Benjamin" <[email-masked]>, <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

> A network manager wanting to change the state or extent of a
> capability would appreciate doing that within a bgp session, wouldn't
> they?
>

The network manager can turn off other features (or capabilities) and thus
communicate such an event to the remote peers via this message. To keep it
simple, the draft does not allow turning OFF "Dynamic-capability" via the
capability message.

srihari...
p.s. If I've misinterpreted your message, I apologise.

From: "Tsillas, Jim" <[email-masked]>
To: "'Srihari R. Sangli'" <[email-masked]>, "Tsillas, Jim" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: Are dynamic capabilities really necessary if peers support gr acef ul restart?
Date: Thu, 11 Jul 2002 14:18:06 -0400
Sender: [email-masked]

I agree we need to see a description for how dynamic
capabilities should work for each capability currently
in use and for each newly introduced capability.

My point on graceful restart was that since it can hide
session resets, couldn't we simply use that to reconfigure
capabilities rather than introduce a new concept. Ben
and Steve's point on being able to turn something on and
off without worrying about affecting the session is well
taken and I now agree with them.

-Jim.

-----Original Message-----
From: Srihari R. Sangli [mailto:[email-masked]]
Sent: Thursday, July 11, 2002 12:43 PM
To: Tsillas, Jim
Cc: [email-masked]'
Subject: Re: Are dynamic capabilities really necessary if peers support
gracef ul restart?

> It seems we are entering major discussion on dynamic
> capabilities and the can of worms is open. Can anyone
> think of a scenario where a dynamic capabilities
> negotiation is still needed if we have graceful restart?

Talking about address-families exchanged between BGP speakers, the
dynamic capability is useful for enabling a new address-family or
disabling a address-family w/o disrupting the session and/or other
address-families. It is also useful for turning ON/OFF other capabilities
(like ORF, etc)  without much disruption. Note that Graceful restart has
its own limitations for control plane changes.

I understand that the draft needs to be clear on the race condition
aspects that was mentioned earlier. Any other issues ?

srihari...

Date: Thu, 11 Jul 2002 11:22:09 -0700 (PDT)
From: "Srihari R. Sangli" <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support  gracef ul restart?
Sender: [email-masked]

> Jim and Enke:
>
> I think the issue with dynamic capabilities is that the current draft is not
> clear on what to do in the case of race conditions with the capability
> messages
> between two routers. The can of worms you speak of will bite us if we do not
> clearify this issue. Each vendor will choose to take a different action
> when this happens, I would prefer that the draft state a way to handle race
> conditions and thus avoid the whole mess of either reseting the session or
> recovering the CAPABILITY state.

Handling the race condition will be clarified in the draft in the next
revision.

>
> I have a question on the current draft-ietf-idr-dynamic-cap-02.txt, section
> 7 - Operation, 3rd and 4th paragraph:
>
>     "The Dynamic Capability itself can not be revised dynamically via a
>      CAPABILITY message. The lifetime of the Dynamic Capability is the
>      duration of the BGP session in which the capability is advertised.

This is referring to the "Dynamic-Capability" itself.  This capability
can ONLY appear in the OPEN message.

>      Upon receiving a CAPABILITY message from its peer, the BGP speaker
>      shall update the capabilities previously received from that peer
>      based on the "Action" in the message, and then function in accordance
>      with the revised capabilities for the peer. Any unrecognized
>      capabilities in the message should be ignored."

This is referring to capabilities other than "Dynamic-Capability".

>
> Sounds like the second paragraph contradicts the first one.

No, it does not.

> So I cannot
> tell if I can renegotiate and replace the CAPABILITY or not during this
> session?

How about now ?

srihari...

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Stephen Waters'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: [email-masked]
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 14:09:32 -0400
Sender: [email-masked]

Absolutely, Agreed.  

Ben

> From: Stephen Waters [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 11:57 AM
> To: Abarbanel, Benjamin
> Cc: [email-masked]
> Subject: RE: Re>Are dynamic capabilities really necessary if peers
> support gracef ul restart?
> 
> 
> 
> I don't see much point in dynamic capability negotiation if 
> you can't turn things on and off,
> (or off and on) to your heart's content. 
> 
> A network manager wanting to change the state or extent of a 
> capability would appreciate doing that within a bgp session, 
> wouldn't they?
> 
> Steve.  
> 
> > From: Abarbanel, Benjamin [mailto:[email-masked]]
> > Sent: Thursday, July 11, 2002 4:15 PM
> > To: 'Tsillas, Jim'; [email-masked]'
> > Cc: [email-masked]'
> > Subject: RE: Re>Are dynamic capabilities really necessary if peers
> > support gracef ul restart?
> > 
> > 
> > Jim and Enke:
> > 
> > I think the issue with dynamic capabilities is that the 
> > current draft is not
> > clear on what to do in the case of race conditions with the 
> capability
> > messages 
> > between two routers. The can of worms you speak of will bite 
> > us if we do not
> > clearify this issue. Each vendor will choose to take a 
> > different action 
> > when this happens, I would prefer that the draft state a way 
> > to handle race
> > conditions and thus avoid the whole mess of either reseting 
> > the session or
> > recovering the CAPABILITY state.
> > 
> > I have a question on the current 
> > draft-ietf-idr-dynamic-cap-02.txt, section
> > 7 - Operation, 3rd and 4th paragraph:
> > 
> >     "The Dynamic Capability itself can not be revised 
> > dynamically via a
> >      CAPABILITY message. The lifetime of the Dynamic 
> Capability is the
> >      duration of the BGP session in which the capability is 
> > advertised.
> > 
> >      Upon receiving a CAPABILITY message from its peer, the 
> > BGP speaker
> >      shall update the capabilities previously received from 
> that peer
> >      based on the "Action" in the message, and then function 
> > in accordance
> >      with the revised capabilities for the peer. Any unrecognized
> >      capabilities in the message should be ignored."
> > 
> > Sounds like the second paragraph contradicts the first one. 
> > So I cannot 
> > tell if I can renegotiate and replace the CAPABILITY or not 
> > during this
> > session?
> > 
> > Ben
> > 
> > > From: Tsillas, Jim [mailto:[email-masked]]
> > > Sent: Thursday, July 11, 2002 9:52 AM
> > > To: [email-masked]'
> > > Subject: FW: Re>Are dynamic capabilities really necessary if peers
> > > support gracef ul restart?
> > > 
> > > 
> > > 
> > > Bruno,
> > > 
> > > if graceful restart is implemented for all supported
> > > address families and can be manually initiated it
> > > seems to me that much of the functionality of capabilities
> > > negotiation can be achieved by gracefully restarting the
> > > session. I'm trying to think of a scenario where this
> > > won't work. Granted that bouncing the session, even with
> > > graceful restart, will cause large numbers of messages
> > > to be sent between peers (which is something to avoid).
> > > 
> > > By can of worms I am implying unknown race conditions
> > > which can result by renegotiating certain capabilities.
> > > For some capabilities it may be harder than others to
> > > identify and handle all the race conditions.
> > > 
> > > -Jim.
> > > 
> > > From: Bruno Rijsman [mailto:[email-masked]]
> > > Sent: Thursday, July 11, 2002 8:30 AM
> > > To: [email-masked]
> > > Cc: [email-masked]
> > > Subject: Re>Are dynamic capabilities really necessary if 
> > peers support
> > > gracef ul restart?
> > > 
> > > 
> > > Re> It seems we are entering major discussion on dynamic
> > > Re> capabilities and the can of worms is open. Can anyone
> > > Re> think of a scenario where a dynamic capabilities
> > > Re> negotiation is still needed if we have graceful restart?
> > > 
> > > One example is: it allows you to activate or de-activate an
> > > address family (e.g. ipv6 or vpnv4) on a session which is
> > > already established and exchanging some other address family
> > > (e.g. ipv4) without having to bounce that session.
> > > 
> > > More generally speaking, it allows you to activate or
> > > de-activate features on a session without needing to bounce the
> > > session.
> > > 
> > > Not needing to bounce a session a good thing.
> > > 
> > > It seems from your e-mail that you are implying that 
> > graceful restart
> > > removes the need or reduces the usefulness of dynamic capability
> > > negotiation. Could you please clarify that?
> > > 
> > > Having personally implemented dynamic capability negotiation
> > > I can tell you that the "can of worms" is not so bad: all you
> > > have to do is to be willing to accept and silently ignore certain
> > > messages after withdrawing a capability (e.g. you have to 
> be willing
> > > to accept and ignore vpnv4 updates if you recently withdrew the
> > > vpnv4 multi-protocol capability).
> > > 
> > > 
> > > 
> > > _________________________________________________________________
> > > Send and receive Hotmail on your mobile device: 
> http://mobile.msn.com
> > 
> 

From: "Tsillas, Jim" <[email-masked]>
To: "'Abarbanel, Benjamin'" <[email-masked]>, "'Yakov Rekhter'" <[email-masked]>, "Srihari R. Sangli" <[email-masked]>
Cc: "Tsillas, Jim" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Are dynamic capabilities really necessary if peers support gr acef ul restart? 
Date: Thu, 11 Jul 2002 14:02:57 -0400
Sender: [email-masked]

Ben,

graceful restart can also be useful when the control plane
is implemented separately from the forwarding plane. When
this is true, the control place can reset (or fail-over)
while the forwarding plane is unaffected. The reset, can
be made transparent to the forwarding plane. Graceful
restart can also serve to limit the scope of the reset to
just the given speaker and its neighbors. In other words,
the reset state doesn't propagate through the network.

The case which you point out (where TCP is broken) is actually
not covered since a broken TCP session implies a broken
forwarding plane.

-Jim.

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]]
Sent: Thursday, July 11, 2002 1:05 PM
To: 'Yakov Rekhter'; Srihari R. Sangli
Cc: Tsillas, Jim; [email-masked]'
Subject: RE: Are dynamic capabilities really necessary if peers support
gr acef ul restart? 

Yakov:
  The only time I think gracefull restart is usefull is if two peers are
operating
fine, but the TCP connection is broken by the network. In the case that
either peer drops the session or either peer reboot/crash, I dont see much
value in gracefull restart.

Did I miss something?

Ben

> From: Yakov Rekhter [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 12:55 PM
> To: Srihari R. Sangli
> Cc: Tsillas, Jim; [email-masked]'
> Subject: Re: Are dynamic capabilities really necessary if 
> peers support
> gracef ul restart? 
> 
> 
> Srihari,
> 
> > > It seems we are entering major discussion on dynamic
> > > capabilities and the can of worms is open. Can anyone
> > > think of a scenario where a dynamic capabilities
> > > negotiation is still needed if we have graceful restart?
> > 
> > Talking about address-families exchanged between BGP speakers, the
> > dynamic capability is useful for enabling a new address-family or
> > disabling a address-family w/o disrupting the session and/or other
> > address-families. It is also useful for turning ON/OFF 
> other capabilities
> > (like ORF, etc)  without much disruption. 
> 
> Just to clarify, in the presence of graceful restart the disruption
> you refer to above is just the disruption in the routing peering
> between the two speakers. There is no disruption in the 
> data/forwarding
> plane and no disruption in any other routing peering. Correct ?
> 
> Yakov.
> 

Date: Thu, 11 Jul 2002 10:55:20 -0700 (PDT)
From: "Srihari R. Sangli" <[email-masked]>
To: Yakov Rekhter <[email-masked]>
Cc: "Tsillas, Jim" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: Re: Are dynamic capabilities really necessary if peers support gracef ul restart? 
Sender: [email-masked]

Yakov,

> Just to clarify, in the presence of graceful restart the disruption
> you refer to above is just the disruption in the routing peering
> between the two speakers. There is no disruption in the data/forwarding
> plane and no disruption in any other routing peering. Correct ?

I was referring to the control plane changes after the BGP speaker restart
event until the point its up and has relearned them.

srihari...

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Yakov Rekhter'" <[email-masked]>, "Srihari R. Sangli" <[email-masked]>
Cc: "Tsillas, Jim" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: Are dynamic capabilities really necessary if peers support gr acef ul restart? 
Date: Thu, 11 Jul 2002 13:05:20 -0400
Sender: [email-masked]

Yakov:
  The only time I think gracefull restart is usefull is if two peers are
operating
fine, but the TCP connection is broken by the network. In the case that
either peer drops the session or either peer reboot/crash, I dont see much
value in gracefull restart.

Did I miss something?

Ben

> From: Yakov Rekhter [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 12:55 PM
> To: Srihari R. Sangli
> Cc: Tsillas, Jim; [email-masked]'
> Subject: Re: Are dynamic capabilities really necessary if 
> peers support
> gracef ul restart? 
> 
> 
> Srihari,
> 
> > > It seems we are entering major discussion on dynamic
> > > capabilities and the can of worms is open. Can anyone
> > > think of a scenario where a dynamic capabilities
> > > negotiation is still needed if we have graceful restart?
> > 
> > Talking about address-families exchanged between BGP speakers, the
> > dynamic capability is useful for enabling a new address-family or
> > disabling a address-family w/o disrupting the session and/or other
> > address-families. It is also useful for turning ON/OFF 
> other capabilities
> > (like ORF, etc)  without much disruption. 
> 
> Just to clarify, in the presence of graceful restart the disruption
> you refer to above is just the disruption in the routing peering
> between the two speakers. There is no disruption in the 
> data/forwarding
> plane and no disruption in any other routing peering. Correct ?
> 
> Yakov.
> 

To: "Srihari R. Sangli" <[email-masked]>
Cc: "Tsillas, Jim" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: Re: Are dynamic capabilities really necessary if peers support gracef ul restart? 
Date: Thu, 11 Jul 2002 09:55:16 -0700
From: Yakov Rekhter <[email-masked]>
Sender: [email-masked]

Srihari,

> > It seems we are entering major discussion on dynamic
> > capabilities and the can of worms is open. Can anyone
> > think of a scenario where a dynamic capabilities
> > negotiation is still needed if we have graceful restart?
> 
> Talking about address-families exchanged between BGP speakers, the
> dynamic capability is useful for enabling a new address-family or
> disabling a address-family w/o disrupting the session and/or other
> address-families. It is also useful for turning ON/OFF other capabilities
> (like ORF, etc)  without much disruption. 

Just to clarify, in the presence of graceful restart the disruption
you refer to above is just the disruption in the routing peering
between the two speakers. There is no disruption in the data/forwarding
plane and no disruption in any other routing peering. Correct ?

Yakov.

Date: Thu, 11 Jul 2002 09:43:21 -0700 (PDT)
From: "Srihari R. Sangli" <[email-masked]>
To: "Tsillas, Jim" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: Are dynamic capabilities really necessary if peers support gracef ul restart?
Sender: [email-masked]

> It seems we are entering major discussion on dynamic
> capabilities and the can of worms is open. Can anyone
> think of a scenario where a dynamic capabilities
> negotiation is still needed if we have graceful restart?

Talking about address-families exchanged between BGP speakers, the
dynamic capability is useful for enabling a new address-family or
disabling a address-family w/o disrupting the session and/or other
address-families. It is also useful for turning ON/OFF other capabilities
(like ORF, etc)  without much disruption. Note that Graceful restart has
its own limitations for control plane changes.

I understand that the draft needs to be clear on the race condition
aspects that was mentioned earlier. Any other issues ?

srihari...

content-class: urn:content-classes:message
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 16:56:44 +0100
From: "Stephen Waters" <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: <[email-masked]>
Sender: [email-masked]

I don't see much point in dynamic capability negotiation if you can't turn things on and off,
(or off and on) to your heart's content. 

A network manager wanting to change the state or extent of a capability would appreciate doing that within a bgp session, wouldn't they?

Steve.  

> From: Abarbanel, Benjamin [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 4:15 PM
> To: 'Tsillas, Jim'; [email-masked]'
> Cc: [email-masked]'
> Subject: RE: Re>Are dynamic capabilities really necessary if peers
> support gracef ul restart?
> 
> 
> Jim and Enke:
> 
> I think the issue with dynamic capabilities is that the 
> current draft is not
> clear on what to do in the case of race conditions with the capability
> messages 
> between two routers. The can of worms you speak of will bite 
> us if we do not
> clearify this issue. Each vendor will choose to take a 
> different action 
> when this happens, I would prefer that the draft state a way 
> to handle race
> conditions and thus avoid the whole mess of either reseting 
> the session or
> recovering the CAPABILITY state.
> 
> I have a question on the current 
> draft-ietf-idr-dynamic-cap-02.txt, section
> 7 - Operation, 3rd and 4th paragraph:
> 
>     "The Dynamic Capability itself can not be revised 
> dynamically via a
>      CAPABILITY message. The lifetime of the Dynamic Capability is the
>      duration of the BGP session in which the capability is 
> advertised.
> 
>      Upon receiving a CAPABILITY message from its peer, the 
> BGP speaker
>      shall update the capabilities previously received from that peer
>      based on the "Action" in the message, and then function 
> in accordance
>      with the revised capabilities for the peer. Any unrecognized
>      capabilities in the message should be ignored."
> 
> Sounds like the second paragraph contradicts the first one. 
> So I cannot 
> tell if I can renegotiate and replace the CAPABILITY or not 
> during this
> session?
> 
> Ben
> 
> > From: Tsillas, Jim [mailto:[email-masked]]
> > Sent: Thursday, July 11, 2002 9:52 AM
> > To: [email-masked]'
> > Subject: FW: Re>Are dynamic capabilities really necessary if peers
> > support gracef ul restart?
> > 
> > 
> > 
> > Bruno,
> > 
> > if graceful restart is implemented for all supported
> > address families and can be manually initiated it
> > seems to me that much of the functionality of capabilities
> > negotiation can be achieved by gracefully restarting the
> > session. I'm trying to think of a scenario where this
> > won't work. Granted that bouncing the session, even with
> > graceful restart, will cause large numbers of messages
> > to be sent between peers (which is something to avoid).
> > 
> > By can of worms I am implying unknown race conditions
> > which can result by renegotiating certain capabilities.
> > For some capabilities it may be harder than others to
> > identify and handle all the race conditions.
> > 
> > -Jim.
> > 
> > From: Bruno Rijsman [mailto:[email-masked]]
> > Sent: Thursday, July 11, 2002 8:30 AM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: Re>Are dynamic capabilities really necessary if 
> peers support
> > gracef ul restart?
> > 
> > 
> > Re> It seems we are entering major discussion on dynamic
> > Re> capabilities and the can of worms is open. Can anyone
> > Re> think of a scenario where a dynamic capabilities
> > Re> negotiation is still needed if we have graceful restart?
> > 
> > One example is: it allows you to activate or de-activate an
> > address family (e.g. ipv6 or vpnv4) on a session which is
> > already established and exchanging some other address family
> > (e.g. ipv4) without having to bounce that session.
> > 
> > More generally speaking, it allows you to activate or
> > de-activate features on a session without needing to bounce the
> > session.
> > 
> > Not needing to bounce a session a good thing.
> > 
> > It seems from your e-mail that you are implying that 
> graceful restart
> > removes the need or reduces the usefulness of dynamic capability
> > negotiation. Could you please clarify that?
> > 
> > Having personally implemented dynamic capability negotiation
> > I can tell you that the "can of worms" is not so bad: all you
> > have to do is to be willing to accept and silently ignore certain
> > messages after withdrawing a capability (e.g. you have to be willing
> > to accept and ignore vpnv4 updates if you recently withdrew the
> > vpnv4 multi-protocol capability).
> > 
> > 
> > 
> > _________________________________________________________________
> > Send and receive Hotmail on your mobile device: 
http://mobile.msn.com

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Tsillas, Jim'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 11:15:15 -0400
Sender: [email-masked]

Jim and Enke:

I think the issue with dynamic capabilities is that the current draft is not
clear on what to do in the case of race conditions with the capability
messages 
between two routers. The can of worms you speak of will bite us if we do not
clearify this issue. Each vendor will choose to take a different action 
when this happens, I would prefer that the draft state a way to handle race
conditions and thus avoid the whole mess of either reseting the session or
recovering the CAPABILITY state.

I have a question on the current draft-ietf-idr-dynamic-cap-02.txt, section
7 - Operation, 3rd and 4th paragraph:

    "The Dynamic Capability itself can not be revised dynamically via a
     CAPABILITY message. The lifetime of the Dynamic Capability is the
     duration of the BGP session in which the capability is advertised.

     Upon receiving a CAPABILITY message from its peer, the BGP speaker
     shall update the capabilities previously received from that peer
     based on the "Action" in the message, and then function in accordance
     with the revised capabilities for the peer. Any unrecognized
     capabilities in the message should be ignored."

Sounds like the second paragraph contradicts the first one. So I cannot 
tell if I can renegotiate and replace the CAPABILITY or not during this
session?

Ben

> From: Tsillas, Jim [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 9:52 AM
> To: [email-masked]'
> Subject: FW: Re>Are dynamic capabilities really necessary if peers
> support gracef ul restart?
> 
> 
> 
> Bruno,
> 
> if graceful restart is implemented for all supported
> address families and can be manually initiated it
> seems to me that much of the functionality of capabilities
> negotiation can be achieved by gracefully restarting the
> session. I'm trying to think of a scenario where this
> won't work. Granted that bouncing the session, even with
> graceful restart, will cause large numbers of messages
> to be sent between peers (which is something to avoid).
> 
> By can of worms I am implying unknown race conditions
> which can result by renegotiating certain capabilities.
> For some capabilities it may be harder than others to
> identify and handle all the race conditions.
> 
> -Jim.
> 
> From: Bruno Rijsman [mailto:[email-masked]]
> Sent: Thursday, July 11, 2002 8:30 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re>Are dynamic capabilities really necessary if peers support
> gracef ul restart?
> 
> 
> Re> It seems we are entering major discussion on dynamic
> Re> capabilities and the can of worms is open. Can anyone
> Re> think of a scenario where a dynamic capabilities
> Re> negotiation is still needed if we have graceful restart?
> 
> One example is: it allows you to activate or de-activate an
> address family (e.g. ipv6 or vpnv4) on a session which is
> already established and exchanging some other address family
> (e.g. ipv4) without having to bounce that session.
> 
> More generally speaking, it allows you to activate or
> de-activate features on a session without needing to bounce the
> session.
> 
> Not needing to bounce a session a good thing.
> 
> It seems from your e-mail that you are implying that graceful restart
> removes the need or reduces the usefulness of dynamic capability
> negotiation. Could you please clarify that?
> 
> Having personally implemented dynamic capability negotiation
> I can tell you that the "can of worms" is not so bad: all you
> have to do is to be willing to accept and silently ignore certain
> messages after withdrawing a capability (e.g. you have to be willing
> to accept and ignore vpnv4 updates if you recently withdrew the
> vpnv4 multi-protocol capability).
> 
> 
> 
> _________________________________________________________________
> Send and receive Hotmail on your mobile device: http://mobile.msn.com
> 

Date: Thu, 11 Jul 2002 07:47:09 -0700
From: Kunihiro Ishiguro <[email-masked]>
To: "Bruno Rijsman" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Aggregation of the overlapping routes and ATOMIC_AGGREGATE
Sender: [email-masked]

>Re> Consider scenario:
>Re> BGP Router is configured to aggregate routes 192.0.0.0/8
>Re> This router receives 2 NLRIs: 192.0.0.0/8 and 192.1.0.0/16
>Re> Received NLRIs are installed in RIB, but are suppressed by
>Re> aggregation and are not advertised. As a result of aggregation
>Re> the router sends update that contains only one NLRI: 192.0.0.0/8
>Re> Does this update should contain ATOMIC_AGGREGATE attached?
>
>I my experience this is what the major deployed implementations do:
>
>1) If the aggregate is an "as-set" aggregate (i.e. the as-path and
>   other path attributes of the aggregate are constructed by
>   "summarizing" the attributes of the covered routes) then the
>   ATOMIC_AGGREGATE attribute is not attached.
>
>2) If the aggregate is not an "as-set" aggregate (i.e. the as-path
>   and other path attributes are constructed as if the aggregate
>   route is a locally originated route) then the ATOMIC_AGGREGATE
>   attribute is attached.

Adding to that, when "as-set" is not specified, created aggregated
route has origin IGP.  For example RFC1771 and the latest draft said

      ORIGIN attribute: If at least one route among routes that are
      aggregated has ORIGIN with the value INCOMPLETE, then the
      aggregated route must have the ORIGIN attribute with the value
      INCOMPLETE. Otherwise, if at least one route among routes that are
      aggregated has ORIGIN with the value EGP, then the aggregated
      route must have the origin attribute with the value EGP. In all
      other case the value of the ORIGIN attribute of the aggregated
      route is INTERNAL.

When "as-set" is specifed above rule is applied but when "as-set" is
not specified, the aggregate route always has orgin IGP like self
originated route.  This is little bit implementation specific but some
people may consider.
-- 
Kunihiro Ishiguro

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Chandrashekhar Appanna'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Thu, 11 Jul 2002 10:28:42 -0400
Sender: [email-masked]

> 
> You can be tolerant in your implementation today if you choose
> and if your customers will accept it. I do not see how you hope
> that a new INFORM message will let an implementation correct itself 
> when the implementation failed to do the right thing in the first 
> place.
> 

The case I am refering to is the "ships in the night" condition where the
messages cross on the wire, thus causing improper processing. With the use
of the INFORM like message, the two routers can get synchronized on what 
capabilities to honor without a session restart. Thus preserving the state
of all their routes.

Ben

> -
> Chandra A.
> 
> 
> On Wed, Jul 10, 2002 at 02:49:12PM -0400, Abarbanel, Benjamin wrote:
> > I am in favour of providing a tolerance counter that will 
> drop the session
> > only after a max number of bad update messages are 
> received. Thus if the
> > offending
> > peer does not correct or adjust itself in terms of 
> capabilities, after a
> > number
> > of INFORM responses are sent to it, then and only then 
> should the receiving
> > router drop the session. This will cover the case, where 
> the INFORM is also
> > mis-processed in the "ships in the night" scenario.
> > 
> > Ben
> > 
> > 
> > > From: Chandrashekhar Appanna [mailto:[email-masked]]
> > > Sent: Wednesday, July 10, 2002 2:09 PM
> > > To: Abarbanel, Benjamin
> > > Cc: 'Bruno Rijsman'; [email-masked]
> > > Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> > > 
> > > 
> > > 
> > > But that and the ability to INFORM is not the same. In the
> > > example you provide, the INFORM is not providing any extra 
> > > advantage. If the peer did not process/rcv the withdraw 
> > > why do you hope that it will rcv/process the INFORM?
> > > 
> > > I think the behavior of the withdraw itself is the problem here,
> > > It has to handle the case of a message cross that you describe, 
> > > in a well defined manner. 
> > > 
> > > -
> > > Chandra A.
> > > 
> > > On Wed, Jul 10, 2002 at 11:50:32AM -0400, Abarbanel, 
> Benjamin wrote:
> > > > Correct. The protocol has to be elastic, since it carries 
> > > so much weight in 
> > > > controlling stability in the Internet.
> > > > 
> > > > Ben
> > > > 
> > > > > From: Bruno Rijsman [mailto:[email-masked]]
> > > > > Sent: Wednesday, July 10, 2002 11:47 AM
> > > > > To: [email-masked]
> > > > > Cc: [email-masked]
> > > > > Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> > > > > 
> > > > > 
> > > > > Re> Just because you receive an Update message with 
> > > capabilities that
> > > > > Re> were not negotiated at the OPEN message, should not 
> > > require the
> > > > > Re> peering session to terminate. The affect of all that 
> > > peer routes
> > > > > Re> going down will destabilize the network.
> > > > > 
> > > > > Also, keep in mind that in the case of dynamic capability 
> > > negotiation
> > > > > is possible and perfectly legal to receive a message 
> > > which relies on
> > > > > a capability which was recently withdrawn. Consider for 
> > > example the
> > > > > case where a route-refresh message and a capability 
> message which
> > > > > withdraws the route-refresh capability cross each other.
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> _________________________________________________________________
> > > > > Send and receive Hotmail on your mobile device: 
> > http://mobile.msn.com
> > > > 
> 

From: "Tsillas, Jim" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: FW: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 09:52:27 -0400
Sender: [email-masked]

Bruno,

if graceful restart is implemented for all supported
address families and can be manually initiated it
seems to me that much of the functionality of capabilities
negotiation can be achieved by gracefully restarting the
session. I'm trying to think of a scenario where this
won't work. Granted that bouncing the session, even with
graceful restart, will cause large numbers of messages
to be sent between peers (which is something to avoid).

By can of worms I am implying unknown race conditions
which can result by renegotiating certain capabilities.
For some capabilities it may be harder than others to
identify and handle all the race conditions.

-Jim.

-----Original Message-----
From: Bruno Rijsman [mailto:[email-masked]]
Sent: Thursday, July 11, 2002 8:30 AM
To: [email-masked]
Cc: [email-masked]
Subject: Re>Are dynamic capabilities really necessary if peers support
gracef ul restart?

Re> It seems we are entering major discussion on dynamic
Re> capabilities and the can of worms is open. Can anyone
Re> think of a scenario where a dynamic capabilities
Re> negotiation is still needed if we have graceful restart?

One example is: it allows you to activate or de-activate an
address family (e.g. ipv6 or vpnv4) on a session which is
already established and exchanging some other address family
(e.g. ipv4) without having to bounce that session.

More generally speaking, it allows you to activate or
de-activate features on a session without needing to bounce the
session.

Not needing to bounce a session a good thing.

It seems from your e-mail that you are implying that graceful restart
removes the need or reduces the usefulness of dynamic capability
negotiation. Could you please clarify that?

Having personally implemented dynamic capability negotiation
I can tell you that the "can of worms" is not so bad: all you
have to do is to be willing to accept and silently ignore certain
messages after withdrawing a capability (e.g. you have to be willing
to accept and ignore vpnv4 updates if you recently withdrew the
vpnv4 multi-protocol capability).

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com

From: "Bruno Rijsman" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re>Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 08:30:05 -0400
Sender: [email-masked]

Re> It seems we are entering major discussion on dynamic
Re> capabilities and the can of worms is open. Can anyone
Re> think of a scenario where a dynamic capabilities
Re> negotiation is still needed if we have graceful restart?

One example is: it allows you to activate or de-activate an
address family (e.g. ipv6 or vpnv4) on a session which is
already established and exchanging some other address family
(e.g. ipv4) without having to bounce that session.

More generally speaking, it allows you to activate or
de-activate features on a session without needing to bounce the
session.

Not needing to bounce a session a good thing.

It seems from your e-mail that you are implying that graceful restart
removes the need or reduces the usefulness of dynamic capability
negotiation. Could you please clarify that?

Having personally implemented dynamic capability negotiation
I can tell you that the "can of worms" is not so bad: all you
have to do is to be willing to accept and silently ignore certain
messages after withdrawing a capability (e.g. you have to be willing
to accept and ignore vpnv4 updates if you recently withdrew the
vpnv4 multi-protocol capability).

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com

From: "Tsillas, Jim" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: Are dynamic capabilities really necessary if peers support gracef ul restart?
Date: Thu, 11 Jul 2002 08:16:02 -0400
Sender: [email-masked]

It seems we are entering major discussion on dynamic
capabilities and the can of worms is open. Can anyone
think of a scenario where a dynamic capabilities
negotiation is still needed if we have graceful restart?

thanks,
-Jim.

From: "Bruno Rijsman" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Aggregation of the overlapping routes and ATOMIC_AGGREGATE
Date: Thu, 11 Jul 2002 06:47:26 -0400
Sender: [email-masked]

Re> Consider scenario:
Re> BGP Router is configured to aggregate routes 192.0.0.0/8
Re> This router receives 2 NLRIs: 192.0.0.0/8 and 192.1.0.0/16
Re> Received NLRIs are installed in RIB, but are suppressed by
Re> aggregation and are not advertised. As a result of aggregation
Re> the router sends update that contains only one NLRI: 192.0.0.0/8
Re> Does this update should contain ATOMIC_AGGREGATE attached?

I my experience this is what the major deployed implementations do:

1) If the aggregate is an "as-set" aggregate (i.e. the as-path and
   other path attributes of the aggregate are constructed by
   "summarizing" the attributes of the covered routes) then the
   ATOMIC_AGGREGATE attribute is not attached.

2) If the aggregate is not an "as-set" aggregate (i.e. the as-path
   and other path attributes are constructed as if the aggregate
   route is a locally originated route) then the ATOMIC_AGGREGATE
   attribute is attached.

I have seen nothing in RFC1771 or the current draft that describes
this behavior but -as far as I know- that is what happens in real
life.

_________________________________________________________________
Chat with friends online, try MSN Messenger: http://messenger.msn.com

From: "Fryczke, Pawel" <[email-masked]>
To: [email-masked]
Subject: Aggregation of the overlapping routes and ATOMIC_AGGREGATE
Date: Thu, 11 Jul 2002 12:32:14 +0200
Sender: [email-masked]

Latest BGP draft states:
"If a BGP speaker receives overlapping routes, the Decision Process
MUST consider both routes based on the configured acceptance policy.
If both a less and a more specific route are accepted, then the
Decision Process MUST either install both the less and the more
specific routes or it MUST aggregate the two routes and install the
aggregated route, provided that both routes have the same value of
the NEXT_HOP attribute.
If a BGP speaker chooses to aggregate, then it MUST add
ATOMIC_AGGREGATE attribute to the route."

What does happen if BGP router installs both routes in RIB and next
aggregates them ?
Should ATOMIC_AGGREGATE be added to aggregate or not ?

Consider scenario:
BGP Router is configured to aggregate routes 192.0.0.0/8
This router receives 2 NLRIs: 192.0.0.0/8 and 192.1.0.0/16
Received NLRIs are installed in RIB, but are suppressed by aggregation and
are not advertised.
As a result of aggregation the router sends update that contains only one
NLRI: 192.0.0.0/8
Does this update should contain ATOMIC_AGGREGATE attached ?

Why does RFC assume that aggregation is done during decision process?
It is possible to aggregate during update process or some other process,
isn't it?

Pawel

From: "Natale, Jonathan" <[email-masked]>
To: "'Gargi Nalawade'" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 18:31:36 -0400
Sender: [email-masked]

Good example.

I had also suggested a new NOTIFICATION for the case when it IS configured
to kill the con:

-----Original Message-----
From: Natale, Jonathan 
Sent: Monday, April 29, 2002 4:13 PM
To: 'Russ White'; Pedro Roque Marques
Cc: [email-masked]
Subject: RE: BGP-17 Draft Comments...

I would like to suggest another UPDATE Message Error subcode be introduced
~= "Maximum Number of Allowed Prefixes Exceeded" (if this has been done,
then please direct me to where it is documented).  Currently ( 12.1(5)T4 ),
a "Malformed Attribute List" notification is sent when the user configured
maximum number of allowed prefixes is exceeded.  Alternatively, this current
(or the "correct" behavior) should be specified in the draft-ietf-idr-bgp4
draft which recently added:

   "A BGP speaker may support the ability to impose an (locally
   configured) upper bound on the number of address prefixes the speaker
   is willing to accept from a neighbor. When the upper bound is
   reached, the speaker (under control of local configuration) may
   either (a) discard new address prefixes from the neighbor, or (b)
   terminate the BGP peering with the neighbor. If the BGP speaker
   decides to terminate its peering with a neighbor because the number
   of address prefixes received from the neighbor exceeds the locally
   configured upper bound, then the speaker must send to the neighbor a
   NOTIFICATION message with the Error Code Cease."

-$0.02

From: Gargi Nalawade [mailto:[email-masked]] 
Sent: Wednesday, July 10, 2002 5:12 PM
To: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)

Hi All,

A few clarifications about the INFORM message & its motivations. This is
supposed to be purely an informational message. Does not propose to affect
the functionality of the protocol. Rather, it proposes to give flexibility
to implementations in that; they dont need to reset the sessions, just so
they can inform the peer of a specific noteworthy event happening. 

Take the example of max-prefix limit. If the limit is hit, the receiving
peer would just generate a local warning (if so configured) and continue
with the session. There is no mechanism of informing the other peer that
it has exceeded the prefix-limit value, other than churning through the log,
noticing the warnin & then picking up the telephone (or oops - writing an
email) & informing the peer's administrator.

This is just one such hypothetical example where the INFORM would be useful.
But the INFORM message gives us the capability to do just that. There are 
several other such scenarios seen in deployment, that necessitate the
existence of just such a mechanism.

-Gargi

From: "Natale, Jonathan" <[email-masked]>
To: "'Abarbanel, Benjamin'" <[email-masked]>, "'Gargi Nalawade'" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 18:34:06 -0400
Sender: [email-masked]

Rxing mcast nlri, when only doing unicast
The draft added this clarification--"ignore them" (seems like a job for the
INFORM???)

-$0.03

-----Original Message-----
From: Abarbanel, Benjamin [mailto:[email-masked]] 
Sent: Wednesday, July 10, 2002 5:39 PM
To: 'Gargi Nalawade'; [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)

Thanks for the clarification on the original intent.

Clearly I think there are many cases, were we want to send potential
operational affecting information to a neighboring peer without reseting
the session. INFORM type message could provide that communication.

I can see the following type of data it would convey:
  1. CPU Utilization levels in an effort to slow down the rate of update.
  2. Memory utilization in an effort to control the amount of routing
     information sent, possibly forbid any optional capabilities while
     router is in a memory depleted state.
  3. Reaching maximum limits on configured or defaulted resources. Number
     of prefixes, attributes, capabilities, etc.
  4. Protocol related WARNING conditions that do not cause a session 
     reset. Repair race conditions, identify FSM state, etc.  

Thanks,
Ben 

> From: Gargi Nalawade [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 5:12 PM
> To: [email-masked]
> Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Hi All,
> 
> A few clarifications about the INFORM message & its 
> motivations. This is
> supposed to be purely an informational message. Does not 
> propose to affect
> the functionality of the protocol. Rather, it proposes to 
> give flexibility
> to implementations in that; they dont need to reset the 
> sessions, just so
> they can inform the peer of a specific noteworthy event happening. 
> 
> Take the example of max-prefix limit. If the limit is hit, 
> the receiving
> peer would just generate a local warning (if so configured) 
> and continue
> with the session. There is no mechanism of informing the 
> other peer that
> it has exceeded the prefix-limit value, other than churning 
> through the log,
> noticing the warnin & then picking up the telephone (or oops 
> - writing an
> email) & informing the peer's administrator.
> 
> This is just one such hypothetical example where the INFORM 
> would be useful.
> But the INFORM message gives us the capability to do just 
> that. There are 
> several other such scenarios seen in deployment, that necessitate the
> existence of just such a mechanism.
> 
> -Gargi
> 

Date: Wed, 10 Jul 2002 15:21:20 -0700
From: Chandrashekhar Appanna <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Chandrashekhar Appanna'" <[email-masked]>, "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

You can be tolerant in your implementation today if you choose
and if your customers will accept it. I do not see how you hope
that a new INFORM message will let an implementation correct itself 
when the implementation failed to do the right thing in the first 
place.

-
Chandra A.

On Wed, Jul 10, 2002 at 02:49:12PM -0400, Abarbanel, Benjamin wrote:
> I am in favour of providing a tolerance counter that will drop the session
> only after a max number of bad update messages are received. Thus if the
> offending
> peer does not correct or adjust itself in terms of capabilities, after a
> number
> of INFORM responses are sent to it, then and only then should the receiving
> router drop the session. This will cover the case, where the INFORM is also
> mis-processed in the "ships in the night" scenario.
> 
> Ben
> 
> 
> > From: Chandrashekhar Appanna [mailto:[email-masked]]
> > Sent: Wednesday, July 10, 2002 2:09 PM
> > To: Abarbanel, Benjamin
> > Cc: 'Bruno Rijsman'; [email-masked]
> > Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> > 
> > 
> > 
> > But that and the ability to INFORM is not the same. In the
> > example you provide, the INFORM is not providing any extra 
> > advantage. If the peer did not process/rcv the withdraw 
> > why do you hope that it will rcv/process the INFORM?
> > 
> > I think the behavior of the withdraw itself is the problem here,
> > It has to handle the case of a message cross that you describe, 
> > in a well defined manner. 
> > 
> > -
> > Chandra A.
> > 
> > On Wed, Jul 10, 2002 at 11:50:32AM -0400, Abarbanel, Benjamin wrote:
> > > Correct. The protocol has to be elastic, since it carries 
> > so much weight in 
> > > controlling stability in the Internet.
> > > 
> > > Ben
> > > 
> > > > From: Bruno Rijsman [mailto:[email-masked]]
> > > > Sent: Wednesday, July 10, 2002 11:47 AM
> > > > To: [email-masked]
> > > > Cc: [email-masked]
> > > > Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> > > > 
> > > > 
> > > > Re> Just because you receive an Update message with 
> > capabilities that
> > > > Re> were not negotiated at the OPEN message, should not 
> > require the
> > > > Re> peering session to terminate. The affect of all that 
> > peer routes
> > > > Re> going down will destabilize the network.
> > > > 
> > > > Also, keep in mind that in the case of dynamic capability 
> > negotiation
> > > > is possible and perfectly legal to receive a message 
> > which relies on
> > > > a capability which was recently withdrawn. Consider for 
> > example the
> > > > case where a route-refresh message and a capability message which
> > > > withdraws the route-refresh capability cross each other.
> > > > 
> > > > 
> > > > 
> > > > _________________________________________________________________
> > > > Send and receive Hotmail on your mobile device: 
> http://mobile.msn.com
> > > 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'John G. Scudder'" <[email-masked]>
Cc: "'Pedro Roque Marques'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 18:03:53 -0400
Sender: [email-masked]

John:

What if the dynamic capabilities was controlled by an operator's script
that was dynamic and automatic and configured from NMS outside the router. 
The script would add/delete capabilities based on the current state of the
router. Lets say this situation occured continuosly.

Obviously, it would not be sensible to run a live network this way. But this
is one hypthetical condition in which we could re-enter the race condition
after
the session restarted. I guess maybe, one has to judge how relient you want
the router to be. 

In practice, if you can create it with a script, there is a (maybe small)
chance it would happen in a live network.

Ben

> From: John G. Scudder [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 5:53 PM
> To: Abarbanel, Benjamin
> Cc: 'Pedro Roque Marques'; Abarbanel, Benjamin; [email-masked]'
> Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> At 5:46 PM -0400 7/10/02, Abarbanel, Benjamin wrote:
> >I was not saying that two different TCP sessions will cross 
> each other and
> >convey each other messages in a mixed up way. I was saying that the
> >potential
> >for the "ships crossing in the night" that caused the first 
> race condition
> >with
> >the capabilities could be there again, after the new session 
> is established.
> >If some combination of routes or capabilities or just router 
> execution
> >timing
> >caused it, it could happen again with similar environment 
> and thus cause
> >the condition. Murphy's law always strikes the same way if 
> the conditions
> >are right. Simply reseting a session I would believe could replay the
> >condition
> >that cause the first race condition and thus throw the two 
> routers into a
> >perpetual session restart loop.
> 
> Since the hypothetical problem relates to dynamic capability 
> advertisement, why would it recur?  Surely any sensible 
> implementation will only dynamically advertise a capability as a 
> result of a configuration change.  The changed configuration will be 
> used to produce the capabilities in the initial OPEN of the new 
> session.  Thus, no need to use dynamic capability on the new session 
> (until and unless the configuration changes again).
> 
> --John
> 

Date: Wed, 10 Jul 2002 17:52:33 -0400
To: "Abarbanel, Benjamin" <[email-masked]>
From: "John G. Scudder" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Cc: "'Pedro Roque Marques'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>, "[email-masked]'" <[email-masked]>
Sender: [email-masked]

At 5:46 PM -0400 7/10/02, Abarbanel, Benjamin wrote:
>I was not saying that two different TCP sessions will cross each other and
>convey each other messages in a mixed up way. I was saying that the
>potential
>for the "ships crossing in the night" that caused the first race condition
>with
>the capabilities could be there again, after the new session is established.
>If some combination of routes or capabilities or just router execution
>timing
>caused it, it could happen again with similar environment and thus cause
>the condition. Murphy's law always strikes the same way if the conditions
>are right. Simply reseting a session I would believe could replay the
>condition
>that cause the first race condition and thus throw the two routers into a
>perpetual session restart loop.

Since the hypothetical problem relates to dynamic capability 
advertisement, why would it recur?  Surely any sensible 
implementation will only dynamically advertise a capability as a 
result of a configuration change.  The changed configuration will be 
used to produce the capabilities in the initial OPEN of the new 
session.  Thus, no need to use dynamic capability on the new session 
(until and unless the configuration changes again).

--John

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 17:46:25 -0400
Sender: [email-masked]

Pedro:
  
See below

Ben

> From: Pedro Roque Marques [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 5:24 PM
> To: Abarbanel, Benjamin
> Cc: [email-masked]'
> Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Abarbanel, Benjamin writes:
> 
> >> 
> > But the potential for the race condition with the new session is
> > still there.
> 
> I'm sorry but i really disagree with your statement. It would also
> appear to me that if that was the truth we would have bigger 
> issues than
> the one we are discussing...
> 
> Would you please detail, under what scenario(s) you can have data from
> an old BGP session being accepted as valid in a new BGP session ?
> 

I was not saying that two different TCP sessions will cross each other and
convey each other messages in a mixed up way. I was saying that the
potential
for the "ships crossing in the night" that caused the first race condition
with
the capabilities could be there again, after the new session is established.
If some combination of routes or capabilities or just router execution
timing
caused it, it could happen again with similar environment and thus cause
the condition. Murphy's law always strikes the same way if the conditions
are right. Simply reseting a session I would believe could replay the
condition
that cause the first race condition and thus throw the two routers into a 
perpetual session restart loop.

Ben 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Gargi Nalawade'" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 17:39:17 -0400
Sender: [email-masked]

Thanks for the clarification on the original intent.

Clearly I think there are many cases, were we want to send potential
operational affecting information to a neighboring peer without reseting
the session. INFORM type message could provide that communication.

I can see the following type of data it would convey:
  1. CPU Utilization levels in an effort to slow down the rate of update.
  2. Memory utilization in an effort to control the amount of routing
     information sent, possibly forbid any optional capabilities while
     router is in a memory depleted state.
  3. Reaching maximum limits on configured or defaulted resources. Number
     of prefixes, attributes, capabilities, etc.
  4. Protocol related WARNING conditions that do not cause a session 
     reset. Repair race conditions, identify FSM state, etc.  

Thanks,
Ben 

> From: Gargi Nalawade [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 5:12 PM
> To: [email-masked]
> Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Hi All,
> 
> A few clarifications about the INFORM message & its 
> motivations. This is
> supposed to be purely an informational message. Does not 
> propose to affect
> the functionality of the protocol. Rather, it proposes to 
> give flexibility
> to implementations in that; they dont need to reset the 
> sessions, just so
> they can inform the peer of a specific noteworthy event happening. 
> 
> Take the example of max-prefix limit. If the limit is hit, 
> the receiving
> peer would just generate a local warning (if so configured) 
> and continue
> with the session. There is no mechanism of informing the 
> other peer that
> it has exceeded the prefix-limit value, other than churning 
> through the log,
> noticing the warnin & then picking up the telephone (or oops 
> - writing an
> email) & informing the peer's administrator.
> 
> This is just one such hypothetical example where the INFORM 
> would be useful.
> But the INFORM message gives us the capability to do just 
> that. There are 
> several other such scenarios seen in deployment, that necessitate the
> existence of just such a mechanism.
> 
> -Gargi
> 

Date: Wed, 10 Jul 2002 14:24:04 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Abarbanel, Benjamin writes:

>> 
> But the potential for the race condition with the new session is
> still there.

I'm sorry but i really disagree with your statement. It would also
appear to me that if that was the truth we would have bigger issues than
the one we are discussing...

Would you please detail, under what scenario(s) you can have data from
an old BGP session being accepted as valid in a new BGP session ?

Due to the fact that these are 2 different TCP sessions, that would
require that TCP itself accepts the packet of an old session... to my
knowledge it is not commonly known for TCP to exhibit such a problem. TCP
in fact as a series of mechanisms designed to address this issue.

Of course, it could always be that i'm misinterpreting your statement,
case in which i would apologize and ask for a clarification.

regards,
  Pedro.

Date: Wed, 10 Jul 2002 14:12:26 -0700
From: Gargi Nalawade <[email-masked]>
To: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Hi All,

A few clarifications about the INFORM message & its motivations. This is
supposed to be purely an informational message. Does not propose to affect
the functionality of the protocol. Rather, it proposes to give flexibility
to implementations in that; they dont need to reset the sessions, just so
they can inform the peer of a specific noteworthy event happening. 

Take the example of max-prefix limit. If the limit is hit, the receiving
peer would just generate a local warning (if so configured) and continue
with the session. There is no mechanism of informing the other peer that
it has exceeded the prefix-limit value, other than churning through the log,
noticing the warnin & then picking up the telephone (or oops - writing an
email) & informing the peer's administrator.

This is just one such hypothetical example where the INFORM would be useful.
But the INFORM message gives us the capability to do just that. There are 
several other such scenarios seen in deployment, that necessitate the
existence of just such a mechanism.

-Gargi

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 17:04:50 -0400
Sender: [email-masked]

> >>  btw: If a BGP speaker, receives an inform that claims that it is
> >> sending an NLRI that in fact it is not configured to send what is
> >> the suggested behaviour ? I'd be tempted by default to reset the
> >> session since the peer clearly doesn't know what it is talking
> >> about.
>  
> > Could we not fall into this kind of race condition after session
> > resets?
> 
> It would not be of the same session, correct... ? session implies a
> TCP socket. If you do have a close, there is an implicit sequencing
> event and no confusion with previous messages.
> 
But the potential for the race condition with the new session is still 
there. If its not a fluky timing problem it might reappear, if it was fluky
it will go away. My only point is, if we were gracefull the first time it 
happened we would have told the offending peer without dropping his routes
and
perhaps it could correct the error and renegotiate the capabilities and
stabilize the problem without a network outage.

Ben

Date: Wed, 10 Jul 2002 13:55:43 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Abarbanel, Benjamin writes:

> How will the offending peer know it is doing something wrong if the
> receiving peer does not send some indication of the problem?

There is no problem in this scenario... just messages that cross
themselfs in transit. I see no benifit in informing the sender of
this.

>>  btw: If a BGP speaker, receives an inform that claims that it is
>> sending an NLRI that in fact it is not configured to send what is
>> the suggested behaviour ? I'd be tempted by default to reset the
>> session since the peer clearly doesn't know what it is talking
>> about.
 
> Could we not fall into this kind of race condition after session
> resets?

It would not be of the same session, correct... ? session implies a
TCP socket. If you do have a close, there is an implicit sequencing
event and no confusion with previous messages.

  Pedro.

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 15:37:48 -0400
Sender: [email-masked]

> From: Pedro Roque Marques [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 3:06 PM
> To: Abarbanel, Benjamin
> Cc: [email-masked]'
> Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Abarbanel, Benjamin writes:
> 
> > pedro: As was mentioned in earlier email:
> 
> > " Also, keep in mind that in the case of dynamic capability
> > negotiation is possible and perfectly legal to receive a message
> > which relies on a capability which was recently withdrawn. Consider
> > for example the case where a route-refresh message and a capability
> > message which withdraws the route-refresh capability cross each
> > other."
> 
> > In this case there was no corruption in the payload of the Update
> > message and yet the state machine was in a inconsistent state for a
> > particular capability due to the race condition.
> 
> > In your approach
> > you would have considered that peer's routes invalid and terminated
> > the session.
> 
> No. It has been a while since i read that particular proposal but i
> seem to rememeber and i believe that the following algotihm is
> recomended there or has been discussed...
> 
> At an nlri withrawl:
> - Keep track of nlri which where enabled until that event
> - Start a timer (something like 30 secs)
> - For the duration of the timer ignore updates that match the
> remebered nlri.
> - When the timer kicks, cleanup the 'old nlri'...
> 
> No additional spec changes... implementation details that afect the
> behaviour during a shortest interval of time.
> 
> > In my, approach I would not, but yet send back an
> > INFORM to the inconsistent peer and expect it to take the proper
> > corrective action.
> 
> imho, that isn't the correct behaviour on a 'race condition' kind of
> scenario... you want to discard those messages silently without
> generating new messages in response.

How will the offending peer know it is doing something wrong if the
receiving 
peer does not send some indication of the problem? 

I would agree sending more than a few consecutive INFORM messages could lead
to a very big burst of INFORM messages if the offending peer does not stop
sending the invalid update messages. 

> 
> btw: If a BGP speaker, receives an inform that claims that it is
> sending an NLRI that in fact it is not configured to send what is the
> suggested behaviour ? I'd be tempted by default to reset the session
> since the peer clearly doesn't know what it is talking about.
>
 
Could we not fall into this kind of race condition after session resets? 

Ben

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Jeffrey Haas'" <[email-masked]>, Pedro Roque Marques <[email-masked]>
Cc: "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 15:12:45 -0400
Sender: [email-masked]

> From: Jeffrey Haas [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 3:08 PM
> To: Pedro Roque Marques
> Cc: Abarbanel, Benjamin; [email-masked]
> Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> I want to post something more thoughtful on this thread, but have
> to do it later due to time constraints.
> 
> I couldn't let this pass though.
> 
> On Wed, Jul 10, 2002 at 11:54:41AM -0700, Pedro Roque Marques wrote:
> > If the stability of the Internet is the concern, then i believe that
> > one should avoid any changes to the protocol unless they are
> > absolutely necessary.
> 
> This made me laugh extremely loudly - but not at you.
> 
> I count 9 RFC extensions past RFC1771 base functionality.
> I count at least 10 protocol extensions in ID form in just
> my personal watch list of stuff that we may implement or have
> implemented.  This list is by no means complete.
> 
> When talking to people, I often refer to BGP as the "Border 
> Glue Protocol".
> People keep adding stuff.
> 
> > By default the answer should be 'no'. Don't change the protocol, add
> > flexibility that you desire, and is fact required operationally, in
> > the implementations. There is plenty of space to compete there...
> 
> The trick is to do it in an interoperable way.

I totally agree, I also think we should add things in a way that is
phaseable
into existing networks. Thus causing little if any network disburbances
between 
the old routers and the new.

Ben

Date: Wed, 10 Jul 2002 15:07:40 -0400
From: Jeffrey Haas <[email-masked]>
To: Pedro Roque Marques <[email-masked]>
Cc: "Abarbanel, Benjamin" <[email-masked]>, [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

I want to post something more thoughtful on this thread, but have
to do it later due to time constraints.

I couldn't let this pass though.

On Wed, Jul 10, 2002 at 11:54:41AM -0700, Pedro Roque Marques wrote:
> If the stability of the Internet is the concern, then i believe that
> one should avoid any changes to the protocol unless they are
> absolutely necessary.

This made me laugh extremely loudly - but not at you.

I count 9 RFC extensions past RFC1771 base functionality.
I count at least 10 protocol extensions in ID form in just
my personal watch list of stuff that we may implement or have
implemented.  This list is by no means complete.

When talking to people, I often refer to BGP as the "Border Glue Protocol".
People keep adding stuff.

> By default the answer should be 'no'. Don't change the protocol, add
> flexibility that you desire, and is fact required operationally, in
> the implementations. There is plenty of space to compete there...

The trick is to do it in an interoperable way.

>   Pedro.

-- 
Jeff Haas 
NextHop Technologies

Date: Wed, 10 Jul 2002 12:06:24 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Abarbanel, Benjamin writes:

> pedro: As was mentioned in earlier email:

> " Also, keep in mind that in the case of dynamic capability
> negotiation is possible and perfectly legal to receive a message
> which relies on a capability which was recently withdrawn. Consider
> for example the case where a route-refresh message and a capability
> message which withdraws the route-refresh capability cross each
> other."

> In this case there was no corruption in the payload of the Update
> message and yet the state machine was in a inconsistent state for a
> particular capability due to the race condition.

> In your approach
> you would have considered that peer's routes invalid and terminated
> the session.

No. It has been a while since i read that particular proposal but i
seem to rememeber and i believe that the following algotihm is
recomended there or has been discussed...

At an nlri withrawl:
- Keep track of nlri which where enabled until that event
- Start a timer (something like 30 secs)
- For the duration of the timer ignore updates that match the
remebered nlri.
- When the timer kicks, cleanup the 'old nlri'...

No additional spec changes... implementation details that afect the
behaviour during a shortest interval of time.

> In my, approach I would not, but yet send back an
> INFORM to the inconsistent peer and expect it to take the proper
> corrective action.

imho, that isn't the correct behaviour on a 'race condition' kind of
scenario... you want to discard those messages silently without
generating new messages in response.

btw: If a BGP speaker, receives an inform that claims that it is
sending an NLRI that in fact it is not configured to send what is the
suggested behaviour ? I'd be tempted by default to reset the session
since the peer clearly doesn't know what it is talking about.

> My way, gives a little room for the protocol to
> tolerate temporary inconsistency.  Yours wont.

I believe it is a weak argument...
Temporary, triggered inconsistencies, should not be handled at the
expense of ignoring the problem of inconsistencies that happen due to
failure in a more generic case...

btw: i've omitted my usual rant on how i believe that dynamic capability
negociation for NLRI is a questionable enhancement from a cost/benefit
point of view. Shouldn't be monopolizing the soap box...

  Pedro.

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>, "Abarbanel, Benjamin" <[email-masked]>
Cc: [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 15:03:43 -0400
Sender: [email-masked]

I think we are getting very philosophical here, but as we all know no
protocol
that is heavily used stands still. BGP is one that is constantly in a state
of 
change because of the various users it has and will continue to add on.
To say one cannot change it and improve it, is to say we dont make progress.

We know there are many issues with the way it works and I am sure many
experts
will tell you what they are. So I wont get into it. All I and others here
are 
trying to do is add a little elasticity/gracefullness/resiliency, you name
it what you want.

Ben

> From: Pedro Roque Marques [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 2:55 PM
> To: Abarbanel, Benjamin
> Cc: [email-masked]
> Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Abarbanel, Benjamin writes:
> 
> > Correct. The protocol has to be elastic, since it carries so much
> > weight in controlling stability in the Internet.
> 
> <meta>
> 
> If the stability of the Internet is the concern, then i believe that
> one should avoid any changes to the protocol unless they are
> absolutely necessary.
> 
> If we where talking about a protocol in early development then adding
> extra 'bells and whistles' would probably be ok (although i can think
> of quite a large number of new protocols which deployment has been
> very negativly impacted by constant churn).
> 
> But since this an essential protocol to the internet, the burden of
> proof required to make a modification to the protocol should be much
> higher and should be on the side of the proponents.
> 
> By default the answer should be 'no'. Don't change the protocol, add
> flexibility that you desire, and is fact required operationally, in
> the implementations. There is plenty of space to compete there...
> 
> </meta>
> 
>   Pedro.
> 

Date: Wed, 10 Jul 2002 11:54:41 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Abarbanel, Benjamin writes:

> Correct. The protocol has to be elastic, since it carries so much
> weight in controlling stability in the Internet.

<meta>

If the stability of the Internet is the concern, then i believe that
one should avoid any changes to the protocol unless they are
absolutely necessary.

If we where talking about a protocol in early development then adding
extra 'bells and whistles' would probably be ok (although i can think
of quite a large number of new protocols which deployment has been
very negativly impacted by constant churn).

But since this an essential protocol to the internet, the burden of
proof required to make a modification to the protocol should be much
higher and should be on the side of the proponents.

By default the answer should be 'no'. Don't change the protocol, add
flexibility that you desire, and is fact required operationally, in
the implementations. There is plenty of space to compete there...

</meta>

  Pedro.

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Chandrashekhar Appanna'" <[email-masked]>
Cc: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 14:49:12 -0400
Sender: [email-masked]

I am in favour of providing a tolerance counter that will drop the session
only after a max number of bad update messages are received. Thus if the
offending
peer does not correct or adjust itself in terms of capabilities, after a
number
of INFORM responses are sent to it, then and only then should the receiving
router drop the session. This will cover the case, where the INFORM is also
mis-processed in the "ships in the night" scenario.

Ben

> From: Chandrashekhar Appanna [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 2:09 PM
> To: Abarbanel, Benjamin
> Cc: 'Bruno Rijsman'; [email-masked]
> Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> 
> But that and the ability to INFORM is not the same. In the
> example you provide, the INFORM is not providing any extra 
> advantage. If the peer did not process/rcv the withdraw 
> why do you hope that it will rcv/process the INFORM?
> 
> I think the behavior of the withdraw itself is the problem here,
> It has to handle the case of a message cross that you describe, 
> in a well defined manner. 
> 
> -
> Chandra A.
> 
> On Wed, Jul 10, 2002 at 11:50:32AM -0400, Abarbanel, Benjamin wrote:
> > Correct. The protocol has to be elastic, since it carries 
> so much weight in 
> > controlling stability in the Internet.
> > 
> > Ben
> > 
> > > From: Bruno Rijsman [mailto:[email-masked]]
> > > Sent: Wednesday, July 10, 2002 11:47 AM
> > > To: [email-masked]
> > > Cc: [email-masked]
> > > Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> > > 
> > > 
> > > Re> Just because you receive an Update message with 
> capabilities that
> > > Re> were not negotiated at the OPEN message, should not 
> require the
> > > Re> peering session to terminate. The affect of all that 
> peer routes
> > > Re> going down will destabilize the network.
> > > 
> > > Also, keep in mind that in the case of dynamic capability 
> negotiation
> > > is possible and perfectly legal to receive a message 
> which relies on
> > > a capability which was recently withdrawn. Consider for 
> example the
> > > case where a route-refresh message and a capability message which
> > > withdraws the route-refresh capability cross each other.
> > > 
> > > 
> > > 
> > > _________________________________________________________________
> > > Send and receive Hotmail on your mobile device: 
http://mobile.msn.com

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 14:44:15 -0400
Sender: [email-masked]

pedro:

As was mentioned in earlier email:

" Also, keep in mind that in the case of dynamic capability negotiation
  is possible and perfectly legal to receive a message which relies on
  a capability which was recently withdrawn. Consider for example the
  case where a route-refresh message and a capability message which
  withdraws the route-refresh capability cross each other."

In this case there was no corruption in the payload of the Update message
and yet the state machine was in a inconsistent state for a particular
capability due to the race condition. In your approach you would have 
considered that peer's routes invalid and terminated the session. In my,
approach I would not, but yet send back an INFORM to the inconsistent 
peer and expect it to take the proper corrective action. My way, gives a 
little room for the protocol to tolerate temporary inconsistency. 
Yours wont. 

> 
> As i've detailed in previous e-mails, the receipt of an UPDATE message
> for an NLRI that has not been negociated, can, imho, only be cause by
> either corruption of the state machine on your peer or of the
> message. When detecting such a scenario, you want to reset all
> information received from the corrupted peer as fast as possible,
> since poison information can be just about the most disturbing thing
> to a network.
> 
> 
> > I think a more gracefull mechanism suggested by others on this list
> > will help inform the originating error'd peer of this case and allow
> > it to either
> > remove the unapproved capability from future updates or self
> > terminate the peering session.
> 
> 
> I'm sure we must have very different experiences when it comes to
> UPDATE messages that include invalid NLRI...

I am in favour of ignoring the entire update message if it has invalid
capabilities. Thus previous updates are still maintained. I am also in
favour of defining a tolerance counter for invalid update messages such
that after a number of invalid updates, only then should the receiving peer
drop the session.

> 
> All the instances i've seen either in the lab or in the wild have been
> cases in which either one of the lenght fields have been incorrectly
> calculated or the message has been inadvertily trimed by the
> sender. You see an invalid NLRI not because the peer actually intended
> to send such NLRI but because, typically, you are reading garbage.
> 
> I fail to see the wisdom of adding new procedures to the protocol that
> concern itself with how to deal with semantically incorrect
> updates. In my experience, quite often you are not lucky enought that
> the message just becomes corrupted at the byte deailing the NLRI...
>
If we are talking of gracefull ways of handling invalid updates, then an
immediate session drop is unacceptable. If we are talking of non-gracefull
way, then you are right. 

The problem is, in my experiences taking abrupt action most of the time is
not necessary and usually leads to unstable and unconverged network. Cause
if you drop the peer, and the race condition occurs again and again after
a session restarts, you will be in a perpetual loop and you will experience
oscillations in routes and then that opens the topic to our previous thread
of converstation here on the list. Also if the offending router has a
software
bug that causes this condition, what makes you think if you drop the session
the offending router will magically correct its code?
 
> In most of the cases you have already received and accepted corrupted
> data that fail to trigger any validation checks.
> 
 Can you explain in detail the scenario you are talking about here?

> All scenarios that i've seen lead me to conclude that the desired
> outcome is to:
> 1) Make sure that you do not propagate junk received in the UPDATE
> that you detected the error.

Agree on this point.

> 2) Drop all information from the peer ASAP...

Disagree on this point. Why all previous good updates from this peer bad?

> 
> RFC 2858, probably because it was drafted before capability
> negociation was in place, is incorrect in specifying that one should
> ignore an update with a missing NLRI. It might have been the best
> aproach at the time it was drafted but it isn't now. It is just
> another instance where deployment experience advises one against
> following the RFCs literally... which is a concept that should be
> quite familiar to people dealing with BGP.
> 

I sure hope the people who drafted this, have deployment experience and are 
saying this for the majority of the cases.

Can we get some voices on this from the authors of this draft?

>   Pedro.

Thanks,
Ben

Date: Wed, 10 Jul 2002 11:13:39 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Abarbanel, Benjamin writes:

> Just because you receive an Update message with capabilities that
> were not negotiated at the OPEN message, should not require the
> peering session to terminate. The affect of all that peer routes
> going down will destabilize the network.

As i've detailed in previous e-mails, the receipt of an UPDATE message
for an NLRI that has not been negociated, can, imho, only be cause by
either corruption of the state machine on your peer or of the
message. When detecting such a scenario, you want to reset all
information received from the corrupted peer as fast as possible,
since poison information can be just about the most disturbing thing
to a network.

I believe that proponents of this mechanism do need to demonstrate how
it would be useful...

> I think a more gracefull mechanism suggested by others on this list
> will help inform the originating error'd peer of this case and allow
> it to either
> remove the unapproved capability from future updates or self
> terminate the peering session.

I'm sure we must have very different experiences when it comes to
UPDATE messages that include invalid NLRI...

All the instances i've seen either in the lab or in the wild have been
cases in which either one of the lenght fields have been incorrectly
calculated or the message has been inadvertily trimed by the
sender. You see an invalid NLRI not because the peer actually intended
to send such NLRI but because, typically, you are reading garbage.

I fail to see the wisdom of adding new procedures to the protocol that
concern itself with how to deal with semantically incorrect
updates. In my experience, quite often you are not lucky enought that
the message just becomes corrupted at the byte deailing the NLRI...

In most of the cases you have already received and accepted corrupted
data that fail to trigger any validation checks.

All scenarios that i've seen lead me to conclude that the desired
outcome is to:
1) Make sure that you do not propagate junk received in the UPDATE
that you detected the error.
2) Drop all information from the peer ASAP...

RFC 2858, probably because it was drafted before capability
negociation was in place, is incorrect in specifying that one should
ignore an update with a missing NLRI. It might have been the best
aproach at the time it was drafted but it isn't now. It is just
another instance where deployment experience advises one against
following the RFCs literally... which is a concept that should be
quite familiar to people dealing with BGP.

  Pedro.

Date: Wed, 10 Jul 2002 11:08:41 -0700
From: Chandrashekhar Appanna <[email-masked]>
To: "Abarbanel, Benjamin" <[email-masked]>
Cc: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

But that and the ability to INFORM is not the same. In the
example you provide, the INFORM is not providing any extra 
advantage. If the peer did not process/rcv the withdraw 
why do you hope that it will rcv/process the INFORM?

I think the behavior of the withdraw itself is the problem here,
It has to handle the case of a message cross that you describe, 
in a well defined manner. 

-
Chandra A.

On Wed, Jul 10, 2002 at 11:50:32AM -0400, Abarbanel, Benjamin wrote:
> Correct. The protocol has to be elastic, since it carries so much weight in 
> controlling stability in the Internet.
> 
> Ben
> 
> > From: Bruno Rijsman [mailto:[email-masked]]
> > Sent: Wednesday, July 10, 2002 11:47 AM
> > To: [email-masked]
> > Cc: [email-masked]
> > Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> > 
> > 
> > Re> Just because you receive an Update message with capabilities that
> > Re> were not negotiated at the OPEN message, should not require the
> > Re> peering session to terminate. The affect of all that peer routes
> > Re> going down will destabilize the network.
> > 
> > Also, keep in mind that in the case of dynamic capability negotiation
> > is possible and perfectly legal to receive a message which relies on
> > a capability which was recently withdrawn. Consider for example the
> > case where a route-refresh message and a capability message which
> > withdraws the route-refresh capability cross each other.
> > 
> > 
> > 
> > _________________________________________________________________
> > Send and receive Hotmail on your mobile device: http://mobile.msn.com
> > 

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Bruno Rijsman'" <[email-masked]>, [email-masked]
Cc: "Abarbanel, Benjamin" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 11:50:32 -0400
Sender: [email-masked]

Correct. The protocol has to be elastic, since it carries so much weight in 
controlling stability in the Internet.

Ben

> From: Bruno Rijsman [mailto:[email-masked]]
> Sent: Wednesday, July 10, 2002 11:47 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Re> Just because you receive an Update message with capabilities that
> Re> were not negotiated at the OPEN message, should not require the
> Re> peering session to terminate. The affect of all that peer routes
> Re> going down will destabilize the network.
> 
> Also, keep in mind that in the case of dynamic capability negotiation
> is possible and perfectly legal to receive a message which relies on
> a capability which was recently withdrawn. Consider for example the
> case where a route-refresh message and a capability message which
> withdraws the route-refresh capability cross each other.
> 
> 
> 
> _________________________________________________________________
> Send and receive Hotmail on your mobile device: http://mobile.msn.com
> 

From: "Bruno Rijsman" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 11:47:17 -0400
Sender: [email-masked]

Re> Just because you receive an Update message with capabilities that
Re> were not negotiated at the OPEN message, should not require the
Re> peering session to terminate. The affect of all that peer routes
Re> going down will destabilize the network.

Also, keep in mind that in the case of dynamic capability negotiation
is possible and perfectly legal to receive a message which relies on
a capability which was recently withdrawn. Consider for example the
case where a route-refresh message and a capability message which
withdraws the route-refresh capability cross each other.

_________________________________________________________________
Send and receive Hotmail on your mobile device: http://mobile.msn.com

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Pedro Roque Marques'" <[email-masked]>, Jeffrey Haas <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Wed, 10 Jul 2002 11:40:08 -0400
Sender: [email-masked]

Pedro and Jeff:

> From: Pedro Roque Marques [mailto:[email-masked]]
> Sent: Tuesday, July 09, 2002 1:45 PM
> To: Jeffrey Haas
> Cc: [email-masked]'
> Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
> 
> 
> Jeffrey Haas writes:
> 
> > The operational community continues to demand resiliency from the
> > BGP protocol.
> 
> I believe you get less resiliency, not more, from accepting corrupted
> messages. If you have a peer that is sending you a given NLRI and it
> didn't negociate it then either:
> 1) The peers state machine is completly out of wack, case in which you
> want to reset it.

Just because you receive an Update message with capabilities that were not
negotiated at the OPEN message, should not require the peering session to
terminate. The affect of all that peer routes going down will destabilize
the network. 

I think a more gracefull mechanism suggested by others on this list will 
help inform the originating error'd peer of this case and allow it to either

remove the unapproved capability from future updates or self terminate the 
peering session. It is more gracefull for the originating peer to decide if
its 
publicized routes are compromised. The assumption being is that the
advertised 
routes are most likely stable and reachable and do not constitute any harm
to 
the topology. 

The receiving peer should ignore all subsequent capabilities not informed
during 
the session OPEN exchange. We would need a NON session terminating warning
message like the INFORM suggested by others to do this:

  "INFORM would at least make this non-silent.  However, additional pushback
   might be warranted.  Upon receipt of an INFORM (or similar) message
   the receiving peer may wish to choose one or more of the following
actions:

   o Just log it per INFORM
   o Stop sending this AFI/SAFI
   o Reset the peering session (contrary to the wishes of the peer?)
  "

> 2) It is a corrupted message, case in which you wish to reset it.
> 
> >  If we can avoid sending a NOTIFICATION and gracefully
> > deal with someone elses buggy code, thats a Good Thing.

It is very difficult to determine if the offending peer has buggy code or a
minor bug. Just taking down all its routes on the first sign of trouble
would not be wise.

Also, multiprotocol capabilities are new and optional things they should not
destablize the network if there is doubt on their intentions.

> 
> > Sending an INFORM would be nice here, if such a thing existed.
> 
> Personally i believe that this message is completly unnecessary.
> 
>   Pedro.
>

I think BGP needs many mechanisms for gracefullness and tolerance of
imprecise
information in order to combat the Global route instability that is caused
when
BGP routers just tear down their peering sessions at the first sign of
trouble.

Ben 

Date: Tue, 9 Jul 2002 10:45:04 -0700 (PDT)
From: Pedro Roque Marques <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Jeffrey Haas writes:

> The operational community continues to demand resiliency from the
> BGP protocol.

I believe you get less resiliency, not more, from accepting corrupted
messages. If you have a peer that is sending you a given NLRI and it
didn't negociate it then either:
1) The peers state machine is completly out of wack, case in which you
want to reset it.
2) It is a corrupted message, case in which you wish to reset it.

>  If we can avoid sending a NOTIFICATION and gracefully
> deal with someone elses buggy code, thats a Good Thing.

> Sending an INFORM would be nice here, if such a thing existed.

Personally i believe that this message is completly unnecessary.

  Pedro.

From: "Abarbanel, Benjamin" <[email-masked]>
To: "'Natale, Jonathan'" <[email-masked]>, "'Jeffrey Haas'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Tue, 9 Jul 2002 12:54:32 -0400 
Sender: [email-masked]

> Also:
> "If a BGP speaker that supports a certain capability determines that
>    its peer doesn't support this capability, the speaker MAY send a
>    NOTIFICATION message to the peer, and terminate peering 
> (see Section
>    "Extensions to Error Handling" for more details). The Error Subcode
>    in the message is set to Unsupported Capability. The message SHOULD
>    contain the capability (capabilities) that causes the 
> speaker to send
>    the message.  The decision to send the message and 
> terminate peering
>    is local to the speaker. If terminated, such peering SHOULD NOT be
>    re-established automatically. --draft-ietf-idr-rfc2842bis-02.txt,
> "Capabilities Advertisement with BGP-4"
> 
> Seems harsh, I would think that the preferred behavior is to 
> re-open w/o the
> offending capability.  A major vendor does not and they have 
> a back door cmd
> to work around it.  They also have a bug on it but based on history
> (AS_CONFED_SET <-> AS_CONFED_SEQUENCE, deterministic med, etc...) the
> rfc/draft seem to follow them NOT the other way around. Any 
> thoughts???

For network deployment phasing reasons and backward compatibility reasons.
We should not drop a peer just because we recieved a message that contained
unrecognized capability(s). I vote the spec be changed to accomodate an
evolving network. The lesser capable BGP router should silently ignore the
unrecognized capability(s) or the entire message if there is nothing else
recognizable in it. The more advanced BGP capable router upon detecting a
peer that does not understand certain advanced capability(s) should merely
not send any more advanced capability(s) to this peer. The two routers
should also log the event for their operators to review.

Also, if we drop peers do to unrecognized messages or capabilities, we make
it easier for hacker attacks causing network outages, or the topology to get
unstable.  

Right?

Ben

From: "Natale, Jonathan" <[email-masked]>
To: "'Jeffrey Haas'" <[email-masked]>, "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Tue, 9 Jul 2002 12:35:19 -0400 
Sender: [email-masked]

Thank you very much for the prompt reply.  I agree completely.

My question is this: If you receive a route refresh when you did not
advertise that capability at all (and/or your code does not support it at
all) then you should reset the peer with a 1/3 (Message Header Error/Bad
Message Type) notification. Right?

Also:
"If a BGP speaker that supports a certain capability determines that
   its peer doesn't support this capability, the speaker MAY send a
   NOTIFICATION message to the peer, and terminate peering (see Section
   "Extensions to Error Handling" for more details). The Error Subcode
   in the message is set to Unsupported Capability. The message SHOULD
   contain the capability (capabilities) that causes the speaker to send
   the message.  The decision to send the message and terminate peering
   is local to the speaker. If terminated, such peering SHOULD NOT be
   re-established automatically. --draft-ietf-idr-rfc2842bis-02.txt,
"Capabilities Advertisement with BGP-4"

Seems harsh, I would think that the preferred behavior is to re-open w/o the
offending capability.  A major vendor does not and they have a back door cmd
to work around it.  They also have a bug on it but based on history
(AS_CONFED_SET <-> AS_CONFED_SEQUENCE, deterministic med, etc...) the
rfc/draft seem to follow them NOT the other way around. Any thoughts???

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Tuesday, July 09, 2002 12:26 PM
To: Natale, Jonathan
Cc: [email-masked]'
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)

On Tue, Jul 09, 2002 at 12:19:23PM -0400, Natale, Jonathan wrote:
> : "If a BGP speaker receives from its peer a ROUTE-REFRESH message with
>    the <AFI, SAFI> that the speaker didn't advertise to the peer at the
>    session establishment time via capability advertisement, the speaker
>    shall ignore such a message." - RFC2918, "Route Refresh Capability for
> BGP-4"
> 
> shouldn't this be an error/notification???

The operational community continues to demand resiliency from the BGP
protocol.  If we can avoid sending a NOTIFICATION and gracefully
deal with someone elses buggy code, thats a Good Thing.

Sending an INFORM would be nice here, if such a thing existed.

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 9 Jul 2002 12:30:31 -0400
From: Jeffrey Haas <[email-masked]>
To: "Natale, Jonathan" <[email-masked]>
Cc: "'Fryczke, Pawel'" <[email-masked]>, [email-masked]
Subject: Re: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Sender: [email-masked]

On Tue, Jul 09, 2002 at 10:48:04AM -0400, Natale, Jonathan wrote:
> I raised this issue more than two years ago.  What is the story on bgp?  Why
> is the draft/rfc so far off base?  I would be happy to provide a laundry
> list of things, if that will help.

That list would be good.  At the very least, it lets us adjust the
spec to "should" versus "shall" if we decide that it makes sense to
do so.

> I think the MED was fixed in a major vendor's code with the ~= deterministic
> med cmd (or so I was told then, but now the story on that cmd seem
> different).

deterministic med fixes the MED election to follow the spec.  I
don't believe it has anything to do with aggregation.

> The next hop was never "fixed".  The only answer for why not to aggregate if
> nxhp not the same was that it has something to do w/ mcast RPF... but I
> never fully digested this.

Once Upon a Time (-12?) the active route was "echoed" back to
the external peer which it was learned from to facilitate multicast
RPF checks.  No one appeared to use this, so it was removed from the
spec.  If the only reason for the nexthop to be the same in aggregation
was for a similar reason, it may be worth removing the restriction.

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 9 Jul 2002 12:25:42 -0400
From: Jeffrey Haas <[email-masked]>
To: "Natale, Jonathan" <[email-masked]>
Cc: "[email-masked]'" <[email-masked]>
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

On Tue, Jul 09, 2002 at 12:19:23PM -0400, Natale, Jonathan wrote:
> : "If a BGP speaker receives from its peer a ROUTE-REFRESH message with
>    the <AFI, SAFI> that the speaker didn't advertise to the peer at the
>    session establishment time via capability advertisement, the speaker
>    shall ignore such a message." - RFC2918, "Route Refresh Capability for
> BGP-4"
> 
> shouldn't this be an error/notification???

The operational community continues to demand resiliency from the BGP
protocol.  If we can avoid sending a NOTIFICATION and gracefully
deal with someone elses buggy code, thats a Good Thing.

Sending an INFORM would be nice here, if such a thing existed.

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 9 Jul 2002 12:22:55 -0400
From: Jeffrey Haas <[email-masked]>
To: "Natale, Jonathan" <[email-masked]>
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

On Tue, Jul 09, 2002 at 12:17:28PM -0400, Natale, Jonathan wrote:
> Where is this proposal? URL???

:         Title           : BGPv4 INFORM message
:         Author(s)       : G. Nalawade et al.
:         Filename        : draft-nalawade-bgp-inform-00.txt
:         Pages           : 10
:         Date            : 24-Jun-02
:
: This document defines a new message type, the BGP INFORM
: message that communicates Informational data and operational
: warnings without resetting the peering session.
:
: A URL for this Internet-Draft is:
: http://www.ietf.org/internet-drafts/draft-nalawade-bgp-inform-00.txt

-- 
Jeff Haas 
NextHop Technologies

From: "Natale, Jonathan" <[email-masked]>
To: "[email-masked]'" <[email-masked]>
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Tue, 9 Jul 2002 12:19:23 -0400 
Sender: [email-masked]

: "If a BGP speaker receives from its peer a ROUTE-REFRESH message with
   the <AFI, SAFI> that the speaker didn't advertise to the peer at the
   session establishment time via capability advertisement, the speaker
   shall ignore such a message." - RFC2918, "Route Refresh Capability for
BGP-4"

shouldn't this be an error/notification???

From: "Natale, Jonathan" <[email-masked]>
To: "'Jeffrey Haas'" <[email-masked]>, [email-masked]
Subject: RE: INFORM proposal and MP-BGP (RFC 2858)
Date: Tue, 9 Jul 2002 12:17:28 -0400 
Sender: [email-masked]

Where is this proposal? URL???

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Monday, July 08, 2002 2:43 PM
To: [email-masked]
Subject: INFORM proposal and MP-BGP (RFC 2858)

The proposal for the INFORM message gives the opportunity to redress
an open issue in the MP-BGP specification:

: 6. Error Handling
: 
:    If a BGP speaker receives from a neighbor an Update message that
:    contains the MP_REACH_NLRI or MP_UNREACH_NLRI attribute, and the
:    speaker determines that the attribute is incorrect, the speaker must
:    delete all the BGP routes received from that neighbor whose AFI/SAFI
:    is the same as the one carried in the incorrect MP_REACH_NLRI or
:    MP_UNREACH_NLRI attribute. For the duration of the BGP session over
:    which the Update message was received, the speaker then should ignore
:    all the subsequent routes with that AFI/SAFI received over that
:    session.

This amounts to silently ignoring reachability.

INFORM would at least make this non-silent.  However, additional pushback
might be warranted.  Upon receipt of an INFORM (or similar) message
the receiving peer may wish to choose one or more of the following actions:

o Just log it per INFORM
o Stop sending this AFI/SAFI
o Reset the peering session (contrary to the wishes of the peer?)

Thoughts?

-- 
Jeff Haas 
NextHop Technologies

Date: Tue, 9 Jul 2002 12:14:57 -0400
From: Jeffrey Haas <[email-masked]>
To: Manav Bhatia <[email-masked]>
Cc: [email-masked]
Subject: Re: NLRI in both update and withdraw
Sender: [email-masked]

>From -17:

   An UPDATE message should not include the same address prefix in the
   WITHDRAWN ROUTES and Network Layer Reachability Information fields,
   however a BGP speaker MUST be able to process UPDATE messages in this
   form. A BGP speaker should treat an UPDATE message of this form as if
   the WITHDRAWN ROUTES doesn't contain the address prefix.

On Tue, Jul 09, 2002 at 03:41:50PM +0530, Manav Bhatia wrote:
> Hi,
> I want to know as to what the current implementations out there in the
> field do and what is the appropriate and the correct method for dealing
> with UPDATE messages  having the same NLRI listed in both the withdrawn and
> the feasible routes.
> 
> Regards,
> Manav

-- 
Jeff Haas 
NextHop Technologies

From: "Natale, Jonathan" <[email-masked]>
To: "'Fryczke, Pawel'" <[email-masked]>, [email-masked]
Subject: RE: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Date: Tue, 9 Jul 2002 10:48:04 -0400 
Sender: [email-masked]

I raised this issue more than two years ago.  What is the story on bgp?  Why
is the draft/rfc so far off base?  I would be happy to provide a laundry
list of things, if that will help.

I think the MED was fixed in a major vendor's code with the ~= deterministic
med cmd (or so I was told then, but now the story on that cmd seem
different).

The next hop was never "fixed".  The only answer for why not to aggregate if
nxhp not the same was that it has something to do w/ mcast RPF... but I
never fully digested this.

-$0.02

-----Original Message-----
From: Fryczke, Pawel [mailto:[email-masked]] 
Sent: Thursday, July 04, 2002 8:08 AM
To: [email-masked]
Subject: Aggregation and MULTI_EXIT_DISC, NEXT_HOP

The latest draft, draft-ietf-idr-bgp4-17.txt states:
Routes that have the following attributes shall not be aggregated
unless the corresponding attributes of each route are identical:
MULTI_EXIT_DISC, NEXT_HOP.

Many vendors (e.g. Cisco/Juniper) does not follow this requirement. Why?
What is the purpose of this requirement?
Pawel

From: "Natale, Jonathan" <[email-masked]>
To: "[email-masked]'" <[email-masked]>, BGP mailing list <[email-masked]>
Cc: Gilles Geerts <[email-masked]>
Subject: RE: TCP state machine doubts
Date: Tue, 9 Jul 2002 10:42:33 -0400 
Sender: [email-masked]

-----Original Message-----

all,

Thanks,
  Hans De Vleeschouwer.

Date: Tue, 09 Jul 2002 10:08:49 -0400
From: Anindya Basu <[email-masked]>
To: Susan Hares <[email-masked]>, [email-masked]
Cc: [email-masked], yakov Rekhter <[email-masked]>
Subject: Re: IDR agenda
Sender: [email-masked]

Dear WG Chairs,

I would like a 10-15 minute slot for presenting the work in
draft-wilfong-ibgp-oscillations-00.txt that addresses the route
oscillation problem identified in
draft-ietf-idr-route-oscillation-00.txt. 

Thanks for your consideration,
-Anindya Basu
 MTS, Bell Laboratories
 Lucent Technologies

Susan Hares wrote:
> 
> Please send any requests for the agenda to the list.
> The final agenda will be sent on Sunday.
> (I am traveling to Japan until that date.)
> 
> Sue Hares

Date: Tue, 09 Jul 2002 06:22:06 -0400
To: [email-masked]
From: Susan Hares <[email-masked]>
Subject: IDR agenda 
Cc: [email-masked], yakov Rekhter <[email-masked]>
Sender: [email-masked]

Please send any requests for the agenda to the list.
The final agenda will be sent on Sunday.
(I am traveling to Japan until that date.)

Sue Hares

Date: Tue, 09 Jul 2002 15:41:50 +0530
From: Manav Bhatia <[email-masked]>
Subject: NLRI in both update and withdraw
To: [email-masked]
Reply-To: Manav Bhatia <[email-masked]>
Sender: [email-masked]

Hi,
I want to know as to what the current implementations out there in the
field do and what is the appropriate and the correct method for dealing
with UPDATE messages  having the same NLRI listed in both the withdrawn and
the feasible routes.

Regards,
Manav

----
"When you are courting a nice girl an hour seems like a second. When you
sit on a red-hot cinder a second seems like an hour. That's relativity."

-Albert Einstein, on relativity

Date: Tue, 9 Jul 2002 00:30:28 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: Mathew Richardson <[email-masked]>, [email-masked]
Subject: Re: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Sender: [email-masked]

> It would be nice to know why the restriction was placed in the
> specification in the first place before we suggest it be removed.

If we perform aggregation as part of the Decision process (as the
document says), it would mean that we're installing aggregated route
and not installing any specifics in our local RIB. The restriction
in question is to make sure it is safe to do so.

The question is whether anyone actually does this instead of
aggregating during the Update process...

Alex

Date: Mon, 8 Jul 2002 15:22:59 -0700
From: Alex Zinin <[email-masked]>
Reply-To: Alex Zinin <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

Jeff,

 BTW, a more generic question: if we can't trust an attribute
 as a whole (since it's invalid), how can we trust specific fields
 (AFI/SAFI) inside, i.e., if an MP_* attribute is malformed, it is
 unsafe to assume that AFI/SAFI fields are correct... so, we may
 very well end up deleting and ignoring a wrong (but valid) AFI/SAFI.

 [I thought I brought up this issue already, but I couldn't find
 any evidence]

-- 
Alex

Monday, July 08, 2002, 11:42:48 AM, Jeffrey Haas wrote:
> The proposal for the INFORM message gives the opportunity to redress
> an open issue in the MP-BGP specification:

> : 6. Error Handling
> : 
> :    If a BGP speaker receives from a neighbor an Update message that
> :    contains the MP_REACH_NLRI or MP_UNREACH_NLRI attribute, and the
> :    speaker determines that the attribute is incorrect, the speaker must
> :    delete all the BGP routes received from that neighbor whose AFI/SAFI
> :    is the same as the one carried in the incorrect MP_REACH_NLRI or
> :    MP_UNREACH_NLRI attribute. For the duration of the BGP session over
> :    which the Update message was received, the speaker then should ignore
> :    all the subsequent routes with that AFI/SAFI received over that
> :    session.

> This amounts to silently ignoring reachability.

> INFORM would at least make this non-silent.  However, additional pushback
> might be warranted.  Upon receipt of an INFORM (or similar) message
> the receiving peer may wish to choose one or more of the following actions:

> o Just log it per INFORM
> o Stop sending this AFI/SAFI
> o Reset the peering session (contrary to the wishes of the peer?)

> Thoughts?

Date: Mon, 8 Jul 2002 17:12:12 -0400 (EDT)
From: Russ White <[email-masked]>
Reply-To: Russ White <[email-masked]>
To: Jeffrey Haas <[email-masked]>
Cc: [email-masked]
Subject: Re: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

When we originally conceived of the INFORM, we had included an
"action code" as well. This turned out to be rather cumbersome,
defining all of them. Probably the best thing to do is to make
certain this code is included in the spec, and perhaps loosen up
the text as to what may be done when receiving an INFORM to
include user configurable options, such as dropping the session
or just the impacted updates, etc.

:-)

Russ

On Mon, 8 Jul 2002, Jeffrey Haas wrote:

> The proposal for the INFORM message gives the opportunity to redress
> an open issue in the MP-BGP specification:
> 
> : 6. Error Handling
> : 
> :    If a BGP speaker receives from a neighbor an Update message that
> :    contains the MP_REACH_NLRI or MP_UNREACH_NLRI attribute, and the
> :    speaker determines that the attribute is incorrect, the speaker must
> :    delete all the BGP routes received from that neighbor whose AFI/SAFI
> :    is the same as the one carried in the incorrect MP_REACH_NLRI or
> :    MP_UNREACH_NLRI attribute. For the duration of the BGP session over
> :    which the Update message was received, the speaker then should ignore
> :    all the subsequent routes with that AFI/SAFI received over that
> :    session.
> 
> This amounts to silently ignoring reachability.
> 
> INFORM would at least make this non-silent.  However, additional pushback
> might be warranted.  Upon receipt of an INFORM (or similar) message
> the receiving peer may wish to choose one or more of the following actions:
> 
> o Just log it per INFORM
> o Stop sending this AFI/SAFI
> o Reset the peering session (contrary to the wishes of the peer?)
> 
> Thoughts?
> 
> -- 
> Jeff Haas 
> NextHop Technologies
> 

__________________________________
[email-masked] CCIE <>< Grace Alone

Date: Mon, 8 Jul 2002 14:42:48 -0400
From: Jeffrey Haas <[email-masked]>
To: [email-masked]
Subject: INFORM proposal and MP-BGP (RFC 2858)
Sender: [email-masked]

The proposal for the INFORM message gives the opportunity to redress
an open issue in the MP-BGP specification:

: 6. Error Handling
: 
:    If a BGP speaker receives from a neighbor an Update message that
:    contains the MP_REACH_NLRI or MP_UNREACH_NLRI attribute, and the
:    speaker determines that the attribute is incorrect, the speaker must
:    delete all the BGP routes received from that neighbor whose AFI/SAFI
:    is the same as the one carried in the incorrect MP_REACH_NLRI or
:    MP_UNREACH_NLRI attribute. For the duration of the BGP session over
:    which the Update message was received, the speaker then should ignore
:    all the subsequent routes with that AFI/SAFI received over that
:    session.

This amounts to silently ignoring reachability.

INFORM would at least make this non-silent.  However, additional pushback
might be warranted.  Upon receipt of an INFORM (or similar) message
the receiving peer may wish to choose one or more of the following actions:

o Just log it per INFORM
o Stop sending this AFI/SAFI
o Reset the peering session (contrary to the wishes of the peer?)

Thoughts?

-- 
Jeff Haas 
NextHop Technologies

Date: Mon, 8 Jul 2002 13:58:38 -0400
From: Jeffrey Haas <[email-masked]>
To: Mathew Richardson <[email-masked]>
Cc: [email-masked]
Subject: Re: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Sender: [email-masked]

On Sat, Jul 06, 2002 at 02:05:54AM -0400, Mathew Richardson wrote:
> The current mandate for the BGP spec. is to document current
> implementations.  If vendors are currently aggregating routes
> with different MEDs, or NEXT_HOPs, then this restriction should
> be made a suggestion ("It is recommended that routes that have
> differing MULTI_EXIT_DISC, or NEXT_HOP, attributes not be
> aggregated"), or removed entirely.

It would be nice to know why the restriction was placed in the
specification in the first place before we suggest it be removed.

> mrr

-- 
Jeff Haas 
NextHop Technologies

From: "Tsillas, Jim" <[email-masked]>
To: "'Daniel Kushner'" <[email-masked]>, [email-masked]
Cc: [email-masked]
Subject: RE: Route Reflector and client interaction questions
Date: Mon, 8 Jul 2002 13:42:23 -0400 
Sender: [email-masked]

Daniel,

what you are describing is a "poison reverse" strategy
which would have the benefit of reducing the amount of
state stored in the network. In your example, when R1
sends a withdrawl back to R2, R2 can free the otherwise
useless state since R1's decision process has determined
that R2's route is better. This is usually done between
external peers, but could also be done between RRs.
Not sending the withdraw is not incorrect since the
topology is not affected.

On the topic of prohibiting attribute modification: each
vendor will decide for themselves, but is there really
any value to limiting this function? If I buy a hammer
I can read the (fictional) warning about using it to relieve
headaches and decide if I want to use it in that manner.
Should that hammer then be redesigned to make it impossible
for me to bang my head?

regards,
-Jim.

-----Original Message-----
From: Daniel Kushner [mailto:[email-masked]]
Sent: Monday, July 08, 2002 1:18 PM
To: [email-masked]
Cc: [email-masked]
Subject: Route Reflector and client interaction questions

I have some questions concerning Juniper Route 
Reflector operation where Juniper and Cisco 
differ in operation.

Given:
R1 = Juniper route reflector (RR)
R2 = RR client to Juniper

- I noticed that if a client peer (R2) advertises a 
   route (route A) to a Juniper route reflector (R1), 
   that Juniper route reflector (R1) sends a 
   withdrawal of the route (route A) back to the 
   client (R2) that advertised the route to the 
   Juniper route reflector. Cisco does not do this. 

My questions are:
1) Does Juniper withdraw the route from the client 
     sender on purpose, or is it a Juniper bug?
2) If it is not a bug, what condition/topology exactly

     does sending the withdrawal back to the client 
     protect against?
3) From the provider standpoint, if it is not a bug, 
     is withdrawing the route from the client sender 
     preferred behavior?

- I also noticed that neither Cisco or Juniper 
   prohibits modification to the AS_PATH, 
   NEXT_HOP, LOCAL_PREF, or MED for 
   routes reflected, as required by the RFC. 
   I'm not sure if this is important when 
   considering the above questions.

4) From the provider standpoint, is allowing 
    attribute modification for reflected routes
    preferred behavior?

Thanks,
- Daniel Kushner

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com

Date: Mon, 8 Jul 2002 10:17:58 -0700 (PDT)
From: Daniel Kushner <[email-masked]>
Subject: Route Reflector and client interaction questions
To: [email-masked]
Cc: [email-masked]
Sender: [email-masked]

I have some questions concerning Juniper Route 
Reflector operation where Juniper and Cisco 
differ in operation.

Given:
R1 = Juniper route reflector (RR)
R2 = RR client to Juniper

- I noticed that if a client peer (R2) advertises a 
   route (route A) to a Juniper route reflector (R1), 
   that Juniper route reflector (R1) sends a 
   withdrawal of the route (route A) back to the 
   client (R2) that advertised the route to the 
   Juniper route reflector. Cisco does not do this. 

My questions are:
1) Does Juniper withdraw the route from the client 
     sender on purpose, or is it a Juniper bug?
2) If it is not a bug, what condition/topology exactly

     does sending the withdrawal back to the client 
     protect against?
3) From the provider standpoint, if it is not a bug, 
     is withdrawing the route from the client sender 
     preferred behavior?

- I also noticed that neither Cisco or Juniper 
   prohibits modification to the AS_PATH, 
   NEXT_HOP, LOCAL_PREF, or MED for 
   routes reflected, as required by the RFC. 
   I'm not sure if this is important when 
   considering the above questions.

4) From the provider standpoint, is allowing 
    attribute modification for reflected routes
    preferred behavior?

Thanks,
- Daniel Kushner

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com

Date: Sat, 06 Jul 2002 04:31:22 -0700
From: Kunihiro Ishiguro <[email-masked]>
To: Mathew Richardson <[email-masked]>
Cc: Kunihiro Ishiguro <[email-masked]>, "Fryczke, Pawel" <[email-masked]>, [email-masked]
Subject: Re: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Sender: [email-masked]

>The current mandate for the BGP spec. is to document current
>implementations.  If vendors are currently aggregating routes
>with different MEDs, or NEXT_HOPs, then this restriction should
>be made a suggestion ("It is recommended that routes that have
>differing MULTI_EXIT_DISC, or NEXT_HOP, attributes not be
>aggregated"), or removed entirely.

I think so too.
-- 
Kunihiro Ishiguro

Date: Sat, 6 Jul 2002 02:05:54 -0400
From: Mathew Richardson <[email-masked]>
To: Kunihiro Ishiguro <[email-masked]>
Cc: "Fryczke, Pawel" <[email-masked]>, [email-masked]
Subject: Re: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Sender: [email-masked]

> Kunihiro Ishiguro <[email-masked]> [Fri, Jul 05, 2002 at 08:33:18PM -0700]:
>>The latest draft, draft-ietf-idr-bgp4-17.txt states:
>>Routes that have the following attributes shall not be aggregated
>>unless the corresponding attributes of each route are identical:
>>MULTI_EXIT_DISC, NEXT_HOP.
>
>Well, this comes from RFC1771 and we didn't change it.
>
>9.2.4.2 Aggregating Routing Information
>
>   Aggregation is the process of combining the characteristics of
>   several different routes in such a way that a single route can be
>   advertised.  Aggregation can occur as part of the decision  process
>   to reduce the amount of routing information that will be placed in
>   the Adj-RIBs-Out.
>
>   Aggregation reduces the amount of information that a BGP speaker must
>   store and exchange with other BGP speakers. Routes can be aggregated
>   by applying the following procedure separately to path attributes of
>   like type and to the Network Layer Reachability Information.
>
>   Routes that have the following attributes shall not be aggregated
>   unless the corresponding attributes of each route are identical:
>   MULTI_EXIT_DISC, NEXT_HOP.
>
>IHMO, the intention of this part is "BGP should aggregate the routes
>which have same network reachability".  Diffrent MULTI_EXIT_DISC or
>NEXT_HOP means different network reachability.  So the specification
>said "shall not be aggregate".

<snip>

The current mandate for the BGP spec. is to document current
implementations.  If vendors are currently aggregating routes
with different MEDs, or NEXT_HOPs, then this restriction should
be made a suggestion ("It is recommended that routes that have
differing MULTI_EXIT_DISC, or NEXT_HOP, attributes not be
aggregated"), or removed entirely.

mrr

Date: Fri, 05 Jul 2002 20:49:18 -0700
From: Kunihiro Ishiguro <[email-masked]>
To: [email-masked]
Cc: BGP mailing list <[email-masked]>, Gilles Geerts <[email-masked]>
Subject: Re: TCP state machine doubts
Sender: [email-masked]

>i have a few questions related to the new state machine proposed in the

Current draft-17 has some known problems.  I think Sue aleady prepared
draft of draft-18.  So best way is updating the draft to draft-18 then
process IDR working group review to it.

>re-establsihed, even if actually it only fails like every 2 hours.

My understanding is once BGP peer is establieshed, the ConnectRetryCnt
is cleared.
-- 
Kunihiro Ishiguro

Date: Fri, 05 Jul 2002 20:33:18 -0700
From: Kunihiro Ishiguro <[email-masked]>
To: "Fryczke, Pawel" <[email-masked]>
Cc: [email-masked]
Subject: Re: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Sender: [email-masked]

>The latest draft, draft-ietf-idr-bgp4-17.txt states:
>Routes that have the following attributes shall not be aggregated
>unless the corresponding attributes of each route are identical:
>MULTI_EXIT_DISC, NEXT_HOP.

Well, this comes from RFC1771 and we didn't change it.

9.2.4.2 Aggregating Routing Information

   Aggregation is the process of combining the characteristics of
   several different routes in such a way that a single route can be
   advertised.  Aggregation can occur as part of the decision  process
   to reduce the amount of routing information that will be placed in
   the Adj-RIBs-Out.

   Aggregation reduces the amount of information that a BGP speaker must
   store and exchange with other BGP speakers. Routes can be aggregated
   by applying the following procedure separately to path attributes of
   like type and to the Network Layer Reachability Information.

   Routes that have the following attributes shall not be aggregated
   unless the corresponding attributes of each route are identical:
   MULTI_EXIT_DISC, NEXT_HOP.

IHMO, the intention of this part is "BGP should aggregate the routes
which have same network reachability".  Diffrent MULTI_EXIT_DISC or
NEXT_HOP means different network reachability.  So the specification
said "shall not be aggregate".

>Many vendors (e.g. Cisco/Juniper) does not follow this requirement. Why?
>What is the purpose of this requirement?

Today's network operator take control the use of aggregation.  So even
if we aggregate diffrent network rechability routes, that should be
ok.
-- 
Kunihiro Ishiguro

Date: Fri, 05 Jul 2002 12:21:48 +0200
From: [email-masked]
To: BGP mailing list <[email-masked]>
Cc: Gilles Geerts <[email-masked]>
Subject: TCP state machine doubts
Sender: [email-masked]

all,

Thanks,
  Hans De Vleeschouwer.

From: "Fryczke, Pawel" <[email-masked]>
To: [email-masked]
Subject: Aggregation and MULTI_EXIT_DISC, NEXT_HOP
Date: Thu, 4 Jul 2002 14:08:02 +0200 
Sender: [email-masked]

The latest draft, draft-ietf-idr-bgp4-17.txt states:
Routes that have the following attributes shall not be aggregated
unless the corresponding attributes of each route are identical:
MULTI_EXIT_DISC, NEXT_HOP.

Many vendors (e.g. Cisco/Juniper) does not follow this requirement. Why?
What is the purpose of this requirement?
Pawel

Date: Wed, 03 Jul 2002 19:32:13 -0700
From: Kunihiro Ishiguro <[email-masked]>
To: Susan Harris <[email-masked]>
Cc: [email-masked]
Subject: BGP4-draft-18
Sender: [email-masked]

Hi, this is Kunihiro

Sue, when do we publish draft-18 of BGP4?  Well, some people start
using draft-17 with IdleHold status and claim that all of other
current implementation is wrong...  If you need any help or review or
some other things, please let me know.  I really want to update the
draft.
--
Kunihiro Ishiguro