Subject: FSM words for the draft-18

From: Bill Fenner <[email-masked]>
Date: Wed, 17 Jul 2002 23:13:59 -0700

Sue asked me to forward this since it didn't appear on the list.
Sorry if this causes duplicates.

  Bill

----- Begin forwarded message:

From: Susan Hares <[email-masked]>
Subject: FSM words for the draft-18
Date: Wed, 17 Jul 2002 23:24:40 -0400
To: [email-masked]
Cc: yakov Rekhter <[email-masked]>, [email-masked],
    [email-masked]

I would like to propose that section 8 be replaced with the
attached words.

Sue Hares

----- End forwarded message:

---

From: [email-masked]
Date: Thu, 18 Jul 2002 16:04:40 +0200

All,

I have reviewed the FSM document below, upto the Active state. I have added my
comments inline in the text.
If it felt that it makes sense to continue the review in this way, please let
me know.

Kind regards,
  Hans De Vleeschouwer.

Bill Fenner wrote:

>   ------------------------------------------------------------------------
>
>    FSM-words-05b.txtName: FSM-words-05b.txt
>                     Type: Plain Text (text/plain)
>
>   ------------------------------------------------------------------------
> General questions/doubts/remarks: 
>
> - Several versions of chapter 8 are floating around.
>   The version below was posted by Bill Fenner on July 17th.   
>    
> - I have placed comments inline in the text. The comment
>   lines start with a >. 
>
> - For timers I noticed that several syntaxes are being used.
>   e.g. 
>     Connect Retry Timer
>     ConnectRetry Timer
>     ConnectRetryTimer
>
>   As the second notation seems most used, I assumed that this was    
>   the correct notation.
>
> - for timers I suggest in the text to use consistently:
>   'start timer' and 'stop timer'. 
>    
> - Section 8.1 decribes all possible events. Would it not be good
>   Also to have a section describing the purpose of eah timer.
>
>

Note: (this is version 5 of the changes to
       the text.) 

8.0	BGP Finite State Machine

This section specifies the BGP operation in terms of a 
Finite State Machine (FSM).  The section falls into 2 
parts:

    1)	Description of Events for the State machine (section 
        8.1
    2)	Description of the FSM (section 8.2)

Session Attributes required for each connection are; 

   1)	state
   2)	Connect Retry timer

>>
> Naming Consistency:  ConnectRetry timer
>>

   3)	Hold timer
   4)	Hold time
   5)	Keepalive timer

>>
> Naming Consistency: KeepAlive timer
>>

8.1 Events for the BGP FSM

8.1.1   Administrative Events (events 1-5)

Please note that only Event 1 (manual start) and Event 2 
(manual stop) are mandatory administrative events. All 
other administrative events are optional.

 Event1: Manual start 

	   Definition: Administrator manually starts peer
                       connection
>>
> Syntax: Administrator -> Local system administrator
>         For consistency with Event2 below. 
>>

	   Status: 	Mandatory

 Event2: Manual stop

	   Definition: Local system administrator manually 
                       stops the peer connection.

	   Status: 	mandatory

 Event3: Automatic Start

 	   Definition: Local system automatically starts the 
                       BGP peer session

	   Status:     Optional depending on local system

 Event4: Manual start with passive Transport Establishment

	   Definition: Administrator manually start the peer
>>
> Syntax: Administrator maually start->
>          Local system administrator manually starts
>
>>
                       Connection, but has the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing the connection.

	   Status:     Optional depending on local system

 Event5: Automatic start with passive Transport establishment

	  Definition:  Local system automatically starts the
                       BGP session with the passive flag 
                       enabled.  The passive flag indicates 
                       that the peer will listen prior to 
                       establishing a connection.

	  Status:      Optional depending on local system use
		       of a passive connection.

 Event6: Automatic start with bgp_stop_flap option set

	   Definition: Local system automatically starts the 
                       BGP peer session with persistent peer 
                       oscillation damping enabled.  The exact 
                       method of damping persistent peer 
                       oscillations is left up to the 
                       implementation.   These methods of 
                       damping persistent BGP adjacency 
                       flapping are outside the scope of this 
                       document.

           Status: 	Optional, used only if the bgp peer has 
	  	        Enabled a method of damping persistent
		        BGP peer flapping.

 Event7: Auto stop

	   Definition: Local system automatically stops the
                       BGP peer session.  
	
	   Status:     Optional depending on local system

8.1.2 Timer Events (events 8-11)

 Event8:  idle Hold timer expires 

>>
> Naming Consistency: idle Hold timer --> IdleHold timer
>>

           Definition: Idle Hold timer expires.  The Idle 
                       Hold Timer is only used when persistent  
>>
> Naming Consistency: idle Hold timer --> IdleHold timer
>>

                       BGP oscillation damping functions are 
                       enabled.  

           Status: 	Optional.  Used when persistent
			BGP peer oscillation damping functions
			are enabled. 

 Event9: connect retry timer expires

>>
> Naming Consistency: connect retry timer --> ConnectRetry timer
>>
           Definition: An event triggered by the expiration of 
                       the ConnectRetry timer.
  
	   Status:     Mandatory

 Event10: hold time expires

	   Definition: An event generated when the HoldTimer 
                       expires.

>>
> Naming COnsistency: HoldTimer --> Hold timer
>>

	   Status:     Mandatory

 Event11: keepalive timer expires

>>
> Naming Consistency: Keepalive --> KeepAlive
>>

	   Definition: A periodic event generated due to the 
                       expiration of the KeepAlive Timer. 
 
	   Status:     Mandatory

 Event12: DelayBGP Open timer expires [changed]

>>
> NamingConsistency: DelayBGP Open -> DelayBGPOpen 
>>
	
	   Definition: A timer that delays sending of the BGP
	               Open message for n seconds after the
                       Transport connection has been completed. 

	   status:     Optional

8.2.3) Tranport Message based (13-16) 

 Event13: Transport Connection Indication & valid remote peer[changed]

	   Definition: Event indicating that transport
                       Request valid source IP address and TCP 
                       port, and valid destination IP address 
                       and TCP Port.  The definition of 
                       invalid Source, and invalid Destination 
                       IP address is left to the implementation.
                       BGP's destination port should be port 
                       179 as defined by IANA.

>>
> Comment:
>   If it is the intention of the text to allow BGP to use other
>   transport protocol besides TCP, then this text will probably 
>   need to be modified a bit in the lines of:
>
>           Event indicating that a transport connection is 
>           initiated by a peer with a valid source and 
>           destination address
>                      
>           The definition of valid source, and valid destination 
>           address is left to the implementation.
>                                              
>           If BGP is using TCP as transport protocol, the event 
>           is denoted by the local system receiving a TCP SYN 
>           on port  179 as defined by IANA from a peer, with a valid 
>           source and destination IP address, and a valid source
>            TCP source port.
>
>>

	   Status:     Mandatory

 Event14: RCV Transport Connection Indication and Invalid Source or 
          Destination [changed]

           Definition: Transport request received with either
 		       an invalid source address or port
		       number or an invalid destination 
                       address or port number. BGP destination 
                       port  number should be 179 as defined 
                       by IANA.

	    Status: 	 Mandatory
	
>>
> Same comment as for Event13. Here the text could become something like:
>
>                       Event indicating that a transport connection is 
>                       initiated by a peer with an invalid source and/or
>                       destination address
>                       
>                       The definition of invalid source, and invalid
>                       destination address is left to the implementation.
>                                              
>                       If BGP is using TCP a transport protocol, the event 
>                       is denoted by the local system receiving a TCP SYN 
>                       on port 179 with an invalid source IP address, 
>                       source TCP port, or destination IP address.
>>

 Event15: Transport Connection request sent received an ACK.

           Definition: Local system's request to establish a transport
		       Connection to the remote side received
		       an ACK.
>>
> Typo:  remote side received an ACK -> remote side, and received an ACK.
>>

	    Status:    Mandatory

 Event16: Transport Connection Confirmed [changed]

	   Definition: The local system has received a Confirmation that
                       the transport connection has been established by
		       the remote site. 

	  Status:      Mandatory

 Event17: Transport connection fails [changed]

           Definition: This BGP peer receives a transport
		       connection failure notice.  This
		       connection notice could be caused by a
		       Transport disconnect message or a
		       Timeout in the transport session.

                       If this connection is using TCP, the
		       remote BGP peer's TCP machine could
		       have sent a FIN.  The local peer would
		       respond with a FIN-ACK. Another
		       alternative is that the local peer
		       indicated a timeout in the TCP session
		       and downed the connection.
>>
> Comment: what about the TCP signal RST. Should this not also be
>          considered as an error?
>>

	    Status:    Mandatory

8.1.4 BGP Messages (18-27)

 Event18: BGPOpen

           Definition:  An event indicating that a valid Open 
>>
> Naming Consistency:  Open -> BGP Open
>>
			message has been received.

	   Status: 	Mandatory

 Event19: BGPOpen with BGP Delay Open Timer running [changed]

           Definition: An event indicating that a valid Open 
                       Message has been successful 
                       established for a peer that is 
                       currently delaying the sending of an 
                       BGP Open message. 
>>
> 
>"...for a peer that is currently delaying the sending of an 
> BGP Open message." 
>
> In fact I guess it is not the peer who is delaying the sending of 
> the BGP Open, but the local system. proposed rephrase"
>
>"...for a peer for which the local peer is currently delaying the
>    sending of a BGP Open message." 
>
>>

	    Status:    Optional

 Event20: BGPHeaderErr

	   Definition: BGP message header is not valid.  

	   Status:     Mandatory

 Event21: BGPOpenMsgErr 
	   Definition: An BGP Open message has been received 
                        with errors. 
 
	   Status: 	Mandatory

 Event22: Open collision discard

           Definition: An event generated administratively 
                       when a connection Collision has been 
                       detected while processing an incoming  
                       Open message. This connection has been 
>>
> Naming Consystency: Open message -> BGP Open message
>>
                       selected to disconnected.  See section 

>>
> Typo: selected to disconnected -> selected to be disconnected.
>>

                       6.8 for more information on collision
	               detection.  

                       Event 22 is an administrative could 
                       occur if FSM is implemented as two 
                       linked state machines.  

>>
> Typo:  Event 22 is an administrative could occur if FSM -> 
>        Event22 is an administrative event that could occur if the FSM
>>

	  Status:      Optional

 Event23: NotifMsgVerErr

	    Definition: An event is generated when a 
                        NOTIFICIATION message with "version   
                        error" is received.

	    Status:     Mandatory

 Event24: NotifMsg

	    Definition: An event is generated when a 
                        NOTIFICATION messages is received and
                        the error code is anything but 	 
			"version error".

	     Status:	Mandatory
		        
 Event25: KeepAliveMsg

	    Definition: An event is generated when a KEEPALIVE
			Message is received.

		Status: Mandatory

 Event26: UpdateMsg

	    Definition: An event is generated when a valid 
			Update Message is received.

		Status: Mandatory

 Event27: UpdateMsgErr

	    Definition: An event is generated when an invalid
			Update message is received.

		Status:	Mandatory

8.2) Description of FSM

Idle state

   Initially BGP is in the Idle state.   

>>
>
> In fact BGP itself is not in the Idle state, but the FSM for a given 
> peer. Therefore I would suggest a rephrase to:
>
>   Initially the BGP FSM for a peer is in the Idle state. 
>
>
>>

   In this state BGP refuses all incoming BGP connections.  No 
   resources are allocated to the peer.    In response to a 
   manual start event(Event1) or an automatic start 
   event(Event3), the local system 
      -	initializes all BGP resources, 

>>
>
> The text says that "No resources are allocated to the peer. "
> Therefore i would suggest to rephrase this bullet as:
>
>     - allocates and initializes all BGP resources,
>
>  This also matches better with the text furtheron where 
>  sometimes I read " release all BGP resources and return to Idle state"
>>

      -	sets the Connect retry counter to zero
      -	starts the ConnectRetry timer with initial value, 
      -	initiates a transport connection to the other BGP 
        peer, 
      -	listens for a connection that may be initiated by 
        the remote BGP peer, and
        [Action A in the FSM table]
      -	changes its state to connect.

>>
>1. Naming Consistency:
>      -	sets the ConnectRetryCnt to zero
>      -	changes its state to Connect.
>
>
>2. I see nowhere a place where the value of the ConnectRetry timer
>   is changed. Is then not enough to say
>     - start the ConnectRetry timer.
>>

  An manual stop event (Event2) is ignored in the Idle state.

  In response to a manual start event with the passive Transport
  flag (Event 4) or automatic start with the passive Transport flag 
  (Event 5), the local system:
      -	initializes all BGP resources,
>>
> Same remark:  suggested new text:
>  
>     - allocates and initializes all BGP resources,
>>

      -	sets the connect retry count to zero,
      -	start the Connect Retry timer with initial value, 
>>
> Naming Consistency: 
>      -	sets the ConnectRetryCnt to zero,
>      -	start the ConnectRetry timer with initial value, 
>>

      -	listens for a connection that may be initiated by 
        the remote peer
        [Action B in the FSM table] 
      -	changes its state to Active.

  The exact value of the ConnectRetry timer is a local 
  matter, but it should be sufficiently large to allow TCP 
  initialization.   

  If a persistent BGP peer oscillation damping function is 
  enabled, two additional events may occur within Idle state:  
      -	Automatic start with bgp_stop_flap set [Event6],
      -	Idle Hold Timer expired [Event 8].

  The method of preventing persistent BGP peer oscillation is 
  outside the scope of this document.

  Any other events [Events 9-27] received in the Idle state, 
  are noted by the MIB processing as FSM Errors [action V]
  and the local peer stays in the Idle State.

Connect State:

  In this state, BGP is waiting for the transport protocol 
  connection to be completed.

  If the transport connection succeeds [Event 15 or 
  Event 16], the local system checks the "Delay Open 
  Flag".  If the Delay Open flag is set, the local system: 
     -	clears the ConnectRetry timer, 
     -	set the BGP Open Delay timer to the initial 
        value.

>>
> Naming Consistency:
>    
>   - stop the ConnectRetry timer, 
>   - start the BGPOpenDelay timer with its initial value
>
>
>>

        [Action ZZ in FSM table]

  If the Delay Open flag is not set, the local system: 
     - clears the ConnectRetry timer, 
     - completes BGP initialization
     - send an Open message to its peer,
     - sets Hold timer to a large value,
>>
> Naming Consistency:
>
>    - stop the ConnectRetry timer,
>    - send a BGP Open message to its peer,
>    - start the Hold timer with a large value
>
>>

     - Change the state to Open Sent 
   [Action H in the FSM table]

>>
> In all other cases the state change is not part of the action itself.
> Should therefore the order not be reversed to:
>
>  [Action H in the FSM table], and
>  - Change the state to Open Sent 
>   
>>

  A hold timer value of 4 minutes is suggested.

>>
> Question:
>The text above assumes the same behavior for both event 15 and 16. In 
>reality I  wonder whether most implementations would not use 2 linked
> FSM machines for this. In this case a scenario could be:
>- if 15 arrives: continue as described in the text;
>- if 16 arrives: As this is in fact a new connection that comes in 
>orginated by the remote peer, create a new FSM.
>  The new FSM then eithers starts the delay timer and goes into connect 
>state, or sends an OPEN message and goes into OPEN sent state.
>
>This behavior is it seems also more in line with section 6.8 of the BGP RFC 
>on collisions.  
>
>Would it make sense to work this out any further?
>
>>

  If the Open Delay timer expires [Event 12] in the connect 
  state,
>>
> Naming consistency BGPOpenDelay
>>

     - send an Open message to its peer,
     - set the Hold timer to a large value,
>>
> Naming consistency:
> - start the Hold timer with a large value
>>

	[Action H in the FSM Table], and
     - change the state to Open Sent. 

  If the BGP port receives a Transport connection indication 
  [Event 13], the Transport connection is processed (actions AA or
  AB in the FSM table) and the connection remains
  in the connected state. 

>>
> If it is the intention to make this text independent of the Transport
> protocol, it might be better to remove the term port:
>
>    If a Transport connection indication [Event13] is received, the event 
>    is processed (actions AA orAB in the FSM table) and the connection
>    remains in the Connect state. 
>>

  If the transport connection receives an Transport indication
  that is invalid or unconfigured. [Event 14]:
     - the transport connection is rejected.
	[Action L in the FSM table]
>>
> This seems to me like a strange case. We are in the Connect state for
> a session. The session is identified sofar with a source and destination 
> IP address and (in case of TCP the correct TCP port). 
>
> If a Transport indication would come in that either has invalid info
> or which is not configured, I can not really see how this event could be
> 'dropped' inside this FSM instance. I would assume that this request
> would be treated outside the scope of any peer FSM, in the
> 'BGP-main-session'
>
>>

  If the transport connection fails (timeout or transport
  disconnect) [Event17], the local system:
      - restarts the ConnectRetry timer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
     [Action G in the FSM table]
      - changes its state to Active. 

  If an Open is received with the BGP Delay Open timer is 
  running [Event 19], the local system:
	- clears the connect retry timer (cleared to zero),
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message

>>
>
> 1. I think an action is missing here: The setting of the Hold timer.
>
> 2. For naming consistency: 
>
>       - stops the ConnectRetry timer
>	- completes the BGP initialization, 
>	- Stops the BGPOpenDelay timer
>	- Sends an Open message
>       
>     ? - set the hold timer to a large value?
>>

	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 The start events [Event 1, 3-6] are ignored in connect 
 state.

 A manual stop event[Event2], the local system:
       - drops the transport connection, 
       - releases all BGP resources,
       - sets connect retry count to zero
       - resets the connect retry timer (sets to zero)
      [Action Z in the FSM table]
       - goes to Idle state.

>>
>  Would it not be enough to say:
>
>  If a manual stop event[Event2] is received, the local system:
>    - drops the transport connection, 
>    - stops all running timers,
>    - releases all BGP resources,    
>      [Action Z in the FSM table], and
>    - goes to Idle state.
>
>   Note that it is indicated in the idle state that
>   - the connct retry count is set to 0; and
>   - the ConnectRetry timer is started with its inital value.
>
>   Also note that I moved the clearing of the resources to the end
>   as, I guess , resources are needed to clear the timers.
>
>>

  In response to the ConnectRetry timer expired event(Event 
  9), the local system: 
      - Sets the MIB FSM error information with ConnectRetry 
        expired, 
      - drops the transport connection 
      - restarts the ConnectRetry timer
      - initiates a transport connection to the other BGP 
        peer, 
      - continues to listen for a connection that may be 
        initiated by the remote BGP peer, and
      [Action O in the FSM table]
      - stays in Connect state.

 In response to any other events [Events 7-8, 10-11, 18, 20-
 27] the local system:

     - resets the connect retry timer (sets to zero),
     - drops the Transport connection,
     - release all BGP resources, 
     - increments the ConnectRetryCnt by 1,
     - [optionally] performs bgp peer oscillation damping, and
     [Action D in the FSM table],
     - goes to Idle state.

>>
> 
> Note that in the Idle state 'all resources' for this peer will
> be re-allocated and initialised.
> 
> So, with the current definition of the Idle state you will not be
> able to increment the connectRetryCnt if you move into Idle state.
> A solution might be to create a new init-state where resources are
> allocated an intialised, and which is then braught into idle state.
> In this way the resources intialisation can be removed from the Idle
> state and hence the ConnectRetryCnt can be kept.
>
>>

Active State:

 In this state BGP is trying to acquire a peer by listening 
 for and accepting a transport protocol connection.  

 A transport connection succeeds [Event 15 or Event 16], the 
 local system: process the transport connection flags
	- If the BGP delay open flag is set:
          o clears the ConnectRetry timer, 
          o completes the BGP initialization,
          o sets the BGP delay Open timer
         [Action ZZ]
>>
>
>Naming consistency:
>          o stops the ConnectRetry timer, 
>          o completes the BGP initialization,
>          o starts the BGPOpenDelay timer
>
>> 

        - If the BGP delay open flag is not set:
          o clears the ConnectRetry timer,
          o completes the BGP initialization, 
          o sends the Open message to it's peer, 
          o sets its Hold timer to a large value, 
         [Action H in the FSM table]
          and changes its state to OpenSent.  

>>
>Naming consistency
>
>          o stops the ConnectRetry timer,
>          o sends the BGP Open message to it's peer, 
>          o start the Hold timer with a large value, 
>>

 A Hold timer value of 4 minutes is suggested.  

 If the local system receives a valid Transport Indication
 [Event 13], the local system processes the Transport flags
  (actions aa or ab in FSM section 2.3.4).  

>>
>
> This way of describing actions, means that the complete sepcification
> for basic BGP is not longer complete without the FSM document.
>
> Is this the way the RFC is intended to evolve?
>>

 If the local system receives a Transport indication
 that is invalid for this connection [Event 14]:
      - the transport connection is rejected
        [Action L in the FSM table]

>>
> Same comment as for [event 14] in the previous state.
>>

 If the local system receives a Transport connection
 failed [Event 17] (timeout or receives Transport
 disconnect), the local system will:
	- set Transport disconnect in the MIB reason code,
	- restart ConnectRetry timer (with initial value)
	- release all BGP resources
	- Acknowledge the drop of Transport connection if
          transport disconnect (If TCP, send a FIN ACK),
	- Increment ConnectRetryCnt by 1,
	- perform the BGP peer oscillation damping process [2]
	[Action Y in the FSM table]

>>
>1.  To what state is the FSM going here? As all BGP resources are released
>    it seems to me that we can do little more then to go to Idle state?
>
>2. In case of TCP a RST signal is also possible I guess. In this case
>   no FIN ACK is needed.
>
>> 

 If the local system has the Delay Open timer expired [event 
 12] local system: 
        - clears the Connect Retry timer (set to zero),
	- stops and clears the Delay Open timer (set to zero) 
        - completes the BGP initialization,
        - sends the Open message to it's remote peer,
        - sets it's Hold timer to a large value,
        [Action H in the FSM table]
        - and set the state to Open Confirm.

>>
> 1. Naming consistency (not repeated in detail hereafter.
>
> 2. Should we not move to the state Open Sent? In the State Open Confirm
>    the OPEN message from the peer will not longer be accepted.
>>

 A Hold timer value of 4 minutes is also suggested for this 
 state transition.

 If an Open is received with the BGP Delay Open timer is 
 running [Event 19], the local system 
	- clears the connect retry timer (cleared to zero),
	- Stops and clears the BGP Open Delay timer  
	- completes the BGP initialization, 
	- Stops and clears the BGP Open Delay timer
	- Sends an Open message
	- Set the Hold timer to a large value (4 minutes),
	[Action H in the FSM table], and
	- changes its state to Open Confirm.

 In response the ConnectRetry timer expired event[Event9], 
 the local system:
        - restarts the ConnectRetry timer (with initial value),
        - initiates a transport connection to the other BGP 
          peer,
        - Continues to listen for transport connection that may be 
          initiated by remote BGP peer, 
        [Action F in the FSM table]
	- and changes its state to Connect.  

 The start events [Event1, 3-6] are ignored in the Active 
 state.

 A manual stop event[Event2], the local system:
        - Sets the administrative down in the MIB reason code,
	- Sends a notification with a Cease,
	- If any BGP routes exist, delete the routes
	- release all BGP resources, 
	- drops the Transport connection, 
        - sets connect retry count to zero
        - resets the connect retry timer (sets to zero)
        [Action S in the FSM table]
        - goes to Idle state.

 In response to any other event (Events 7-8, 10-11,18, 20-
 27), the local system:
        - stores the MIB information to indicate appropriate 
          error [FSM for Events 7-8, 10-11, 18, 20-27]
        - reset the connect retry timer (sets to zero),
        - release all BGP resources, 
	- drops the transport connection, 
        - increments the ConnectRetryCnt by one,
        - optionally performs BGP peer oscillation damping, 
  	  [Action D in FSM table],
	- and goes to the idle state