Subject: Re: bgp18 WG Last Call fsm missing next state

From: "Tom Petch" <[email-masked]>
Date: Mon, 2 Dec 2002 16:06:30 -0000

Jeff

I appreciate your comments, agreeing with many.

A minor supplementary; Open Confirm, valid open [event 18], decide to
drop this connection and so the FSM but what about the actions of
incrementing the ConnectRetryCnt (and optional peer oscillation
damping)?  Are we doing that in the other FSM?  on a per router id or
per IP address pair independent of FSM?

And a slightly bigger issue in the Active state (which is the passive
state because we have entered it from Idle with Passive TCP flag set
Events 4/5).  In which case, TCP Indication valid [event 13] or
invalid [event 14] are what I would expect with event 14 keeping us in
Active state; and if event 13 also keeps us in Active state (while
sending a TCP SYN-ACK) then TCP Connection Confirmed [event 16] will
ensue while TCP Connection Acknowledged [event 15] would be an FSM
error.

Except that you suggest that event 14 in the Connect state takes us to
Active state which seems fine but for the fact that we will have sent
a TCP SYN (because that is what the Connect state is), this invalid
Indication may be a coincidence, in which case our SYN is about to
receive a SYN-ACK back ie event 15 is valid after all.  And I am then
in agreement with you over events 15/16 in the Active (ie passive)
state (delay open flag keeps us in the Active state).

Tom Petch
[email-masked]

-----Original Message-----
From: Jeffrey Haas <[email-masked]>
To: Tom Petch <[email-masked]>
Cc: idr <[email-masked]>
Date: 30 November 2002 01:33
Subject: Re: bgp18 WG Last Call fsm missing next state

>Tom,
>
>On Thu, Nov 28, 2002 at 02:53:56PM -0000, Tom Petch wrote:
>> In seven places in the FSM, the actions listed for the occurrence
of
>> an event do not include any indication of what the next state
should
>> be, nor can I be sure I can guess what it should be.
>>
>> I believe the FSM should specify the new state in the cases listed
>> below.
>>
>>       Connect State:
>>
>>         If the TCP connection succeeds [Event 15 or
>>         Event 16], the local system checks the "Delay Open
>>         Flag".  If the delay Open flag is set, the local system:
>> **enters what state
>
>Remains in the connect state awaiting either an open message or
>for the opendelay timer to expire.
Agreed
>
>>         If the TCP connection receives an indication
>>         that is invalid or unconfigured. [Event 14]:
>> **enters what state
>
>Enters the active state.
>Connectretrytimer should be started.
Agreed
>
>>       Active State:
>>
>>         A TCP connection succeeds [Event 15 or Event 16], the
>>         local system: process the TCP connection flags
>>          - If the BGP delay open flag is set:
>> ** enters what state (I think this is an FSM error in TCP because
it
>> has not initiated a connection!)
>
>In the active state, we are awaiting incoming tcp connections to be
>completed.  Here we should:
>
>remain in the active state
>wait for an open message or the opendelay timer to expire.
>
>>         If the local system receives a valid TCP Indication
>>         [Event 13], the local system processes the TCP connection
>> flags.
>> ** enters what state
>
>I'm also confused by this event.  Since we are not initiating a TCP
>connection, but are awaiting one to complete, why would we get an
>Indication here?  FSM error?
>
>>         If the local system receives a TCP indication
>>         that is invalid for this connection [Event 14]:
>> ** enters what state
>
>Ditto.
>
>>
>>         If the local system receives a TCP connection
>>         failed [Event 17] (timeout or receives connection
>>         disconnect), the local system will:
>> ** enters what state
>
>Ditto.
>
>>       Open Confirm State:
>>
>>         If the Open messages is valid [Event 18], the collision
>>         detect function is processed per section 6.8.  If this
>>         connection is to be dropped due to call collision, the
>>         local system:
>> ** enters what state
>
>This instance of the FSM should be discarded since it implies there
is
>another FSM instance running that will proceed to Established?
>
>> Tom Petch
>> [email-masked]
>
>--
>Jeff Haas
>NextHop Technologies

---

From: "Susan Hares" <[email-masked]>
Date: Wed, 4 Dec 2002 21:17:46 -0500

Tom and Jeff:

Thanks for the great review of the state machine. 
Please confirm I've got my 7 changes numbered right.

Here's where we have consensus:
	change 1 - consensus, editorial
		state: Connect
		Events: 15 and 16

	change 2 - no consensus
		 I've given two alternatives.
		 We need to get implementers to pick 1.
		 thread: 
		State: Connect
		Event: 14

	change 3 - consensus, editorial
		state: Active:
		Events: 15 and 16

	change 4 and 5:
		State: Active
		Events: 13 and 14

	On changes 4 and 5 - moved to a different thread
	thread titled:  fsm missing next state - Events 13 - 17 and the TCP connection   

	on change 6 - no consensus
		State: Active
		Event: 17
	
		Review my explanation and the new text, and
		see if we have consensus.

	on change 7: no consensus.
		State: Open Confirm
		Event: 18

		Review my explanation and see if we have
	      consensus.

Sue Hares
([email-masked]) 
=============

Warning - put the screen on wide mode for the following text: 

-------------------

Here's my response on the 7 changes

1) Connect State - Event 15 or Event 16 [consensus, editorial]

note:  The delay retry timer is utilized instead of the
	 connect retry timer for the next two changes. 

Previous text:
 If the TCP connection succeeds [Event 15 or Event 16], 
 local system checks the "Delay Open Flag".  If the delay
 open flag is set, the local system:
	- clears the connect retry timer,
	- sets the BGP open delay timer to initial value

   If the Delay Open flag is not set, the local system:
	- clears the connect retry timer,
	- clear BGP Open Delay timer (set to zero), 
	- completes the BGP initialization,
	- send an Open message to its peer,
	- sets hold timer to a large value, and
	- change the state to Open Sent.

New text: 
 If the TCP connection succeeds [Event 15 or Event 16], 
 local system checks the Delay Open flag prior to
 processing:   If the Delay Open flag is set, the local system:
	- clears the connect retry timer,
	- sets the BGP open delay timer to initial value, and
	- stays in the Connect state.

 If the Delay Open flag is not set, the local system:
	- clears the connect retry timer,
	- clears the BGP Delay timer (sets to zero), 
	- completes the BGP initialization,
	- sends an Open message to its peer,
	- sets the hold timer to a large value, and
	- changes the state to Open Sent. 

2)Connect State - Event 14  [no consensus] 

Current Text:
	If the TCP connection receives an indication that
	that is invalid or unconfigured [Event 14], 
	- the TCP connection is rejected.

At the very least this section needs more "word smithing",
so I'd like to change it for more clarity at least.

I'm not sure this represents the implementations.
What I'd like to do is query the implementations
to see what they do if they receive a valid TCP
connection with an invalid or unconfigured peer.

Two options:

Alternative 1: Count it as a valid response

 New Text: If a TCP connection is received that has
	     an invalid format, or an unconfigured host [Event 14],
	     the local system:
		- rejects the TCP connection,
		- increments the connect retry counter,
		- performs bgp peer oscillation checks.
	
		If bgp peer oscillation checks allow for a new
		connection, the bgp peer
		- restarts the Connect retry timer with configured 
		  value, and
		- enters the Active state. 

FSM table:
           Idle   Connect Active Open-Sent Open-Confirm Establish
Event-14   =======================================================
Next state Idle  | Active|Active|Open-Sent|Open-Confirm|Establish|  
	     =======================================================
 action       V  | Y2    | L    | Ignore  | Ignore     | Ignore  |
           =======================================================

Alternative 2: Reject the connection and see if valid or
		   configured one appears within the 
		   connect timer window. 

New Text: If a TCP connection is received that has an invalid format,
	    or an unconfigured host [Event 14], the local system:
		- rejects the TCP connection,
		- and stays in the Connect state. 

FSM table:
           Idle   Connect Active Open-Sent Open-Confirm Establish
Event-14   =======================================================
 Nxt state Idle  |Connect|Active|Open-Sent|Open-Confirm|Establish|  
	     =======================================================
 action       V  | L    | L     | Ignore  | Ignore     | Ignore  |
	     =======================================================
  
3) Active State - Event 15 or Event 16 [consensus, editorial]

Previous text: 
 A TCP connection succeeds [Event 15 or Event 16], 
 the local system: process the TCP connection flags 
 - If the BGP delay open flag is set:
	- clears the connect retry timer

   [through the following text:
 - and changes its state to Open Sent.

New text: 
 If the TCP connection succeeds [Event 15 or Event 16], 
 local system checks the "Delay Open Flag" prior to
 processing:   If the delay open flag is set, the local system:
	- clears the connect retry timer,
	- sets the BGP open delay timer to initial value, and
	- stays in the Active state.

 If the Delay Open flag is not set, the local system:
	- clears the connect retry timer,
	- clears the BGP Delay timer (sets to zero),  
	- completes the BGP initialization,
	- sends an Open message to its peer,
	- sets the hold timer to a large value, and
	- changes the state to Open Sent. 
  
4) Active State, Event 13 [no consensus] 
5) Active State, Event 14 [no consensus] 

  The problem with this state is it is difficult to
  exactly specify without discussing the TCP
  Messages that FSM document covers.
  
  I'll query if the implementors require all
  of events 13-17 as mandatory. 
  
  We'll 

6) Active State, Event 17 [no consensus] 

comment: 
In the Active state, we may already have a connection and be awaiting
the Open Delay timer.  The TCP disconnect or timeout could occur in this
state due to the "Open Delay Timer".   If the TCP Disconnect is ignored,
we could have some peer oscillation. 

If the we wait, then the connection retry timer needs to be kept running.
The text below allows this timer.  The real question is what is the status
of the current implementations.  

I agree, the Active state and the connect state should match.  

Old Text:
	If the TCP connection fails (timeout or disconnection) [Event 17],
	the local system:
		- set TCP disconnect in the MIB reason code,
		- restart Connect retry timer (with initial value),
		- release all BGP resources,
		- Acknowledge the Drop of the TCP connection if TCP disconnect (FIN ACK),
		- Increment ConnectRetryCnt (connect retry count) by 1, and
		- performs the BGP peer oscillation damping process.  

Applicable FSM State table:

FSM table old: 

 Event 17
 current:  Idle   Connect Active Open-Sent Open-Confirm Establish
           =========================================================
Next state Idle  |Active |Idle   |Active | Idle       |Idle       |  
		     |       |       |       |            |           |
	     =========================================================
 action       V  | Y2     | G     | Ignore| Track 2nd  | Track 2nd |
                 |       |       |       | connection | connection|
	     =========================================================

Alternative 1: 

FSM table new: 

 Event 17
 current:  Idle   Connect Active Open-Sent Open-Confirm Establish
           =========================================================
Next state Idle  |Active |Active   |Active | Idle       |Idle       |  
		     |       |        |       |            |           |
	     =========================================================
 action       V  | G     | G      | Ignore| Track 2nd  | Track 2nd |
                 |       |        |       | connection | connection|
	     =========================================================

G:  The local system:
	- restarts the connect retry timer (at intial value),
	- continues to listen for a connection that may be initiated
	  by the remote peer, and
	- sets its next state to Active. 

New Text: (for Connect and Active state)
		If the TCP connection fails (timeout or disconnect) 
		[Event 17], the local system:
		- restarts the connect retry timer,
		- continues to listen for a connection that may be 
		  initiated by the remote BGP peer, and
		- changes it state to Active.

Alternative 2: 
FSM table new: 

 Event 17
 current:  Idle   Connect Active Open-Sent Open-Confirm Establish
           =========================================================
Next state Idle  |Idle   |Idle   |Active | Idle       |Idle       |  
		     |       |       |       |            |           |
	     =========================================================
 action       V  | Y2    | Y2    | Ignore| Track 2nd  | Track 2nd |
                 |       |       |       | connection | connection|
	     =========================================================

Next Text: 
	If the location system receives a TCP connection failed [Event 17],
	the local system will:
		- increment the ConnectRetrycnt (connect retry count) by 1,
		- release all BGP resources associated with this connection,
		- perform BGP peer oscillation (if configured), and
		- go to Idle 

Y2 - is:
	The local system: 
		1) increments the ConnectRetryCnt (connect retry count) by 1,
		2) releases all BGP resources associated with this connection, and
		3) performs the BGP peer oscillation damping process

	if the damping process allows for a new connection, the local system
		- restarts the connect retry timer (with initial value, and 
		- goes to Idle

	If the damping process does not allow for a new connection, the local
	system
		- set the flags to damp the creation of a new bgp connection
	        until a manual start occurs, and
		- goes to Idle.

7) Open Confirm state, Event 18

Here's my proposed text. Please let me know what you think.
I think this is an editorial change.

Old text:  If the open message is valid, the collision detect 
           function is processed per section 6.8.  If this
           connection is to be dropped due to call collision, the local
	     system:
		- sends a Notification with a Cease
		- resets the Connect timer (to zero),
		- releases all BGP resources,
		- Drop the TCP connection (sends a TCP FIN),
		- increments the ConnectRetryCnt by 1 (connect retry count), and
		- performs an BGP peer oscillation damping process.

New text:  If the open message is valid, the BGP peer connection
	     is check to detect a collision per section 6.8.  If this
           connection is to be dropped due to call collision, the local
	     system:
		- sends a Notification with a Cease
		- resets the Connect timer (to zero),
		- releases all BGP resources,
		- Drop the TCP connection (sends a TCP FIN),
		- increments the ConnectRetryCnt by 1 (connect retry count), and
		- performs an BGP peer oscillation damping processing, and
		- enters the Idle State. 

	notes: Collision detect impacts Open Sent, Open Confirm, and
		 Established states. 

Applicable FSM state

> This instance of the FSM should be discarded since it implies there is

-----Original Message-----
From: Tom Petch [mailto:[email-masked]]
Sent: Monday, December 02, 2002 11:07 AM
To: Jeffrey Haas
Cc: idr
Subject: Re: bgp18 WG Last Call fsm missing next state

Jeff

Tom Petch

-----Original Message-----
From: Jeffrey Haas <[email-masked]>
To: Tom Petch <[email-masked]>
Cc: idr <[email-masked]>
Date: 30 November 2002 01:33
Subject: Re: bgp18 WG Last Call fsm missing next state

of
should

Agreed

Agreed

it

>Indication here?  FSM error?

We would get an indication from the remote side that it had
send us a TCP Syn.  The real question is 

is

---

From: "Tom Petch" <[email-masked]>
Date: Thu, 5 Dec 2002 22:14:46 -0000

Sue, Jeff
Agree with 1 (with a slight change of words put **inline to make the
name the Open Delay timer as Yakov accepted
2 wait on implementors -  but do we need a then clause for when peer
oscillation damping does not allow a new connection in alternative
1(also flagged inline)?
agree with 3
4,5 on a different thread
6 as you say, it depends on the implementations; I always like
failures such as Event 17 to take us to Idle so I hope the answer is
alternative 2
7 I am still struggling with; we are in OpenConfirm so we already have
received an Open from the remote peer and Event 18 is a second Open
from the same peer.  Perhaps my struggle is that I think in terms of
two (or more) FSM for a given IP address pair so the Open Collision
detection will occur when the/an- other FSM receives a valid Open in
states Active/Connect/Open Sent and will generate Event 22 into this
FSM so Event 18 cannot occur.  But yes, if Event 18 can occur in this
FSM and this connection is to be dumped, then Idle state it should be
as you suggest.  I have slotted in [optionally] in front of the peer
oscillation damping in your text because I think it should be
optional:-)
Collection collision is simple until I think about it :-(

Tom Petch

-----Original Message-----
From: Susan Hares <[email-masked]>
To: Tom Petch <[email-masked]>; Jeffrey Haas
<[email-masked]>
Cc: idr <[email-masked]>
Date: 05 December 2002 02:17
Subject: RE: bgp18 WG Last Call fsm missing next state

Tom and Jeff:

change 2 - no consensus

change 4 and 5:
State: Active
Events: 13 and 14

On changes 4 and 5 - moved to a different thread
thread titled:  fsm missing next state - Events 13 - 17 and the TCP
connection

Sue Hares
([email-masked])
=============

-------------------

Previous text:

New text:
- sets the BGP** Open Delay timer to initial value, and

- clears the BGP**Open Delay timer (sets to zero),

**tp agreed with changes to make it Open Delay timer

Current Text:

Two options:

**tp  need a then clause here 'if bgp peer oscillation damping does
not allow for a new connection, then the local system ???'

FSM table:

FSM table:

Previous text:

New text:

  We'll

comment:
In the Active state, we may already have a connection and be awaiting
the Open Delay timer.  The TCP disconnect or timeout could occur in
this
state due to the "Open Delay Timer".   If the TCP Disconnect is
ignored,
we could have some peer oscillation.

If the we wait, then the connection retry timer needs to be kept
running.
The text below allows this timer.  The real question is what is the
status
of the current implementations.

Old Text:
- Acknowledge the Drop of the TCP connection if TCP disconnect (FIN
ACK),

FSM table old:

Event 17

Alternative 1:

FSM table new:

Event 17

G:  The local system:

Alternative 2:
FSM table new:

Event 17

Next Text:

Y2 - is:
The local system:

local

local
- **[optionally] performs an BGP peer oscillation damping processing,
and
- enters the Idle State.

Applicable FSM state

is

-----Original Message-----
From: Tom Petch [mailto:[email-masked]]
Sent: Monday, December 02, 2002 11:07 AM
To: Jeffrey Haas
Cc: idr
Subject: Re: bgp18 WG Last Call fsm missing next state

Jeff

Tom Petch

-----Original Message-----
From: Jeffrey Haas <[email-masked]>
To: Tom Petch <[email-masked]>
Cc: idr <[email-masked]>
Date: 30 November 2002 01:33
Subject: Re: bgp18 WG Last Call fsm missing next state

of
should

Agreed

Agreed

it

is

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 5 Dec 2002 19:00:13 -0500

Tom:

Thanks for the input.

1 - done  
2 - sent to the list for implementations
3 - done, and question about
	     1 addition.

4,5 - I'll look for different thread

6 - sent to list for implementations
    I'm happy with alternative 2 if
    no one else works on it.

7 - this mechanism allows a single fsm to
    handle both.  2 fsm and 1 fsm BGP FSM
    seem to exist.  (I queried implementors
    a few times on this one.  So, I just
    put in this change to provide the 
    flexibility.  

	Collision detect tends to give
	scrambled brains for most people..
	As Dennis Ferguson said 2 years ago,
	that's the hardest part of the FSM.

Soo.. bottom line.  I'll look to see if
any implementors work on 2, 6 or 7.
I'll query a few of the implementors in
a questionnaire as well.

Thanks for the hard work!

Sue

3's question
---------
	Should I add completes BGP initialization in
	the following list:

If the TCP connection succeeds [Event 15 or Event 16],
local system checks the Delay Open flag prior to
processing:   If the Delay Open flag is set, the local system:
- clears the connect retry timer,
%- sets the BGP** Open Delay timer to initial value, and
%- complets the BGP initialization 
- stays in the Connect state.

-----Original Message-----
From: Tom Petch [mailto:[email-masked]]
Sent: Thursday, December 05, 2002 5:15 PM
To: Susan Hares; Jeffrey Haas
Cc: idr
Subject: Re: bgp18 WG Last Call fsm missing next state

Sue, Jeff

	- done

	- I think I sent out the query to the implementors. 
	  I'll resend to the list. 

agree with 3

Tom Petch

-----Original Message-----
From: Susan Hares <[email-masked]>
To: Tom Petch <[email-masked]>; Jeffrey Haas
<[email-masked]>
Cc: idr <[email-masked]>

Tom and Jeff:

change 2 - no consensus

change 4 and 5:
State: Active
Events: 13 and 14

Sue Hares
([email-masked])
=============

-------------------

Previous text:
- sets the BGP Open delay timer to initial value

New text:

Current Text:

Two options:

FSM table:

FSM table:

Previous text:

New text:

  We'll

comment:

Old Text:

FSM table old:

Event 17

Alternative 1:

FSM table new:

Event 17
current:  Idle   Connect Active Open-Sent Open-Confirm Establish
           ========================================================
Next state Idle  |Active |Active  |Active | Idle       |Idle      |
         |       |        |       |       |            |          |
     ==============================================================
action        V  | G     | G      | Ignore| Track 2nd  | Track 2nd |
                 |       |        |       | connection | connection|
     ===============================================================

G:  The local system:

Alternative 2:
FSM table new:

Event 17

Next Text:

Y2 - is:
The local system:

Applicable FSM state

is

-----Original Message-----
From: Tom Petch [mailto:[email-masked]]
Sent: Monday, December 02, 2002 11:07 AM
To: Jeffrey Haas
Cc: idr
Subject: Re: bgp18 WG Last Call fsm missing next state

Jeff

Tom Petch

-----Original Message-----
From: Jeffrey Haas <[email-masked]>
To: Tom Petch <[email-masked]>
Cc: idr <[email-masked]>
Date: 30 November 2002 01:33
Subject: Re: bgp18 WG Last Call fsm missing next state

of
should

Agreed

Agreed

it

is