Subject: Re: Evaluation of Gateway Location proposals

From: Matt Squire <[email-masked]>
Date: Thu, 29 Apr 1999 13:45:55 -0400

>
>2. Matt raised numerous transport issues (TCP or not TCP). I think its
>fair to say we want to avoid designing a new transport protocol. There
>is work underway elsewhere (sigtran and slums) to do this, which we can
>leverage. It would seem to me that if something new is needed, this
>would also be independent of whether the protocol baseline is BGP or
>SCSP, since presumably one could use this new transport protocol with
>either.
>

I certainly agree that we don't want a new transport protocol.  But well
short of designing a new transport is to design reliability into our
application protocol (ie with acks, timeouts, re-transmissions, etc), just
like most other application protocols.  

>3. To me, the MAIN differences between a BGP based approach and SCSP
>based approach are:
>  a. how the data is synchronized between peers
>  b. the intra-domain topology
>
>I think we are clear on (b), so (a) is the issue to explore. Hussein had
>made reference to a 2-stage exchange only being needed with BGP, while
>its three with SCSP. These are the kind of differences I think we need
>to explore here. I'd invite both authors to comment on the differences
>in how the data is exchanged and updated between peers. Lets stick with
>inter-domain for the moment...
>

Here's a brief synopsis of the data exchanges for each alternative.  In
both cases, the exchanges are just peer-to-peer.  

BGP
***  

  BGP1) OPEN.  OPEN messages are exchanged to bring the link up.
Basically, two peers connect and send an open message to the other.  A peer
either acks the OPEN with a KEEP-ALIVE, or nacks the OPEN with a
NOTIFICATION.  When your OPEN gets ack'd, the peer connection is considered
to be established (ie have bi-directional connectivity).  BGP can also
perform version negotiation and capabilities exchanges with the OPEN
messages. 
 
  BGP2) UPDATE.  Once a peer connection is established (via ack'd OPENs),
BGP peers exchange UPDATE messages.  UPDATE messages are the messages that
contain the routing information.  Each UPDATE message may contain a number
of withdrawn routes and a single feasible route.  A withdrawn route is a
route that should be removed from the peer's RIB-IN.  A withdrawn route is
defined by a address/prefix-length.  A feasible route is defined as the
NLRI (network layer reachability information, ie a list of
address/prefix-length combinations) and its attributes.   A key mandatory
attribute is the NEXT-HOP attribute which gives the next-hop for the NLRI.
Basically, you send an UPDATE for each possible NEXT-HOP in your routing
database.  

KEEP-ALIVE messages are periodically going in the background to ensure that
the link is up.  

BGP UPDATE and NOTIFICATION messages are not acknowledged as TCP
acknowledgment is considered sufficient.  

SCSP
****
  SCSP1) Hello.  Peers exchange Hello messages to ensure bi-directional
connectivity between peers.  This is equivalent to the BGP OPEN exchanges
but without anything like version negotiation or capabilities exchange.
Hello messages are also periodically exchanged in the background to ensure
the link us still up.  

  SCSP2) Cache Aligment.  SCSP distinguishes between the state where two
peers have had their databases synchronized and when they have not.  When a
peer connection first comes up (via Hello), the peers enter the Cache
Alignment phase where the entire databases must be exchanged.  First, there
is a negotiation to determine who is the master and slave of the peering
relationship.  Next, the peers exchange summary records with each other,
where a summary record is a short-version of the record - enough to
identify it, but doesn't have the real data.  This phase has no parallel in
BGP.  

  SCSP3) Cache State Update.  A server maintiains a list of the records
that it needs to get from its peer (the summary records in the previous
phase).  A server requests these records via a Solicit message (ie I need
the full record for these summary records).  The response to the the
request is a Update Request message containing some/all of the records.  A
server continues to send further Solicit messages for records that it has
not yet received.  A server ack's that the records have been received with
an Update Reply containing the summaries of recieved records.  The Update
Request messages are similar to the UPDATE messages of BGP.  

Acknowledgements in SCSP are done per-record (ie requests for a record are
acknowledged by sending the record, record transmissions are acknowledged
by sending a record summary in an Update Reply).  

SOME DIFFERENCES
****************

I think Hussein's "2 vs 3" stages are identified above.  SCSP has the added
stage of Cache Alignment where database summaries are exchanged.

BGP uses TCP, SCSP uses X (pick your transport).  

BGP exchanges and message formats are certainly more efficient, SCSP more
generic.  In the peer-to-peer case (ie inter-domain), some SCSP header
fields are redundant (example: reciever and sender ID).  

SCSP does per-record acks and requests, which is flexible, but can be a
pain in the ass and inefficient.  BGP UPDATE/NOTIFICATION messages are
unacknowledged and assume TCP provides reliability for the application.  

SCSP Update Requests can contain many records.  BGP UPDATEs contain a
single route (maybe multiple withdrawn routes).  

SCSP bases record identification on record summaries (source identifier,
cache 'key' for record), so to replace a record you keeep the record
summary and replace the data contents.  BGP identifies records based on the
contents (ie NLRI), so the data content is examined to determine new vs
replacement. 

BGP includes version negotiation and capabilities exchange in the peering
session establishment.  SCSP has no such concept, the function would have
to be included in the application above SCSP. 

Both use a similar keep-alive mechanism to determine peer status.  

SOME COMMENTARY
***************

My personal view is that TCP is not enough for reliable application
communications.  I haven't heard anyone else argue for or against this
position, so I'll stick with it.

In order to get application reliability from a lower layer, you need a
transport that either maintains the data or one that gives the application
control over the acks.  SLUMS is a potential transport that seems to take
the latter approach.   From the SLUMS BOF announcement, one of the in-scope
requirements is

    - ability for application to indicate "a reply is coming" versus
      "no more coming now, go ahead and ack, don't delay"

So SLUMS recognizes that apps may need to have input into the ack timing.
I don't know enough about sigtran yet to say anything.  

Another alternative is that we put reliability into GLP so that messages
(or records) get ack'd, retransmitted, timed-out, etc.  This is the
direction most other protocols have gone, and is as viable a solution as
anything else.  If we go this way, then we most likely need a "GLP Route
Reflector" spec, etc, to parallel BGP.  I don't think we can just say "we
want to use that attribute/feature of that protocol in a similar way."  The
fact that we're an incompatible protocol to BGP means that any features
that we want from BGP have to be recreated in a spec(s) for GLP.  This may
turn out to be mostly copying, or it may require some technical changes to
the underlying operation.  

I suggested SCSP because it seemed to solve our transport needs.  I
wouldn't argue that SCSP is a simple protocol.  We could build a reliable
protocol that is more efficient and less complicated than SCSP.  If SCSP
gets used for more things in the future, then the costs of its
implementation get leveraged as time passes.  I have no idea whether that
will or will not come to be.  It certainly has limited applications now.  

So SCSP may be an ugly shoe, but it looked like it fit.  I don't know
whether the cost/complexity of being one of the earlier applications of
SCSP is worth while.  I thought it might be.  I might have similar concerns
about SLUMS after it gets defined.  My crystal ball is a wee bit fuzzy at
the moment.  

But that's enough out of me.

- Matt

---------
This message came from the IETF IPTEL Working Group Mailing List.

---

From: Jose Costa Requena <[email-masked]>
Date: Fri Apr 30 13:49:49 1999

Hello,

I just want to make some comments about the SCSP, because I have implemented 
it for distributed database synchronization.

The SCSP has generic messages format that could seems inefficient but in 
this way it can be used for many applications.

There are some header fields  that seem redundant like the receiver and 
sender ID in the Mandatory Common part. From the implementation point of 
view it is useful when we have to choose the right Finite State Machine 
(FSM) that has to attend the DCS from which we have received that message. 
Thus checking the Sender ID we select the FSM that is implemented as an 
idividual object and inside the FSM we can check if the Receiver ID belongs 
to the DCS list that has the FSM to know if the packet is from one server in 
the SG.

>unacknowledged and assume TCP provides reliability for the application. 

That's right, the SCSP does a little excess of message transactions to reach 
a consistent state in the database. A good alternative is to use a TCP 
connection at the beginning just to check if the connectivity is 
bidirectional or not. Thus, when the Hello Finite machine (HFSM) transits to 
the waiting state we use UDP for the rest of transactions. The unreliability 
of the UDP is corrected with the excess of acks and requests.

Regarding the 2-stage exchange only being needed with BGP.
In the SCSP we have 3-stage, but in Cache Alignment that is the other stage 
that the SCSP performs there is performed an initially information 
alignment.It could be inefficient, but in that stage each peer exchange only 
the summaries of the total data, in this way the next stage knows exactly 
the records that are needed to be updated. Those summaries are stored in the 
CRL instead of allocating the total amount of information. 
This feature is more efficient in the case that the records are quite big, 
but I agree that it will be inefficient in the case that the summary is 
comparable to the total record that must be updated.

I hope that I have clarify some details that I have seen from the 
implementor point of view.

Best Regards.

Jose

Helsinki University of Technology
Laboratory of Telecommunications Technology
Jose Costa Requena
Research Scientist
Room: SG 227B
Tel: + 358 9 451 2475
email(internet): [email-masked]

---------

---

From: Ami Amir <[email-masked]>
Date: Fri, 30 Apr 1999 19:59:37 +0300

Hi

I am definitely NOT an expert - but one thing that should be kept in mind - 
the need to provide fault tollerance and redundancy. One cannot deploy a 
functional public phone system that deos not provide for redundancy at 
alkost every level.

Ami

-----Original Message-----
From:	Matt Squire [SMTP:[email-masked]]
Sent:	Thursday, April 29, 1999 8:46 PM
To:	Jonathan Rosenberg
Cc:	IPTEL IETF Mailing List
Subject:	Re: Evaluation of Gateway Location proposals

BGP
***

SCSP
****

SOME DIFFERENCES
****************

SOME COMMENTARY
***************

- Matt

---------

---------

---

From: Ami Amir <[email-masked]>
Date: Fri, 30 Apr 1999 19:59:37 +0300

Hi

Ami

-----Original Message-----
From:	Matt Squire [SMTP:[email-masked]]
Sent:	Thursday, April 29, 1999 8:46 PM
To:	Jonathan Rosenberg
Cc:	IPTEL IETF Mailing List
Subject:	Re: Evaluation of Gateway Location proposals

BGP
***

SCSP
****

SOME DIFFERENCES
****************

SOME COMMENTARY
***************

- Matt

---------

---------