Subject: Re: Evaluation of Gateway Location proposals

From: Matt Squire <[email-masked]>
Date: Thu, 08 Apr 1999 12:16:38 -0400

I'm already sorry for the length of this message, but I had to
respond...

So here are my comments on your comparison.  Not surprisingly, I have a
different view on several important aspects.  

>
>1. Introduction:
>GLP is based on SCSP which is a generic link-state based protocol for
>synchronizing the databases of a server group. GLP uses SCSP's message
>formats and defines new fields specific for call routing and gateway
>location. GLP uses SCSP's transport mechanism only for intra-domain
>synchronization of LSs. For inter-domain communication, GLP proposes a
>model that's based on BGP. At runtime, GLP defines a server group (the
>set of LSs) which uses SCSP for synchronization and update
>advertisement, i.e., GLP is a user of SCSP.
>
>TBGP is based on the BGP-4 and its multi-protocol extensions. It uses
>BGP's transport mechanism for both inter-domain and intra-domain. TBGP
>uses the same message formats defined by BGP, and benefits from the
>attributes already defined 
>in BGP. In addition, TBGP defines new attributes to serve the specific
>goals of call routing and gateway location. With regards to
>implementation, BGP and TBGP are completely independent protocols. The
>two protocols do not interfere with each other at runtime. A TBGP
>speaker (or LS) does not have to be a BGP speaker.

I've tried to make this clear before, but have evidently not succceeded
in full.  GLP is based on the combination of BGP4 and SCSP, not just
SCSP.  SCSP is just used to move the data between servers and replaces
the BGP transport mechansism.  Behavior above the
synchronization/transport is based on BGP4 as catered to the needs of IP
telephony.  

Both proposals have the same basic BGP architecture for an LS:

   |-----------------------------------------|
   |            Local routes                 |
   |                 /\                      |
   |                  |                      |
   |               Decision                  |
   |                Process                  |
   |               /\     |                  |
   |                |     \/                 |
   |         in-routes  out-routes           |
   |-----------------------------------------|

Both proposals have a decision process based on local policies.  Both
proposals have routing objects which carry phone numbers, next-hops, and
all kinds of other attributes.  Any attribute supported in one proposal
can be made to work in the other proposal.  

Lets look at some of the basic aspects of BGP4 (I'm just paging through
the BGP spec now, so this list is not intended to be complete).  
  1) Attributes
  2) Transport/communications (ie open, keep-alive, update, notification)
  3) FSMs
  4) Update processing (decision process, update-send, route selection)

I'll briefly discuss each aspect below before getting into more specific
comments on your comparison later.  

1) Attributes

I fully recognize that the attribute set as listed in the GLP draft is
incomplete.  I've concentrated the initial draft on the potential new 
attributes for iptel that do not exist in BGP.  Other BGP attributes
will be forthcoming, 

The TBGP draft has taken the approach that everything in BGP is fair
game for iptel, and has assumed that other BGP attributes will work for
us just fine.  Although I agree that other BGP attributes are fair game,
I've taken the approach that we need to inspect every BGP attribute in
terms of the problem we are trying to solve, and have not assumed that
they will work as defined in BGP4.  BGP4 attributes that exist have been
defined for a reason, so we need to understand that reason, see how it
applies to call-routing, and adjust the attribute appropriately.

The 7 attributes defined in the BGP4 RFC are ORIGIN, NEXT_HOP, AS_PATH,
MED, LOCAL_PREF, ATOMIC_AGGREGATE, and AGGREGATOR.  Just running through
some thoughts off the top of my head:

 A) NEXT_HOP.  We've already had discussions how the concept of a BGP4
 next-hop differs from an iptel next-hop.  GLP is not a hop-by-hop
 routing protocol like BGP4, and this has impacts on many aspects of the
 protocol.

 B) AS_PATH.  I've already agreed that we need something like the
 AS_PATH.  But we've also had discussions on the list about whether we
 want to talk about the path that the advertisement took, the path that
 call-routing messages will take, or both.  In BGP4, this attribute
 serves both purposes.  Do we need one or both kinds of paths?  How does
 this affect other aspects of the protocol (route preferences and
 selection, loop prevention, attribute propagation, etc.)?  Our
 definition of what paths we need has an effect on other aspects of the
 protocol.

 C) MED.  The multi-exit discard attribute is used in BGP4 to differentiate
 multiple links between neighboring ASs so that one link is 'preferred'
 over the other.  But we don't route hop-by-hop, and that brings up some
 issues.  How is MED used when an routing advertisement contains a
 next-hop that is not in the advertising domain?  In other words, is MED
 a property of the advertisement path, or of the potential signalling
 path, or what exactly?  In any case, I think its definition as in BGP4
 would need some reworking.

 D) ATOMIC_AGGREGATE.  Atomic aggregate is used to indicate that a
 "less-specific NLRI" has been chosen without selecting a 'more specific
 NLRI".  GLP needs to have multi-attributed routing decisions, hence the
 concept of more/less specific is not dependent only on the addresses/NLRI.
 Also, we have yet to agree that what we want is address prefixes - if
 we choose something else, then things get even more confusing.  In
 general, the NLRI-centric nature of BGP is too restrictive for our
 needs.

So I have issues with 4 of the 7 well-defined attributes of BGP4 as they
are defined and used within BGP4.  The application of these attributes
to GLP is not the same as it is in BGP4.  We don't route hop-by-hop, we
use mutli-valued routing decisions, etc.  These differences affect many
aspects of the protocol, including attribute definitions and processing.
Every attribute needs to be re-examined in our context before we ok its
use.

I think we've taken different approaches to including other BGP
attributes in our proposals.  I wanted to understand the reasons behind
every attribute, see how it applies to iptel, and then come up with an
attribute that works for us.  I think you've taken the approach that all
attributes are ok and we can fix how they work later.  In either case, I
hope we agree that the BGP4 definitions of some of its attributes are
not sufficient for iptel.  We need something like them, but not exactly
the same thing.

Independent of the manner in which we choose to implement GLP, we need
something like BGP, but not really BGP.  We have to write an entirely
new specification, we can't just reference BGP and say that we're
redefining these attributes, this process, these algorithms, etc.  In
either implementation scenario, to say that GLP has something because
BGP has it is a flawed approach until we've performed analysis of the
feature and how it is effected by things like the fact that GLP isn't a
hop-by-hop routing algorithm, that the GLP routing decision is based on
things other than the NLRI, etc.  

In general, attribute decisions are independent of the differences
between the proposals.  I knew my proposal was lacking in attributes
when I wrote it.  I also think the TBGP proposal needs to redefine BGP4
attributes for our purposes and not to just reference their use in BGP4.
We need a new protocol, not a reference to BGP4.  

I also believe that any attribute can work with either approach.

2) Transport/communications.

Will address later.

3) FSMs.  These basicly describe how one gets to the 'established'
state, so these are really dependent on the transport/communications.

4) Update processing (decision process, update-send)

  4a) Decision process.  The BGP4 decision process is NLRI-centric.
  This must be changed to support multi-attributed routing decisions.
  We certainly want to maintain the structure of the decision process of
  BGP4, but it CAN NOT be the same decision process.  BGP4s decision
  process must be made more general to support our requirements.  The
  whole concept of what aggregation is and how it is performed is
  completely differnet for GLP than it is for BGP4.  We have strings not
  bits, we have routes with attributes, we have black holes, we haven't
  agreed on prefixes as our destination representation, etc.  Our
  decision process is not the same as the BGP4 decision process.

  4b) Update-send. Again, we should keep the structure but we have to
  get rid of the NLRI-centric aspects.  Things like jitter, advertisement
  frequency, etc., can be handled independently of the transport (UPDATE
  vs SCSP).  Although this is not in the first draft of GLP, it would
  need to go in there.  

>
>2. Reliability:
>GLP runs on a reliable transport, e.g. TCP. In addition, GLP/SCSP uses
>request/response transactions.  Responses are not simple Acks. They are
>long messages. This significant overhead is needed with intra-domain
>link-state protocols to achieve transitive closure and to make sure that
>all LSs in a domain have synced up following a routing change. However,
>it is still a significant overhead that may be avoided. The
>request/response mechanism does not add any benefit for inter-domain
>GLP.  
>
>TBGP, on the other hand, relies on TCP for reliability. It uses Update
>messages only with no responses. This is sufficient.
>

Onto a discussion of transport.  To me, this is the major functional
difference between the proposals.  

To begin, I strongly disagree with your last statement.  TCP provides
transport reliability, it does NOT provide for reliable communications
between applications.  Look at the numerous protocols that the IETF has
defined over TCP.  The majority of them have recognized the difference
between application reliability and transport reliability, and include
things like acks, timeouts, retries, etc.  

Generally, BGP4 implementations don't use off-the-rack TCP/IP stacks.
In order to achieve application reliability using just TCP, you really
have to integrate your application with the TCP stack.  BGP4 TCP/IP stacks
are finely tuned for BGP, the OS is catered toward routing and routing
protocols, etc.  They don't use a standard socket type interface over
TCP-IP because they would lose their reliability.  

TCP is not sufficient for application reliability unless you application
is integrated with TCP.  Whatever we do, I hope we recognize this fact
and avoid the pitfall of forcing GLP developers to hack the TCP stack
because we don't build application reliability.  

On the other hand, SCSP is sufficient.  SCSP was designed to get
information from A to B (and to C, D, ...).  When used between just two
servers, as you point out later, it has more than is needed.  We could
certainly build another reliable transport protocol by adding retries,
timeouts, acks, etc., into the BGP4 transport.  If restricted to
point-to-point relationships, then it would have less overhead than
SCSP.  

I just question why we would want to do this.  The IETF has already
defined SCSP as a general method of moving data between servers.  Why
not just use a defined protocol instead of defining JARAP (just another
reliable application protocol).  The overhead of SCSP's generality is not
wasted if we use that generality within a domain.  Using SCSP frees us
from worrying about transport and allows us to concentrate on what data
we need and how we should process it.  These are really the novel iptel
specific aspects.  Moving data around has been done many times by many
protocols in many ways.  We don't have to build another one.

The use of SCSP would replace the open, update, and keep-alive in BGP4.
This decision is independent of the changes required of BGP4 above
transport, which I discussed earlier.  However, deciding how version
negotiation and error notification work when using SCSP is yet TBD.  These
functions are provided in the BGP4 transport which would need some work
in my suggested approach.

>3. Inter-Domain Topologies:
>Both protocols can support any general topology.
>
>4. Intra-Domain Topologies:
>GLP can support any general topology.
>
>TBGP requires either a full mesh configuration or the use of route
>reflectors or confederations. It is possible to configure each TBGP
>speaker in a domain as a route reflector, and thereby to support any
>general topology.  
>
>Questions: 
>- What's so bad about route reflectors and confederations? 
>- What's the need for supporting a general topology? All that's required
>is synchronizing the LSs of the same domain. It doesn't matter how they
>interconnected, as long as there sufficient redundancy in the topology.
>

Some people evidently think route reflecting has some downsides.  See 
draft-dube-route-reflection-harmful-00.txt "Route Reflection Considered
Harmful".  

In general, I think unnecessary restrictions are, in a word,
unnecessary.  They limit deployment scenarios.  BGP4s topological
restrictions are a reflection of its choice of transport, not of the
routing processes above them.  

We are designing a new protocol.  We need to leverage the experience of
BGP, both good and bad.  That means we don't have to make the same
mistakes.  In hindsight, the original BGP restriction on meshes was bad
thing.  There have been numerous proposals over the years to fix it
(route reflectors, route servers, confederations, clusters, etc).
However, all of these proposals have had to live within the confines of
interoperability with existing BGP.  We don't have to be interoperable
with existing BGP implementations.  We have the freedom to fix this
problem in a more general way without interoperability concerns.  

GLP needs to get information from every server to every other server
within the domain.  That is the problem we have to solve.  If we
recognize that this is the issue, and that we don't have to limit
ourselves to solutions that are compatible with BGP1 through BGP4, then
we can do better than what was done in BGP.  Let's step out of the box
and look at the problem without the BGP interop constraints.  

>5. Loop Avoidance:
>5.1 Intra-domain Loop Detection: 
>5.2 Inter domain Loop Detection:
>6. Hold-down Timers and Route Flaps:
>7. Policy Enforcement:
>8. Coping with Aggregation Side Effects:
> 

These are all attribute issues that can be addressed by either approach.
TBGP does need to define new formats, in my approach we would.  With
either approach, we also need to re-examine the exact definition of the
attributes and modify them as appropriate.

>9. Addressing Formats:
>GLP proposes to advertise E.164 ranges. During the meeting, it has been
>agreed that E.164 prefixes are more space-efficient.
>
>The multi-protocol extensions for BGP-4 (RFC 2283) have already defined
>fields for advertising E.164 prefixes in BGP messages. These extensions
>are used by TBGP.

Although we agreed that prefixes are more space efficient, I don't think
we agreed that prefixes are the way to go.  I think it was Scott that
brought up the point of a company having the same number in many area
codes.  Although I have nothing against a decision for prefixes, I just
don't think it was agreed upon.  

>
>10. Leveraging Existing Attributes:  
>Most of BGP-4's fields and attributes can be used by TBGP, as evident
>from the above discussion. Still there are some attributes which are of
>no use to TBGP. for example the layer 3 NEXT_HOP attribute.
>
>GLP, on the other hand, will have to redefine several of BGP's existing
>attributes within the SCSP's message formats. Consider for example, the
>policy attributes and aggregation attributes discussed above. Also SCSP
>defines several mandatory fields which are redundant or of no use for
>GLP. E.g., the Sender ID and Recvr ID are not needed in every Update
>messages. The peers already recognize each other. Also the Cache Key
>field is not needed since the tables will be indexed by the E.164
>prefixes.

As I mention above, there are subtle points with several attributes
that arise because call routing differs from ip routing.  Yes, TBGP can
use the same attribute format.  I don't consider this particular
important.  We still need to look at every attribute and redefine it as
necessary.  Formats are the least of our obstacles.

SCSP fields are necessary for general database sync.  In a peer-to-peer
inter-domain link, you mention some that are redundant.  In the
intra-domain case where there are multiple servers, these fields are
required.  

The cache key is a very useful and compact identifier for a data object.
One can issue updates or withdraws on the data object without having to
understand the data.  This means that one can withdraw a route by
performing a lookup on the cache key instead of an e164 address.  This
allows for a more efficient lookup for changing an existing data object.
Yes, its redundant, but for a reason.

>
>TBGP can define its own message formats and remove any useless fields
>from the current BGP messages, because BGP and TBGP are separate
>protocols. On the other hand, GLP can not remove useless fields from
>SCSP messages, because it is just a another user of SCSP.

The other view is that TBGP MUST define its own message formats and
remove useless fields from BGP messages.  With SCSP we only define the
data, not the protocol. To me, that means more work with the TBGP approach.

I certainly agree that we could a protocol with less overhead than SCSP.
See my previous arguments on why the BGP transport is insufficient for
applications.

>
>11. Protocol Phases:  
>GLP/SCSP has three phases: Hello, Cache Alignment, and Update. 
>
>TBGP/BGP consists of two phases only: Hello and Update. It uses the same
>mechanisms of the update phase for aligning the call routing tables when
>a new connection between wo peers is established.
>
>12. Security Consideration:  
>Both protocols define optional fields to include authentication data.
>Both of them  permit the use of different authentication mechanisms, and
>both provide sufficient space to include the 128-bit MD5 digests. 
>
>However, TBGP messages can't carry authentication data larger than 128
>bits, while SCSP messages can.
>
>13. Routing Experience:
>This is not a feature of the protocol, but I think it is worth repeating
>that SCSP was not designed for, and was never used for, inter-domain
>routing.
>
>On the other hand, BGP has been specifically designed for inter-domain
>routing. It (a) is known to work, (b) is widely deployed in the
>Internet, (c) has an extensive operational experience, and (d) known to
>scale.

First, SCSP is not being proposed as an inter-domain routing protocol.
It is being proposed as a method of moving data between servers.  The
data that it is being proposed to move is data that will be used for
routing.   So although you're right in that SCSP isn't a routing
protocol of any sorts, I'm not proposing that it be made one.  I want to
separate the routing process from the process of moving the data around,
so that this wg can build the routing process and algorithms.  SCSP
frees us from worrying about moving the data around.  To me, this is a
good thing.  

Second, you're absolutely right about BGP.  But TBGP is not BGP.  Its a
new protocol.  It has different attributes, different aggregation
algorithms, different route selection, different routing decisions, etc.
I consider it flawed logic to assume that a property of BGP translates
into the same property in TBGP.  We have no operational experience with
TBGP, for example.

Yes BGP works.  But that's because of the experience developed over the
years in its aggregation algorithms, routing decisions, experience with
internet routing deployment and address distribution, etc.  The IETF
does not have years of experience with deployed call routing protocols.
We don't have experience with the aggregation algorithms for the
aggregation of strings and multi-valued routing objects.  We don't have
that experience with multi-valued routing decisions and processes.  We
can't rely on our handing out addresses in certain ways to help with
aggregation.  Etc, etc, etc.  

This is a new problem.  We need to leverage our experience with BGP, but
not be restricted by it, or to just assume that what is good for ip
routing is good for call routing.  

In my opinion, there are really two major differences between the
proposals.  The first is ideological.  I took the approach that we have
to re-examine every attribute in BGP before accepting it, you took the
approach where we accept it and re-examine it later.  I think we both
agree that this is a new protocol that requires a new spec, and that
cannot just reference BGP4 to describe its behavior.  We each want to
leverage BGPs attributes.  TBGP has an easier job cause it leverages the
formats as well as the function, where with my proposal we have to
redefine formats.

The second aspect is the transport issue.  I am in strong disagreement
with the assumption that TCP provides application reliability.   Hence,
as an application protocol, BGP4's reliability measures are not
sufficient.  I proposed SCSP to give us this reliabilty and to free us
from worring about transport issues such as retries, timeouts, etc.
SCSP also has beneficial side effects as an intra-domain distribution
system.  If the wg feels that we'd rather design our own reliable
application protocol, then I could live with that.  I think we'd be
wasting effort that could be better applied to other iptel problems, but
these things happen.  

Counting on TCP for application reliability is just a bad idea.

- Matt

---------
This message came from the IETF IPTEL Working Group Mailing List.

---

From: "Eric Ho" <[email-masked]>
Date: Wed, 21 Apr 1999 10:53:30 +0800

Hi,

Could anyone please advise how VoIP is going to run over ATM
(i.e., what kind of encapsulation or native ATM)?

Thanks in advance!

eh

=====================================================================
Eric Ho                                         Direct: +852-27222766
FORE Systems, Inc.                                Main: +852-27222722

---------

---

From: Jonathan Rosenberg <[email-masked]>
Date: Tue, 20 Apr 1999 22:46:26 -0400

> different view on several important aspects.

Let me summarize here:

1. I think we all agree that attributes will need to be treated, one at
a time, thought out, and specified, for GLP. The difference between
Matt's and Hussein's approach is one of direction - starting with BGP
and modifying, or taking a more fresh approach. In the end, it doesn't
really matter. So, for this thread, lets focus on things not having to
do with attributes (which includes AS_PATH, MED, NEXT_HOP, etc.)

2. Matt raised numerous transport issues (TCP or not TCP). I think its
fair to say we want to avoid designing a new transport protocol. There
is work underway elsewhere (sigtran and slums) to do this, which we can
leverage. It would seem to me that if something new is needed, this
would also be independent of whether the protocol baseline is BGP or
SCSP, since presumably one could use this new transport protocol with
either.

3. To me, the MAIN differences between a BGP based approach and SCSP
based approach are:
  a. how the data is synchronized between peers
  b. the intra-domain topology

I think we are clear on (b), so (a) is the issue to explore. Hussein had
made reference to a 2-stage exchange only being needed with BGP, while
its three with SCSP. These are the kind of differences I think we need
to explore here. I'd invite both authors to comment on the differences
in how the data is exchanged and updated between peers. Lets stick with
inter-domain for the moment...

-Jonathan R.

-- 
Jonathan D. Rosenberg                       Lucent Technologies
Member of Technical Staff                   101 Crawfords Corner Rd.
High Speed Networks Research                Holmdel, NJ 07733
FAX: (732) 834-5379                         Rm. 4C-526
EMAIL: [email-masked]
URL: http://www.cs.columbia.edu/~jdrosen

---------

---

From: Nadia KAUSAR <[email-masked]>
Date: Wed, 21 Apr 1999 12:11:09 +0100

Hi

I believe some people in a company called INESC are looking at various aspects 
of H.323 conferencing over ATM.  also if it is H.323 conferencing and ATM 
issues you are interested in may be [email-masked]  will be a 
better mailing list?

thanks
Nadia

---------