Subject: Re: Support for multicast in the ForCES protocol

From: "Khosravi, Hormuzd M" <[email-masked]>
Date: Thu, 20 Nov 2003 11:19:10 -0800

Hi Zsolt,

I think you have brought up a very good point, 
although based on some assumptions. 
(For now lets consider the assumptions are true.)
I have been resisting getting into this thread since it
was meant for folks "not directly involved with protocols".
However I don't see that as the case any more :)

In any case, what you are suggesting is that in order
To use multicast efficiently, you will need to have this
Tree in which multicast groups are assigned FE:LFB basis.
(I will try to keep things simple for now by assuming 1 table per LFB)
I agree its an efficient way to use multicast. 
Now what this means is that, just in terms of no. of 
connections from CE-to-FE, the ratio of unicast:multicast, 
is the ratio of -> sum of Distinct LFBs per FE (M): no of FEs (N).

So in case of a medium size Edge network element with 10 FEs 
but 15 distinct LFBs (since it has to handle many diff protocols),
This ratio is 15:10, which means that there will be more multicast 
Groups than unicast connections ! This clearly has worse performance 
than unicast.

However, in case of a core router with 100 FEs and 10 distinct LFBs,
This ration is 10:1, which means less multicast connections but not
By a huge ratio.

Now, this brings another question to mind... as to how  "command
Bundling" will be supported with this concept of multicast groups.
Say I want to update entries in 3 LFBs (in 2 FEs) which belong to 
different groups. Using multicast groups, you can send 3 messages
On the 3 groups but how do you bundle that in an elegant way ?
One way which I can think of is to bundle all 3 messages in a single
Message and send it on one multicast connection. This does not
Seem to be very elegant and there is no performance gain that I can
think of.
(Instead of sending 3, x size msgs, you sent 1 3x size msg)
The only other way is to use a 2-phase commit in which you will land
Up sending 6 messages, 3 for request and 3 to commit. This will
definitely
Be bad in terms of performance, etc and doesn't even meet the command
Bundling requirement in spirit (single message).

Let me know if you can think of an elegant, efficient way of doing this
?

Thanks
Hormuzd
-----Original Message-----
From: Forwarding and Control Element Separation
[mailto:[email-masked]] On Behalf Of Zsolt Haraszti
Sent: Wednesday, November 19, 2003 3:49 PM
To: [email-masked]
Subject: Re: [FORCES] Support for multicast in the ForCES protocol

On Wed, 2003-11-19 at 16:58, Alan DeKok wrote:
>   That is, assuming that there *is* a multicast-capable transport
under
> ForCES, a "multicast" update can simple appear to ForCES to be another
> FE.  The system administrator would be responsible for maintenance of
> multicast group membership.

It is important to keep in mind that the target audience of
potential multicast configuration updates are a set of LFBs of
identical type across one or more FEs, and not the FEs
themselves.  For example, just because the same LPM table must
be loaded into two FEs', it does not mean that all other
LFBs of the two FEs will also be programmed identically.  In
fact it is more probable that they won't.

To be even more precise, the entities that need to be addressed
by the same config information may not be even full LFBs, but
rather some attributes of (e.g., tables) of these LFBs.

It appears to me that using a layered approach, in which the ForCES
protocol simply relies on multicast support in the lower layer, does
not address the whole problem, since that only addresses the
efficient distribution to a group of FEs and not a group of LFBs
across these FEs.  (Am I wrong?)  That may be fine, provided that
the rest of the problem can also be solved.  But I am not sure it
can be without support from the ForCES protocol itself.

To explain what I mean by target audience of multicast config
requests, consider the following (small-scale) example.

CE---+---> FE1 ---+---> LFB1(*)
     |            +---> LFB2(*)
     |            +---> LFB3(LPM)--+--> prefixtab1 [VRF1]
     |            |                +--> prefixtab2
     |            +---> LFB4(*)
     |            +---> LFB5(LPM)--+--> prefixtab1
     |            |                +--> prefixtab2 [VRF1]
     |            :
     |
     +---> FE2 ---+---> LFB1(*)
     |            +---> LFB2(*)
     |            :
     |
     +---> FE3 ---+---> LFB1(LPM)-----> prefixtab1 [VRF1]
     |            +---> LFB2(*)
     |            +---> LFB3(LPM)--+--> prefixtab1
     |            |                +--> prefixtab2
     :            :                +--> prefixtab3 [VRF1]

The CE controls a number of FEs, each has a set of LFBs (the
FEs may not have the same LFB topology). LFB(LPM) denotes
LPM prefix lookup LFBs (which for simplicity  are instances
of the same LFB class). LFB(*) denotes other types
of LFBs.  In a VRF capable router the LPM LFBs may contain
multiple LPM tables, and the number of LPM tables may be
different from one LFB to the other (as indicated in the
example).  Suppose that the following tables are created to
provide LPM for the same VRF (e.g., VRF1 as marked in the
figure):

FE1 : LFB3 : prefixtab1
FE1 : LFB5 : prefixtab2
FE3 : LFB1 : prefixtab1
FE3 : LFB3 : prefixtab3

In a large system where there is a large number of such tables,
one should use the ForCES model in a clever way such that these
tables are kept identical (not only the prefixes, which are
obvious, but also the next hop identifiers associated with
each prefix to address next hop entries in another LFB).  If
the tables are identical, then the exact same methods can be
and should be sent to all of them to maintain the tables, and
that is a case for multicast support.  In other words,
the above list of tables are the real and final target of the
multicast messages.

I am not sure what part of the problem would the underlying
multicast layer solve, but I think it is only from the CE to
the FEs (which is useful, but not sufficient).  If so, there
must be additional mechanisms to set up the "rest of the tree"
on each FE, and I don't see how you can do that without explicit
support from the ForCES protocol.  What is needed is that the CE
should be able to say: "I want to have
FE1 : LFB3 : prefixtab1
FE1 : LFB5 : prefixtab2
FE3 : LFB1 : prefixtab1
FE3 : LFB3 : prefixtab3
to form a muticast group, accessible by a fictive address
FEx : LFBy : prefixtab_z".  To implement that the address
triplet must be registered at the CE, a CE-to-FE multicast
tree must be established (if one with identical membership
does not exist yet), and then a table must be set up at
the FE (and possibly even in each involved LFB).

My main point, again, is that CE-to-FE multicast is useless
unless the full tree can be set up (the leaves are tables within
LFBs), and I suspect the ForCES protocol will need to pitch in
to address the second and perhaps the third layer of the tree
(the third can actually be solved within the LFB-specific protocol
data, instead of the base protocol).

My $0.02.

Zsolt Haraszti

---

From: Zsolt Haraszti <[email-masked]>
Date: Thu, 20 Nov 2003 15:52:45 -0500

Hormuzd,

See my comment inline.

> Groups than unicast connections ! This clearly has worse performance

You make an assumption that does not hold: you assume that all
15 LFBS will share information with their counterpart in all
other FEs.  That is unlikely the case in practice.  In fact
only a very small number of LFBs will share information (though
they may be the ones updated most frequently with large amount
of information).  A more realistic numerical example would be
when 2 out of the 15 LFBs are part of multicast trees.  So in
this case I would have 10 unicast connections for normal LFB
configuration, and 2 multicast trees.

/zsolt