Subject: Re: Multiple Levels of LMM agents (was: RE: [mobile-ip] Revised L ocalized Mobility Management Requirement s)

From: Erik Nordmark <[email-masked]>
Date: Wed, 9 May 2001 09:20:09 +0200 (CEST)

Tom,

> Consider you have one level hierarhy with one "element" (I call those
> FAs, but they may be called something else in some of the recent drafts
> I have not read) in the highest level and N FAs on the lowest level.
> Now, let us assume that considering only signaling, this system can
> handle k MNs simultaneously. Those MNs are handling, e.g. VoIP calls and
> are moving fast, e.g. on a highway. So, with all the VoIP call data and
> localized mobility management signaling of the k MobileNodes, the
> highest FA is fully employed.

Good example since it raises an interesting question.
The highest FA could be fully utilized for multiple reasons:
1. running out of processing power to handle the rate of binding updates
2. running out of memory to store the state for the bindings
3. running out of bandwidth on its network interfaces for all
   the traffic to (and from?) the MNs

If you add a hierarchy of LMMs below this box and have the box continue to
serve the same number of MNs you can improve #1 but it doesn't change
#2 or #3.
If you instead of a hierarchy add N equal LMM agents at the top level
(e.g., with each MN in the domain randomly selecting which LMM agent to use)
then you can address scaling along all of the above axis.

So it all depends along what axis you think the agent will max out first.

  Erik

---

From: Jim Bound <[email-masked]>
Date: Wed, 9 May 2001 11:02:41 -0400 (EDT)

Erik,,

> serve the same number of MNs you can improve #1 but it doesn't change

#2 and #3 in a hierarchial model is a funciton of implementation and we
know how to fix that out here in vendor land.  I can build co-located
router-switch-gateway/server to solve number 2 with bookoo RAM, 200 port
at a cheap port density, and terabytes of aggregated thru-put.

Problem #1 should be the focus of how we design LMM not 2 and 3 IMO.

> then you can address scaling along all of the above axis.

But there are 3 axises in my view only one being call flows for when I
need access to the top LMM axis and I would argue that it can be designed
without having to go to that axis for most cases.

Likewise the network for which MNs will need to operate are 4 dimensional
because of the signaling plane and reducing that plane for traffic and
distributed it b definition assists scalability.  Which is the critical
problem to solve for Mobile IP replacing the existing bearer services for
wireless.

> So it all depends along what axis you think the agent will max out first.

Without hiearchy it will always be the top tier and I think we can do
better than that.

/jim

---

From: "Charles E. Perkins" <[email-masked]>
Date: Wed, 09 May 2001 13:30:03 -0700

Hello Erik,

> So it all depends along what axis you think the agent will max out first.

I think the main reasons for intermediate levels are:

- Shorter signaling paths -- fewer chances for congestion and delays,
  fewer nodes to process the control messages, and (O.K. maybe not huge)
  better average performance -- but probably more consistent performance

- Natural fit to hierarchical network architectures
  ---- Backbone/Core Network/IP RAN at least, and possibly in some
       situations one (or rarely two) more levels of hierarchy.

Besides that, since the multi-level hierarchy doesn't have to be so much
more complicated anyway, I don't understand why you are arguing against it.
It seems like an obvious improvement.  I read most of the arguments supporting
the single-level structure to be "Well, we don't really need that improvement"
or "Well, we can program around that".  But why should we have to botch the
obvious network architecture in order to support something only marginally
easier (if at all!) to implement, and get lower performance as a reward?

Regards,
Charlie P.

---

From: Tom Weckström <[email-masked]>
Date: Thu, 10 May 2001 09:14:58 +0300 (EET DST)

Erik,

Very good points. I agree.

For fast handoffs (increasing speed on the highway), #1 may be very
important, while #2 is most important for scalability in the number of MNs
and #3 is most important for high throughput of data.

I think a good protocol should not make too much assumptions on what
problem types (within the area of localized mobility management) the users
of the protocol are trying to solve. The optimal protocol would then allow
the deployment to be N LMMs in a row or N+m+n agents in a three level
hierarchy. If the solution even allowed transparent clustering in the
hierarchy's root, then I would be rather happy. :-)

Regards,
	Tom

On Wed, 9 May 2001, Erik Nordmark wrote:

Erik.N >
Erik.N >Tom,
Erik.N >
Erik.N >> Consider you have one level hierarhy with one "element" (I call those
Erik.N >> FAs, but they may be called something else in some of the recent drafts
Erik.N >> I have not read) in the highest level and N FAs on the lowest level.
Erik.N >> Now, let us assume that considering only signaling, this system can
Erik.N >> handle k MNs simultaneously. Those MNs are handling, e.g. VoIP calls and
Erik.N >> are moving fast, e.g. on a highway. So, with all the VoIP call data and
Erik.N >> localized mobility management signaling of the k MobileNodes, the
Erik.N >> highest FA is fully employed.
Erik.N >
Erik.N >Good example since it raises an interesting question.
Erik.N >The highest FA could be fully utilized for multiple reasons:
Erik.N >1. running out of processing power to handle the rate of binding updates
Erik.N >2. running out of memory to store the state for the bindings
Erik.N >3. running out of bandwidth on its network interfaces for all
Erik.N >   the traffic to (and from?) the MNs
Erik.N >
Erik.N >If you add a hierarchy of LMMs below this box and have the box continue to
Erik.N >serve the same number of MNs you can improve #1 but it doesn't change
Erik.N >#2 or #3.
Erik.N >If you instead of a hierarchy add N equal LMM agents at the top level
Erik.N >(e.g., with each MN in the domain randomly selecting which LMM agent to use)
Erik.N >then you can address scaling along all of the above axis.
Erik.N >
Erik.N >So it all depends along what axis you think the agent will max out first.
Erik.N >
Erik.N >  Erik
Erik.N >

-- 
	Tom Weckström		Dynamics group
				Helsinki University of Technology
				[email-masked]
				http://www.cs.hut.fi/Research/Dynamics/

---

From: "Vinod Kumar Choyi" <[email-masked]>
Date: Thu, 10 May 2001 15:26:43 -0500

Hello charles,

comments below

----- Original Message -----
From: Charles E. Perkins <[email-masked]>
To: Erik Nordmark <[email-masked]>
Cc: <[email-masked]>; Tom Weckström <[email-masked]>
Sent: Wednesday, May 09, 2001 3:30 PM
Subject: Re: Multiple Levels of LMM agents (was: RE: [mobile-ip] Revised L
ocalized Mobility Management Requirement s)

>   better average performance -- but probably more consistent performance

Agreed, shorter signaling paths and therefore some advantage, but data
packets adrressed to MN have to be encapsulated and decapsulated or by some
other means from one higher LMM agent to a lower LMM agent and so on
(depending upon how many levels). The delay that occurs at LMM agents for
this forwarding from one agent to another could be large depending upon the
levels again.

> easier (if at all!) to implement, and get lower performance as a reward?

We should not be debating about the levels of hierarchy IMO instead focus
more on the INTERACTION and signalling between LMM agents.

regards
vinod

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 10 May 2001 21:26:37 +0200 (CEST)

> more complicated anyway, I don't understand why you are arguing against it.

Charlie,

I'm not arguing agaist it - I'm trying to understand the detailed technical
arguments for and against it and I'm frustrated because I the arguments
are thin on technical aspects like numbers showing performace differences.

I have some high-level concerns around LMM in general and hierarachical
LMM in particular based on the observation that any LMM scheme
essientially builds an overlay routing structure on top of regular
IP routing.

1. The interaction between failure detection done by the IP routing protocol
   (OSPF and IS-IS) and the LMM agents is likely to be operationally difficult.
   Assuming there are multiple LMM agents with IP routers in between them.
   When a link or a router fails the IGP will try to detect this and route
   around it. But in parallel (and indepedently of this) you'll need
   the LMM agents to detect failures (to detect when an LMM agent fail or the
   path between LMM agents fail).
   Assuming you'd like to detect and recover from all such failures (links,
   routers, LMM agents) quickly you will have independent failure
   detection operating at two different layers - how do you prevent them
   working against eachother? If you don't do this then the LMM agents
   might decide to reconfigure the LMM hiearachy paths just before the
   routers have routed around the failure i.e. the LMM reconfig was not
   necessary.

   A possible way to avoid this is to not have any regular IP routers in
   the picture i.e. all IP forwarding in the sub-cloud is done by LMM
   agents. I don't know how paletable that would be.

   Another way to mitigate this issue would be to make sure 
   that the time constants for the two
   layers are orders of magnitude apart. For instance, if the IGP
   is configured to route around a failure in 100 milliseconds and
   the LMM agents will not consider there being a failure until more
   than 100 millseconds have passed, you'd be fine. But in that case the
   LMM agents might not be able to detect and recover from an LMM agent failure
   in less than a few seconds. Would that be good enough? If there were
   no LMM agents the routing would be restored in 100 millseconds.

2. I think there is some experience indicating that protocols that have
   been designed assuming a hierarchical topology are far from future proof.
   One example being EGP which assumed a hierarchical relationship between
   autonomous systems which was replaced by BGP which has no such assumption.

   I fear the same risk here - the assumption that there is a hierarchy might
   prove invalid as both the topology of these networks evolve (driven
   by the cost of wired connectivity and the traffic patterns) and as the
   business relationships between operators evolve (which might drive a desire
   to optimize across different groupings of operators as Jim Kempf has 
   pointed out).

If folks take the above two concerns to heart I suspect that the LMM
schemes would look quite different than what we've seen so far.

  Erik

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 11 May 2001 17:56:12 +0200 (CEST)

Jim,

> Problem #1 should be the focus of how we design LMM not 2 and 3 IMO.

But just like we can add memory and network ports to a 
router-switch-gateway/server we can add CPUs to improve processing capacity.
Sure, in all 3 cases there are likely to be some hard limits 
imposed by the actual box (number of slots; amount of power etc).

So my question was really intended to explore whether there are numbers
that indicate that we'll hit one of the three limiting factors before the
others so that we can make sure that an LMM scheme solves the right
problem.

> problem to solve for Mobile IP replacing the existing bearer services for

I agree that you need scalability here. But it is hard because
something in the network need to track all the visiting mobiles.
Perhaps this state can be partitioned (in the database sense) over N
different parallel LMM agents.

  Erik

---

From: Theo Pagtzis <[email-masked]>
Date: Fri, 11 May 2001 10:53:06 -0700

Hello Erik,

>    routers have routed around the failure i.e. the LMM reconfig was not

The main point here is to consider which should kick in first. I would expect that
dynamic routing would heal itself slower than LMM routing since MN dictate how
fast the heal should be effected not the routers themselves.

Thus the LMM healing would have effected the next hop LMM which need not be the
next hop AR supported by the dynamic routing healing. I cannot see that the two
would work against each other.

Consider:

a,b are LMM aware routers and of course at the same time ARs (if they are not AR
simultaneously then all the better!!!)

        a.1
         |
a-------------1---------b
|____|___2                                |
               |_____3                     |
                           |___________|

'a' fails and a.1 is an LMM-aware router that can recover from failure. If a.1 is
an AR it would have known the best IP path anyway so dynamic routing would have
done its work to provide the Shortest path towards b. If a.1 is aware of the LMM
hierarchy it can work out who to send to ...that is b...and thus the packet would
still be enroute to b without conflicting routing goals..

so I would like to see a scenario that you can have conflicting interests from the
LMM-aware routing hierarchy and dynamic routing.

I personaly believe there is _no_ such issue. By the way the worst that can happen
is suboptimal dynamic route provision which takes a little longer to get optimal
upon LMM-aware routing healing, but that is what dynamic routing can give on a
need to have basis unless we could push that.

Also we must assume that failure of an LMM and the failure of an AR is either

   1) atomic
    2) decoupled

If it is atomic the recovering LMM-aware router (a.1) would have already
established paths (maybe not superoptimal depending when the dynamic routing comes
up in action)

if it is decoupled then the failure of the LMM agent does not imply failure of
dynamic routing. It would just start LMM routing from a different point.

>    agents. I don't know how paletable that would be.

I don't think we need to go that far...

>    no LMM agents the routing would be restored in 100 millseconds.

Now just a minute...

we are not talking about IGP failures here !!!! If IGP fails then tough...it is
not a problem of the LMM routing hierarchy nor has it create it...It is a problem
for the IGP people to solve how fast the routing should come back..No LMM scheme
would be able to do anything here anyway...

So we want to talk how fast would the LMM-aware router would be able to recover
from the
previous LMM failure. I want to support the following

    "a hell lot faster than any dynamic underlying routing protocol."

How...? lets write a draft... :))

Why ?  Because this is one of the reasons for having the LMM-aware routing (of
course amongst others) fast healing...not another slow one....

Now of course if there were no LMM agents then of course the routing would be
restored in 100 msec. But in this case we wouldn't have  LMM extensions, would we?

(No LMM agents == core MIPv6)  evaluates to true... and I think this is not the
point of having requirements for LMM protocols..

So I think that the above point is invalid...

>    been designed assuming a hierarchical topology are far from future proof.

This is a bit vague since I cannot relate the folloing statement to the specific
issue at hand.

>    autonomous systems which was replaced by BGP which has no such assumption.

Why did they replace it? Because of the original assumption about hierarchical
relationship? How did such assumption impede the operation of EGP so as to be
replaced with BGP (if the assumption had to do anything with the replacement in
the first place???)  ?

And why should that relate to whether LMM encompass multi-level hierarchies. If
the hierarchy can shring to just a depth of one it is just using the very basic
LMM semantics..

Theo

UCL/ Mobile Systems

---

From: Jim Bound <[email-masked]>
Date: Fri, 11 May 2001 23:05:37 -0400 (EDT)

ACK Erik I agree with your come back for measurability and scalability. If
we an determine the cost to do one or the other that will tell us a lot.

/jim