Subject: POOLREQ proposal for failover

From: Kim Kinnear <[email-masked]>
Date: Tue, 21 Dec 1999 11:39:39 -0500

Steve,

These are what I see as the problems to solve concerning
balancing available pools among failover servers: (from my
previous mail on the subject):

    1.  Keeping the available pools of each server balanced in the
    way that they have been configured.  This needs activity at
    two discrete times:

	a.  Whenever either server leases out an IP address, some
	rebalancing of the available pools might need to occur.

	b.  Whenever either server senses a lease expiration or
	release, there is the question of what to do with it.

    2.  At time (b) above, trying to ensure that the client that
    used it last can get it again.

----------------------

I propose the following...

[I wanted to propose junking the POOLREQ, but realized that the
one useful purpose served by the POOLREQ is to signal one server
that its partner has updated it with all relevant information.
This is an issue because it is easy to have one server try to
make available pool balancing decisions prior to it having enough
information to make cogent decisions on the subject.  The POOLREQ
is one way for one server to inform its partner that the partner
now *has* sufficient information to make such a decision.]

Assume the existing protocol mechanisms with slight extensions:
the POOLREQ can be used the secondary to get the primary to
balance its available pools of IP addresses.  And, the BNDUPD can
be used by either server to attempt to "get" or to "give" any
available IP address from or to the other server.

These mechanisms would be used as follows:

1.  Whenever the secondary enters NORMAL state (including the
     first time):

   The secondary would send a POOLREQ to the primary after it has
   updated the primary with all outstanding information (if any)
   that it has the the primary didn't.  [Just like in the current
   draft.]

   The primary would scan all of its available IP address pools,
   and figure out how many IP addresses it should send to the
   secondary.  It would then look at the loadbalancing
   configuration (if any), and send the secondary available
   addresses that had been used most recently by clients that the
   secondary is configured to support.  If the addresses found
   using that technique do not satisfy the needs of the secondary
   server, the primary would send the least recently used IP
   addresses to the secondary.

   The algorithm used here is the same as that used in #3 below,
   except that it is run over *all* of the address pools in which
   these servers are jointly participating in failover.  [This is
   similar to what is in the draft today, with the addition of
   knowledge of the loadbalancing algorithm and sending to the
   secondary IP addresses that match its client base under
   loadbalancing.]

2.  Whenever a lease was granted in NORMAL state:

   The server granting a lease would check the number of available
   addresses it had in the pool from which the lease was granted,
   and how many its partner had.  When the number it needs to meet
   the configuration guidelines increases above a certain
   threshold, it will ask the other server for some specific
   available IP addresses using BNDUPD.  It will ask first for any
   which were recently used by clients which it supports (via
   loadbalancing), if any, and then for the lease recently used IP
   addresses.  [This is all new, not in the current draft.]

3.  Whenever a lease expires or is released in NORMAL state:

   The server processing the expiration or release will compare
   the number of available IP addresses it has in this pool with
   the number of IP addresses that its partner has in this pool,
   and if the difference is above some threshold, it will send
   some available IP addresses to its partner.

   It will send first any IP addresses which were recently used by
   clients which its partner supports (via loadbalancing), if any,
   and then those least recently used.

   It may or may not send the currently expiring or released IP
   address to the partner -- it depends on which server was
   configured to process the most recent client's MAC address due
   to loadbalancing.  [This is all new, not in the current draft.]

In any of the above cases, the fact that a client has recently
used an IP address and that one server or the other is configured
due to loadbalancing to support that client will be considered
when making allocation decisions between servers.  However, the
first priority is to balance the pools of available IP addresses
(within some threshold), and this balance should take priority
over which server is configured to process the client that most
recently leased the IP address.

------------------------

I *believe* that this proposal solves the problems that I've
stated (which flow from, but are not identical to, the problems
that Steve brought up).

Comments?

Thanks -- Kim

---

From: Steve Gonczi <[email-masked]>
Date: Tue, 21 Dec 1999 15:05:39 -0500

Kim, 

This seems a bit complicated. 
If you want to define a self-tuning heuristic, we should make it 
optional and define it in a separate paragraph in the draft.
Seems like a lot of work for a marginal benefit.

Consider, that load balancing is to some degree self-tuning.
If both servers are running, and one server is out of addresses in one
or more pools, and the other server has such addresses, then
new clients that need these will be served by the other server.

sG

---

From: Kim Kinnear <[email-masked]>
Date: Tue, 21 Dec 1999 15:31:39 -0500

>Seems like a lot of work for a marginal benefit.

         I view this proposal as considerably less complicated (in
         practice, not necessarily in description) than the one
         that you made several weeks ago, where servers "own" IP
         addresses even when they are leased.

         If the benefit is truly marginal (something of which I'm
         not convinced), then let's leave the draft the way it is.
         I thought you brought up your proposal because you saw a
         problem with pools getting out of balance.  Perhaps I've
         misunderstood your motivation.

>new clients that need these will be served by the other server.

         I don't see this at all.  My understanding of the
         loadbalancing algorithm and its relationship to failover
         is that it defines exactly which MAC addresses each
         server is allowed to respond to for DHCPDISCOVER requests
         when the servers are in NORMAL state.

         It isn't in any way self-tuning, at least as I understand
         it.  If a server is out of IP addresses, then it doesn't
         respond, and the other server *can't* respond as long as
         it is in NORMAL state.  Is there something I missed about
         the algorithm that makes this untrue?

         I quote from draft-ietf-dhc-loadb-00.txt:

         Section 4, paragraph 5:

           "Resource allocation is accomplished by assigning
           specific hash values to each participating server.

           Each server is assigned ownership of one or more hash
           values, and will only service requests where the STID
           hash matches one of these values.

           Any hash buckes not assigned to servers will result in
           some client ST-s being entirely ignored.  (In some
           scenarios, this may be the desired outcome.)"

         Now, I think it would be *nice* it were self-tuning
         with respect to pool size.  I just don't think it
         is now.

         Thanks -- Kim

---

From: Steve Gonczi <[email-masked]>
Date: Tue, 21 Dec 1999 20:03:45 -0500

>         addresses even when they are leased.

Let me give it a second look. I got a bit carried away.

>         misunderstood your motivation.

I was concerned about the backup server simply running out of leases,
unless we introduced some way of replenishment while in normal mode
(periodic POOLREQ-s, retaining ownership or something along the 
lines of your proposal.)

>         when the servers are in NORMAL state.

You are absolutely right. For a moment, I lost sight of what happens when 
a specific client is not able to get a lease.  

At one point, we have discussed making the loadbalancing calculation into a
bias 
instead of a hard yes/no. e.g.: we could allow a server to respond to a
DISCOVER even if the hash says it should not, based on the 'secs' field, or 
after the client sent the DISCOVER several times. 
That could make loadbalancing self-tuning ...?
My current thinking is that load balancing should only be applied to
DISCOVER-s.

sG