Subject: IPv6 w.g. Last Call on "IPv6 Host to Router Load Sharing"

From: Bob Hinden <[email-masked]>
Date: Mon, 14 Jan 2002 11:23:04 -0800

This is a IPv6 working group last call for comments on advancing the 
following document as an Proposed Standard:

	Title		: IPv6 Host to Router Load Sharing
	Author(s)	: B. Hinden
	Filename	: draft-ietf-ipv6-host-load-sharing-00.txt
	Pages		: 4
	Date		: 10-Jan-02

Please send substantive comments to the ipng mailing list, and minor
editorial comments to the author.  This last call period will end two
weeks from today on January 28, 2002.

Bob Hinden / Steve Deering

--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Francis Dupont <[email-masked]>
Date: Tue, 15 Jan 2002 15:01:41 +0100

There was a long and not very clear discussion about the level of
requirement for this proposal...
The current spec (RFC 2461 6.3.6 end of 1)) says:

        An implementation may choose to always return the same router or
        cycle through the router list in a round-robin fashion as long
        as it always returns a reachable or a probably reachable router
        when one is available.

i.e. there are two proposed solutions with a small may (I interpret
this as an implementation hint). The new proposal is:

        An implementation SHOULD pick routers from the default router
        list in random order while making sure it always returns a
        reachable or a probably reachable router when one is available.

I have two concerns about this:
 - the random order is not defined (this is a formal concern because
   the intention is clear)
 - we moved from an implementation hint to a SHOULD, this will make
   old implementations not conformant and in some situations there
   can be far better solutions. So I prefer to get a MAY (enough to
   get this proposal implemented in the future by everybody without
   harm) and an explicit reference to the default router preference
   draft (first because I am in favor of this from the beginning,
   i.e. since many years, second because this is the best reply to
   the "special case" argument (not so special case because 100% of
   the IPv6 networks I used have several routers with a better one)).

Regards

[email-masked]

---

From: Bob Hinden <[email-masked]>
Date: Wed, 16 Jan 2002 11:02:56 -0800

Francis,

>  - the random order is not defined (this is a formal concern because

I think most people will understand the intent of "random order".  Does 
anyone else think more definition is needed?  If so, I could add some text.

>    the IPv6 networks I used have several routers with a better one)).

I disagree.  The intent of the document is to change the behavior from a 
hint to a common behavior.  Hence the SHOULD.  Note that the current "hint" 
for round robin is less than perfect.

I think the current implementations are covered by the SHOULD.  If it was a 
MUST there might be a problem with current behavior.

In Salt Lake City where the issue of combining this with the default router 
preferences was discussed, the consensus was to keep this separate.  The 
plan was to also add this behavior to the default router preferences draft 
for the case where there are multiple routers at the same preference.

Bob

---

From: Francis Dupont <[email-masked]>
Date: Thu, 17 Jan 2002 10:54:33 +0100

>  - the random order is not defined (this is a formal concern because
   
   I think most people will understand the intent of "random order".

=> as I've written this is a *formal* concern... (formal = on the form)

   I disagree.  The intent of the document is to change the behavior from a 
   hint to a common behavior.  Hence the SHOULD.

=> if this is the intent the document is underspecified (i.e. it have to
remember when the default router selection is done (this is at the
beginning of 6.3.6)). There is too a real issue about the required
minimal default router list length (two issues in fact: do we allow
small implementations to use a short list (possibly shorter than
the real number of default routers)? What is the reply to the bad guy
who tries to give us 2^64 default routers?) Note there is something
about this in 5.3 GC and Timeouts and 6.3.4 Processing rcvd RAs.

   Note that the current "hint" for round robin is less than perfect.

=> I agree but my concern is more about the requirement level.
   
   I think the current implementations are covered by the SHOULD.

=> they are not compliant with the SHOULD, not a real problem for round
robin (as it can be looked at a bad random order) but implementations
which keep the same router have a very different behavior (than the
recommended one).

   If it was a MUST there might be a problem with current behavior.

=> I agree but a SHOULD in a draft standard (i.e. the intent) is
really quite strong.
   
   In Salt Lake City where the issue of combining this with the default router 
   preferences was discussed, the consensus was to keep this separate.

=> my concern is there are a lot of situations where some potential
default routers are better than others. Without the default router
preferences the only way to control the default router choice is
to use zero router lifetimes. This is far too drastic and can't
express any kind of nuance... So I'd really like to get preferences
before or at the same time than random selection.

   The plan was to also add this behavior to the default router
   preferences draft for the case where there are multiple routers at
   the same preference.
   
=> we agree but there is clearly a problem in priorities.

Regards

---

From: JINMEI Tatuya / 神明達哉<[email-masked]>
Date: Thu, 17 Jan 2002 22:25:45 +0900

>>>>> On Thu, 17 Jan 2002 10:54:33 +0100, 
>>>>> Francis Dupont <[email-masked]> said:

   

> which keep the same router have a very different behavior (than the

I agre with Francis.  And, actually, our implementation (i.e. all *BSD
variants) basically keeps the same router (of the same priority), so
it will be non-compliant with the SHOULD.  I believe our
implementation is not the only example of this "non-compliant" behavior.

If the SHOULD is really the consensus, I'll change the implementation
so that it will be compliant again.  However, I'm still not fully
convinced that the change is worth making (perhaps many) existing
applications non-compliant.

As Francis pointed out, there are many cases of "several default
routers with a better one."  In this case, picking up random routers
(without the knowledge of router preference) will just be meangless,
because we'll eventually converge on the better router with redirect
messages.

> => I agree but a SHOULD in a draft standard (i.e. the intent) is

I agree, too.

So, I'm okay with a MAY, instead of the SHOULD.

I'll probably okay with a SHOULD if the proposal is combined with the
router preference proposal, because the preferenece proposal is also a
new stuff.
   
>    preferences was discussed, the consensus was to keep this separate.

Yes, but I disagree with the SHOULD as well as separating those two.

					JINMEI, Tatuya
					Communication Platform Lab.
					Corporate R&D Center, Toshiba Corp.
					[email-masked]
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Pekka Savola <[email-masked]>
Date: Thu, 17 Jan 2002 16:28:06 +0200 (EET)

> implementation is not the only example of this "non-compliant" behavior.

True.
 
> because we'll eventually converge on the better router with redirect

I agree, as I commented on this when the draft first appeared.

IMO it's _much_ better practise have predictable behaviour.
Load-distribution is not that.  When there are two default routes of equal
weight, I want to only use one, and if the one I use fails, failover
nicely to the other.

-- 
Pekka Savola                 "Tell me of difficulties surmounted,
Netcore Oy                   not those you stumble over and fall"
Systems. Networks. Security.  -- Robert Jordan: A Crown of Swords

---

From: "Jim Fleming" <[email-masked]>
Date: Thu, 17 Jan 2002 08:58:44 -0800

----- Original Message -----
From: "Pekka Savola" <[email-masked]>
To: "JINMEI Tatuya / ????" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, January 17, 2002 6:28 AM
Subject: Re: IPv6 w.g. Last Call on "IPv6 Host to Router Load Sharing"

> weight, I want to only use one, and if the one I use fails, failover

With RIFRAF Routing, people can use both links with 50/50 (coin-toss)
selection for traffic when the links are working. If one link fails, then
all of the traffic has to go to the other link. If that link fails, they are
out
of luck. RIFRAF could be easily extended to 3, 4, or more links, with
the traffic distributed 33%, 25%, etc. Since the current implementation
does a 50/50 coin toss, and selects 4 bits for the packet marking, one
can also build a 50/50 system with 16 traffic flows. Couple that with the
16 QoS classes and you have lots of ways to go.

Jim Fleming
2002:[IPv4]:000X:03DB
http://www.IPv8.info

---

From: Bob Hinden <[email-masked]>
Date: Mon, 21 Jan 2002 17:22:08 -0800

Instead of trying to reply each email I will attempt to summarize and 
respond to the comments in the last set of email.

1) Requirement level of SHOULD would make existing implementations 
non-compliant.

I believe that the definition of SHOULD (see RFC2119) allows for alternate 
behavior if there is a good reason for it.  I think this can cover both 
better algorithms for the selection of default routers and if the 
implementation predates this spec.  I will note that the intent of the this 
document is to change the default behavior of implementations.  That is way 
I think SHOULD is appropriate.  Also, I think this is independent of the 
level of the standard (Proposed, Draft, Standard).

2) Relationship of this change to Neighbor Discovery and Default Router 
Preferences draft.  Should they be combined?

The reason why the conclusion of the w.g. in Salt Lake City was to keep the 
documents separate is that this document is changing the base behavior of 
Neighbor Discovery when there is more than one default router.  The Default 
Router Preferences draft is an optional mechanism (see the draft).  It may 
not always be used and/or implemented.  It was agreed that the Router 
Preferences draft should also incorporate this behavior when there are more 
than one router of the same preference.

Note this draft is not replacing the router preferences draft.  Both are 
expected to go forward.  This draft only changes the case when there is 
more than one default routers.  It does not provide the optimum solution 
when there are multiple routers with different characteristics.  That case 
is covered by the router preferences draft.

3) General statement load-distribution is not predictable behavior and is 
undesirable (e.g., using a single router until it fails is better).

There is considerable current practice that disputes this.  The specific 
case that this draft addresses (host to router traffic) is very widely 
deployed using protocols such as VRRP and Cisco's HSRP.  The advantage of 
distributing the load over multiple routers is that it takes advantage of 
the resources that are available and insures that the backup is working 
when it is needed.  Backup systems that are not being used have a nasty 
habit of not working when they are needed.

Regards,
Bob

---

From: Robert Elz <[email-masked]>
Date: Tue, 22 Jan 2002 12:54:03 +0700

Date:        Mon, 21 Jan 2002 17:22:08 -0800
    From:        Bob Hinden <[email-masked]>
    Message-ID:  <[email-masked]>

  | The reason why the conclusion of the w.g. in Salt Lake City was to keep the 
  | documents separate is that this document is changing the base behavior of 
  | Neighbor Discovery when there is more than one default router.  The Default 
  | Router Preferences draft is an optional mechanism (see the draft).  It may 
  | not always be used and/or implemented.

My general feeling is that without router preferences being implemented,
the new load sharing stuff should not be implemented either.   That is,
it is fine, but only if I'm guaranteed a way to override it.

Expecting everyone to implement load sharing, and making preferences just
an optional add on that some can implement if they feel like it, is not
acceptable.  (The "some" being the hosts that receive the RA's of course,
I'll simply assume that any router worth having will implement router
preferences).

Because of that, I'd much prefer that the two be in the same document, so
they get exactly the same status.

kre

---

From: Francis Dupont <[email-masked]>
Date: Tue, 22 Jan 2002 15:55:26 +0100

=> what we asked is not really a combinaison, it was to get the
default router preference not after the load sharing.

   The Default Router Preferences draft is an optional mechanism (see
   the draft). It may not always be used and/or implemented.

=> I'd like to see the same argument than in (1) applied to the default
router preferences!

=> you know what we'd (people who did comments) like: to get both.

   There is considerable current practice that disputes this.

=> there are situations where load sharing is better and others where it
is a disaster. The only good solution is to get load-sharing with a way
to control it, i.e. the default router preferences.
I sent in July 1995 a mail about default router preferences, I never
changed my opinion about this...

Regards

---

From: Bob Hinden <[email-masked]>
Date: Tue, 22 Jan 2002 17:02:47 -0800

kre,

>I'll simply assume that any router worth having will implement router

I think there are two cases.  One is where there are two routers that 
provide equivalent service.  The other is where the are two or more routers 
that have different characteristics (performance, connectivity, etc.).  In 
the first case the proposed load sharing mechanism is fine.  In the second 
case the router preferences is useful.

I think that the first case is very common so it is worthwhile to keep the 
documents separate.

>they get exactly the same status.

I think in practice both will be implemented and used as needed.  Neither 
is very complicated.  There is even a rumor that one company is already 
shipping router preferences :-)

Bob

p.s. I don't really care too much about this (one or two documents) as long 
as we get the mechanisms.  I will continue to push the current path as it 
was the conclusion reached in Salt Lake City.  I asked the question in my 
presentation and the conclusion was to continue as a separate document.

---

From: JINMEI Tatuya / 神明達哉<[email-masked]>
Date: Thu, 24 Jan 2002 02:44:34 +0900

>>>>> On Mon, 21 Jan 2002 17:22:08 -0800, 
>>>>> Bob Hinden <[email-masked]> said:

> behavior if there is a good reason for it.

Yes, but the wording is not very clear at least to me.  In particular
it is not clear how to define the "good reason."  But,

> I think this can cover both 
> implementation predates this spec.

if this is the common interpretation of SHOULD and existing
implementation (that seem to be "non-compliant" to the new spec) can
just exit as is, I'm perhaps okay with it.

> Preferences draft.  Should they be combined?

As for this I share the same opinion with Francis.  I actually don't
stick to combining the two drafts, but would like to point out that
the described load-sharing can often be meaningless without the router
preferences.  There are of course some cases where the load-sharing is
useful even without the preferences, as you mentioned in this thread.

So, again, I guess my position depends on the constraint level of the
SHOULD.  If it is a strong constraint like I imagined, I disagree with
the spec because the spec is not always useful.  If it is not so
strong (like you mentioned above), I'm perhaps okay because the spec
is sometimes useful.

> undesirable (e.g., using a single router until it fails is better).

(just to make my position clear) I don't care about this argument.a

					JINMEI, Tatuya

---

From: Robert Elz <[email-masked]>
Date: Fri, 25 Jan 2002 17:46:38 +0700

Date:        Tue, 22 Jan 2002 17:02:47 -0800
    From:        Bob Hinden <[email-masked]>
    Message-ID:  <[email-masked]>

  | I think there are two cases.  One is where there are two routers that 
  | provide equivalent service.  The other is where the are two or more routers 
  | that have different characteristics (performance, connectivity, etc.).  In 
  | the first case the proposed load sharing mechanism is fine.  In the second 
  | case the router preferences is useful.

Sure.

  | I think that the first case is very common so it is worthwhile to keep the 
  | documents separate.

Ah, I think I see the trap.   You're concerned about the router end of
things, I'm concerned about the hosts.

There's no way a random host implementation can know which kind of
environment it is about to be thrust into - it has no idea at all (without
router preferences) whether the two routers sending RA's are in any way
equivalent.   A host that implements load sharing, and doesn't implement
router preferences, is simply a disaster in your above 2nd scenario.

  | I think in practice both will be implemented and used as needed.  Neither 
  | is very complicated.  There is even a rumor that one company is already 
  | shipping router preferences :-)

I have no doubts that router vendors will ship router preferences.  That's
not what I'm concerned with.   I'm concerned that host implementors implement
it, or that they don't implement load sharing.  That is, a host that simply
picks one default router, and sticks with it as long as it is functional
(until NUD says go elsewhere) I can cope with.   A host that implements router
preferences I can cope with.   A host that simply insists on picking any
random router that is sending RAs and sends packets at it, and distributes its
load, I cannot cope with - that combination must not be allowed to happen.

  | p.s. I don't really care too much about this (one or two documents) as long 
  | as we get the mechanisms.

If we get both mechanisms, actually implemented in hosts, that's just fine.
If we get Router Prefs as a PS, and require its implemenation in hosts, then
that's also fine.   If your doc says "MUST NOT implement unless router
prefs is also implemented", then that's fine as well.

There are lots of ways forward here - just the one your previous message
suggested, where load sharing is a PS expected to become the default on all
hosts, and Router Prefs is a random option that some people might happen to
choose to implement, is the worse possible scenario - worse than having
neither of them.

kre

---

From: Tim Chown <[email-masked]>
Date: Fri, 25 Jan 2002 15:04:11 +0000 (GMT)

> load, I cannot cope with - that combination must not be allowed to happen.

I share Robert and Francis' view.   There is also the possibility that
misconfigured hosts will send RAs; having these targetted by load sharing
hosts will only add to the pain of such events.
 
> prefs is also implemented", then that's fine as well.

I would assume a reasonable mode of working is to only load share between
routers of equal preference, rather than some "random" algorithm picking 
routers weighted by preference.  

Tim

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 29 Jan 2002 16:42:24 +0100 (CET)

> (until NUD says go elsewhere) I can cope with.   A host that implements
> router preferences I can cope with.   A host that simply insists on picking
> any random router that is sending RAs and sends packets at it, and
> distributes its load, I cannot cope with - that combination must not be
> allowed to happen.

Hmm - I know of a large server vendor who has been spreading the load
over multiple default routes for IPv4 for about 10 years.
I guess the difference is that even with ICMP router discovery (RFC 1256)
there is a preference field so the admins have a knob.

Is it just the absense of such a knob to the administator that is lacking?
Or is there a need to have a rich set of preferences? (RFC 1256 has 256
different values vs. router preferences has only 3 values).

   Erik

---

From: Francis Dupont <[email-masked]>
Date: Wed, 30 Jan 2002 17:21:42 +0100

=> the knob.

=> no, I believe we can live with only 3 values.

Regards