Subject: Anycast & NHRP

From: Joel Halpern <[email-masked]>
Date: Mon, 24 Nov 1997 13:13:48 -0500 (EST)

Yakov and I have been discussing his questions about the use3 of anycast
in NHRP, in order to determine what the actual problem is.  What follows
is a note that he and I have worked up.  I may have still managed to get
some of his concerns wrong, but it should at least help people understand
what the problem is and how it might be addressed.  Thank you.

1. Abstract.

The current Internet Drafts ( draft-ietf-rolc-nhrp-12.txt) defines the
use of NBMA group, anycast, or multicast addresses as follows:

   If an NBMA technology offers a group, an anycast, or a multicast
   addressing feature then the NHC may be configured with such an
   address which would be assigned to the appropriate NHSs.  The NHC
   might then submit NHRP Resolution Requests to such an address,
   eliciting a response from one or more NHSs, depending on the
   response strategy selected.

The following outlines cases where the above behavior could lead to
incorrect operations.  At the end we will propose a replacement section.

2. Clarification of Applicability

Although the paragraph quoted above is quite clear, it is in fact too
general.  As it says later in the same section:

   With the exception of NHRP Registration Requests (see Section 5.2.3
   and 5.2.4 for details of the NHRP Registration Request case), an NHC
   MUST send NHRP messages over a direct NBMA level connection between
   the serving NHS and the served NHC.

Therefore, the first of the two pargraphs ought to be clarified to
indicate that it applies to Registration packets, not requests and
responses.  It also needs to distinguish between "appropriate" for
receiving an initial registration request, and "appropriate" for
receiving requests from NHCs.

3. Multiple Routing Realms

Consider an NBMA network that is used by several routing realms (e.g.,
several VPNs). In this case use of group/anycast/multicast NBMA
addresses by NHRP could result in crossing routing realm's boundaries. This
may cause establishment of L3 paths that cross such boundaries, which
in turn may result in incorrect operations.

Thus unless there is either (a) a way to guarantee that the whole NBMA
network is used just by a single routing realm, or (b) a way to
guarantee that NHRP messages wouldn't be allowed to cross routing
realm's boundaries, anycast/group/multicast addresses of NHSs on such a
network have to be on a per routing realm basis.  Procedures by which
either (a) or (b) are satisfied are outside the scope of this document.
Thus, currently anycast/group mechanisms can not be used for NHRP 
registration on NBMAs which are shared by multiple domains.  To avoid
customer misconfiguration, it may be appropriate that such (anycast)
behavior default to off, rather than on, with the concomitant loss of
ease of installation.

Note that using anycast/group/multicast addresses on a per routing
realm basis (rather than on a per NBMA network basis) has negative
impact on NBMA routing, as such addresses are not aggregatable for the
purpose of NBMA routing procedures.  This situation is still
more scalable than the use of an anycast address per LAG.

4. Multiple egress NHSs

When a destination could be reachable through more than one egress NHS
(e.g., a path to destination includes more than one segment through an NBMA
network)
use of anycast/group/multicast address may result in a situation where
an NHC that sends a Resolution Request for the destination would
receive a Resolution Reply from an NHS that is not along the path from
the NHC to the destination, even if the NHS is along a path to the
destination (e.g., this could happen when the NHC is in the path from
the NHS to the destination). Use of the information contained in the
Reply by the NHC may result in incorrect operations (e.g., persistent
forwarding loop).

To avoid the above problem one need to guarantee that either (a) a
destination is always reachable though at most one NHS, or (b) the set
of NHSs to which an NHC could submit a Resolution Request are in the
same LIS/LAG as the NHC. Procedures for satisfying (a) are outside the
scope of this document.  Use of anycast/group/multicast addresses on a
per LIS/LAG basis allows to satisfy (b). However, note that using
group/anycast/multicast addresses on a per LIS/LAG basis (rather than
on a per NBMA network basis) has negative impact on NBMA routing, as
such addresses are not aggregatable for the purpose of NBMA routing
procedures.  As an alternative technique for achieving (b), one could
note that the anycast registration procedure results in connectivity
between an NHC and an NHS serving its LIS/LAG.  The current behavior
should be tightened to require that that connection be used for sending
request.

5. Partition

There is one additional case where the use of anycast even for registration
may be problematic.  Registration requests are forwarded, according to 
routing, from a receving NHS towards the home LIS/LAG.  This is designed to
deliver the registration request to an appropriate NHS.  Normally, this
will work.  However, it is possible due to failures of routers or 
conenctions, or just due to oddities of routing, that the routed path from
the first NHS towards the home LIS/LAG may actually exit the NBMA.  If
that occurs, the Registration will be undeliverable.  No response will be
received.  The specification should note this, and indicate that the client
MUST NOT send NHRP resolution requests until the registration succeeds, 
and SHOULD retry the registration request periodically.

6. Replacement Paragraph

For issues 2, 3, and 4, the paragraph described in 1 clearly needs to be
clarified.

   If an NBMA technology offers a group, an anycast, or a multicast
   addressing feature then the NHC may be configured with such an
   address (appropriate to the routing realm it participates in) which
   would be assigned to all NHS serving that routing realm.  This address
   can then be used for establishing an initial connection to an NHS to
   transmit a registration request.  This address may not be used for
   sending NHRP requests.  The resulting VC may be used for NHRP requests
   if and only if the registration response is received over that VC,
   thereby indicating that one happens to have anycast connected to an
   NHS serving the LIS/LAG.  In the case of non-connection oriented
   networks, or of multicast (rather than anycast) addresses, the addres
   MUST NOT be used for sending NHRP resolution requests.

We will also need to examine the remaining text to determine if there are
other places that need to be clarified.

Yours,
Joel M. Halpern                         [email-masked]
Newbridge Networks Inc.

---

From: Albert Manfredi <[email-masked]>
Date: Mon, 24 Nov 1997 14:38:38 -0500

Joel Halpern wrote:

[ ... ]

>    MUST NOT be used for sending NHRP resolution requests.

I'm probably missing something, but I don't see why multicasts
should even be mentioned at all in this NHRP context. I'd say that
anycast or static configuration are allowable.

If one specifies only anycast or static configuration, for
registration, is one not guaranteed that the NHS will be in the same
LIS as the NHC?

And I would further stipulate that the same VC one used for
registration can only be used for resolution requests _if_ there is
only one NHS in that LIS. Not _just_ if the registration response is
received over that VC.

A large ATM LIS could have many routers connecting to the outside
world. Just because a client received a registration response from
one particular router (NHS) does not guarantee that this is the
correct first hop in a routed path, right? The actual shortcut VC
might be in the opposite direction. So I would simply separate the
two issues completely, unless, of course, that NHS router is the
only one in the LIS.

If a LIS has many NHSs, how are they all to know about a given NHC,
when responding to a resolution request from outside? They know
because they all use SCSP.

So again, my take would be to not even mention multicast for
registration.

Bert
[email-masked]

---

From: Joel Halpern <[email-masked]>
Date: Mon, 24 Nov 1997 14:46:58 -0500 (EST)

> anycast or static configuration are allowable.
I mention it because the working group chose to include it in the original
text.  I do not hink multi-cast is the likely case, but it is important to
understand how it behaves.

> registration, is one not guaranteed that the NHS will be in the same
NO.  For a simple example if one had a private corporate ATM network, one
might well use a single anycast address for all NHS.  Hence when one
connected using anycast, one might get any NHS, not necessarily one that
is serving you home LAG.
If we require one anycast per LAG, we might as well not bother.

> ...

Yours,

---

From: Albert Manfredi <[email-masked]>
Date: Mon, 24 Nov 1997 15:18:09 -0500

Joel Halpern wrote:

[ ... ]

> If we require one anycast per LAG, we might as well not bother.

Okay, point taken, but that wasn't the crucial part. If you have a
corporate network with very few egress points, then it's clear that
having an NHS per LIS is not important. The ultimate goal of that
NHS to LIS ratio is, I think, to ensure that the ATM shortcut be
established based on a sensible route. Since ATM knows nothing about
the L3 topology, having the IP frame take the "correct" IP route
before establishing the best ATM shortcut makes a lot of sense.

In a large corporate ATM net with few egress points, each LIS would
not need an NHS to ensure an optimum (or just plain workable)
shortcut. The large corporate net is not a crucial part of the
Internet's infrastructure.

Still, the fundamental point I think remains: static configuration
and anycast alone ought to be specified for NHRP registration. I
don't disagree with you that the behavior of multicast shouldn't be
investigated. I merely question its relevance.

Bert

---

From: Diane Schuster <[email-masked]>
Date: Fri, 05 Dec 1997 09:41:05 -0500

Hi,

It seems to me that the problem of using anycast to locate a server will
apply to many more applications than just NHRP. Anycast, by definition,
implies that any server that can provide this service is just as good as
any other server that can provide the service. If the ability of a
server to serve a particular client depends on "routing realm"
affiliation (which I would expect to be quite common), then the basic
premise for using anycast is not met. 

Perhaps a better service for NHRP and many other applications to use to
locate an appropriate server would be "manycast." This would be a
service provided by the NBMA network whereby there are MANY servers
which can  provide the service being sought equally well. However, not
ANY server in the NBMA network can provide the service equally well. The
difference between manycast and group or multicast addresses is that
only one server receives a packet sent to a manycast address (i.e. not
all servers with the manycast address receive the packet).

-- 
Schuster Consulting	
[email-masked]
(919) 831-9019

---

From: Albert Manfredi <[email-masked]>
Date: Fri, 05 Dec 1997 10:39:25 -0500

Diane Schuster wrote:
> any other server that can provide the service.

Not really. Anycast implies that the topologically _closest_ server
answering to that particular anycast address is the best server to
use. In principle, that one topologically closest could also be the
_only_ one a particular host can use, at that time.

> If the ability of a

Interesting idea. As far as I can tell, the real difference between
your manycast and anycast is that manycast would depend on a set of
criteria which are _not_ solely proximity, based on ATM topology.

Bert

---

From: Paul Koning <[email-masked]>
Date: Fri, 5 Dec 1997 12:10:24 -0500

>>>>> "Albert" == Albert Manfredi <[email-masked]> writes:

 Albert> Diane Schuster wrote:
 >>  Hi,
 >> 
 >> It seems to me that the problem of using anycast to locate a
 >> server will apply to many more applications than just
 >> NHRP. Anycast, by definition, implies that any server that can
 >> provide this service is just as good as any other server that can
 >> provide the service.

 Albert> Not really. Anycast implies that the topologically _closest_
 Albert> server answering to that particular anycast address is the
 Albert> best server to use. In principle, that one topologically
 Albert> closest could also be the _only_ one a particular host can
 Albert> use, at that time.

Not quite, because PNNI routing does not impose specific rules for
route selection.  While it is reasonable to expect that an anycast
call will be routed to the "best" destination, "best" does NOT equal
"topologically closest".  It may be the destination with the lowest
total delay, largest available path bandwidth, lowest cumulative admin
weight, or any combination of those, or none of the above.

	paul

---

From: Albert Manfredi <[email-masked]>
Date: Fri, 05 Dec 1997 12:38:48 -0500

Paul Koning wrote:

> > Not really. Anycast implies that the topologically _closest_
> > server answering to that particular anycast address is the
> > best server to use. In principle, that one topologically
> > closest could also be the _only_ one a particular host can
> > use, at that time.
> 

Hmmm. But Anycast would typically use scoping, no? Which means it
looks up the PNNI peer group hierarchy to find the first available
server with that address. But okay, you're saying that other
criteria can also be established.

In principle then, perhaps one could set anycast up to do what Diane
called manycast. Assuming the rules she would impose can somehow be
coded into rules understandable by the PNNI architecture.

Bert

---

From: Paul Koning <[email-masked]>
Date: Fri, 5 Dec 1997 13:25:54 -0500

Albert> Paul Koning wrote:
 >> call will be routed to the "best" destination, "best" does NOT
 >> equal "topologically closest".  It may be the destination with the
 >> lowest total delay, largest available path bandwidth, lowest
 >> cumulative admin weight, or any combination of those, or none of
 >> the above.

 Albert> Hmmm. But Anycast would typically use scoping, no? Which
 Albert> means it looks up the PNNI peer group hierarchy to find the
 Albert> first available server with that address. But okay, you're
 Albert> saying that other criteria can also be established.

If you ignore scope, the set of possible targets for an anycast call
is any one of the nodes that advertise that address.  When you include
scope, you take a subset, which is all the nodes advertising the
address that are reachable without going above the specified level in
the PNNI hierarchy.

Once you have this set, the PNNI route selection chooses one member of
the set using implementation-dependent criteria.

It certainly is possible for an implementation to search for a target
anycast from the bottom of the hierarchy upward, but that's not
required and implementations may do otherwise.  For example, I did an
implementation which effectively does a route calculation to the set
of nodes that match on address and scope, where selection among them
is done using the conventional "equal cost path splitting" approaches,
and the evaluation of which is best is done by a metric optimization
as would be done for "ordinary" call routing.  So the destination
chosen may not be the one that involves the fewest hiearchy levels (so
long as scope is satisfied).  And it may not have the lowest hop count
(except perhaps if you chose admin weight as the metric to be
optimized). 

>From the point of view of protocol and application design, you really
have to follow the rule that a particular anycast address maps to a
set of servers any one of which is equally capable of giving you the
answers you want.  And the call routing rule is that you will be
connected to a random member of that set (subject only to the scope
restriction) -- and indeed not necessarily to the same one twice in a
row even if the topology has not changed at all.

If you need other properties, i.e., the servers aren't all
interchangeable or you need to prefer one over the other, then you
cannot use anycast as the way to chose amongst them.

	paul

---

From: Eric W Gray <[email-masked]>
Date: Sun, 07 Dec 1997 06:47:17 -0500

Bert,

...

It might be that you are being entirely too picky about word usage
here.  Your observation that "toplogically closest" is more accurate
than "any" is perhaps disingenuous - since "topologically closest" 
(in the ATM topology) is unlikely to be significantly different from
"any" when topological proximity is clearly unrelated to usage - as
is the case when an ATM switch is selecting a server based on anycast
address when the selection of an "appropriate" server should be based
on something other than (ATM) topological proximity.

-- 
Eric Gray
EMail at mailto:[email-masked]
Phone (603) 659-6859
FAX   (603) 659-6859
================================================
Have a nice day!

---

From: Albert Manfredi <[email-masked]>
Date: Sun, 07 Dec 1997 16:42:46 -0500

> on something other than (ATM) topological proximity.

Oh, I don't know about "disingenuous." To me, it depends on how one
plans to use anycast. ATM topological proximity as a criterion to
reach most or all of an ATM net's servers can make a lot of sense,
depending how one designs the ATM net, and should probably be used
extensively for that purpose. I mean, once you have a tool, you can
certainly put it to good use. Just like DHCP is useful for more than
just economizing on IP address space assigned to ISPs. (And, as was
discussed, you can ignore "scope" and implement other criteria for
anycast, if that's desirable for any reason.)

I never assumed that anycast would only be used in cases where "any"
server is equally good. To me, it's a convenient way of finding the
myriad servers that IP over ATM schemes demand. Once a server is
found using anycast, it is conceivable that the client register with
that server and continue to use it, unless it can't be reached for
any reason, at which point the client would register with another
one, also reached with that same anycast address. And registering
with a server should not necessarily mean that the server's unique
ATM addressed must be used after this registration process, does it?
The client could continue to use the anycast address.

In short, I disagree that anycast should be limited to uses where
"any" server is equally good. I think it's potentially a very
powerful tool which would be used for virtually any client-server
situation. To me, anycast should be used for ATM in ways that "all
1s" broadcast or perhaps multicast is use in IEEE LANs.

Bert

---

From: Navin Agrawal <[email-masked]>
Date: Sat, 10 Jan 1998 16:37:55 +0000 (GMT)

Hello,
	
	I have one query regarding NHS. What happens if the NHS of a
LIS fails? I mean,if the NHS gets failed, How hosts will resolve their
NHRP query?
	Thanks in advance.

regards
-Navin

X-Info: To unsubscribe, email [email-masked]' with
X-Info: 'unsubscribe ion' in the body of the message.

---

From: Albert Manfredi <[email-masked]>
Date: Sun, 11 Jan 1998 16:30:02 -0500

> LIS fails? I mean,if the NHS gets failed, How hosts will resolve their

If the NHS fails, one answer is that they might not be able to
establish an IP session anyway, let alone a shortcut, to another IP
subnet. If the NHS is the only router out of your LIS, say the
default IP router, you'd be out of luck.

Alternatively, there might be other NHRP-capable routers out of this
LIS. Or at least another router, which would at least allow the IP
session to be established, even if sans shortcut.

The problem of a failed NHS is not really different from a problem
of failed IP router.

Bert