Subject: RE: runumbering (was: Re: IPv6: Past mistakes repeated?)

From: Christian Huitema <[email-masked]>
Date: Tue, 25 Apr 2000 17:13:57 -0700

> Now consider the NATv6 alternative.  The average net admin is already
> comfortable with NAT at the ISP boundary (hell, some even like it).
> She will already be running NAT, if for no other reason than to deal
> with IPv4-IPv6 transition.  NATv6 is much less onerous than NATv4,
> because the address mapping is one to one (and in fact is probably
> a simple prefix rewriting, not a large table lookup), not many to
> one like with IPv4.  What's more, NATv6 will give you some rudimentary
> form of multihoming (can advertise different prefixes at different
> ISPs)!

I think your description is somewhat biased, Paul. Suppose that you execute
the strategy you describe, and that the address 10.3.2.1, that was mapped to
129.87.64.32, gets transparently remapped to 130.123.45.67. It seems to me
that you are going to loose all existing TCP connections, just like with any
other form of renumbering. Suppose then that the external prefix of your
site, 129.87.64.0/25, was used in the firewall access control lists of your
business partners. Well, unless these partners somehow reprogram their
firewall, the packets sourced from 130.123.45.67 are not going to get
through. What? Numeric addresses in ACL is bad? Come on -- if none were
used, if we only used DNS names, then renumbering would not be the problem
that it is known to be! Besides, do you believe that using a NAT rather than
plain renumbering would make your external DNS records to upgrade any
sooner?

Suppose now that you want to use NAT for multihoming. Suppose then that I
really care that my TCP connection survives when I loose contact with ISP-A,
the one that provided me with the prefix 129.87.64.0/25. Can I just go on
routing the packets through ISP-B, that provides me with the address
130.123.45.0/25? Well, you can source them, but not receiving them, unless
you convince ISP-B to announce your other /25 prefix in its BGP tables --
just like the old fashion no-NAT multihoming. And then, how exactly do you
tell your customers that, among the two IP addresses of your web site, they
should really use 130.123.45.3, not 129.87.64.5?

Turn it any way you want, TCP sessions can only survive renumbering through
end to end mechanisms, in effect some form of TCP-NG, and effective
multihoming requires either a single address prefix supported by multiple
ISPs, or end to end smartness to pick the best of N addresses (hey, we could
use *your* work for that!) As Steve pointed out, trying to convince your ISP
to announce random non topological prefixes will get you laughed out of
court. There is, in fact, no alternative to end-to-end smartness -- the
smarter your end systems, the more likely they are to use the network
correctly. Now, if your systems are smart and now how to choose one of many
available addresses, then we have a solution with v6 multi-homing -- the
support of multiple prefixes per site, multiple addresses per interface.

---

From: Karl Auerbach <[email-masked]>
Date: Tue, 25 Apr 2000 18:41:46 -0700 (PDT)

> end to end mechanisms...

Which raises the interesting (to me anyway) question: Is there value in
considering a new protocol, layered on top of TCP, but beneath new
applications, that provides an "association" the life of which transcends
the TCP transports upon which it is constructed?

I believe that if we had such a protocol that it would be a useful tool to
solve many of the juggling acts that transpire under the heading of
"mobile networking" as well as providing a way to continue (or
"resume") connectivity after IP address changes.

(I will, of course, be suitably embarrassed if someone points out that
work is already going on to do this.)

		--karl--

---

From: Tripp Lilley <[email-masked]>
Date: Wed, 26 Apr 2000 02:49:32 +0000 (/etc/localtime)

> "resume") connectivity after IP address changes.

Like you, I'll shuffle away in embarassment if this is already going on,
but it seems to me a heck of a good idea. I really don't like the idea of
tunnelling all the way back home to my measly DSL line when I'm sitting at
Interop behind a pair of DS3's, just so I can keep my sessions persistent
:)

I once accidently put my Sparc to sleep before I left for the evening, and
came back in the morning, realizing my mistake. I unslept it, and my ssh
session back home was still kicking. I was pleasantly surprised.

I shouldn't have been.

That is, that's the way it should work... I should be able to put my
laptop to sleep, hop on the plane, land in San Francisco, pull it back
out, unsleep it, and pick up where I left off, _without_ tunneling all the
way back home (ie: MobileIP). That's about as bad as going through MAE
East just to get to the ISP down the street...

And, yes, I'd gladly implement apps that were "association-aware", if such
a mechanism existed (and, Karl, if it doesn't exist, and you start hacking
it out, I'll write "running code" for ya :) ).

-- 
   Tripp Lilley  *  [email-masked]  *  http://stargate.sg505.net/~tlilley/
------------------------------------------------------------------------------
  "I get a lot of letters like, 'Dear John, I've got a dead alien. What
   should I do with it?'  One word: barbecue!"

   - John Lovitz, A Yellow Pages commercial

---

From: Bill Manning <[email-masked]>
Date: Tue, 25 Apr 2000 20:16:10 -0700 (PDT)

% > Turn it any way you want, TCP sessions can only survive renumbering through
% > end to end mechanisms...
% 
% Which raises the interesting (to me anyway) question: Is there value in
% considering a new protocol, layered on top of TCP, but beneath new
% applications, that provides an "association" the life of which transcends
% the TCP transports upon which it is constructed?
% 
% I believe that if we had such a protocol that it would be a useful tool to
% solve many of the juggling acts that transpire under the heading of
% "mobile networking" as well as providing a way to continue (or
% "resume") connectivity after IP address changes.
% 
% (I will, of course, be suitably embarrassed if someone points out that
% work is already going on to do this.)
% 
% 		--karl--

 The most visable one was implemented at UCLA in 1996. Built using 
 a per-session 32bit sequence. My idea was to use the DNS lable instead
 of a fixed 32bit number. A bit harder... :)

--bill

---

From: Keith Moore <[email-masked]>
Date: Tue, 25 Apr 2000 23:51:17 -0400

> the TCP transports upon which it is constructed?

been there, done that.  yes, it is valuable.  but it is expensive
in terms of the amount of protocol overhead and support that you 
need to make it work reliably in the face of various failures.
and it can be slow to recover from such failures.

for instance, you have to build your own data acks on top of TCP,
because if the other end of a connection changes IP addresses you
cannot assume that any data already acknowledged at the TCP level
was actually received by the application at the other end...and 
you therefore have to keep track of data that is unacknowleged by the 
other end in case you have to resynchronize.  

and you need above-TCP keepalives and redirects.

and you need a separate mechanism to keep track of the current
IP address and port number for each of your connection endpoints...
and which gets updated independently (in case your connection
with a peer drops before it has a chance to tell you where it's
moving to)  this separate mechanism has its own failure modes, 
and parts of that mechanism might also be mobile.

if you had to build mobility entirely at the above-tcp level
this might be a useful approach.  but there are better ways to 
implement mobile networking.

Keith

---

From: "C. Perkins/D. Reese" <[email-masked]>
Date: Tue, 25 Apr 2000 21:10:01 -0700

Hello again,

From the note below, it's obvious that there are some misconceptions
about Mobile IP that I'd like to correct.

> East just to get to the ISP down the street...

In the mobile-ip working group, we had many discussions in the years
before RFC 2002 about this problem.  We also had a solution, called
"route optimization".  The decision was taken to standardize the
"base protocol" and work on standardizing "route optimization"
later.  The reason was exactly because doing this sort of route
optimization needs kernel modifications.

I think that all the details have been worked out pretty well for
Mobile IP route optimization.  The specification has been implemented,
works, and may eventually get through last call if there is sufficient
interest.  The strategy to isolate route optimization from the base
Mobile IP protocol may be questioned with the benefit of hindsight,
but after all there are now millions of implementations of Mobile IP
deployed (according to mail sent to the mobile-ip mailing list).
We don't know if that number would be bigger or smaller if the
working group had included route optimization features with
the base protocol.

I think it would be preferable to put effort behind a known pretty-good
approach that takes modifications to the networking stack, instead
of starting over on a relatively unknown approach that also requires
modifications to the networking stack.  By known pretty-good, I mean
that it has a good story on security, has been studied for scalability,
and does the job.  Many other approaches could be taken with more
engineering effort, but I don't think any of the other possibilties are
anywhere near as close to being finished.  If there is something about
Mobile IP route optimization that should be improved, let's talk
about it; but let's also recognize that a lot of work has been done.

Regards,
Charlie P.

---

From: Karl Auerbach <[email-masked]>
Date: Tue, 25 Apr 2000 21:16:24 -0700 (PDT)

> for instance, you have to build your own data acks on top of TCP,

There is a protocol design that did this, but we tend to shield our eyes
when we look that way - OSI Session layer.  It was a hideous thing indeed
to read on paper and way overburdened with options.  But when one got down
to the wire, the actual protocol traffic was small.  It didn't try to do
any kind of reliability or sequencing or even sensing that a transport had
died.  Rather it maintained a sequence of restart points - and it was only
at those points (which were triggered by the application saying "mark this
point") that there were things that could be called "acks".

So what I am suggesting is that it seems that there is evidence that one
can do an "association" protocol that is relatively lightweight in terms
of machinery, packets, packet headers, and end-node state if one leaves
the heavy lifting of reliability to the underlying TCP protocol.

I bet Marshall Rose has some good comments on this as he actually went and
did some of this.

(By-the-way, I'm not in any way suggesting OSI Session, I'm only trying to
learn from the past.)

		--karl--

---

From: Keith Moore <[email-masked]>
Date: Wed, 26 Apr 2000 10:20:01 -0400

> the heavy lifting of reliability to the underlying TCP protocol.

the on-the-wire protocol overhead is not that great.  the computational
overhead to the host and application, and the resulting loss in maximum
bandwidth, are fairly expensive.

basically it's a lot more efficient to do some variant of mobile IP.

Keith

---

From: Karl Auerbach <[email-masked]>
Date: Wed, 26 Apr 2000 08:33:00 -0700 (PDT)

> bandwidth, are fairly expensive.

I tend to disagree.  An association protocol only really does its work on
connect/reconnect - hopefully a rear event -- and when the application
says "let's establish a work-mark here".

And given that many of our applications are bursty/transactional vehicles
we're not talking about supercomputers transfering bulk files of
simulation data.

> basically it's a lot more efficient to do some variant of mobile IP.

Mobility is not the issue.  Rather, there is use, distinct from mobilty,
for an association that can persever longer than the underlying
transport(s), including changes of IP addresses.  Had we had such a tool
it would have perhaps obviated some of things the mobility folks had to
do.  But having an standardized association protocol is useful in its own
right, distinct from mobility.

In my eyes, an application that uses an association protocol to deal with
changes and faults in underlying connectivity is a lot more in accord with
end-to-end principles than if it relied on ad hoc transport/network
address juggling.

		--karl-

---

From: Paul Francis <[email-masked]>
Date: Wed, 26 Apr 2000 11:23:02 -0700 (PDT)

>  I think your description is somewhat biased, Paul. Suppose that you execute

Sure...I was making a point, not publishing a full analysis.  But
my position did take into consideration everything you mention.

>  other form of renumbering. Suppose then that the external prefix of your

I agree with everything you say here about the difficulties
of NAT as a renumbering-avoidance mechanism.  But I would
point out that these difficulties effect external connectivity
only, not internal connectivity.  (The same external difficulties
also exist even if you do full renumbering, so the fact that
they also exist in the NAT case does not necessarily change my
assertion that NAT will be the path of least resistance for
net managers.)

For most net managers, intra-company connectivity is way more
important than external connectivity.  Renumbering problems
effect intra-company connectivity, whereas NAT problems only
effect external connectivity. 

>  really care that my TCP connection survives when I loose contact with ISP-A,

I also agree with the statements you make about the problems
of multihoming with NAT.  You'll notice that I specifically
said "rudimentary form of multihoming".  I considered going
into what that meant when I wrote it, but didn't want to water
down my main point.   (Good communications sometimes necesitates
telling a biased and partial story.)

>  support of multiple prefixes per site, multiple addresses per interface. 

I agree completely with what you say about needing to push
the multi-address complexity to the host.  As you kindly
pointed out (and I self-servingly expand on here), this is
an architecture I put forth about a decade ago in a sigcomm
paper (in Zurich, I don't remember the year).

But that architecture (hosts having multiple addresses
representing a site's multiple aggregation prefixes and
selecting among them) requires some method of identifying
hosts when they switch from one address to another
mid-connection.  I would assume that what people have in
mind for this are the mobility mechanisms?  (The alternative
is 8+8 or some variant, which I understand to be contentious
enough that it is a defacto non-starter.)

PF

---

From: [email-masked]
Date: Wed, 26 Apr 2000 13:48:00 -0800 (PST)

> work is already going on to do this.)

  draft-xie-stewart-sigtran-ddp-00.txt

				ned

---

From: "Stephen Sprunk" <[email-masked]>
Date: Wed, 26 Apr 2000 16:33:54 -0500

draft-xie-stewart-sigtran-ddp-00 addresses redundancy and failover of
sessions within a server pool, where uncoordinated failover of sessions from
one endpoint to another is a requirement.  There is signifcant overheard and
indirection added to the session to achieve this.

We seem to be discussing a simpler requirement: coordinated movement of a
session from one ip:port pair on a single endpoint to a different ip:port
pair on the same endpoint.  Windows, buffer states, sequence numbers, etc.
could all remain the same.

I would think the latter requirement could be implemented as a simple TCP
"forward me" option.  For ESP/AH-protected sessions, no TCP-level
anti-hijacking protection seems necessary.  This could even be performed if
the original IP is suddenly not available and the other endpoint hasn't
given up on the connection yet; you send a "forward me" packet sourced from
the first IP, then listen for an ACK on the new IP.

I can think of no simple way (ie. without recreating IKE&AH inside TCP) to
do this for unprotected sessions; I'm not sure it's worth the effort to
solve either.

I'm sure there's something I'm missing here, or else this would have been
implemented 15 years ago...  Thoughts?

S

     |          |         Stephen Sprunk, K5SSS, CCIE #3723
    :|:        :|:        Network Consulting Engineer, NSA
   :|||:      :|||:       14875 Landmark Blvd #400; Dallas, TX
.:|||||||:..:|||||||:.    Email: [email-masked]

----- Original Message -----
From: [email-masked]
To: Karl Auerbach
Cc: IETF
Sent: Wednesday, April 26, 2000 16:48
Subject: RE: runumbering (was: Re: IPv6: Past mistakes repeated?)

ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 26 Apr 2000 21:53:17 -0400

> says "let's establish a work-mark here".

no, that's not the bulk of the work.  the bulk of the work is just in
managing the extra copies of data (applications don't want to deal
with this explicitly, they want it handled by lower layers), in
detecting failures, distinguishing one kind of failure from another,
and in recovery.  for instance, to make the handoff efficient you
want to have the ability to say to your peer "please contact me on 
this new address" and have it do so.  but that message may be delayed
and meanwhile some new data might have been sent your way.  so the
host that is moving might also arrange to have data which was
sent to its old location forwarded to the new location for a time -
this saves the burden of getting the peer to retransmit everything.
but the "I'm moving" message might get dropped - this is actually 
likely because hosts don't usually know that they're moving and 
you have to assume that hosts that are mobile will be disconnected 
from time to time.  so you still need a fallback way to find your
peer's connections again - and this generally means some sort 
of resolution service for connection endpoints.   and the resolution
service probably also needs some redundancy.

now realize that it's quite possible for two peers to be moving
on the network at the same time, with their redirect messages
crossing each other and/or being dropped.  etc.  

there are a lot of states that have to be dealt with if you want
the system to be both reliable and efficient.
 
> we're not talking about supercomputers transfering bulk files of

if you're talking about a general purpose facility then it probably 
does need to span a wide range of applications' needs.  perhaps not
supercomputers exchanging large matrices, but those are hardly the
only applications that care about performance and bandwidth.
even lowly web servers care about performance and bandwidth.

> transport(s), including changes of IP addresses.  

if you build a level of indirection to handle this then it had better
handle mobility also.

and we don't need the kind of facility you're talking about just to
do renumbering.  nor would such a facility be sufficient, because
IP addresses are kept in other places besides just TCP connection state.

> end-to-end principles than if it relied on ad hoc transport/network

either approach is end-to-end.  it's just that by putting this functionality
above the transport layer you end up duplicating the functionality that
was already implemented at lower layers, and you have to make the resulting
system fairly complex before you gain much for your trouble.

OTOH, by putting the mobility at the IP layer, you can let TCP take care of
the sequencing etc. and you don't have to implement it again in a higher
layer.

Keith

---

From: [email-masked]
Date: Wed, 26 Apr 2000 15:31:03 -0800 (PST)

> could all remain the same.

The stated requirement was "an association that will outlast the TCP
connections it is based on". I saw (and see) nothing that limited this to
coordinated moves set up in advance.

You're right that it's a simpler problem, but the general problem is what's
on the table IMO.

And if ISO session layer is seen as comparable, we're *definitely* in the more
general problem space. (Although whether or not ISO session layer actually
solved such problems is another matter -- it sorta did on paper if you read the
specifications optimistically, but it never solved any of these problems in
practice. And I have the misfortunate of having had to deal with ISO session
layer stuff a *lot*.)

				Ned

---

From: Randall Stewart <[email-masked]>
Date: Thu, 27 Apr 2000 12:05:20 +0000

Ok,

I will stop being a lurker and chime in since our draft was
mentioned :->

> indirection added to the session to achieve this.
Yes, I think you sum up what DDP attempts to do but I strongly
disagree with your "signifcant overhead" argument. Qiaobing and I
have been playing with DDP now for quite a number of years. It has
some overhead on the wire (not much) since the overhead is mainly
upfront i.e. when you want to go hold a talk to someone you
may need to do a query to find out where it is and where all
its redundant pools are. This drops in to a cache in any sensible
implementation and then within some period will need to expire
or be updated. Even when the cache is updated it does NOT interfere
with the ongoing communication (its done in parrallel). So basically
the delay overhead to talk to a peer is possibly one round trip to
the local ENRP server (of course local is not necessarrily on the
machine you are on but it could be :-0). In some cases there may
be no delay. In particular, the ability to send to a address (pre-known)
and then do a parrallel query to the ENRP deamon. We did not
get this one in the draft but I am sure future updates will have
this ...

The point is I can't see what your "signifcant overhead" is. Yes
it will take a query to get some redundancy but after 5 years of
work I think we have the overhead as slim as we can... of course
perhaps the working group will show us a way to reduce it further
but we will see....

> could all remain the same.

In its basic state this is what DDP does do.. i.e. if you don't have
multiple peers. But of course if the route fails between you and
your peer, the conversations over.. until the path is back up.

> implemented 15 years ago...  Thoughts?

Yes, having worked at solving this problem for 5 years you are missing
some of the subtle nature of where the different state is. You really
must have a seperate state around, unless the "new IP" is attached to
the same machine. In that case you end up with something that looks
like SCTP to provide the redundancy, i.e. use the "new IP" address.

R

-- 
Randall R. Stewart
Member Technical Staff
Network Architecture and Technology (NAT)
847-632-7438 fax:847-632-6733

---

From: Karl Auerbach <[email-masked]>
Date: Sat, 29 Apr 2000 14:38:51 -0700 (PDT)

> and in recovery.

You are describing a super-transport, a TCP on TCP.

I don't want a super-transport.

All I'm trying to do is to create a means by which an aware application
can recognize transport (TCP) connection failures in a way that allows the
communicating parts of that aware application to resynchronize their
activity over a new TCP connection.  That's hardly the heavyweight engine
that you are positing, and there is no need for the protocol to retain
data or do anyting other than allow the communicating applications to
declare to one-another that they have reached a namable point in their
interaction.

Need Freed has pointed me to some interesting drafts that might do the
trick (or might be a good headstart)
  draft-xie-stewart-sigtran-ddp-00.txt

> the sequencing etc. and you don't have to implement it again in a higher

Sure, IP address shell games do frequently work.  So does NAT.

By putting long-distance mobility at the IP layer we force the packet
path mechanisms to engage in a game of patting its head while rubbing its
tummy and reciting the alphabet backwards.

That's sometimes the right thing to do, other times it is not - the choice
is application dependent.

Sometimes it is easier to simply break a TCP connection and later create a
new TCP connection - and that's not just for mobility; it has many other
uses.

		--karl--