Subject: [tcpm] "TCP Multipath" with shim6

From: Scott Leibrand <[email-masked]>
Date: Fri, 31 Mar 2006 16:40:55 -0500 (EST)

I recently had an idea that I've brought up to the shim6 list, and would
also like to bring up here.  The basic idea is to extend TCP to use shim6
and its context forking feature to allow a single TCP session to
simultaneously transmit over two paths (defined by two locator pairs) and
use the full bandwidth of each.

Say you have two shim6-capable hosts, and at least one of them has two
transit connections (say DSL + Cable) with two corresponding locators. Now
say the two hosts establish a connection, start sending traffic, and
establish shim6 state.  Now say that the multihomed host has lots of data
to send, and is in congestion avoidance on the default (ULID-ULID) path.
Now say that he has implemented some TCP extensions that allow him to
invoke shim6 context forking and simultaneously send packets with the
shim6 context tag over the second locator pair while continuing to send
packets over the ULID-ULID pair.  The extended TCP stack would
independently track the congestion windows (and run the TCP congestion
avoidance algorithm) for each shim6 context (locator pair).

This would allow the sending host to fully utilize the available bandwidth
on both his links for a single TCP session, which otherwise would be
impossible (without bonding the two links with some sort of per-packet
load balancing, which has all kinds of problems).  It would require some
modifications to the sending host in order to keep a separate set of state
variables for each context/path, but the receiving host need only do
ordinary shim6, and TCP SACK to selectively ack packets as they came in.
Since the shim would be in place, the recipient's TCP stack would be
completely unaware of the fact that his packets are coming over two
different paths.

One obvious challenge the modified TCP stack would have to deal with is
that the use of two paths with differing RTTs will all but guarantee lots
of duplicate ACKs.  The modified TCP sender would need to modify its
duplicate ACK processing with some algorithms to keep track of which
packets were sent over which path, and detect which duplicate ACKs
actually indicate packet drops (which you would know when SACK says you've
received at least 3 additional segments sent over the same path).

In addition, the modified TCP receiver has to be careful not to confuse an
unmodified TCP sender.  The most straightforward way to accomplish this is
to send all ACKs over the first context (the "normal" path), and only send
data packets over the second context/path.  This ensures that the
unmodified TCP sender always measures RTT consistently, and doesn't get
any out-of-order ACKs.

Another optimization I've thought of is to steal SCTP's idea of doing all
retransmissions over a different path/context from the one they were
originally sent on, so as to not be held back by (or exacerbate)
the congestion that caused the retransmission in the first place.

All of this would of course require close integration of the modified TCP
sender with the shim layer via the shim6 API, to do the context forking,
pick a divergent pair of locators, be notified of and respond
appropriately to any locator rehoming on either context, etc.

Can anyone think of any problems with this approach?  If not, should I
start writing up an Internet-Draft on it?  Would such an I-D eventually be
considered by this WG, or elsewhere?  (Dallas was my first IETF, so if
anyone has any other newbie advice, feel free to speak up.)

If I'm not missing anything big, and this is indeed workable, I think this
could be a very popular feature, and could help push shim6 adoption onto a
wide enough installed base to actually make it useful for multihoming as
well.

Thanks,
Scott

_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: Scott Leibrand <[email-masked]>
Date: Fri, 31 Mar 2006 17:01:28 -0500 (EST)

One other thing I didn't mention: this idea looks to be very similar to
a similar feature being developed for SCTP by the University of Delaware's
Protocol Engineering laboratory:

http://www.eecis.udel.edu/~amer/PEL/poc/index.html#sctp

I came up with the TCP multipath idea (and name) independently, and
another poster on the shim6 mailing list pointed me to the SCTP work,
which looks very similar.  I haven't read the journal articles yet (just
the webpage), but I'm sure they address the same issues I enumerated
below, and others I haven't thought of yet, so it looks like I'll be doing
some reading this weekend.  :)

-Scott

---

From: Joe Touch <[email-masked]>
Date: Fri, 31 Mar 2006 13:59:16 -0800

> use the full bandwidth of each.

It sounds more like you want SCTP than TCP; see below...

> completely unaware of the fact that his packets are coming over two

Except for reordering at the receiver, which would be exacerbated.

> received at least 3 additional segments sent over the same path).

Different RTTs, different congestion windows, etc. I.e., it seems even
more complicated than you're hinting at.

> unmodified TCP sender always measures RTT consistently, and doesn't get

ACKs can come out-of-order on a single TCP connection. And if the
forward channel benefits from multipath, so would the reverse. More
specifically, if the backward channel is simplified by using a single
path, so is the forward - and you're back to existing TCP! ;-)

> the congestion that caused the retransmission in the first place.

Why not just use SCTP at that point?

> Can anyone think of any problems with this approach?

You seem to have listed more than enough above. ;-)

> If not, should I

Hacking TCP to provide something SCTP - or even BXXP over separate TCP
connections - would provide seems like a poor way to justify the need
for Shim6.

Joe

---

From: Tim Shepard <[email-masked]>
Date: Fri, 31 Mar 2006 18:23:20 -0500

For the past year or two, I've heard "SCTP" proffered by many people
as the answer to a variety of questions.

I've given the SCTP specification a very quick scan last week while in
Dallas, and I've learned bits and pieces about it over the years.
SCTP does indeed seem to be cool, and a better framework into which to
put all sorts of desired functionality.

So if SCTP is the answer (to multihoming, multipath, general locator
agility, etc.), and the applications I want to use between my
multi-homed laptop and the various servers that I use are things like
ssh, imap, and xmpp, what steps do I need to take to start
experiencing all of the wonderfulness that is SCTP?

Are the various port numbers assigned by IANA for TCP use that sshd,
imapd, and jabberd are already listening on implicitly assigned for
SCTP use as well?  Is the right thing to do to put something in the
stack such that when a (perhaps legacy) daemon that opens a TCP
listener on some port number to also have the stack listen on the same
numbered SCTP port in case some client wants to use SCTP instead?

Is there a document anywhere describing how the TCP to SCTP transition
is supposed to be accomplished?  Are we going to be running dual-stack
for a number of years or decades while the transition is underway?

I too have been thinking about adding locator agility awareness to
TCP, and I've mentioned it to a few people over the past year.  A few
people have told me I should perhaps contribute my ideas to SCTP
implementations instead of TCP implementations.  But since most or
perhaps all of the applications I care about making work better
when there is locator agility stuff happening run on TCP, it seems
that improving SCTP's behavior wouldn't help my stuff run better.

What should I (or we) do?

Should tcpm's charter be expanded to include similar tweaking of SCTP?
Or should we have an sctpm WG?

In the int-area meeting in Dallas, it was pointed out that some folks
are designing protocols for IPv4 only, with no IPv6 story.   It was
suggested that drafts should have an "IPv6 considerations" section.

Should the various drafts in tcpm WG for tweaks to TCP's behaviour
have an "SCTP considerations" section?

			-Tim Shepard
			 [email-masked]