Subject: RE: [tcpm] "TCP Multipath" with shim6

From: "Caitlin Bestler" <[email-masked]>
Date: Fri, 31 Mar 2006 14:00:15 -0800

Scott Leibrand wrote:
> I recently had an idea that I've brought up to the shim6
> list, and would also like to bring up here.  The basic idea
> is to extend TCP to use shim6 and its context forking feature
> to allow a single TCP session to simultaneously transmit over
> two paths (defined by two locator pairs) and use the full bandwidth
> of each. 
> 
> Say you have two shim6-capable hosts, and at least one of
> them has two transit connections (say DSL + Cable) with two
> corresponding locators. Now say the two hosts establish a
> connection, start sending traffic, and establish shim6 state.
>  Now say that the multihomed host has lots of data to send,
> and is in congestion avoidance on the default (ULID-ULID) path.
> Now say that he has implemented some TCP extensions that
> allow him to invoke shim6 context forking and simultaneously send
> packets with the shim6 context tag over the second locator pair while
> continuing to send packets over the ULID-ULID pair.  The
> extended TCP stack would independently track the congestion
> windows (and run the TCP congestion avoidance algorithm) for
> each shim6 context (locator pair).
> 
> This would allow the sending host to fully utilize the
> available bandwidth on both his links for a single TCP
> session, which otherwise would be impossible (without bonding
> the two links with some sort of per-packet load balancing,
> which has all kinds of problems).  It would require some
> modifications to the sending host in order to keep a separate
> set of state variables for each context/path, but the
> receiving host need only do ordinary shim6, and TCP SACK to
> selectively ack packets as they came in.
> Since the shim would be in place, the recipient's TCP stack
> would be completely unaware of the fact that his packets are
> coming over two different paths.
> 
> One obvious challenge the modified TCP stack would have to
> deal with is that the use of two paths with differing RTTs
> will all but guarantee lots of duplicate ACKs.  The modified
> TCP sender would need to modify its duplicate ACK processing
> with some algorithms to keep track of which packets were sent
> over which path, and detect which duplicate ACKs actually
> indicate packet drops (which you would know when SACK says
> you've received at least 3 additional segments sent over the same
> path). 
> 
> In addition, the modified TCP receiver has to be careful not
> to confuse an unmodified TCP sender.  The most
> straightforward way to accomplish this is to send all ACKs
> over the first context (the "normal" path), and only send
> data packets over the second context/path.  This ensures that
> the unmodified TCP sender always measures RTT consistently,
> and doesn't get any out-of-order ACKs.
> 
> Another optimization I've thought of is to steal SCTP's idea
> of doing all retransmissions over a different path/context
> from the one they were originally sent on, so as to not be
> held back by (or exacerbate) the congestion that caused the
> retransmission in the first place.
> 
> All of this would of course require close integration of the
> modified TCP sender with the shim layer via the shim6 API, to
> do the context forking, pick a divergent pair of locators, be
> notified of and respond appropriately to any locator rehoming on
> either context, etc. 
> 
> Can anyone think of any problems with this approach?  If not,
> should I start writing up an Internet-Draft on it?  Would
> such an I-D eventually be considered by this WG, or
> elsewhere?  (Dallas was my first IETF, so if anyone has any
> other newbie advice, feel free to speak up.)
> 
> If I'm not missing anything big, and this is indeed workable,
> I think this could be a very popular feature, and could help
> push shim6 adoption onto a wide enough installed base to
> actually make it useful for multihoming as well.
> 
> Thanks,
> Scott
> 

The only valid reason not to use SCTP for this type of problem
is the ubiquitous deployment of low-end middleboxes that do not
know how to NAT SCTP connections (beyond passing anything that
they do not recognize to a single internal host).

Has anyone conducted a survey to see how many of these boxes
properly support IPv6? I would be pleasantly surprised if their
limitations only impacted SCTP and that IPv6 was fully supported.

_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: Scott Leibrand <[email-masked]>
Date: Fri, 31 Mar 2006 17:08:16 -0500 (EST)

> they do not recognize to a single internal host).

Do you know if support for multipath SCTP can be implemented independently
on the sender side without support from the receiver side?  (If you don't
know, I'm sure I'll find out when I read those articles).  If not, and
SCTP multipath requires support on both ends, that might be another reason
to use TCP multipath over shim6 instead of SCTP multipath.

> limitations only impacted SCTP and that IPv6 was fully supported.

I haven't done any surveys, but others have expressed frustration with NAT
boxes that doesn't support IPv6 (or allow "unknown" protocols to allow
transitional strategies).

However, I wonder if SCTP (or shim6, for that matter) will be supported
when NAT vendors start to roll out "enhanced NAT" boxes that NAT IPv4 and
also provide (non-NAT'd) IPv6 support.

-Scott