Subject: [multipathtcp] [T3-Rsrc Control] Multipath TCP use cases

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 25 Jun 2009 22:44:42 +0200

Hi all,

So far we've more or less assumed bulk data transfer as the use case  
for multipath TCP. I think multipath TCP and/or multipath transport in  
general and/or some of the underlying mechanisms (testing paths to  
find the best one(s)) are more broadly applicable.

There are basically six different types of communication:

1. Single packet request/response transactions (like the DNS)
2. Multi-packet, but limited size transactions (like typical HTTP  
transfers)
3. Bulk transfers (FTP or HTTP transfer of large files)
4. Low-bandwidth interactive traffic (such as Telnet)
5. Low-bandwidth streaming (VoIP)
6. High-bandwidth streaming (video)

With the exception of 1, all can benefit from using multiple paths. In  
all cases except 6, the most optimal strategy is to distribute packets  
over the paths such that the last bit arrives within the least amount  
of time.

In cases 4 and 5, that means using the path with the shortest round  
trip times exclusively. In case 3, the round trip times of individual  
packets are so short compared to the total length of the transfer that  
choosing the shortest RTT path for an individual packet provides  
virtually no benefit. In that case, it's important to either use all  
available paths fully, or order the paths in order of decreasing  
bandwidth and use them in that order as aggressively as permitted  
within the bounds of fairness.

In case 2, if the size of the transaction is known up front and the  
properties of the paths are also known up front, ore can be learned  
quickly enough, it's possible to distribute the packets over the paths  
such that the last bit arrives in the minimum amount of time possible.

In case 6 there is usually a fairly fixed amount of bandwidth that is  
required. If that bandwidth is available on one of the paths, the  
obviously the transfer can simply use that path. If no single path is  
capable of providing the required bandwidth, then congestion control  
can be executed for multiple paths in order to determine how much  
bandwidth may be used on each path. Packets are then distributed over  
each path within the limits of the congestion window for that path.  
This way, a stream's bandwidth is limited by the baggregate bandwidth  
of the available paths, not by that of a single path.

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 26 Jun 2009 08:13:17 +0100

Iljitsch van Beijnum wrote:
> Hi all,
> 
> So far we've more or less assumed bulk data transfer as the use case for 
> multipath TCP. I think multipath TCP and/or multipath transport in 
> general and/or some of the underlying mechanisms (testing paths to find 
> the best one(s)) are more broadly applicable.
> 
We could also consider bulk transfer that is unattended, like 3, but 
with even more relaxed time constraints. Would that change things?

> over the paths such that the last bit arrives within the least amount of 
> time.
> 
> In cases 4 and 5, that means using the path with the shortest round trip 
> times exclusively. In case 3, the round trip times of individual packets 
> are so short compared to the total length of the transfer that choosing 
> the shortest RTT path for an individual packet provides virtually no 
> benefit. In that case, it's important to either use all available paths 
> fully, or order the paths in order of decreasing bandwidth and use them 
> in that order as aggressively as permitted within the bounds of fairness.
> 
Do you think this is likely to be a common case?

> capable of providing the required bandwidth, then congestion control can 
> be executed for multiple paths in order to determine how much bandwidth 
> may be used on each path. Packets are then distributed over each path 
> within the limits of the congestion window for that path. This way, a 
> stream's bandwidth is limited by the baggregate bandwidth of the 
> available paths, not by that of a single path.
> 

In case 6 - I think this is true for the streaming of video in real-time 
(as in while with video-conference video), rather than receiver-buffered 
video? In video-streaming applications with 10-20 seconds of buffer, or 
so, there is quite some slack-time.

> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp
> 

Gorry

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 26 Jun 2009 10:26:09 +0200

> with even more relaxed time constraints. Would that change things?

Don't think so... How makes the presence of the user any difference to  
the multipath?

Of course it's possible to argue that unattended communication can be  
less agressive about the amount of bandwidth that it claims, but that  
would be true regardless of the use of multiple paths, this is what  
LEDBAT is doing.

>> quickly enough, it's possible to distribute the packets over the  
>> paths such that the last bit arrives in the minimum amount of time  
>> possible.

> Do you think this is likely to be a common case?

Well, this would be your typical HTTP transfer for a web page or an  
image. The question is whether between the minimum number of packets  
that we need to test at least a couple of paths and the requirement  
that the length of the transmission be known early = the transmission  
is small enough that it fits within the send buffer there is a  
reasonable number of transactions that qualify.

> In case 6 - I think this is true for the streaming of video in real- 
> time (as in while with video-conference video), rather than receiver- 
> buffered video? In video-streaming applications with 10-20 seconds  
> of buffer, or so, there is quite some slack-time.

If you're watching a 1.5 Mbps stream but you have two 1 Mbps  
connections to the internet the buffer isn't going to help you  
much.  :-)

Iljitsch

---

From: Michael Scharf <[email-masked]>
Date: Fri, 26 Jun 2009 18:10:26 +0200

> in that order as aggressively as permitted within the bounds of fairness.

Why is it not optimal to order the paths always in order of increasing
RTTs, i. e., the sender first uses the path with the smallest RTT
only, and it sucessively starts to distribute the data over (all)
other paths once it becomes clear that the bandwidth of the first one
is not "sufficient"?  Such a heuristic might not require any
distinction between application types 3,4,5. Do I miss something?

> such that the last bit arrives in the minimum amount of time possible.

What precisely is the "size of the transaction"? A network stack can
easily measure the amount of application data queued in the socket. Is
there any benefit of determining the total amount of data that an
application plans to send?

Michael

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 26 Jun 2009 21:07:27 +0200

> distinction between application types 3,4,5. Do I miss something?

Only in cases 5 and 6 do you (hopefully) know how much bandwidth is  
enough. In case 3 it's never enough, you always want to take more.

>> paths

> application plans to send?

Well, if I have a channel that can take 20 1000-byte packets per  
second with a 100 ms RTT and another channel that can take 100 1000- 
byte packets per second with a 500 ms RTT, then it takes 100 ms for  
the first packet to be ACKed on the low BT/RTT path and then another  
1000 / 20 = 50 ms per packet for the other 4, so a total of 300 ms. On  
the high BW/RTT path the initial ACK is 500 ms and then 1000 / 100 =  
10 ms for the other 4 = 540 total. So in this case I would want to  
send all packets over the low BW/RTT path.

Now if there's another 10 packets following those initial 5, this  
becomes 800 vs 640 ms so it would have been better to send all packets  
over the high BT/RTT path. (Although it's even better if I can send 9  
packets over the low path and 6 over the high path, for a completion  
time of 500 on the low path and 550 on the high path (or 10 / 5 =  
550 / 540).)

After a certain threshold I should always send 1/6th of my packets  
over the low path and 5/6th over the high path, so I don't really need  
to know how large the transfer is exactly, just that it's big enough  
that using the low path exclusively isn't the fastest option.

And of course the fact that we don't know the path properties until we  
start sending packets severely complicates all of this.

---

From: Michael Scharf <[email-masked]>
Date: Fri, 26 Jun 2009 22:56:11 +0200

> enough. In case 3 it's never enough, you always want to take more.

Sure. My point is that one could maybe handle both case 3 _and_ cases
4+5 with the same straightforward strategy. And then there would be no
need to distinguish between these application types.

You can detect quite fast whether the transfer over the path with the
smallest RTT is application-limited or not (at least in static
environments).  In case 5 and 6 this is likely to be the case, i. e.,
a cwnd grows beyond the amount of available app data. Then, this path
is just fine. In case 3, application limitation will not occur.  Then,
other paths beyond the one with the smallest RTT should be used as
well. It should be possible to detect application limitation on the
path with the smallest RTT within few seconds.

Thus, if you always use the path with the smallest RTT first, and if
you start to use additional paths once you detect that the path with
the smallest RTT is not application-limited, you can handle _both_
case 3 and 4+5 with the same simple heuristic, without requiring any
a-priori knowledge about the application type. But of course I may be
wrong.

Michael

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Sun, 28 Jun 2009 16:10:45 +0200

> need to distinguish between these application types.

The difference is that VoIP/Telnet needs a fast turnaround for  
individual packets but bandwidth is almost never an issue, which is  
the opposite for bulk transfers.

However, if we think of VoIP packets and Telnet keystrokes/responses  
as minitransactions and bulk transfers as really large transactions,  
we only need to solve the general case of transactions (both small and  
large) which would be something along the lines of: send on the lowest  
RTT path exclusively for RTT2 - RTT1 milliseconds if the data on hand  
fits within that timeframe, otherwise/after that distribute packets  
over paths 1 and 2 relative to the path bandwidth. (See my earlier  
message.) Since 1-packet "transactions" like in VoIP take zero time,  
you never end up in the situation

> well.

That may be a good starting point, but even if the app doesn't send  
unlimited data there can be value in using multiple paths as this will  
get the last bit to the other side sooner than just using the lowest  
RTT path.

But certainly in the case where the app doesn't fill up an entire  
packet we should just use the lowest RTT path. If the path isn't fast  
enough then the Nagle algorithm will kick in and the data will no  
longer be smaller than a packet size after some time.

---

From: Aleksi Suhonen <[email-masked]>
Date: Mon, 29 Jun 2009 13:13:53 +0300

> 6. High-bandwidth streaming (video)

I would like to add for your consideration two more categories:

* unattended bulk transfers with liveness and timeliness constraints

	such as BGP or IRC intra-server-connections, which last
	"forever", don't consume much bandwidth except during bursts
	but which can't be interrupted or they die and cause havoc

	multi-pathing could potentially be useful in preserving the
	connections during outages and attacks

* unattended bulk transfers without constraints

	such as usenet news and MTA intra-server-connections

	these two example protocols can benefit from multiple
	concurrent single path connections along different paths
	tho, so maybe this isn't as good an example...

Anyway, the common denominator for these examples is that they are 
connections that never finish, or seldom finish, and I couldn't justify 
combining them with any of the existing 6 categories to myself.

One notable feature of these is that they are very likely to wrap around 
TCP sequence space, and some of the discussion surrounding some of the 
MP/TCP proposals has been centered on TCP sequence space tricks to 
appease "connection normalizers".

-- 
	Aleksi Suhonen