Subject: [multipathtcp] Options or Payload?

From: "Ford, Alan" <[email-masked]>
Date: Tue, 10 Nov 2009 00:58:48 -0000

Hi all,

One of the big issues to be raised during yesterday's MPTCP session was
the question of whether TCP Options are really the right place to be
doing this. This is not the first time this has come up but deserves
further exploration.

Specifically, instead of doing this with TCP Options, the same
instructions could be included in the payload. Similar to TLS, the data
could be chunked and each chunk has a data sequence and length header.
These can be interspersed with control blocks to signal addresses,
security of joining subflows to connections, and connection close. A
simple 2-octet TCP option would still be used in the initial SYN to
signal MPTCP capability.

This has the benefit that it would allow the signalling to have
reliability, and we wouldn't be hit with option space limits, and thus
be potentially able to do better security algorithms. It would also give
us greater freedom in signals for future extensibility (for example, if
we wanted to signal ports for additional subflows, not just addresses).

On the downside, there may be cases where this could confuse
middleboxes, e.g. expecting HTTP on port 80 and finding this kind of
data instead. However, since a TCP option would be used at the start to
identify capability, if this were dropped by a middlebox/proxy then
MPTCP would not be used.

What do people think is the best approach?

Regards,
Alan

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 10 Nov 2009 10:35:22 +0900 (JST)

Hello,
How about having a simple option which indicates the offset for real tcp payload?
For example, if mptcp packets conveys 10 bytes control info in the payload, set 
offset to 10 in the option.

Thanks,
--
Yoshifumi Nishida
[email-masked]

From: "Ford, Alan" <[email-masked]>
Subject: [multipathtcp] Options or Payload?
Date: Tue, 10 Nov 2009 00:58:48 -0000
Message-ID: <[email-masked]>

 > _______________________________________________
 > multipathtcp mailing list
 > [email-masked]
 > https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: "Ford, Alan" <[email-masked]>
Date: Tue, 10 Nov 2009 01:52:44 -0000

We had considered that, however the main reason against this was whether
certain middleboxes would split and coalesce TCP payloads thus breaking
the placement of the extended options - the TCP payload is essentially
no longer a continuous stream.

Wes Eddy proposed a "Long Options Option"
(http://tools.ietf.org/id/draft-eddy-tcp-loo-04.txt) which seemed to be
a possible way of doing a very similar thing, by negotiating to ensure
that long options would work along a path. Although, this draft is now
expired, and I'm not sure why it was dropped. Anyone know?

Regards,
Alan

> -----Original Message-----
> From: Yoshifumi Nishida [mailto:[email-masked]]
> Sent: 10 November 2009 01:35
> To: Ford, Alan
> Cc: [email-masked]
> Subject: Re: [multipathtcp] Options or Payload?
> 
> 
> Hello,
> How about having a simple option which indicates the offset for real
tcp
> payload?
> For example, if mptcp packets conveys 10 bytes control info in the
payload,
> set
> Message-ID:
> <[email-masked]>
> 
>  > Hi all,
>  >

---

From: Joe Touch <[email-masked]>
Date: Mon, 09 Nov 2009 17:54:33 -0800

This has been tried in various places in TCP to extend the options
space. It'd be useful to take a look at that work as context.

Joe

---

From: William Herrin <[email-masked]>
Date: Mon, 9 Nov 2009 21:38:10 -0500

On Mon, Nov 9, 2009 at 7:58 PM, Ford, Alan <[email-masked]> wrote:
> doing this.
> instructions could be included in the payload.

Hi Alan,

There isn't enough room in the options to transfer an encryption key,
a hostname or other likely methods of authenticating the offered
addresses. There isn't really enough to transfer all of the candidate
addresses either, not once we start talking v4/v6 agnosticism, so we'd
have to send adds and deletes instead of updates.

When I tinkered with a throwaway concept
(http://bill.herrin.us/network/multipathtcp2.html) I pretty quickly
moved to the data section because trying to keep it all in the options
got funky fast.

Regards,
Bill Herrin

-- 
William D. Herrin ................ [email-masked]  [email-masked]
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004

---

From: "SCHARF, Michael" <[email-masked]>
Date: Tue, 10 Nov 2009 03:48:53 +0100

Hi all,

> Specifically, instead of doing this with TCP Options, the 
> same instructions could be included in the payload. Similar 
> to TLS, the data could be chunked and each chunk has a data 
> sequence and length header.
> These can be interspersed with control blocks to signal 
> addresses, security of joining subflows to connections, and 
> connection close. A simple 2-octet TCP option would still be 
> used in the initial SYN to signal MPTCP capability.
> 
> This has the benefit that it would allow the signalling to 
> have reliability, and we wouldn't be hit with option space 
> limits, and thus be potentially able to do better security 
> algorithms. It would also give us greater freedom in signals 
> for future extensibility (for example, if we wanted to signal 
> ports for additional subflows, not just addresses).

Maybe there is a further advantage: I could imagine that payload chunks
could simplify the MPTCP stack design. Sporadically adding TCP options
other than SACK could result a mess in a real stack (correct MSS
calculation, dealing with segmentation offloading, etc.). At least I
recall that we had such problems in our Quick-Start TCP implementation
in Linux.

Michael

---

From: Joe Touch <[email-masked]>
Date: Mon, 09 Nov 2009 19:04:08 -0800

> expired, and I'm not sure why it was dropped. Anyone know?

Same problems.

Joe

---

From: Costin Raiciu <[email-masked]>
Date: Wed, 11 Nov 2009 11:21:03 +0900

Hi

I tried to assemble a pros and cons list for the two approaches.  
Please add bullets to both lists until we get something reasonably  
exhaustive, and then we can make an informed decision.

At a high level, we need to use options on every subflow at least in  
the syn exchange. Otherwise we can't attach the subflow to the  
existing connection, or can't negotiate mptcp in the first place.
Ideally you'd like fate sharing: if the syn/ack exchange passes, the  
subflow should work.

Options
+ nice with tcp
+ single subflow mptcp if friendly with IDSes, and traffic  
normalizers (i.e. data flow is what the app receives).
- reliability is more difficult (how often do you retransmit?) drop  
packets? what's the failure mode when options don't get through?
- boxes may crash when parsing unknown stuff. The hope is that on SYN  
processing is more robust with unknown options. ECN story is  
frightening :). Murari (Microsoft)
says that even changing position of padding bits in options crashes  
some middleboxes; I've heard a similar story from the Linux kernel  
experience. However, we can't (cleanly) avoid sending options on SYNs.

Payload
+ option space limits no longer matter; can do funkier stuff later,  
like negotiate keys, etc.
+ easier to get reliability; retransmit of control data linked to  
data retransmit
+ easier to deal with middleboxes that resegment (it just works!)
+ new options on syns may be allowed, but not on data (is this true)?
- feels like a hack, not the traditional way to evolve tcp
- implies encoding of any data sent MUST  be ASCII; middleboxes (ids,  
traffic normalizers) that look at content and see binary stuff will  
probably drop the data; finally, if we used binary stuff, there is no  
guarantee for fate sharing. . Text encoding is less space efficient;  
does it matter?

Any more?
Costin

---

From: "SCHARF, Michael" <[email-masked]>
Date: Thu, 12 Nov 2009 02:57:53 +0100

> + easier to deal with middleboxes that resegment (it just works!) new 
> + options on syns may be allowed, but not on data (is this true)?
> - implies encoding of any data sent MUST  be ASCII; 
> middleboxes (ids, traffic normalizers) that look at content 
> and see binary stuff will probably drop the data; finally, if 
> we used binary stuff, there is no guarantee for fate sharing. 
> . Text encoding is less space efficient; does it matter?

I don't understand why ASCII encoding is mandatory. An IDS that parses
the content may anyway be confused by the incomplete content inside one
MPTCP subflow, independent of the question whether options or payload
encoding is used. Or do I miss something?

Michael

---

From: Costin Raiciu <[email-masked]>
Date: Thu, 12 Nov 2009 15:51:46 +0900

Hi,

I am thinking of middleboxes that expect "text-only" protocols and we  
would introduce binary data in the payload. That's not going to go  
down well, but in the end it's just another failure mode.

I guess whichever way we go, we really need to think carefully about  
failure modes.

Let me add one more bullet to the list, after my discussion with  
Murari today.
Payload
- involves mixing the control and data plane in host stacks; will be  
much more difficult to implement