Subject: Re: [multipathtcp] Time for Long Options?

From: Andreas Ripke <[email-masked]>
Date: Mon, 2 Aug 2010 09:14:25 +0000

Not really piggybacking as in fast handshake but more
like extended option space. How do unaware stacks 
handle payload with initial SYN? If the receiver is
aware it should know whether the sender knows and
shouldn't send back SYN/ACK with payload if not.

Andreas

> Did you try piggybacking data on both SYN segments in the handshake? I'd
> expect almost all implementations to drop data arriving with the active side's
> SYN as a result of SYN cookies, reduced SYN state (FreeBSD syncache, Linux
> minisocks, etc). However, data hung on the passive side's SYN/ACK might be
> handled differently, and it would be interesting to know how things
> look there as well.
> 
> Robert

---

From: Andreas Ripke <[email-masked]>
Date: Mon, 2 Aug 2010 09:14:18 +0000

> FWIW, the MacOSX is implementing TCP properly. The data in the SYN is
> part of the data path (unless, presumably, the option declares
> otherwise). Options not defined are required to be silently ignored.

The plan was to figure out how to handle a receiver that does not
know about extended option space. If the initial SYN with extended
options space arrives and the receiver is unaware it treats the extra
as payload. Falling back to regular TCP is then only possible with RST
followed by a new connection request (to avoid extended options to be
delivered to the application - as only the SYN is ack'ed in the first
place and not the payload). But then, the other challenge is how to
detect if middleboxes altered and rectified the packets.

Andreas

> I suspect the other TCPs are choosing to drop data in the SYN regardless
> of what options are set - i.e., they do this to avoid queueing data for
> connections that might never happen. While this is permitted (i.e., TCP
> can always drop data so long as it doesn't ACK it), such behavior is NOT
> required.
> 
> Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 03 Aug 2010 01:44:59 -0700

> detect if middleboxes altered and rectified the packets.

RSTs are not reliable, so now you have to deal with the timeouts and 
retransmissions that result.

As I noted, this problem was explored in numerous contexts in TCPM, and 
the conclusion was that it was not feasible to extend options in the 
SYN. That being the critical case, we didn't explore extending options 
in just the payloads after the connection was established. That may be 
useful to explore here as a separate issue, but SYN extension does not 
seem possible.

Joe

---

From: <[email-masked]>
Date: Tue, 3 Aug 2010 10:50:48 +0100

Joe,

> in just the payloads after the connection was established. 

didn't
http://tools.ietf.org/html/draft-eddy-tcp-loo
do just that?

(yes, should probably be -tcpm- rather than -tcp- in the filename.)

L.

Lloyd Wood
http://sat-net.com/L.Wood/
________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Joe Touch [[email-masked]]
Sent: 03 August 2010 09:44
To: Andreas Ripke
Cc: multipathtcp TCP
Subject: Re: [multipathtcp] Time for Long Options?

Joe
_______________________________________________
multipathtcp mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 3 Aug 2010 16:25:16 +0200

On 29 jul 2010, at 20:30, Alexander Zimmermann wrote:

> Costin you really demonstrate that we need long options

Does the MPTCP really need 20 bytes??

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 3 Aug 2010 16:32:33 +0200

On 30 jul 2010, at 9:43, Joe Touch wrote:

> My recollection is that LO are mostly needed in the SYN, and there is no viable way to extend the space in the SYN.

Additional TCP option could be placed in an IPv4 option or an IPv6 destination option header. In the IPv6 case, despite the large number of additional TLV layers this would allow for a great deal of additional option space.

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 3 Aug 2010 15:34:57 +0100

And to further that question, even if it does (ie. With good security
for JOIN etc), will all that info need to be passed on in a single
segment, or could it be piggy-backed over a number of segments (with
different sequence numbers, to have them delivered reliable).

Richard Scheffenegger

> -----Original Message-----
> From: Iljitsch van Beijnum [mailto:[email-masked]] 
> Sent: Dienstag, 3. August 2010 16:25
> To: Alexander Zimmermann
> Cc: multipathtcp TCP

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 03 Aug 2010 16:47:51 +0200

Iljitsch ,

> Additional TCP option could be placed in an IPv4 option or an IPv6 destination option header. In the IPv6 case, despite the large number of additional TLV layers this would allow for a great deal o…

Putting a tcp option inside an IPv4 option is not a good idea as routers
usually process packets with options in the slow path, if they process
it. IPv6 might be possible, but this kind of cross-layer hack looks
pretty ugly to me

Olivier

-- 
http://inl.info.ucl.ac.be , UCLouvain, Belgium

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 3 Aug 2010 17:01:20 +0200

On 3 aug 2010, at 16:47, Olivier Bonaventure wrote:

> it.

What I've been hearing is that this is no longer the case, and most routers have no trouble with IP options. I'm sure being able to truthfully claim that a box can't be taken down by a relatively minor amount of traffic containing options is good for business so vendors added ignoring options to their silicon implementations.

If routers forward them on the slow path that's their problem, we can't really work around a condition that has creates no visible problems. If they drop these packets we can fall back to the normal option space. (Although if that would give us good results, why did we need the big options in the first place?)

> IPv6 might be possible, but this kind of cross-layer hack looks
> pretty ugly to me

I agree that optionless TCP is cleaner.  :-)

---

From: Arnd Hannemann <[email-masked]>
Date: Tue, 03 Aug 2010 17:10:05 +0200

Hi,

> http://tools.ietf.org/html/draft-eddy-tcp-loo

The draft specifies both kinds of extended options.
TCP Long Options (LO) and TCP SYN Long Options (SLO).

Best Regards,
Arnd

---

From: Joe Touch <[email-masked]>
Date: Wed, 04 Aug 2010 12:31:00 -0700

> http://tools.ietf.org/html/draft-eddy-tcp-loo

Sec 3 did, but had numerous issues (required option ordering - which 
isn't really feasible -- options do not have required ordering, and 
there's no 'registry' to indicate who comes first if multiple claim to). 
It also didn't define how it interacted with other options (e.g., MD5 or 
extended checksums).

Sec 4 attempted to extend the SYN for subsequent connections to the same 
host, but that's not feasible since info on one connection might not 
apply to another (e.g., when connections are distributed inside a 
back-end system).

Joe

---

From: <[email-masked]>
Date: Thu, 5 Aug 2010 08:13:18 +0100

On 4 Aug 2010, at 20:31, Joe Touch wrote:
On 8/3/2010 2:50 AM, [email-masked] wrote:
> Sec 3 did

right.

And doesn't TCP-CT contains a similar mechanism? That is on its way through
the RFC-Editor...
http://tools.ietf.org/html/draft-simpson-tcpct-03

> , but had numerous issues (required option ordering - which 

Lloyd Wood
[email-masked]
http://sat-net.com/L.Wood

---

From: Joe Touch <[email-masked]>
Date: Thu, 05 Aug 2010 07:56:18 -0700

> http://tools.ietf.org/html/draft-simpson-tcpct-03

As an individual submission; the authors chose not to bring their work 
to the IETF. It has numerous problems.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 05 Aug 2010 11:07:51 -0700

A key issue:

In TCP (and MPTCP), the data is delivered reliably and in order.

Options are not. There is no option sequence number. Options with state 
would require their own, independent mechanism for reordering or 
retransmission.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 05 Aug 2010 11:10:32 -0700

On 7/31/2010 3:44 AM, Robert Watson wrote:
>
> On Fri, 30 Jul 2010, Andreas Ripke wrote:
>
>> SYN+payload: we tried this the other day. But as it was already
>> assumed/stated on the TCPM discussion list: Most OSes that are not
>> aware silently drop the payload and carry on (tried with the usual
>> suspects like Linux, OpenSolaris, WindowsXP, and {Free,Open,Net}BSD.
>> Only MacOSX delivers the payload to the application. Doh! The few NAT
>> boxes we tested didn't remove SYN payload when mangling the packet.
>> But there are some middleboxes in the wild rectifying this sin. It is
>> not easy to evaluate as control of both endpoints is needed. Sigh.
>
> expect almost all implementations to drop data arriving with the active
> side's SYN as a result of SYN cookies, reduced SYN state (FreeBSD
> syncache, Linux minisocks, etc). However, data hung on the passive
> side's SYN/ACK might be handled differently, and it would be interesting
> to know how things look there as well.

TCP allows data in the SYN to be for the connection. Although many 
implementations drop that data, it's also entirely legal to retain it 
and ACK it in the SYN-ACK, at which point it would be passed to the 
application in legacy implementations. I.e., once the initiator ACKs the 
received SYN (third part of the TWHS), that data would be handed to the 
receiving side application.

I.e., you would need to drop connections to legacy systems that don't 
ACK the extended SYN option to avoid contaminating the data path.

Joe

---

From: Andreas Ripke <[email-masked]>
Date: Fri, 6 Aug 2010 09:16:07 +0000

Yes, RFC793 states that data in SYN is legal and
must be delivered to the application after
connection establishment (if not dropped). Hence,
putting payload as long options in SYN is doomed
to failure as you cannot figure out what the
recipient is doing. (E.g. MacOS acks payload in SYN
after the TWHS.) So it seems that the only sound
way is to only announce long options in SYN and
then respond accordingly to the SYN-ACK reply.

Andreas

---

From: Joe Touch <[email-masked]>
Date: Fri, 06 Aug 2010 10:10:14 -0700

On 8/6/2010 2:16 AM, Andreas Ripke wrote:
> then respond accordingly to the SYN-ACK reply.\

This was the conclusion TCPM came to. However, in TCPM's case, we 
considered the extension of the SYN the most critical part, and decided 
that non-SYN extensions weren't particularly useful by themselves.

Here they might be, though the issue of transparent 'rewriters' in the 
path leads to the extension space effectively being a shim layer 
encoding (ala TLS, as already proposed here), which then inhibits 
'zero-copy' off-loading optimizations.

The other alternative, as is already proposed in a non-IETF draft 
(TCPCT) is to compress existing options to result in more space (in the 
post-SYN segments); I don't think that helps much here, though.

Joe

---

From: Arnd Hannemann <[email-masked]>
Date: Thu, 12 Aug 2010 11:24:52 +0200

Am 29.07.2010 19:09, schrieb Costin Raiciu:
> Hi,
> 
> There are two separate issues: options on SYN or options during the data flow.
> 
> MPTCP will only use 6B of SYN options. There is still plenty of room left for further extensions.

How only 6B?
MP_CAP has 12B and MP_JOIN 7B, did I miss something?

> 
> For data flow, MPTCP will use 16/20B for sequence mapping, and 10B for ACKs. As people have pointed out, these will leave little space for other signalling.
> 
> I agree that an architecturally clean solution would be to do long options - and only use it when it actually does go through the network.
> 
> Another solution that is not as elegant but could work now is:
> 1. Mandate use of Appropriate Byte Couting when using Multipath TCP
> 2. Turn off delayed acks (or just do ack splitting) to generate more acks than we currently do. Use the option space in those ACKs to convery additional information.
> 
> This will be a lot easier to deploy, but is not as clean.
> 
> Cons?

This would be, well, ugly ;-)

Best regards,
Arnd

---

From: Costin Raiciu <[email-masked]>
Date: Fri, 13 Aug 2010 23:44:18 +0300

Hi,

> MP_CAP has 12B and MP_JOIN 7B, did I miss something?

We found a clean way to remove the initial 6B DSN from MP_CAP (computed as hash of token), so MP_CAP will only have 6B. 

> This would be, well, ugly ;-

Yes. :)