Subject: Re: [multipathtcp] MPTCP connection termination

From: Christoph Paasch <[email-masked]>
Date: Thu, 01 Dec 2011 11:05:49 +0100

Hello,

one minor comment:

On 11/30/2011 06:14 PM, Hampel, K Georg (K Georg) wrote:
> - MPTCP_RST initiator sends DATA_RST on all subflows.
> 
> - Upon reception of at least one DATA_RST, peer responds with 
> DATA_RST and TCP RST on all subflows and closes down right away. 
> (Should we bundle DATA_RST and TCP RST in this case? I'm not so sure.)

He cannot close down right away, because it's own DATA_RST may get lost.
And we have to answer to retransmissions of the original DATA_RST with a
DATA_RST, which won't be possible if MPTCP got already closed (assuming
that we put at least a DSN in the DATA_RST).

Thus, the receiver of the original DATA_RST has to enter TIME_WAIT-state.

Christoph

-- 
Christoph Paasch
PhD Student

IP Networking Lab --- http://inl.info.ucl.ac.be
MultiPath TCP in the Linux Kernel --- http://inl.info.ucl.ac.be/mptcp
Université Catholique de Louvain
--

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Thu, 1 Dec 2011 10:46:14 -0600

Christoph,

No. The sender of DATA_RST enters RST_WAIT state! In THIS state and ONLY in this state, NO subflow is equivalent to NO connection. This means that teardown of the last subflow also tears down the connection. In all other connection states, tear down of the last subflow does NOT translate into (immediate) tear down of the connection.

When host B receives DATA_RST, it knows that host A is in RST_WAIT state and that TCP RSTs on all subflows would automatically kill host A's connection. For that reasons, host B can tear down the connection right away. This guarantees that future DATA_RST retransmissions by host A will cause TCP RST replies by host B, which (self-consistently) close down host A's connection.
Host B sends Data_RST reply + TCP RSTs solely to reduce the timeframe host A has to spend in RST_WAIT.

- Georg

-----Original Message-----
From: Christoph Paasch [mailto:[email-masked]] 
Sent: Thursday, December 01, 2011 5:06 AM
To: Hampel, K Georg (K Georg)
Cc: Yoshifumi Nishida; multipathtcp; Olivier Bonaventure
Subject: Re: [multipathtcp] MPTCP connection termination

Hello,

one minor comment:

Christoph

-- 
Christoph Paasch
PhD Student

---

From: Yoshifumi Nishida <[email-masked]>
Date: Sat, 3 Dec 2011 15:30:00 -0800

Hi Georg,

It looks better to me. Although I think I'll need to think about
simultaneous cases
more carefully.
I think one interesting point of this is we might be able to
distinguish two cases with
one mptcp option. I mean,

 1: sending DATA_RST .... indicate the sender doesn't receive DATA_RST
from the peer yet
 2: sending DATA_RST + TCP_RST .. indicate the sender already received
DATA_RST from the peer

Since case 2 can be treated as an ACK for DATA_RST, I'm feeling that
we might able
to find a convergence point between Olivier's idea and this idea.

Thanks,
--
Yoshifumi Nishida
[email-masked]

2011/11/30 Hampel, K Georg (K Georg) <[email-masked]>:
> Hi Yoshifumi,
>
> We have to file out the details. One solution to your problem may be:
>
> - Upon reception of at least one DATA_RST, peer responds with DATA_RST and TCP RST on all subflows and closes down right away. (Should we bundle DATA_RST and TCP RST in this case? I'm not so sure.)
>
> - Upon reception of at least on DATA_RST, MPTCP_RST initiator sends TCP RST on all subflows (to close middleboxes) and closes down connection.
>
> This solution would also address the situation where both hosts initiate DATA_RST at the same time.
>
> Does this look better? Thanks.
>
> Regards,
>
> Georg
>
>
>
>
>
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Yoshifumi Nishida
> Sent: Wednesday, November 30, 2011 6:18 AM
> Cc: Olivier Bonaventure; multipathtcp
> Hi Georg,
> Sorry, my point was not clear enough.
> Please correct me if I'm wrong. In my understanding, your proposal
> requires to receive
> TCP RST from ALL subflows to shutdown a MPTCP session.
>
> Let's say we try to reset a MPTCP session which has 10 subflows. In this case,
> even if it successfully receives 9 TCP RSTs, it might have to wait for
> one TCP RST
> from a non-response-path, which might take long time.
>
> Thanks,
> --
> Yoshifumi Nishida
> [email-masked]
>
>
>
> 2011/11/29 Hampel, K Georg (K Georg) <[email-masked]>:
>> Hi Yoshifumi,
>>
>> In case the host doesn't hear anything from its peer for an extended period of time, I think MPTCP should react in the same way as TCP.
>>
>> How does TCP react under such circumstances? To the best of my knowledge, this depends on if keepalives are set or not. If not, TCP keeps hanging until the app is closed down.
>>
>> Regards,
>> Georg
>>
>>
>>
>> From: [email-masked] [mailto:[email-masked]] On Behalf Of Yoshifumi Nishida
>> Sent: Tuesday, November 29, 2011 6:46 AM
>> Cc: Olivier Bonaventure; multipathtcp
>> Hi Georg, Olivier,
>> Thanks for the response. I basically agree with your comments.
>> The corner case 1 might be a minor issue compared to the benefits.
>>
>> As Olivier mentioned, the corner case 2 can be solvable by using a timer to
>> address the case where no response comes back from the peer due to routing
>> failures or middilebox failures or something.
>> But, I think this means if one of paths becomes no-response status, the peer
>> will need to wait for a certain amount of time. I'm wondering that this would be
>> acceptable or not..
>>
>> Thanks,
>> --
>> Yoshifumi Nishida
>> [email-masked]
>>
>> 2011/11/28 Hampel, K Georg (K Georg) <[email-masked]>:
>>> Yoshifumi, Olivier,
>>>
>>> I have inserted comments on top of Olivier's comments. Thanks.
>>>
>>> Georg
>>>
>>>
>>> Yoshifumi,
>>>>
>>>> Sorry for my slow response. I'm now back from vacation.
>>>> My little concern for this is it relies on a little implicit signaling
>>>> compared to using
>>>> acks for data_RST.
>>>> I think there'll be the following corner cases, although they might be
>>>> minor points.
>>>> and might be solved with some additional logics.
>>>>
>>>> Corner case 1:
>>>>
>>>>   Host A sent data_RST, but it was lost and didn't reach Host B.
>>>>   At the same time, Host B sent TCP RST to all connections in order to
>>>> switch networks.
>>>>
>>>>   In this case, Host A considers data_RST arrived at Host B since it
>>>> received TCP RSTs,
>>>>   although this is not true.
>>>
>>> At least they both consider the MPTCP to be closed and have removed
>>> their state.
>>>
>>>
>>> [GH] As I understand this scenario, Host B's MPTCP connection is still up while host A's connection is already closed. This, indeed, is a corner case where data_RST does not do proper connection teardown. In such a scenario, host B would hang in until some timeout has been exceeded.
>>> Here is why we should not be too concerned about this corner case:
>>> 1. The data_RST solution properly handles the vast majority of abrupt teardown scenarios.
>>> 2. This remaining corner case is not handled worse WITH data_RST than it is handled WITHOUT data_RST. Further, the only penalty is that host B's connection-related state is released with some delay.
>>> 3. At the moment, there doesn't seem to be a solution in sight that would address this corner case any better.
>>>
>>>>
>>>> Corner case 2:
>>>>
>>>>   Host A sent data RST and Host B received it.
>>>>   Host B sent TCP RST on all subflows and tore down the connection, however
>>>>   one of paths somehow discarded all packets from both ends.
>>>
>>> At this point, Host B has lost its state for the MPTCP connection
>>>>
>>>>   In this case, Host A continues to send data RST to wait for TCP RST.
>>>
>>> If A sends a Data_RST, it will receive a RST from B and this will allow
>>> A to close the connection. If A never receives the RST, then it will
>>> have to wait for the expiration of a timer to remove its state.
>>>
>>> [GH] If host A does not receive Data_RST reply or TCP RSTs on all synchronized subflows within a short timeframe (e.g. RTO ~ 2 sec), it will retransmit Data_RST. Since this retransmission will occur on all synchronized subflows, it will produce the TCP RSTs needed to tear down the connection.
>>>
>>>
>>> Olivier
>>

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Mon, 5 Dec 2011 10:40:27 -0600

Hi Yoshifumi, all,

I think we have to see the various solutions in light of their respective tradeoffs, i.e. complexity, time-to-shutdown and the corner cases they miss.

I have tried to summarize 3 essential candidate solutions and their respective tradeoffs:

1. Candidate solution:
- Host A sends DATA_RST on all subflows.
- Host B, after receiving one DATA_RST, sends TCP RST on all subflows and closes down.
- Host A receives TCP RSTs on all subflows and closes down.

Reliability: host A retransmits DATA_RST on all subflows that haven't shown TCP RST.

2. Candidate solution:
- Host A sends DATA_RST on all subflows.
- Host B, after receiving one DATA_RST, responds DATA_RST on some subflows and TCP RST on all others.
- Host A, after receiving either one DATA_RST or TCP RSTs on all subflows, sends TCP RST on all remaining subflows and closes down.

Reliability: host A retransmits DATA_RST on all subflows that haven't shown TCP RST.

Tradeoff: More complicated than first candidate but faster since host A only needs to receive one DATA_RST rather than TCP RSTs on all subflows.

3. Candidate solution:
- Host A sends DATA_RST_REQ on at least one subflow.
- Host B, after receiving one DATA_RST_REQ, responds DATA_RST_ACK on at least one subflow.
- Host A, after receiving one DATA_RST_ACK, sends TCP RST on all subflows and closes down.
- Host B, after receiving TCP RST on all subflows, closes down.

Reliability: Host A retransmits DATA_RST_REQ if no DATA_RST_ACK arrives. In case host A's TCP RSTs get lost any further packet by host B causes TCP RST since host A has already closed down. This also closes down host B.

Tradeoff: This candidate avoids the corner case mentioned by Yoshifumi (host B does BBM while host A tries to close down). However, it is more complicated and delays host B's tear down.

Any preference? Any other solution?

Regards,

Georg

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yoshifumi Nishida
Sent: Saturday, December 03, 2011 6:30 PM
To: Hampel, K Georg (K Georg)
Cc: Olivier Bonaventure; multipathtcp
Subject: Re: [multipathtcp] MPTCP connection termination

Hi Georg,

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 06 Dec 2011 15:52:57 +0100

Georg,
> I have tried to summarize 3 essential candidate solutions and their respective tradeoffs:

Thanks for this summary.

> Reliability: host A retransmits DATA_RST on all subflows that haven't shown TCP RST.

Personally, I'm not in favor of waiting for RST on all subflows to
close. A single RST, but generated by the other endpoint should be
sufficient.

> Tradeoff: More complicated than first candidate but faster since host A only needs to receive one DATA_RST rather than TCP RSTs on all subflows.

I'd propose to use a special option for the DATA_RST. This option would
contain the Key of the other entity. It could be encoded as

                           1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +---------------+---------------+-------+-------+-+-----------+-+
      |     Kind      |    Length     |Subtype|Version|C| (reservd) |S|
      +---------------+---------------+-------+-------+-+-----------+-+
      |                         Receiver's Key                        |
      |                            (64 bits)                          |
      |                                                               |
      +---------------------------------------------------------------+

Let's call this the MPTCP_RST option for this email.

If A wants to close an MPTCP connection abruptly, it could :

- Host a sends a regular TCP segment containing the MPTCP_RST option
with B's key on one or more subflows (implementation choice)
- Host B, after receiving a regular segment containing a valid MPTCP_RST
on one subflow sends a TCP RST segment containing the MPTCP_RST option
(and thus A's key) on all subflows and closes.
- Host A, after receiving one valid RST segment with the MPTCP_RST
option closes down (the other RST segments that it may receive after
will be ignored since they are RST)

Reliability : host A retransmits MPTCP_RST as long as it hasn't received
a RST segment that contains MPTCP_RST from the other end.

If both A and B send a regular segment with MPTCP_RST at the same time,
they will both answer by sending TCP RST with MPTCP_RST on all subflows.

If A does not receive any of the RST segments sent by B, A will
retransmit MPTCP_RST on one or more subflows and receive RST since there
is no state on B for these subflows. A should leave this state after a
timeout and consider the connection to be closed

This solution asks host B to send more segments to close the MPTCP
connection that host A.

Using MPTCP_RST inside RST segment should avoid having problems with RST
segments generated by middleboxes.

> Tradeoff: This candidate avoids the corner case mentioned by Yoshifumi (host B does BBM while host A tries to close down). However, it is more complicated and delays host B's tear down.

This solution seems long for an abrupt release.

Olivier

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Tue, 6 Dec 2011 13:28:21 -0600

Olivier,

On a high level, we seem to converge toward a solution that's close to candidate 2.

I'm fine with the MPTCP_RST option format.

Opposed to my candidate 2 solution, you propose to piggyback MPTCP_RST onto TCP_RST. This creates an advantage and a disadvantage over candidate 2:

Advantage over candidate 2:

Since host A can distinguish between host-B's TCP_RST and TCP_RSTs sent by middleboxes, it avoids the rare corner case where host A sends MPTCP_RST while host B undergoes BBM and a middlebox sends TCP_RST on behalf of host B.

Disadvantage over candidate 2:

The reset procedure is not reliable since it keeps host-A hanging when host B's TCP_RST/MPTCP_RST packet gets lost.

I claim that my proposal has better performance, because:

- The rare corner case is far less likely than losing host-B's MPTCP_RST packet.

- The damage of either incident is the same: One of the hosts has shut down while the other hangs in waiting.

There may be other concerns about piggybacking "higher-layer" information onto TCP RST. Since TCP RST is the "lower layer" message, it should be evaluated first, which means that the subflow and all corresponding state is torn down. At that point higher layer content cannot be forwarded anymore.

Regards,

Georg

-----Original Message-----
From: Olivier Bonaventure [mailto:[email-masked]]
Sent: Tuesday, December 06, 2011 9:53 AM
To: Hampel, K Georg (K Georg)
Cc: Yoshifumi Nishida; [email-masked]; [email-masked]; [email-masked]; Olivier Bonaventure; multipathtcp
Subject: Re: [multipathtcp] MPTCP connection termination

Georg,

Thanks for this summary.

sufficient.

                           1                   2                   3

      |                         Receiver's Key                        |

      |                            (64 bits)                          |

      |                                                               |

connection that host A.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 06 Dec 2011 21:08:24 +0100

Georg,
> 
>  
> 
> On a high level, we seem to converge toward a solution that’s close to
> candidate 2.

Let's focus the discussion on this case.

> - Host A, after receiving either one DATA_RST or TCP RSTs on all
subflows, sends TCP RST on all remaining subflows and closes down.
> Tradeoff: More complicated than first candidate but faster since host
A only needs to receive one DATA_RST rather than TCP RSTs on all subflows.

1. I don't think that we should mandate to transmit DATA_RST on all
subflows. There are many cases where the transmission of a single
DATA_RST would be sufficient and we should try to minimise the number of
packets that are used to close an MPTCP connection. Some connections are
very short.

2. The behaviour of host B is unclear to me. How does host B decides on
which  connection it sends a DATA_RST and on which it sends a regular RST ?

3. If host A received a RST on all subflows, it should not send a RST on
any subflow. In fact, it should only send a RST on the subflows over
which it has not yet received a RST. Otherwise, there is a risk of
answering to a RST by another RST

I think that we should minimise the total number of segments that are
sent and also the number of segments that are sent on each subflow. An
abrupt release should be as fast as possible.

Olivier

-- 
INL, ICTEAM, UCLouvain, Belgium, http://inl.info.ucl.ac.be

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Tue, 6 Dec 2011 14:20:36 -0600

Olivier,

Comments inserted below.

Georg

-----Original Message-----
From: Olivier Bonaventure [mailto:[email-masked]] 
Sent: Tuesday, December 06, 2011 3:08 PM
To: Hampel, K Georg (K Georg)
Cc: Yoshifumi Nishida; [email-masked]; [email-masked]; [email-masked]; multipathtcp
Subject: Re: [multipathtcp] MPTCP connection termination

Georg,

[GH] We can simplify things in the following way: 
- Host A sends DATA_RST on one subbflow and TCP_RST on all others (The TCP RSTs must be sent anyway to clear middleboxes and host A is the culprit who wants to kill the connection, so let him pay the price). 
- Host B replies to DATA_RST with DATA_RST on the same subflow and tears down its connection.
- Host A tears down its connection as soon as it receives DATA_RST or TCP_RST on the remaining subflow. 
Reliability: If host B's DATA_RST got lost, host A will simply retransmit DATA_RST on this one subflow. This will create a TCP RST.

[GH] I tried to keep it rather flexible. This may add confusion.

[GH] That's right.

Olivier

---

From: Olivier Bonaventure <[email-masked]>
Date: Tue, 06 Dec 2011 21:53:33 +0100

Georg,

I like the idea of "penalising" the host that wants to perform the
abrupt release.

> Reliability: If host B's DATA_RST got lost, host A will simply retransmit DATA_RST on this one subflow. This will create a TCP RST.

However, we need to take into account the possibility of collisions when
both A and B send DATA_RST at the same time and RST on different subflows.

Consider that A and B have one IPv4 and one IPv6 address.

To close, A sends DATA_RST over IPv6 and RST over IPv4
At the same time, B sends DATA_RST over IPv4 and RST over IPv6.

Each host will receive a RST in response to its DATA_RST. This implies
that B cannot always respond to a DATA_RST by sending a DATA_RST on the
same subflow.

I would propose to rewrite your proposal as follows :

We have one option, called MPTCP_RST that contains a 64 bits key. This
option can be sent inside any TCP segment.

- Host A sends a regular TCP segment containing MPTCP_RST(B's key) on
one subflow and TCP RST segments on all other subflows

- Upon reception of a regular TCP segment containing MPTCP_RST and its
key, a host responds by sending a TCP RST segment and releases the
entire MPTCP connection

- Host A tears down the MPTCP connection as soon as it receives a TCP
segment containing MPTCP(A's key) or a RST segment on the remaining subflow

Reliability :

If A's MPTCP_RST is lost, A will retransmit and B will respond with a
RST segment (whether the subflow still exists or not).

If B's RST is lost, A will retransmit MPTCP_RST and B will respond with
RST as it does not have state anymore.

If A and B send MPTCP_RST exactly at the same time, the RST sent in step
one will collide but without causing any problem and A and B will
receive the MPTCP_RST without having any state. They will thus both
reply by sending RST.

Performance :

If there are no losses, the initiator of the abrupt release sends one
segment per subflow while the other host only needs to send one segment.

Olivier

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Tue, 6 Dec 2011 17:52:08 -0600

Olivier,

Almost. I'm still stumbling over one point:

[GH] On what subflow does the host send TCP RST? Obviously on the one where it received MPTCP_RST.

[GH] In case B's MPTCP_RST segment (holding A's key) arrives on a subflow where A already sent a TCP RST, host B would have to send a TCP RST on the remaining subflow i.e. the one where it sent its own MPTCP_RST. This is not covered by the second bullet and not contained in the third.

Recast:
- A host (host A), which wants to tear down its MPTCP connection, sends a regular TCP segment containing MPTCP_RST (B's key) on one subflow and TCP RST segments on all other subflows, and then it goes into RST_WAIT state.

- A host (host B), which is not in RST_WAIT state, and receives a regular TCP segment containing MPTCP_RST and its key responds by sending a TCP RST segment on the subflow where it received the MPTCP_RST segment and releases the entire MPTCP connection

- A host in RST_WAIT state (host A) tears down the MPTCP connection as soon as it receives a TCP segment containing MPTCP_RST (A's key) or a RST segment on the remaining subflow. If it receives a MPTCP_RST segment, it sends TCP RST on this remaining subflow.

In other words: There is a difference between host B receiving MPTCP_RST and host A receiving MPTCP_RST.

Georg  

-----Original Message-----
From: Olivier Bonaventure [mailto:[email-masked]] 
Sent: Tuesday, December 06, 2011 3:54 PM
To: Hampel, K Georg (K Georg)
Cc: Yoshifumi Nishida; [email-masked]; [email-masked]; [email-masked]; multipathtcp
Subject: Re: [multipathtcp] MPTCP connection termination

Georg,

Reliability :

Performance :

Olivier

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Thu, 8 Dec 2011 12:43:26 -0600

Olivier,

I'm taking back the modification I proposed in my last email on MPTCP_RST. Opposed to what I suggested, host A should not be able to receive MPTCP_RST on a subflow, where it already sent TCP RST before. In case both hosts send MPTCP_RST at the same time on different subflows, MPTCP_RST retransmissions would provoke TCP RSTs and resolve the problem. 

This means we would agree on the following (simple) procedure:

- Host A sends a regular TCP segment containing MPTCP_RST(B's key) on
one subflow. It sends TCP RST segments on all other subflows and tears them down.

- Upon reception of a regular TCP segment containing MPTCP_RST and its
key, a host responds by sending a TCP RST segment on the same subflow and releases the entire MPTCP connection.

- Host A tears down the MPTCP connection as soon as it receives a TCP
segment containing MPTCP(A's key) or a RST segment on the remaining subflow.

Georg

-----Original Message-----
From: Hampel, K Georg (K Georg) 
Sent: Tuesday, December 06, 2011 6:52 PM
To: [email-masked]'
Cc: Yoshifumi Nishida; [email-masked]; [email-masked]; [email-masked]; multipathtcp
Subject: RE: [multipathtcp] MPTCP connection termination

Olivier,

Recast:

Georg  

-----Original Message-----
From: Olivier Bonaventure [mailto:[email-masked]] 
Sent: Tuesday, December 06, 2011 3:54 PM
To: Hampel, K Georg (K Georg)
Cc: Yoshifumi Nishida; [email-masked]; [email-masked]; [email-masked]; multipathtcp
Subject: Re: [multipathtcp] MPTCP connection termination

Georg,

Reliability :

Performance :

Olivier

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 12 Dec 2011 02:49:45 -0800

Hi Georg, Olivier,

I have the following minor concerns on this, but otherwise it looks fine to me.
So, are we getting close?

1: I think we'll need to feed some data to subflow in order to send
MPTCP_RST otherwise it will look pure ACK packet.

2: we should have a certain retransmission timeout for MPTCP_RST since
there'll be a case where no response comes back from the peer.

3: there'll be a case where host B or middlebox sends a TCP_RST
without intending to terminate the MPTCP session, while it might
trigger host A to tear down the session. But, is this a corner case
that we can ignore since it will rarely happen and the penalty won't
be fatal?

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: "Hampel, K Georg (K Georg)" <[email-masked]>
Date: Mon, 12 Dec 2011 10:41:43 -0600

Hi Yoshifumi,

I have inserted comments. Thanks.

Georg

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yoshifumi Nishida
Sent: Monday, December 12, 2011 5:50 AM
To: Hampel, K Georg (K Georg)
Cc: [email-masked]; multipathtcp
Subject: Re: [multipathtcp] MPTCP connection termination

Hi Georg, Olivier,

[GH] What if the client doesn't have any data to send? There are other cases where MP options are piggybacked on ACKs (DSS with DATA_ACK, MP_PRIO, ADD_ADDRESS etc). Is there any difference to MP_RST?

[GH] I agree. Host A could go to TIME_WAIT as soon as it sends the first MP_RST and apply the same timer it generally uses for TIME_WAIT.

[GH] Indeed, this is a corner case which only happens if host B undergoes a BBM at the same time as host A sends MP_RST. In think the behavior of host B in this case would fall under the topic "MP connection management" which is currently under discussion on the ML: What should do a host do who receives TCP RST on the last subflow or who doesn't see any response from its peer. We need to resolve this as well.

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 14 Dec 2011 02:50:52 -0800

Hi Georg,

> [GH] What if the client doesn't have any data to send? There are other cases where MP options are piggybacked on ACKs (DSS with DATA_ACK, MP_PRIO, ADD_ADDRESS etc). Is there any difference to MP_RS…

In case of the other cases, I think you can wait until there is a
chance to be piggybacked.
However, we don't want to wait in case of MP_RST. I think we might
have to feed dummy data to a subflow in such case even though it's a
bit ugly.

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 14 Dec 2011 14:48:39 +0100

Hi all,

> segment containing MPTCP(A's key) or a RST segment on the remaining subflow.

I have now implemented this behavior in our MPTCP-kernel.

And it works very well.

I don't see the need to put dummy data in the MP_RST.

The option-format I have chosen is the following (subtype 0x7):

                           1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +---------------+---------------+-------+-------+-+-----------+-+
      |     Kind      |    Length     |Sub:0x7|      Reserved         |
      +---------------+---------------+-------+-------+-+-----------+-+
      |                         Receiver's Key                        |
      |                            (64 bits)                          |
      |                                                               |
      +---------------------------------------------------------------+

Christoph

-- 
Christoph Paasch
PhD Student

IP Networking Lab --- http://inl.info.ucl.ac.be
MultiPath TCP in the Linux Kernel --- http://mptcp.info.ucl.ac.be
Université Catholique de Louvain
--

---

From: Olivier Bonaventure <[email-masked]>
Date: Wed, 14 Dec 2011 22:12:21 +0100

Georg, Yoshifumi,

> [GH] What if the client doesn't have any data to send? There are other cases where MP options are piggybacked on ACKs (DSS with DATA_ACK, MP_PRIO, ADD_ADDRESS etc). Is there any difference to MP_RS…

There is no difference. In both cases, we need to retransmit if the
segment got lost.

> [GH] I agree. Host A could go to TIME_WAIT as soon as it sends the first MP_RST and apply the same timer it generally uses for TIME_WAIT.

I'm not sure that this should be mixed with TIME_WAIT. I'd prefer a
specific state where the host waits for a RST in response to its MPTCP_RST

> [GH] Indeed, this is a corner case which only happens if host B undergoes a BBM at the same time as host A sends MP_RST. In think the behavior of host B in this case would fall under the topic "MP…

If host A sends an MPTCP_RST, this means that it does not want to
continue the connection. It might receive a TCP_RST that was not related
to the abrupt release of the MPTCP connection, but at this point this
does not really matter as it has sent TCP_RST on all other subflows.
Host B will keep unnecessarily state for some time, but this cannot be
avoided.

Olivier

---

From: Yoshifumi Nishida <[email-masked]>
Date: Thu, 15 Dec 2011 02:19:49 -0800

Hi Olivier,

> There is no difference. In both cases, we need to retransmit if the

My concern was if the packets which have MPTCP options don't have
payload, it will look like dup ACKs. Because seqno and ackno will not
be advanced. The might be a situation where dup ACKs might trigger
some unnecessary actions on middleboxes or end nodes. Do you think
it's OK since we're going to shutdown the connection anyway?

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: Olivier Bonaventure <[email-masked]>
Date: Thu, 15 Dec 2011 14:50:53 +0100

Yoshifumi,
> it's OK since we're going to shutdown the connection anyway?

If dupack means retransmit, then this is not issue as we close.
If dupack means that the middlebox will drop the dupack, then we will
send a RST after x retransmission attempts. I don't see how we could add
data to ensure that the segment that only contains an option will be
delivered by all middleboxes

Olivier

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 19 Dec 2011 01:47:55 -0800

Hi Olivier,

> delivered by all middleboxes

OK. As long as we can consider it won't be harmful, I'm fine with it.
Sorry for bringing up minor thing.
I think most of the points for this feature have been addressed and clarified.
If you think it's ready, could you update the draft? Or, are we still
missing something?

Thanks,
--
Yoshifumi Nishida
[email-masked]

---

From: Alan Ford <[email-masked]>
Date: Thu, 29 Dec 2011 18:20:05 +0000

All,

Apologies for the delay in catching up with these recent developments...

A few thoughts:

I don't particularly like the term MPTCP_RST or DATA_RST, since this is not a direct analogy to the TCP RST; rather, it is almost new functionality - a "fast close" mechanism, so it may be better termed DATA_CLOSE or similar.

What is the reason for the reuse of the key in this context? Is there significant additional security?

Do we really need it as a new option? Could we not just do it as an extra flag in the DSS? I thought that was the original proposal and I didn't see a rationale for a change. This would have the benefit that it requires no additional options, and could be included with data (it would seem unlikely to be able to fit an option with a key as well as a DSS option on a data packet).

In summary: I think the proposed mechanism below will cover most standard needs for this option, and any corner cases will be handled by normal session expiry. I think, however, that it should be implemented simply as a flag on the DSS option.

Regards,
Alan

On 8 Dec 2011, at 18:43, Hampel, K Georg (K Georg) wrote:

> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp