Subject: [multipathtcp] rfc6824bis - RST after MP_FASTCLOSE retransmission

From: François Finfe <[email-masked]>
Date: Tue, 23 May 2017 10:55:32 +0200

Hello,

I encountered an issue with MP_FASTCLOSE and a stateful firewall.

Let's explain it with the following scenario. See figure 1.
Firewalls M and N are stateful firewall which don't analyse MPTCP
options and ignore them.

- An MPTCP connection has been established between host A and host B.
- Host A sends a ACK with the MP_FASTCLOSE option.
- Firewall M forwards the packet to host firewall N.
- Firewall N forwards the packet to host B.
- Host B receives the MP_FASTCLOSE and replies with a TCP RST.
- Firewall N forwards the TCP RST packet to firewall M.
   Due to the TCP RST, the stateful firewall removes the connection
   state.
- The TCP RST is lost due to a lossy link, network congestion, etc.

- As host A didn't receive the expected TCP RST packet, a timeout fires
   a MP_FASTCLOSE retransmission.
- Firewall M forwards the packet to firewall N.
- For firewall N, this connection no more exists. It sees the
   MP_FASTCLOSE as an TCP ACK packet without any related connection.
   Firewall N drops the packet.
- MP_FASTCLOSE are retransmitted until the limit of MP_FASTCLOSE
   retransmission is reached.
- If nothing is done, firewall M will retain the connection state for
   some time until a connection tracking timeout occurs.
   In a production environment, with a lot of simultaneous connection,
   this kind of entries (erroneous connection state for an already closed
   connection) can accumulate in the firewall.
   Due to ressources limitation, this might lead to performance issue
   where new connections might be rejected.

To mitigate this issue, here is a proposal for rfc6824bis:
- When the limit of MP_FASTCLOSE retransmission is reached, a TCP RST
   could be sent by host A.
- In this scenario, firewall M forwards the TCP RST packet and removes
   the connection state.

This TCP RST packet could contain the MP_FASTCLOSE option.

   Host A                                                          Host B
    |                 Firewall M             Firewall N                |
    |                      |                      |                    |
    |  ACK(MP_FASTCLOSE)   |  ACK(MP_FASTCLOSE)   | ACK(MP_FASTCLOSE)  |
    |--------------------->|--------------------->|------------------->|
    |                      |             TCP RST  | TCP RST            |
    |                      |           x----------|<-------------------|
    |                      |                      |                    |
    |                      |                      |                    |
    |                      |                      |                    |
    |  ACK(MP_FASTCLOSE)   |  ACK(MP_FASTCLOSE)   |                    |
    |--------------------->|--------------------->x                    |
    |                      |                      |                    |
    |                      |                      |                    |
    |  ACK(MP_FASTCLOSE)   |  ACK(MP_FASTCLOSE)   |                    |
    |--------------------->|--------------------->x                    |
    |                      |                      |                    |
    :                      :                      :                    :
    :                                                                  :
    :  Multiple MP_FASTCLOSE retransmissions until limit is reached    :
    :                                                                  :
    :                      :                      :                    :
    :                      :                      :                    :
    :                      :                      :                    :
    |  Last retransmission |                      |                    |
    |                      |                      |                    |
    |  ACK(MP_FASTCLOSE)   |  ACK(MP_FASTCLOSE)   |                    |
    |--------------------->|--------------------->x                    |
    |                      |                      |                    |
    | (timeout)            |                      |                    |
    |  RST(MP_FASTCLOSE)   |  RST(MP_FASTCLOSE)   |                    |
    |--------------------->|--------------------->x                    |
    |                      |                      |                    |

    Figure 1

Best regards,

François Finfe

-- 

------------------------------
DISCLAIMER.
This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. 
If you have received this email in error please notify the system manager. 
This message contains confidential information and is intended only for the 
individual named. If you are not the named addressee you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately by e-mail if you have received this e-mail by mistake and 
delete this e-mail from your system. If you are not the intended recipient 
you are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this information is strictly 
prohibited.

---

From: Christoph Paasch <[email-masked]>
Date: Tue, 23 May 2017 22:05:27 -0700

Hello Francois,

> This TCP RST packet could contain the MP_FASTCLOSE option.

wouldn't this exact same scenario happen with regular TCP as well?

For example, host A is sending data while host B at one point decides to
drop the connection with a TCP RST. This TCP RST gets lost between firewall
N and M. A will keep on retransmitting its data until it gives up and
signals an ETIMEDOUT to the application upon which the socket gets silently
closed.

If we deem the problem severe enough in MPTCP to address it for the
MP_FASTCLOSE scenario, then I think it should in general be considered for
TCP that when a connection times out it must be closed with a TCP RST.

Christoph

> immediately by e-mail if you have received this e-mail by mistake and delete
> this e-mail from your system. If you are not the intended recipient you are
> notified that disclosing, copying, distributing or taking any action in
> reliance on the contents of this information is strictly prohibited.
> 
> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Olivier Bonaventure <[email-masked]>
Date: Wed, 24 May 2017 10:28:30 +0200

Christoph, François,

> wouldn't this exact same scenario happen with regular TCP as well?

Yes, indeed. The difference in MPTCP is that we have sent a RST on all 
subflows except the one where we sent the MP_FASTCLOSE. In fact, the 
MP_FASTCLOSE that we send indicates that we want to remove state for 
this MPTCP connection and the corresponding subflow.

> signals an ETIMEDOUT to the application upon which the socket gets silently

Agreed, but here we are in a situation where the application has already 
accepted to close the connection.

> TCP that when a connection times out it must be closed with a TCP RST.

In general, I think that it would be wise for a TCP stack to send a RST 
when the stack decides to remove state for a connection.

For MPTCP, a possible modification to rfc6824bis could be

    o  If Host A does not receive a TCP RST in reply to its MP_FASTCLOSE
       after one retransmission timeout (RTO) (the RTO of the subflow
       where the MPTCP_RST has been sent), it SHOULD retransmit the
       MP_FASTCLOSE.  The number of retransmissions SHOULD be limited to
       avoid this connection from being retained for a long time, but
       this limit is implementation specific.  A RECOMMENDED number is 3.

We could add. a sentence like : After 3 retransmission of the 
MP_FASTCLOSE without any TCP RST in response, Host A MAY send a TCP RST 
when it releases the state for this MPTCP connection.

Olivier