Subject: [multipathtcp] Multipath TCP Address advertisement 2/5 - Reliability

From: Fabien Duchêne <[email-masked]>
Date: Wed, 3 Aug 2016 17:47:05 +0200

Hello,

As agreed in Berlin during IETF96, I'm sending a series of emails to
discuss the different contributions proposed
inhttps://datatracker.ietf.org/doc/draft-duchene-mptcp-add-addr/

This is the part 2/5 : reliability.

In RFC6824, ADD_ADDR options can be attached to segments carrying data
or pure acknowledgements.
In practice, notably given the length of ADD_ADDR with IPv6 addresses
and the HMAC, it is very likely that they will be often sent as pure
acknowledgements.
This implies that ADD_ADDR are sent unreliably, which could be
problematic when the ADD_ADDR is required to allow the establishment of
additional subflows, as in load balancing scenarios. 
We propose to rely on the "E" (Echo) flag in the ADD_ADDR option.
This echo flag is used to acknolwedge a received ADD_ADDR by echoing it.
If the acknowledgement is not received, the ADD_ADDR option will be
retransmitted up to N times.

Fabien

---

From: Alan Ford <[email-masked]>
Date: Wed, 3 Aug 2016 17:11:19 +0100

I am neutral on the addition on this flag… We have spare bits, so could easily allocate this. But is it worth it? Although not explicitly written, I believe we had expected senders to send the ADD_ADDR option multiple times.

If the ADD_ADDR was not echoed, what would the sender do? Retransmit a few times I guess. I also guess it could learn that a path may not support duplicate ACKs with options.

It could be useful, but I do not feel strongly here… If this were an IETF room, I’d hum weakly ;-)

Regards,
Alan

> On 3 Aug 2016, at 16:47, Fabien Duchêne <[email-masked]> wrote:
> 
> Hello,
> 
> _______________________________________________
> multipathtcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/multipathtcp

---

From: Fabien Duchêne <[email-masked]>
Date: Wed, 3 Aug 2016 19:05:11 +0200

> I am neutral on the addition on this flag… We have spare bits, so could easily allocate this. But is it worth it? Although not explicitly written, I believe we had expected senders to send the ADD_…
The idea here is, with the load balancing in mind, we *really* want the
ADD_ADDR to be delivered, and sending it multiple times doesn't really
guarantee that it will be delivered.
> If the ADD_ADDR was not echoed, what would the sender do? Retransmit a few times I guess. I also guess it could learn that a path may not support duplicate ACKs with options.
Retransmit it a few times, and if not echoed, we could know that it
hasn't been delivred.
Because there's the case where the ADD_ADDR is received but the client
doesn't elect to open a subflow on this address.
With the echo bit, if such thing happen basically you would know "ok, he
received my add_addr but doesn't want to establish a subflow for now",
otherwise you won't be able to tell the difference between a lost
ADD_ADDR and this case.

I agree with the dupacks with options.

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 03 Aug 2016 23:01:30 +0200

> I am neutral on the addition on this flag… We have spare bits, so could
> easily allocate this. But is it worth it?

I haven't read the draft.  But it looks to my uneducated eyes that adding
this flag carries little additional implementation complexity, so even if
it turns out to be useless it won't harm much.  On the other hand, should
it turn out to be useful at a later date, it will be difficult to add in
a backwards compatible manner.

Can somebody who has implemented this flag confirm that it's easy to
implement, and that there are no hidden gotchas?

-- Juliusz

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 03 Aug 2016 20:52:40 -0700

> On Aug 3, 2016, at 8:47 AM, Fabien Duchêne <[email-masked]> wrote:
> 
> Hello,
> 

I support the addition of the E-flag. Especially, as it improves reliability if one chooses to do the load-balancer deployment as described.

Christoph

---

From: Yoshifumi Nishida <[email-masked]>
Date: Wed, 3 Aug 2016 23:36:26 -0700

Hi Fabien,

> retransmitted up to N times.

I have several comments on this. Please let me know if I miss something.

1： Do we really need ADD_ADDR reliability in all cases? An end node might
want to send ADD_ADDR for 'just in case' rather than for "I want you to use
this"
BTW, I'm not sure if the draft tries to replace the current ADD_ADDR or to
propose an additional ADD_ADDR option.

2: The sender cannot be sure whether the info in ADD_ADDR will be used or
not. It depends on the peer's decision.
I might prefer re-transmitting ADD_ADDR up to N times when the sender
doesn't receive MP_JOIN to the address for a certain amount of time.

3: "the receiving host MUST return the exact option.." Although it is MUST,
can we always do this? There might be a situation where there is no option
space or no segment to send.

Thanks,
--
Yoshi

---

From: Alan Ford <[email-masked]>
Date: Thu, 4 Aug 2016 08:57:57 +0100

Hi Yoshi,

Inline…

> On 4 Aug 2016, at 07:36, Yoshifumi Nishida <[email-masked]> wrote:
> 
> Hi Fabien,
> 
> On Wed, Aug 3, 2016 at 8:47 AM, Fabien Duchêne <[email-masked] <mailto:[email-masked]>> wrote:
> Hello,
> 
> inhttps://datatracker.ietf.org/doc/draft-duchene-mptcp-add-addr/ <http://datatracker.ietf.org/doc/draft-duchene-mptcp-add-addr/>
> 
> 1： Do we really need ADD_ADDR reliability in all cases? An end node might want to send ADD_ADDR for 'just in case' rather than for "I want you to use this" 
> BTW, I'm not sure if the draft tries to replace the current ADD_ADDR or to propose an additional ADD_ADDR option.

The argument is that by making it reliable, you only need to send it once to know it’s been received, even if the far end does not use it.

> 
> 2: The sender cannot be sure whether the info in ADD_ADDR will be used or not. It depends on the peer's decision.   
> I might prefer re-transmitting ADD_ADDR up to N times when the sender doesn't receive MP_JOIN to the address for a certain amount of time.
> 

As above, by making it reliable you only need to send it once to know it’s been received, even if the far end chooses not to use it.

I think I’m sold on this idea now, given the additional recent comments. I support this.

> 3: "the receiving host MUST return the exact option.." Although it is MUST, can we always do this? There might be a situation where there is no option space or no segment to send.  

I would assume the same rules as ADD_ADDR are followed, so it’s sent on a duplicate ACK in that case.

Regards,
Alan

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 5 Aug 2016 00:17:40 -0700

Hi Alan,

On Thu, Aug 4, 2016 at 12:57 AM, Alan Ford <[email-masked]> wrote:

> On Wed, Aug 3, 2016 at 8:47 AM, Fabien Duchêne <fabien.duchene@
> uclouvain.be> wrote:
>
>> Hello,
>>
> The argument is that by making it reliable, you only need to send it once
> to know it’s been received, even if the far end does not use it.
>
> As above, by making it reliable you only need to send it once to know it’s
> been received, even if the far end chooses not to use it.
>

Hmm. if a sender wants the peer to use specific address (such as a load
balancer case), I can understand that we want to make it reliable. But, if
the sender doesn't care, I'm not very sure the reason to make it reliable
or send ADD_ADDR frequently.

I personally think this is a trade-off point. We are trying to introduce
some complexities in the protocol to send TCP options reliably.  I might
want to see some benefits to compensate it.

> 3: "the receiving host MUST return the exact option.." Although it is
> MUST, can we always do this? There might be a situation where there is no
> option space or no segment to send.
>
>

Hmm. If we apply the same rules to this, it may need to send a dupack for a
dupack and may be repeated N times. I still need to think about it, but it
doesn't look pretty..

Thanks,
--
Yoshi

---

From: Alan Ford <[email-masked]>
Date: Sat, 6 Aug 2016 21:35:53 +0100

Hi Yoshi,

Inline...

On Friday, 5 August 2016, Yoshifumi Nishida <[email-masked]> wrote:

> Hi Alan,
>
> On Thu, Aug 4, 2016 at 12:57 AM, Alan Ford <[email-masked]
> <javascript:_e(%7B%7D,'cvml',[email-masked]');>> wrote:
>
>> Hi Yoshi,
>>
>> Inline…
>>
>> On 4 Aug 2016, at 07:36, Yoshifumi Nishida <[email-masked]
>> <javascript:_e(%7B%7D,'cvml',[email-masked]');>> wrote:
>>
>> Hi Fabien,
>>
>> On Wed, Aug 3, 2016 at 8:47 AM, Fabien Duchêne <fabien.duchene@uclouv
>> ain.be <javascript:_e(%7B%7D,'cvml',[email-masked]');>>
>> wrote:
>>
>>> Hello,
>>>
>> BTW, I'm not sure if the draft tries to replace the current ADD_ADDR or
>> to propose an additional ADD_ADDR option.
>>
>>
>> As above, by making it reliable you only need to send it once to know
>> it’s been received, even if the far end chooses not to use it.
>>
>

If local policy means the subflow won't be immediately established, there's
no way of knowing if it's been received.

Similarly, if the subflow can't be created due to e.g. firewall/network
issues, it would be useful to know if the ADD_ADDR had nevertheless been
received, so you could stop sending it.

> Hmm. If we apply the same rules to this, it may need to send a dupack for
> a dupack and may be repeated N times. I still need to think about it, but
> it doesn't look pretty..
>

OK, we wouldn't need to retransmit the echo, unless a new ADD_ADDR comes
in. You raise a good point though about rate limiting however, an
implementation couldn't just keep on retransmitting for ever, and needs to
assume the options are being lost and stop sending them.

Regards,
Alan

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 8 Aug 2016 01:07:03 -0700

Hi Alan,

On Sat, Aug 6, 2016 at 1:35 PM, Alan Ford <[email-masked]> wrote:

>>> ain.be> wrote:
>>> 1： Do we really need ADD_ADDR reliability in all cases? An end node
>>> might want to send ADD_ADDR for 'just in case' rather than for "I want you
>>> to use this"
>>> The argument is that by making it reliable, you only need to send it
>>> once to know it’s been received, even if the far end does not use it.
>>>
> If local policy means the subflow won't be immediately established,
> there's no way of knowing if it's been received.
>

Right. But, let's say we send ADD_ADDR at once in 3 mins, is it such
serious waste of resources?  If we don't care if the peer uses the address
or not, we might not need to send it so frequently and the logic can be
simpler. Also, if the peer uses the address in 3 mins, all we need to send
is one segment.

> assume the options are being lost and stop sending them.

I think we can prepare some rules here to avoid potential issues and corner
cases.
if we can make them simple enough, then we are good. But, I'm just not very
sure yet.

--
Yoshi