Subject: Re: [multipathtcp] 6824bis -11 review

From: Christoph Paasch <[email-masked]>
Date: Wed, 01 Aug 2018 10:26:34 -0700

On 31/07/18 - 13:07:06, Alan Ford wrote:
> Hi Rahul, all,
> 
> > On 31 Jul 2018, at 01:21, Rahul Arvind Jadhav <[email-masked]> wrote:
> >  
> > c) sending a new ADD_ADDR with same ID + address and different port is fine
> > [RJ] again this is what the text already says, but I’m not still clear about this. Why ID + address has to match? Only ID should be enough. 
> >  
> > Consider:
> >  
> > a) ADD_ADDR comes in saying ID=2, addr=10.0.0.1 (private)
> > b) MP_JOIN comes in saying ID=2, addr=8.9.10.11 (public)
> > c) ADD_ADDR comes in saying ID=2, addr=192.168.0.1
> >  
> > What does this mean? Is (c) an implicit REMOVE+ADD? And thus does this mean the subflow created in (b) is now no longer valid?
> >  
> > By keeping the address / ID mapping, if an address is lost, then so is the subflow.
> >  
> > [RJ2] Rather than viewing ( c) as an implicit REMOVE+ADD, can we see it as an UPDATE? Because ID remains same. Subflow created in (b) remains valid (not sure if I’m missing anything here, I see no reason to lose this subflow).
> > In step ( c), the receiver can override Address field in context to ID=2 to the new addr (192.168.0.1) ……   8.9.10.11 is kept separately as the learnt actual rcvd IP (from MP_JOIN) in the same context ID=2. We have to agree that an implementation needs to maintain this IP separately in context to ID=2 anyways.
> > So if we do an explicit REMOVE+ADD, the end result is the same as this UPDATE (but with much less overhead & impl ease).
> 
> I would be interested in hearing from other implementers here… Is there an issue with “updating” using ADD_ADDR?
> 
> Given we say that REMOVE_ADDR should not remove a subflow if it’s still alive, we have precedent for implying a subflow can exist to an address which does not have an Address ID - to - IP address mapping.
> 
> So given that, does updating, as Rahul suggests, cause a problem?

I haven't yet implemented MPTCP path-management in a way that both sides can
trigger the creation of a subflow - which is the scenario we are concerned
about here.

If we would try to do this, I think we should always "believe" the latest
information that we are receiving. In the above scenario, this would mean
that (c) would update the Address-ID to IP mapping.

Yes, it's a private IP and probably not routable - but that simply means
that there is an implementation-bug on the client as he should not have
announced the IP in the first place.

Should (c) trigger a removal the subflow established in (b) ? I don't think
so. As long as the subflow is working fine,... we should keep it. Only
REMOVE_ADDR should trigger an explicit removal.

That's just my opinion.

Christoph

---

From: Christoph Paasch <[email-masked]>
Date: Wed, 01 Aug 2018 10:26:58 -0700

On 28/07/18 - 18:19:16, Alan Ford wrote:
> Hi Rahul,
> 
> Thank you very much indeed for this review.
> 
> > On 28 Jul 2018, at 08:38, Rahul Arvind Jadhav <[email-masked]> wrote:
> > 
> > Hi,
> > 
> > As promised  in IETF102, here is my review for 6824bis-11. As Phil had suggested, I didn't take the trouble to check if my comments overlap with any of the previous discussions/comments. 
> > 
> > 1. Section 3.6: Subflow Reset
> > [RJ]: Is MP_TCPRST a SHOULD or MUST to be included along with the TCP_RST ? Can this be stated explicitly in section 3.6?
> 
> It’s a MAY :)  We say:
> 
> "These semantics are carried in the MP_TCPRST option that can be included on a TCP RST packet.”
> 
> We should change that to “MAY be included”.
> 
> > 2. Section 3.4.2. Remove Address
> >   "The sending and receipt (if no keepalive response was received) of
> >   this message SHOULD trigger the sending of RSTs by both hosts on the
> >   affected subflow(s) "
> > [RJ]: Since a MP_TCPRST should be associated with this RST, what should be the corresponding Reason Code that can be used in MP_TCPRST for this case? I checked existing reason codes in section 3.6 and none of those made sense (may be except for 0x0 ?).
> 
> That’s an interesting one. There may be some benefit (maybe for middleboxes, or maybe for security) to sharing this information in the RST with an explicit reason code. We could add one - but I would be interested in hearing other opinions.

We could add one that indicates that the RST is upon a response of an
incoming MPTCP-option (e.g., responding to a REMOVE_ADDR, a
MP_FASTCLOSE,...).

Christoph

---

From: Rahul Arvind Jadhav <[email-masked]>
Date: Thu, 2 Aug 2018 00:57:56 +0000

> > > [RJ]: Since a MP_TCPRST should be associated with this RST, what should
> be the corresponding Reason Code that can be used in MP_TCPRST for this
> case? I checked existing reason codes in section 3.6 and none of those made
> sense (may be except for 0x0 ?).
> >
> > That’s an interesting one. There may be some benefit (maybe for
> middleboxes, or maybe for security) to sharing this information in the RST
> with an explicit reason code. We could add one - but I would be interested in
> hearing other opinions.
> 

Some more points in context to MP_TCPRST:
1. There are several cases of MP_TCPRST where there is no mention of Reason Code to be used. Can the Reason Code be explicitly stated for known cases?  Consider foll sections:
	a. Section 3.2. Starting a New Subflow
	"If the token is unknown, or the host wants to refuse
	subflow establishment (for example, due to a limit on the number of
   	subflows it will permit), the receiver will send back a reset (RST)
   	signal, analogous to an unknown port in TCP, containing a MP_TCPRST
   	option (Section 3.6) with an appropriate reason code."

	-----[There is a duplicate sentence in the same section]----

	"If the token received at Host B is unknown or local policy prohibits
   	the acceptance of the new subflow, the recipient MUST respond with a
   	TCP RST for the subflow, with a MP_TCPRST option (Section 3.6) with
   	an appropriate reason code."

	b.  Section 3.3.2.
	"In this case, the MPTCP sender can decide to terminate the subflow 
	that is behaving badly by sending a RST, using an appropriate MP_TCPRST 
	(Section 3.6) error code."

	c. Section 3.3.6.
	"A RST for this purpose SHOULD be accompanied with
	an appropriate MP_TCPRST option"

	d. Section 3.7.
	"This RST SHOULD include the MP_TCPRST option
	 (Section 3.6) with an appropriate reason code."

	e. Section 3.7
	"The data would not be DATA_ACKed
   	(since there is no mapping for the data), and the subflow can be
   	closed with a RST, containing a MP_TCPRST option (Section 3.6) with
   	an appropriate reason code."

2. Can we have an user-defined Reason-Code block for implementations to define their own Reason-code within the block without worrying for future reason-code addons?

---

From: Yoshifumi Nishida <[email-masked]>
Date: Mon, 6 Aug 2018 15:25:29 -0700

On Tue, Jul 31, 2018 at 5:07 AM, Alan Ford <[email-masked]> wrote:

> [RJ] again this is what the text already says, but I’m not still clear
> about this. Why ID + address has to match? Only ID should be enough.
>
> Consider:
>
> What does this mean? Is (c) an implicit REMOVE+ADD? And thus does this
> mean the subflow created in (b) is now no longer valid?
>
> [RJ2] Rather than viewing ( c) as an implicit REMOVE+ADD, can we see it as
> an UPDATE? Because ID remains same. Subflow created in (b) remains valid
> (not sure if I’m missing anything here, I see no reason to lose this
> subflow).
> In step ( c), the receiver can override Address field in context to ID=2
> to the new addr (192.168.0.1) ……   8.9.10.11 is kept separately as the
> learnt actual rcvd IP (from MP_JOIN) in the same context ID=2. We have to
> agree that an implementation needs to maintain this IP separately in
> context to ID=2 anyways.
> So if we do an explicit REMOVE+ADD, the end result is the same as this
> UPDATE (but with much less overhead & impl ease).
>
>
> Given we say that REMOVE_ADDR should not remove a subflow if it’s still
> alive, we have precedent for implying a subflow can exist to an address
> which does not have an Address ID - to - IP address mapping.
>
Hmm.. I am wondering why we want to re-use the same address ID while we
have 0-255 values for it?
Do we have some advantage for it? I thought picking up a new ID would be
safer and simpler..
As Alan mentioned, when we try to "update" the address ID, we will have to
keep both addresses until we confirm that old one is not used anymore.
Also, in the case described above, what will happen if (b) was delayed and
arrived after (c)?

Thanks,
--
Yoshi

---

From: Rahul Arvind Jadhav <[email-masked]>
Date: Wed, 8 Aug 2018 15:20:21 +0000

Please find my response [RJ3] inline.

Best,
Rahul

On 31 Jul 2018, at 01:21, Rahul Arvind Jadhav <[email-masked]<mailto:[email-masked]>> wrote:

Consider:

Hmm.. I am wondering why we want to re-use the same address ID while we have 0-255 values for it?
Do we have some advantage for it? I thought picking up a new ID would be safer and simpler..

[RJ3]: Thanks for covering this angle. Now if we do not re-use the same address ID then the scenarios are different. What happens when REMOVE_ADDR sent for previous ID is ignored because the flow is active on the receiver? The new ADD_ADDR with new ID will result in new ID getting added on the receiver. This requires more state across multiple IDs, and requires more messaging since REMOVE_ADDR has to done. And there is still an uncertainty regarding what happens if the REMOVE_ADDR is ignored because the old flow is active. Thus this is not simpler.
ADD_ADDR doing an update on existing ID is easier to implement and requires less messaging.
There are advantages with an update on existing ID unless other implementers have some other angle to cover.

As Alan mentioned, when we try to "update" the address ID, we will have to keep both addresses until we confirm that old one is not used anymore.
Also, in the case described above, what will happen if (b) was delayed and arrived after (c)?

[RJ3] I’m not sure what is meant by, “confirm that old one is not used anymore”. A receiver anyways is ready to receive an MP_JOIN from source address not advertised in the ADD_ADDR.
Considering update is used,.. in the scenario above, if MP_JOIN (b) arrives after ADD_ADDR update (c) is issued, it would not result in any mishap imo.

---

From: Yoshifumi Nishida <[email-masked]>
Date: Fri, 10 Aug 2018 11:49:37 -0700

Hi Rahul,

Thanks for bringing up an interesting point before we finalize the draft!
I put my comments in lines.

On Wed, Aug 8, 2018 at 8:20 AM, Rahul Arvind Jadhav <[email-masked]
> wrote:

> [RJ3]: Thanks for covering this angle. Now if we do not re-use the same
> address ID then the scenarios are different. What happens when REMOVE_ADDR
> sent for previous ID is ignored because the flow is active on the receiver?
> The new ADD_ADDR with new ID will result in new ID getting added on the
> receiver. This requires more state across multiple IDs, and requires more
> messaging since REMOVE_ADDR has to done. And there is still an uncertainty
> regarding what happens if the REMOVE_ADDR is ignored because the old flow
> is active. Thus this is not simpler.
>
> There are advantages with an update on existing ID unless other
> implementers have some other angle to cover.
>

Yes, if we pick up a new ID, mptcp stack will maintain multiple IDs until
we find the old path is not active anymore. But, I think this is ok as
these are different paths.

Please correct me if I'm wrong, but It seems to me that when we update the
flow, it is expected to inherit the state of the flow in this approach.
But, in this case, I'm not very sure if this is right thing to do. When
address has been changed, I guess the state of the path might be changed as
well..
Also, when we try to update an ID, what if the old flow with the ID is
still active? I think we should not remove it and keep it until closed.

> [RJ3] I’m not sure what is meant by, “confirm that old one is not used
> anymore”. A receiver anyways is ready to receive an MP_JOIN from source
> address not advertised in the ADD_ADDR.
>
> Considering update is used,.. in the scenario above, if MP_JOIN (b)
> arrives after ADD_ADDR update (c) is issued, it would not result in any
> mishap imo.
>

Sorry. I was not clear enough. "confirm" means that checking if the subflow
is already closed or not by following what is described Section 3.4.2. This
may take a certain time as we might need to wait for timeout.

About race condition case, I am thinking about an example like this.
(a) ADD_ADDR comes in saying ID=2, addr=10.0.0.1 (private)
(b) MP_JOIN comes in saying ID=2, addr=8.9.10.11 (public)  (internal
10.0.0.1)
(c) ADD_ADDR comes in saying ID=2, addr=9.10.11.12 (public)

In this case, when a node sent (b), ID=2 is associated with internal
address 10.0.0.1
But, after that, the node learned he got new addr 9.10.11.12 which is
public. So, he decided to replace ID=2 for some reasons. (because the
current ID=2 has significant delay?)
Now, because of significant delay in the ID=2 path, (b) has arrived after
(c). How to handle this kind of case is not very clear to me yet.

Thanks,
--
Yoshi