Subject: [Ipoverib] ipoib-cm multiple connection model

From: Dror Goldenberg <[email-masked]>
Date: Wed, 19 Oct 2005 12:11:48 +0200

I think that the current model for multiple connections establishment
between 
two peers works well for 1 connection but is broken for more than one
connection.
Because of cases where CM packets are dropped or travel on different VLs,
they 
might get reordered on the way and observed in a different order than were
originally
sent. This breaks the current model which assumes that the REQs are sent and

processed in order. 
 
I think that we need to think of a model which is robust to CM packets
reordering
which can be one of:
- Adding a "channel ID" in the private data. The field goes 0,1,2 and on. It
helps 
   synchronizing both peers on which is the channel at question now. The
rest
   of the decision (per channel) of whether to accept or reject can be the
same 
   as defined in the draft.
- Not supporting multiple connections between the same peers.
 
-Dror
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Vandana Rao <[email-masked]>
Date: Wed, 19 Oct 2005 09:39:31 -0700

Hi Dror,
The purpose of the MAD transaction ID is to allow one to do what you 
say below, i.e. have multiple connection establishment MADs between 2 
peers. It is essentially the same as the "channel ID" that you talk 
about below. Its just that it is a generic channel ID for any MAD 
transactions, not just CM MADs.
I do not believe there is any issue with existing IB CM in this regard.
-Vandana

At 03:11 AM 10/19/2005, Dror Goldenberg wrote:
>I think that the current model for multiple connections establishment between
>two peers works well for 1 connection but is broken for more than 
>one connection.
>Because of cases where CM packets are dropped or travel on different 
>VLs, they
>might get reordered on the way and observed in a different order 
>than were originally
>I think that we need to think of a model which is robust to CM 
>packets reordering
>which can be one of:
>- Adding a "channel ID" in the private data. The field goes 0,1,2 
>and on. It helps
>    synchronizing both peers on which is the channel at question now. The rest
>    of the decision (per channel) of whether to accept or reject can 
> be the same
>    as defined in the draft.

---

From: Bill Strahm <[email-masked]>
Date: Fri, 21 Oct 2005 20:32:56 -0700

Let me try to understand what you are worried about:
I am assuming that each TCP connection (5-tuple) can be on its own CM
connection, and not multiplex a single TCP connection over multiple CM
connections.

Is this your understanding ?

Now if this is what we are talking about - we don't have a problem.  TCP
only guarantees in order delivery to a single connection - if I have
multiple TCP connections to the same hosts, IP packets can be completely
reordered/jumbled by the intermediate network - and all I am guaranteed
is that each TCP stream will have in order delivery relative to itself.

I am not aware of any applications where in order IP packet delivery is
required between multiple streams.  If this is the case - it has to be
handled above TCP anyway, as TCP does not make this guarantee.

Comments ?
Bill

Dror Goldenberg wrote:

> might get reordered on the way and observed in a different order than 
> were originally
> - Adding a "channel ID" in the private data. The field goes 0,1,2 and 
> on. It helps
>    of the decision (per channel) of whether to accept or reject can be 
> the same
>    as defined in the draft.
>------------------------------------------------------------------------
>

---

From: Michael Krause <[email-masked]>
Date: Mon, 24 Oct 2005 09:46:07 -0700

>Comments ?

Your understanding is correct.   IP packets can arrive in any order on any 
interface though all implementations prefer them to arrive close to in 
order and on the same interface (Ethernet aggregation technology attempts 
to maintain the same interface is used consistently).  Now, having multiple 
IB connections between endnode pairs is not an issue - does not matter what 
VL, path, etc. is taken.  However, ideally, the IP datagrams for a given 
connection are injected for a given tuple on the same path consistently as 
this leads to optimal performance - it is not a correctness issue though as 
ordering is the TCP layer's responsibility.

Mike

>>- Adding a "channel ID" in the private data. The field goes 0,1,2 and on. 
>>It helps