Subject: Re: [Ipoverib] ipoib-cm multiple connection model

From: "H.K. Jerry Chu" <[email-masked]>
Date: Mon, 24 Oct 2005 10:40:36 -0700 (PDT)

I think Dror and Vandana are talking about connection "establishment",
not how to use multiple connections after they have been established.
It looks like there are some robustness issues wrt the handshake for
setting up multiple IB connections.

But I may be wrong of course.

Jerry

>At 08:32 PM 10/21/2005, Bill Strahm wrote:
>>Let me try to understand what you are worried about:
>>I am assuming that each TCP connection (5-tuple) can be on its own CM
>>connection, and not multiplex a single TCP connection over multiple CM
>>connections.
>>
>>Is this your understanding ?
>>
>>Now if this is what we are talking about - we don't have a problem.  TCP
>>only guarantees in order delivery to a single connection - if I have
>>multiple TCP connections to the same hosts, IP packets can be completely
>>reordered/jumbled by the intermediate network - and all I am guaranteed
>>is that each TCP stream will have in order delivery relative to itself.
>>
>>I am not aware of any applications where in order IP packet delivery is
>>required between multiple streams.  If this is the case - it has to be
>>handled above TCP anyway, as TCP does not make this guarantee.
>
>>Comments ?
>
>Your understanding is correct.   IP packets can arrive in any order on any 
>interface though all implementations prefer them to arrive close to in 
>order and on the same interface (Ethernet aggregation technology attempts 
>to maintain the same interface is used consistently).  Now, having multiple 
>IB connections between endnode pairs is not an issue - does not matter what 
>VL, path, etc. is taken.  However, ideally, the IP datagrams for a given 
>connection are injected for a given tuple on the same path consistently as 
>this leads to optimal performance - it is not a correctness issue though as 
>ordering is the TCP layer's responsibility.
>
>Mike
>
>>Bill
>>
>>Dror Goldenberg wrote:
>>
>>>I think that the current model for multiple connections establishment between
>>>two peers works well for 1 connection but is broken for more than one 
>>>connection.
>>>Because of cases where CM packets are dropped or travel on different VLs, 
>>>they
>>>might get reordered on the way and observed in a different order than 
>>>were originally
>>>sent. This breaks the current model which assumes that the REQs are sent and
>>>processed in order.
>>>
>>>I think that we need to think of a model which is robust to CM packets 
>>>reordering
>>>which can be one of:
>>>- Adding a "channel ID" in the private data. The field goes 0,1,2 and on. 
>>>It helps
>>>    synchronizing both peers on which is the channel at question now. The 
>>> rest
>>>    of the decision (per channel) of whether to accept or reject can be 
>>> the same
>>>    as defined in the draft.
>>>- Not supporting multiple connections between the same peers.
>>>
>>>-Dror
>>>
>>>------------------------------------------------------------------------
>>>
>>>_______________________________________________
>>>IPoverIB mailing list
>>>[email-masked]
>>>https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>
>>
>>
>>_______________________________________________
>>IPoverIB mailing list
>>[email-masked]
>>https://www1.ietf.org/mailman/listinfo/ipoverib
>

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Vandana Rao <[email-masked]>
Date: Tue, 25 Oct 2005 11:18:05 -0700

Yes, Jerry is right. This does not have to do with how TCP 
connections are mapped to IB connections but whether there is an 
issue with  multiple active-active IB connections between 2 peers. I 
believe there is none.
-Vandana