Subject: Re: [Ipoverib] ipoib-cm nits/clarifications

From: "H.K. Jerry Chu" <[email-masked]>
Date: Wed, 19 Oct 2005 09:56:46 -0700 (PDT)

>- page 10, section 6.0 - what is the final MTU of a connection ? is it the
>minimum between
>   the two "receive MTU" , or is it hybrid - different MTU per half
>connection ?

Although there is really no reason to demand for a single MTU value
per connection as long as a sender never sends a message large than
the "Receive MTU" of the receiver, I'd expect most implementations
would want to choose one value for both directions for simplicity.
For that I'd suggest to take the min as the final MTU for both directions
so that the side that advertises a 32KB MTU doesn't need to post
32KB receive buffers and waste memory if the other side will never send
msgs larger than 16KB, for example.

>I think that the current model for multiple connections establishment
>between 
>two peers works well for 1 connection but is broken for more than one
>connection.
>Because of cases where CM packets are dropped or travel on different VLs,
>they 
>might get reordered on the way and observed in a different order than were
>originally
>sent. This breaks the current model which assumes that the REQs are sent and
>processed in order.

Isn't this a problem at the IB layer already, or this problem is introduced
by the 20byte comparison for the REQ crossing case?

I wish the connection setup can be kept as simple as possible, e.g.,
active-active as you've suggested. But that requires a common service-ID,
and the logical choice of using the P_Key takes away the flexbility of
allowing multiple IP subnets/IPoIB-UD QPs to occupy the same partition...

Jerry

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Dror Goldenberg <[email-masked]>
Date: Thu, 20 Oct 2005 09:37:14 +0200

> From: H.K. Jerry Chu [mailto:[email-masked]] 
> Sent: Wednesday, October 19, 2005 6:57 PM
> 
> >- page 10, section 6.0 - what is the final MTU of a 
> connection ? is it 
> >the minimum between
> Although there is really no reason to demand for a single MTU 
> value per connection as long as a sender never sends a 
> message large than the "Receive MTU" of the receiver, I'd 
> expect most implementations would want to choose one value 
> for both directions for simplicity. For that I'd suggest to 
> take the min as the final MTU for both directions so that the 
> side that advertises a 32KB MTU doesn't need to post 32KB 
> receive buffers and waste memory if the other side will never 
> send msgs larger than 16KB, for example.

I agree. 

> >between two peers works well for 1 connection but is broken for more 
> >than one connection.
> Isn't this a problem at the IB layer already, or this problem 
> is introduced by the 20byte comparison for the REQ crossing case?

I think that it's a new problem because we're not letting the CM 
perform Active/Active on behalf of the ULP. However I think that the 
fundamental issue here is also that we're trying to use both
Active/Active and Active/Passive semantics on the same "service" while 
expecting to end up with more than one simultaneous connection.
Take a look at the pdf I've sent in the other mail, it tries to illustrate
the 
problem.

-Dror

---

From: Dror Goldenberg <[email-masked]>
Date: Thu, 20 Oct 2005 09:39:26 +0200

> From: Vivek Kashyap [mailto:[email-masked]] 
> Sent: Wednesday, October 19, 2005 7:21 PM

> The receive MTU gives you the peer's minimum value. An 
> implementation can work this way. However, it might be useful 
> to state it as the minimum of the two in the draft..since an 
> implementation is likely to use the 
> the same size buffers when sending or receiving. That is if A 
> advt. recv of 
> 32K it will very likely send packets which are only 32K even 
> if the peer 
> send a receive MTU of 64K.
> 
> thoughts?

Sounds good to me.