Subject: RE: [Ipoverib] A Couple of IPoIB Questions

From: Dror Goldenberg <[email-masked]>
Date: Sat, 20 Nov 2004 18:49:08 +0200

-----Original Message-----
From: Michael Krause [mailto:[email-masked]] 
Sent: Friday, November 19, 2004 3:45 AM
To: Vivek Kashyap
Cc: IPoverIB
Subject: Re: [Ipoverib] A Couple of IPoIB Questions

At 05:14 PM 11/18/2004, Vivek Kashyap wrote:

RC and UC both have benefits. There is almost no difference other than
the connection flag between the two. 

Many host OS implementations do not support UC as RC and UD are all that is
really required within the industry.  The ACK overhead associated with RC is
truly noise and the end-to-end credits are very nice as IB now supports
three signaling rates combined with 4 link widths (though only three are
really being implemented).  Such a permutation in bandwidth capability makes
RC a more tenable / good citizen as we designed it to be so I'd prefer RC. 

[DG] Mike,
 A few reasons I think that the end to end credits / RNR  in an RC
connection is a problem.
It may be worth discussing it:
1) Lack of receive WQEs in the responder implies a slow responder. Getting
the messaged dropped 
    in this case is desirable from protocols that have injection control
such as TCP.  In this case it
    is supposed to back off and restart slowlier. While UC/UD result  in a
similar behavior of messages
    being dropped at the receiver when it's slow, RC does not. Instead,
there is persistence in getting the
    message transmitted and the receiver won't be able to tell the requester
that it's being slow.
2) How would you configure the RNR retry counters. Would they be configured
to infinity ? Doesn't sound
    good. Would they be configured to a finite value (should be <7), in
which case, in the case of a slow
    receiver you'd end up recreating connections that had end to end credits
problem, which is a real 
    overhead on the protocol.
3) What happens with implementations that don't support RNR Nak generation ?
That poses more
    difficulties on (2).

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Michael Krause <[email-masked]>
Date: Mon, 22 Nov 2004 10:36:56 -0800

At 08:49 AM 11/20/2004, Dror Goldenberg wrote:
>Many host OS implementations do not support UC as RC and UD are all that 
>is really required within the industry.  The ACK overhead associated with 
>RC is truly noise and the end-to-end credits are very nice as IB now 
>supports three signaling rates combined with 4 link widths (though only 
>three are really being implemented).  Such a permutation in bandwidth 
>capability makes RC a more tenable / good citizen as we designed it to be 
>so I'd prefer RC.
>
>[DG] Mike,
>     is supposed to back off and restart slowlier. While UC/UD result  in 
> a similar behavior of messages
>     message transmitted and the receiver won't be able to tell the 
> requester that it's being slow.

TCP on the sending side will regulate due to lack of update 
window  credits.   Hence, there is no need to restart the large messages 
that are put forth as the reason for using *C instead of UD.

>2) How would you configure the RNR retry counters. Would they be 
>configured to infinity ? Doesn't sound
>     receiver you'd end up recreating connections that had end to end 
> credits problem, which is a real

RNR would be no different for IP over IB than for any other IB RC instance.

>3) What happens with implementations that don't support RNR Nak generation 
>? That poses more
>     difficulties on (2).

A HCA is required to support RNR NAK.  A TCA has the option.  If you don't 
support RC, then use UD.  Where is the real problem as nothing shown here 
on either side is more than speculation?

Mike