Subject: RE: [rddp] MPA issue (3): Rejected Connection bit - Rough Consensus

From: "Carrier, John" <[email-masked]>
Date: Mon, 19 Apr 2004 12:26:05 -0700

Hi Tom,
 
If by 'idle' mode, you mean that MPA is no longer consuming the TCP stream (what I think I called 'inactive' mode), then I agree.
 
If we specify a return to this 'idle' mode, as Tom suggests, then after MPA sends the reply frame with the rejected bit set, the next step is under ULP control.  MPA only actively terminates the connection for an invalid Request or Reply frame.  MPA has no other active role in connection termination (eg, once in full MPA mode, if there are errors in the FPDUs, MPA reports the error to DDP and DDP is responsible for closing the connection).
 
During connection startup, after MPA has sent the Reply frame with the rejected bit set, the ULP at the MPA Responder can do one of the following:
 
* keep the connection in TCP only mode, 
* terminate the connection, or
* re-start MPA mode (ie enable MPA to receive another Request frame)
 
To keep the two half-connections symmetrical, the MPA Initiator should also transition to 'idle' mode after it has received a Reply frame with the rejected bit set.  The ULP at the initiator can then do one of the following:
 
* keep the connection in TCP only mode
* terminate the connection, or
* re-start MPA mode by sending another MPA Request frame
 
I don't have a usage model in mind, so does this make sense?
 
--jc

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of Talpey, Thomas
Sent: Thursday, April 15, 2004 7:07 AM
To: Caitlin Bestler
Cc: [email-masked]; [email-masked]
Subject: Re: [rddp] MPA issue (3): Rejected Connection bit - Rough Consensus

At 01:31 AM 4/15/2004, Caitlin Bestler wrote: 
>However, I believe it has also been shown that the option to return 
>the TCP connection to the ULP is mostly a theoretical nicety. There 
>is no identified *need* for such a feature. So if anyone put forward 
>a scenario where it was difficult to return the TCP connection then 
>it would probably be agreed to allow the MPA layer to simply 
>terminate the connection after the Reset bit. 

The NFS/RDMA mode of the NFSv4/Session protocol supports a mode 
where RDMA mode is offered by the client via NFS/TCP exchange, but 
may be declined by the server (for example, inability to support 
requested RDMA parameters). Currently, this exchange is handled 
in TCP mode, but we are examining options which may lead us to 
also perform it via MPA, using private data and reject. In this case, 
leaving the session in TCP mode will enhance availability and 
robustness. 

While we have no current implementation with this requirement, i.e. 
we have no proof it is compelling, it is trivially easy for the RDMA 
provider. Therefore I argue for inclusion of the requirement that the 
MPA layer return to idle mode. 

Tom.

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 19 Apr 2004 22:47:38 +0300

On Apr 19, 2004, at 10:26 PM, Carrier, John wrote:

>
>  
> If by 'idle' mode, you mean that MPA is no longer consuming the
> TCP stream (what I think I called 'inactive' mode), then I agree.
>  
> If we specify a return to this 'idle' mode, as Tom suggests, then
> after MPA sends the reply frame with the rejected bit set, the
> next step is under ULP control.  MPA only actively terminates the
> connection for an invalid Request or Reply frame.  MPA has no other
> active role in connection termination (eg, once in full MPA mode,
> if there are errors in the FPDUs, MPA reports the error to DDP and
> DDP is responsible for closing the connection).
>
>  
> During connection startup, after MPA has sent the Reply frame with
> the rejected bit set, the ULP at the MPA Responder can do one of
> the following:
>  
> To keep the two half-connections symmetrical, the MPA Initiator should
> also transition to 'idle' mode after it has received a Reply frame with
> the rejected bit set.  The ULP at the initiator can then do one of the
> following:
>  

If you add the caveat that the ULP may (and quite frequently will) 
exercise the
local interface to instruct the MPA mode to terminate the TCP 
connection immediately
if MPA mode cannot be entered.

It is a purely local matter, but it may save a user/kernel round-trip, 
so we don't want
to imply that the local interface should avoid offering such an option.

"RDMA or bust" protocols would use this. "Optional RDMA" protocols 
currently
negotiate RDMA in pure streaming mode, and then hence are "RDMA or bust"
once they make that decision. Tom's earlier post essentially said that 
had a back-out
from RDMA entry been present in the MPA Private Data exchange from the 
very
beginning, that the NFS over RDMA draft might well have made use of it 
--
and presumably future ULPs could find such a capability useful.