Subject: RE: [rddp] Adaption Indication for TCP

From: [email-masked]
Date: Tue, 10 Dec 2002 16:12:32 -0700

Caitlin,

D is very useful for the iSCSI case and for any other case where one would like to complete authentication of ones partner and negotiation of parameters that affect the RDMA/DDP connection setup before provisioning the queues for the connection. 

Regards,
Pat

-----Original Message-----
From: Caitlin Bestler [mailto:[email-masked]]
Sent: Monday, December 09, 2002 11:07 AM
To: Williams, Jim; [email-masked]
Subject: Re: [rddp] Adaption Indication for TCP

On 12/9/02, Williams, Jim wrote:

>
>The current thinking is to allow each ULP to negotiate use
>of DDP and start using DDP at an agreed upon point within
>an already established connection.  The advantage of this
>is that it allows pre-existing ULPs to negotiate an
>upgrade to DDP mode without causing any backward
>incompatability. If either end doesn't understand DDP,
>then the negotiation defaults to non-DDP mode with no ill
>effects.  The down side of this is that the negotiation
>must necessarily by ULP specific.  Therefore each ULP must
>solve this problem for itself.
>

There are a few distinct scenarios that need to be
supported. The current approach supports all of them, and
one additional scenario which is nice, but in my opinion not
essential.

A - Pre-designated port / Daemon level:

The ULPs have pre-designated the DDP/RDMAP configuration for
certain ports. Incoming connections for those ports are
routed automatically to a DDP/RDMAP specific connection
accepting daemon (whether hardware and/or software).

Immediate pre-posting of receive buffers may be slightly
tricky here, but there are solutions.

B - LLP Establishment / Instant DDP/RDMAP configuration:

The LLP Connection is established using existing LLP means.
This includes any automatic logging of connections,
validation of remote addresses, etc. (Some hosts may require
these steps, and hence not allow a NIC to accept a
connection).

The ULP having been delivered the LLP connection (by
whatever existing means, including INETD and TCPSERVER)
immediately places it in DDP/RDMAP mode. This is "instant"
but the "reflex" response is being exercised by the ULP
itself, rather than by a transport/DDP level daemon.

C - LLP Establishment / Private Data Exchange before Config:

The LLP is established as above, but each ULP sends a single
message (MSS size or smaller). Each side uses the message
received to determine the DDP/RDMAP configuration.

This is equivalent to InfiniBand's exchange of Private Data.

D - LLP Establishment / Extended Login before Config:

The LLP is established as in scenario B, followed by an
indefinite period of LLP negotiation before entering
DDP/RDMAP mode.

This is the scenario that I do not feel is essential, but
the current draft encompasses it as well. It places some
demands on DDP-only RNICs in terms of their ability to
retroactively interpret material as DDP messages when
notified asynchronously of DDP mode entry.

But this is solveable in pure software, assuming the
existing IP stack can be modified to "give back" either the
Ethernet Frames it was fed or the raw TCP stream.
(Interestingly the local native OS IP stack could end up
becoming one of those dreaded alignment destroying
middleboxes).

I see very high value in each of scenarios A, B and C.
Unlike SCTP, an adaptation indication in TCP cannot be
obtained "for free". It would be an additional message
sent after the three-way handshake had completed.

I know that theoretically data can be sent with the SYN,
but I do not believe you can reliably get that data through
two arbitrary IP stacks all the way to the other ULP.

Given that limiation, I do not think we should add an
additional round-trip to the setup period when it is not
required by the ULP.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Tue, 10 Dec 2002 21:38:31 -0600

On 12/10/02, [email-masked] wrote:

>Caitlin,
>
>D is very useful for the iSCSI case and for any other case
>where one would like to complete authentication of ones
>partner and negotiation of parameters that affect the
>RDMA/DDP connection setup before provisioning the queues
>for the connection. 
>
DAFS also wants a prolonged negotiation before fully
enabling RDMA operations.

However this can be achieved by enabling DDP, but deferring
assignment of a Protection Zone. With no Protection Zone
assigned, no STag could be enabled for the DDP Stream.

That would leave the only issues that had to be negotiated
prior to entering DDP mode being the number of credits and
maximum untagged message size.

To maintain a consistent programming model with IB it would
be very desirable to also negotiate what the Protection Zone
*will* be once authentication is approved.

D is not essential unless there is a need for extended
negotiations *before* it is known whether this will be an
LLP-only or a DDP connection.

There is clearly a need to complete TCP connection
establishment before entering DDP mode. Existing security
policies, firewalls and load balancers don't leave another
option.

I know that some connection-establishers (TCPServer for one)
have the ability to talk over the connection before passing
the approved connection to the consumer. I do not know how
common this is. Out-of-band communications, such as IDENTD
or reverse DNS lookups are far more common, and are the type
of "existing security policy" that I believe needs to be
accomodated.

Is there an example of a complex negotiation that must take
place before basic RDMA/DDP parameters can be known (as
opposed to enabled)?

Caitlin Bestler