Subject: [rddp] Adaption Indication for TCP

From: "Williams, Jim" <[email-masked]>
Date: Mon, 9 Dec 2002 11:58:50 -0500

David Black wrote:
> (4) Assurance of no confusion about presence of framing.
> 
> The "gold standard" here is the SCTP adaption indication -
> if the two ends of the connection don't agree on the usage
> of DDP or RDMA, the result is no connection.  At the moment,
> MPA depends on the ULP to get this right, including when to
> turn MPA on after a connection has been established, so every
> time a new ULP is developed on MPA, there's a new opportunity
> to get this wrong.  Asserting that MPA is a ULP to TCP (and hence
> nothing needs to be done here) is unlikely to fly unless one
> is prepared to abandon the current link between markers and
> PDU alignment at TCP senders.

Personally I find the idea of depending on each ULP to get
adoption negotiation right less troubling that relying on each
ULP to get the buffer ownership model correct.  However I will
save that for a separate thread.

The current thinking is to allow each ULP to negotiate use
of DDP and start using DDP at an agreed upon point within
an already established connection.  The advantage of this
is that it allows pre-existing ULPs to negotiate an upgrade
to DDP mode without causing any backward incompatability.
If either end doesn't understand DDP, then the negotiation
defaults to non-DDP mode with no ill effects.  The down
side of this is that the negotiation must necessarily
by ULP specific.  Therefore each ULP must solve this problem
for itself.

If the above is unacceptable, as it appears it may be, then
there aren't a lot of alternatives left.  I would bring
up the following possibility as a discussion point.

All RDDP connections always use a well known port on the server
side, and all such connections are in DDP mode at creation.
Since the port number is no longer used as a ULP indication,
a login phase is required for DDP.  This login would exchange
information required to define the RDMA/DDP connection points
(i.e. which ULP), and define whether in RDMA mode or not.
This model follows closely the model used in "VI" where 
discriminators are exchanged end to end before the VI connection
is actually made.  I will leave it to others to comment
on how well this matches the connection models used
by Infiniband, DAT, etc.

This model may also be helpful to implementations that wish
to accelerate DDP mode connection on a NIC, but defer
non-DDP connections to the host based stack.  Since the
DDP-ness of the connection is explicit on the initial 
SYN segment (via the port number)  there is no ambiguity
as to where the connection should reside, and no need
to migrate an already established connection to or from 
the host stack.

This means that if the client side wants
to use DDP and the server doesn't support it, the client
must first establish or try to establish a connection the
DDP port, and then establish another connection to the
normal ULP port when the first fails.

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 9 Dec 2002 13:07:24 -0600

On 12/9/02, Williams, Jim wrote:

>is that it allows pre-existing ULPs to negotiate an
>upgrade to DDP mode without causing any backward
>incompatability. If either end doesn't understand DDP,
>then the negotiation defaults to non-DDP mode with no ill
>effects.  The down side of this is that the negotiation
>must necessarily by ULP specific.  Therefore each ULP must
>solve this problem for itself.
>

There are a few distinct scenarios that need to be
supported. The current approach supports all of them, and
one additional scenario which is nice, but in my opinion not
essential.

A - Pre-designated port / Daemon level:

The ULPs have pre-designated the DDP/RDMAP configuration for
certain ports. Incoming connections for those ports are
routed automatically to a DDP/RDMAP specific connection
accepting daemon (whether hardware and/or software).

Immediate pre-posting of receive buffers may be slightly
tricky here, but there are solutions.

B - LLP Establishment / Instant DDP/RDMAP configuration:

The LLP Connection is established using existing LLP means.
This includes any automatic logging of connections,
validation of remote addresses, etc. (Some hosts may require
these steps, and hence not allow a NIC to accept a
connection).

The ULP having been delivered the LLP connection (by
whatever existing means, including INETD and TCPSERVER)
immediately places it in DDP/RDMAP mode. This is "instant"
but the "reflex" response is being exercised by the ULP
itself, rather than by a transport/DDP level daemon.

C - LLP Establishment / Private Data Exchange before Config:

The LLP is established as above, but each ULP sends a single
message (MSS size or smaller). Each side uses the message
received to determine the DDP/RDMAP configuration.

This is equivalent to InfiniBand's exchange of Private Data.

D - LLP Establishment / Extended Login before Config:

The LLP is established as in scenario B, followed by an
indefinite period of LLP negotiation before entering
DDP/RDMAP mode.

This is the scenario that I do not feel is essential, but
the current draft encompasses it as well. It places some
demands on DDP-only RNICs in terms of their ability to
retroactively interpret material as DDP messages when
notified asynchronously of DDP mode entry.

But this is solveable in pure software, assuming the
existing IP stack can be modified to "give back" either the
Ethernet Frames it was fed or the raw TCP stream.
(Interestingly the local native OS IP stack could end up
becoming one of those dreaded alignment destroying
middleboxes).

I see very high value in each of scenarios A, B and C.
Unlike SCTP, an adaptation indication in TCP cannot be
obtained "for free". It would be an additional message
sent after the three-way handshake had completed.

I know that theoretically data can be sent with the SYN,
but I do not believe you can reliably get that data through
two arbitrary IP stacks all the way to the other ULP.

Given that limiation, I do not think we should add an
additional round-trip to the setup period when it is not
required by the ULP.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp