Subject: [rddp] Options and Markers

From: [email-masked]
Date: Fri, 11 Apr 2003 13:40:40 -0400

Pulling another topic out of Paul Culley's post:

> In my mind "enabling" these scenarios, while important, should not mean
> that we go to great lengths to optimize or reduce overhead on the
> software side at the expense of making the overall protocol less robust,
> or adding multiple options that applications will have to sort through
> and all concerned will have to test.

I have no problem with robustness considerations, but the vehemence
of opposition to options in any form from a significant portion of
this WG has been among the things that have hindered progress.

I think some form of option is going to be hard to avoid for
markers, as there is "running code" out there for a class of
implementations (RDDP + TCP with full buffering offloaded to
a souped up NIC) that do not need markers.  Requiring that this
class of implementation use markers solely to avoid putting an
option in the protocol appears to be very hard to justify.

One alternate possibility would be to make marker usage optional,
but put control of this "option" entirely in the hands of
the receiver - senders MUST implement both markers and not-markers,
receivers MUST implement one OR the other, and each receiver
gets to pick which one is used as the benefits of markers are
primarily on the receiver side.

Comments?

Thanks,
--David

----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 11 Apr 2003 16:20:43 -0500

On 4/11/03, [email-masked] wrote:

>
>One alternate possibility would be to make marker usage
>optional, but put control of this "option" entirely in the
>hands of the receiver - senders MUST implement both
>markers and not-markers, receivers MUST implement one OR
>the other, and each receiver gets to pick which one is
>used as the benefits of markers are primarily on the
>receiver side.
>
>Comments?
>

A very solid suggestion.

Omitting markers simplifies packet handling, improves
network utilization slightly, but at the cost of increasing
the receivers buffering for out-of-order segments.

A receiver that knew it had sufficient buffering would have
every reason to request that it not be sent markers. As
pointed out during the prior discussions on this issue, once
you had an external memory device you might as well add a
lot. Low memory RNICs are most likely those that are trying
to work without an external memory device.

There is one gotcha that would have to be worked out.
All of the other options discussed involve a sender
describing how they send. That makes it compatible with
a single initialization FULPDU that proceeds the data
FULPDUs. There is no need for special marking or for an
extra round trip.

With this option, however, the receiver must know *when* its
partner received its configuration. That means either one of
two things:

-- The establishment of "FULPDU" types, analogous to SCTP
   chunk types, so that configuration changes can be
   acknowledged even after data chunks have been sent.
-- Requiring a round-trip so that each side can confirm
   its modes *after* hearing what the other side wants.
   
In either case, this could be optimized for the case where
one side believes it knows what the other side wants.
The "init" chunk would include not only the sender's
marker mode, but what it believes the other side's is.

When an "init" chunk is received that correctly predicts
the receivers marker mode, no further action is required.
The connection can proceed to "data" mode immediately.
Otherwise a "correction" chunk must be sent.

Similiarly, the receiver can predict whether the other
side would have to send a "data" chunk.

When an incorrect prediction is made to an unknown host,
a round trip is still required. Unless there is a convention
for public servers, this could be a problem (an example of a
connection would be providing a mode hint in the URL).

The alternate would require an "escape" FULPDU. Such a frame
could be designed. The best "escape" protocol I can think of
is that control chunks could be placed immediately after a
zero length FULPDU.

Even so, changing modes would be tricky, because the change
would have to be negotiated without any data in flight. The
benefit is that a receiver could take the first data chunk
in the "wrong" mode, and then decide if it was worth going
idle in order to change.

A third option would be to explicitly allow receivers to
mark themselves as being "sender compliant". This might be
the best solution for public servers that deal with a large
number of intermittent clients, especially for short lived
connections (i.e., an RDMA optimized web server).

BTW, I still dislike options. Every option has a permanent
and ongoing cost in QA testing.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp