Subject: RE: [rddp] DDP: Ordering Among DDP Messages (7.3) Data Source rules

From: "Culley, Paul" <[email-masked]>
Date: Wed, 6 Nov 2002 08:14:55 -0600

The authors believe that implementations can get some significant optimizations at the receiver if the segments arrive (mostly) in order.  So that's the reason for adding an increasing MO/TO order rule to the sender.  The idea is that certain types of context caching operations at the receiver will perform better if the segments are in sequentially increasing order.

As you say, the receivers must handle out of order segments, but that may take a bit more time, context lookup bandwidth etc.  Since it was not demonstrated that a sender could really get any advantage when filling reasonable sized (1500 byte or so) segments (which would occur when sending messages large enough to require multiple segments), the authors went in favor of the receiver.

We must make sure that "extra" error checking is not allowed at receivers, since that would not work when naturally occurring out of order segments arrived...

Paul Culley
HP Sr. Fellow
281-514-5543

-----Original Message-----
From: Caitlin Bestler [mailto:[email-masked]]
Sent: Tuesday, November 05, 2002 1:33 PM
To: [email-masked]
Subject: [rddp] DDP: Ordering Among DDP Messages (7.3) Data Source rules

The following rule was added:

> At the Data Source, DDP:
>
> ...
> 
> * SHOULD transmit DDP Segments within a DDP Message in
> increasing MO order for Untagged DDP Messages and in
> increasing TO order for Tagged DDP Messages.

I believe this topic was thoroughly discussed, and that was
NOT the consensus. While it may seem innocuous, I do not
believe that a SHOULD clause is appropriate when there are
known valid reasons for at least some implementations not to
do so.

Further, in this case, there is absolutely no benefit to
this clause. It will only encourage sloppy implementations
to add extraneous "error checking" code looking for
"incorrectly" sequences segments.

As previously covered, a simple compliant receiver will not
even *notice* when segments are not in a natural order. It
has no *need* to examine their relative offsets at all.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 6 Nov 2002 10:45:40 -0600

On 11/6/02, Culley, Paul wrote:

>The authors believe that implementations can get some
>significant optimizations at the receiver if the segments
>arrive (mostly) in order.  So that's the reason for adding
>an increasing MO/TO order rule to the sender.  The idea is
>that certain types of context caching operations at the
>receiver will perform better if the segments are in
>sequentially increasing order.
>

What are those optimizations?

An additional requirement should be justified by algorithms
presented in public *before* a requirement is added.

As for caching of the receive buffer. The receiver can
easily just assume that the entire untagged buffer will be
updated. And flush it after it is complete.

Unless you are optimizing for extremely large untagged
buffers, I do not see any benefit. And optimizign for large
untagged buffers would be a mistake, untagged buffers SHOULD
be small, we should not complicate the specification for the
benefit of a practice that is contrary to the intent of the
entire RDMA model.

>As you say, the receivers must handle out of order
>segments, but that may take a bit more time, context
>lookup bandwidth etc.  Since it was not demonstrated that
>a sender could really get any advantage when filling
>reasonable sized (1500 byte or so) segments (which would
>occur when sending messages large enough to require
>multiple segments), the authors went in favor of the
>receiver.
>

This is based upon the assumption that the source of a Send
is a contiguous in-memory buffer. If it comes from a
scatter-gather list, or is a virtual constructor (a "NIC
Side Include"), then the segments can easily be "available"
at different times. Hence sending them "out of order" can
make considerable sense.

Further, the bulk of the data transfers are tagged, for
which the receiver can make no orderign assumptions at all.

Again, I do not believe this requirement has been justified
by any publicly presented algorithm or scenario. It should
not be added.

Caitlin Bestler