Subject: RE: [rddp] A different thought on CRCs

From: "Mukund, Shridhar" <[email-masked]>
Date: Mon, 16 Dec 2002 18:30:59 -0800

Caitlin,

> If you have a scenario where DDP would be deployed and
> alignment destroying middle boxes would be common, whether
> immediate or delayed, then I could see adding the second
> CRC. Otherwise, why not stick with what is already proposed
> and *is* the best solution for virtually every envisioned
> deployment?

Well, there you go. Other than the fact that the 1CRC 
argument is really a link-level digest(implementation wise) 
pretending to be a DDP+(MPA) level digest, it is further
predicated on the *alignment* requirement!

Do you really buy the alignment argument? I may not have a
problem with *some implementations* choosing to optimize 
around that assumption for *some specific* applications. 
But to base an IETF RDMA-over-TCP protocol spec. around 
that assumption is preposterous.

I agree with the arguments made on the iSCSI reflector and
here that synchronization(markers) are required to assuage
buffering requirements resulting from *long* fat pipes(LFN).
The argument " ... but the world is going to be *aligned* for
data center(non LFN) application" is an oxymoron. :-)

What is the RDMA/iWARP vision? Pl. educate me ...

A. Granted, low-latency data center application is one of 
   them. The need for synchronization(markers) is probably 
   the least for this application - given that such pipes
   will be *short* and sufficiently over-provisioned.

B. Storage better be the other compelling ap. for RDMA. Are
   we saying that storage aps. such as iSCSI when layered
   over RDMA have to depend on alignment? There goes the 
   ubiquitous ness of iSCSI down the drain.

C. How does one expect to encourage ap. developers to develop
   new aps. if RDMA can not be made to work with existing TCP
   stacks? All one needs for a lot of these aps. is the 
   steering header part of DDP and integrity. We need to 
   grow the market first before selling h/w RNICs :-)

D. Innovative networked games, streaming aps and what not
   over the ubiquitous TCP/IP network - in time, of course.

E. Are we saying that it is REQUIRED to replace the TCP 
   send s/w stack because the receiver stack is optimized 
   *sharply* for alignment? Why did we not think of that 
   in the iSCSI context? 

F. Are the middle boxes in the network going to decrease or
   increase over time? What is the life expectancy of 
   RDMAoverTCP? I hope it is intended to survive as long as
   TCP, which to me could very well be eternity (for good
   reasons). Although this is a debatable topic in itself,
   I believe that we may see a combination of rate based
   flow control for streaming applications and hop-to-hop
   flow control(different MSS between hops?) for bursty aps. 
   Do we want to depend on assumptions around MSS, alignment, 
   send-segmentation that are *tightly* coupled with the 
   closed-loop congestion control function? For most part, 
   TCP is an encapsulation of the congestion control 
   function and is evolutionary-by-design for good reasons.
   ( Oops, I was hoping to stay away from this one :-) )

G. ...

So I am asking that we do not optimize the protocol so very
*sharply* for the *alignment* case.

-Shridhar Mukund

 
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 16 Dec 2002 23:55:38 -0600

> that assumption is preposterous.
Even looking ahead, I see non-allignment to be an exception.
One that must be dealt with for completeness, but not an issue
that will materially affect performance.

This is based upon two assumptions. If either is false, or even
suspect, I would fully support using split CRCs (because unlike
some of the critics, I agree that it is *very* feasible).

Those assumptions:

-   Most DDP senders will send originate the data aligned. It is
    easily implemented in any custom stack and/or RNIC. The only
    exceptions will be user-mode code written on top of pre-existing
    TCP/IP stacks. These will mostly be clients. The fact that the
    TCP/IP stack has not been modified to support alignment would
    be indicative that *most* of the traffic for this host in *not*
    DDP enabled. If there is a substantial amount of DDP traffic
    the DDP/TCP interface will be optimized and/or an RNIC will
    be installed.
    
-   Terminating and re-originating a TCP stream is work. Doing it
    for an unknown application will not be done routinely. It will
    happen, but the only common reason for doing so will be
    to gateway between IP networks with radically different
    communications characteristics (round trip time, MTU,
    latency or some combination thereof). In other words,
    re-originating an unknown TCP stream is most likely when
    normal end-to-end flow control would get confused.
    
    Most connections will *not* cross such a barrier, although
    some clients will cross it virtually all of the time.
    
    No server would place itself on the other side of such
    a barrier from any substantial portion of its clientele.
    
The most common meddlesome middlebox once we break out of the pure
data-center environment will be a firewall/NAT. The IP and TCP headers
get munged, but IP datagrams are passed through on a one-to-one basis.
Alignment will not be broken (applications that pass IP addresses in
the payload, are a different but very well known problem).

If I were making a data-center *only* assumption, I would be totally
opposed to markers. In a data-center-only deployment there is no
justification for using TCP rather than SCTP. A data center has
no excuse for not fixing middleboxes without SCTP support.

It is the "second generation" of RDMA applications that justify the
use of TCP: those where home clients access content from remote servers
(or use remote storage services). While these will be slow to develop,
there can be major benefits of RDMA for these applications. Having worked
consumer electronics, I am quite confident in predicting that many of the
SCTP unfriendly middleboxes that are in place today will still be there
in ten years. Lack of OS support doesn't strike me as much of issue, by
the time the applications are ready to RDMA, SCTP will be supported in
all OSs. A turnkey box that already does all that is required - those
are left in place until they stop working. Which can be a very long time.

---

From: Michael Krause <[email-masked]>
Date: Tue, 17 Dec 2002 15:04:54 -0800

>no excuse for not fixing middleboxes without SCTP support.

Economics play a major role in any decision so there are plenty of reasons 
why this isn't such a slam dunk decision.

>in ten years.

True for data centers - existing infrastructure has a level of inertia no 
matter the deployment.

>Lack of OS support doesn't strike me as much of issue, by

The issue for SCTP will be RNIC development not software stack 
implementation availability.

Mike