Subject: RE: [rddp] Options and Markers

From: "Barron, Dwight" <[email-masked]>
Date: Tue, 15 Apr 2003 11:12:07 -0500

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Sunday, April 13, 2003 2:52 PM
> To: Barron, Dwight; [email-masked]
> Subject: RE: [rddp] Options and Markers
> 
> 
> Dwight,
> 
> > > I have no problem with robustness considerations, but the 
> vehemence
> > > of opposition to options in any form from a significant portion of
> > > this WG has been among the things that have hindered progress.
> > 
> > I do not believe it is a "vehemence of opposition to options" 
> > per se, but rather a strong desire to see valid engineering 
> > justification for such options. I believe we established, per 
> > your comments at the last F2F, that receive side copying is 
> > not likely to be avoided, regardless of marker interval and 
> > we went on to focus on transmit side impact of marker 
> > interval options.
> 
> The discussion in San Francisco concerned a receive side full-software
> implementation.  It did not cover a "full-offload with full 
> TCP buffering"
> class of implementation, as the discussion would up considering only
> full-software and "flow-through" RNIC implementations.  
> There's a third
> class that is relevant.

I haven't heard anyone assert that this relevant third class implementation does not fall into one of the two previously mentioned catagories. I would suspect, given the embedded nature of such designs, that they have relatively unencumbered access to a NIC with scatter/gather capability. 

> 
> [... material on full-software implementations snipped ... ]
> 
> > > I think some form of option is going to be hard to avoid for
> > > markers, as there is "running code" out there for a class of
> > > implementations (RDDP + TCP with full buffering offloaded to
> > > a souped up NIC) that do not need markers.  Requiring that this
> > > class of implementation use markers solely to avoid putting an
> > > option in the protocol appears to be very hard to justify.
> > 
> > How does this argument not cross the line from what I would 
> > consider debate in search of best architectural practice for 
> > RDDP to that of defending an existing RDDP implementation of 
> > "running code". What RDDP specification was this running code 
> > conformant to such that it should impact the design of future 
> > RDDP specifications? 
> 
> The assertion that there is a single "best architectural practice"
> for RDDP and it is a "flow-through RNIC" with minimal buffering
> is part of the problem.  

I'm sorry, but I disagree. This is the Remote Direct Data Placement WG, not the Remote Store and Forward WG. From the WG Description at https://www1.ietf.org/html.charters/rddp-charter.html ..RDDP capabilities refer to the ability to place data directly from the NIC into application buffers, without intensive CPU usage. This strategy avoids the costs of data copying and enables using IP as a method for high speed buffer to buffer transfer....
I have no quarrel with anyone that wants to use assembly buffers in their design, every design will need some amount of buffering. I do take exception that we mandate an option that essentially diminishes or precludes a flow-through design for those of us that are taking the *Direct* part seriously. 

> The implementation I referred to above
> is based on the VI architecture and runs it over TCP/IP with
> all RDMA processing done outboard.  AFAIK, the protocol used
> is specific to that implementation; among the reasons for this is
> that VI is an interface, not a protocol.
> The reason that this discussion doesn't cross the line, is that
> the "running code" serves to demonstrate the viability of the
> third class of implementations, speaking to your above desire for
> "valid engineering justification" for protocol design concerns
> raised by that class.  While I can't speak for the implementers,
> I think they plan to update the implementation in question to
> whatever specifications the RDDP WG adopts.  Given the viability
> of that class of implementation (FWIW, it was extensively discussed
> prior to formation of the RDDP WG), design decisions which exclude
> that class will tend to run afoul of the admonition not to
> impose requirements on implementors that are not required for
> interoperability (see Section 6 of RFC 2119).

I cannot find said product available for sale today, I have been told it is withdrawn from the market and it is not on the vendor's website (if this comment seems untowardly callous, the same is true for my own company's VI product). That said, I have not heard that the product is incapable of supporting markers. Given the general catagory of a dedicated CPU and souped up NIC capable of being upgraded by firmware, does that product become (more) commercially unviable due to the overhead of inserting markers? As for excluding a class as a whole, I think we are down to quibbling whether an implementation should use an XXX or YYY MHz embedded processor to hit ZZZ% of wire speed. 

> 
> Thanks,
> --David
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> [email-masked]        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> 
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Somesh Gupta" <[email-masked]>
Date: Tue, 15 Apr 2003 10:40:21 -0700

Dwight,

My reading of the charter of the workgroup is that
the rddp protocol address the problem of data copying
in the host (from the NIC into application buffers).

The charter does not indicate how the NIC is implemented.

When the Work Group was being chartered, there was a
fairly big debate about whether there should even be
a TCP mapping, since TCP would always require some
buffering or roto-tilling of TCP (the argument is over
how much buffering). Those of us who wanted
a mapping over TCP pushed for it on the ground that
we could achieve the mapping without trying to create a
different version of TCP - and it did not matter from
the perspective of the problem being solved what we did in
the NIC.

Maybe I was being overly optimistic at that time, not
in any way about the engineering challenges, but about
the process. Seems like there is a strong desire to get
the legitimacy provided by using the TCP protocol, but
at the same time by significant modifications to it
(or by creating a new supertransport to overcome the
limitations of TCP).

Regards,
Somesh

> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Barron, Dwight
> Sent: Tuesday, April 15, 2003 9:12 AM
> To: [email-masked]; [email-masked]
> I haven't heard anyone assert that this relevant third class 
> implementation does not fall into one of the two previously 
> mentioned catagories. I would suspect, given the embedded nature 
> of such designs, that they have relatively unencumbered access to 
> a NIC with scatter/gather capability. 
> 
> > 
> I'm sorry, but I disagree. This is the Remote Direct Data 
> Placement WG, not the Remote Store and Forward WG. From the WG 
> Description at 
> https://www1.ietf.org/html.charters/rddp-charter.html ..RDDP 
> capabilities refer to the ability to place data directly from the 
> NIC into application buffers, without intensive CPU usage. This 
> strategy avoids the costs of data copying and enables using IP as 
> a method for high speed buffer to buffer transfer....
> I have no quarrel with anyone that wants to use assembly buffers 
> in their design, every design will need some amount of buffering. 
> I do take exception that we mandate an option that essentially 
> diminishes or precludes a flow-through design for those of us 
> that are taking the *Direct* part seriously. 
> 
> I cannot find said product available for sale today, I have been 
> told it is withdrawn from the market and it is not on the 
> vendor's website (if this comment seems untowardly callous, the 
> same is true for my own company's VI product). That said, I have 
> not heard that the product is incapable of supporting markers. 
> Given the general catagory of a dedicated CPU and souped up NIC 
> capable of being upgraded by firmware, does that product become 
> (more) commercially unviable due to the overhead of inserting 
> markers? As for excluding a class as a whole, I think we are down 
> to quibbling whether an implementation should use an XXX or YYY 
> MHz embedded processor to hit ZZZ% of wire speed. 
> 
> > 
> > Thanks,
> > --David