Subject: [rddp] RE: Marker Progress?

From: "Barron, Dwight" <[email-masked]>
Date: Wed, 16 Apr 2003 16:59:18 -0500

David, how does this proposal relate to previous discussions about configurable marker intervals? Would the ability for the receiver to disable markers at the transmitter be sufficient to mitigate the need for additional marker interval options? 

Regards,
Dwight

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, April 16, 2003 2:28 PM
> To: Barron, Dwight; [email-masked]
> Subject: Marker Progress?
> 
> 
> > > > I do take exception that we mandate an option that essentially
> > > > diminishes or precludes a flow-through design
> > >
> > > Could we see "valid engineering justification" for the 
> latter statement,
> > > especially the "precludes" assertion?  Keep in mind that 
> the "option"
> > > under discussion allows a receiver who benefits from markers 
> > > to require their use.
> > 
> > You are correct, the "preclude" assertion does not apply if 
> > we get to the point that markers are mandatory to 
> > implement/optional to use at all receivers (are you putting 
> > your WG Chair hat on?).
> 
> Not exactly, and not yet.  Here's the proposal that was floated to
> start this thread:
> 
> 	One alternate possibility would be to make marker usage 
> optional,
> 	but put control of this "option" entirely in the hands of
> 	the receiver - senders MUST implement both markers and 
> not-markers,
> 	receivers MUST implement one OR the other, and each receiver
> 	gets to pick which one is used as the benefits of markers are
> 	primarily on the receiver side.
> 
> See Jeff Mogul's email for a long explanation of why something like
> this has a better chance of making progress than the "MUST always
> use markers" and "markers are completely OPTIONAL" positions.
> 
> > However, this means that markers are 
> > mandatory to implement at the transmitter also, which seems 
> > to be where the argument for the most marker pain has 
> > manifested itself for those that are opposed to markers.
> 
> Yes.  This now puts responsibility for the transmitter consequences
> on the receiver.  Receiver implementers who believe that there are
> no significant transmitter consequences can implement receivers that
> always require markers.  Receiver implementers who are concerned about
> transmitter consequences can choose to support not using markers.
> OTOH, we could go on arguing about this for the next 6 months ...
> 
> > Once implemented, is there sufficient delta to a) warrant an 
> > option and b) operate the connection in an asymmetric mode? I 
> > apologize in advance for slipping into "commercial merits 
> > diatribe" but options cost real money once the ink is dry on 
> > the spec. I like to reserve their use/cost for things that 
> > are to the left of the decimal point, rather than the right. 
> 
> This is an unfortunate example of the "vehemence of opposition
> to options" that is causing delay.
> 
> For asymmetric mode, consider that some all-software "client"
> implementations might be more willing to put in the markers to
> help out the receiving "server" if the "client" didn't have to
> go to the pointless additional trouble of stripping markers from
> the data returned by the server.  Consider that the "client"
> might be a small portable device like a PDA or mobile phone.
> 
> Thanks,
> --David 
> ----------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 176 South St., Hopkinton, MA  01748
> +1 (508) 293-7953             FAX: +1 (508) 293-7786
> [email-masked]        Mobile: +1 (978) 394-7754
> ----------------------------------------------------
> 
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: [email-masked]
Date: Wed, 16 Apr 2003 18:31:25 -0400

> David, how does this proposal relate to previous discussions 
> about configurable marker intervals? Would the ability for 
> the receiver to disable markers at the transmitter be 
> sufficient to mitigate the need for additional marker 
> interval options? 

That appears to be likely, but let me go ahead and ask --

	Does anyone want to see support for additional marker
	interval sizes (other than the 512 bytes that was
	deemed important in San Francisco)?

If so, speak up quickly, please.

Thanks,
--David

---

From: Renato Recio <[email-masked]>
Date: Wed, 16 Apr 2003 19:13:27 -0500

What is the strong value of having variable marker interval sizes?  In my
view, without a really, really good reason, and considering the increase in
complexity (receiver, additional negotiation/mgt/debug field) associated
with variable marker sizes,  I would not want to see support for additional
marker interval sizes.

Thanks,

Renato J Recio
IBM Distinguished Engineer
Member IBM Academy of Technology
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685

                      [email-masked]                                                                                 
                      om                       To:       [email-masked], [email-masked]                            
                      Sent by:                 cc:                                                                      
                      [email-masked]        Subject:  [rddp] RE: Marker Progress?                                    
                      rg                                                                                                

                      04/16/2003 05:31                                                                                  
                      PM                                                                                                

Thanks,
--David

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 17 Apr 2003 11:11:41 -0500

On 4/16/03, Renato Recio wrote:

>
>
>
>
>What is the strong value of having variable marker
>interval sizes?  In my view, without a really, really good
>reason, and considering the increase in complexity
>(receiver, additional negotiation/mgt/debug field)
>associated with variable marker sizes,  I would not want
>to see support for additional marker interval sizes.
>

I agree.

As I understood it, the intent of variable marker sizes was
to minimize their overhead while still ensuring one marker
per TCP segment.

But for all the reasons previously discussed on why there is
no such thing as an "aligned connection", merely one that
has been aligned so far, a variable marker size cannot be
tailored to the PMTU. PMTU is a measured characteristic
of a connection, not a configured attribute.

Worse, the entire simplicity of markers requires that they 
be unchanged over the lifespan of the connection. A protocol
that allowed the interval to change to X at stream offset Y
is possible, but certain to be complex. For one thing, it
would require MPA to define in-stream "control chunks".

The benefit of a greater marker interval is at best margin.
The complexity it would introduce is considerable,
especially for testing and monitoring.

I believe the real debate is between mandatory fixed
interval markers, and David's suggestion for having the
receiver mandate whether it expected markers or not.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Talpey, Thomas" <[email-masked]>
Date: Thu, 17 Apr 2003 12:31:25 -0400

I believe the three interesting values are:
	none (infinity)
	512
	8192
("None" for software receivers, 512 for full generality,
and 8K for local networks with jumbo frames.)

Therefore your question might be rephrased, do we need an
optional 8K marker interval?

I say it's not compelling enough to be worthwhile.

Tom.

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 17 Apr 2003 15:50:09 -0500

On 4/17/03, Michael Krause wrote:

>>It is a valid question as to whether the existence of the
>>option is justified. That is, how much of a benefit is
>>no-marker reception to buffer-rich receivers, and are
>>buffer-rich receivers a valid design likely to be
>>implemented by at least a signifigant portion of the
market?
>
>They may be valid for a 100 Mbps or perhaps even a Gbps
>link (but with GbE prices falling so quickly why would
>anyone tolerate the price differential) but anything
>faster and the need for dual-port, high-speed SRAM, etc.
>far outweighs any benefits.

Agreed.

Which is why I am firmly in the camp that says that
high-speed low-buffer RNICs MUST be able to mandate
that they will receive markers.

But I believe RDMA has a far wider applicability than
machines that justify 10 GbE ports. Even if the 10 GbE
port were free, a 10 GbE link may not be available.

A Gbit NIC may more than cover all of the traffic a given
host may ever see. That does not mean that it, or the
servers it connects to, would not benefit from RDMA.

It is a fair point that the overhead of doing markers in
software needs to be considered at the single Gbit range.
Identifying TCP connections alone would seem to make a pure
software solution at 10 Gbits unfeasible, with or without
markers. So if no consensus can be reached on
receiver-driven optional markers, I would fully expect that
software-driven receivers will still be feasible and will
still be deployed.

>>Other than QA testing, I don't see the cost to a sender
>>of making this receiver specifiable. QA testing *is* a
>>serious cost, but the benefit to some receivers, IMHO,
>>clearly justifies it.
>
>QA often becomes the dominant cost in delivery technology
>- it isn't just a one-time thing but is on-going and
>applies to multiple permutations thus options have a
>multiplying effect.  There is also the cost issue on the
>transmitter to support both options - its design may be
>such that removing markers increases complexity (I believe
>this was noted by several people prior to the recent IETF
>event) and thus cost.  Development, QA, etc. costs all
>impact the market adoption rate, TTM for solutions, etc. 
>Each option also comes at an opportunity cost as gates,
>engineers, etc. spent on the option are not free and delay
>or prevent other technologies from being incorporated into
>design points (numerous examples of this in the
>marketplace).

This is the crucial point of comparison. In the long run,
QA and compliance testing will dwarf development costs.
I reamin opposed to variable CRC coverage precisely because
the benefits of header-only coverage seem too slight to
cover the long term testing requirement.

If the consensus is that TOEs will not be part of the long
term solution mix and/or that the impact on TOE/software
stacks for markers is minimal then the QA cost is not
justified.

However, you have to be careful about what you define the
"long term" to be.

In my opinion, during the lifespan of RDDP over MPA/TCP,
there is likely to be a signifigant portion of traffic
that is non-RDMA over TCP through the GigE (or 10 GigE)
ports. If anything, the inertia against changing
applications will take longer to overcome than the inertia
to shift to RDDP over SCTP.

Over that timeframe I strongly suspect that low buffer
RNICs will probably be the dominant solution, but I doubt
that they will be the only one.

Caitlin Bestler