Subject: Re: [rddp] RE: Marker Progress?

From: Renato Recio <[email-masked]>
Date: Thu, 17 Apr 2003 11:44:19 -0500

Folks,

   So, it seems the real issue is whether normatively defined that:
A)  RDMA/DDP over TCP requires MPA in the middle, or
B)  RDMA/DDP over TCP may use MPA or may not.

If that is the case, then it seems to me we could either decide now, or let
the market decide later.  In my view, the market will go with (A), because
of the benefits it provides.  If we go with (B), we would add complexity to
the (initialization) protocols and a slight increase in implementation
complexity, which would, in my view, not be used much (if at all) in the
market.

So, what's the gain in going with (B)? If its to allow a NIC to be fully
buffered and hence it would not need markers for placement?  It seems to me
that type of NIC would need a lot of buffer capacity to support the number
of connections demanded by future Servers, Appliances, Storage, etc...   It
seems to me the cost of that additional capacity, even considering $/MB
take-down rates, is not going to be competitive when compared to a
flow-through design that doesn't require the additional capacity.

Thanks,

Renato J Recio
IBM Distinguished Engineer
Member IBM Academy of Technology
IBM I/O Architecture and Technology
Tel 512-838-3685, T/L 678-3685

                      Caitlin Bestler                                                                                   
                      <[email-masked]>         To:       [email-masked]                                                  
                      Sent by:                 cc:                                                                      
                      [email-masked]        Subject:  Re: [rddp] RE: Marker Progress?                                
                      rg                                                                                                

                      04/17/2003 11:11                                                                                  
                      AM                                                                                                

On 4/16/03, Renato Recio wrote:

>
>
>
>
>What is the strong value of having variable marker
>interval sizes?  In my view, without a really, really good
>reason, and considering the increase in complexity
>(receiver, additional negotiation/mgt/debug field)
>associated with variable marker sizes,  I would not want
>to see support for additional marker interval sizes.
>

I agree.

As I understood it, the intent of variable marker sizes was
to minimize their overhead while still ensuring one marker
per TCP segment.

But for all the reasons previously discussed on why there is
no such thing as an "aligned connection", merely one that
has been aligned so far, a variable marker size cannot be
tailored to the PMTU. PMTU is a measured characteristic
of a connection, not a configured attribute.

Worse, the entire simplicity of markers requires that they
be unchanged over the lifespan of the connection. A protocol
that allowed the interval to change to X at stream offset Y
is possible, but certain to be complex. For one thing, it
would require MPA to define in-stream "control chunks".

The benefit of a greater marker interval is at best margin.
The complexity it would introduce is considerable,
especially for testing and monitoring.

I believe the real debate is between mandatory fixed
interval markers, and David's suggestion for having the
receiver mandate whether it expected markers or not.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 17 Apr 2003 12:58:50 -0500

>B)  RDMA/DDP over TCP may use MPA or may not.

I believe that MPA is required no matter what. See my
immediate prior posting on Header CRCs being mandatory.
So the real question is whether Markers are optional where
the option is dictated by the receiver. I believe there is a
strong consensus that receivers MUST have the option to
mandate that streams sent to them have markers.

>If that is the case, then it seems to me we could either
>decide now, or let the market decide later.  In my view,
>the market will go with (A), because of the benefits it
>provides.  If we go with (B), we would add complexity to
>the (initialization) protocols and a slight increase in
>implementation complexity, which would, in my view, not be
>used much (if at all) in the market.
>
To be of value to the recievers, the standard really needs
to mandate that senders MUST comply with receiver
requirements. So, I do not see how market forces will have
an impact here. Even if 99% of the market goes with markers,
the 1% that decided otherwise would be entitled to have
senders comply with their receive requirements.

It is a valid question as to whether the existence of the
option is justified. That is, how much of a benefit is
no-marker reception to buffer-rich receivers, and are
buffer-rich receivers a valid design likely to be
implemented by at least a signifigant portion of the market?

>So, what's the gain in going with (B)? If its to allow a
>NIC to be fully buffered and hence it would not need
>markers for placement?  It seems to me that type of NIC
>would need a lot of buffer capacity to support the number
>of connections demanded by future Servers, Appliances,
>Storage, etc...   It seems to me the cost of that
>additional capacity, even considering $/MB take-down
>rates, is not going to be competitive when compared to a
If I'm desiging and/or buying an RNIC, I agree. I'd prefer a
low to zero buffer RNIC with minimal functionality. But then
I've gotten used to the idea that my preferences are not
binding upon the market. Given that AmigaDOS is my favorite
OS to date, we can go further and conclude that my
preferences are not even a valid predictor of the market.

However, the alternate design is a valid approach. It
particularly offers benefits when optimizing for a mix
of TCP and RDDP/TCP connections. A buffer rich NIC offers
optimizations for *all* TCP connections.

I have never bought the argument that markers imposed a
severe cost on "software" implementations. I remain opposed
to creating a mode marker *for this purpose*. But there is
a benefit to eliminating markers over these connections.

So let me flip the question. Other than QA testing, what
is the cost of omitting markers when the receiver will
not benefit from them?

Caitlin Bestler