Subject: Re: [Tsvwg] RE: [rddp] RE: Placement "outside of" TCP

From: David Borman <[email-masked]>
Date: Thu, 19 Dec 2002 15:19:12 -0600 (CST)

> Date: Thu, 19 Dec 2002 13:19:18 -0700
> From: "Brian F. G. Bidulock" <[email-masked]>
> Subject: Re: [Tsvwg] RE: [rddp] RE: Placement "outside of" TCP
>
> David,
>
> On Thu, 19 Dec 2002, David Borman wrote:
> > 
> > From a protocol standpoint, what is really needed is an extension
> > to TCP to allow out-of-order delivery of data, ...
>
> This extension would give TCP an option to be un-TCP.  If you want

No, it would be TCP with an *optional* extension that would provide
a *standard* interface for allowing TCP to deliver out-of-order
(and I better add un-acknowledged) data to the next layer up.

> out-of-order delivery, use UDP.  If you want reliable out-of-order
> delivery, use RUDP, or more recently, SCTP.

I'm not saying that I want it.  I'm *trying* to point out that the
whole MPA mess is due to adding markers which breaks the strict
protocol layering design in order to allow implementations to blur
the protocol layers more easily in their implementations, strictly
for optimization reasons.  So, take a step back from the whole markers
issue and see how TCP would need to be modified to provide the necessary
things to allow the implementations to be optimized without muddying
up the MPA/DDP/TCP protocol design.

If we do that, then we can decouple the proposed changes to TCP from
MPA/DDP, and evaluate them on their own merits.

> Regardless of your beliefs on the values of APIs, RFC 793 happens
> to place requirements on the TCP API.  Anyone wanting to get away
> from that needs to strike a few MUSTs from that standard.

Yep.  That's what modifying a protocol is all about. :-)

Personally, I don't view TCP as sacred.  To be highly regarded, yes,
but not sacred.  I've been working with TCP since about 1984, and have
been directly involved in modifications to it.  TCP is an extensible
protocol, that is one of its beauties.  But I'll still scrutinize any
proposed changes/extensions to TCP, whether they be from someone else
or from me.   RFC 793 represents the understandings that the designers
had over 20 years ago.  They could not envision all potential uses for
TCP, but they provided mechanisms for TCP to be extended when needed.
Some of the real obvious modifications/extensions to TCP are in RFC
1122, RFC 1323 and RFC 3168.

> If the only problem were efficient reodering, why not reorder the
> delivery of IP packet indications from the NIC to TCP's lower
> boundary, using only information in the TCP header (without even
> copying)?  Then TCP sees a network where everything arrives in sequence
> and there is no need for markers or out-of-order delivery.  TCP can
> direct place to the user, perhaps in response to a standard readv(),
> and maintain in-sequence delivery.

You're missing the point.  One of the arguments of those arguing for
markers is that the NIC won't have enough memory to buffer everything
up in the case of packet loss, waiting for the missing packet to
arrive.  (And even if it does, then you have a burst of activity to
get all the back-logged data off the NIC and up into the host memory
once the missing piece arrives.)

> Why not?  I suspect the answer is because DDP applications really
> want framing too.  Then use a record-based protocol (UDP, RUDP, SCTP)
> instead of a stream-based protocol (TCP) or take the performance
> disadvantage of using the wrong tool for the wrong job.

And that may well be the right answer.  Let's call a spade a spade.
Adding markers for doing out-of-order placement of TCP data crosses
strict protocol layering design, even if it doesn't change "TCP on the
wire".  At best markers are an engineering solution based on the
constraints that they can't/won't change TCP, markers are not a protocol
design solution.  Let's yank it from the shadows and put it into the
bright light, and evaluate it for what it really is.  And the best way
I know to do that is to propose clean modifications to TCP to provide
the functionality that is being requested, and then see if those proposed
modifications can stand by themselves.  If they can't, then TCP is the
wrong tool for the job.

		-David Borman, [email-masked]
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: Caitlin Bestler <[email-masked]>
Date: Thu, 19 Dec 2002 16:07:07 -0600

On 12/19/02, David Borman wrote:

>
>And that may well be the right answer.  Let's call a spade
>a spade. Adding markers for doing out-of-order placement
>of TCP data crosses strict protocol layering design, even
>if it doesn't change "TCP on the wire".  At best markers
>are an engineering solution based on the constraints that
>they can't/won't change TCP, markers are not a protocol
>design solution.  Let's yank it from the shadows and put
>it into the bright light, and evaluate it for what it
>really is.  And the best way I know to do that is to
>propose clean modifications to TCP to provide the
>functionality that is being requested, and then see if
>those proposed modifications can stand by themselves.  If
>they can't, then TCP is the wrong tool for the job.
>
Any clear "start from scratch" analysis of what DDP required
from its LLP would come up with something very much like
SCTP without multi-homing. TCP wouldn't even be close.

But that misses the whole point. Nobody believes that TCP is
the ideal transport for RDMA protocols. The goal is to allow
consenting ULP to experience the benefits of RDMA *without*
having to change the intermediate network, or deploy a new
network (such as InfiniBand).

Engineering to be compatible with equipment that is already
deployed seldom results in pretty solutions -- but the
effort is important. The investment millions of users have
placed in their existing network equipment is something
that should take precedence over our desire to design
an "elegant" protocol.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 19 Dec 2002 15:03:43 -0700

David,

Sounds like a resonable approach to me.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp