Subject: [Tsvwg] Status of TCP ULP Framing

From: Andreas Jungmaier <[email-masked]>
Date: Tue, 14 Aug 2001 11:49:20 +0200

Dear all,

I was a bit surprised not to find anything on the TCP ULP framing
draft on the agenda of the TSVWG meeting in London.

What is the current status of this document ? I believe to have 
read somewhere, that a newer version than the current
draft-ietf-tsvwg-tcp-ulp-frame-00.txt is being prepared  which  
incorporates some of the comments that have been made so far.

Could someone please provide a link for that newer version,
or say when it is due to be out ? Thank you.

One question I have to  the approach so far: would applications
that might use a framing aware TCP stack where possible and 
a normal TCP stack where there is no framing aware TCP stack
have to cater for a negotiation of which TCP to use ?
Aren't there interoperability problems ? 

Best regards,

Andreas

- -- 
- ------------------------------------------------------------------------
Dipl.-Ing. Andreas Jungmaier              |                         _
Computer Networking Technology Group      | ASCII ribbon campaign  ( )
University of Essen                       |  - against HTML email   X
http://www.exp-math.uni-essen.de/~ajung   |              & vcards  / \

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Stephen Bailey <[email-masked]>
Date: Tue, 14 Aug 2001 20:56:42 -0400

> or say when it is due to be out ?

Shortly.  I'd expect that means around the end of the month.

> have to cater for a negotiation of which TCP to use ?

Yes.  The draft points out that this negotiation is performed as
specified by the ULP.

> Aren't there interoperability problems ? 

As a purely existential question, interoperability problems abound :^)

If you're talking specifically about ULPs which chose to use the
framing protocol, the answer is no.  Each ULP must define a mechanism
to enable framing, and provide the relevant parameters to each peer.
The most likely mechanism would be a negotiation.

This is described in section 4 (Negotiating Use Of The Framing
Protocol) of the current draft.  If something in this section is
unclear please bring it to the authors' attention.

A receiver which desires the use of PDU alignment mode (which implies
a framing-aware TCP sender at the remote peer), must be prepared to
not have its desire granted.  In this case, it can either wish for
markers as a fall back, or simply not use the framing protocol at all.
The mechanics by which use of framing (or not) and its mode is
selected is what is ULP-specific.

Steph

---

From: Andreas Jungmaier <[email-masked]>
Date: Wed, 15 Aug 2001 13:25:16 +0200

Stephen, dear all,

please find my comments inline.

> As a purely existential question, interoperability problems abound :^)

Okay, I understand that. What I was thinking about is just that 
there are tons of applications out there using TCP. Now, these
will have to be enhanced by some negotiation of transport protocol
features. 
This strikes me as an architecturally unfortunate decision...

In doing it that way, you take considerable comlexity away from the
framing protocol, just to burden it onto the application processes...
Please correct me, if I am wrong here !

> unclear please bring it to the authors' attention.

Again, that would mean that all TCP application protocols that could 
possibly want to use the TCP ULP framing extension would need to be
enhanced, doesn't it ?
I agree, that for new application layer protocols that might not be 
too much of a problem, but for existing ones this is not fortunate.

> markers as a fall back, or simply not use the framing protocol at all.

...or it might use a transport protocol that does support message
oriented transfer of ULP PDUs in the first place - like SCTP !
The only problem I have with the approach you suggest is what you 
point out in the last part of your last statement:
An application might end up not using the framing protocol at all.

Best regards,

Andreas
- --

---

From: Stephen Bailey <[email-masked]>
Date: Wed, 15 Aug 2001 22:22:58 -0400

Andreas,

> possibly want to use the TCP ULP framing extension would need to be

Yes.

The assumption is that existing applications are either a) fine
without framing assist (thank you very much) or b) sufficiently
motivated by the performance improvement (to THAT particular
application) that it's worth some change to the protocol or its
implementations.

As the draft points out, one relatively unintrusive way to introduce
framing support for a ULP is to offer a framed version of the ULP on a
different port.

If you wish to accelerate an existing ULP implementation, you should
use a different technique, such as a ULP-agnostic or ULP-aware
hardware TCP offload, or possibly something which might be
API-compatible but not wire compatible, like Microsoft's Winsock
Direct (presently only works for system-area networks).

> oriented transfer of ULP PDUs in the first place - like SCTP !

I agree that using a transport which provides the framing capability,
and specifically PDU alignment `natively' is the ultimate answer.
However, for protocols being developed now (e.g. iSCSI), the approach
the hardware vendors are comfortable with is to start on unmodified
TCP, migrate to TCP + the framing protocol when framing-aware
(hardware) receivers become available, and finally, move to a new
transport (e.g. SCTP) when it becomes widely deployed.

> An application might end up not using the framing protocol at all.

Correct.  The emphasis is on motivated ULPs (and peers).  In cases
where the framing protocol is not used, because one end or the other
does not have the appropriate support, the application will still run,
but possibly not with the same level of performance.  If avoiding
degraded mode is not an adequate motivation to modify the endpoints,
the ULP's probably not a good candidate for using the framing
protocol.

The TCP framing protocol is not a cure for all performance ills.  It's
a minimum-delta assist for one particular case.

Steph

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 16 Aug 2001 12:00:46 -0700

Stephen,

I have noticed that you failed to answer the most salient question asked
which is why not use SCTP.  The intent of the ULP Framing for TCP
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-tcp-ulp-frame-00.txt
is to allow a "packet filter" to process the TCP segments out of sequence to
ensure a bounded limit on the buffering required due to packet loss and to
eliminate the need to further extract the payload contained within the
application's PDU.  To do this properly for applications such as SCSI or
RDMA which are the intended immediate uses of this modification to TCP,
there needs to be an exchange of information between the portion of the
application sitting above TCP with that portion sitting within the "packet
filter."  SCTP performs this operation by using streams.  The stream
concept, which was considered too heavy weight to bother with by those
advocating this TCP modification, is the mechanism that allows these
associations to take place without corrupting the network layering as is the
case for this modification of TCP.

SCTP has documented those aspects of application interaction which was not
even touched by this draft.  In addition, as there will be extensive changes
needed to the application, the excuse for not using SCTP seems to wane.  In
addition, as there is no means to ensure proper error handling with this ULP
Framing for TCP in the event of an inadvertent segmentation, it seems likely
to be unstable.  David suggests that if there is an error, this off-load
process will simply backup and re-arrange the memory placement once an error
is discovered.  One can only wonder the state of all the odd pieces of
padding and discarded headers that also need to be recovered.  Here is a
list of differences between this draft and your modification of TCP.  Your
level of modification no longer allows one to claim the high ground with
respect to stability.  SCTP however has the advantage of having been
designed for this type of use whereas this draft is a kludge at best.

-------------------------------------|ULPfTCP| SCTP |
Defines the Application Interface    | No    | Yes  |
Defines MTU Application interaction  | No    | Yes  |
Improves Error detection             | No    | Yes  |
Adds multi-homing (in iSCSI)         | No    | Yes  |
Aggregation of streams (in iSCSI)    | No    | Yes  |
Protection from DoS attacks          | No    | Yes  |
Protection from Blind Spoofing       | No    | Yes  |
Provides data recovery on failure    | No    | Yes  |
Optionally allows chunk fragmentation| No    | Yes  |
Able to detect segmentation fault    | No    | Yes  |
Changes optimal gateway behavior     | Yes   | No   |
Modifies existing Protocol Stacks    | Yes   | No   |
Extensively Modifies Layer Boundaries| Yes   | No   |
Changes the NIC Driver Interface     | Yes   | No   |
ULP Negotiation with NIC Driver      | Yes   | No   |
-----------------------------------------------------

Doug

---

From: Stephen Bailey <[email-masked]>
Date: Thu, 16 Aug 2001 21:52:15 -0400

Doug,

> which is why not use SCTP.

See:

> > the approach the hardware vendors are comfortable with is to start
> > on unmodified TCP, migrate to TCP + the framing protocol when
> > framing-aware (hardware) receivers become available, and finally,
> > move to a new transport (e.g. SCTP) when it becomes widely
> > deployed.

Steph

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 16 Aug 2001 19:19:12 -0700

Stephen,

The step from unmodified TCP to SCTP could be a direct path that avoids the
havoc created by this surgery on TCP.  In effect, this is nothing more than
a short term solution.

Doug

---

From: Joe Touch <[email-masked]>
Date: Sat, 25 Aug 2001 00:22:10 -0700

> framing support for a ULP is to offer a framed version of the ULP on a

Why not on a different protocol number? 
That would be much less obtrusive.

Joe

---

From: Stephen Bailey <[email-masked]>
Date: Sat, 25 Aug 2001 10:48:27 -0400

> Why not on a different protocol number? 

That's certainly an option.  A ULP which uses framing must specify how
the framing protocol is enabled.  A ULP could chose to specify a
different protocol number for framed operation.

However, the objective of the framing protocol is to allow endpoints
with minimally modified or completely unmodified TCPs to support
efficient, direct data placement on the remote endpoint.

In either mode, if you watched the exchanges of endpoints using the
framing protocol from some point in the middle, you could not say they
were not talking TCP.  The TCP segments exchanged conform completely
to the specifications TCP.  If they don't, please tell us so we can
fix it!  For marker mode, this is vacuously true.  PDU alignment mode
only specifies a sender behavior change, not a protocol change.

Steph

---

From: Joe Touch <[email-masked]>
Date: Sat, 25 Aug 2001 10:24:22 -0700

Stephen Bailey wrote:
> fix it!  

A protocol, in this case TCP, is more than a wire specification; 
it defines the behavior of the endpoints, as well as the API.

At least two of those three are changing.

Further, not that I'm an advocate of this, but there are ways to
transform packets on the wire (split one into two, etc.) which
are compliant with the existing wire, endpoint behavior, and API,
which could destroy the meaning of the packets to a ULP receiver
without a) intending to and b) the receiver detecting it.

Joe

---

From: Andreas Jungmaier <[email-masked]>
Date: Mon, 27 Aug 2001 20:55:53 +0200

Stephen, dear all,

please find some comments below.

On Saturday, 25. August 2001 16:48, Stephen Bailey wrote:

- ----------------- snip -----------------------------------------------
> efficient, direct data placement on the remote endpoint.
- ----------------- snap -----------------------------------------------

The focus should be on a sound design, and not on a quick hack...
Your approach gives a whole new semantics to an established old
proven protocol. I am not sure  that is a good idea.
As a matter of fact I am quite sure, that this is not a good
thing to do.

If it is only for a short term solution, then why go about introducing 
modifications to TCP implementations (and even their applications),
that are kludgy (IMHO :-) and also need quite a bit of interoperability
testing before they can be deployed ?

As far as I see, you will get similar problems that will arise with 
a wide-spread deployment of SCTP. 

Best regards,

Andreas
- --