Subject: Re: [Tsvwg] Status of TCP ULP Framing

From: Stephen Bailey <[email-masked]>
Date: Mon, 03 Sep 2001 09:09:46 -0400

Joe,

> A protocol, in this case TCP, is more than a wire specification; 
> it defines the behavior of the endpoints, as well as the API.
> 
> At least two of those three are changing.

To the best of our understanding, the behavior described for a
framing-aware TCP supporting PDU alignment-mode is not a change from
what is allowed by the TCP protocol.  In fact, if the application
knew, or guessed at the EMSS, and tiled PDUs appropriately, some
current TCPs would be highly likely to behave exactly like a
framing-aware TCP.  In the cases where the two would differ, the
framing-aware TCP behavior is a change from current implementation
practice, but it is not a change from what is specified.

The API in the appendix is informative, not a part of what is
specified.  All the framing protocol does is insert in-band hints in
TCP's byte stream.  As such, there are other APIs that might as well
be used.  For example, the framing hints look suspiciously like TCP's
urgent pointer.  The semantics of the urgent pointer are subtly
different (multiple urgents can be collapsed), but that behavior of
urgent is probably why nobody uses it.  You could equally well use an
urgent-like API (FLAG_OOB) which is pretty much the same as what
exists today.  You could also use a setsockopt() to position the hints
in the TCP data stream.  The semantics of the byte-stream remain the
same.

Urgent is carried out-of-band (in the TCP header) whereas the ULP
framing hints are carried in-band.  You could imagine the framing
protocol (either mode) as building a noncollapsing urgent pointer on
top of TCP.  Urgent is also a hint carried beside the TCP byte stream
that applications use for their own particular purposes.

We will say something to this effect in the next draft.

> Further, not that I'm an advocate of this, but there are ways to
> transform packets on the wire (split one into two, etc.) which
> are compliant with the existing wire, endpoint behavior, and API,
> which could destroy the meaning of the packets to a ULP receiver
> without a) intending to and b) the receiver detecting it.

Agreed.

Marker mode is layered completely on top of TCP, so resegmentation by
intermediaries is not an issue.

The design goal of PDU alignment-mode is to detect this
resegmentation.  A PDU alignment-mode receiver definitely can not
blindly assume that PDU alignment is guaranteed.  PDU alignment-mode
uses trust-but-verify model.  If you feel the algorithm described has
flaws, by all means, give us the details.

> Interoperable standards need not be standardized within the IETF,
> especially when they are short term.

Yes.

Perhaps the term `transitional' is better than `short term'.  The
duration of the transition is unknown.  There are many
transition-related standards within the IETF.

While this need not be standardized within the IETF, the implementors
would like to standardize this within the IETF.  Given its
relationship to TCP, there is no other standards body which could
provide useful input on the proposal.  Given the goal of implementing
the protocol in hardware, the small stake of control by any individual
implementor, and the number of interested implementors, including new
ones popping up every day who were not members of the original
community that developed the proposal, the participants believe it is
very important to standardize.  There are already some ad hoc,
noninteroperable approaches in the field which those and prospective
implementors agree do not fully serve the goals.  The defined, open
process of IETF RFCs culminating in a standard owned by a
disinterested third-party seems to be exactly what's needed to get
everybody moving in the right direction.  We have not yet found anyone
who sees benefit to pursuing a separate, proprietary, or
consortium-based approach if a standard exists.

We understand that it is not for the authors to decide what IETF
should standardize.  We only propose, participate in the defined
process and the IETF decides.

Thanks,
  Steph

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Douglas Otis" <[email-masked]>
Date: Mon, 3 Sep 2001 13:39:34 -0700

Stephen,

You are advocating more than just an alteration of the TCP send.  In the
draft,
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-tcp-ulp-frame-00.txt
it both alters the send and the receive behavior.

Abstract
"The framing protocol accepts PDUs from a ULP (upper level protocol)
 and transports them over a TCP connection.  This is done in such a
 way that the PDUs can be recovered at the receiver even if
 preceding TCP segments have not yet been received.  This is useful
 when the PDUs are self describing within the context of a protocol
 TCP connection.  In this case, the framing protocol allows incoming
 packets to be parsed (but not processed) in the order received and
 their data to be placed directly in the ultimate destination memory
 instead of TCP reassembly buffers."

Clearly, there is a layer being used below TCP or it would be impossible to
provide for the placement of data while preceding data is pending.  In
addition, TCP does not delineate segments based on send boundaries nor will
existing implementations provide this behavior especially in high bandwidth
modes.  Again you have glossed over details missing from this draft to allow
the "ultimate destination memory" to be identified for the intended
applications such as iSCSI as advocated and you are confusing inter-layer
signaling with the TCP byte stream.  These "hints" are not placed within the
TCP byte stream.  For iSCSI as example, there would be a need to identify
the "ultimate destination memory" with the SCSI tag.

This "in-band" (inter-layer) signaling represents a substantial structural
change for both the send and the receive functions for TCP.  By removing
much of the details from the draft that will be added later in a proprietary
fashion, the send looks only slightly altered but that is illusory.  The
proprietary nature of these missing details ensures both hardware and
application interfaces will amount to chaos.  Unlike the architectural
destruction created by this draft that obliterates any concept of layering,
SCTP provides a solution that is both well documented and allows for all the
needed features sought by this draft while preserving the concept of layers.
Add an endless list of "in-band" (inter-layer) signals, and there is no
longer a conventional TCP interface nor a system that can be stable or
secure.

Doug

---

From: Stephen Bailey <[email-masked]>
Date: Thu, 06 Sep 2001 19:24:07 -0400

Doug,

> By removing much of the details from the draft that will be added
> later in a proprietary fashion, the send looks only slightly altered
> but that is illusory.

You are conflating protocol specification and implementation.

There is no claim that using the framing cues to perform direct data
placement is trivial or would require little or no modification to
existing network stack implementations.  However, the protocol for
communicating the cues does layer cleanly and is succinctly specified.
Further, those working on candidate applications seem to agree that
the cues are what is needed to enable direct data placement.

Steph

---

From: "Douglas Otis" <[email-masked]>
Date: Thu, 6 Sep 2001 23:30:15 -0700

Stephen,

I do not agree that adding "cues", "hints", or "in-band" signaling layers
cleanly, after all, application unique inter-layer signaling erodes layer
boundaries while also requiring additional layers added below the transport.
You are asking for Carte Blanche as to the form these signals take and wish
to reduce TCP to be nothing more than a wire specification with such a
statement about wishing to split specification from implementation.  There
can not be a flourishing community able to exchange information should this
attitude prevail with such stalwarts as TCP.  This is draft is pernicious.

Your statement "We have not yet found anyone who sees benefit to pursuing a
separate, proprietary, or consortium-based approach if a standard exists."
seems to repeat the oversight that SCTP is a standard that does provide
these cues and it is called streams.  If you wish to isolate information at
the transport, stream is the mechanism for doing so when using SCTP.  It is
well defined and is independent of the application.  You do not need to
reinvent this transport to provide the needed behavior attempted by this
draft.  You do not need to change TCP handling at the gateway, through
security layers, or at the API.  Instead, implement the same techniques
using SCTP and inherit a formidable list of other improvements that will be
needed for the intended applications mentioned in this draft.

I can not see any advantage for the short or long term from mucking with TCP
in such a destructive manner.  In less time  SCTP will be adopted and stable
well before getting this draft accepted.  Your goal seems to be focused on
ignoring SCTP and reinventing SCTP lite using TCP while also ignoring one of
the most important innovations offered by SCTP.  A standard cueing scheme
that is independent of the application.  Yes, it is called streams.  It is a
concept still foreign to you.  Embrace it, it is a good thing and provides
for the desired direct placement.  It gets rid of the application unique
inter-layer signaling and added layers.  Such would be a bad thing.

Doug