Subject: [Tsvwg] Comments on draft-ietf-tsvwg-ulp-frame-00.txt

From: Bernard Aboba <[email-masked]>
Date: Tue, 9 Oct 2001 10:27:43 -0700 (PDT)

Section 2

"No reassembly buffer is required at all."

This statement seems to imply that direct placement enables NIC high
speed memory requirements to avoid having to scale with the bandwidth-delay
product. While it seems logical that direct placement will reduce the
NIC high speed memory requirements, the requirements will still increase
with bandwidth. For example, a 64-octet frame sent over Ethernet requires
roughly 600 bit times to transmit (512 + 96 bit inter-frame gap). At 1
Gbps this requires 600 ns; at 10 Gbps, 60 ns; at 100 Gbps, 6ns. Assuming a
1 Ghz processor, this represents 600, 60 and 6 cycles,  
respectively. Somewhere along this continuum, in at least some
circumstances, bottlenecks will conspire to prevent NIC memory buffers
from being emptied as fast as they are filled. This becomes more likely
as speeds increase. The result is that high speed NIC memory *will* need
to increase with bandwidth-delay product so as to be able to handle the
resultant queuing. On that basis, it seems that the statement
represents an exaggeration. The degree to which this is true is probably
worthy of detailed study. 

Section 3

"When the framing protocol is in PDU alignment mode, each PDU is limited
to the smaller of 65528 and the size that will fit entirely within a TCP
segment."

Presumbably this statement attempts to take into account the case
of oversize TCP segments which are subsequently fragmented. Given the MTU
sizes of existing media, and assuming that path MTU discovery is used,
won't the TCP segment size always be smaller?

Section 5

Later on, it is noted that the framing header is not required when IPsec
or TLS is used, or when there are no intermediate boxes resegmenting  
TCP. The circumstances in which the framing header is negotiated should
probably be explicitly discussed at the beginning of this section.

"the use of oversize TCP segments sent by means of IP fragmentation is
discouraged due to the limited size of the IP header Identification field
and the potential for undetected errors due to wrapping of the
Identification value."

I think you need to be more specific on the undetected errors you are
referring to, and the situations in which they won't be detected. For
example, are you referring to errors in the IP headers or
payloads? Payload errors will presumably be detected either by the TCP
checksum (modulo concerns about its use at high speeds), or by other   
protection mechanisms in place, such as IPsec or TLS.

Section 7.3 Using TLS with the Framing Protocol

"Using the framing protocol with TLS requires modification of a TLS
implementation for the combination to perform effectively."

The modifications required to TLS are substantial, and the
effectiveness even with these changes is unclear, so overall I come
away unconvinced that use of TLS with ULP framing really makes sense. 
I think that the issues needs to be collected in one sub-section and that
an overall recommendation would be helpful. 

For example, it is stated "it is probably not appropriate to negotiate
the use of a stream cipher when securing the framing protocol." Given that
the RC4 stream cipher is by far the most commonly negotiated cipher with
TLS, this not be compatible with most existing TLS implementations. Given
the desire to use ULP framing in high speed applications, presumably we
would need TLS implementations with AES support.

However, when a CBC mode cipher is used, section 7.3.1 states "the need to
receive the previous TCP segment before completing TLS processing of
current TCP segment means that using the framing protocol in PDU alignment
mode... defeats one of the primary advantages of PDU alignment mode." So
moving to a block CBC-mode cipher doesn't work well either. 

Section 7.3.1 and 7.3.2

"There is no need to encapsulate TLS PDUs in a framing PDU". 

This statment applies to PDU Alignment mode, but is not included within
section 7.3.2 which covers TLS in Marker Mode. So overall, I am left with
the impression that in Marker Mode, it *is* necessary to encapsulate TLS
PDUs in a framing PDU. Is this correct?

If so, this is problematic, since a TLS utilizing a framing header will
not be backward compatible with RFC 2246, unless framing headers could be
turned on after the initial TLS handshake completes. This would mean that
TLS implementations with framing headers would need to operate on a
separate port. This is awkward, since some applications might require 3
ports: one of the "naked" protocol, one for RFC 2246 TLS, and another port
from TLS with framing headers. 

Overall, from reading section 7.3, I come away with the impression that
use of TLS with ULP framing is not very practical, and even where
feasible, has dubious benefits. For example, in section 7.3.1 it states
"while it is possible to use TLS to secure the framing protocol in PDU
alignment mode, IPsec would be a more appropriate choice". Based on the
discussion of section 7.3, I agree with this statement, but think that it
should probably be placed up front and be made stronger, such as "Use of
TLS with ULP framing involves significant issues and is not recommended."

Section 5.2

"the sender SHALL ensure that one of the following is true for every
segment with a partial framing PDU:

A. octets 0-1 do not equal the segment length minus 8
B. octets 2-8 do not match the framing key value
C. The total segment legnth is less than the framing PDU header of 8
   octets"

In this case, it would appear that the framing header is being used as a
kind of "integrity check" against resegmentation by an intermediate
box. Given that such a integrity check is not required when other framing
PDU integrity mechanisms are in place (e.g. IPsec), and that ULPs
frequently contain their own CRCs which may map 1-1 with framing PDUs in
PDU alignment mode, it is worth considering whether in PDU alignment
mode the framing header is needed at all, and if, in the absence of 
IPsec or TLS, whether a framing CRC might be substituted instead. This
would potentially save some space (e.g. 4 octets instead of 8), and would
not impose additional cost since presumably ULP CRCs will be used anyway.

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Stephen Bailey <[email-masked]>
Date: Thu, 11 Oct 2001 11:37:48 -0400

Bernard,

> product.

It doesn't seem to imply that to me.

The buffering and control requirements of performing complete
reassembly in hardware are certainly substantial.  This statement
simply says that the framing protocol can remove the need to implement
reassembly logic in the NIC.

What remains after you have removed the reassembly mechanics is
certainly not trivial, and as you point out there's a large,
substantially unstudied contiuum of engineering possibilities.

The point of this draft is that many (not all) hardware vendors think
that by removing the need to implement reassembly mechanics, hardware
acceleration becomes much more tractable (in terms of engineering and
solution cost) with the current, and anticipated future circuit
technology.

All that said, if you want to suggest some text that makes this point
clearer, I would appreciate it.

> won't the TCP segment size always be smaller?

Actually, it's not about fragmentation of `oversized' TCP segments.
The MTU sizes of existing (as opposed to common) media range well
beyond what an IPv4 datagram can accommodate.  The IPv4 limit for the
payload of a single TCP segment, which can certainly be transmitted in
unfragmented form on many media, is 65535-20-20 = 65495 octets.

The 65528 octet number is the maximum payload that the structure of a
framing PDU can accomodate---the 16-bit length field covers the
framing PDU header + payload, and the framing PDU header size is 8
bytes.  The IPv4 limit for a framing PDU would be 65495-8=65487 octets
Yes, this is smaller than the structural limit of the framing PDU, but
the `within a TCP segment' clause certainly covers this.

> protection mechanisms in place, such as IPsec or TLS.

It sounds more like you're advocating that we not mention this at all.
The specific error is aliasing of IP fragments (a subsequent fragment
is mistaken for a previous one).  As you say, various integrity
mechanisms (TCP checksum or crypto check) should catch such aliasing,
but the integrity checks were designed under a particular error model
(frequency & type), and I'm certainly not sure if the results of
fragment aliasing are inside or outside the bounds of that model.

Whatever the case, the problem is exactly the same whether the framing
protocol is used or not.  It amounts to a choice about whether an
implementation wants to resegment or fragment, and as I understand it,
existing TCP implementations have taken both choices.

We will look at the wording of this section again.  Any other concrete
text suggestions you have would be appreciated.

> Section 7.3 Using TLS with the Framing Protocol

We are hard at work on a new draft, which incorporates comments from a
previous discussion along these exact lines with Andreas Jungmaier
(look for `Comments on TCP ULP Framing and TLS' in the archives).

Basically, he said pretty much what you've said (`it looks impractical
to use TLS with ULP framing, and furthermore, your description of how
to do it isn't even complete'), and I said (at tedious, wheezing
length):

Just because something `looks' impractical doesn't mean it actually
is.  TLS cipher acceleration looks pretty impractical WITHOUT framing,
yet there are many attempts at it.  Using framing to frame TLS PDUs
may actually make TLS cipher acceleration easier.  Certainly it
doesn't make it any harder.  Once you've paid the table stakes for TLS
cipher acceleration, using the ULP PDU framing cues is probably a drop
in the bucket.

The section on using TLS with framing, particularly PDU alignment
mode, was intended to outline the issues that we are aware of.
Specifically, it describes in some detail why you wouldn't expect
TLS+framing to `just work'.  It was expected that if there was real
motivation to do TLS+framing (none has come to our attention yet), a
subsequent standards effort would address it.

What has changed in the text of the forthcoming draft is an up-front
statement to this effect, as you have suggested.

> box.

That is EXACTLY what it is doing.  In fact, that is the ONLY role of
the framing header.

We've also had quite a bit of previous discussion of this which you
could find in the archives.

> it is worth considering whether in PDU alignment mode the framing
> header is needed at all, and if, in the absence of IPsec or TLS,
> whether a framing CRC might be substituted instead.

We definitely did consider this.  Everybody would have been much
happier if we did not have to shim a header into the ULP data stream.
Objections abound---`it's heavyweight', `it's gross', `I don't see the
need'.

ULP integrity checks (non-crypto digests & field validation) are not
designed for this particular error model (resegmentation), and, in
general, DO NOT WORK effectively.  These integrity checks are designed
assuming that you are checking what is, or once was, a PDU that came
from the sender.  In other words, there is no question that the
receiver/checker is synchronized with the PDU stream.  A condition
where these checks are not an effective synchronization indicator is
when the ULP payload stream is filled with data that looks exactly
like valid ULP PDUs.  The worst case chances are roughly
1:length(minimum ULP PDU) that such an integrity check would give a
false positive indication that a PDU is aligned with the beginning of
the TCP segment when it is actually not.

Without a crypto integrity check, if we want framing cues to be
in-band, we must either use stuffing (which was ruled out because of
its inefficiency for software implementations), or there must be a
secret with respect to the ULP data stream (the random key).  Perhaps
the desire to take advantage of ULP PDU aligment, but without the
distasteful framing header will be the impetus required to make use of
crypto ubiquitous :^)

Steph