Subject: RE: [Tsvwg] fragmentation and PMTU

From: [email-masked]
Date: Tue, 15 Jan 2002 09:27:37 -0500

> > > This all seems fine until one looks carefully at one little aspect:
> > >
> > > - If I an implementation supports fragmentation it MUST fragment.
> > >
> > > The issue I have with this is what if an application layer says:
> > >
> > > "hey I know you can fragment, but please don't, I want you
> > >  to reject sends that exceed the P-MTU."
> > 
> >         Excuse my ignorance, but, why would you like such kind of
> > behavior? Couldn't the upper user simply discard those messages
> > beforehand? The upper user should be aware of the variations in the
> > P-MTU but... :-/
> 
> 
> One place such a thing MIGHT be wanted is in something like
> SCSI over IP... there are other applications as well that
> may NOT want to ever send anything over a PMTU.

Not exactly.  On its own, SCSI has little interest in PMTU - SCSI
tends to transfer data in units of 2k, 4k, 16k, etc., driven by
what's convenient for the systems it connects.  IMHO, forcing a
SCSI implementation to fragment to PMTU is not a real good idea.
I believe the actual motivation here is the desire to define a
Direct Data Placement (DDP) layer between SCTP and a protocol
like SCSI over IP - one of the ways of doing this entails adding
information to every chunk and hence wants to ensure that SCTP
never fragments so that DDP can do its own fragmentation and
add its info to every fragment.

> I believe also Doug Otis posted something along this line
> as well for RDMA... not sure of the details.. I will have
> to go back and look in the email archives...

DDP is an important enabling technology for RDMA.  There's
a lot more about this on the [email-masked] list.

Thanks,
--David

---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA  01748
+1 (508) 249-6449 *NEW*      FAX: +1 (508) 497-8500
[email-masked]         Cell: +1 (978) 394-7754
---------------------------------------------------

_______________________________________________
tsvwg mailing list
[email-masked]
http://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Randall Stewart <[email-masked]>
Date: Tue, 15 Jan 2002 08:52:05 -0600

David:

Thanks for the correction.. I knew it had something
to do with IPS and RDMA... sorry about getting them
mixed up :>

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: "Douglas Otis" <[email-masked]>
Date: Tue, 15 Jan 2002 08:12:30 -0800

David,

Allowing the Direct Data Placement shim to sit above SCTP needs two features
from the SCTP API.  One is the ability to do the chunk fragmentation.  This
allows automation of reassembly by the shim through insertion of an offset
and is independent of the message size being sent by the ULP.  This also has
advantages with respect to partial writes to a Stream, in that there is no
sequential TSN requirement across a message.  This feature has been
accommodated in the latest Socket draft -03, and should be included in the
Implementers guide.

The second requirement is equally important.  As these fragmented chunks are
sent Unordered to ensure no buffering is done by SCTP, completion signaling
then requires that the Transmission Sequence Number as well as the
Cumulative Transmission Sequence Number be visible.  As messages are
fragmented and reassembled by the shim as received, there is no need for the
Stream Sequence Number (it is invalid for Unordered delivery) and its 16 bit
size *may* represent a potential limitation at very high bandwidths in the
future.  A message is signaled complete to the ULP when the last chunk in a
message containing a marker is received and its TSN is equal to or less than
the cumulative TSN per serial math.  This marker signal is held at the shim
layer until the TSN condition is met.

The shim concept is to allow Direct Data Placement (DDP) to be added as a
layer above SCTP.  This differs substantially from framing techniques used
on TCP, such that no packet filter is inserted below the transport to
intercept packets.  SCTP remains unaltered and its path management,
bundling, retry algorithms, etc. remain intact and unaltered.

Doug

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 15 Jan 2002 16:28:24 -0800 (PST)

> layer until the TSN condition is met.

Can you explain why the shim cannot put in its own sequence number and
the stack has to expose the internal TSN to the ULP?

							K. Poon.
							[email-masked]

---

From: "Douglas Otis" <[email-masked]>
Date: Tue, 15 Jan 2002 17:23:18 -0800

Kacheong,

I understand a desire to hide internals of SCTP from the ULP.  In the case
of TSN, the transport is doing a significant amount of work providing a
cumulative TSN as well as tracking the TSN on each chunk.  As these are
structures already available within the transport and on the wire and, at
one time were exposed, these values available to the ULP will remove
overhead both with respect to size on a per chunk basis as well as the score
boarding otherwise required by the shim.  As you know, the Stream Sequence
Number is of no use for Unordered delivery as this value is invalid.  If
there is an application running in the Unordered mode, there would be no
means to synchronize completion of a set of chunks across a Stream as SCTP
does not provide sufficient information although it could.

I see this feature assisting both a TCP model able to employ partial writes
in a software implementation as well as an SCTP off-load device that of
course would have no difficulty seeing this transport information.  I
suspect the reason for exclusion of TSN information from the ULP was that
there were few applications at the time considering Unordered delivery and
thus relied upon ordering provided by SCTP and SSN to isolate messages.

By allowing this feature, this allows the most expedient means of providing
signaling that would not get tangled in the transport's decision to hold a
stream and to transmit another out of sequence of the ULP thus increasing
possible HOL blocking at the shim.  For applications such as RDMA that make
little use of the Stream feature of SCTP, there is a desire to treat Streams
in unison and thus create more HOL blocking at the send should there be any
disparity between the shim send order and the SCTP send order.  As you may
know, there is ongoing work related to Stream flow control that could easily
exasperate this problem.

Doug

> > The second requirement is equally important.  As these
> > fragmented chunks are sent Unordered to ensure no buffering
> > is done by SCTP, completion signaling then requires that the
> > Transmission Sequence Number as well as the Cumulative
> > Transmission Sequence Number be visible.  As messages are
> > fragmented and reassembled by the shim as received, there is
> > no need for the Stream Sequence Number (it is invalid for
> > Unordered delivery) and its 16 bit size *may* represent a
> > potential limitation at very high bandwidths in the future.
> > A message is signaled complete to the ULP when the last chunk
> > in a message containing a marker is received and its TSN is
> > equal to or less than the cumulative TSN per serial math.
> > This marker signal is held at the shim layer until the TSN
> > condition is met.
>