Subject: RE: [rddp] MPA TCP analysis draft; avoiding confusion on use

From: "Culley, Paul" <[email-masked]>
Date: Tue, 18 Feb 2003 11:17:12 -0600

I have always been curios why the ADs are placing requirement 3 on MPA.
Since MPA is now characterized as an adaptation layer for DDP, and as
such, cannot get turned on unless DDP is turned on, why is it necessary
to "prove" that MPA/DDP cannot ever get accidentally (or otherwise)
connected to an inappropriate port/address and (presumably) cause some
kind of trouble?

Has this kind of requirement been applied to other applications that use
TCP?  For example, has it been shown that an HTTP client cannot ever
cause a problem when connecting to some other non-http server or other
client?  I suspect not.

Of course it may be that the ADs placed this requirement on MPA when MPA
was being put forth as a general framing mechanism; depending on how it
got enabled, it might have been enabled unexpectedly under some
unsuspecting application.  I never thought that was very likely either,
but...

Attempting to "fix" this "problem" by adding additional Hello exchanges,
merely seems to move the problem to "proving" that the Hello messages,
cannot cause problems themselves, putting us back where we started...

Paul R. Culley
HP Senior Fellow
281-514-5543

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Tuesday, February 18, 2003 10:42 AM
To: [email-masked]
Subject: [rddp] MPA TCP analysis draft

This draft (draft-elzur-iwarp-mpa-tcp-analysis-00.txt) has been out for
over a week and generated no discussion.  Let me see if I can engender
some discussion by offering some observations in my role as WG chair.
All of these are preliminary observations for further discussion, but I
would ask that folks pay particular attention to the "prove" issue in
(3) to avoid the WG chair having to get out a clue-by-four.

The Elzur MPA TCP analysis draft covers 3 topics:
(1) Packing ULPDUs into a TCP segment
(2) Sender alignment of ULPDUs to the start of TCP segments
(3) Avoiding endpoint confusion about whether MPA is enabled Discussion
of each follows:

-- (1) Packing

The draft recommends that packing of multiple ULPDUs into a single TCP
segment be supported (sender MAY pack, receiver MUST support receipt of
packed segments).  This strikes me as a good thing and a welcome change.

-- (2) Sender ULPDU alignment to TCP segments

Some of the buffer size calculations seem questionable given the prior
WG consensus that senders using unmodified TCP stacks (which cannot
support alignment in all circumstances) be supported - a sender running
over an unmodified stack will lose alignment despite its best efforts,
so the claimed benefits of alignment are probably overstated, BUT - see
the end of the discussion of
(3) below on identifying sender behavior.

OTOH, the calculation of buffering required for a non-aligned MPA sender
seems to assume no drops or reordering, although considering those would
only increase the advantage of reliable sender alignment.

Overall, I tend to agree that there are useful receiver implementation
optimizations enabled by a sender who can align ULPDUs to TCP segments.
One that I didn't see mentioned is that checking the MPA CRC when each
packet is received allows an MPA CRC failure to be dealt with by
dropping the offending packet and using TCP retransmission - an approach
that is not as readily usable if a CRC spans multiple packets that do
not arrive back-to-back.  While this is a layer violation (CRC check
"above" TCP triggers TCP drop), I would be prepared to argue with the
"layering police" that this is acceptable and desirable because it
strengthens an existing TCP function (detection and rejection of packets
corrupted in transmission), and the pure TCP alternative (TCP Alternate
Checksum - RFC 1146) is undeployable in practice due to the changes it
requires to both TCP stacks and the wire format (lots of stuff will
break).

A key design challenge/goal is to ensure that receivers can take
advantage of receiver alignment performance optimizations 
*without* locking out senders that use unmodified TCP stacks due to
their inherent ability to lose alignment.

-- (3) Avoiding Endpoint Confusion about MPA use

I want to remind everyone that our AD asked the WG to "prove" that
senders and receivers cannot be confused about whether MPA is in use; I
assure everyone our AD chose the word "prove" deliberately.  With
suitable design attention to the MPA Hello/Hello ACK/ACK details, I
believe such a mechanism has a good possibility of providing a suitable
"proof" - Figure 5 looks like a good starting point, and its conciseness
is a positive indication.

OTOH, I have serious concerns about the protocol analysis required
(Section 8.2) for transition via a well known port or service location
protocol (Figure 4).  With apologies to the authors (whose hard work on
this draft is acknowledged and appreciated), that analysis reminds me of
an old joke about how to "prove" that all odd numbers > 1 are prime:
	3 is prime
	5 is prime
	7 is prime
	9 is ... experimental error	
	11 is prime
	13 is prime
	lacking a counterexample, they must all be prime ... :-) :-)
[for "experimental error", see the DNS discussion in Section 8.2.4.3]

In all seriousness, the inability to find a counterexample does not
prove anything, as there are many more protocols that run over TCP than
those considered in Section 8.2, and who knows whether 32 bits of zero
is a command code in some private protocol that hasn't been considered.

It seems to me that efforts might be better directed towards using an
MPA Hello/Hello ACK/ACK or similar mechanism in all cases, even when a
well-known port or service location protocol is used for the transition
into MPA mode.  Such a design would also provide an opportunity to
address a nagging buffering issue - a receiver that can't distinguish an
MPA-modified aligning TCP sender from an unmodified TCP sender who is
nonetheless using MPA (e.g., for the CRC) will have to pessimistically
assume that all senders are unmodified TCPs and allocate the larger
buffering resources discussed in Section 7.1.

In the current MPA design, I don't believe a receiver can make that
distinction.  In contrast, if different MPA Hello codes were used for
the two classes of senders, the receiver could more reliably optimize
the buffering for the MPA-modified aligning TCP that should require less
buffering.  A receiver will still have to watch out for unexpected
alignment failures even from MPA-modified aligning TCP senders (e.g.,
courtesy of Caitlin's resegmenting middleboxes that delay making
themselves known).

This is all for further discussion ... I would suggest that replies to
this message pick out one of the three topics above and change the
message subject accordingly.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: [email-masked]
Date: Tue, 18 Feb 2003 18:28:16 -0500

> >> I have always been curios why the ADs are placing
> >> requirement 3 on MPA. Since MPA is now characterized as
> >> an adaptation layer for DDP, and as such, cannot get
> >> turned on unless DDP is turned on, why is it necessary
> >> to "prove" that MPA/DDP cannot ever get accidentally (or
> >> otherwise) connected to an inappropriate port/address
> >> and (presumably) cause some kind of trouble?
> >
> >MPA proposes to change TCP - see the sender alignment
> >discussion. If MPA did not propose to change TCP (i.e.,
> >all senders could always use unmodified existing stacks)
> >then (portions of) your "applications that use TCP"
> >discussion would apply ...
> >
> True, MPA makes some minor changes for an MPA aware sender.
> However, the resulting stream is fully TCP compliant.
> 
> Specifically, it is impossible to examine a MPA/TCP stream
> and *prove* that it could not have been generated by an
> extremely annoying ULP that just wanted the stream to
> look like it was an MPA/TCP stream.

Unfortunately, that's called "missing the point."  A lot of
time was spent in Atlanta and afterwards on the topic of
TCP being more than just a specification of the byte stream
on the wire - surely we don't need to re-cover that ground ...

> -- Given the flexibility of base TCP for ad hoc ULPs, there
> will never be a method to absolutely ensure rapid detection
> of cross-ULP mismatches for TCP. Any such method would have
> to alter TCP itself.

As I said in an earlier post, "absolutely" is a red herring.
OTOH, the WG is welcome to take this position - the likely
result will be the removal of sender alignment from MPA.

> -- An MPA/TCP stream arriving at the wrong port is no more
> likely to be mistakenly accepted than any other TCP stream.
> Indeed, the draft does a heroic job of showing that it is
> *less* likely.

[... snip ...]

> I agree with Paul, there is no justification for this to be
> documented. The message needs to be clear that no ULP is
> going to be dragooned into using DDP or MPA. Hence the
> normal procedures for matching ULPs and ports are fully
> applicable, as complete and/or as flawed as they are for
> all other IP based ULPs.

And if MPA were not proposing to change TCP, that would be a
reasonable end to this discussion.  If DDP/MPA is positioned
as just another application that uses TCP (like HTTP, FTP,
etc.), tsvwg's response to a request to change TCP will be
to enquire what part of the word "NO" has not been understood.
If people are serious about wanting to take this approach,
sender alignment will probably have to be removed from MPA,
at which point the "prove" requirement about avoiding confusion
should no longer apply.

Launching and losing another argument on this topic will not
impress the Transport Area Directors when it comes time to decide
whether the RDDP WG will be allowed to continue working after
the March IETF meeting ... (yes, rddp has a rechartering event
coming up fast - check the WG charter on the IETF web site).

Thanks,
--David

---

From: "Larsen, Roy K" <[email-masked]>
Date: Tue, 18 Feb 2003 16:05:31 -0800

> Sent: Tuesday, February 18, 2003 3:28 PM
> To: [email-masked]; [email-masked]
> Subject: RE: [rddp] MPA TCP analysis draft; avoiding confusion on use
> 
> 

David,

At the risk of missing the point yet again, I have looked at the MPA draft
and could not find a requirement that TCP must change to support MPA.  It
clearly differentiates performance benefits that can be derived with a "MPA
aware" TCP implementation but allows interoperable operation with an
unmodified TCP stack.  As Catlin and others have pointed out above, it is
not possible for a receiver to detect whether I am sending from a MPA aware
implementation or not.

I was not at the Atlanta meeting.  Was there a fundamental change to the MPA
draft that has not been published yet?

Roy

---

From: [email-masked]
Date: Wed, 19 Feb 2003 12:55:41 -0500

Caitlin actually quoted the RFC 1122 text that explains why
PUSH won't get the job done:

> > The PSH bit is not a record marker and is independent of
> > segment boundaries.  The transmitter SHOULD collapse
> > successive PSH bits when it packetizes data, to send the
> > largest possible segment.

The problem is that if there is a transmit queue larger than
the segment size, existing TCP stacks collapse the multiple
PUSH requests into one at the tail of the transmit queue, and
then (re)segment the queue into maximally sized segments for
transmission, breaking sender alignment.  This behavior has
been pointed out previously on the tsvwg list.

> MPA/TCP overrides that recomendation *for the purpose of
> improving performance*. In my mind this is precisely the
> type of justification that is required to justify overriding
> a SHOULD.

This "justify overriding a SHOULD" is the sort of discussion
that I seem to recall asking people to avoid.  The TSV ADs
and the tsvwg WG own this decision, and I would think
it's clear by now that their views on this matter may not
match those of the MPA advocates.

> The criteria has always been interoperability. MPA/TCP does
> not do anything to a TCP stream that any ULP has not been
> free to do all along. Therefore it does not change TCP.

Sorry, incorrect in several ways, and a waste of time ... unless
the preferred outcome is to repeat the Atlanta experience in San
Francisco and see the rddp group charter cancelled or revised
to remove TCP support ...

--David

> From: Caitlin Bestler [mailto:[email-masked]]
> Sent: Wednesday, February 19, 2003 11:05 AM
> To: [email-masked]
> On 2/19/03, David Borman wrote:
> 
> 
> >>
> >>
> >> MPA 5.1.1's SHOULD can be implemented using a 4.2.4.3
> >> compliant flush call. Therefore it does contradict RFC
> 1122.
> >
> >Perhaps you meant "does not contradict"?  But no matter,
> >because the FLUSH call *throws away* any unsent data (the
> >data to the right of the send window).  I don't think that
> >is what you want.
> >
> >You can implement sender alignment over an unmodified TCP
> >stack. Unfortunately, the most portable way to do that is
> >to have application level per-segment acks, and only allow
> >one outstanding segment at a time. Thus, it'd probably be
> >a performance hit.  (Another implementation could be done
> >if you could query the TCP stack to see how much data is
> >waiting to be ACKed, and just wait for that to go to zero
> >before writing the next segment.)
> >
> 
> Oops, up too late yesterday. 
> 
> The section I should have cited was
> 
> >4.2.2.2  Use of Push: RFC-793 Section 2.8
> > When an application issues a series of SEND calls without
> > setting the PUSH flag, the TCP MAY aggregate the data
> > internally without sending it.  Similarly, when a series
> > of segments is received without the PSH bit, a TCP MAY
> > queue the data internally without passing it to the
> > receiving application. 
> >
> > The PSH bit is not a record marker and is independent of
> > segment boundaries.  The transmitter SHOULD collapse
> > successive PSH bits when it packetizes data, to send the
> > largest possible segment.
> > 
> > A TCP MAY implement PUSH flags on SEND calls.  If PUSH
> > flags are not implemented, then the sending TCP: (1) must
> > not buffer data indefinitely, and (2) MUST set the PSH
> > bit in the last buffered segment (i.e., when there is no
> > more queued data to be sent).
> > 
> > The discussion in RFC-793 on pages 48, 50, and 74
> > erroneously implies that a received PSH flag must be
> > passed to the application layer.  Passing a received PSH
> > flag to the application layer is now OPTIONAL .
> > 
> > An application program is logically required to set the
> > PUSH flag in a SEND call whenever it needs to force
> > delivery of the data to avoid a communication deadlock. 
> > However, a TCP SHOULD send a maximum-sized segment
> > whenever possible, to improve performance (see Section
> > 4.2.3.4 ).
> 
> Note that it does not send TCP MUST send a maximum-sized
> segment whenever possible to improve performance.
> 
> MPA/TCP overrides that recomendation *for the purpose of
> improving performance*. In my mind this is precisely the
> type of justification that is required to justify overriding
> a SHOULD.
> 
> An argument could be made that the use of this facility is
> not what was intended when the RFC was drafted. But by that
> criteria a vast majority of the Web is RFC non-compliant.
> 
> The criteria has always been interoperability. MPA/TCP does
> not do anything to a TCP stream that any ULP has not been
> free to do all along. Therefore it does not change TCP.
> 
> Caitlin Bestler
> http://asomi.com/CaitlinBestler/