Subject: Re: [rddp] Let's discuss markers

From: "Vadim Makhervaks" <[email-masked]>
Date: Tue, 3 Dec 2002 09:43:32 +0200

Sorry for previous note, please discard it.

I would propose to separate Alignment and Markers.

If one could assume a perfect Alignment, no Markers are needed.

Markers are needed to handle a case when the TCP segment carrying DDP
header is lost,
and no alignment assumption can be made.

I do not think that receiver is allowed to make any assumption regarding
alignment,
at the same time I think that Alignment suggestion (should and not must)
and Marker do help to make receiver implementation efficient (not
necessarily simple).
Therefore no negotiation process is needed.

Let me describe my view on the receiver decision-flow:

1. New TCP segment arrives.

2. This segment passes all TCP validations.

3. If the segment arrives in-order, then the location of DDP header is
known
(the length field of the previous DDP segment can be used for this
purpose).
Alignment would help in this case, because the receiver DDP segment can be
directly placed to the memory.
Without alignment, if only part of DDP segment was received, this part
would need to be stored locally
in the RNIC memory.

4. If the segment arrives out-of-order, we need Markers to find the nearest
DDP header.
In out-of-order case, Markers allows reduce the memory resources of RNIC.
Without Markers, to recover from segment lost, RNIC would need either store
in all arriving TCP segments
in its internal buffers, till it won't receive the missing TCP segment,
and only then proceed DDP segments processing.
Markers limit pretty well the memory resources needed to recover from TCP
segment lost:
MSS*number-of-holes, and number of supported holes is anyway limited by TCP
implementation.

5. Note that once DDP header is found, all following DDP segments are
treated as received in-order,
till the next hole.

To conclude: on my view Markers is not a first thing that I would wish to
implement,
but by now I do not see any other solution for efficient recovery from TCP
segment lost.

Vadim

|---------+---------------------------->
|         |           [email-masked]|
|         |           om               |
|         |           Sent by:         |
|         |           [email-masked]|
|         |           rg               |
|         |                            |
|         |                            |
|         |           03/12/02 08:26 AM|
|         |                            |
|---------+---------------------------->
  >-------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                               |
  |       To:       [email-masked]                                                                                                 |
  |       cc:                                                                                                                     |
  |       Subject:  [rddp] Let's discuss markers                                                                                  |
  |                                                                                                                               |
  |                                                                                                                               |
  >-------------------------------------------------------------------------------------------------------------------------------|

Since an important technical issue underlying this debate
about not-quite-bufferless NICs is markers, let's take a
closer look at markers.  This entire message is with my WG
Chair hat off for the purpose of provoking discussion ...

... so, for the sake of discussion, let's assume that we can
run a framing-modified TCP transmitter in aligned mode - the TCP
ULP framing draft that was ready to go experimental not that
long ago proposed to do this, so this is not an entirely
unreasonable assumption.

Under this assumption, what's the purpose of markers?  MPA was
apparently designed under this assumption, and the MPA draft says:

             7.4.1  Re-segmenting Middle boxes and non-conforming senders

Non-conforming senders sure sounds like a matter for connection setup
negotiation - get the other side to say "yes, I do aligned FPDUs"
on setup, and then close the connection in error if the receiver
ever detects that the sender broke this promise (e.g., as if a CRC
error not recoverable via retransmits occurred).

That leaves re-segmenting middle boxes.  First of all, it appears
to be clear from list discussion that the designers of (and advocates
for) MPA view them as a rare or error case.  Even so, many statically
resegmenting middleboxes ought to be detectable via some sort of
"training sequence" test run at connection setup designed to cause
resegmentation - if resegmentation happens, disable framing before
sending any data that depends on it.

This would appear to reduce the scenarios of concern to a subset
of a rare error case.  Given this, the impact of markers on the
operation of framing seems to be rather disproportionate -
e.g., in their current "every 512 bytes" form, they all but
exclude software implementations.

Nonetheless, there's still a problem here.  What if TCP drops a packet
and at that precise point in the bytestream, a dynamic resegmenting
middlebox decides to kick in and screw up everything?  That should be
sufficiently rare that it may be ok to just close the connection ...
and tell the poor user to get a real network ... ;-).

... but a whole bunch of data is going to come in before the retransmit
arrives that indicates the receiver's been fooled by the middlebox - what
gets done with it?  One possibility is to process it on the assumption
that it's all aligned frames, and close the connection in error when
the retransmit shows up indicating that they're not, and the receiver
was fooled by what looked like frames in data (in most cases, errors
will happen very quickly, but the "frame in data" case has to work
correctly) ...

... so what about the frames that were processed incorrectly between the
original drop and the retransmit?  Well, they could only have been placed,
because the missing retransmit prevents delivery to the ULP (TCP delivers
in order, and framing can't change that).  So, when the retransmit arrives,
close the connection, and complete all the pending receives with errors
indicating that the buffers are invalid.  This has to be implemented
anyway to deal with a CRC error that can't be recovered via retransmits,
so it's almost "free" :-) ...

... but what if the frames that were processed incorrectly affected
memory that wasn't supposed to be affected by any pending receive?  Any
memory that it could have modified has to be declared invalid - this is
somewhat worse than the CRC error as any exposed memory in the protection
domain gets invalidated ... but I think I've seen folks headed towards
a design approach of being rather conservative in how much memory is
exposed when (e.g., to provide application assurances that the memory
from completed receives can't be overwritten).  This might not be an
important difference, but it is clearly fodder for discussion, and
reinforces the importance of precisely specifying protection domain
access control functionality.

One more thing - anytime one comes up with an error mechanism that
closes a connection, denial of service becomes a possibility via
the attacker causing the error.  In this case it's a definite possibility,
as a clever attacker could corrupt a TCP packet and then send
a fake one that looks like a retransmit that's been resegmented by a
middlebox, causing the connection to close.  This doesn't look like
a major increase in risk, as an attacker who can send the latter packet
can probably also send a TCP RST which will close the connection far
more effectively (yes, one could in theory do TCP RST filtering at
[some] routers, but in practice this is not done, and hence can't be
relied upon).  IPsec is needed to defend against this class of attacks.

Ok, that should be more than enough to provoke discussion ...
... comments?

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953 **NEW**     FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Talpey, Thomas" <[email-masked]>
Date: Tue, 03 Dec 2002 09:40:00 -0500

Vadim thanks for the insightful summary. I agree that
Alignment and Markers are useful to distinguish.

What if the marker interval were negotiable on each
side of the connection? Software implementations
would request zero (no markers) on their receive side,
but "consenting adults" could agree otherwise?

The obvious advantage for tuning the marker interval
relative to the MSS, as well as enabling software
implementations on highly optimized peers (e.g.
embedded) would seem to far outweigh the addition
of optional behavior.

Tom.

At 02:43 AM 12/3/2002, Vadim Makhervaks wrote:
>>-----------------------------------------------------------------------
>--------------------------------------------------------|
>  |
>|
>  |       To:       [email-masked]
>|
>  |       cc:
>|
>  |
>|
>  |
>|
>>-----------------------------------------------------------------------
>--------------------------------------------------------|
>
>
>