Subject: RE: [rddp] MPA draft status/situation

From: "Vadim Makhervaks" <[email-masked]>
Date: Thu, 5 Dec 2002 22:58:55 +0200

I would add packing as addition thing that need to be reconsidered

Vadim

|---------+------------------------------------>
|         |           "Somesh Gupta"           |
|         |           <somesh_gupta@silverbacks|
|         |           ystems.com>              |
|         |           Sent by:                 |
|         |           [email-masked]      |
|         |                                    |
|         |                                    |
|         |           05/12/02 08:18 PM        |
|         |                                    |
|---------+------------------------------------>
  >-------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                               |
  |       To:       "Caitlin Bestler" <[email-masked]>, <[email-masked]>                                                     |
  |       cc:       <[email-masked]>, <[email-masked]>                                                                           |
  |       Subject:  RE: [rddp] MPA draft status/situation                                                                         |
  |                                                                                                                               |
  |                                                                                                                               |
  >-------------------------------------------------------------------------------------------------------------------------------|

Considering the amount of variance in the scenarios
(10G data center config with hardware acceleration all
the way to software implementations across WANs with
segmenting middleboxes), and the lack of uncertainty
about maintaining alignment at all times etc etc.,
it is better to go with protocol design approach
that is more flexible and does not burden (on the
send side)/relegate (on the the receive path)
software implementations to the slow path.

I believe some of the issues arise from the desire
to have an IB replacement as a data center interconnect,
but at the same time as not limiting (or being able to
limit) the scope of the problem to only that configuration.

Perhaps a compromise of a separate header and data CRC,
combined with optional markers is the right compromise.
Alignment is something that high-speed data center configurations
could use to further accelerate the processing.

Somesh

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]On Behalf Of
> Caitlin Bestler
> Sent: Wednesday, December 04, 2002 10:55 PM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: Re: [rddp] MPA draft status/situation
>
>
> On 12/5/02, [email-masked] wrote:
>
> >
> >The following serious technical issues have to be resolved
> >via compelling design rationale and/or changes to MPA:
> >
> >(1) Markers in their current form appear to have a rather
> >   disproportionate impact on the protocol to deal with a
> >   relatively rare error case (resegmenting middlebox kicks
> >   in at precisely the point of a drop).
> >
> There is also the case of the stream terminating and
> re-originating middlebox.
>
> Such a middlebox
>
> a) totally destroy any alignment, and do so consistently.
>
> b) is an appropriate strategy when acting as a gateway
> between networks with radically different L2 layers.
>
> It is also possible to have a resegmenting middlebox that
> attempts more mild optimizations, and hence 'kicks in'
> during the middle of an in-progress session. I contend that
> this possibility eliminates any chance of ULP testing for
> middleboxes before ULP work begins.
>
> >(2) Markers in their current form appear to (deliberately?)
> >   rule out software implementations.
> >
>
> I do not see how they rule out software implementations.
> Having implemented both convolutional encoding and Reed
> Solomon Forward Error Correction in software I am very
> reluctant to state that software cannot do something.
>
> It does require software to compact the buffer, or more
> likely transfer data from a raw buffer to a post MPA buffer.
> But I suspect this is fully compatible with network speeds
> that you would implement in software anyway.
>
> Additionally, software CRC calculations can be made to have
> very low CPU impact with enough memory for lookup tables.
>
> Hardware performs these functions *better* than software
> does, which is generally true for any simple fixed function,
> but they are quite doable in software.
>
> One of the reasons I continue to favor markers, even though
> I consider them inelegant, is that they strike me as at most
> a nuisance and a slight bandwidth waste. Their impact on
> software based solutions is slight. This is compared to the
> very serious impact that the absence of markers would have
> on a limited buffer RNIC.
>
> The absence of markers would have a far more dictatorial
> impact on RNIC design than their inclusion does.
>
> >(3) The prohibition on packing multiple ULPDUs into a
> single TCP segment departs seriously from RFC 1122.
>
> It also is wasteful of network capacity. Under many
> protocols, the untagged message signalling transaction
> closer will be *very* short. Unless the prior tagged
> transfers aligned perfectly with the PMTU there is a very
> high chance that the ULPDU containing the untagged message
> could fit within the same TCP segment.
>
> I think this issue is worth serious review.
>
> >
> >(4) Assurance must be provided that in no case can two
> >   communicating TCPs (MPA-modified or not) get confused
> >   about whether the TCP bytestream is framed or not.
> >
> The only way to leave MPA mode is to terminate the
> connection. No new connection can be initiated without the
> consent of both ends. Therefore once MPA mode has been
> agreed to, there is no chance of the non-MPA traffic being
> interpreted as though it were MPA traffic.
>
>
>
>
> You also left out the valid, but in my opinion insufficient,
> arguments in favor of a header only CRC to allow more prompt
> placement of resegmented headers. As that resegmenting
> middleboxes *can* be avoided for the vast majority of
> connections I believe that the complexity of the second CRC
> is not justified. However this is based upon the assumption
> that it is acceptable to optimize only for cases where
> alignment is not broken, as long as the connection functions
> even when it is.
>
> If the consensus were to be that the protocol must support
> high levels of performance (in excess of plain TCP) even for
> connections where alignment is broken, then the second CRC
> would be justified.
>
> Caitlin Bestler
> http://asomi.com/CaitlinBestler/
> _______________________________________________
> rddp mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rddp
_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp

---

From: "Uri Elzur" <[email-masked]>
Date: Thu, 5 Dec 2002 13:32:53 -0800

PLS Ignore prev message

David,

Thanks for straightening me up and focusing the discussion again on the technical issues. 

As for the technical issues pls see embedded comments below [UE] and in the other mail sent out yesterday.

Uri
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Wednesday, December 04, 2002 9:20 PM
To: Uri Elzur; [email-masked]
Subject: MPA draft status/situation
Importance: High

Uri,

WG chair hat on for this message ...

> Seems like we have 2 camps:  one camp (Jim, anyone else?) 
> claiming RNIC can not be made cheaper with Markers and a 
> larger group claiming it does simplify the RNIC and makes it cheaper. 
> Should we keep re hashing it again and again on this list?
> 
> Citing a vendor specific existing implementation as a reason 
> to stop any progress essential for the feasibility of high 
> speed (10G+) NICs, is not a compelling reason.

That's definitely an off-beam summary of the current situation.

In hopes of focusing discussion on productive technical matters,
let me get out the clue-by-four and be explicit about what this
WG will have to do if the MPA draft is going to go anywhere.

[UE] My understanding of how a middle box works is that it can re-segment a TCP byte stream without a drop. This makes the receiver sub-optimal. In that case in order to get Direct Data Placement, some buffering must take place before the receiver gets a complete FPDU with its placement information. This will be the case without a drop. 

Another scenario is a  drop due to a simple congestion in a switch or a router, may cause  the receiver to either buffer all the following out-of-order or be able to place it. [UE]

[UE] agree that there is a need to show more analysis to make sure, the heavy weight operation of touching every byte for CRC is evidently the performance hitter more than inserting or removing markers. There is NO intention to rule out SW implementations nor do I think MPA prevents them. [UE]

(3) The prohibition on packing multiple ULPDUs into a single
	TCP segment departs seriously from RFC 1122.

[UE] absolutely agree this might be a place where the current MPA draft may have to change.[UE] 

[UE] "two communicating TCPs (MPA-modified or not) get confused about whether the TCP bytestream is framed or not". I believe framing is an ULP for TCP. Framed stream should not be communicating with non-framed stream in the same way HTTP has no interoperability with another app say NFS. I believe, correct me if I'm wrong, that the goal is to ensure non-framing-aware-application using framing-aware-TCP (with framing disabled for THIS app) has no interoperability issues with another non-framing-aware-application using non-framing-aware-TCP. All similar cases like, framing-aware-application using framing-aware-TCP communicating with framing-aware-application using non-framing-aware-TCP should be shown to interoperate with no issues. [UE]

In the absence of solutions (compelling design rationale and/or
design changes) to at least these four issues, WG rough consensus
will not be sufficient to advance the MPA draft.  The above
list of issues may not be complete (e.g., NFSv4 as a ULP is
not on it), but that should be enough to get some useful work
underway.  Sorry to have to wield the clue-by-four, but some
people seem to have lost sight of the situation that MPA is in.

Thanks,
--David
----------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953 **NEW**     FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: [email-masked]
Date: Thu, 5 Dec 2002 19:24:53 -0700

Caitlin,

Good points. Also, drops might be due to a failure that causes a path change. Therefore, David's case of a resegmenting middle box kicking in at the point of a drop is not necessarily rare.

Pat

-----Original Message-----
From: Caitlin Bestler [mailto:[email-masked]]
Sent: Wednesday, December 04, 2002 10:55 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: [rddp] MPA draft status/situation

Such a middlebox

Caitlin Bestler