Subject: Re: [rddp] SCTP Multi-homing issues

From: Michael Krause <[email-masked]>
Date: Mon, 04 Nov 2002 08:47:59 -0800

At 10:08 AM 10/29/2002 -0600, Caitlin Bestler wrote:
>The multi-homing capabilities of SCTP raises some questions
>for DDP and the accessibility of host memory across multiple
>interfaces.
>
>An SCTP endpoint on a multi-homed host is a collection of
>eligible IP addresses. Presumably these are on different
>physical interfaces, doing otherwise just slows down the
>failover process that is supposed to provide fault tolerance
>for network failures.
>
>This raises a few possibilities:
>
>-- An RDMA/DDP SCTP endpoint may only select IP addresses
>    that are managed by the same RNIC. This would represent
>    a restriction on RDMA/DDP SCTP associations that would
>    not apply to other SCTP associations.
>
>-- All STags enabled for any stream in the association must
>     be enabled for ALL RNICs that are part of the SCTP
>     endpoint. This would require that SCTP RNICs accept
>     host processor assigned STags.
>
>
>If my understanding of the feature is correct, an SCTP
>endpoint must be ready to recieve packets on any of the
>eligible interfaces at any time. This is a sharp contrast to
>automatic path migration under InfiniBand. It also mandates
>full real-time duplication of all STag related resources on
>every interface that is part of an SCTP endpoint. This is
>clearly undesirable if the alternate interface was intended
>solely as a fallback (i.e., a primary interfacewould carry
>the entire traffic, as opposed to having n interfaces load
>balance the traffic and willingly reduce to n-1 interfaces
>with a reduced capacity).

Agreed - highly undesirable.  If fault tolerance is required for the 
particular application, then it really isn't a suitable candidate for RDMA 
as the fault zone is the RNIC.  The performance objective is to enable OS 
bypass, copy avoidance, and copy avoidance - these are in direct conflict 
with the desire to checkpoint state across multiple layers / hardware 
devices in order to provide transparent fail-over at any given time.  In 
fact, one can argue that it is nearly impossible to implement such 
fail-over with a protocol off-load solution as it would require a highly 
integrated system design (not just an I/O device but integration at the 
system level including changes to the memory subsystem / controller).  All 
of this was considered during the development of Future I/O and InfiniBand 
and was rejected as being too expensive / complex to implement at a 
commodity design point (even in the high-end, it is far to expensive to 
implement and thus would not be ubiquitous and therefore not a subject for 
standardization).

>In my opinion, many to most RDMA ULPs would prefer to handle
>path migration on their own. This is especially true of
>those that are using RDMA to achieve very low predictable
>latencies. For those ULPs, restricting an SCTP endpoint to
>the scope where the hardware can pool STag resources "for
>free" would be the optimal choice.

Agreed.

>For others, fully conforming to standard SCTP behavior with
>its fault tolerance and congestion management would be
>preferred.

Mike

_______________________________________________
rddp mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rddp