Subject: RE: [Ipoverib] recharter consideration

From: Yaron Haviv <[email-masked]>
Date: Tue, 10 Sep 2002 23:04:14 +0300

Caitlin,

I'm merely pointing out a requirement, which I think is very important to
the success of IB
It could be addressed either by IBTA, or by IPoIB, and in several ways
The issue I raise, is do we take it into consideration (charter), and not
how to implement it
If this is of no value to the group I can just drop it

Few corrections to your assumptions:

Path migration is not applicable in this case since it deals with failure of
paths (switches) and not nodes
APM doesn't enable migrating to a different node, which is the case I'm
presenting

As to SDP (and I'm a great fan of SDP), it cannot replace IPoIB entirely
since it doesn't deal with ICMP/UDP
And in some cases IPoIB is more resource effective than SDP
Also don't neglect the fact that IP traffic can be routed in to IB through
hosts or bridges/routers
We must support IP over InfiniBand for real traffic passing; we cannot leave
those issues open and expect SDP/DAPL to solve them
IPoIB must support the basic scenarios dealt in Ethernet, and fail-over is
one of them

Routers that implement VRRP for example don't break the TCP connections
during the fail-over phase
And also there might be unconnected traffic (UDP) too
So it is not a problem the IB CM can solve

If you refer to the CM in the case of opening an IPoIB RC connection and
mandating an ARP request prior to that
It is a reasonable solution, might not solve all issues but will work for RC
Any way it needs to appear in the spec, which requires a charter

I would be happy to here some positive ideas how this issue can be targeted

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Caitlin Bestler
Sent: Monday, September 09, 2002 6:55 PM
To: [email-masked]
Subject: RE: [Ipoverib] recharter consideration

On 9/8/02, Yaron Haviv wrote:

>Caitlin,
>
>The issue is really that TCP/IP works mostly over Ethernet
>When running over IB few limitations arouse, one is the
>inability to do MAC faking (some may consider it a
>benefit) This has nothing to do with DAT, SDP, .. it
>relates to how addresses are resolved And by the way SDP,
>DAPL relay on IPoIB to provide the address resolution
>capability
>
>Just a simple example in Ethernet a backup router sends a
>packet with the Master router MAC address in case of a
>fail-over This automatically configures the switches to
>route the traffic destined to that MAC address to the
>backup router In IB there is no way to do such a thing,
>since the HW addresses might be even fixed (base GID=GUID)
>(VRRP protocol standardize just that over Ethernet and
>other mediums) Other scenarios could be when one host
>wants to take over HW address on another this could be
>replaced by gratuitous ARP's, how ever they are limited in
>some apps (e.g. in case of multiple IP addresses,
>active/active scenarios, no client support, ..)
>
>The solution might be either change the IB L2, enable GID
>changes on the fly + immediate notifications to update the
>switches Or it could fall within the scope of IPoIB (RC)
>
>One solution that could work with IPoIB RC is for example
>in the connection setup headers pass an alternate address
>That would be used in case of failure, since RC connection
>setup headers are in the scope of IPoIB RC, this is Ok
>

I believe you are confusing an existing method (MAC
spoofing) with a requirement. The functional goal is to
allow an alternate servver to be selected automatically.

I would contend that that problem can be solved either
through the IB Communications Manager or by simply using
IPoIB to do IP to GID resolution before establishing the
connection.

MAC spoofing as a failover does not preserve an existing
connection. Path migration does. If the goal is to achieve
this sort of connection transparent as to whether an IB or
IP network is in use then SDP and DAPL would seem to be more
appropriate.

If the goal is to make a byte or message stream ULP
connection over an IB network, what benefit is there in
insisting on tunneling that connection in IP format?

If IPoIB is going to expand its charter, it should be on the
basis of solving ULP problems, not merely trying to make it
a universal solution. That becomes an effort in "fixing" IB
to be more IP like. In my opinion, the end result will be
unappealing to any target market.

One legitimate goal that has been cited was supporting a
larger MTU to allow atomic delivery of larger IP frames, at
least matching jumbo Ethernet frames. I think that RD and UC
should be considered for this goal. They meet the objective
while imposing fewer additional semantics than found in the
orignal link layer.

Caitlin Bestler
http://www.asomi.com/CaitlinBestler/
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Caitlin Bestler <[email-masked]>
Date: Tue, 10 Sep 2002 16:04:33 -0500

On 9/10/02, Yaron Haviv wrote:

>Caitlin,
>
>I'm merely pointing out a requirement, which I think is
>very important to the success of IB It could be addressed
>either by IBTA, or by IPoIB, and in several ways The issue
>I raise, is do we take it into consideration (charter),
>and not how to implement it If this is of no value to the
>group I can just drop it
>

The question, at charter time, is whether this is a problem
that should be solved in this specific forum.

Relaying of intact IP datagrams to/from IB hosts is a valid
problem, and what I would understand to be the current
charter.

Taking on additional options to fulfil that mission would
certainly be consistent with the current charter. Debating
which mode is the best method for relaying jumbo Ethernet
sized IP datagrams is certainly valid. I happen to think
that UC or RD would be better, but it is totally possible
when all factors are considered that RC is the best
solution.

What I am objecting to is chartering reliable connections as
a new feature of IPoIB, in the sense that it suggests this
is a better way to establish an end-to-end ULP connection.

Providing reliable identification of alternate servers on an
IB network is an IB problem. It should be dealt with in the
IBTA, or perhaps in SDP.

Once a solution exists it should be valid whether or not
there is an IP client or not. It could even be claimed that
the Communications Manager already provides for service
failover -- it just does not provide for transparent
switching out of a server behind the clients back.

It could even be argued that this is a security feature of
InfiniBand networks, not a defect.

For example, in the discussion of the definition of a
connection for DAT (Direct Access Transport) the DAFS users
(Direct Access File System) were insistent that a connection
could choose a different path, or even a different end-port,
but that it MUST be to the same host -- That server failure
was a ULP issue, and MUST NOT be handled at the transport
layer.

That is also basically true with TCP/IP. You cannot
successfully handoff an aribtrary TCP connection to a new
server transparently to the client. You have to have planned
for a virtualized connection within a cluster to begin with.

In any event, in the chartering sense, failover between
nodes within an InfiBand subnet is an InfiniBand problem.
It should be solved by IBTA, for all IB modes and hosts, not
just those using IPoIB. Further, it can be argued that it
HAS been solved.

If the goal is to define server fail-over on an end-to-end
basis, it should be solved in pure-IP terms. Such that the
replacement server could be an IP over Ethernet host, or
even failing from IP/Ethernet to IPoverIB. In other words it
should be a totally seperate draft proposal.

Dealing with server failure specifically in the context of
bridging IP and IB does not make any sense. Nor does
suggesting that IPoIB should be the 'normal' mode of traffic
within an IB network.

Caitlin Bestler
http://asomi.com/CaitlinBestler/
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib