Subject: RE: [Ipoverib] recharter consideration

From: Larry Butler <[email-masked]>
Date: Fri, 6 Sep 2002 13:51:33 -0700

Yaron,

A fail-over mechanism in the IPoIB domain is interesting to me especially
when it will allow master/slave relationships in the clustering server
world. I can't speak for the companies that write clustering server
applications (e.g. the database vendors), but I think this capability is
desired if not a requirement in high-availability configurations. Question:
how is this done on non-IB clusters today?

Thanks,

Larry

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Friday, September 06, 2002 12:56 PM
To: 'Larry Butler'; [email-masked]; 'H.K. Jerry Chu';
[email-masked]
Cc: [email-masked]
Subject: RE: [Ipoverib] recharter consideration

Larry,

Obviously it is very helpful for routers (its like VRRP)
However many applications, clusters use this mechanism to enable IP take
over by the slave in case of fail-over
(And clustered apps will run on IB as we know)

As to bill's remark I rather first see if there is interest for it in IPoIB
and maybe even get some ideas before I go and write a complete draft for it

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Larry Butler
Sent: Friday, September 06, 2002 6:57 PM
To: [email-masked]'; 'H.K. Jerry Chu'; [email-masked]
Cc: [email-masked]
Subject: RE: [Ipoverib] recharter consideration

Yaron,

Are you suggesting that an RFC 2338-like Router Redundancy Protocol be
looked at for IPoIB? Or does it go beyond routers and also apply to any IP
end station?

Larry

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Thursday, September 05, 2002 1:40 PM
To: 'H.K. Jerry Chu'; [email-masked]
Cc: [email-masked]
Subject: RE: [Ipoverib] recharter consideration

Jerry,

We can try and target fail-over with IPoIB
In Ethernet a system can just use a MAC faking technique to migrate to a
standby station
In IB a client station cannot take over an HW address of a different machine
This poses restrictions on fail-over mechanisms
(One might suggest ARP's for it, however it doesn't solve the same problems)

We can try to have a mechanism, especially in conjunction with RC
Where we have a standby HW address configured, in case of failure we move to
the alternate address
(It is different from APM since the two paths direct us to two different end
nodes, not one)

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
H.K. Jerry Chu
Sent: Thursday, September 05, 2002 2:32 AM
To: [email-masked]
Cc: [email-masked]
Subject: [Ipoverib] recharter consideration

In Yokohama we discussed a few new items that people thought
may be useful and have suggested to add them to the WG charter.
Note that these work items still need to be blessed by the
ADs and IESG before they can be adopted by the WG. For each
work item the ADs are looking for

a. clear benefit (over the added cost) and sufficient interest
from the WG

b. concise definition and scope

c. deliverables and milestones

The following is a summary of the work items that have been
brought up so far and the questions/issues raised by the audience
in the Yokohama meeting.

1) encapsulating IP datagrams using a non-UD transport (e.g.
UC, RC)

question - scope is not clear

2) demux IP datagrams over multiple QPs

issue - dubious benefit; it may introduce routing and session
establishment problems?

(Two opposed this item in f2f meeting.)

3) other client capability, e.g. SDP

issue - non-IP therefore out of scope

4) skipping TCP/UDP checksum when running over RC/UC.

issues - serious concern from many. difficult to ensure end2end
property. could be made to work in a limited set of cases but the
added complexity seems far outweigh the benefit.

All the above fall into the category of advanced features,
and their usage is completely optional. This way they should
not affect the basic IP over IB operation defined by the
existing IPoIB drafts.

Given the amount of concerns raised so far against 4), I'd like to
remove it from the list of consideration. If you have a problem with
that please bring it up to me and Bill. For the rest three I'll need
you to

1. show your interest and/or volunteer for the work
2. provide a concise description on what the item is, why it should
be considered (e.g. the benefit), and the deliverables

Jerry and Bill,
IPoIB co-chairs

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Caitlin Bestler <[email-masked]>
Date: Fri, 6 Sep 2002 16:39:30 -0500

On 9/6/02, Larry Butler wrote:

>Yaron,
>
>A fail-over mechanism in the IPoIB domain is interesting
>to me especially when it will allow master/slave
>relationships in the clustering server world. I can't
>speak for the companies that write clustering server
>applications (e.g. the database vendors), but I think this
>capability is desired if not a requirement in
>high-availability configurations. Question: how is this
>done on non-IB clusters today?
>

Why would you introduce non-IP semantics to IPoIB? Wouldn't
it just be easier to shift to IB semantics? Maintaining IP
semantics is valuable. But when you extend beyond IP
semantics you are merely creating new transport definitions.
Why not just use IB, SDP or DAT?
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Yaron Haviv <[email-masked]>
Date: Sun, 08 Sep 2002 22:28:11 +0300

Caitlin,

The issue is really that TCP/IP works mostly over Ethernet
When running over IB few limitations arouse, one is the inability to do MAC
faking (some may consider it a benefit)
This has nothing to do with DAT, SDP, .. it relates to how addresses are
resolved
And by the way SDP, DAPL relay on IPoIB to provide the address resolution
capability

Just a simple example in Ethernet a backup router sends a packet with the
Master router MAC address in case of a fail-over
This automatically configures the switches to route the traffic destined to
that MAC address to the backup router
In IB there is no way to do such a thing, since the HW addresses might be
even fixed (base GID=GUID)
(VRRP protocol standardize just that over Ethernet and other mediums)
Other scenarios could be when one host wants to take over HW address on
another
this could be replaced by gratuitous ARP's, how ever they are limited in
some apps
(e.g. in case of multiple IP addresses, active/active scenarios, no client
support, ..)

The solution might be either change the IB L2, enable GID changes on the fly
+ immediate notifications to update the switches
Or it could fall within the scope of IPoIB (RC)

One solution that could work with IPoIB RC is for example in the connection
setup headers pass an alternate address
That would be used in case of failure, since RC connection setup headers are
in the scope of IPoIB RC, this is Ok

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Caitlin Bestler
Sent: Saturday, September 07, 2002 12:40 AM
To: [email-masked]
Subject: RE: [Ipoverib] recharter consideration

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 9 Sep 2002 10:55:06 -0500

On 9/8/02, Yaron Haviv wrote:

>When running over IB few limitations arouse, one is the
>inability to do MAC faking (some may consider it a
>benefit) This has nothing to do with DAT, SDP, .. it
>relates to how addresses are resolved And by the way SDP,
>DAPL relay on IPoIB to provide the address resolution
>capability
>
>Just a simple example in Ethernet a backup router sends a
>packet with the Master router MAC address in case of a
>fail-over This automatically configures the switches to
>route the traffic destined to that MAC address to the
>backup router In IB there is no way to do such a thing,
>since the HW addresses might be even fixed (base GID=GUID)
>(VRRP protocol standardize just that over Ethernet and
>other mediums) Other scenarios could be when one host
>wants to take over HW address on another this could be
>replaced by gratuitous ARP's, how ever they are limited in
>some apps (e.g. in case of multiple IP addresses,
>active/active scenarios, no client support, ..)
>
>The solution might be either change the IB L2, enable GID
>changes on the fly + immediate notifications to update the
>switches Or it could fall within the scope of IPoIB (RC)
>
>One solution that could work with IPoIB RC is for example
>in the connection setup headers pass an alternate address
>That would be used in case of failure, since RC connection
>setup headers are in the scope of IPoIB RC, this is Ok
>

I believe you are confusing an existing method (MAC
spoofing) with a requirement. The functional goal is to
allow an alternate servver to be selected automatically.

I would contend that that problem can be solved either
through the IB Communications Manager or by simply using
IPoIB to do IP to GID resolution before establishing the
connection.

MAC spoofing as a failover does not preserve an existing
connection. Path migration does. If the goal is to achieve
this sort of connection transparent as to whether an IB or
IP network is in use then SDP and DAPL would seem to be more
appropriate.

If the goal is to make a byte or message stream ULP
connection over an IB network, what benefit is there in
insisting on tunneling that connection in IP format?

If IPoIB is going to expand its charter, it should be on the
basis of solving ULP problems, not merely trying to make it
a universal solution. That becomes an effort in "fixing" IB
to be more IP like. In my opinion, the end result will be
unappealing to any target market.

One legitimate goal that has been cited was supporting a
larger MTU to allow atomic delivery of larger IP frames, at
least matching jumbo Ethernet frames. I think that RD and UC
should be considered for this goal. They meet the objective
while imposing fewer additional semantics than found in the
orignal link layer.

Caitlin Bestler
http://www.asomi.com/CaitlinBestler/
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Caitlin Bestler <[email-masked]>
Date: Mon, 9 Sep 2002 11:01:38 -0500

On 9/8/02, Yaron Haviv wrote:

>And by the way SDP, DAPL rely on IPoIB to provide the
>address resolution capability
>

SDP relies on IPoIB. DAPL MAY rely upon IPoIB.

DAPL requires that an IA Address have IPv6 characteristics.
On an IP network that may be achieved by using GIDs where
the Network IDs have been selected to be IPv6 compatible, or
through the use of IPoIB.

Caitlin Bestler

---

From: Vivek Kashyap <[email-masked]>
Date: Mon, 9 Sep 2002 16:28:41 -0700 (PDT)

<snip..>
> connection. Path migration does. If the goal is to achieve

Automatic path migration is one advantage that can be derived through
IPoIB-RC. 

> IP network is in use then SDP and DAPL would seem to be more

If the underlying connection failover is transparent, then the
solution is transparent to the upper layer - whatever it may be
IP/SDP/DAPL. How does it make one protocol more appropriate than the
other? 

> insisting on tunneling that connection in IP format?

We are talking about IP over different IB modes and the benefits that
accrue to the IP application such as APM and large MTU. You seem to be
considering replacing IP and using a different ULP. Those are
different discussions.

> unappealing to any target market.

I for one am interested in a failover of the type mentioned. It really
doesn't matter at which level it is done i.e. at IB level or in IPoIB.
If it can be done at IB level then it doesn't belong in IPoIB but if
it cannot be, then we could consider it here. Does anyone have any
solutions in either space?

> while imposing fewer additional semantics than found in the

Yes, one gets larger MTU and in the case of RC/RD Automatic path
migration. Any connection/QPN mapping to a particular service reduces
the lookup/scheduling needed in a server.

I have no objection to considering RD and UC but I don't think they
are in any way 'semantically closer' in context of IPoIB. They are all
'connection' oriented links setup using CM REQ/REP as is the case with
RC. They are point to point exactly as is IPoIB-RC and don't support
multicast. As far as IPoIB is concerned all these modes are
equivalent. If we describe the method for IPoIB over one of these
modes the method can be trivially extended to the other modes. See
section 3.0 in draft-kashyap-ipoib-advanced-capabilities-00.txt.

Vivek

-- 
Vivek Kashyap
Linux Technology Center, IBM
[email-masked] 
[email-masked] 
503 578 3422 (o)
_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib