Subject: RE: [Ipoverib] RE: ARP vs SIDR

From: "H.K. Jerry Chu" <[email-masked]>
Date: Mon, 11 Mar 2002 10:22:45 -0800 (PST)

Roy,

>The use of SIDR has been seriously considered and debated. I personally
>had a number of concerns about SIDR. It looks like SIDR may not
>be universally available. It may not be available early enough during
>system boot for RARP to work. It's not clear whether SIDR will work
>across IB routers. All these reasons (and saving a trip to CM is not
>on the list) made me hesitate to include the use of SIDR as a
>requirement.
>
>/rkl
>Don't these concerns equally apply to RC/UC transports?  The concern as to
>whether connection management MADs can cross subnet boundaries caused me to
>start another thread related to consistent IPoIB link boundaries.
>rkl/

But RC/UC are the *optional* transports for IPoIB. There will be many
other cases when RC/UC are advertised but the node can't really support
them due to, e.g. temporary resources shortage. In that case one must fall
back to UD, and we must make UD bullet proof to ensure the basic
mechanism is always there. Requiring SIDR for UD adds risks.

What do you think about the idea of making the use of SIDR (and QPN)
optional for UD as described below?

Jerry

>
>              I've adopted a design principle to stick as close to
>the existing, proven IP mechanism as possible. SIDR doesn't seem
>to offer much value to the basic IPoIB/UD. (Note that it doesn't
>help the EtherType issue as the latter proposes to use a QP more
>like an "interface endpoint", than like a "service endpoint".)
>
>Of course SIDR can be very useful beyond the basic IPoIB/UD. E.g.
>Vivek's advanced capability proposal is mainly based on SIDR. But
>the key is that the use of SIDR is *optional" thus the risk is
>much reduced.
>
>What I proposed a couple of months ago was to allow the QPN field
>in the hardware/link-layer addr to be zero. This signifies to the
>requrester to use SIDR to resolve QPN. The motivation is to make
>the "MAC address" more persistent by removing the volatile component
>- QPN.
>
>Ideally each node is allowed to decide whether to include QPN
>or not in its hardware/link-layer addr. The only requirement is
>that a node must include QPN in its hardware/link-layer addr
>exposed to a remote node when either side does NOT support SIDR.
>
>But I fail to figure out how to make the above work within the
>ARP/ND protocol. So I have not pursued this idea further. If
>someone else can solve this puzzle I'm certainly interested.
>
>Jerry
>
>>Jerry,
>>
>>I wouldn't be in favor of providing optional address resolution methods.
>>One is plenty and I can live with the currently defined method.
>>
>>Had I been involved when this topic was first explored, my preference would
>>have been to do address resolution something similar to what you outlined
>>below.  Use broadcast ARP/ND to discover the endpoint GID (actually I am in
>>the camp that prefers to stay within the IB subnet so LID would have worked
>>for me to keep hardware address length reasonable) then use SIDR to lookup
>>the LSAP of interest.
>>
>>For IPoIB, the ServiceIDs would have been derived from the well known
>>ethertypes.  Enhanced transport capabilities, other than UD, could be
>>derived with a bit oring function to construct their well-known ServiceIDs
>>so we have one consistent method for obtaining a QPN.  The private data
>>field provides a mechanism for returning useful information associated with
>>an enhanced transport.  It keeps the choice of the number of QPs a node
>>wants to use for IPoIB/UD a local decision.  Since ARP is involved, the OS
>>network stack is still handling the ARP/neighbor cache and its semantics.
>>Because it's ServiceID/LSAP based, this is a general facility that could be
>>used for other non-IP protocols.  
>>
>>I realize this would involve three transactions, ARP+SIDR+PathRecord
>lookup.
>>But, ARP is a relatively low frequency event on a low latency / high
>>bandwidth link and this is only three out of may thousands of transactions
>>with the end node so it's lost in the noise.  If someone felt they could
>>implement the current address resolution scheme without doing a PathRecord
>>lookup, then this would definitely add complexity.  Since I don't know how
>>to do that in a generalized fabric, the extra SIDR adds little complexity
>to
>>my design.
>>
>>I want to be clear.  This is not the start of a campaign to change the
>>current ARP method.  If folks found it compelling, that would be cool but
>it
>>was not meant to spark public debate.  It just provides some backdrop to my
>>views on encapsulating the LSAP in the IPoIB header given the current APR
>>method.
>>
>>Roy
>>
>>
>>-----Original Message-----
>>From: H.K. Jerry Chu [mailto:[email-masked]]
>>Sent: Friday, March 08, 2002 4:39 AM
>>To: [email-masked]
>>Cc: [email-masked]
>>Subject: RE: ARP vs SIDR
>>
>>
>>Roy,
>>
>>To use SIDR one must obtain the remote IB node address (GID or LID)
>>first. THerefore an "IP to IB" address resolution protocol is still
>>required. (It'd be a two steps resolution process: ARP/ND for GID
>>followed by SIDR for UD-QPN.)
>>
>>I've proposed the *optional* use of SIDR for resolving the UD-QPN. This
>>way we can exclude QPN from the "IPoIB link layer addr" thereby
>>providing some additional property such as static hardware addr
>>(one that doens't change across system reboot). But I couldn't figure
>>out a way to make the use of SIDR truly optional (e.g. if node A decides
>>to use SIDR, it may require all the remote nodes communicating with
>>itself to also use SIDR...)
>>
>>If enough people think the above is useful and the requiring of SIDR
>>for IPoIB is acceptable, we can still pursue it. (It only requires
>>a flag "use-SIDR" being added to the hardware addr.)
>>
>>Rgds,
>>
>>Jerry
>>
>>>From: "Larsen, Roy K" <[email-masked]>
>>>To: "'H.K. Jerry Chu'" <[email-masked]>
>>>Cc: [email-masked]
>>>Subject: RE: ARP vs SIDR
>>>Date: Thu, 7 Mar 2002 09:34:40 -0800 
>>>MIME-Version: 1.0
>>>
>>>But then, 20 byte hardware addresses, PathRecord lookups in the ARP path,
>>>and peeking into network headers don't fit well in existing network stacks
>>>either.  But alas, I joined the reflector too late to participate in the
>>>early address resolution discussion, and as my mail said, I was *not*
>>>attempting to have this group change its address resolution method.  Just
>a
>>>cautionary note about straying from long standing network practices and
>>>underlying link architecture.
>>>
>>>Roy
>>>
>>>-----Original Message-----
>>>From: H.K. Jerry Chu [mailto:[email-masked]]
>>>Sent: Wednesday, March 06, 2002 6:19 PM
>>>To: [email-masked]
>>>Cc: [email-masked]
>>>Subject: ARP vs SIDR
>>>
>>>
>>>Roy,
>>>
>>>I believe we've made a conscientious choice to use ARP/ND instead of
>>>SIDR in order to be more compatible with IP semantics and implementations.
>>>E.g. if we were to use SIDR instead of ARP, one would have to simulate
>>>ARP flushes (gratuitous ARP) and proxy ARP in SIDR. It can be quite
>>>a headache. It may also require a lot of changes to the existing IP
>>>stack implementation to support IPoIB, which is unacceptable to
>>>most OS vendors.
>>>
>>>Jerry
>>>
>>>>> > At 11:28 AM 2/26/2002 -0800, Vivek Kashyap wrote:
>>>>> > >         Please comment soon so that the encapsulation draft can be
>>>>> > >         updated and submitted before the cut off of march 2nd.
>>>>> > >
>>>>> > >         For that matter please send in your comments on 
>>>>> the draft(s)
>>>>> > >         in general in the next couple of days so that they can be
>>>>> > >         included before the cut off.
>>>>> > 
>>>>> > For a UD-based IP over IB implementation, Ethertype isn't 
>>>>> required if the 
>>>>> > rest of the IB infrastructure has its act together to allow 
>>>>> non-IP services 
>>>>> > to be properly identified. Hence, I don't have a problem 
>>>>> with it not being 
>>>>> > included within the specification.
>>>>> > 
>>>>> > Mike
>>>>> 
>>>>> Yes, you are right. Ethertype is also not required for RC (or other
>>>>> mode) IP over IB implementations.
>>>>
>>>>But of course, I would assume we wouldn't be using ARP for RC (or other
>>>>modes) either so I'm not sure how this is relevant.  ARP is at the root
>of
>>>>this "evil".  The IBTA defined SIDR to deal with the situation but we
>>chose
>>>>not to use it.  For connected modes, we'll have to use well-known service
>>>>IDs.  So the logical question readers of our work might ask is, if
>>>well-know
>>>>service IDs are be good enough for all the other IPoIB modes, why aren't
>>>>they good enough for UD?  
>>>>
>>>>Now don't get me wrong. I'm *not* asking the group to consider changing
>>the
>>>>UD address resolution mechanism at this late date.  Just pointing out
>that
>>>>when you stray from using the services provided by the link architecture,
>>>it
>>>>causes some awkward and inconsistent design choices.  I think
>>encapsulating
>>>>the LSAP designated by the sender is a way of redeeming the choice of ARP
>>>>because it means the IPoIB protocol becomes self-contained and doesn't
>>rely
>>>>on peeking and interpreting user payload data.
>>>>
>>>>Roy
>>>>
>>>>> 
>>>>> Vivek
>>>>> 
>>>>> > 
>>>>> > 
>>>>> > _______________________________________________
>>>>> > IPoverIB mailing list
>>>>> > [email-masked]
>>>>> > https://www1.ietf.org/mailman/listinfo/ipoverib
>>>>> > 
>>>>> 
>>>>> 
>>>>> -- 
>>>>> Vivek Kashyap
>>>>> Linux Technology Center, IBM
>>>>> [email-masked] 
>>>>> [email-masked] 
>>>>> 503 578 3422 (o)
>>>>> 
>>>>> _______________________________________________
>>>>> IPoverIB mailing list
>>>>> [email-masked]
>>>>> https://www1.ietf.org/mailman/listinfo/ipoverib
>>>>> 
>
>
>
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib