Subject: Re: [ipoverib] summerising the last week & open issues

From: [email-masked]
Date: Tue, 05 Feb 2002 20:44:40 +0900

Roy, Vivek, Bill, Yaron, .. All,

>I'm convinced that the Ethertype is not required for IP over IB.

Let me explain about "Why 'Ethertype in payload' is needed".

I know IETF/IPoIB focuses on IPv4/ARP/IPv6, but I think IETF/IPoIB
 should enable for vendors to be able to implement other protocols
 over IPoIB than IPv4/ARP/IPv6. # And we really try to do it.

I tried to categorize "protocols over IPoIB" into three a),b),and c).
# They all are what we will design(or as-is),implement(or as-is) and provide.

 a) IPv4, ARP, IPv6  - They are focused and argued by IETF/ipoib mailing-list.

    [description]

    They use IPoIB driver directly.
    Without Ethertype,
      Layer-2 (=IPoIB drver) needs to know layer-3 packet formats,
      snoops/peeks layer-3 packets, and dispatches the payload of
      received packets to layer-3 (IPv4, ARP, IPv6).
    Else
      Layer-2 doesn't need to know layer-3 packet formats and
      doesn't need to snoop/peek layer-3 packets.
      It dispatches the payload (skip Ethertype) of received packets
      to layer-3 just based on the Ethertype on the payload.

 b) RARP    - Not focused and argued by IETF/ipoib mailing-list.

    [description]

    It also uses IPoIB driver directly.
    Without Ethertype,
      "layer-2's snooping layer-3 packets method" cannot be used
      because top 4bits of a RARP packet is top of MAC address
      and so both top 4bits of RARP and ARP may be the same, i.e. zero.
      Otherwise, Can I use an additional QP may be needed only for RARP ?
      If it is possible, it would be used only for distinguish RARP.
      Is this QP allocation usage suitable for a IPoIB native style ?

      What is merit of using multiple QPs per only one interface ?
      I think more suitable usage exists for multiple QPs for IPoIB using UD.
      e.g. Load-balancing.
	   define multiple IPoIB interfaces per HCA port.
	   Each IPoIB interface uses just a QP and a different VL (if possible)
	   . Very huge traffic and parallel processing can be enabled.
    Else
      Simple. the same as a) Else.
     
 c) Multiplexer/Demultiplexer for High Availability and/or Load Balancing
	   (- Not focused and argued by IETF/ipoib mailing-list.)

    [aspect of layers of modules]
      -------------------------------------------------
                           TCP, UDP
      -------------------------------------------------
                     IPv4, ARP, IPv6, RARP
      -------------------------+
         Multiplex/Demultiplex |
         module "X"            |
      -------------------------+-----------------------
                         IPoIB driver
      -------------------------------------------------

    [aspect of IP interfaces(addresses) used by application user]

         |
     virtual IP    (which is defined to a Multiplexer/Demultiplexer instance)
      |       |
    IP(1)   IP(2)  (which are defined to two IPoIB driver instances)

      Module "X" ties two (general) IP interfaces, IP(1) and IP(2) (which are
      assigned/defined to an two IPoIB driver instances) into a bandle which is
      used as a (virtual) IP interface.
      The "virtual IP" address is assigned/defined to Module "X".
        cf. "virtual IP" is NOT so-called "logical IP". It's a real IP address.
      Module "X" provides a virtual IP
         , High Availability by multiplexing using IP(1) and IP(2).
         , and Load Balancing by multiplexing using ... IP(1) and (2).

    [description]
    They(Module "X") also use IPoIB driver directly.
    They are in-kernel modules which multiplexes/demultiplexes
    multiple IPoIB interfaces.
    cf. Multiple modules which have function like "X" exist.
        One is for load-balancing, another is mainly for high-availability..

    Without Ethertype,
      When receiving "X"'s packets, "layer-2's snooping layer-3 packets method"
      cannot be used. Because this method is based on 4bits special mark
      only on IPv4/ARP/IPv6 packet, not defined for Module "X".
      In this case, I want to dispatch this packet to Module "X" but I cannot
      judge if this received packet is for "X" or not.

      What is merit of using multiple QPs per only one interface ?
      I think more suitable usage exists for multiple QPs for IPoIB using UD.
      e.g. Load-balancing.
	   define multiple IPoIB interfaces per HCA port.
	   Each IPoIB interface uses just a QP and a different VL (if possible)
	   . Very huge traffic and parallel processing can be enabled.
    Else
      Simple, too. the same as a) Else.
      When receiving "X"'s packets, I can dispatch the "X" received packet.
      And Module "X" can handle it based on the packet contents as it likes.
      Module "X" processes the packet by itself or dispatch it to upper layer
      (IPv4,ARP,IPv6),or do any ohter thing.

>But if there are folks who want to take advantage of a traditional
>differentiator that's always been available, it seems reasonable
>to accomodate them. Making up for the lack of the LLC/SNAP
>header field that does this is the only way to retain the same
>demuxing paradigm that's been used since the 70s.
>Some comments are also embedded below:
>
>RR
>----- Original Message ----- 
>From: "Yaron Haviv" <[email-masked]>
>To: <[email-masked]>; <[email-masked]>
>Sent: Thursday, January 31, 2002 12:19 PM
>Subject: RE: [ipoverib] summerising the last week & open issues
>
>
>> Lets review your arguments:
>
>Their "argument" is that their desire to mux protocols in the fashion
>that has been traditionally supported and that they would like this support
>to continue in IP over IB. I believe that this is not an argument, but a
>statement of fact.
>
>> You want to use Ethertype to do non IPoIB stuff (even without the ethertype
>> you could support future IP stuff like maybe IP v 8)
>> This is not the focus of this group, even if you do that than it might not
>> work with other implementations that work strictly with IP
>
>It's a no-brainer that non-IP protocols won't get handled properly 
>where there are not implemented. I doubt that anyone expects or 
>wants different. They do want the "option" to support those protocols.
>
>> Regarding the second argument you could use different QP's in order to have
>> multiple IPoIB networks
>> And even then having Ethertype field doesn't help you multiplexing things
>> beyond layer 2
> 
>I doubt that anyone wants any help in demuxing beyond layer 2. They do
>not "want" to uses different QPNs since that is a different demuxing method
>than they use today.

I use multiple QPNs for load-balancing.
      e.g. Load-balancing.
	   define multiple IPoIB interfaces per HCA port.
	   Each IPoIB interface uses just a QP and a different VL (if possible)
	   . Very huge traffic and parallel processing can be enabled.

>> The reason for saying it's a philosophical question is that for the things
>> defined in the IPoIB charter (IP only)
>> You don't need the Ethertype
>
>They want the Ethertype. It provides a value for them. 

Yes.

>> Yaron
>> 
>>  -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]]  On Behalf
>> Of [email-masked]
>> Sent: Thursday, January 31, 2002 3:30 AM
>> To: Yaron Haviv
>> Cc: 'Bill Strahm'; [email-masked]'
>> Subject: Re: [ipoverib] summerising the last week & open issues
>> 
>> All,
>> 
>> >Message-ID: <20083A3BAEF9D211BDB600805F8B14F301CBF1F0@NTSERVER>
>> >From: Yaron Haviv <[email-masked]>
>> >To: "'Bill Strahm'" <[email-masked]>
>> >Cc: "[email-masked]'" <[email-masked]>
>> >Subject: [ipoverib] summerising the last week & open issues
>> >Date: Wed, 30 Jan 2002 15:13:52 +0200
>> >
>> >It looks to me like we had a lot of progress done in the last 2 weeks
>> >I will try and summaries the issues, and status, to enable people to
>> respond
>> >on the last issues
>> 
>> >4. Ethertype field in data (need to close)
>> >many people has mentioned that this issue is not critical, and doesn't harm
>> >anyone's ability to implement a solution
>> >it is more of a philosophical debate around strict layering vs. efficiency
>> >(and maybe also possibility to enable Ethernet over IB one day)
>> >since it is not a critical issue I would tend to go with the Author (Vivek)
>> >and leave it as it is specified already
>> 
>> If "ethertype' arguments was summarized so,
>> we would have some critical problems.
>> 
>> We don't think these debate/argument as philosophical.
>> It effects our InfiniBand product development.
>> Without ethertype in payload, probably other vendors can't implement other
>>  protcols (than IPv4/v6/ARP) on IPoIB neither.
>> 
>> Of course, as Vivek wrote, this mailing-list focuses ipv4/ipv6/arp.
>> But I think IPoIB is infrastructure for network applications/protocols
>>  over InfiniBand environment.
>> If Ethertype exists in payload, many other developers could also
>>  use this IPoIB protocol to implement many applications to realize many
>> market
>>  requirements. If not, vendors can't do so.
>> 
>> So we absolutely need ethertype field in payload (by using IBTA 1.0a. not
>> 2.0).
>> 
>> For example, we implement some "protocols" over IPoIB framework, such as
>>  'multiplexing software (between arp/ipv4/ipv6 and ipoib driver) which uses
>>   multiple IPoIB interfaces',
>>  'cluster nodes patrol software which uses multiple IPoIB interfaces.'
>>  'rarp',etc.
>> 
>> They all are implemented by IPoIB driver's using ethertype in payload and
>> IPoIB driver's dispatching protocols by ethertype value.
>> If info is only 4bit in arp/ipv4/ipv6 packet itself, we cannot implement
>> any other protocols/applications over IPoIB at all.
>> 
>> >this one to my understanding is the only real open issue right now
>> >I hope we finalize it and get a consensus quick
>> >
>> >Thanks, Yaron
>> 
>> Thanks,
>> ----------------------------------------------------------------------------
>> --
>> Akihiko Imai/Development Dept.5, Infrastructure Systems Division, Fujitsu
>> Ltd.

Thanks,
  Akihiko
------------------------------------------------------------------------------
Akihiko Imai/Development Dept.5, Infrastructure Systems Division, Fujitsu Ltd.
  e-mail: [email-masked]  TEL: +81-044-754-3456 (ext)7113-8706
  address: 4-4-1 Kamiodanaka Nakahara-ku, Kawasaki 211-8588, Japan

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib