Subject: RE: [ipoverib] resolving the Ethertype issue

From: "Larsen, Roy K" <[email-masked]>
Date: Fri, 25 Jan 2002 16:16:03 -0800

Yaron,

I believe I addressed your comments in a response to Paul with the exception
of #3.

In the OSs I work with, none of the link drivers wonder into the buffer
beyond their own header.  For example, the Ethernet header includes the
len/type field which we have been referring to as the "ethertype".  It
belongs to the link layer to specify the destination protocol of the
payload.  Supplying destination protocol identifiers so a packet can be
routed from service provider to service user within the network stack is
basic networking 101.  All protocol layers use this mechanism which is why
I'm a little surprised at the resistance this concept is getting from the
list.  The network stack of interesting OSs have simple and elegant
mechanisms to "strip" protocol headers as it traverses the stack so it can't
be a complexity issue.

Roy

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Friday, January 25, 2002 2:21 PM
To: 'Larsen, Roy K'; [email-masked]
Subject: RE: [ipoverib] resolving the Ethertype issue

Roy, similar arguments like Paul, see embadded

 -----Original Message-----
From: 	[email-masked] [mailto:[email-masked]]  On Behalf
Of Larsen, Roy K
Sent:	Friday, January 25, 2002 10:36 PM
To:	[email-masked]'; [email-masked]
Subject:	RE: [ipoverib] resolving the Ethertype issue

Although my feelings aren't strong enough to drag the reflector into a never
ending rat hole, I believe the IPoIB draft should specify a
protoco/"Ethertype" as the first field of the payload which is treated as an
IPoIB protocol specific field.  The reasons are as follows:

1.  Interesting OSs require the link driver to identify the destination
protocol of received frames.
<yh> since you know its IP, you could fill it out

2.  QPNs do not provide a clean mechanism to do this because, at a minimum,
both ARP responses and IPv4 frames must share the same QPN.
<yh> the idea Vivek suggested is to put zero in the IP ver field in cases of
an ARP, this resolves the problem

3.  Requiring the IPoIB driver to peek into an upper-level protocol frame is
architecturally displeasing. (Read that as an euphemism for KLUDGE.)
<yh> also Ethernet IP drivers do some of it today, not sure I get what you
mean as upper-level in this case

4.  Arguments that some day the IB header may contain such a field are
irrelevant in defining a protocol today.  (Same reasoning we've seen for IB
subnet routing issues.)
<yh> this was not assumed, just was argued that it might also solve the
problem of limiting it to IP (& not also other Ethernet stuff)

Roy

-----Original Message-----
From: Yaron Haviv [mailto:[email-masked]]
Sent: Friday, January 25, 2002 9:46 AM
To: [email-masked]
Subject: [ipoverib] resolving the Ethertype issue

As Arvind mentioned the Ethertype field issue is one of the last items to
agree up on

Currently there are 2 proposals:
1.	(Vivek) no need for ethertype, we could see its an ARP from the
first
fields in the Packet (IP Ver)
2.	put the Ethertype field in the first few bytes of each message

I personally don't see it as a critical decision choosing 1 or 2, the only
thing that Vivek proposal might not enable is doing Ethernet over IB which
is not in the scope of this RFC anyway
I would tend to agree with Vivek for the simplicity reasons (not the
performance ones)

If any one has a firm reason why to put the Ethertype field in the payload
please make your arguments
And it would help in settling this matter too
Please try to give real reasons in this case too, and not things like "it is
better", etc'

Yaron

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Koji takahara <[email-masked]>
Date: Mon, 28 Jan 2002 14:08:20 +0900

I strongly agree with Roy.

I think the best method to identify the destination protocol is
the following.
(1) to specify a Ethertype as the first field of the payload

If we do not use method (1), we have to use one of the following
methods.
(2) to use different QPNs for different Ethertypes, or
(3) to peek into an upper-level protocol frame in link layer

I also think both (2) and (3) are not clean.

And I think we should not assume Ethertypes are only ARP, IPv4,
IPv6.
That assumption restricts the expantion of ther upper-layer of
the ipoverib link layer driver.

_______________________________________________________________
Koji Takahara  Email: [email-masked]
Development Dept.5,  Infrastructure Systems division,
Computer Group, Fujitsu Limitted

---

From: "Bill Strahm" <[email-masked]>
Date: Mon, 28 Jan 2002 00:24:20 -0800

Comments inline...

I guess my big question is why are people afraid of allocating QPN for a
given use of Infiniband.  I thought that was the purpose of QPN.

Can someone tell me how they can redirect packets to the correct Layer 3
(with IB UD being a Layer 2 protocol for this discussion) using Ethertype
without incurring either an additional copy, or a page flip ?  The way I see
it is you will have to install a software "protocol demuxer" above the IB
layer to receive the IB frame, look at the first 4 bytes and then put it on
the appropriate protocols receive buffer.  I am most interested in listening
to peoples implementations that involve zero-copy going up the TCP stack.

Bill

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Koji takahara
Sent: Sunday, January 27, 2002 9:08 PM
To: [email-masked]
Subject: RE: [ipoverib] resolving the Ethertype issue

I think the best method to identify the destination protocol is
the following.
(1) to specify a Ethertype as the first field of the payload
<Bill>
How do you propose handling the case where an Layer 3 (for IB that is)
protocol does not put a n-byte (I assume 4-byte) header in the message.  In
that case you CAN NOT determine if you are looking at a "EtherType" or the
first 4 bytes of a payload therefore...
</Bill>
If we do not use method (1), we have to use one of the following
methods.
(2) to use different QPNs for different Ethertypes, or
<Bill>
I claim you will have to do this anyway
</Bill>
(3) to peek into an upper-level protocol frame in link layer
<Bill>
You consider looking at the first 4 bytes of the ULP acceptable, yet here
you claim it is unclean... which is it ???
</Bill>
I also think both (2) and (3) are not clean.

And I think we should not assume Ethertypes are only ARP, IPv4,
IPv6.
That assumption restricts the expantion of ther upper-layer of
the ipoverib link layer driver.
<Bill>
Correct.  I claim option 2 is the way to go.  Then you don't have to worry
about forcing all other encapsulations to include a 4 byte header in their
framing, I do not see a way of enforcing this anyway..

What are peoples objections to using a small number of QPN to handle upper
layer protocols on top of Infiniband.  My assumptions are QPN are very
similar to Ethertypes except they can be dynamic...
</Bill>

Billhttps://www1.ietf.org/mailman/listinfo/ipoverib

---

From: Yaron Haviv <[email-masked]>
Date: Mon, 28 Jan 2002 11:13:32 +0200

Bill,

No copy is/should be done between L2-L3, you just pass the buffer pointer
(sk_buff in linux, m_buff in BSD)
The mandatory copy is between the kernel and user space (TCP/IP private
buffers to Application receive buffers)
If an application can work with the packets when they are in the TCP/IP
private buffers chains (without doing a receive(buf) op for example) so it
could avoid the copy, usually it is not so simple, might work better for
kernel Apps
Another approach is to build TOE NIC's which place the data directly in the
application receive buffers (where the private buffers are on the NIC, no
magic)

The protocol demuxing relates to the execution process
The L2 look @ the protocol number, and calls the relevant protocol function
(or soft IRQ) based on the value, this could also be done from L3 to L4

By the way adding more QP's doesn't solve any problem, it just requires more
resources (I had a long email on that in the pass)

My take on the Ethertype issue is that who ever needs it when its not
provided could generate it for him self
Where who ever wants to ignore it when its available could also do that
This is why I don't think it is a critical decision if we do have it or we
don't, since the raw information is provided
It is up to the group to see if someone feels (and could also explain) that
the Ethertype provides him an Info he must have
And not just for future Ethernet over IB stuff since we deal with IP now

I personally could sleep well with the two decisions with/without, I just
want to resolve it quick

Yaron

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Bill Strahm
Sent: Monday, January 28, 2002 10:24 AM
To: [email-masked]; [email-masked]
Subject: RE: [ipoverib] resolving the Ethertype issue

Comments inline...

Bill