Subject: RE: [Ipoverib] IPoIB architecture/encapsulation

From: Yaron Haviv <[email-masked]>
Date: Wed, 12 Dec 2001 11:05:33 +0200

Bill,

My personal experience is with BSD & Linux stacks since they are public
domain 
The network adapter drivers specifically put the entire Ethernet+IP headers
in the sk_buf/m_buf
From my little experience with NDIS drivers, I assume it is also the case in
Windows
(See TCP/IP Illustrated Vol 2, page 35)
You could use both offsetting, and the 2 data segments approach I mentioned
to resolve issues around it in other cases
I understand from the responses in the group that this option is gaining
wide acceptance, more than the other alternatives 

I could try and write a draft for IPoIB over RC/RD, any one who likes to
help is welcomed !

I mentioned UDP since most multimedia protocols using IP Multicast (e.g.
RTP/RTCP) use UDP, and TCP cannot use multicast by nature 
In essence I meant IP multicast   

Yaron

-----Original Message-----
From: Bill Strahm [mailto:[email-masked]]
Sent: Wednesday, December 12, 2001 9:16 AM
To: Yaron Haviv; [email-masked]
Subject: RE: [Ipoverib] IPoIB architecture/encapsulation

There are varying implementations for what the IP stack looks for.  Most of
them that I am aware are looking for a byte stream that starts with the
first byte of the IP header.  That is not to say if you did
ipstream[-ethhdrsize] you wouldn't accidentally find the beginning of the
Ethernet header.  However the stacks that I have worked with DO NOT do
things like this (and I would personally shoot any developer that did <G>)

We are currently trying to define IP over UD because it is easiest and most
supported.  Beyond this I am willing to look at proposals to do any of the
other IB transports... Are you going to write the draft ???

By the way... What is UDP Multicast ??? I am assuming you mean IP multicast
and the fact that UDP is running above it is just a coincidence...

Bill

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf
Of Yaron Haviv
Sent: Tuesday, December 11, 2001 1:26 PM
To: 'Vivek Kashyap'; [email-masked]
Subject: RE: [Ipoverib] IPoIB architecture/encapsulation

Few questions & suggestions

I'm trying to understand way not to take the straightforward approach for
the ethrtype field
And just add that before the IP header in the payload area, if alignment is
needed so align it

I might be missing something here so forgive me if I do
It looks to me like TCP stacks are used to get a packet with everything
including Ethernet header, it is not like we are doing full header/data
separation that could help the stack
On the worst case if the TCP implementation has problems with it we could
have 2 data segments in the receiving side, one for the first word, second
for the rest

Another thing that was mentioned in the recent emails was that IPoIB is
defined just for UD
In the discussion in LWG few months ago I recall few capability bits in one
of the suggestions
I personally think we should not preclude the use of RC/RD for IPoIB unicast
traffic and leave it as an option
This could be used in future enhancements to increase MTU to "Jumbo Frames",
add help avoiding the Checksum calculations

In the near future RC might assist in supporting profile A chips that
doesn't have large MTU's, and will be deployed in environments where they
provide enough value (I would have hoped to have just Profile B chips,
however they are here, with us, and probably for some time)
For some of those limited chips that also doesn't support Multicast we could
implement a multicast server that will either convert the traffic to
multicast or send it to the multiple clients, it might also proxy the ARP's
and answer based on his internal table & SA information
To be clear I don't like it !!! (see ATM), but it could do the work as an
interim solution that will help accelerate IB adoption with the given HW
I assume that someone building a multimedia solution relying on UDP
multicast will probably wont use such HW, however it might be sufficient for
ARP's, DHCP's, etc'

Yaron

_______________________________________________
IPoverIB mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ipoverib

---

From: Michael Krause <[email-masked]>
Date: Wed, 12 Dec 2001 06:05:17 -0800

At 11:05 AM 12/12/2001 +0200, Yaron Haviv wrote:
>domain The network adapter drivers specifically put the entire Ethernet+IP 
>headers
>in the sk_buf/m_buf From my little experience with NDIS drivers, I assume 
>it is also the case in
>Windows (See TCP/IP Illustrated Vol 2, page 35) You could use both 
>offsetting, and the 2 data segments approach I mentioned to resolve issues 
>around it in other cases I understand from the responses in the group that 
>this option is gaining wide acceptance, more than the other alternatives

In general, most networking stacks post receive buffers that contain 
sufficient length to accommodate all headers from all layers allowing one 
to implement a single DMA for the packet when it arrives - performance win 
for most implementations.  Each layer within the stack then parses their 
component of the packet and advances the pointer so that the next layer 
only views their headers and nothing before - agree with Bill that it would 
be unwise to look earlier through pointer arithmetic as you never know if 
the packet was part of a scatter operation (should header / data split be 
used).  Given this is a well-known paradigm, the advantages of using the 
Ethertype right after the IB headers I hope are apparent.

>help is welcomed

Would like to see the workgroup first get through the requirements and a 
common UD draft so we can get it implemented and start interop testing as 
many of us require IP over IB to deliver solutions to our customers in the 
coming months.

>defined just for UD In the discussion in LWG few months ago I recall few 
>capability bits in one
>of the suggestions I personally think we should not preclude the use of 
>RC/RD for IPoIB unicast
>traffic and leave it as an option This could be used in future 
>enhancements to increase MTU to "Jumbo Frames", add help avoiding the 
>Checksum calculations

Unless you are absolutely sure that the packet was originated and I mean 
absolutely originated within an IB subnet, one cannot just turn off 
checksum calculations just because IB provides strong end-to-end data 
integrity.  Given that endnodes may act as routers between the Ethernet / 
etc. fabrics and IB and there is no way to signal this has occurred at the 
IP layer, checksums must be calculated by all endnodes.  A smart CA vendor 
might come up with some technique to accelerate this calculation for 
software to then fix-up (much like we do checksum off-load today with 
Ethernet) but that is not part of the specification (IP or IB).

>multicast will probably wont use such HW, however it might be sufficient for

The requirements document is focused on what would be called profile B 
endnodes.  Given just the sample of the problems you note here, most people 
appear to be comfortable with staying with a profile B focus and getting 
the requirements and associated specification implemented first.  Several 
of us will then be willing to help develop the connection oriented variants 
should there be sufficient interest.

Mike