Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4

From: Ajoy C Siddabathuni <[email-masked]>
Date: Tue, 16 Apr 2002 19:41:30 -0700 (PDT)

>4 byte header:Signal is 4 bytes but intelligence is only in 2
>bytes(ethertype). Only half of the signal provides any information.
>
>LLC/SNAP: Signal is 8 bytes but intelligence is only in 2 bytes. So we get
>25% return. We have added another 4 bytes with no increase in intelligence.
>
>Hence a '4-byte ethertype' is better than the 8 byte LLC/SNAP.

The conclusion drawn here does not consider all the discussion that was put forth
in the last msg of the thread.

If you consider what would be extra over head of 8 bytes verses 4 bytes aganist
an MTU of 2k. It is negligible or nothing.

I think one must get outside the "try to save some bytes here and there" line
of thinking when defining wire protocols and make sure that you dont have to
write another rfc next year to obsolete this one. All those RFCs that defined the
LLC/SNAP headers in them were not mis-guided, would you think? You would'nt endup
with an IB header of upto 68 bytes for just UD if saving bytes from the header
was the goal. Recall that at wire speeds of 2-10Gbps these should not matter.
Things like alignment, cache-line sizes, receive queue depths and page-sizes are
the bigger fishes to fry at those rates.

~Ajoy

>
>Also, I don't think you are right in dismissing non-LLC/SNAP use. LLC/SNAP
>is not the favored mode in general - just look at 802.3 vs. DIX ethernet.

>                      Ajoy C                                                                                                       
>                      Siddabathuni             To:       
[email-masked], [email-masked]                          
>                      <Ajoy.Siddabathun        cc:       [email-masked], 
[email-masked], [email-masked],            
>                      [email-masked]>                [email-masked]                                                                  
>                      Sent by:                 Subject:  Re: [Ipoverib] 
consensus call on EtherType, min MTU and DHCPv4            
>                      ipoverib-admin@ie                                                                                            
>                      tf.org                                                                                                       
>                                                                                                                                   
>                                                                                                                                   
>                      04/15/2002 03:52                                                                                             
>                      PM                                                                                                           
>                      Please respond to                                                                                            
>                      Ajoy C                                                                                                       
>                      Siddabathuni                                                                                                 
>                                                                                                                                   
>                                                                                                                                   
>
>
>
>
>
>>It takes another 4 bytes and provides no new information. The issues
>>you indicate above are equally addressed by a 'type' field.
>
>Ofcourse it provides no new information - if you are running IPoIB
>in vacum. It is much cleaner than putting a 4 byte header with reserved
>fields. Also expect (in a few years) that there will be multi-protocol
>routers that will be sitting at the edges of an
>IB-Ethernet-osi-apple-network.
>They would definetly like to handle packets in a std way - especially when
>cp'ing pkts from one kind of "link" to another. They dont need to handle
>cases such as - "oh by the way, if this link was an IPoIB link..I needed to
>strip the top 4bytes...and not interpret the next 16 bits before I look at
>the eth-type".
>
>>
>>'No header case' is disallowed as per WG consensus. Otherwise, the
>>4-byte ethertype field is equally viable as the 8 byte LLC/SNAP field.
>>It has the advantage of being 4 bytes smaller.
>
>I would'nt consider "4 bytes smaller" as an advantage. Especially if you
>are
>working with IB header lengths upto 68 bytes. And then if you are working
>with
>TCP/IPv4/v6 add upto 60+.
>
>>
>>How about RFC2734 ? There is nothing standard about this.
>Well - time to learn from another's mistakes.
>
>Here is my summary -
>
> - Having no-header at all will give you alignment. No ethtype to mux on.
>   But you could have always figured out all IP'ish types from the
>IP-hdrtype
>   field.
>
> - Having 4byte "non-std" header will also give you alignment. Gives you
>   all the info you need. But yes, there are certain non-std elements in
>   the header that implementations would have to be aware of  - especially
>   if they were porting an existing stack.
>
> - Having 8byte std-header will be standards conforming for future
>multi-prot-
>   col enviroments. Ofcourse, retains alignment.
>
>~Ajoy
>
>
>>>
>>> Frame formats -
>>>
>>> a) If LLC/SNAP was used then you automatically get a 32bit-word
>alignment
>>>    for the IP-Datagram.
>>>
>>>    The frame would look something like --
>>>
>>>    LRH (8bytes)
>>>    GRH (40 bytes)*
>>>    BTH (12 bytes)
>>>    DETH (8 bytes)
>>>                LLC       OUI     EthType
>>>    LLC/SNAP <AA-AA-03><00-00-00><xx-xx>   (8bytes)
>>>    IPv4/IPv6 Datagram
>>>    ICRC
>>>    VCRC
>>>
>>> b) Embedding just the  ethtype - the frame would look like -- also gives
>>>    you the alignment - but is a non-std way of doing things (with
>reserved
>>>    fields etc in the format).
>>>
>>>    LRH (8bytes)
>>>    GRH (40 bytes)*
>>>    BTH (12 bytes)
>>>    DETH (8 bytes)
>>>
>>>     Rsvd   EthType
>>>    <??-??><xx-xx>  (4bytes)
>>>    IPv4/IPv6 Datagram
>>>    ICRC
>>>    VCRC
>>> ----
>>>
>>> My opinion on this is that if EthType is ever supported, it should be
>>> supported using the std mechanism (a) to be consistent with how other
>>> IPoverXXX protocols have been defined in the past. (See RFCs -ATM: 2225,
>>> 1577, 1483 SMDS:1209 FR:1294)
>>
>>How about RFC2734 ? There is nothing standard about this.
>>
>>[ As another example one can even use a 2 byte header if it works out
>>for the hardware -i.e. if one can DMA to a 2 byte boundary, the IP
>>header will land at a 4 byte boundary. ]
>>
>>Vivek
>>
>>>
>>> If the WG feels that in the foreseeable future, only IP will ever be
>>> supported, then yes there is no need for the Ethtype - ie. leave it
>alone,
>>> you can still get the IP datagram 32bit aligned right after DETH. On the
>>> other hand, if the WG feels that we do it the way our ancestors have
>done it
>>> there is enough argument already in place for supporting llc/snap hdrs.
>>>
>>> ~Ajoy
>>>
>>>
>>> >X-Sender: [email-masked]
>>> >Date: Mon, 15 Apr 2002 10:20:49 -0700
>>> >To: Yaron Haviv <[email-masked]>, "'H.K. Jerry Chu'"
>>> <[email-masked]>, "[email-masked]'" <[email-masked]>,
>>> "[email-masked]'" <[email-masked]>
>>> >From: Michael Krause <[email-masked]>
>>> >Subject: RE: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4
>>> >Mime-Version: 1.0
>>> >X-Mailman-Version: 1.0
>>> >List-Id: IP over InfiniBand WG Discussion List <ipoverib.ietf.org>
>>> >X-BeenThere: [email-masked]
>>> >
>>> >At 10:31 AM 4/14/2002 +0200, Yaron Haviv wrote:
>>> >>Jerry,
>>> >>
>>> >>We must stop expending the definition somewhere so we get the spec out
>>> >>This is IPoIB we are talking about, there is no need for LLC/SNAP, and
>we
>>> >>just need the Ethertype
>>> >
>>> >The consensus is that people want to have the full range of Ethertype
>>> >values supported.  As such, this needs to be specified independent of
>the
>>> >IP over IB specification so that everyone may use these values without
>>> >requiring them to implement IP over IB.
>>> >
>>> >Mike
>>> >
>>> >
>>> >_______________________________________________
>>> >IPoverIB mailing list
>>> >[email-masked]
>>> >https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>>
>>> _______________________________________________
>>> IPoverIB mailing list
>>> [email-masked]
>>> https://www1.ietf.org/mailman/listinfo/ipoverib
>>>
>>
>>
>>--
>>Vivek Kashyap
>>Linux Technology Center, IBM
>>[email-masked]
>>[email-masked]
>>503 578 3422 (o)
>
>
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib
>
>
>
>
>
>_______________________________________________
>IPoverIB mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/ipoverib

_______________________________________________
IPoverIB mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipoverib

---

From: "Bill Strahm" <[email-masked]>
Date: Tue, 16 Apr 2002 22:51:08 -0700

I don't agree with the consideration that "Its just 4 bytes out of
thousands"

Like the politicians say "A million here, A million there, next thing you
know we are talking about real money"

Every byte matters.  If a byte doesn't add value why have it...  I am
comfortable with a 4 byte Ethertype because that maintains DWORD alignment,
and is not horrible... I get concerned that people who won the 4 byte
debate, now want to start talking about 8 bytes... Where next 16, 32 ???

I don't see a full LLC/SNAP header adding any value, so why have it then ?

-----Original Message-----
From: Ajoy C Siddabathuni [mailto:[email-masked]]
Sent: Tuesday, April 16, 2002 7:42 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked];
[email-masked]; [email-masked]; [email-masked];
[email-masked]; [email-masked]
Subject: Re: [Ipoverib] consensus call on EtherType, min MTU and DHCPv4

>Hence a '4-byte ethertype' is better than the 8 byte LLC/SNAP.

The conclusion drawn here does not consider all the discussion that was put
forth
in the last msg of the thread.

If you consider what would be extra over head of 8 bytes verses 4 bytes
aganist
an MTU of 2k. It is negligible or nothing.

I think one must get outside the "try to save some bytes here and there"
line
of thinking when defining wire protocols and make sure that you dont have to
write another rfc next year to obsolete this one. All those RFCs that
defined the
LLC/SNAP headers in them were not mis-guided, would you think? You would'nt
endup
with an IB header of upto 68 bytes for just UD if saving bytes from the
header
was the goal. Recall that at wire speeds of 2-10Gbps these should not
matter.
Things like alignment, cache-line sizes, receive queue depths and page-sizes
are
the bigger fishes to fry at those rates.

~Ajoy

---

From: Vivek Kashyap <[email-masked]>
Date: Tue, 16 Apr 2002 23:49:38 -0700 (PDT)

> in the last msg of the thread.

	See below.

> an MTU of 2k. It is negligible or nothing.

        I have suggested that there is no need to have an overhead
        when it provides no intelligence. Whether the overhead is
        negligible or not, it should have a reason to exist.

> write another rfc next year to obsolete this one. All those RFCs that defined the

	And why will it become obsolete ? 

> LLC/SNAP headers in them were not mis-guided, would you think? You would'nt endup

        Oh c'mon! you are the one who belittled the most recent RFC in
        the list: RFC2734. You exhorted us to "learn from others'
        mistakes" regarding this RFC. My comment was that just because
        some RFCs used LLC/SNAP, it does not make it a standard but
        just one way of doing things.

        And I do think that LLC/SNAP is useless - the 3 bytes of LLC
        provide nothing. Then you have 3 bytes of OUI before getting
        to the type field. Those 6 bytes might as well not exist.

> was the goal. Recall that at wire speeds of 2-10Gbps these should not matter.

        Just because one can afford to be lax doesn't mean one should
        be. What good is deliberately adding bytes that provide
        nothing? 

> the bigger fishes to fry at those rates.

        Good point. And how are all these helped by adding unnecessary
        overhead? 

> >is not the favored mode in general - just look at 802.3 vs. DIX ethernet.

        You mentioned working with multiprotocols in 'edge routers' -
        what about the ubiquitous DIX ethernet ? No header at all,
        what of LLC/SNAP! You mentioned ATM - as far as I know it
        suggests that the encapsulation format is negotiable. So, you
        can work with or without LLC/SNAP. LLC/SNAP doesn't seem to be
        the common encapsulation format.

        I, for one, need a lot more convincing before I can support
        the idea of LLC/SNAP being the common encapsulation in the
        future.  

Vivek

---

From: Michael Krause <[email-masked]>
Date: Wed, 17 Apr 2002 06:27:55 -0700

>I don't see a full LLC/SNAP header adding any value, so why have it then ?

2/3 of IP packets are less than 256 bytes (1/3 are less than 50 bytes) of 
payload so this continual adding of, IMO, unnecessary overhead creates more 
wire inefficiency for the bulk of traffic and increases the chance of 
crossing into another cache line for various workloads which directly 
impacts the memory controllers, increases cache pollution, etc.  The whole 
argument of link bandwidth is so high so this isn't a problem is 
irrelevant.  It isn't link bandwidth that is the problem within any system 
- it is memory bandwidth that is and will remain potentially for all time 
the primary bottleneck.   Any new communication technology that does not 
focus on memory bandwidth as one of its primary concerns is not a well 
conceived and potentially not a well used technology.

Mike

---

From: Ajoy Siddabathuni <[email-masked]>
Date: Wed, 17 Apr 2002 08:59:14 -0700 (PDT)

> 	And why will it become obsolete ? 

Well, if the router vendors come along and discover that it is so
inefficient to convert frames from one format to another - they
might just write a new one.

>         just one way of doing things.

It is a std. when most protocols defined so far chose to do it in that
"one way of doing things" - what more is needed to make a std.
There are already  'n' different standards out there - my  point was to
stick with the one with the highest prevalence and not invent a new one.
I think the WG is also required to look and understand existing  IP encap-
sulation protocols that are in wide use before going ahead and defing a
new format/protocol. I am sure the WG will need to refer back and forth
to the those RFCs when we get to the IPoverIB(RC) - UD is a nobrainer
relative to the complexities of RC.

Most IPoverXXX protocols defined so far went by the book to accomodate
that format. There may be 1 or 2 exceptions, I just dont want us to go
down the path of those exceptions.

>         to the type field. Those 6 bytes might as well not exist.

My point has been that it is, yes, useless when you are only concerned
about IPoverIB and not even consider how this meshes with a multi-protocol
enviroment.

You might come around and tell me, we will deal with it when it becomes
a problem. That is the time for the new rfc.

>         be. What good is deliberately adding bytes that provide

IBA already provides a Raw-Datagram format with a RWH, it seems like we
made one full circle to get there - why not just use raw-datagrams  and
get it over with it.

>         the common encapsulation format.

Infact, LLC/SNAP is the default encapsulation format for Classical IP over
ATM. Also see the other RFCs that I pointed out. This had a lot to do
with how all edge devices such as routers and bridges like to see commonly
understood formats in thier internal memory structures. Beyond that I am
not allowed to speak, the RFCs can speak for themselves.

~Ajoy