Subject: [nvo3] comment on herbert-gue-01

From: Lucy yong <[email-masked]>
Date: Mon, 10 Mar 2014 21:51:07 +0000

Hi Tom,

I read this draft. It is interesting proposal. It is indeed another tunneling encapsulation proposal and aims in applying to NVO as well (not limited to).

Regarding the semantics, it suggests using the flag on the header to indicate the option field presence in the header. Three flags are specified in current proposal. For such kinds of semantics, IMO, it is very important to specify the processing order in the flags that derives the option field presence. This is because that these optional fields are present independently. If the flag is on, the corresponding field present, but the processor does not know where the option fields present. If all the option field length is vary, it mandates that the processor look at the header flag in the order too.  Therefore, specifying flag processing order is required in such semantics.

For the same above reason, the statement "A middle box may interpret some flags and optional fields of the GUE
header for classification purposes, but is not required to understand all flags and fields in GUE packets." has some problem. How does middle box know where the needed fields present on the GUE packets without knowing all the flows and option fields format? I do not know why you want middle box to perform the treatment on the inner payload without tunnel termination?

It is my impression that IPsec maybe used by a underlay network that carries NVO traffic when necessary. Here you proposal that using IPsec within the overlay network. Does that mean that the overlay app. does not rely on the underlay network to provide security?

      o Type: type of header. The rest of the fields in the header are
        defined based the type.

Do you mean here? a typo?

The proposal of encapsulating a layer 2 protocol in GUE is interesting. It means that, for different protocol type value, GUE header may be different. Why do you think that is a good design? How can hardware implement in an easy way?

Regards,
Lucy

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Mar 2014 17:14:25 -0700

Hi Lucy, thanks for the comments!

On Mon, Mar 10, 2014 at 2:51 PM, Lucy yong <[email-masked]> wrote:
> Hi Tom,
>
>
>
> I read this draft. It is interesting proposal. It is indeed another
> tunneling encapsulation proposal and aims in applying to NVO as well (not
> limited to).
>
>
>
> Regarding the semantics, it suggests using the flag on the header to
> indicate the option field presence in the header. Three flags are specified
> in current proposal. For such kinds of semantics, IMO, it is very important
> to specify the processing order in the flags that derives the option field
> presence. This is because that these optional fields are present
> independently. If the flag is on, the corresponding field present, but the
> processor does not know where the option fields present. If all the option
> field length is vary, it mandates that the processor look at the header flag
> in the order too.  Therefore, specifying flag processing order is required
> in such semantics.
>
The order of fields follows the order of the option flag(s) and the
size of a particular optional field is fixed. So for a specific set of
flags, the layout and sizes of the fields are fixed. For instance, if
the flags are 0xa000, the fields are precisely the four byte vni
followed by 8 byte security header. There is no other possible
interpretation, so in fact I can use such properties to create a fast
path:

if (gue->flags == 0xa000) {
   vgue = (struct vgue *) gue;
   sec_key = lookup(vgue->vni, ip addresses, ...)
   if (vgue->sec !=  sec_key)
      goto drop_packet;

   strip_gue_hdr(pkt);

   proto_ctx = proto_table[gue->protocol];
   (*proto_ctx->process_pkt)(pkt);
}

Because TLVs have no requisite ordering, it is much more difficult to
code a simple fast path like this (see TCP code).

>
>
> For the same above reason, the statement “A middle box may interpret some
> flags and optional fields of the GUE
>
> header for classification purposes, but is not required to understand all
> flags and fields in GUE packets.” has some problem. How does middle box know
> where the needed fields present on the GUE packets without knowing all the
> flows and option fields format? I do not know why you want middle box to
> perform the treatment on the inner payload without tunnel termination?
>
We assume that new flags and bits are always added at the end (right
of last bit that was defined). In this way, if a device implements
some processing of the n'th bit, if needs to know how to compute it's
field offset which is a function of the first n-1 bits which were
previously been defined. New bits added at n+1 and above can't change
the offset of the n'th bit so a device can continue to process it
without knowing anything about the new bits. The header length field
allows skipping over new unknown bits to find the next header.

>
>
> It is my impression that IPsec maybe used by a underlay network that carries
> NVO traffic when necessary. Here you proposal that using IPsec within the
> overlay network. Does that mean that the overlay app. does not rely on the
> underlay network to provide security?

Yes, IPsec may always carry nvo3 traffic, but the problem is how to
maintain visibility of the encpasulation to intermediate network
devices. For instance, the UDP portion of a VXLAN packet could be
encapsulated in transport mode-- this gives headers:
IP-ESP-UDP-VXLAN-packet. The problem is that the VXLAN header is no
longer in the outside header (could in fact be encrytped) so if we
want do firewall in the network based on vni we have lost that
ability. My solution is to encapsulate by IP-UDP-GUE-ESP-packet.

> Do you mean here? a typo?
Yes, should read "defined based on the type"

>
>
>
> The proposal of encapsulating a layer 2 protocol in GUE is interesting. It
> means that, for different protocol type value, GUE header may be different.
> Why do you think that is a good design? How can hardware implement in an
> easy way?

The GUE header did not change, in this case a GRE packet is being
encapsulated within GUE. With IP protocol we can directly encapsulate
IPv4, IPv6, GRE, and Ethernet frames using EtherIP (as a small bonus,
EtherIP includes an extra two bytes before encapsulated Ethernet
header which retains 32 bit alignment of following IP protocols--
vxlan and nvgre don't account for that). Using GRE encapsulation is
only necessary for other "L3" protocols.

Tom

---

From: Lucy yong <[email-masked]>
Date: Tue, 11 Mar 2014 16:49:41 +0000

Hi Tom, 

Please see in-line below.

-----Original Message-----
From: Tom Herbert [mailto:[email-masked]] 
Sent: Monday, March 10, 2014 7:14 PM
To: Lucy yong
Cc: [email-masked]
Subject: Re: comment on herbert-gue-01

On Mon, Mar 10, 2014 at 2:51 PM, Lucy yong <[email-masked]> wrote:
> tunneling encapsulation proposal and aims in applying to NVO as well 
> (not limited to).
>
>
>
> indicate the option field presence in the header. Three flags are 
> specified in current proposal. For such kinds of semantics, IMO, it is 
> very important to specify the processing order in the flags that 
> derives the option field presence. This is because that these optional 
> fields are present independently. If the flag is on, the corresponding 
> field present, but the processor does not know where the option fields 
> present. If all the option field length is vary, it mandates that the 
> processor look at the header flag in the order too.  Therefore, 
> specifying flag processing order is required in such semantics.
>
The order of fields follows the order of the option flag(s) and the size of a particular optional field is fixed. So for a specific set of flags, the layout and sizes of the fields are fixed. For instance, if the flags are 0xa000, the fields are precisely the four byte vni followed by 8 byte security header. There is no other possible interpretation, so in fact I can use such properties to create a fast
path:
[Lucy] OK. Then it is necessary to make the field order constraint explicitly clear. BTW using flag bits achieve different sizes of a field has pros and cons. 

   strip_gue_hdr(pkt);

Because TLVs have no requisite ordering, it is much more difficult to code a simple fast path like this (see TCP code).

>
>
> For the same above reason, the statement “A middle box may interpret 
> some flags and optional fields of the GUE
>
> header for classification purposes, but is not required to understand 
> all flags and fields in GUE packets.” has some problem. How does 
> middle box know where the needed fields present on the GUE packets 
> without knowing all the flows and option fields format? I do not know 
> why you want middle box to perform the treatment on the inner payload without tunnel termination?
>
We assume that new flags and bits are always added at the end (right of last bit that was defined). In this way, if a device implements some processing of the n'th bit, if needs to know how to compute it's field offset which is a function of the first n-1 bits which were previously been defined. New bits added at n+1 and above can't change the offset of the n'th bit so a device can continue to process it without knowing anything about the new bits. The header length field allows skipping over new unknown bits to find the next header.
[Lucy] yes, if the field presence order and field length are fixed. The middle box needs to understand GUE format including all optional fields but not the content in the fields. Having the private field at last and requiring a field length indication there becomes unique privilege for this field. 

>
>
> It is my impression that IPsec maybe used by a underlay network that 
> carries NVO traffic when necessary. Here you proposal that using IPsec 
> within the overlay network. Does that mean that the overlay app. does 
> not rely on the underlay network to provide security?

Yes, IPsec may always carry nvo3 traffic, but the problem is how to maintain visibility of the encpasulation to intermediate network devices. For instance, the UDP portion of a VXLAN packet could be encapsulated in transport mode-- this gives headers:
IP-ESP-UDP-VXLAN-packet. The problem is that the VXLAN header is no longer in the outside header (could in fact be encrytped) so if we want do firewall in the network based on vni we have lost that ability. My solution is to encapsulate by IP-UDP-GUE-ESP-packet.
[Lucy] Are these controversial requirements? A firewall may be used to inspect flow within VNI, then what do you do?

Yes, should read "defined based on the type"
[Lucy] then do you mean that a bit in a GUE header w/ a different type value may mean different things? This draft only defines the fields with type value 0. 

>
>
>
> The proposal of encapsulating a layer 2 protocol in GUE is 
> interesting. It means that, for different protocol type value, GUE header may be different.
> Why do you think that is a good design? How can hardware implement in 
> an easy way?

The GUE header did not change, in this case a GRE packet is being encapsulated within GUE. With IP protocol we can directly encapsulate IPv4, IPv6, GRE, and Ethernet frames using EtherIP (as a small bonus, EtherIP includes an extra two bytes before encapsulated Ethernet header which retains 32 bit alignment of following IP protocols-- vxlan and nvgre don't account for that). Using GRE encapsulation is only necessary for other "L3" protocols.
[Lucy] So this is just a GRE in GUE. No need to make it special and limit it to encap. L2 protocols.

Lucy

Tom

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Mar 2014 10:48:54 -0700

> [Lucy] OK. Then it is necessary to make the field order constraint explicitly clear. BTW using flag bits achieve different sizes of a field has pros and cons.

Indeed. I had considered making all fields one size (say 32 bits),
where each bit indicates once field, and contiguous larger fields
could be created by grouping continuous bits together. This would make
header length and offset calculation very easy. Problems were that
this precludes true flags (no associated field), large fields would
consume a lot of the bits (e.g. a 256 bit field requires 8 flag bits),
and a grouped field could have multiple representations yielding the
same length.

> [Lucy] yes, if the field presence order and field length are fixed. The middle box needs to understand GUE format including all optional fields but not the content in the fields. Having the private…

That's why we needed to put private bits at the end.

> [Lucy] Are these controversial requirements? A firewall may be used to inspect flow within VNI, then what do you do?

We'll have a similar problem with any form of encryption in the data
center. The purpose of encryption is to hide the contents which means
that devices in the network without the decrypt keys won't have
visibility any more. In the case of network virtualization, I would
prefer to encrypt the client's whole packet and place minimal
identifying information about the inner header in the outside header
that firewalls may consume. To be honest, in the most conservative
approach, I'd prefer not to send anything in the clear that might
easily identify the tenant which originated the packet, eventually
we'd even want the vni to be obfuscated (another reason why it needs
to be a least 32 bits!).

> [Lucy] then do you mean that a bit in a GUE header w/ a different type value may mean different things? This draft only defines the fields with type value 0.
Yes. Only the first byte is specified. Type #1 could have a completely
different format beyond that, or exactly same format but completely
different semantics. I don't see a reason to bake in the format for
all types. I think the OAM messages might fit well into a new type
(although I really don't have a use case to define it so I can't
provide specifics). Also, when we add NAT support, we'll probably want
a keepalive to be a different type. The important thing is that new
types/versions can be defined without effecting the primary
encapsulation data path. "if (gue->type == 0)" is first branch into
the fast path.

> [Lucy] So this is just a GRE in GUE. No need to make it special and limit it to encap. L2 protocols.
Correct.