Subject: [nvo3] FW: New Version Notification for draft-gross-geneve-00.txt

From: Pankaj Garg <[email-masked]>
Date: Sat, 15 Feb 2014 00:22:05 +0000

As a co-author on this draft, feedback is requested.

Sent from my Windows Phone
________________________________
From: [email-masked]<mailto:[email-masked]>
Sent: ‎2/‎15/‎2014 4:05 AM
To: T.Sridhar<mailto:[email-masked]>; Ilango Ganga<mailto:[email-masked]>; Jesse Gross<mailto:[email-masked]>; Ilango Ganga<mailto:[email-masked]>; Pankaj Garg<mailto:[email-masked]>; Chris Wright<mailto:[email-masked]>; Pankaj Garg<mailto:[email-masked]>; Chris Wright<mailto:[email-masked]>; T. Sridhar<mailto:[email-masked]>; Jesse Gross<mailto:[email-masked]>
Subject: New Version Notification for draft-gross-geneve-00.txt

A new version of I-D, draft-gross-geneve-00.txt
has been successfully submitted by Jesse Gross and posted to the
IETF repository.

Name:           draft-gross-geneve
Revision:       00
Title:          Geneve: Generic Network Virtualization Encapsulation
Document date:  2014-02-14
Group:          Individual Submission
Pages:          23
URL:            http://www.ietf.org/internet-drafts/draft-gross-geneve-00.txt
Status:         https://datatracker.ietf.org/doc/draft-gross-geneve/
Htmlized:       http://tools.ietf.org/html/draft-gross-geneve-00

Abstract:
   Network virtualization involves the cooperation of devices with a
   wide variety of capabilities such as software and hardware tunnel
   endpoints, transit fabrics, and centralized control clusters.  As a
   result of their role in tying together different elements in the
   system, the requirements on tunnels are influenced by all of these
   components.  Flexibility is therefore the most important aspect of a
   tunnel protocol if it is keep pace with the evolution of the system.
   This draft describes Geneve, a protocol designed to recognize and
   accommodate these changing capabilities and needs.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Tom Herbert <[email-masked]>
Date: Sat, 15 Feb 2014 14:21:37 -0800

I have a general question on vsid, vnid's.

nvgre, vxlan, and now Geneve all define a 24 bit virtual network
identifier in their packet formats. What is so magical about this
size? I can understand that nvgre needs to not use full 32 bits in
keyid and wants some bits for flow hash, but UDP based encapsulations
should not have that consideration. While on paper these might allow
16M ids, in practice even a moderately large deployment will want to
do hierarchical allocation, reserve some high order bits for special
classes (like "trusted", "internal", etc.), and might do masked block
assignments for customers-- so with 24 bits we may be facing future
scaling issues. In GUE we defined a 32 bit vnid which should allow
more scaling, but if we need to obfuscate the vni for things like
security then even that might not be large enough!

Thanks,
Tom

On Fri, Feb 14, 2014 at 4:22 PM, Pankaj Garg <[email-masked]> wrote:
> From: [email-masked]
> To: T.Sridhar; Ilango Ganga; Jesse Gross; Ilango Ganga; Pankaj Garg; Chris
> Wright; Pankaj Garg; Chris Wright; T. Sridhar; Jesse Gross
> URL:
> http://www.ietf.org/internet-drafts/draft-gross-geneve-00.txt
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3
>

---

From: Pankaj Garg <[email-masked]>
Date: Sun, 16 Feb 2014 03:51:09 +0000

For NVGRE, the 24-bit was chosen, to keep few reserved bits in the Key fields for future use and 24-bit ID seemed enough, which we ended up reclaiming for FlowId in a future revision.

I am not sure I understand why reserve bits are needed for categorizing the networks. Isn't it possible to use the range? in any case, I don't have a good reason for 24-bit vs 32-bit vs 64-bit, beside the fact that 24-bit seemed good enough and so far we haven't heard any problems with it, at least, in the context of NVGRE.

As you mention, if obfuscation based security is needed, even 32-bits won't be sufficient. I don't believe we are trying to solve the security problem of snooping VNI. However, if it needs to be solved, one option is to use IPsec or some other encryption to make the header completely opaque to the transit network as described in the NVGRE draft.

Pankaj 
________________________________________
From: Tom Herbert <[email-masked]>
Sent: Sunday, February 16, 2014 3:51
To: Pankaj Garg
Cc: [email-masked]
Subject: Re: [nvo3] FW: New Version Notification for draft-gross-geneve-00.txt

Thanks,
Tom

---

From: Tom Herbert <[email-masked]>
Date: Mon, 17 Feb 2014 10:47:14 -0800

> For NVGRE, the 24-bit was chosen, to keep few reserved bits in the Key fields for future use and 24-bit ID seemed enough, which we ended up reclaiming for FlowId in a future revision.
Yes, but unfortunately nvgre shortchanges both the vni and flow ID. A
new protocol without legacy constraints should do better than that!

> I am not sure I understand why reserve bits are needed for categorizing the networks. Isn't it possible to use the range? in any case, I don't have a good reason for 24-bit vs 32-bit vs 64-bit, bes…

At one point IPv4 address space seemed large enough also! In any case,
even if 24 bits were sufficient, why use the upper 24 bits of the
word? If this was in the lower 24 bits then that would at least allow
the option of expanding the space into the upper 8 bits with backwards
compatibility.

> As you mention, if obfuscation based security is needed, even 32-bits won't be sufficient. I don't believe we are trying to solve the security problem of snooping VNI. However, if it needs to be so…
Security is a major issue, and I believe should be a top consideration
in the development of these encapsulation protocols (personally I'm
hoping encryption becomes ubiquitous in the lifetime of these
protocols even within DC). Consider the real ramifications of using
IPsec with something like vxlan. To secure the packet headers would
entail doing IPsec over the UDP/vxlan header, for example we'd have
something like IP-ESP-UDP-vxlan-inner_packet. This has several
deficiencies:

1) We no longer have UDP in the outer header so we lose its usefulness
to providing a flow hash in network devices. We can recover this by
another UDP encapsulation giving IP-UDP-ESP-UDP-vxlan-inner_packet,
but encap'ing packets twice in UDP is hardly elegant or efficient.
2) The vni is not in the outermost headers and might be encrypted
anyway. This eliminates the ability to firewall within the network
based on vni. Strict partitioning of tenants in the physical network
will be an important feature even with the use of IPsec.
3) When inner headers are encrypted we lose visibility within the
network for flows. This substantially impacts using sflow and other
taps to diagnose networking issues within the fabric.

What I think with we want in such a case looks like
IP-UDP-encap-IPsec-inner_packet. That is have UDP and encap header in
the outer header which includes vni and probably a flow identification
token.

In any case, whether or not IPsec is used, the encap header really
needs to be secured somehow. Maintaining isolation between virtual
networks is super critical, in the case of third party vms we can't
assume the monitoring and detection which is already deployed in the
native infrastructure.

Thanks,
Tom

---

From: Pankaj Garg <[email-masked]>
Date: Tue, 18 Feb 2014 06:15:01 +0000

Inline.

> -----Original Message-----
> From: Tom Herbert [mailto:[email-masked]]
> Sent: Tuesday, February 18, 2014 12:17 AM
> To: Pankaj Garg
> Cc: [email-masked]
> > For NVGRE, the 24-bit was chosen, to keep few reserved bits in the Key
> fields for future use and 24-bit ID seemed enough, which we ended up
> reclaiming for FlowId in a future revision.
> >
> Yes, but unfortunately nvgre shortchanges both the vni and flow ID. A new
> protocol without legacy constraints should do better than that!
> 
> > I am not sure I understand why reserve bits are needed for categorizing
> the networks. Isn't it possible to use the range? in any case, I don't have a
> good reason for 24-bit vs 32-bit vs 64-bit, beside the fact that 24-bit seemed
> good enough and so far we haven't heard any problems with it, at least, in
> the context of NVGRE.
> 
> At one point IPv4 address space seemed large enough also! In any case, even
> if 24 bits were sufficient, why use the upper 24 bits of the word? If this was in
> the lower 24 bits then that would at least allow the option of expanding the
> space into the upper 8 bits with backwards compatibility.

[PG] There are two separate items here, so let me comment on each one separately. 
(1) I don't understand the backward compatibility argument for mandatory bits in the frame. For example, if the VNI is right shifted today but kept 24-bits, I am not sure if it really helps in future, because for all 32-bits to be recognized by the endpoints or transit devices they would all need to be updated to handle 32-bits or otherwise they may act on 24-bits resulting in incorrect behavior. If backward compatibility is needed, then the reserve bits can only be used for adding optional information that can be ignored by the receivers. 
(2) As to having 32-bits in future, we have to pick a number and one can argue even 32-bits won't be sufficient in future, so where do we draw the line? We can chat about it during next NVO3 meet along with other authors of the draft. 

To summarize, I am not against making it 32-bits (if there is consensus on that), but I don't think right shifting the VNI provides much value.

> 
> >
> > As you mention, if obfuscation based security is needed, even 32-bits
> won't be sufficient. I don't believe we are trying to solve the security
> problem of snooping VNI. However, if it needs to be solved, one option is to
> use IPsec or some other encryption to make the header completely opaque
> to the transit network as described in the NVGRE draft.
> >
> Security is a major issue, and I believe should be a top consideration in the
> development of these encapsulation protocols (personally I'm hoping
> encryption becomes ubiquitous in the lifetime of these protocols even within
> DC). 

[PG] I am not against having a tunnel format that has security and encryption defined into it. What I meant was that the tunnel formats such as NVGRE, VXLAN and Geneve, are not defining that as a lot of use cases are fine without it. That said, maybe as part of NVO3 charter, the security requirements should be enumerated and an optional tunnel format proposed to address those. That should cover various attack vectors including but not limited to Snooping/Spoofing VNI or other fields in outer frame, Snooping/Spoofing Inner Frame, etc. That is definitely out of scope of this document.

>Consider the real ramifications of using IPsec with something like vxlan.
> To secure the packet headers would entail doing IPsec over the UDP/vxlan
> header, for example we'd have something like IP-ESP-UDP-vxlan-
> inner_packet. This has several
> deficiencies:
> 
> 1) We no longer have UDP in the outer header so we lose its usefulness to
> providing a flow hash in network devices. We can recover this by another
> UDP encapsulation giving IP-UDP-ESP-UDP-vxlan-inner_packet,
> 2) The vni is not in the outermost headers and might be encrypted anyway.
> This eliminates the ability to firewall within the network based on vni. Strict
> partitioning of tenants in the physical network will be an important feature
> even with the use of IPsec.
> 3) When inner headers are encrypted we lose visibility within the network
> for flows. This substantially impacts using sflow and other taps to diagnose
> networking issues within the fabric.
> 
> What I think with we want in such a case looks like IP-UDP-encap-IPsec-
> inner_packet. That is have UDP and encap header in the outer header which
> includes vni and probably a flow identification token.
> 
> In any case, whether or not IPsec is used, the encap header really needs to
> be secured somehow. Maintaining isolation between virtual networks is
> super critical, in the case of third party vms we can't assume the monitoring
> and detection which is already deployed in the native infrastructure.
> 
> Thanks,
> Tom
> 
> > Pankaj
> >> To: T.Sridhar; Ilango Ganga; Jesse Gross; Ilango Ganga; Pankaj Garg;
> >> Chris Wright; Pankaj Garg; Chris Wright; T. Sridhar; Jesse Gross
> >> A new version of I-D, draft-gross-geneve-00.txt has been successfully
> >> submitted by Jesse Gross and posted to the IETF repository.
> >>
> >> Name:           draft-gross-geneve
> >> Revision:       00
> >> Please note that it may take a couple of minutes from the time of
> >> submission until the htmlized version and diff are available at
> tools.ietf.org.
> >>
> >> The IETF Secretariat
> >>
> >>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 18 Feb 2014 17:57:31 -0800

> (2) As to having 32-bits in future, we have to pick a number and one can argue even 32-bits won't be sufficient in future, so where do we draw the line? We can chat about it during next NVO3 meet a…
You could make the vni be another TLV, then having to worry about
setting the vni length in stone is no longer an issue. This also has
the benefit that instead of geneve being an extensible virtual
encapsulation protocol, it would be a generic and extensible
encapsulation protocol that happens to support virtualization as one
use case-- that might make it more compelling.