Subject: [nvo3] comments and suggestions to draft-ietf-nv03-arch-01

From: Linda Dunbar <[email-masked]>
Date: Wed, 14 May 2014 21:50:46 +0000

David, Jon, Larry, Marc, and Thomas,

Here are some questions, comments, and suggestions to draft-ietf-nv03-arch-01:

*       Section 4.2.1 (Tenant VLAN handling in Split-NVE case):  Even with virtualized servers,  NVE function could be implemented on the first switch to which the (virtualized) servers is attached.   Do you call this scenario "Split-NVE" case?

Under this case, the NVE should have access to the complete original packet. why does the last sentence say "... additional mechanisms would be used"?

*       Section 5.1: Most Network service appliances are for applying treatment to data packets between hosts/applications, i.e. most likely the Network Service Appliances are in the same address domain of the overlay, not underlay.  Probably using the same VLAN tags from TS for the tenants' configuration rules.

I find the following text to comparing Appliances VN differentiation with Server virtualization is inappropriate, therefore should be removed from the section.

"Unlike server virtualization, however, the Network
Service Appliance may not be running a traditional hypervisor and the
VM Orchestration system may not interact with the Network Service
Appliance."

*       Section 5.3: "Gateways" should be in the overlay address domain, i.e. same as Tenant Systems.  Gateway is for relay traffic from one VN to another. The relay is specific to the traffic among TSs.

In Figure 1, should add a box for Gateways. They are in the same positions as TS, i.e. outside the NVEs.

Even if NVEs support Gateway function, the gateway should be a separate logical entity connected to VAP (Figure 2).

*       Section 7.3: Why is it necessary for NVA to appear as one single entity? For a network that supports tens of thousands VNs, each NVA may only need to support a subset of VNs. Why can't we let each NVA to announce itself on the range of VNs that it supports, and let the NVE reach to the appropriate NVA depending on the data frame's VN?

Cheers,
Linda

---

From: "Black, David" <[email-masked]>
Date: Thu, 15 May 2014 12:42:23 -0400

Hi Linda,

> Here are some questions, comments, and suggestions to draft-ietf-nv03-arch-01:

Thank you for taking a detailed look.

> Section 4.2.1 (Tenant VLAN handling in Split-NVE case):  Even with virtualized
> servers,  NVE function
> could be implemented on the first switch to which the (virtualized) servers is attached.   Do you call
> this scenario "Split-NVE" case?

If I understand the question, I believe the answer is "no" - the Split-NVE case treats the hypervisor-NVE VLANs as private to the hypervisor and external NVE - in effect each VLAN is used to extend the virtual network over an unencapsulated link; they would not be visible to the TS on a VLAN-tagged interface.  More in next paragraph ...

> Under this case, the NVE should have access to the complete original packet. why does the last sentence say
> "... additional mechanisms would be used"?

That sentence is about something else - namely, if the VLAN is assigned to extend a VN by the combination of the hypervisor and NVE, then it's not carrying VLAN info within that VN for the Tenant Systems.  Adding such TS VLAN info involves additional mechanisms, such as VLAN mapping, e.g., if TSs on VNs A and B are both using VLANs 1-8 at their interfaces, then the hypervisor and NVE have to remap those VLAN tags into the set VLAN tags used for hypervisor-NVE communication - e.g., between the hypervisor and NVE, VN A's VLANs 1-8 might be mapped to/from VLANs 101-108, and VN B's VLANs 1-8 might be mapped to/from VLANs 151-158.  We could add that explanation to the draft.

OTOH, it feels like your comments are about a different model where existing VLANs are attached to virtual networks instead of attaching Tenant Systems - I'm not sure whether that's appropriate to add to the base model, as this feels like some sort of gateway between the VN and an existing VLAN, which is covered, albeit briefly in 5.3.  What do other people think?

> Section 5.1: Most Network service appliances are for applying treatment to data packets between hosts/applications,
> i.e. most likely the Network Service Appliances are in the same address domain of the overlay, not underlay

Ok, but please keep in mind that existing network service appliances can't decode the overlay/underlay encapsulation - that's a major motivation for the text in this section.

> I find the following text to comparing Appliances VN differentiation with Server virtualization
> is inappropriate, therefore should be removed from the section.

"Unlike server virtualization, however, the Network Service Appliance may not be running a traditional hypervisor and the VM Orchestration system may not interact with the Network Service Appliance."

Why?  I believe the text is correct and describes situations that may occur in practice with existing appliances.

> Section 5.3: "Gateways" should be in the overlay address domain, i.e. same as Tenant Systems.  Gateway
> is for relay traffic from one VN to another. The relay is specific to the traffic among TSs.

That is the intent - if the text is not clear, please suggest specific changes.

> In Figure 1, should add a box for Gateways. They are in the same positions as TS, i.e. outside the NVEs.

That Figure is a copy of Figure 3 in the framework draft - I would prefer not to change it for that reason, but a separate figure for gateways seems fine - do you want to try to draw one?

> Even if NVEs support Gateway function, the gateway should be a separate logical entity connected to VAP (Figure 2).

Ok, but I think a separate gateway figure would be clearer.

> Section 7.3: Why is it necessary for NVA to appear as one single entity? For a network that supports
> tens of thousands VNs, each NVA may only need to support a subset of VNs. Why can't we let each NVA
> to announce itself on the range of VNs that it supports, and let the NVE reach to the appropriate NVA
> depending on the data frame's VN?

I agree that this scenario is reasonable, although I'd prefer to think about it differently from an architectural viewpoint, and that viewpoint is not captured in the architecture draft, so thank you for raising the topic.

If a single NVE implementation is talking to multiple NVAs, I would prefer to view that NVE implementation as consisting of multiple logical or virtual NVE instances.  My rationale is that I think of an NVE as controlled by the NVA, and viewing the NVA as having its own logical or virtual NVE instance makes the domain of control of each NVA clearer.

Regardless of whether the WG agrees with this viewpoint, I think the architecture draft could use some text to discuss this scenario.

Thanks,
--David

From: nvo3 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Wednesday, May 14, 2014 5:51 PM
To: Thomas Narten; [email-masked]; Black, David; Jon Hudson; Larry Kreeger; LASSERRE, MARC (MARC)
Subject: [nvo3] comments and suggestions to draft-ietf-nv03-arch-01

Cheers,
Linda

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 15 May 2014 18:37:38 +0000

David,

Thanks for the prompt reply. To make it easier for the community to chime in, I break the issues based on their category and add a distinct subject title.

My comments are inserted below.

From: Black, David [mailto:[email-masked]]

[Linda] Bear in mind, the data frames sent out from Applications (or Tenant Systems) don't usually have VLANs (unless the application itself is configured with multiple IP addresses under different subnets). It is the first switch that add the VLAN for the purpose of differentiating the traffic on the link that aggregate traffics from different TSs.  In another words, the VLAN is usually NOT visible to TS.

The VLAN between Hypervisor <-> NVE should be locally significant anyway. When a TS is moved from being attached one NVE (say NVE#1) to being attached to another NVE (say NVE#2), the local VLAN under NVE#2 to represent the Tenant Network might be different from the local VLAN under the NVE#1 because the same VLAN number under NVE#2 might be used for another Tenant network.

[Linda] The described mechanism is needed when NVE is not collocated with Hypervisor. The VLANs are south of NVE's VAP, i.e. between TS and NVE. In order to support VM migration and more than 4K TNs, the VLANs under each NVE should be locally significant. A mechanism is needed assign an unique VLAN for TN under each NVE to avoid VLAN collision.

So I only see two cases:

1.       NVE & Hypervisor collocated

2.       NVE & Hypervisor not collocated.

I don't see why Splict-NVE case being different from #2 above.

Linda

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 15 May 2014 18:44:36 +0000

David,

My suggestion to remove the following text from 5.1 is due to there is no such a term of "traditional hypervisor" for Network Service Appliances.

Most Network service appliances are for applying treatment to data packets between hosts/applications,
i.e. most likely the Network Service Appliances are in the same address domain of the overlay, not underlay.

In another word, they don't terminate Overlay header.

Linda

[Linda] That is why I say that the "Network Service Appliances" are in the same address domain of the overlay not underlay.

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 15 May 2014 21:36:03 +0000

David,

Attached is my suggested wording change for Section 5.3 Gateways. In case the NVO3 mailing list can't take attachment, here are the suggested wording (the attachment has change marked, so easier to read).

Linda

5.3.  Gateways

   Gateways of VNs relay traffic onto and off of a virtual network.
   Tenant Systems use gateways to reach destinations outside of the
   local VN.  If the Tenant Systems are in Layer 2 network, the Gateways are the L2/L3 boundary nodes, i.e. mapping the external Layer 3 traffic to the corresponding Layer 2 traffic expected by the Tenant Systems, and vice versa. If the Tenant Systems are in Layer 3 network, the Gateways are the boundary between external WAN network (such as Internet, VPN, etc) and the internal network expected by the Tenant Systems.

When a gateway relays traffic from one VN to another, it has to decapsulte the overlay header of the source VN, relay the traffic, and encapsulate a new header with the target VN. Therefore, there is always a NVE function, whether embedded or external, to perform the encapsulation/decapsulation function.
Strictly speaking, the VN Gateways are outside the underlay network, i.e. in the same address domains of the Tenant Systems.
Taking Gateways into consideration, the NVO3 model depicted in Figure 1 will be like the following figure:

                              _,....._
                           ,-'        `-.
                          /   External  `.
                         |     Network   |
                         `.             /
                           `.__     _,-'
                               `''''
                                  |
                             +---------+
                             | Gateway |
                             +----+----+
                             +----+----+
                             |   NVE   |
                             +-----+---+
       +--------+                  |                          +--------+
       | Tenant +--+               |                     +----| Tenant |
       | System |  |               |                    (')   | System |
       +--------+  |          ................         (   )  +--------+
                   |  +-+--+  .              .  +--+-+  (_)
                   |  | NVE|--.              .--| NVE|   |
                   +--|    |  .              .  |    |---+
                      +-+--+  .              .  +--+-+
                      /       .              .
                     /        .  L3 Overlay  .   +--+-++--------+
       +--------+   /         .    Network   .   | NVE|| Tenant |
       | Tenant +--+          .              .- -|    || System |
       | System |             .              .   +--+-++--------+
       +--------+             ................
                                     |
                                   +----+
                                   | NVE|
                                   |    |
                                   +----+
                                     |
                                     |
                           =====================
                             |               |
                         +--------+      +--------+
                         | Tenant |      | Tenant |
                         | System |      | System |
                         +--------+      +--------+

   The dotted line indicates a network connection (i.e., IP).

                  Figure 3: NVO3 Reference Model with Gateways

   Gateways can be either virtual (i.e., implemented as a VM) or
   physical (i.e., as a standalone physical device).  For performance
   reasons, standalone hardware gateways may be desirable in some cases.
   Such gateways could consist of a simple switch forwarding traffic
   from a VN onto the local data center network, could embed router
   functionality, or/and service functions (such as firewall, IPS/IDS, etc).  [L1]
As in the case with Network Service
   Appliances that don't terminate NV header, gateways [L2] will need an
   appropriate control plane protocol to obtain the information needed
   to provide NVO3 service.

   Gateways handle several different use cases.  For example, a virtual
   network could consist of systems supporting overlays together with

   legacy Tenant Systems that do not.  Gateways could be used to connect
   legacy systems supporting, e.g., L2 VLANs, to specific virtual
   networks, effectively making them part of the same virtual network.[L3]
   Gateways could also forward traffic between a virtual network and
   other hosts on the data center network or relay traffic between
   different VNs.  Finally, gateways can provide external connectivity
   such as Internet or VPN access.

Thanks,
--David

From: nvo3 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Wednesday, May 14, 2014 5:51 PM
To: Thomas Narten; [email-masked]<mailto:[email-masked]>; Black, David; Jon Hudson; Larry Kreeger; LASSERRE, MARC (MARC)
Subject: [nvo3] comments and suggestions to draft-ietf-nv03-arch-01

Cheers,
Linda

________________________________

 [L1]When a gateway relays traffic from one VN to another, it has to decapsulte the overlay header with the source VN, relay the traffic, and encapsulate the new header with the target VN. Therefore, there is always a NVE function, whether embedded or external, to perform the encapsulation/decapsulation function.

That is why I move the next sentence to the beginning of the section to replace the description of Gateway encapsulating/decapsulating paragraph.

 [L2]Never heard a term about Hypervisor for Gateway.

Do you intend to say that when Network Service Appliances don't support NV header, the Gateway has to use some Network label to differentiate traffic belonging to different VNs?

 [L3]This is NVE function, not Gateway function.

You can have a bridged LAN that connecting to many legacy tenant systems attached to one NVE. For example, NVE on Aggregation switch.

---

From: "Black, David" <[email-masked]>
Date: Fri, 16 May 2014 00:26:13 -0400

Linda,

Would removing the word "traditional" help?  Nothing in the quoted text implies termination of the overlay header.

Thanks,
--David

From: Linda Dunbar [mailto:[email-masked]]
Sent: Thursday, May 15, 2014 2:45 PM
To: Black, David; Thomas Narten; [email-masked]; Jon Hudson; Larry Kreeger; LASSERRE, MARC (MARC)
Subject: What is "Hypervisor" for Network Service Appliances? (was RE: comments and suggestions to draft-ietf-nv03-arch-01)

David,

Linda

---

From: "Black, David" <[email-masked]>
Date: Fri, 16 May 2014 01:12:41 -0400

Linda,

It looks like I did not understand your question ...

I believe that you're describing the Split NVE case where encap/decap is implemented external to the hypervisor, e.g., on a switch, and the VLAN between NVE and Hypervisor is only locally significant (I used the word "private").  In that case, some of the NVE function winds up on the hypervisor in order that the NVE function on the switch and the hypervisor agree on use of those locally significant VLANs.

There are VMs that use VLAN-tagged interfaces, independent of whether they are the "usual" case, and that usage requires the "additional mechanisms" referred to in the last sentence of 4.2.1 - the VLAN mapping that I described is one (but not the only) possibility.

I think we're in some form of violent agreement, and I'm not (yet?) seeing the need for text changes here.

Thanks,
--David

From: nvo3 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Thursday, May 15, 2014 2:38 PM
To: Black, David; Thomas Narten; [email-masked]; Jon Hudson; Larry Kreeger; LASSERRE, MARC (MARC)
Subject: [nvo3] What is Splict-NVE and the VLAN between Hypervisor and NVE (was RE: comments and suggestions to draft-ietf-nv03-arch-01)

David,

So I only see two cases:

Linda

---

From: Zu Qiang <[email-masked]>
Date: Tue, 20 May 2014 18:34:28 +0000

Inline.
Have a nice day
Zu Qiang

[Zu Qiang] I prefer the text "Conceptually, from the perspective of an NVE, an NVA is a single entity." This statement only specifies that one NVE is always under controlled by one NVA. One may have multiple NVA boxes developed in one DC, which each NVA may only support one subset of VNs. But from the NVE viewpoint, there is only one NVA as controller.

Thanks,
--David

Cheers,
Linda