Subject: Re: [nvo3] draft-khasnabish-vmmi-problems-02

From: Richard Bin liu <[email-masked]>
Date: Mon, 5 Nov 2012 00:01:58 +0800

Hello Pat Thaler:

First of all, thank you for the careful review on our draft. They are very
helpful to us.
A lot of your comments are very typical. My explanations will also answer
many similar problems proposed by other experts. And some of your comments
are very constructive.

The following content, brown is my reply content

3 – migration from system to system that stays within the same data center
and administrative domain isn’t listed.
OK. We will independently list the L3 migration from system to system that
stays within the same data center and administrative domain in the later
version.

In 3 (C), IEEE 802.1Q PBB already provides for a larger (24-bit) ID when
the 12-bit VLAN ID isn’t large enough. VXLAN and NVGRE aren’t needed to
address that problem. What VXLAN and NVGRE provide that PBB doesn’t is the
ability to carry L2 over an underlying layer 3 network – so VXLAN and NVGRE
should be in (D) but aren’t.

VXLAN not only has the ability to carry L2 over an underlying L3 network,
but also has the ability to support Multi-tenancy. In the past, tenants are
isolated by VLAN. But VLAN only supports 4K isolated domains. Now we can
implement up to 16,000,000 isolated domains through VXLAN. So I think VXLAN
solve the problem of insufficient number of VLANs through carrying L2 over
an underlying L3 network. So there is no problem in my view. Maybe I will
add some descriptions.

There is a problem with NVGRE alluded to but not explained: And, NVGRE

   “[draft-sridharan-virtualization-nvgre-00] attempts to solve similar

   problems, but artificially causes interoperability problems between

   domains.”

Why?

There is more than one reason, for example, the network equipment and
firewalls currently don’t support NVGRE / VXLAN.

Also, 24 bits is to address the scaling needs within an administrative
domain. It isn’t clear that it provides enough space to allocate compatible
assignments between administrative domains. Mapping may be needed at the
boundary between administrative domains.

I think you have proposed a good problem. But some experts believe that
this point of view is very new, and still have not seen the specific
explanation. Would you like to explain the mapping problem in detail? I'm
willing to adopt it as a new problem statement in the later version.
Otherwise I don’t know how to update it.

(E) Aren’t A through D about mobility of VMs? I don’t see what this adds.

We will remove (E) in the later version. Thank you for your comments.

3.1

“In addition, when re-configuring the VLAN in the traditional DC

   network, STP (MSTP) will lead to the VLAN isolation.  It is a very

   serious problem in the DC network, especially in the storage network,

   because the services that storage networks support demand

   uninterrupted operation.  In the Cloud computing and DC network, the

   problem with VM migration is fatal.”

?? STP or MSTP don’t cause VLAN isolation and IEEE 802.1Q defines MVRP to
handle dynamic configuration and distribution of VLAN membership
information. That there are times when a data center operator chooses to
structure their data center(s) with multiple layer 3 subnets and make that
transparent to VMs operating in their own IP address space doesn’t need to
be justified by attacking the capabilities of Layer 2.

What I mean is re-configuring the VLAN, rather than STP or MSTP, .will lead
to VLAN isolation.

MVRP is used by IPTV clients in MAN, not in IDC. It has been removed by
CCSA in the IDC standards.

If you think people may choose running and migrating VM on L3 network
without attacking the capabilities of Layer 2, it shows that we agree with
the necessity to solve the need for L3-based VM migration. Thank you for
your commendation.

“For example, in the process of VM migration in IDC, there are

   scenarios that VM in the traditional three-tier topological network

   is migrated through WAN to Fat-Tree topological network, or to a

   variety of other topological networks.  For the seamless online

   migration in these scenarios, a very intelligent seamless VM online

   migration is needed to be implemented on the control plane.”

Lacks specificity – what problem is posed here and what features need to be
provided in the control plane to solve it?

 Recommend you to read section 3.1.2. There are more detailed descriptions.

   Especially for the specific architectures like Fat-Tree, specific

   network topology and the protocol architecture of specific routing

   strategy (such as OSPF) should be utilized.  The VM migration process

   must be adapted to these aspects, and cannot be copied for purely

   Layer-2 migration approach.  So VM migration is inherently related to

   network topology and network routing protocols.

For related explanations you can also refer to the following paragraphs,
which are not listed here.

“If VM migration is only implemented in the L2 domain, its concern on

   the network is that the expansion of the number of VLANs or isolated

   domains, such as the 16,000,000 isolated domains in PBB.”

24 bits is the same size as used by VXLAN and NVGRE so they also provide
for 16 million tenant domains.

I agree with your point of view.

“How the source network environment adapts its configuration to the

   target environment?”

What do you mean by source network environment – “source” is used in the
Framework and problem statement as the sending end of the data plane path;
i.e. as source and destination, but that doesn’t seem to be how you are
using it here. Also explain “target environment”.

Thank you for your comments. It may not be clearly described in this
sentence. “Target environment” refers to “destination network environment”.
I will update it in the later version. What I mean is that, before VM
migrates, some configuration information, such as VLAN configuration,
routing configuration, ACL configuration, etc., may exist in the source
network environment (i.e. the network between the VM or its server and the
connected ToR). Once the VM is migrated to the destination server, the
corresponding configuration should be updated in the destination network
environment (i.e. the network between the destination server and the
connected ToR), so as to prevent authorization issues such as unauthorized
access for data.

3.1.2 Part of the purpose of NVO is to make VM addresses independent of the
underlying network addressing or routing and we have examples that do so by
using an outer IP address for routing. Furthermore, we want these addresses
to remain unchanged on moves so location independence for them is a
requirement. This section appears to be saying that they still should be
location and underlying topology dependent and I disagree. The only
constraint on them should be to be unique within the virtual network.

To ensure that the VM's IP address keeps the same is an eternal
requirement. Please do not deduce that the VM's IP address must have
changed and the VM's IP address is location and underlying topology
dependent from there is L3 migration between different topologies of DC. We
have enough methods to avoid these problems.

Your way of using an outer IP address for routing is the same with my
thoughts, but we need further consideration for dealing with new problems.
These problems perhaps do not exist inside a DC, but we must face them when
VM migrates across different networking forms of DCs.

VM IP address can be the inner layer address and the outside part of the VM
IP address (outer layer address) can be divided into the underlying
network. So VM IP address is naturally independent of the underlying
network. It is in full compliance with the requirements of nvo3. This
method is also not contradictory to the charter of nvo3 because nvo3 hasn’t
defined the structure of the underlying network. You cannot assume that
network structure of the DC all over the world is the same. We must face
the problems of seamlessly VM migration across different networking
environment and topologies of the DC (such as Fat tree, PortLand, the Bcube
/ Dcell, etc.). Even in the absence of my draft, these problems still
exist. But none of other drafts have proposed these problems.

Using a figurative example, the IP address is commonly applied to Ethernet
or PPP network. The physical layers of Ethernet and PPP network are two
different networks. Their data link layers need to provide services for
network layers based on their physical layers. Thus the IP network is an
overlay network. It is independent of the underlying network addressing or
routing through a variety of data link layer protocols. For the scenarios
discussed in our draft, various DCs’ network topologies are like different
physical networks. These sorts of physical networks need to shield their
respective differences through an adaptation layer module (similar to data
link layer), so as to make their VMs’ IP addresses independent of the
underlying network. Therefore, if there is no processing for the adaptation
layer module, the objective of the nov3’s charter (i.e. making VM addresses
independent of the underlying network) would not be able to achieve. And
the adaptation layer will be able to do it as long as upgrading the
software. If not in nvo3 WG, where can we put it? I am afraid no one will
agree with me to apply to the Chairman to create a new working group?

So I cannot agree with your conclusion of ‘they still should be location
and underlying topology dependent’. At this stage, the problems we discuss
here are a bit advanced, but the problems exist objectively and we must
face them.

More comments:

In 3.4.2:

In this case, the VID used by the VM has a global significance.

   Various rules and usage range of the VID are required to be provided.

   The VLAN-ID used by a given VM is referred to the VLAN-ID carried by

   the traffic that is originated by that VM and within the same L2

   physical domain as the VM.

The VID used doesn’t need to have global significance. It can be inserted
by the virtual switch (in the hypervisor or NIC on the end system) and have
been assigned to map to the VNI on the OBP in the ToR. In that case, it is
stripped when the OBP does the encapsulation and doesn’t go end to end.  In
IEEE 802.1Qbg, we use that approach to map a VID to an I-CID on the a
Provider Edge Bridge (e.g. at the ToR). VDP provides the communication
between the End System and the Bridge so that the Bridge can assign the
locally significant VID to map to the I-CID and so that the End System can
signal that migration is occurring.

That should be I-SID below rather than I-CID. Pat

I appreciate this paragraph of your comments. But currently many experts in
the maillist may be controversial on this issue. I suggest that the
President arrange a time slot for face-to-face discussion in the IETF
meeting, in order to eventually unify everyone’s’ understanding.

Richard

On Wed, Oct 24, 2012 at 6:58 AM, Pat Thaler <[email-masked]> wrote:

>  3 – migration from system to system that stays within the same data
> center and administrative domain isn’t listed.****
>
> ** **
>
> In 3 (C), IEEE 802.1Q PBB already provides for a larger (24-bit) ID when
> the 12-bit VLAN ID isn’t large enough. VXLAN and NVGRE aren’t needed to
> address that problem. What VXLAN and NVGRE provide that PBB doesn’t is the
> ability to carry L2 over an underlying layer 3 network – so VXLAN and NVGRE
> should be in (D) but aren’t.****
>
> ** **
>
> There is a problem with NVGRE alluded to but not explained: And, NVGRE****
>
>    “[draft-sridharan-virtualization-nvgre-00] attempts to solve similar***
> *
>
>    problems, but artificially causes interoperability problems between****
>
>    domains.”****
>
> Why?****
>
> ** **
>
> Also, 24 bits is to address the scaling needs within an administrative
> domain. It isn’t clear that it provides enough space to allocate compatible
> assignments between administrative domains. Mapping may be needed at the
> boundary between administrative domains.****
>
> ** **
>
> (E) Aren’t A through D about mobility of VMs? I don’t see what this adds.*
> ***
>
> ** **
>
> 3.1****
>
> “In addition, when re-configuring the VLAN in the traditional DC****
>
>    network, STP (MSTP) will lead to the VLAN isolation.  It is a very****
>
>    serious problem in the DC network, especially in the storage network,**
> **
>
>    because the services that storage networks support demand****
>
>    uninterrupted operation.  In the Cloud computing and DC network, the***
> *
>
>    problem with VM migration is fatal.”****
>
> ?? STP or MSTP don’t cause VLAN isolation and IEEE 802.1Q defines MVRP to
> handle dynamic configuration and distribution of VLAN membership
> information. That there are times when a data center operator chooses to
> structure their data center(s) with multiple layer 3 subnets and make that
> transparent to VMs operating in their own IP address space doesn’t need to
> be justified by attacking the capabilities of Layer 2.****
>
> ** **
>
> “For example, in the process of VM migration in IDC, there are****
>
>    scenarios that VM in the traditional three-tier topological network****
>
>    is migrated through WAN to Fat-Tree topological network, or to a****
>
>    variety of other topological networks.  For the seamless online****
>
>    migration in these scenarios, a very intelligent seamless VM online****
>
>    migration is needed to be implemented on the control plane.”****
>
> ** **
>
> Lacks specificity – what problem is posed here and what features need to
> be provided in the control plane to solve it?****
>
> ** **
>
> “If VM migration is only implemented in the L2 domain, its concern on****
>
>    the network is that the expansion of the number of VLANs or isolated****
>
>    domains, such as the 16,000,000 isolated domains in PBB.”****
>
> 24 bits is the same size as used by VXLAN and NVGRE so they also provide
> for 16 million tenant domains.****
>
> ****
>
> “How the source network environment adapts its configuration to the****
>
>    target environment?”****
>
> What do you mean by source network environment – “source” is used in the
> Framework and problem statement as the sending end of the data plane path;
> i.e. as source and destination, but that doesn’t seem to be how you are
> using it here. Also explain “target environment”.****
>
> ** **
>
> 3.1.2 Part of the purpose of NVO is to make VM addresses independent of
> the underlying network addressing or routing and we have examples that do
> so by using an outer IP address for routing. Furthermore, we want these
> addresses to remain unchanged on moves so location independence for them is
> a requirement. This section appears to be saying that they still should be
> location and underlying topology dependent and I disagree. The only
> constraint on them should be to be unique within the virtual network.****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Richard Bin liu
> *Sent:* Thursday, October 18, 2012 9:44 AM
> *To:* [email-masked]
> *Subject:* [nvo3] draft-khasnabish-vmmi-problems-02****
>
> ** **
>
> Greeting all,
>
>
> We had published this draft and wanted to bring it to the NVO3 WG's
> attention.  We plan to present/discuss this draft at the upcoming meeting
> as well, consequently please discuss here on the maillist as well.
>
> Thanks,
>
> Bhumip Khasnabish, et al.
>
>
> A new version of I-D, draft-khasnabish-vmmi-problems-02.txt
> has been successfully submitted by Bin Liu and posted to the
> IETF repository.
> Filename:        draft-khasnabish-vmmi-problems
> Revision:        02
> Title:           Requirements for Mobility and Interconnection of Virtual
> Machine and Virtual Network Elements
> Creation date:   2012-10-15
> WG ID:           Individual Submission
> Number of pages: 46
> URL:
> http://www.ietf.org/internet-drafts/draft-khasnabish-vmmi-problems-02.txt
> Status:
> http://datatracker.ietf.org/doc/draft-khasnabish-vmmi-problems
> Htmlized:
> http://tools.ietf.org/html/draft-khasnabish-vmmi-problems-02
> Diff:
> http://www.ietf.org/rfcdiff?url2=draft-khasnabish-vmmi-problems-02
> Abstract:
>    In this draft, we discuss the challenges and requirements related to
>    migration, mobility, and interconnection of Virtual Machines (VMs)and
>    Virtual Network Elements (VNEs).  VM migration scheme across IP
>    subnets is needed to implement virtual computing resources sharing
>    across multiple network administrative domains.  For the seamless
>    online migration in various scenarios, many problems are needed to be
>    resolved on the control plane.  The VM migration process should be
>    adapted to these aspects.  We also describe the limitations of
>    various types of virtual local area networking (VLAN) and virtual
>    private networking (VPN) techniques that are traditionally expected
>    to support such migration, mobility, and interconnections.****
>
>