Subject: [nvo3] Comments to draft-khasnabish-vmmi-problems-02

From: Linda Dunbar <[email-masked]>
Date: Fri, 2 Nov 2012 15:49:48 +0000

Bhumip, et, al,

This draft has covered issues associated with many aspects of data centers. Some of those aspects are out of scope of NVo3. For example: Requirement of Resource Allocation Gateway which deals with compute & storage resource management. It is definitely out of the scope of NVo3. I am not even sure if they are within the scope of IETF.
"Firewall state maintenance" after VM migration should be out of the scope of NVo3 as well.

"Migration scheduling" should be out of scope of NVo3.

The draft also describes switches within data center, I don't think  it is necessary to have yet another draft describing data center typical architecture. Please refer to "http://datatracker.ietf.org/doc/draft-ietf-armd-problem-statement/"  which has a very comprehensive description of typical DC architecture (Section 6: Generalized Data Center Design).

The draft has many sections in describing why there is VM migration in data centers and the limitation of 4K VLAN to  "accommodate a large number of tenants", which have been very well documented in the "NVo3 problem statement ". I don't think it is necessary to have another draft repeating the same thing.

There are also some description which are not quite correct:

-        3.2.14: It is incorrect to list IEEE802.1Qbg and Vepa as solution to East - West traffic problem.

-        Section 3.1 (The Evolution Problems of the Logical Network Topology in VMMI environments):
I don't think the following statement is valid:  "Since a large number of VMs and their applications are running in the same Layer-2 domain, it (VM migration) may be very stressful from bandwidth utilization viewpont of the data center switching network."

Applications communicate with each other via IP addresses. Yes, some instances are placed in the same subnet. VM migration is not the main reason for bandwidth utilization. There are many reasons for (poor) bandwidth utilization.

"In order to improve the bandwidth utilization, it is required to upgrade the load balancing capability of the network which has numerous ECMP between different points."

This "load balance" has nothing to do with VM mobility.

I don't understand how "Matrix DCN" have anything to do with VM mobility either.

-        The whole section of Cloud Service Virtualization requirement is out of scope of NVo3.

-        3.2.7 VN requirements: it doesn't have anything new which haven't been addressed by "nvo3 problem statement".

-         3.2.8: packet encapsulation problems: The section describes several encapsulation mechanisms: PBB, NvGRE, VxLAN. There is nothing new here. I can understand if you write some real issues associated with each of those encapsulation. But simply listing them down for education purpose is not necessary.

-        3.2.12: "traffic roundabout problem" has been very well described in "draft-rekhter-nvo3-vm-mobility-issues" under "optimal routing section". I don't think it is necessary to repeat.

-        I don't quite understand why you say "VxLAN"/NvGRE are encapsulation, NVo3 is tunneling (Section 3.2.13)

Linda Dunbar

From: [email-masked] [mailto:[email-masked]] On Behalf Of Richard Bin liu
Sent: Thursday, October 18, 2012 12:44 PM
To: [email-masked]
Subject: [nvo3] draft-khasnabish-vmmi-problems-02

Greeting all,

We had published this draft and wanted to bring it to the NVO3 WG's attention.  We plan to present/discuss this draft at the upcoming meeting as well, consequently please discuss here on the maillist as well.

Thanks,

Bhumip Khasnabish, et al.

A new version of I-D, draft-khasnabish-vmmi-problems-02.txt
has been successfully submitted by Bin Liu and posted to the
IETF repository.
Filename:        draft-khasnabish-vmmi-problems
Revision:        02
Title:           Requirements for Mobility and Interconnection of Virtual Machine and Virtual Network Elements
Creation date:   2012-10-15
WG ID:           Individual Submission
Number of pages: 46
URL:             http://www.ietf.org/internet-drafts/draft-khasnabish-vmmi-problems-02.txt
Status:          http://datatracker.ietf.org/doc/draft-khasnabish-vmmi-problems
Htmlized:        http://tools.ietf.org/html/draft-khasnabish-vmmi-problems-02
Diff:            http://www.ietf.org/rfcdiff?url2=draft-khasnabish-vmmi-problems-02
Abstract:
   In this draft, we discuss the challenges and requirements related to
   migration, mobility, and interconnection of Virtual Machines (VMs)and
   Virtual Network Elements (VNEs).  VM migration scheme across IP
   subnets is needed to implement virtual computing resources sharing
   across multiple network administrative domains.  For the seamless
   online migration in various scenarios, many problems are needed to be
   resolved on the control plane.  The VM migration process should be
   adapted to these aspects.  We also describe the limitations of
   various types of virtual local area networking (VLAN) and virtual
   private networking (VPN) techniques that are traditionally expected
   to support such migration, mobility, and interconnections.

---

From: Richard Bin liu <[email-masked]>
Date: Sat, 3 Nov 2012 02:05:58 +0800

Hi Linda,

If any part does not belongs to nvo3 scope, we could consider to put it to
another draft.

3.2.12: "traffic roundabout problem" has been very well described in "draft
- rekhter - nvo3 - vm - mobility - issues" under "optimal routing section".
I don "t think it is necessary to repeat.

>From this numbers(3.1.2) we can see, you read a old version, and I had read
your draft cafully( rekhter - nvo3 - vm - mobility - issues), my 'traffic
roundabout problem' and your "optimal routing" section, the contents of the
discussion is not the same.
3.2.14: It is incorrect to list IEEE802.1Qbg and Vepa as solution to East –
West traffic problem.
Collaborative scheme of Server-etheric network(Qbg and Vepa ), it is a kind
of mutual compromise solutions, it is a idea of combination of server
network and the etheric network, its flow model  is different from the
traditional patterns, so you can't arbitrarily said it is not correct. In
logical thinking, either this or that is not a good way.

Thank you for your comments, next, all of your commenets will be reply
earnestly.

Richard
On Fri, Nov 2, 2012 at 11:49 PM, Linda Dunbar <[email-masked]>wrote:

>  Bhumip, et, al, ****
>
> ** **
>
> This draft has covered issues associated with many aspects of data
> centers. Some of those aspects are out of scope of NVo3. For example:
> Requirement of Resource Allocation Gateway which deals with compute &
> storage resource management. It is definitely out of the scope of NVo3. I
> am not even sure if they are within the scope of IETF. ****
>
> “Firewall state maintenance” after VM migration should be out of the scope
> of NVo3 as well. ****
>
> ** **
>
> “Migration scheduling” should be out of scope of NVo3. ****
>
> ** **
>
> ** **
>
> The draft also describes switches within data center, I don’t think  it is
> necessary to have yet another draft describing data center typical
> architecture. Please refer to “
> http://datatracker.ietf.org/doc/draft-ietf-armd-problem-statement/”
>  which has a very comprehensive description of typical DC architecture
> (Section 6: Generalized Data Center Design).****
>
> ** **
>
> The draft has many sections in describing why there is VM migration in
> data centers and the limitation of 4K VLAN to  “accommodate a large number
> of tenants”, which have been very well documented in the “NVo3 problem
> statement “. I don’t think it is necessary to have another draft repeating
> the same thing. ****
>
> ** **
>
> ** **
>
> There are also some description which are not quite correct:****
>
> ** **
>
> **-        **3.2.14: It is incorrect to list IEEE802.1Qbg and Vepa as
> solution to East – West traffic problem. ****
>
> **-        **Section 3.1 (The Evolution Problems of the Logical Network
> Topology in VMMI environments):****
>
> I don’t think the following statement is valid:  “Since a large number of
> VMs and their applications are running in the same Layer-2 domain, it (VM
> migration) may be very stressful from bandwidth utilization viewpont of the
> data center switching network.”****
>
> ** **
>
> Applications communicate with each other via IP addresses. Yes, some
> instances are placed in the same subnet. VM migration is not the main
> reason for bandwidth utilization. There are many reasons for (poor)
> bandwidth utilization. ****
>
> ** **
>
> “In order to improve the bandwidth utilization, it is required to upgrade
> the load balancing capability of the network which has numerous ECMP
> between different points.”****
>
> ** **
>
> This “load balance” has nothing to do with VM mobility.  ****
>
> ** **
>
> I don’t understand how “Matrix DCN” have anything to do with VM mobility
> either. ****
>
> ** **
>
> **-        **The whole section of Cloud Service Virtualization
> requirement is out of scope of NVo3. ****
>
> **-        **3.2.7 VN requirements: it doesn’t have anything new which
> haven’t been addressed by “nvo3 problem statement”. ****
>
> **-        ** 3.2.8: packet encapsulation problems: The section describes
> several encapsulation mechanisms: PBB, NvGRE, VxLAN. There is nothing new
> here. I can understand if you write some real issues associated with each
> of those encapsulation. But simply listing them down for education purpose
> is not necessary.   ****
>
> **-        **3.2.12: “traffic roundabout problem” has been very well
> described in “draft-rekhter-nvo3-vm-mobility-issues” under “optimal
> routing section”. I don’t think it is necessary to repeat. ****
>
> **-        **I don’t quite understand why you say “VxLAN”/NvGRE are
> encapsulation, NVo3 is tunneling (Section 3.2.13)****
>
> ** **
>
> ** **
>
> Linda Dunbar****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
> Of *Richard Bin liu
> *Sent:* Thursday, October 18, 2012 12:44 PM
> *To:* [email-masked]
> *Subject:* [nvo3] draft-khasnabish-vmmi-problems-02****
>
> ** **
>
> Greeting all,
>
> We had published this draft and wanted to bring it to the NVO3 WG's
> attention.  We plan to present/discuss this draft at the upcoming meeting
> as well, consequently please discuss here on the maillist as well.
>
> Thanks,
>
> URL:
> http://www.ietf.org/internet-drafts/draft-khasnabish-vmmi-problems-02.txt
> Status:
> http://datatracker.ietf.org/doc/draft-khasnabish-vmmi-problems
> Htmlized:
> http://tools.ietf.org/html/draft-khasnabish-vmmi-problems-02
> Diff:
> http://www.ietf.org/rfcdiff?url2=draft-khasnabish-vmmi-problems-02
> Abstract:
>    to support such migration, mobility, and interconnections.****
>
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3
>
>

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 2 Nov 2012 21:10:07 +0000

Richard,

IEEE802.1Qbg is for hypervisor to inform port profile, regardless if the traffic is east-west or north-east.

VEPA is to push communication among VMs on a server to an external switch, regardless if the traffic is east-west or north east.

That is why it is incorrect to say that IEEE802.1Qbg and VEPA are the solution to "east-west traffic" problem.

I will read your draft again to understand "roundabout" issue if it is not same as the "optimal routing in our draft.

It would make it easier for the community if  you can break the draft to multiple ones with some focusing on NVo3 relevant aspects. For the aspects which are out of scope of NVo3, you can have the discussion in more relevant groups or SDOs.

Linda

3.2.14: It is incorrect to list IEEE802.1Qbg and Vepa as solution to East - West traffic problem.
Collaborative scheme of Server-etheric network(Qbg and Vepa ), it is a kind of mutual compromise solutions, it is a idea of combination of server network and the etheric network, its flow model  is different from the traditional patterns, so you can't arbitrarily said it is not correct. In logical thinking, either this or that is not a good way.

Thank you for your comments, next, all of your commenets will be reply earnestly.

Richard
On Fri, Nov 2, 2012 at 11:49 PM, Linda Dunbar <[email-masked]<mailto:[email-masked]>> wrote:
Bhumip, et, al,

Linda Dunbar

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Richard Bin liu
Sent: Thursday, October 18, 2012 12:44 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [nvo3] draft-khasnabish-vmmi-problems-02

Greeting all,

Thanks,

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: Richard Bin liu <[email-masked]>
Date: Thu, 8 Nov 2012 23:40:39 +0800

Hello Linda,

The whole section of Cloud Service Virtualization requirement is out of
scope of NVo3.
Many content which you discussed is belong to this section, so you say
'whole' is not
appropriate. I think, except 3.6.2, 3.6.4, 3.6.19, other are nvo3 relevant
part.
So, these sections(3.6.2,3.6.4,3.6.19) would be moved to another draft.
Thank you for your comments.
Section 3.1 (The Evolution Problems of the Logical Network Topology in VMMI
environments):

I don’t think the following statement is valid:  “Since a large number of
VMs and their applications are running in the same Layer-2 domain, it (VM
migration) may be very stressful from bandwidth utilization viewpont of the
data center switching network.”

bandwidth utilization.

“In order to improve the bandwidth utilization, it is required to upgrade
the load balancing capability of the network which has numerous ECMP
between different points.”

This “load balance” has nothing to do with VM mobility.

About these comments, you could read reply from ytsun.

And I have read your draft(draft-rekhter-nvo3-vm-mobility-issues-03).I
quite agree with you about

the content of section 3.4 (Optimal IP Routing):

+ Optimal routing of a VM's outbound traffic. This means that as a given VM
moves from one server to another, the VM's default gateway should be in a
close topological proximity to the ToR that connects the server presently
hosting that VM.

Obviously, you think that there is close relation between 'the Optimal
routing of a VM's

outbound traffic' and network topology, and this with our draft in Section
3.1 (The Evolution

Problems of The Logical Network Topology in VMMI environments) want to
express the same point of

view, and yhe target is consistent. So, from your draft we can know, your
draft of view is agree

with our draft's point of view. Don't you then changed your mind, want to
overthrow viewpoint of

yourself?

In fact, we are all from different perspective, discussing the same problem.

Similarly, for data center fat tree network in multipath forward, TRILL can
make good use of

network bandwidth. But in fact, for TRILL, fat tree also is not the best
topology, so in the

control plane (NVO3 / TRILL) still exist promotion space.

Richard

---------------

横看成岭侧成峰，
远近高低各不同。
不识庐山真面目，
只缘身在此山中。

---------------

---

From: ytsun <[email-masked]>
Date: Thu, 8 Nov 2012 16:47:27 -0500

Hi Linda,

You give good comments to the draft. But  I have some different opinions on VM migrations.

For DCN, VM migration is a very important feature. When a server breaks down,  the applications or VMs on the server can move to other physical servers dynamically.
 and in a mega data center, server breaking down is very common.  VM migration need transfer large number of information, it will conduct a sharp traffic requirement 
and put great stress on bandwidth. 

On the other hand, for load balancing or energy saving, some papers have proposed methods to rearrange the applications by VM migration in the layer-2 domain. If those methods are used in DCN, VM migration will come to more frequent than today.

So in my thinking,  even if VM migration is not very frequent, it must be often in future data center.  It will be considered at the beginning of design and deployment of 
a data center. The architecture of  DCN and its bisection bandwidth do affect the efficiency  of VM migration.

By the way, MatrixDCN is a new architecture for DCN like Fat-Tree.  It is easy deployed and managed.

- Section 3.1 (The Evolution Problems of the Logical Network Topology in VMMI environments):
I don’t think the following statement is valid: “Since a large number of VMs and their applications are running in the same Layer-2 domain, it (VM migration) may be very stressful from bandwidth utilization viewpont of the data center switching network.”
Applications communicate with each other via IP addresses. Yes, some instances are placed in the same subnet. VM migration is not the main reason for bandwidth utilization. There are many reasons for (poor) bandwidth utilization. 
“In order to improve the bandwidth utilization, it is required to upgrade the load balancing capability of the network which has numerous ECMP between different points.”
This “load balance” has nothing to do with VM mobility. 
I don’t understand how “Matrix DCN” have anything to do with VM mobility either. 

ytsun