Subject: Re: [OPSAWG] discussion on network automatic adaption to VM migration?

From: "Yingjie Gu(yingjie)" <[email-masked]>
Date: Sat, 26 Mar 2011 05:59:20 +0000

Hi Juergen and all,
I briefly conclude problem statement of virtualization Data Center management and the proposal I would suggest. Hope this can help to make the problem and proposal clearer.

Problem statement:
  The problem I would discuss is not the whole part of DC management, it's only the part related to VM Migration.
  Typically 'Server power-off and power-on again' is beyond scope, I focus on the VM hot migration without running service being significantly interrupted. Also, I focus on network adaption to VM migtation, don't touch OS and memory migration on server side.
  Two network side information need to migrate with VM: Dynamic Information and static policies. Among them I believe the former is much more significant. This doesn't mean the latter is less important, and they could be resolved by the same mechanism.
  Dynamic Information refers to the non-configurable states that is generated during service running, examples are TCP connection table (or say TCP connection state) on middleboxes, dynamic ACL on switches/Firewall, DHCP snooping table item on switches, IGMP snooping table(or say Multicast Membership) on network devices' ports, cumulative data on switches/routers/middleboxes, statistical data on switches/routers/middleboxes, etc. Without Dynamic Information being migrated accurately, services would be disrupted.

KEY COMPONENTS for a valid mechanism to this problem include:
  WHEN: VM migration may take minutes even longer, during which Dynamic Information could keep changing, e.g. new TCP connection is established, or sequence number increased. When we look into VM migration process in attached picture, there is a time called 'VM-not-running', this is the best period to migrate Dynamic Information. A valid mechanism should include efficient way to learn the particular time.
  WHERE: To learn two ends of the migration,i.e. source and destination network devices. No matter using a centralized way or distributed way(directly communication between two ends), a valid mechanism should include the way to learn the two ends. Currently, I don't propose any specific mechanism, this is only a general discussion.
  HOW: Existing NM doesn't cover information migration from one element to another. What's more, the Dynamic Information migration is not definitely one-to-one. Since network architecture could be asymmetrical, so that could also be multiple-to-one and one-to-multiple.

Conclusion and Proposal:
  Currently, I haven't found an existing and valid mechanism, with good interoperability among different KEY COMPONENTS and different parts of a possible mechanism, to resolve the problem.
  So I would suggest to define a new mechanism, including architecture and protocols, to handle this.

It's really good to have discussion on this topic. Any questions and suggestions are highly appreciated.

Best Regards
Yingjie
________________________________________
Send From: [email-masked] [[email-masked]] On behalf of Yingjie Gu(yingjie) [[email-masked]]
2011年3月26日 4:41
To: Juergen Schoenwaelder
Cc: 'yingjie gu'; [email-masked]
Subject: Re: [OPSAWG] discussion on network automatic adaption to VM migration?

Hi Juergen,
See inline please.

____________
____________________________
From:Juergen Schoenwaelder [[email-masked]]
Time : 2011年3月24日 17:57
To: Yingjie Gu(yingjie)
Cc: 'Bert (IETF) Wijnen'; [email-masked]; 'yingjie gu'
Subject: Re: [OPSAWG] discussion on network automatic adaption to VM migration?

On Thu, Mar 24, 2011 at 05:39:41PM +0800, Yingjie Gu(yingjie) wrote:

> I have been following and participating IEEE 802.1 Qbg. Qbg doesn't
> define MIB, it defines protocol that works only between Hypervisor
> (general name for Virtual Switch) and adjacent physical switch to
> associate a VM to an adjacent switch. The following is VDP TLV, you
> can find detail information at IEEE.

So what about the MIB listed here?

http://www.ieee802.org/1/pages/802.1bg.html
[Yingjie] Sorry for the mistake.
And terminology wise, I disagree with the notion that a Hypervisor is
a general name for a Virtual Switch.
[Yingjie] Well, not formally, but Hypervisor refers to VEPA/VEB/other kind of virtual switch type.

> Let’s skip the VDP part for now. Assuming the adjacent switch already learned VM. Here are the parts that we look for to make a complete mechanism for policy and dynamic information migration:
>
> 1) When? For static polices, NM can start polices reconfiguration any time (operated by a human) or start at the same time as dynamic information migration (automatically, without human intervention), but should be finished not too later than VM restart at destination devices.  For dynamic information, we need to know the exact time when VM-NOT-RUNNING.
>
> 2) Where? Network manager needs to know the source and destination devices. API from VM Manager can not provide such information. Both adjacent switch and upper layer devices, e.g aggregation switch, router and middleboxes, need to know the migration state if we want to migrate dynamic information. VDP can not achieve this.
>
> 3) How? NETCONF and other existing NMP can work for policy reconfiguration, as long as the above two parts can be resolved pretty well.
>
> For dynamic information, we need a new mechanism (including architecture and protocols), I don’t think NETCONF covers this.

Sounds too abstract too me. You are already sure you need new
architecture and protocols while I am still not sure what the problem
to be solved is. It sounds like you want communication between the VM
management system and the network management system. If so, to have a
chance of success in terms of implementation of deployment, you would
have to have organizations building VM management system and network
management systems excited about the idea.

[Yingjie] Not communication between VM manager and network manager. This is only a way of migration notification, and I said that is not a standadized way, and even there is a need for standard, it's not IETF SCOPE I guess.

> If you can review my draft, I list some examples in my drafts.
>
> Examples are TCP connection table, Multicast membership on ports, DHCP Snooping, cumulative data, etc.
>

I already choked up on the TCP connection table. What I call the TCP
connection table sits in the kernel of the OS being migrated, so I
fail to see what needs to be done there. I had the impression you were
thinking more about connection state maintained by middleboxes, but
your document just did not make this clear enough to me.
[Yingjie] TCP connection table on OS needn't to be done, you are right.
I am talking the TCP connection table on Middlebox, and it's only an example. Maybe there is another way to name it.
I will try to make it clearer.
Thank you for your comments.
[end]

/js

--
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>
_______________________________________________
OPSAWG mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/opsawg