Subject: [armd] Layer 2 vs Layer 3 in data centers

From: Rahul Aggarwal <[email-masked]>
Date: Wed, 30 Mar 2011 05:18:13 -0700

To quote the ARMD charter:

"Various requirements for the deployment of VMs in data center networks, 
such as support for VM mobility, has led to architectures in which 
broadcast domains are scaling up to span more switching devices and VM 
servers, and to interconnect more hosts (as represented by VMs)."

The above text seems to make an assumption that "VM mobility" requires 
broadcast domains with large number of switching devices. This assumption 
needs careful evaluation and this evaluation is necessary for the ARMD WG 
to come up with a useful problem statement.

First there is a need to separate "resource fungibility" requirements from 
"seamless VM Mobility" requirements.

Resource fungibility requires layer 2 domains to be stretched across racks 
and even data centers, as when a VM is re-located its layer 2 domain must 
not change. One of the reasons for this is that a layer 2 domain is an 
administrative property of a VM.

However seamless VM mobility does _not_ require VMs, which are in 
different administrative pools, to be placed in the same layer 2 domain. 
If there are optimal routing solutions to route across layer 2 domains 
(read subnets) in the presence of seamless VM mobility, then layer 2 domains can 
be restricted in size to only what is necessary. There is a mis-conception 
today that optimal forwarding in the presence of VM mobility requires 
VMs to be in the same layer 2 domain.

To summarise the above, if one were to "route when you can and switch when 
you must" then it reduces the ARP scaling problem. Ofcourse this may 
require re-using existing routing technologies in novel ways or 
extensions to them to support optimal forwarding across subnets. Also this 
does not mean that ARP scaling solutions within a given layer 2 domain 
are not relevant.

The ARMD WG needs to spell out precise reasons for large layer 2 domains 
and determine which of these are necessary in which environments. A side 
effect of this is that ARMD may need to produce requirements for routing 
technologies. Looking at the entire system i.e., layer 2 and layer 3 will 
lead ARMD to a clearer problem statement and eventually to better 
solutions to scaling ARP.

rahul

---

From: xuxiaohu 41208 <[email-masked]>
Date: Wed, 30 Mar 2011 14:54:45 +0200

Hi Rahul,

Did you mean that VMs just need to be located within a subnet, rather than a layer 2 domain, so as to support VM mobility? 

Best regards,
Xiaohu

----- 原邮件 -----
发件人: Rahul Aggarwal <[email-masked]>
日期: 星期三, 三月 30日, 2011 下午2:24
主题: [armd] Layer 2 vs Layer 3 in data centers
收件人: [email-masked]

> _______________________________________________
> armd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/armd
>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 30 Mar 2011 12:59:12 +0000

If you get into the debate why Layer 2 (but Layer 3 can do everything), then this will turn into a never ending debate. 

Linda

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Rahul Aggarwal
Sent: Wednesday, March 30, 2011 7:18 AM
To: [email-masked]
Subject: [armd] Layer 2 vs Layer 3 in data centers

rahul

---

From: Benson Schliesser <[email-masked]>
Date: Wed, 30 Mar 2011 08:01:34 -0500

Rahul -

> servers, and to interconnect more hosts (as represented by VMs)."

This text provides context for our work.  The topic of Virtual Machines,
mobility, etc, is out of scope as a primary topic of investigation for ARMD.

> and determine which of these are necessary in which environments.

Your message has a number of interesting technical points.  However, the
charter of ARMD is intentionally narrow, and the chairs have received clear
direction from our AD - we will focus on the ARP/ND problem only, in order
to make progress on a specific problem.

The chairs will discuss this at our upcoming session.

Cheers,
-Benson

---

From: Rahul Aggarwal <[email-masked]>
Date: Thu, 31 Mar 2011 05:46:52 -0700

Linda,

On Wed, 30 Mar 2011, Linda Dunbar wrote:

> If you get into the debate why Layer 2 (but Layer 3 can do everything),

Apparently you didn't read my email. I didn't say "Layer 3 can 
do everything". To just quote one sentence of 
my email:

" Looking at the entire system i.e., layer 2 and layer 3 will
lead ARMD to a clearer problem statement and eventually to better
solutions to scaling ARP."

And:

"Also this does not mean that ARP scaling solutions within a given layer 2 
domain are not relevant."

> then this will turn into a never ending debate.
>

Any debate that helps this WG to get operator input and do any useful work 
is good. From the meeting yesterday its fairly clear that at present the 
foundation of this WG is based on fairly little operator input.

rahul

---

From: Rahul Aggarwal <[email-masked]>
Date: Thu, 31 Mar 2011 05:54:26 -0700

Benson,

Rahul> The ARMD WG needs to spell out precise reasons for large layer 2 domains
Rahul> and determine which of these are necessary in which environments.

Benson> Your message has a number of interesting technical points.  However, the
> to make progress on a specific problem.

Your reading of the charter is fairly different from mine and also that of 
other people as shown by comments made in the meeting yesterday. The 
charter as written is not "narrow". For example from the charter:

"Problem statement and review of current L2/L3 architectures"

"Recommendations on data center L2/L3 architectures and
identification of opportunities for protocol development work"

Why do you think the above statements in the charter are "narrow"?

rahul

---

From: Benson Schliesser <[email-masked]>
Date: Thu, 31 Mar 2011 09:15:32 -0500

Rahul -

> charter as written is not "narrow". For example from the charter:

My reading of the charter is consistent with the direction of our ADs.  If
the charter text needs to be updated, to make clear the IESG's intent, then
we can discuss that.

> Why do you think the above statements in the charter are "narrow"?

To be perfectly clear: I didn't write our current charter; I was asked to
help manage the WG after it was written.  My personal view is that the above
statements, taken by themselves, are so broad as to be meaningless in the
context of a working group - they do not lead us toward a specific /
actionable goal.  Thus, they must be interpreted in the context of our
objectives: to define possible scale problems in L3-L2 address resolution.

During yesterday's ARMD session I presented an overview of our charter and
problem statement scope.  You can review the presentation slides at
http://tools.ietf.org/agenda/80/slides/armd-4.pdf.  At this time,
contributions to the WG will be held against that standard.  If you want to
work on something else, then ARMD is not the working group you¹re looking
for.  But I encourage you to contribute within the WG scope.

Cheers,
-Benson

---

From: "Shah, Himanshu" <[email-masked]>
Date: Thu, 31 Mar 2011 11:06:13 -0400

I agree with Ben's interpretation of the charter.
Why-not-L3-solutions or lets-first-see-if-network-buildout-is-correct
would be an open ended exercise with no foreseeable outcome.

IMO,
himanshu 

-----Original Message-----
From: [email-masked] on behalf of Benson Schliesser
Sent: Thu 3/31/2011 10:15 AM
To: Rahul Aggarwal
Cc: [email-masked]
Subject: Re: [armd] Layer 2 vs Layer 3 in data centers
 
Rahul -

Cheers,
-Benson

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 31 Mar 2011 11:45:51 -0500

Rahul,

 One of the ARMD goals is to get the operators' opinions expressed to make
sure we are addressing the right problems and to increase the community
input. This is explicit in the charter.

We have talked to many data center operators (e.g. Microsoft being one of
the authors). We will talk to more operators at the next NANOG. Hopefully
the next version of problem statements will reflect more problems from
operator's view. Of course, there are thousands of Data centers, all with
different network designs. Some may (and will) be missed.

Linda
On Thu, Mar 31, 2011 at 7:46 AM, Rahul Aggarwal <[email-masked]> wrote:

> Apparently you didn't read my email. I didn't say "Layer 3 can do
> everything". To just quote one sentence of my email:
>
>