Subject: [arp222] ARP222 Charter statement and milestones

From: Linda Dunbar <[email-masked]>
Date: Thu, 12 Aug 2010 18:14:26 -0500

Thank you all for coming to ARP222 Bar BOF at the 78th IETF and giving us
comments and suggestions on and between sessions. 

I put together the initial ARP222 Charter Statement and Milestones. Please
provide comments and suggestions.  

 

ARP 222: Address Resolution Protocol for Layer 2 to Anything to Layer 2

Description of Working Group:

As server virtualization is introduced to data centers, the number of hosts
in a data center can grow dramatically because each physical server, which
used to host one end-station, now can host many end-stations, or Virtual
Machines (20, 30, or hundreds of). Virtual Machines, with its flexible
add/delete and mobility features, not only makes it possible for achieving
better performance and better utilization on servers, they are also a very
important building block for Cloud Computing service to offer virtual
subnets and virtual hosts. The virtual subnets offered by Cloud Computing
service could allow clients to define their own subnets with its own IP
addresses and policies. 

This rapid growth of virtual hosts could tremendously impact to networks and
servers. One huge issue is frequent address resolution (IPv4) or neighbor
discovery (IPv6) requests from hosts. All hosts frequently send out those
requests due to their cache being aged out in minutes. With tens of
thousands of hosts (each with a distinct MAC address) in one Data Center,
the amount of address resolution packets per second is potentially more than
1,000 to 10,000/second. This rate imposes tremendous computational burden on
many hosts. 

Another big issue associated with huge number of virtual hosts in a data
center is potentially duplicated IP addresses within one VLAN which will
make traditional ARP or ND not working properly. Some load balance design
requires multiple hosts serving the same application to have the same IP
address but with different MAC addresses. Cloud Computing service could
allow users to have their own subnets with IP addresses and self defined
policies among those subnets. Some network designs need to put multiple
client subnets into one VLAN because the number of client subnets could be
in hundreds of thousands which is much more than 4095 VLANs. Under this
scenario, there could be duplicated IP addresses which are from different
client subnets ending up in one VLAN.  

The goal of this working group is to develop interoperable solutions to
solve those problems.  

The design should consider the following properties:

*	All solutions developed by ARP222 WG should not expect any behavior
changes on hosts, applications, or Virtual Machines being deployed in the
market. 
*	All solutions developed should not break DHCP. 
*	Evaluating the impact to IPv6 ND, and develop solutions accordingly
if needed. 
*	Should consider variety of solutions, including directory based,
proxy based, or cache based solutions. 
*	Include analysis of security concerns of IPv4 ARP requests from
malicious users. Evaluating potential security solutions and conclude if the
security threat can justify solutions. 
*	ARP222 assumes the direct links to individual hosts and virtual
machines are IEEE802.3 Ethernet links.  
*	Should consider scenarios of one Ethernet network being
interconnected by another network, which can be L2VPN, pure IP, Ethernet, or
others. 
*	Should consider address resolution solutions for one VLAN with small
number of duplicated IP addresses.  

Here are the items which should not be in the scope of the working group: 

*	Re-define DHCP behavior
*	Re-define security concern to IPv6 ND 
*	Direct links from hosts and virtual hosts are non Ethernet links 
*	 

Goals and Milestones:

*	Charter statement
*	Problem Statements
*	Gap analysis 
*	Study of NHRP (RFC2332) & SCSP,  and their applicability to Ethernet
networks
*	Study and Analysis of MOOSE as a potential solution
*	Study and Analysis of SEATTLE as a potential solution.

Best Regards, Linda Dunbar

---

From: T Sridhar <[email-masked]>
Date: Fri, 13 Aug 2010 12:01:36 -0700

Linda,

I am not comfortable with the following bullets in the Goals & Milestones section. There may be other approaches that could be considered once we have the problem statement nailed down, but the specific approaches shouldn't be part of the goals.

 *   Study of NHRP (RFC2332) & SCSP,  and their applicability to Ethernet networks
 *   Study and Analysis of MOOSE as a potential solution
 *   Study and Analysis of SEATTLE as a potential solution.

Thoughts?

Thanks,
Sridhar

From: [email-masked] [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Thursday, August 12, 2010 4:14 PM
To: [email-masked]
Subject: [arp222] ARP222 Charter statement and milestones

Thank you all for coming to ARP222 Bar BOF at the 78th IETF and giving us comments and suggestions on and between sessions.

I put together the initial ARP222 Charter Statement and Milestones. Please provide comments and suggestions.

As server virtualization is introduced to data centers, the number of hosts in a data center can grow dramatically because each physical server, which used to host one end-station, now can host many end-stations, or Virtual Machines (20, 30, or hundreds of). Virtual Machines, with its flexible add/delete and mobility features, not only makes it possible for achieving better performance and better utilization on servers, they are also a very important building block for Cloud Computing service to offer virtual subnets and virtual hosts. The virtual subnets offered by Cloud Computing service could allow clients to define their own subnets with its own IP addresses and policies.

This rapid growth of virtual hosts could tremendously impact to networks and servers. One huge issue is frequent address resolution (IPv4) or neighbor discovery (IPv6) requests from hosts. All hosts frequently send out those requests due to their cache being aged out in minutes. With tens of thousands of hosts (each with a distinct MAC address) in one Data Center, the amount of address resolution packets per second is potentially more than 1,000 to 10,000/second. This rate imposes tremendous computational burden on many hosts.

Another big issue associated with huge number of virtual hosts in a data center is potentially duplicated IP addresses within one VLAN which will make traditional ARP or ND not working properly. Some load balance design requires multiple hosts serving the same application to have the same IP address but with different MAC addresses. Cloud Computing service could allow users to have their own subnets with IP addresses and self defined policies among those subnets. Some network designs need to put multiple client subnets into one VLAN because the number of client subnets could be in hundreds of thousands which is much more than 4095 VLANs. Under this scenario, there could be duplicated IP addresses which are from different client subnets ending up in one VLAN.

The goal of this working group is to develop interoperable solutions to solve those problems.
The design should consider the following properties:

 *   All solutions developed by ARP222 WG should not expect any behavior changes on hosts, applications, or Virtual Machines being deployed in the market.
 *   All solutions developed should not break DHCP.
 *   Evaluating the impact to IPv6 ND, and develop solutions accordingly if needed.
 *   Should consider variety of solutions, including directory based, proxy based, or cache based solutions.
 *   Include analysis of security concerns of IPv4 ARP requests from malicious users. Evaluating potential security solutions and conclude if the security threat can justify solutions.
 *   ARP222 assumes the direct links to individual hosts and virtual machines are IEEE802.3 Ethernet links.
 *   Should consider scenarios of one Ethernet network being interconnected by another network, which can be L2VPN, pure IP, Ethernet, or others.
 *   Should consider address resolution solutions for one VLAN with small number of duplicated IP addresses.

Goals and Milestones:

 *   Charter statement
 *   Problem Statements
 *   Gap analysis
 *   Study of NHRP (RFC2332) & SCSP,  and their applicability to Ethernet networks
 *   Study and Analysis of MOOSE as a potential solution
 *   Study and Analysis of SEATTLE as a potential solution.

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 13 Aug 2010 14:30:26 -0500

Sridhar, 

Absolutely, other approaches should be considered.  The ones listed down
there are the ones already being published by research institutes and NHRP
is developed by IETF for ATM network. The reason we listed them is because
there are people already signed up to write the drafts. The proposed working
group is to solicit more solutions and possibly finalize to a few
interoperable ones.  

Linda 

  _____  

From: T Sridhar [mailto:[email-masked]] 
Sent: Friday, August 13, 2010 2:02 PM
To: Linda Dunbar; [email-masked]
Subject: RE: [arp222] ARP222 Charter statement and milestones

Linda,

I am not comfortable with the following bullets in the Goals & Milestones
section. There may be other approaches that could be considered once we have
the problem statement nailed down, but the specific approaches shouldn't be
part of the goals.  

*        Study of NHRP (RFC2332) & SCSP,  and their applicability to
Ethernet networks

 

Thoughts?

Thanks,

Sridhar

 

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Linda Dunbar
Sent: Thursday, August 12, 2010 4:14 PM
To: [email-masked]
Subject: [arp222] ARP222 Charter statement and milestones

 

*        Should consider address resolution solutions for one VLAN with
small number of duplicated IP addresses.  

*         

Goals and Milestones:

*        Charter statement

*        Problem Statements

*        Gap analysis 

*        Study of NHRP (RFC2332) & SCSP,  and their applicability to
Ethernet networks

---

From: Benson Schliesser <[email-masked]>
Date: Thu, 26 Aug 2010 15:11:50 -0500

Linda -

I'd like to see a few modifications to the charter you've outlined:

1) I've already explained why I think the last design goal (overlapping IP addresses within a single VLAN) should be removed.  I won't reiterate that here.

2) The goal to "not break DHCP" should be modified to include any broadcast and/or multicast protocols.  (Alternatively, broadcast and multicast support can be required by a new goal to be added.)

3) The goal of supporting interconnect scenarios should explicitly include L3VPN.  In case it isn't clear, that implies a goal of support for overlapping IP addresses within the datacenter. (Albeit in different broadcast domains, my comments in #1 above notwithstanding.)

4) A new goal should include a performance analysis of all solutions considered.  This might explicitly acknowledge a role for negative caching approaches, but that shouldn't be a requirement.

Cheers,
-Benson

On 12 Aug 10, at 6:14 PM, Linda Dunbar wrote:

> All solutions developed by ARP222 WG should not expect any behavior changes on hosts, applications, or Virtual Machines being deployed in the market.
> All solutions developed should not break DHCP.
> Evaluating the impact to IPv6 ND, and develop solutions accordingly if needed.
> Should consider variety of solutions, including directory based, proxy based, or cache based solutions.
> Include analysis of security concerns of IPv4 ARP requests from malicious users. Evaluating potential security solutions and conclude if the security threat can justify solutions.
> ARP222 assumes the direct links to individual hosts and virtual machines are IEEE802.3 Ethernet links. 
> Should consider scenarios of one Ethernet network being interconnected by another network, which can be L2VPN, pure IP, Ethernet, or others.
> Should consider address resolution solutions for one VLAN with small number of duplicated IP addresses.  
>  
> Re-define DHCP behavior
> Re-define security concern to IPv6 ND
> Direct links from hosts and virtual hosts are non Ethernet links
>  
>  
> Goals and Milestones:
> Charter statement
> Problem Statements
> Gap analysis
> Study of NHRP (RFC2332) & SCSP,  and their applicability to Ethernet networks
> Study and Analysis of MOOSE as a potential solution
> Study and Analysis of SEATTLE as a potential solution.
>  
>  
> _______________________________________________
> arp222 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/arp222

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 26 Aug 2010 16:17:13 -0500

Benson, 

Thank you very much for the suggestion. Let's see if other people have
similar comments or other suggestions. 

The goal of circulating the tentative Charter statement is to make needed
changes before submitting to the IESG in mid Sept. 

Linda 

  _____  

From: Benson Schliesser [mailto:[email-masked]] 
Sent: Thursday, August 26, 2010 3:12 PM
To: Linda Dunbar; [email-masked]
Subject: Re: [arp222] ARP222 Charter statement and milestones

Linda -

1) I've already explained why I think the last design goal (overlapping IP
addresses within a single VLAN) should be removed.  I won't reiterate that
here.

2) The goal to "not break DHCP" should be modified to include any broadcast
and/or multicast protocols.  (Alternatively, broadcast and multicast support
can be required by a new goal to be added.)

3) The goal of supporting interconnect scenarios should explicitly include
L3VPN.  In case it isn't clear, that implies a goal of support for
overlapping IP addresses within the datacenter. (Albeit in different
broadcast domains, my comments in #1 above notwithstanding.)

4) A new goal should include a performance analysis of all solutions
considered.  This might explicitly acknowledge a role for negative caching
approaches, but that shouldn't be a requirement.

Cheers,

-Benson

 

*         

Goals and Milestones:

*        Charter statement

*        Problem Statements

*        Gap analysis

---

From: Ron Cohen <[email-masked]>
Date: Mon, 30 Aug 2010 19:43:04 +0300

Hi,

My comments - I'm going to echo some of Benson's and Anoop's points.

Separate the problem scope into:

1. Singe Data Center (DC)
2. Distributed Data Center (DDC)
3. Cloud Computing (CC)

I'm concerned that we don't have sufficient understanding of the cloud
computing networking environment, and at this point in time we should
include an action item or goal to study the requirements of cloud computing
in this area. We should have CC providers participating in the discussion,
clarifying current practice and outstanding issues. I do not think that any
CC service allows multiple clients on the same broadcast domain due to
security concerns, and all services include some form of built-in firewalls
in the hypervisor level to enforce traffic separation. Until this is
clarified all mention of duplicate IP addresses of multiple clients on the
same VLAN, exhaustion of VLANs, etc should be removed from the charter.

The current problem scope does not mention DDC explicitly, and it should. If
ARP222 becomes a working group some standardized version of Cisco's OTV or
similar will need to worked on.

I think that behavior changes within the hypervisor level should be
considered by ARP222. It is not clear to me whether this is in conflict
with  'All solutions developed by ARP222 WG should not expect any behavior
changes on hosts, applications, or Virtual Machines being deployed in the
market.'

I think that study of MOOSE or SEATLE should not be part of the goals and
milestones. This is not to say that these solutions are not adequate or that
they will not be considered.

I'm not clear why it should be explicitly mentioned that it is not allowed
to re-define DHCP. I suggest to remove.

Best,
Ron

On Fri, Aug 27, 2010 at 12:17 AM, Linda Dunbar <[email-masked]> wrote:

>  Benson,
>
>
>
> Thank you very much for the suggestion. Let’s see if other people have
>  ------------------------------
>
> *From:* Benson Schliesser [mailto:[email-masked]]
> *Sent:* Thursday, August 26, 2010 3:12 PM
>
> *To:* Linda Dunbar; [email-masked]
> *Subject:* Re: [arp222] ARP222 Charter statement and milestones
>
>
>
> Linda -
>
>
>
>   Thank you all for coming to ARP222 Bar BOF at the 78th IETF and giving
> us comments and suggestions on and between sessions.
>
>
>
> *ARP 222: Address Resolution Protocol for Layer 2 to Anything to Layer 2*
>
>
>
> *Description of Working Group:*
>
> This rapid growth of virtual hosts could tremendously impact to networks
> and servers. One huge issue is frequent address resolution (IPv4) or
> neighbor discovery (IPv6) requests from hosts. All hosts frequently send out
> those requests due to their cache being aged out in minutes. With tens of
> *The design should consider the following properties:*
>
> ·        All solutions developed by ARP222 WG should not expect any
> behavior changes on hosts, applications, or Virtual Machines being deployed
> in the market.
>
> ·        All solutions developed should not break DHCP.
>
> ·        Evaluating the impact to IPv6 ND, and develop solutions
> accordingly if needed.
>
> ·        Should consider variety of solutions, including directory based,
> ·        Include analysis of security concerns of IPv4 ARP requests from
> ·        ARP222 assumes the direct links to individual hosts and virtual
> ·        Should consider scenarios of one Ethernet network being
> ·        Should consider address resolution solutions for one VLAN with
> * *
> *Here are the items which should not be in the scope of the working group:
> *
>
> ·        Re-define DHCP behavior
>
> ·        Re-define security concern to IPv6 ND
>
> ·        Direct links from hosts and virtual hosts are non Ethernet links
>
> ·
>
>
>
> *Goals and Milestones:*
>
> ·        Charter statement
>
> ·        Problem Statements
>
> ·        Gap analysis
>
> ·        Study of NHRP (RFC2332) & SCSP,  and their applicability to
> Ethernet networks
>
> ·        Study and Analysis of MOOSE as a potential solution
>
> ·        Study and Analysis of SEATTLE as a potential solution.
>
>
>
>
>

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 30 Aug 2010 18:12:39 -0500

Hi Ron, 

Ron, 

Thank you very much for the suggestions. 

My comments are inserted below:

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf Of Ron
Cohen
Sent: Monday, August 30, 2010 11:43 AM
To: Linda Dunbar
Cc: Benson Schliesser; [email-masked]
Subject: Re: [arp222] ARP222 Charter statement and milestones

Hi,

3. Cloud Computing (CC)

[Linda] The first 2 bullets are valid scenarios for DC. But I don't see the
3rd one as an independent category for the ARP222 (or soon to be renamed
ARMD). I see that the Cloud computing could be some applications loaded to
Data Center. 

[Linda] First of all, including duplicated IP in one VLAN is not a main
objective. We can definitely remove it from the Charter. 

We are working together with our company's Cloud service BU on requirement.
Amazon's Virtual Private Cloud (http://aws.amazon.com/vpc/) allows client to
use their own IP addresses. It is definitely true that they won't allow
communication across different VPCs. As of now, Amazon doesn't even allow
any multicast among hosts within one VPC
(http://developer.amazonwebservices.com/connect/thread.jspa?messageID=150951
). 

There is distinction between Client Controlled subnets (e.g. Amazon's VPC)
and the actual subnets which connect all the physical servers. The Client
Controlled subnets are virtual subnets. When they are assigned to servers,
many of them will be assigned to one physical subnet in order to have the
VMmobility. You can use VLAN to distinguish VPCs. But, you will quickly used
up all the VLANs. 

That is my original intention. 

[Linda] The original name ARP222 (Address Resolution for Layer 2 to Anything
to Layer 2) intends to emphasize that one large Layer 2 could consists of
many pockets of Layer 2 which are interconnected by Layer 3 or MPLS. So it
does want to include DDC. Once we change the name to ARMD, we need to bring
it up in the Charter. 

I think that behavior changes within the hypervisor level should be
considered by ARP222. It is not clear to me whether this is in conflict with
'All solutions developed by ARP222 WG should not expect any behavior changes
on hosts, applications, or Virtual Machines being deployed in the market.' 

[Linda] I don't think the behavior changes within the Hypervisor should be
addressed by ARP222. 

[Linda] Agree. They are just some available solutions on the table which
people agreed to write a draft on. We should lump it together to GAP
analysis or existing solution analysis. 

[Linda] We were told by the ADs to state clearly what is in the scope and
what is not in the scope. That is why we have those statements. We don't
anticipate any changes to DHCP. 

Best,

Ron

Benson, 

Linda 

  _____  

Linda -

Cheers,

-Benson

 

*         

Goals and Milestones:

*        Charter statement

*        Problem Statements

*        Gap analysis

---

From: Ron Cohen <[email-masked]>
Date: Tue, 31 Aug 2010 10:44:29 +0300

Hi Linda,

I'm glad I read your reply twice before answering. I hope I didn't have to
read it thrice...

Please see inline

On Tue, Aug 31, 2010 at 2:12 AM, Linda Dunbar <[email-masked]> wrote:

> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Ron
> Cohen
> *Sent:* Monday, August 30, 2010 11:43 AM
> *To:* Linda Dunbar
> *Cc:* Benson Schliesser; [email-masked]
>
> [Linda] The first 2 bullets are valid scenarios for DC. But I don’t see the

ronc: I understand your approach. I agree that this is a desired
architecture: separate infrastructure and application. A clear layer
separation would in principle allow us to implement DDC over CC etc. However
it is not clear to me how you achieve this desired goal in practice, and
whether such clean separation is not in conflict with practical
considerations. Unless we fully understand how CC is overlaid on top of a
DDC, it is hard to draw any useful requirements from the CC application to
ARMD.

> We are working together with our company’s Cloud service BU on requirement.
> Amazon’s Virtual Private Cloud (http://aws.amazon.com/vpc/) allows client
> to use their own IP addresses. It is definitely true that they won’t allow
> communication across different VPCs. As of now, Amazon doesn’t even allow
> any multicast among hosts within one VPC (
> http://developer.amazonwebservices.com/connect/thread.jspa?messageID=150951).
>
>
>
>
> There is distinction between Client Controlled subnets (e.g. Amazon’s VPC)
ronc: What do you mean by 'assigned to one physical subnet in order to have
VMmobility'? Does each VM have two interfaces, one attached to the client
controlled subnet and one connected to the internal CC provider subnet for
VMmobility?

>
>
>
> The current problem scope does not mention DDC explicitly, and it should.
> If ARP222 becomes a working group some standardized version of Cisco's OTV
> or similar will need to worked on.
>
>
>
> [Linda] The original name ARP222 (Address Resolution for Layer 2 to
> Anything to Layer 2) intends to emphasize that one large Layer 2 could
> consists of many pockets of Layer 2 which are interconnected by Layer 3 or
> MPLS. So it does want to include DDC. Once we change the name to ARMD, we
> need to bring it up in the Charter.
>
>
>
> [Linda] I don’t think the behavior changes within the Hypervisor should be
> addressed by ARP222.
>

ronc: What I meant to say is that once the problem scope is known, and
assuming that a solution requires implementing (part of) it in the
hypervisor, there is no point in rejecting it. The main requirement in my
opinion is that the solution developed should not expect any behavior
changes for the guest OS/applications running on the VMs.

> what is not in the scope. That is why we have those statements. We don’t