Subject: Re: [armd] soliciting typical network designs for ARMD

From: <[email-masked]>
Date: Fri, 16 Sep 2011 09:14:15 -0400

And two more drafts on this topic:

http://www.ietf.org/id/draft-mahalingam-dutt-dcops-vxlan-00.txt
http://www.ietf.org/id/draft-sridharan-virtualization-nvgre-00.txt

The edge switches could be the software switches in hypervisors. 

Thanks,
--David

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of Warren Kumari
> Sent: Wednesday, August 31, 2011 3:16 PM
> To: Vishwas Manral
> Cc: [email-masked]
> Subject: Re: [armd] soliciting typical network designs for ARMD
> 
> 
> On Aug 11, 2011, at 11:40 PM, Vishwas Manral wrote:
> 
> > Hi Linda/ Anoop,
> >
> > Here is the example of the design I was talking about, as defined by google.
> 
> Just a clarification -- s/as defined by google/as described by someone who happens to work for google/
> 
> W
> 
> > http://www.ietf.org/id/draft-wkumari-dcops-l3-vmmobility-00.txt
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani <[email-masked]> wrote:
> >
> > >>>>
> > (though I think if there was a standard way to map Multicast MAC to Multicast IP, they could
> probably use such a standard mechanisms).
> > >>>>
> >
> > They can do that, but then this imposes requirements on the
> > equipment to be able to do multicast forwarding, and even if does,
> > because of pruning requirements the number of groups would be
> > very large.  The average data center switch probably won't handle
> > that many groups.
> >
> > On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral <[email-masked]> wrote:
> > Hi Anoop,
> >
> > From what I know they do not use Multicast GRE (I hear the extra 4 bytes in the GRE header is a
> proprietery extension).
> >
> > I think a directory based mechanism is what is used (though I think if there was a standard way to
> map Multicast MAC to Multicast IP, they could probably use such a standard mechanisms).
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani <[email-masked]> wrote:
> > Hi Vishwas,
> >
> > How do they get multicast through the network in that case?
> > Are they planning to use multicast GRE, or just use directory
> > based lookups and not worry about multicast applications
> > for now?
> >
> > Anoop
> >
> > On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral <[email-masked]> wrote:
> > Hi Linda,
> >
> > The data packets can be tunnelled at the ToR over say a GRE packet and the core is a Layer-3 core
> (except for the downstream ports). So we could have encapsulation/ decapsulation of L2 over GRE at the
> ToR.
> >
> > The very same thing can be done at the hypervisor layer too, in which case the entire DC network
> would look like a Layer-3 flat network including the ToR to server link and the hypervisor would do
> the tunneling.
> >
> > I am not sure if you got the points above or not. I know cloud OS companies that provide the service
> and have big announced customers.
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar <[email-masked]> wrote:
> > Vishwas,
> >
> > In my mind the bullet 1) in the list refers to ToR switches downstream ports (facing servers)
> running Layer 2 and ToR uplinks ports run IP Layer 3.
> >
> > Have you seen data center networks with ToR switches downstream ports (i.e. facing servers) enabling
> IP routing, even though the physical links are Ethernet?
> > If yes, we should definitely include it in the ARMD draft.
> >
> > Thanks,
> > Linda
> > On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral <[email-masked]> wrote:
> > Hi Linda,
> > I am unsure what you mean by this, but:
> > 	* layer 3 all the way to TOR (Top of Rack switches),
> > We can also have a heirarchical network, with the core totally Layer-3 (and having seperate
> routing), from the hosts still in a large Layer-3 subnet. Another aspect could be to have a totally
> Layer-3 network.
> >
> > The difference between them is the link between the servers and the ToR.
> >
> > Thanks,
> > Vishwas
> > On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar <[email-masked]> wrote:
> > During the 81st IETF ARMD WG discussion, it was suggested that it is necessary to document typical
> data center network designs so that address resolution scaling issues can be properly described. Many
> data center operators have expressed that they can't openly reveal their detailed network designs.
> Therefore, we only want to document anonymous designs without too much detail. During the journey of
> establishing ARMD, we have come across the following typical data center network designs:
> > 	* layer 3 all the way to TOR (Top of Rack switches),
> > 	* large layer 2 with hundreds (or thousands) of ToRs being interconnected by Layer 2. This
> design will have thousands of hosts under the L2/L3 boundary router (s)
> > 	* CLOS design  with thousands of switches. This design will have thousands of hosts under the
> L2/L3 boundary router(s)
> > We have heard that each of the designs above has its own problems. ARMD problem statements might
> need to document DC problems under each typical design.
> > Please send feedback to us (either to the armd email list  or to the ARMD chair Benson & Linda) to
> indicate if we have missed any typical Data Center network designs.
> >
> > Your contribution can greatly accelerate the progress of ARMD WG.
> >
> > Thank you very much.
> >
> > Linda & Benson
> >
> >
> 
> _______________________________________________
> armd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/armd

---

From: Murari Sridharan <[email-masked]>
Date: Sat, 17 Sep 2011 20:02:19 +0000

FYI, here is a talk that I gave last week in relation to the nvgre draft below. 
http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-442T

Thanks
Murari 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Friday, September 16, 2011 6:14 AM
To: [email-masked]
Subject: Re: [armd] soliciting typical network designs for ARMD

Thanks,
--David

> > because of pruning requirements the number of groups would be very 
> > large.  The average data center switch probably won't handle that 
> > many groups.
> >
> > Are they planning to use multicast GRE, or just use directory based 
> > lookups and not worry about multicast applications for now?
> >
> > Anoop
> >
> (except for the downstream ports). So we could have encapsulation/ 
> decapsulation of L2 over GRE at the ToR.
> >
> would look like a Layer-3 flat network including the ToR to server 
> link and the hypervisor would do the tunneling.
> >
> > 	* layer 3 all the way to TOR (Top of Rack switches), We can also 
> > have a heirarchical network, with the core totally Layer-3 (and 
> > having seperate
> data center network designs so that address resolution scaling issues 
> can be properly described. Many data center operators have expressed that they can't openly reveal their detailed network designs.
> Therefore, we only want to document anonymous designs without too much 
> detail. During the journey of establishing ARMD, we have come across the following typical data center network designs:

---

From: Linda Dunbar <[email-masked]>
Date: Sun, 18 Sep 2011 14:06:30 +0000

Hi Murari, 

Thank you very much for sharing the presentation. 

One question: 

For a host within an Enterprise site which needs to communicate with external peers, the host either uses public IP address which is visible to external peers or uses private IP address which is translated to public address at the Enterprise site's gateway.  

When this host is moved to "Cloud data center", will the "Cloud Data center" advertise this host address to external peers? Or will all external peers go through enterprise's gateway to reach this host which is no longer residing in the enterprise site? 

Thanks, Linda

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Murari Sridharan
> Sent: Saturday, September 17, 2011 3:02 PM
> To: [email-masked]; [email-masked]
> > Just a clarification -- s/as defined by google/as described by
> someone
> > who happens to work for google/
> >
> > W
> >
> > > During the 81st IETF ARMD WG discussion, it was suggested that it
> is
> > > necessary to document typical
> > can be properly described. Many data center operators have expressed
> that they can't openly reveal their detailed network designs.
> > detail. During the journey of establishing ARMD, we have come across
> the following typical data center network designs:
> > > Please send feedback to us (either to the armd email list  or to
> the
> > > ARMD chair Benson & Linda) to

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Sun, 18 Sep 2011 18:01:29 +0000

The easiest from the point of view of configuration would be to route everything back through the enterprise - not necessarily the optimal from the enterprise point of view. Are you referring to a scenario where the VMs subnet is split between the cloud and the enterprise? Otherwise I don't see the implication on virtualization as its no different than getting the traffic routed to the enterprise in the first case.

Simha

________________________________________
From: [email-masked] [[email-masked]] on behalf of Linda Dunbar [[email-masked]]
Sent: Sunday, September 18, 2011 7:06 AM
To: Murari Sridharan; [email-masked]; [email-masked]
Subject: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Hi Murari,

One question:

Thanks, Linda

> [email-masked]

---

From: "Balus, Florin Stelian (Florin)" <[email-masked]>
Date: Mon, 19 Sep 2011 18:36:37 -0500

These two drafts and draft-wkumari-dcops-l3-vmmobility-00.txt seem to be submitted as either informational or experimental. Can the authors of these two drafts clarify what do they plan to do with them? If relevant feedback is required I think they seem to fit the scope of L2VPN WG. At least two of them talk about some new encapsulation used to provide L2 Multi-tenancy in the Data Centers over some sort of overlay and UDP/IP tunnels. A new L2VPN charter was released recently which includes requirements and solutions targeted at the Data Center space.

Florin

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Murari Sridharan
Sent: Saturday, September 17, 2011 1:02 PM
To: [email-masked]; [email-masked]
Subject: Re: [armd] soliciting typical network designs for ARMD

Thanks
Murari 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Friday, September 16, 2011 6:14 AM
To: [email-masked]
Subject: Re: [armd] soliciting typical network designs for ARMD

Thanks,
--David

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 20 Sep 2011 14:58:20 +0000

Simha, 

What if entire subnet is moved to "cloud data center"? For applications which communicate with external peers, do they go through the enterprise data center? What if the enterprise data center is removed after moving all the applications to "cloud"? 

Linda

> From: Narasimhan Venkataramaiah [mailto:[email-masked]]
> Sent: Sunday, September 18, 2011 1:01 PM
> To: Linda Dunbar; Murari Sridharan; [email-masked]; [email-masked]
> Subject: RE: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> advertise its external facing hosts' IP addresses to external world?
> 
> The easiest from the point of view of configuration would be to route
> everything back through the enterprise - not necessarily the optimal
> from the enterprise point of view. Are you referring to a scenario
> where the VMs subnet is split between the cloud and the enterprise?
> Otherwise I don't see the implication on virtualization as its no
> different than getting the traffic routed to the enterprise in the
> first case.
> 
> Simha
> 
> Subject: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> advertise its external facing hosts' IP addresses to external world?
> 
> Hi Murari,
> 
> For a host within an Enterprise site which needs to communicate with
> external peers, the host either uses public IP address which is visible
> to external peers or uses private IP address which is translated to
> public address at the Enterprise site's gateway.
> 
> When this host is moved to "Cloud data center", will the "Cloud Data
> center" advertise this host address to external peers? Or will all
> external peers go through enterprise's gateway to reach this host which
> is no longer residing in the enterprise site?
> 
> Thanks, Linda
> 
> > > From: [email-masked] [mailto:[email-masked]] On
> Behalf
> > > Of Warren Kumari
> > > > (though I think if there was a standard way to map Multicast MAC
> to
> > > > Multicast IP, they could
> very
> > > > From what I know they do not use Multicast GRE (I hear the extra
> 4
> > > > bytes in the GRE header is a
> > > proprietery extension).
> > > >
> > > > I think a directory based mechanism is what is used (though I
> think
> > > > if there was a standard way to
> based
> > > > The data packets can be tunnelled at the ToR over say a GRE
> packet
> > > > and the core is a Layer-3 core
> > > > On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar
> <[email-masked]>
> > wrote:
> > > > Vishwas,
> > > >
> > > > The difference between them is the link between the servers and
> the
> > ToR.
> > > >
> > > > Thanks,
> > > > Vishwas
> > > > On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar
> <[email-masked]>
> > wrote:
> > > >   * CLOS design  with thousands of switches. This design will
> have
> > > > thousands of hosts under the
> > > L2/L3 boundary router(s)

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Tue, 20 Sep 2011 15:15:37 +0000

I am trying to understand the implication to network virtualization. Say an enterprise has 2 data centers in different geographical regions and they decided to move a subnet from one DC to another - whatever is done to route the subnet to the new data center applies here too. What is different for a virtualized network unless the subnet spans 2 data centers?

Simha

-----Original Message-----
From: Linda Dunbar [mailto:[email-masked]] 
Sent: Tuesday, September 20, 2011 7:58 AM
To: Narasimhan Venkataramaiah; Linda Dunbar; Murari Sridharan; [email-masked]; [email-masked]
Subject: RE: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Simha, 

Linda

> external peers, the host either uses public IP address which is 
> visible to external peers or uses private IP address which is 
> translated to public address at the Enterprise site's gateway.
> 
> external peers go through enterprise's gateway to reach this host 
> which is no longer residing in the enterprise site?
> 
> Thanks, Linda
> 
> > > > Here is the example of the design I was talking about, as 
> > > > defined
> > by google.
> > >
> > > > equipment to be able to do multicast forwarding, and even if 
> > > > does, because of pruning requirements the number of groups would 
> > > > be
> very
> > > > large.  The average data center switch probably won't handle 
> > > > that many groups.
> > > >
> > > design will have thousands of hosts under the L2/L3 boundary 
> > > router
> > > (s)

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 16:03:36 -0700

Hi Simha,

The main (Standards Track) change in your draft is the addition of TNI.

A question I have is a TNI identifies a particular tenant and all flows
from/to a tenant will be hashed to the same path (even with the changes in
switches to do hashing to use TNI).

Why do you not use the last 8 bits which you have kept as reserved for
providing the randomization for hashing flows between same to/from on
different paths?

Thanks,
Vishwas
On Sun, Sep 18, 2011 at 11:01 AM, Narasimhan Venkataramaiah <
[email-masked]> wrote:

> everything back through the enterprise - not necessarily the optimal from
> the enterprise point of view. Are you referring to a scenario where the VMs
> subnet is split between the cloud and the enterprise? Otherwise I don't see
> the implication on virtualization as its no different than getting the
> traffic routed to the enterprise in the first case.
>
> Simha
>
> external peers, the host either uses public IP address which is visible to
> external peers or uses private IP address which is translated to public
> address at the Enterprise site's gateway.
>
> center" advertise this host address to external peers? Or will all external
> peers go through enterprise's gateway to reach this host which is no longer
> residing in the enterprise site?
>
> Thanks, Linda
>

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Thu, 22 Sep 2011 23:11:25 +0000

The draft mentions exactly this as one use of the reserved 8 bits in Section 4. An NVGRE endpoint could use the 8 bits to further distribute flows belonging to a particular TNI and the switches use all 32 bits to get entropy. One step further would be for the switches to get full entropy from the inner Ethernet frame. I take it that your comment would be to make it explicit in the draft. Right?

One
   such example could be to use the upper 8 bits of the Key field to
   add flow based entropy and tag all the packets from a flow with an entropy label.

Simha

From: Vishwas Manral [mailto:[email-masked]]
Sent: Thursday, September 22, 2011 4:04 PM
To: Narasimhan Venkataramaiah
Cc: Linda Dunbar; Murari Sridharan; [email-masked]; [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Hi Simha,

A question I have is a TNI identifies a particular tenant and all flows from/to a tenant will be hashed to the same path (even with the changes in switches to do hashing to use TNI).

Why do you not use the last 8 bits which you have kept as reserved for providing the randomization for hashing flows between same to/from on different paths?

Thanks,
Vishwas
On Sun, Sep 18, 2011 at 11:01 AM, Narasimhan Venkataramaiah <[email-masked]<mailto:[email-masked]>> wrote:
The easiest from the point of view of configuration would be to route everything back through the enterprise - not necessarily the optimal from the enterprise point of view. Are you referring to a scenario where the VMs subnet is split between the cloud and the enterprise? Otherwise I don't see the implication on virtualization as its no different than getting the traffic routed to the enterprise in the first case.

Simha

________________________________________
From: [email-masked]<mailto:[email-masked]> [[email-masked]<mailto:[email-masked]>] on behalf of Linda Dunbar [[email-masked]<mailto:[email-masked]>]
Sent: Sunday, September 18, 2011 7:06 AM
To: Murari Sridharan; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Hi Murari,

One question:

Thanks, Linda

> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of
> Murari Sridharan
> To: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of
> [email-masked]<mailto:[email-masked]>
> To: [email-masked]<mailto:[email-masked]>
> > From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf
> > Of Warren Kumari
> > Cc: [email-masked]<mailto:[email-masked]>
> > > On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani
> <[email-masked]<mailto:[email-masked]>> wrote:
> > >
> > > >>>>
> > > On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral
> <[email-masked]<mailto:[email-masked]>> wrote:
> > > Hi Anoop,
> > >
> > > On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani
> <[email-masked]<mailto:[email-masked]>> wrote:
> > > Hi Vishwas,
> > >
> > > On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral
> <[email-masked]<mailto:[email-masked]>> wrote:
> > > Hi Linda,
> > >
> > > On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar <[email-masked]<mailto:[email-masked]>>
> wrote:
> > > Vishwas,
> > >
> > > On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral
> <[email-masked]<mailto:[email-masked]>> wrote:
> > > Hi Linda,
> > > On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar <[email-masked]<mailto:[email-masked]>>
> wrote:

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 16:19:28 -0700

Hi Simha,

I see this as the only difference between VXLAN and the NVGRE solution
(besides ofcourse that TNI needs to be parsed in the intermediate device for
hashing and using lesser number of bytes).

I would think you should add it to your draft immediately. With
tunneling you consolidate the addresses visible to the core and by providing
a hash mechanism, you are providing some level of randomness.

The other thing you should look at is L3 (IPv4/ IPv6) over NVGRE instead of
L2 alone. I guess it would be the same comment for the VXLAN proposal too.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 4:11 PM, Narasimhan Venkataramaiah <
[email-masked]> wrote:

>  The draft mentions exactly this as one use of the reserved 8 bits in
> Section 4. An NVGRE endpoint could use the 8 bits to further distribute
> flows belonging to a particular TNI and the switches use all 32 bits to get
> entropy. One step further would be for the switches to get full entropy from
> the inner Ethernet frame. I take it that your comment would be to make it
> explicit in the draft. Right?****
>
> ** **
>
> One****
>
>    such example could be to use the upper 8 bits of the Key field to****
>
>    add flow based entropy and tag all the packets from a flow with an
> entropy label.****
>
> ** **
>
> Simha****
>
> ** **
>
> *From:* Vishwas Manral [mailto:[email-masked]]
> *Sent:* Thursday, September 22, 2011 4:04 PM
> *To:* Narasimhan Venkataramaiah
> *Cc:* Linda Dunbar; Murari Sridharan; [email-masked]; [email-masked]
> *Subject:* Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> advertise its external facing hosts' IP addresses to external world?****
>
> ** **
>
> Hi Simha,****
>
>  ****
>
> The main (Standards Track) change in your draft is the addition of TNI.***
> *
>
>  ****
>
> switches to do hashing to use TNI).****
>
>  ****
>
> different paths?****
>  ****
> Thanks,****
> Vishwas****
> [email-masked]> wrote:****
>
> advertise its external facing hosts' IP addresses to external world?****
>
>
> Hi Murari,
>
> > <[email-masked]> wrote:
> > > >
> > > > >>>>
> > <[email-masked]> wrote:
> > > > Hi Anoop,
> > > >
> > <[email-masked]> wrote:
> > > > Hi Vishwas,
> > > >
> > <[email-masked]> wrote:
> > > > Hi Linda,
> > > >
> > <[email-masked]> wrote:
> > > > Hi Linda,
> > > >****

---

From: Murari Sridharan <[email-masked]>
Date: Thu, 22 Sep 2011 23:43:47 +0000

Vishwas, Thanks for the feedback we will definitely consider adding that. I am not sure what you mean by doing L3 instead of L2. We allow any arbitrary virtual topology including L3.

Thanks

Sent: Thursday, September 22, 2011 4:19 PM

Hi Simha,

I see this as the only difference between VXLAN and the NVGRE solution (besides ofcourse that TNI needs to be parsed in the intermediate device for hashing and using lesser number of bytes).

I would think you should add it to your draft immediately. With tunneling you consolidate the addresses visible to the core and by providing a hash mechanism, you are providing some level of randomness.

The other thing you should look at is L3 (IPv4/ IPv6) over NVGRE instead of L2 alone. I guess it would be the same comment for the VXLAN proposal too.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 4:11 PM, Narasimhan Venkataramaiah <[email-masked]<mailto:[email-masked]>> wrote:
The draft mentions exactly this as one use of the reserved 8 bits in Section 4. An NVGRE endpoint could use the 8 bits to further distribute flows belonging to a particular TNI and the switches use all 32 bits to get entropy. One step further would be for the switches to get full entropy from the inner Ethernet frame. I take it that your comment would be to make it explicit in the draft. Right?

One

Simha

From: Vishwas Manral [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Thursday, September 22, 2011 4:04 PM
To: Narasimhan Venkataramaiah
Cc: Linda Dunbar; Murari Sridharan; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 16:48:48 -0700

Murari,

What I am saying is the inner header should be allowed to be L3.

>From the diagram you have that does not seem to be the case. Am I missing it
totally?

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 4:43 PM, Murari Sridharan <[email-masked]>wrote:

>  Vishwas, Thanks for the feedback we will definitely consider adding that.
> I am not sure what you mean by doing L3 instead of L2. We allow any
> arbitrary virtual topology including L3. ****
>
> ** **
>
> Thanks****
>
> ** **
>
> *Sent:* Thursday, September 22, 2011 4:19 PM
>
> hashing and using lesser number of bytes).****
>
>  ****
>
> a hash mechanism, you are providing some level of randomness.****
>
>  ****
>

---

From: Murari Sridharan <[email-masked]>
Date: Thu, 22 Sep 2011 23:50:32 +0000

You mean not an Ethernet frame but some IP payload?

From: Vishwas Manral [mailto:[email-masked]]
Sent: Thursday, September 22, 2011 4:49 PM
To: Murari Sridharan
Cc: Narasimhan Venkataramaiah; Linda Dunbar; [email-masked]; [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Murari,

>From the diagram you have that does not seem to be the case. Am I missing it totally?

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 4:43 PM, Murari Sridharan <[email-masked]<mailto:[email-masked]>> wrote:
Vishwas, Thanks for the feedback we will definitely consider adding that. I am not sure what you mean by doing L3 instead of L2. We allow any arbitrary virtual topology including L3.

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 16:53:30 -0700

Hi Murari,

Yes that is what I mean.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 4:50 PM, Murari Sridharan <[email-masked]>wrote:

>  You mean not an Ethernet frame but some IP payload?****
>
> ** **
>
> *Sent:* Thursday, September 22, 2011 4:49 PM
> *To:* Murari Sridharan
> *Cc:* Narasimhan Venkataramaiah; Linda Dunbar; [email-masked];
> [email-masked]
>
> Murari,****
> What I am saying is the inner header should be allowed to be L3. ****
>
>  ****
>
> From the diagram you have that does not seem to be the case. Am I missing
> it totally?****
>
>  ****
>
> Thanks,****
>
> Vishwas****
>
> On Thu, Sep 22, 2011 at 4:43 PM, Murari Sridharan <[email-masked]>
> wrote:****
>
> Vishwas, Thanks for the feedback we will definitely consider adding that. I
> am not sure what you mean by doing L3 instead of L2. We allow any arbitrary
> virtual topology including L3. ****
>
>  ****
>
> Thanks****
>
>  ****
>
> *Sent:* Thursday, September 22, 2011 4:19 PM ****
>
>

---

From: Murari Sridharan <[email-masked]>
Date: Fri, 23 Sep 2011 00:00:26 +0000

Do you have a scenario in mind?

Sent from my Windows Phone
________________________________
From: Vishwas Manral
Sent: 9/22/2011 4:55 PM
To: Murari Sridharan
Cc: Narasimhan Venkataramaiah; Linda Dunbar; [email-masked]; [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Hi Murari,

Yes that is what I mean.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 4:50 PM, Murari Sridharan <[email-masked]<mailto:[email-masked]>> wrote:
You mean not an Ethernet frame but some IP payload?

From: Vishwas Manral [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Thursday, September 22, 2011 4:49 PM
To: Murari Sridharan
Cc: Narasimhan Venkataramaiah; Linda Dunbar; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

Murari,

Thanks,
Vishwas

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 19:37:45 -0700

Murari think IP mobility. :)

On Thu, Sep 22, 2011 at 5:00 PM, Murari Sridharan <[email-masked]>wrote:

> ------------------------------
> From: Vishwas Manral
> Sent: 9/22/2011 4:55 PM
>
> To: Murari Sridharan
> Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
>> The other thing you should look at is L3 (IPv4/ IPv6) over NVGRE instead
>> of L2 alone. I guess it would be the same comment for the VXLAN proposal
>> too.****
>>
>>  ****
>>
>> Thanks,****
>>
>> Vishwas****
>>
>> The main (Standards Track) change in your draft is the addition of TNI.**
>> **
>>
>>  ****
>>

---

From: Murari Sridharan <[email-masked]>
Date: Fri, 23 Sep 2011 02:41:13 +0000

In the context of the proposal here where you can carry your IP around wherever you go doesn't that already equivalent to IP mobility? If I understand you right you are simply saying don't just restrict the draft to MAC-in-GRE but do IP-in-GRE right?

From: Vishwas Manral [mailto:[email-masked]]
Sent: Thursday, September 22, 2011 7:38 PM
To: Murari Sridharan
Cc: Narasimhan Venkataramaiah; Linda Dunbar; [email-masked]; [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

On Thu, Sep 22, 2011 at 5:00 PM, Murari Sridharan <[email-masked]<mailto:[email-masked]>> wrote:
Do you have a scenario in mind?
________________________________
From: Vishwas Manral
Sent: 9/22/2011 4:55 PM

To: Murari Sridharan

Yes that is what I mean.

Thanks,
Vishwas

Murari,

Thanks,
Vishwas

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 20:07:52 -0700

Murari,

You could interpret it that way, but I see the scope increase is
considerably more than just removing the MAC header, though that would be
just one such advantage.

Thanks,
Vishwas

On Thu, Sep 22, 2011 at 7:41 PM, Murari Sridharan <[email-masked]>wrote:

>  In the context of the proposal here where you can carry your IP around
> wherever you go doesn’t that already equivalent to IP mobility? If I
> understand you right you are simply saying don’t just restrict the draft to
> MAC-in-GRE but do IP-in-GRE right?****
>
> ** **
>
> *Sent:* Thursday, September 22, 2011 7:38 PM
>
> *To:* Murari Sridharan
> Murari think IP mobility. :)****
>
>  ****
>
> On Thu, Sep 22, 2011 at 5:00 PM, Murari Sridharan <[email-masked]>
> wrote:****
>
> Do you have a scenario in mind?****
> *From: *Vishwas Manral
> *Sent: *9/22/2011 4:55 PM ****
>
>
> *To: *Murari Sridharan
> *Cc: *Narasimhan Venkataramaiah; Linda Dunbar; [email-masked];
> [email-masked]
> *Subject: *Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"
> Hi Murari,****
> Yes that is what I mean.****
>
>  ****
>
> Thanks,****
>
> Vishwas****
>
> On Thu, Sep 22, 2011 at 4:50 PM, Murari Sridharan <[email-masked]>
> wrote:****
>
> [email-masked] ****

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Fri, 23 Sep 2011 03:12:11 +0000

Do you mean in the context of network virtualization or in general its useful? Mobility is already one aspect of network virtualization and its satisfied by the MAC in GRE option.

Simha

From: Vishwas Manral [mailto:[email-masked]]
Sent: Thursday, September 22, 2011 8:08 PM
To: Murari Sridharan
Cc: Narasimhan Venkataramaiah; Linda Dunbar; [email-masked]; [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Murari,

You could interpret it that way, but I see the scope increase is considerably more than just removing the MAC header, though that would be just one such advantage.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 7:41 PM, Murari Sridharan <[email-masked]<mailto:[email-masked]>> wrote:
In the context of the proposal here where you can carry your IP around wherever you go doesn't that already equivalent to IP mobility? If I understand you right you are simply saying don't just restrict the draft to MAC-in-GRE but do IP-in-GRE right?

To: Murari Sridharan

To: Murari Sridharan

Yes that is what I mean.

Thanks,
Vishwas

Murari,

Thanks,
Vishwas

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 22 Sep 2011 20:17:59 -0700

Hi Simha,

I am talking about the difference between Layer-2 mobility and Layer-3
mobility.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 8:12 PM, Narasimhan Venkataramaiah <
[email-masked]> wrote:

>  Do you mean in the context of network virtualization or in general its
> useful? Mobility is already one aspect of network virtualization and its
> satisfied by the MAC in GRE option. ****
>
> ** **
>
> Simha****
>
> ** **
>
> *Sent:* Thursday, September 22, 2011 8:08 PM
>
> *To:* Murari Sridharan
> just one such advantage.****
>
>  ****
>
> Thanks,****
>
> Vishwas****
>
> On Thu, Sep 22, 2011 at 7:41 PM, Murari Sridharan <[email-masked]>
> wrote:****
>
> *Sent:* Thursday, September 22, 2011 7:38 PM ****
>
>
> *To:* Murari Sridharan

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Fri, 23 Sep 2011 03:34:41 +0000

Say you have a virtual subnet that spans 2 physical subnets. A VM in a virtual subnet moving to another host within the same physical subnet is L2 mobility and a VM moving to another host on another in  a different physical subnet is L3 mobility. These 2 cases are the same for NV-GRE in terms of encapsulation as the resulting packets are the same. What does IP in GRE achieve that MAC in GRE does not?

Simha

From: Vishwas Manral [mailto:[email-masked]]
Sent: Thursday, September 22, 2011 8:18 PM
To: Narasimhan Venkataramaiah
Cc: Murari Sridharan; Linda Dunbar; [email-masked]; [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Hi Simha,

I am talking about the difference between Layer-2 mobility and Layer-3 mobility.

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 8:12 PM, Narasimhan Venkataramaiah <[email-masked]<mailto:[email-masked]>> wrote:
Do you mean in the context of network virtualization or in general its useful? Mobility is already one aspect of network virtualization and its satisfied by the MAC in GRE option.

Simha

To: Murari Sridharan

Murari,

Thanks,
Vishwas

To: Murari Sridharan

To: Murari Sridharan

Yes that is what I mean.

Thanks,
Vishwas

Murari,

Thanks,
Vishwas

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Gary Berger <[email-masked]>
Date: Fri, 23 Sep 2011 09:33:40 -0400

I think these are all interesting approaches.. Whether you demultiplex a
customer on an 802.1ah I-SID, a VXLAN VID,  NVGRE TNI or some other set of
bits,  we still seem to be carrying excessive state across the network which
will continue to make things more fragile. If we continue to drive the
problem into the host-stack (because that¹s where the customer has some
choice) we will lose the benefit of building intelligent services in the
network. 

The problem space is still the same I.e. Because of the TCP pseudo-header
binding the IP address to the point of attachment we have to encapsulate in
order to hide this limiting factor). The arguments about "whatever over GRE"
are beyond the problem space IMHO but certainly a clear indication of the
flawed model. 

Some have suggested the answer is a distributed directory service (for
broadcast containment) which allows the implementor to define their own
bindings. This along with the programability of the network (e.g. OF) in
order to instantiate those bindings into the forwarding plane. Certainly
that solution can only be based on the scope of control but none-the-less, I
thought that this example was where this working group derived its charter.

-g

From:  Vishwas Manral <[email-masked]>
Date:  Thu, 22 Sep 2011 20:17:59 -0700
To:  Narasimhan Venkataramaiah <[email-masked]>
Cc:  "[email-masked]" <[email-masked]>
Subject:  Re: [armd] how does
"draft-sridharan-virtualization-nvgre-00"advertise its external facing
hosts' IP addresses to external world?

Hi Simha,

Thanks,
Vishwas
On Thu, Sep 22, 2011 at 8:12 PM, Narasimhan Venkataramaiah
<[email-masked]> wrote:
> Do you mean in the context of network virtualization or in general its useful?
> Mobility is already one aspect of network virtualization and its satisfied by
> the MAC in GRE option.
>  
> Simha
>  
> You could interpret it that way, but I see the scope increase is considerably
> more than just removing the MAC header, though that would be just one such
> advantage.
> 
>  
> 
> Thanks,
> 
> Vishwas
> 
> wherever you go doesn¹t that already equivalent to IP mobility? If I
> understand you right you are simply saying don¹t just restrict the draft to
> MAC-in-GRE but do IP-in-GRE right?
>  
> Vishwas, Thanks for the feedback we will definitely consider adding that. I am
> not sure what you mean by doing L3 instead of L2. We allow any arbitrary
> virtual topology including L3.
>  
> Thanks
>  
> I would think you should add it to your draft immediately. With tunneling you
> consolidate the addresses visible to the core and by providing a hash
> mechanism, you are providing some level of randomness.
> 
>  
> 
> The other thing you should look at is L3 (IPv4/ IPv6) over NVGRE instead of L2
> alone. I guess it would be the same comment for the VXLAN proposal too.
> 
>  
> 
> Thanks,
> 
> Vishwas
> 
> On Thu, Sep 22, 2011 at 4:11 PM, Narasimhan Venkataramaiah
> <[email-masked]> wrote:
> 
> The draft mentions exactly this as one use of the reserved 8 bits in Section
> 4. An NVGRE endpoint could use the 8 bits to further distribute flows
> belonging to a particular TNI and the switches use all 32 bits to get entropy.
> One step further would be for the switches to get full entropy from the inner
> Ethernet frame. I take it that your comment would be to make it explicit in
> the draft. Right?
>  
> One
> On Sun, Sep 18, 2011 at 11:01 AM, Narasimhan Venkataramaiah
> <[email-masked]> wrote:
> everything back through the enterprise - not necessarily the optimal from the
> enterprise point of view. Are you referring to a scenario where the VMs subnet
> is split between the cloud and the enterprise? Otherwise I don't see the
> implication on virtualization as its no different than getting the traffic
> routed to the enterprise in the first case.
> 
> Simha
> 
> Subject: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise
> its external facing hosts' IP addresses to external world?
> 
> 
> Hi Murari,
> 
> For a host within an Enterprise site which needs to communicate with external
> peers, the host either uses public IP address which is visible to external
> peers or uses private IP address which is translated to public address at the
> Enterprise site's gateway.
> 
> When this host is moved to "Cloud data center", will the "Cloud Data center"
> advertise this host address to external peers? Or will all external peers go
> through enterprise's gateway to reach this host which is no longer residing in
> the enterprise site?
> 
> Thanks, Linda
> 

_______________________________________________ armd mailing list
[email-masked] https://www.ietf.org/mailman/listinfo/armd

---

From: Warren Kumari <[email-masked]>
Date: Fri, 23 Sep 2011 15:46:48 -0400

On Sep 19, 2011, at 7:36 PM, Balus, Florin Stelian (Florin) wrote:

> 
> These two drafts and draft-wkumari-dcops-l3-vmmobility-00.txt seem to be submitted as either informational or experimental. Can the authors of these two drafts clarify what do they plan to do with them?

Honestly? My grand plan was to largely ignore the draft and then feel guilty that I had not incorporated the comments that I have received. Eventually the guilt would have caught up with me and I would have been forced to make a decision as to if I was going to bother writing more, or simply deleting it and pretending nothing had ever happened...

I believe that my co-author ((Joel), who is more level-headed than me) may be been planning on doing something more useful -- I've CCed him.

The draft wasn't really intended to push a new encapsulation technology (the desire was simply to demonstrate that big flat L2 networks are not needed for VM mobility) -- the introduction of an encapsulation was only done because I couldn't easily find an existing solution. If one is created (that satisfied the needs of the protocol), that would be much better...

W 
> If relevant feedback is required I think they seem to fit the scope of L2VPN WG. At least two of them talk about some new encapsulation used to provide L2 Multi-tenancy in the Data Centers over some sort of overlay and UDP/IP tunnels. A new L2VPN charter was released recently which includes requirements and solutions targeted at the Data Center space.
> 
> Florin
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 23 Sep 2011 15:54:25 -0400

My own perspective was that it seemed useful to put on paper the 
approach to using a mapping system at the hypervisor.
The exactly encapsulation is irrelevant to me.

I was hoping it would prompt discussion of that piece of the idea.
If it did, we then had to figure out whether there was a way to take it 
forward.
Also, if there was interest, it meant that it was worth seeing if the 
ARMD data supported or disagreed with the view that this is part of the 
problem.  At the moment, the data does not tell us.  But if no one 
cares, then it is not going to be effective for me to push folks to find 
out more.

Yours,
Joel

On 9/23/2011 3:46 PM, Warren Kumari wrote:
>>>> On Tue, Aug 9, 2011 at 2:50 PM, Anoop Ghanwani<[email-masked]>  wrote:
>>>>
>>>>>>>>
>>>> On Tue, Aug 9, 2011 at 2:41 PM, Vishwas Manral<[email-masked]>  wrote:
>>>> Hi Anoop,
>>>>
>>>> On Tue, Aug 9, 2011 at 2:03 PM, Anoop Ghanwani<[email-masked]>  wrote:
>>>> Hi Vishwas,
>>>>
>>>> On Tue, Aug 9, 2011 at 1:27 PM, Vishwas Manral<[email-masked]>  wrote:
>>>> Hi Linda,
>>>>
>>>> On Tue, Aug 9, 2011 at 11:51 AM, Linda Dunbar<[email-masked]>  wrote:
>>>> Vishwas,
>>>>
>>>> On Tue, Aug 9, 2011 at 12:58 PM, Vishwas Manral<[email-masked]>  wrote:
>>>> Hi Linda,
>>>> On Tue, Aug 9, 2011 at 10:22 AM, Linda Dunbar<[email-masked]>  wrote:
>>>> Please send feedback to us (either to the armd email list  or to the
>>>> ARMD chair Benson&  Linda) to
>>>> Linda&  Benson

---

From: Linda Dunbar <[email-masked]>
Date: Sat, 24 Sep 2011 13:35:57 +0000

Protocols have been developed for IP mobility between Home Gateway and Remote Gateway. The question is if we want similar protocols among TORs or among vSwitches. Handset mobility is random, but VM migration is planned.

Linda

From: [email-masked] [mailto:[email-masked]] On Behalf Of Vishwas Manral
Sent: Thursday, September 22, 2011 9:38 PM
To: Murari Sridharan
Cc: [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

To: Murari Sridharan

Yes that is what I mean.

Thanks,
Vishwas

Murari,

Thanks,
Vishwas

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Sat, 24 Sep 2011 17:36:51 +0000

Posting back some offline emails

From: Narasimhan Venkataramaiah
Sent: Thursday, September 22, 2011 9:49 PM
To: 'Vishwas Manral'
Cc: Murari Sridharan; Linda Dunbar; [email-masked]; Benson Schliesser
Subject: RE: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

In some cases the it may be more efficient to just send IP - but given MAC in GRE is a superset, its simpler to just always do MAC in GRE from an engineering point of view for various hardware devices that parse the packets to provide added functionality in the path. They have to deal with one type of packets. We would need MAC in GRE to stretch L2 to non NV-GRE subnets or to carry non IP payload.

Simha

From: Vishwas Manral [mailto:[email-masked]]
Sent: Thursday, September 22, 2011 9:43 PM
To: Narasimhan Venkataramaiah
Cc: Murari Sridharan; Linda Dunbar; [email-masked]; Benson Schliesser
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

I agree PMIP may not directly work here. In such cases the Layer-2 header however is still of no use.

-Vishwas
On Thu, Sep 22, 2011 at 9:07 PM, Narasimhan Venkataramaiah <[email-masked]<mailto:[email-masked]>> wrote:
Right - with NVGRE L2 mobility happens automatically - however if you were to achieve it without NVGRE in a multitenant environment you would need some VLAN reconfiguration. NVGRE achieves that with Tenant IDs.
PMIP is not multi-tenancy aware in the sense that the proxy won't be able to distinguish between multiple tenants using same IP address space in the virtual network.
Simha
From: Vishwas Manral [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Thursday, September 22, 2011 8:51 PM
To: Narasimhan Venkataramaiah
Cc: Murari Sridharan; Linda Dunbar; [email-masked]<mailto:[email-masked]>; Benson Schliesser

Hi Simha,
With that aspect there is little difference in the functionality, though L3-in-L3 can use PMIP and other mobility work already done.
Also it begs the question why you even need L2 mobility in this case
Moving off the list as Benson feels the replies are not substantial.
Thanks,
Vishwas
On Thu, Sep 22, 2011 at 8:34 PM, Narasimhan Venkataramaiah <[email-masked]<mailto:[email-masked]>> wrote:
Say you have a virtual subnet that spans 2 physical subnets. A VM in a virtual subnet moving to another host within the same physical subnet is L2 mobility and a VM moving to another host on another in  a different physical subnet is L3 mobility. These 2 cases are the same for NV-GRE in terms of encapsulation as the resulting packets are the same. What does IP in GRE achieve that MAC in GRE does not?
Simha

From: [email-masked] [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Saturday, September 24, 2011 6:36 AM
To: Vishwas Manral; Murari Sridharan
Cc: [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00" advertise its external facing hosts' IP addresses to external world?

Linda

To: Murari Sridharan

Yes that is what I mean.

Thanks,
Vishwas

Murari,

Thanks,
Vishwas

Thanks

Hi Simha,

Thanks,
Vishwas

One

Simha

Hi Simha,

Thanks,
Vishwas

Simha

Hi Murari,

One question:

Thanks, Linda

---

From: Benson Schliesser <[email-masked]>
Date: Sat, 24 Sep 2011 13:24:50 -0500

Thanks, Simha.

Leaving aside your comments about support for non-IP payloads, etc, and
focusing strictly on address resolution: Would carrying IP in the overlay
inherently localize the address resolution functions (ARP, ND)? Conversely,
does carrying L2 frames in the overlay suggest the extension or enlargement
of ARP/ND domains? Or would you propose an ARP/ND proxy in the MAC overlay?

It seems to me that scale of the address resolution function will be
affected by the way it¹s distributed (or not) throughout the overlay
network. Likewise, this may have impact on the forwarding efficiency of
traffic (i.e. when there are multiple inbound/outbound paths attached to the
overlay).

Cheers,
-Benson

On 9/24/11 12:36 PM, "Narasimhan Venkataramaiah" <[email-masked]>
wrote:

> In some cases the it may be more efficient to just send IP  but given MAC in
> GRE is a superset, its simpler to just always do MAC in GRE from an
> engineering point of view for various hardware devices that parse the packets
> to provide added functionality in the path. They have to deal with one type of
> packets. We would need MAC in GRE to stretch L2 to non NV-GRE subnets or to
> carry non IP payload.
>  
> Simha
>  
> On Thu, Sep 22, 2011 at 9:07 PM, Narasimhan Venkataramaiah
> Right  with NVGRE L2 mobility happens automatically  however if you were to
> achieve it without NVGRE in a multitenant environment you would need some VLAN
> reconfiguration. NVGRE achieves that with Tenant IDs.
> PMIP is not multi-tenancy aware in the sense that the proxy won¹t be able to
> distinguish between multiple tenants using same IP address space in the
> virtual network. 
> Simha
> With that aspect there is little difference in the functionality, though
> L3-in-L3 can use PMIP and other mobility work already done.
> On Thu, Sep 22, 2011 at 8:34 PM, Narasimhan Venkataramaiah
> Say you have a virtual subnet that spans 2 physical subnets. A VM in a virtual
> subnet moving to another host within the same physical subnet is L2 mobility
> and a VM moving to another host on another in  a different physical subnet is
> L3 mobility. These 2 cases are the same for NV-GRE in terms of encapsulation
> as the resulting packets are the same. What does IP in GRE achieve that MAC in
> GRE does not? 
> Simha
>  
>  
> 
> Protocols have been developed for IP mobility between Home Gateway and Remote
> Gateway. The question is if we want similar protocols among TORs or among
> vSwitches. Handset mobility is random, but VM migration is planned.
>  
> Linda
>  
> 
> Vishwas Manral

---

From: Narasimhan Venkataramaiah <[email-masked]>
Date: Mon, 26 Sep 2011 02:21:18 +0000

Would carrying IP in the overlay inherently localize the address resolution functions (ARP, ND)?
[Simha] Not necessarily. ND can still work over an IP based overlay. ARP can also be made to work by mapping the ARP to some IP packet that performs resolution in the overlay - somewhat like teredo that maps ND packets to UDP bubble packets. But I can't see reason to do so.

Conversely, does carrying L2 frames in the overlay suggest the extension or enlargement of ARP/ND domains? Or would you propose an ARP/ND proxy in the MAC overlay?
[Simha] No. The virtual networks are separating the configuration done on the host vs the configuration done in the network to create a certain network topology. The information that is required to create this separation can be used to make ARP/ND more efficient. One option is to proxy ARP/ND. The other option is to unicast the ARP/ND packet to a tunnel endpoint from where the ARP/ND target is reached without flooding. This is not super useful for ARP. However it is useful for ND, say, if SEND is used.

It seems to me that scale of the address resolution function will be affected by the way it's distributed (or not) throughout the overlay network. Likewise, this may have impact on the forwarding efficiency of traffic (i.e. when there are multiple inbound/outbound paths attached to the overlay).
[Simha] For ARP/ND the forwarding efficiency can be made independent of the number of paths in the overlay by using the information present at the tunnel endpoints. In other words ARP/ND packets will be as efficient to forward as any unicast packet. But i agree with your point for general broadcast that realizing broadcast domains using IP multicast might just move the forwarding efficiency issue from L2 to L3.

Simha

From: Benson Schliesser [mailto:[email-masked]]
Sent: Saturday, September 24, 2011 11:25 AM
To: Narasimhan Venkataramaiah; Linda Dunbar; Vishwas Manral; Murari Sridharan
Cc: [email-masked]
Subject: Re: [armd] how does "draft-sridharan-virtualization-nvgre-00"advertise its external facing hosts' IP addresses to external world?

Thanks, Simha.

Leaving aside your comments about support for non-IP payloads, etc, and focusing strictly on address resolution: Would carrying IP in the overlay inherently localize the address resolution functions (ARP, ND)? Conversely, does carrying L2 frames in the overlay suggest the extension or enlargement of ARP/ND domains? Or would you propose an ARP/ND proxy in the MAC overlay?

It seems to me that scale of the address resolution function will be affected by the way it's distributed (or not) throughout the overlay network. Likewise, this may have impact on the forwarding efficiency of traffic (i.e. when there are multiple inbound/outbound paths attached to the overlay).

Cheers,
-Benson

On 9/24/11 12:36 PM, "Narasimhan Venkataramaiah" <[email-masked]> wrote:
Posting back some offline emails

Simha

-Vishwas
On Thu, Sep 22, 2011 at 9:07 PM, Narasimhan Venkataramaiah <[email-masked]> wrote:

Hi Simha,
On Thu, Sep 22, 2011 at 8:34 PM, Narasimhan Venkataramaiah <[email-masked]> wrote:

Linda

On Thu, Sep 22, 2011 at 5:00 PM, Murari Sridharan <[email-masked]> wrote:

From: Vishwas Manral
Sent: 9/22/2011 4:55 PM

To: Murari Sridharan

Hi Murari,

Yes that is what I mean.

Thanks,

Vishwas

On Thu, Sep 22, 2011 at 4:50 PM, Murari Sridharan <[email-masked]> wrote:

Murari,

Thanks,

Vishwas

On Thu, Sep 22, 2011 at 4:43 PM, Murari Sridharan <[email-masked]> wrote:

Thanks

Hi Simha,

Thanks,

Vishwas

On Thu, Sep 22, 2011 at 4:11 PM, Narasimhan Venkataramaiah <[email-masked]> wrote:

One

Simha

Hi Simha,

Thanks,

Vishwas

On Sun, Sep 18, 2011 at 11:01 AM, Narasimhan Venkataramaiah <[email-masked]> wrote:
The easiest from the point of view of configuration would be to route everything back through the enterprise - not necessarily the optimal from the enterprise point of view. Are you referring to a scenario where the VMs subnet is split between the cloud and the enterprise? Otherwise I don't see the implication on virtualization as its no different than getting the traffic routed to the enterprise in the first case.

Simha

Hi Murari,

One question:

Thanks, Linda