Subject: [armd] Multi-Tenancy without changes

From: Vishwas Manral <[email-masked]>
Date: Wed, 5 Oct 2011 08:10:24 -0700

Hi,

I was thinking of multi-tenancy and the way to achieve > 4094 without any
changes in any layer by just using VLAN's. This is achieved as we already
have a mapping layer already in place.

So assume there are 8000 tenants. We can divide them in groups of 4000 each
and assign each tenant a particular id from say 2 to 4001. Now as long as we
make sure we do not have tenants from 2 different groups on the same
machine, there are just no issues. Only the mapping layer needs to be aware
of the group a machine belongs to. There is thus no change in dataplane.

Any comments?

Thanks,
Vishwas

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 05 Oct 2011 19:06:09 +0000

Vishwas,

Are you assuming that each group of tenants are connected to Gateway router via completely different physical ports? If yes, then you are absolutely correct.

Linda

From: [email-masked] [mailto:[email-masked]] On Behalf Of Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: [email-masked]
Subject: [armd] Multi-Tenancy without changes

Hi,

I was thinking of multi-tenancy and the way to achieve > 4094 without any changes in any layer by just using VLAN's. This is achieved as we already have a mapping layer already in place.

So assume there are 8000 tenants. We can divide them in groups of 4000 each and assign each tenant a particular id from say 2 to 4001. Now as long as we make sure we do not have tenants from 2 different groups on the same machine, there are just no issues. Only the mapping layer needs to be aware of the group a machine belongs to. There is thus no change in dataplane.

Any comments?

Thanks,
Vishwas

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 5 Oct 2011 13:14:41 -0700

Linda,

I am unsure what you mean by Gateway router, but there is always a
Hypervisor or the first hop router that does the encapsulation/
decapsulation.

Thanks,
Vishwas
On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <[email-masked]>wrote:

>  Vishwas, ****
>
> ** **
>
> Are you assuming that each group of tenants are connected to Gateway router
> via completely different physical ports? If yes, then you are absolutely
> correct. ****
>
> ** **
>
> Linda****
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of
> *Vishwas Manral
> *Sent:* Wednesday, October 05, 2011 10:10 AM
> *To:* [email-masked]
> *Subject:* [armd] Multi-Tenancy without changes****
>
> ** **
>
> Hi,****
>
>  ****
>
> have a mapping layer already in place.****
>
>  ****
>
> of the group a machine belongs to. There is thus no change in dataplane.**
> **
>
>  ****
>
> Any comments?****
>
>  ****
>
> Thanks,****
>
> Vishwas****
>

---

From: Linda Dunbar <[email-masked]>
Date: Wed, 05 Oct 2011 20:25:01 +0000

I mean "the first hop router". Each physical port on "the first hop router" can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE's KEY field, or VxLan's Client ID field).

You can make VLAN locally significant at the overlay edge node if the Overlay Encapsulation header has a field to further differentiate the Clients.

Linda

From: Vishwas Manral [mailto:[email-masked]]
Sent: Wednesday, October 05, 2011 3:15 PM
To: Linda Dunbar
Cc: [email-masked]
Subject: Re: [armd] Multi-Tenancy without changes

Linda,

I am unsure what you mean by Gateway router, but there is always a Hypervisor or the first hop router that does the encapsulation/ decapsulation.

Thanks,
Vishwas
On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <[email-masked]<mailto:[email-masked]>> wrote:
Vishwas,

Linda

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Vishwas Manral
Sent: Wednesday, October 05, 2011 10:10 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [armd] Multi-Tenancy without changes

Hi,

Any comments?

Thanks,
Vishwas

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 5 Oct 2011 13:35:29 -0700

Linda,

Thats is a very complicated way of thinking things, where you have multiple
layers.

A simpler solution is to just assume each hypervisor belongs to a particular
group. So it will communicate with only members of the same group.  The
mapping layer/ IP has a unique IP address (not group specific).

You have to understand in the cloud you have an orchesrtator so there is
absolute flexibility in this approach, though there is some additional load
on the orchestrator to know groups + VLAN's to identify a tenant and
ofcourse managing the mappings to best use the physical infrastructure.

Thanks,
Vishwas
On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <[email-masked]>wrote:

>  I mean “the first hop router”. Each physical port on “the first hop
> router” can have its own 4095 VLANs. Each Hypervisor can also have its own
> 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header
> to differentiate clients (e.g. GRE’s KEY field, or VxLan’s Client ID field).
> ****
>
> ** **
>
> You can make VLAN locally significant at the overlay edge node if the
> Overlay Encapsulation header has a field to further differentiate the
> Clients. ****
>
> ** **
>
> Linda****
>
> ** **
>
> *From:* Vishwas Manral [mailto:[email-masked]]
> *Sent:* Wednesday, October 05, 2011 3:15 PM
> *To:* Linda Dunbar
> *Cc:* [email-masked]
> *Subject:* Re: [armd] Multi-Tenancy without changes****
>
> ** **
>
> Linda,****
>
>  ****
>
> decapsulation.****
> On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <[email-masked]>
> wrote:****
>
> Vishwas, ****
>
>  ****
>

---

From: Benson Schliesser <[email-masked]>
Date: Wed, 5 Oct 2011 16:15:18 -0500

Hi, Vishwas -

Yes, what you describe is possible.  But the issue is that it only works if we "do not have tenants from 2 different groups on the same machine".  Keep in mind that this applies to all "machines" in the network.  In addition to the hypervisor, all of the switch paths and router interfaces must also be segregated.

The only way this works is if we partition the hardware into non-overlapping topologies, such that different tenants with a conflicting VLAN ID aren't bridged together.  And, as Linda pointed out, each partition must have a different physical interface to the L3 gateway. (E.g. by having different routers, or discrete interfaces on a shared router)

Effectively, we do this today when we build multiple datacenters (including when we build multiple "logical" datacenters inside the same physical building).  We could pursue a more complicated approach where an arbitrary number of topologies are overlaid on a common physical network, constrained by the number of physical paths etc, but that would be an operational nightmare.

Cheers,
-Benson

On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote:

> Linda,
>  
> Thats is a very complicated way of thinking things, where you have multiple layers.
>  
> A simpler solution is to just assume each hypervisor belongs to a particular group. So it will communicate with only members of the same group.  The mapping layer/ IP has a unique IP address (not group specific).
>  
> You have to understand in the cloud you have an orchesrtator so there is absolute flexibility in this approach, though there is some additional load on the orchestrator to know groups + VLAN's to identify a tenant and ofcourse managing the mappings to best use the physical infrastructure.
>  
> Thanks,
> Vishwas
> On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <[email-masked]> wrote:
> I mean “the first hop router”. Each physical port on “the first hop router” can have its own 4095 VLANs. Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the IP encapsulation with a key in the header to differentiate clients (e.g. GRE’s KEY field, or VxLan’s Client ID field).
> 
>  
> 
> On Wed, Oct 5, 2011 at 12:06 PM, Linda Dunbar <[email-masked]> wrote:
> 
> Vishwas,
> 
>  
> 
> _______________________________________________
> armd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/armd

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 5 Oct 2011 14:21:06 -0700

Hi Benson,

You got the idea perfectly right. Here is the catch however.
> The only way this works is if we partition the hardware into
non-overlapping
> topologies, such that different tenants with a conflicting VLAN ID aren't
bridged
> together.
There is no hard seperation required, because mapping can be done through a
control plane. As the mapping is all logical, the orchestration layer can do
intelligent things here and change group bindings on a logical basis. So if
I know group 1 has VLAN 1, 2 and 3 and Group 2 has 4, 5, 6 we can still use
the same group and same hardware, till there is an overlap.

Thanks,
Vishwas
On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser <[email-masked]>wrote:

> Hi, Vishwas -
>
> Yes, what you describe is possible.  But the issue is that it only works if
> we "do not have tenants from 2 different groups on the same machine".  Keep
> in mind that this applies to all "machines" in the network.  In addition to
> the hypervisor, all of the switch paths and router interfaces must also be
> segregated.
>
> The only way this works is if we partition the hardware into
> non-overlapping topologies, such that different tenants with a conflicting
> VLAN ID aren't bridged together.  And, as Linda pointed out, each partition
> must have a different physical interface to the L3 gateway. (E.g. by having
> different routers, or discrete interfaces on a shared router)
>
> Effectively, we do this today when we build multiple datacenters (including
> when we build multiple "logical" datacenters inside the same physical
> building).  We could pursue a more complicated approach where an arbitrary
> number of topologies are overlaid on a common physical network, constrained
> by the number of physical paths etc, but that would be an operational
> nightmare.
>
> Cheers,
> -Benson
>
>
> A simpler solution is to just assume each hypervisor belongs to a
> particular group. So it will communicate with only members of the same
> group.  The mapping layer/ IP has a unique IP address (not group specific).
>
>> Are you assuming that each group of tenants are connected to Gateway
>> router via completely different physical ports? If yes, then you are
>> absolutely correct. ****
>>
>>  ****
>>
>> Linda****
>>
>>  ****
>>
>> *From:* [email-masked] [mailto:[email-masked]] *On Behalf
>> Of *Vishwas Manral
>> So assume there are 8000 tenants. We can divide them in groups of 4000
>> each and assign each tenant a particular id from say 2 to 4001. Now as long
>> as we make sure we do not have tenants from 2 different groups on the same
>> of the group a machine belongs to. There is thus no change in dataplane.*
>> ***
>>
>>  ****
>>
>> Any comments?****
>>
>>  ****
>>
>> Thanks,****
>>
>> Vishwas****
>>
>> ** **
>>
>

---

From: Benson Schliesser <[email-masked]>
Date: Wed, 5 Oct 2011 17:31:40 -0500

Hi, Vishwas.

On Oct 5, 2011, at 4:21 PM, Vishwas Manral wrote:

> > The only way this works is if we partition the hardware into non-overlapping
> > topologies, such that different tenants with a conflicting VLAN ID aren't bridged
> > together.
> There is no hard seperation required, because mapping can be done through a control plane. As the mapping is all logical, the orchestration layer can do intelligent things here and change group bindings on a logical basis. So if I know group 1 has VLAN 1, 2 and 3 and Group 2 has 4, 5, 6 we can still use the same group and same hardware, till there is an overlap.

Your proposal was a design "just using VLAN's" so that there is "no change in dataplane". Thus I'm not sure what a "group" means, in the context of your response, unless it means that the control plane enforces some kind of hard separation.

Note that I referred to separation of "switch paths" in my previous message. In today's average datacenter, this requires hard separation of switches (or partitions within the switch). But there are alternative switch designs; let me try to imagine what you're describing.  I suppose one could build a switch that completely localizes VLAN ID on each interface, has a very large internal namespace for tenants, and rewrites VLAN ID as frames are forwarded from ingress to egress interface. (Sounds a bit like a Frame Relay switch...)  But even with this architecture, we're limited to 4k VLANs on a given path - that limitation is built into the frame format.

Bringing this back to the ARMD scope:  As I wrote last week, it doesn't really matter how we segregate the tenants.  As long as the mechanism provides a L2 service, the underlying method of segregation is orthogonal to ARP/ND address resolution.  However, the fact that we segregate tenants might allow us to distribute the address resolution function more intelligently.  In your approach, do you imagine distributing L3 gateways and/or ARP proxy functions throughout each topology / partition?

Cheers,
-Benson

> On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser <[email-masked]> wrote:
> Hi, Vishwas -
>

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 5 Oct 2011 17:04:24 -0700

Hi Benson,

> Your proposal was a design "just using VLAN's" so that there is
> "no change in dataplane". Thus I'm not sure what a "group" means,
> in the context of your response, unless it means that the control
> plane enforces some kind of hard separation.

Yes the concept of group is only in the control plane.A device at one point
of time belongs to one group. Yes there is a limitation of 4k instances on a
particular server (however that sounds like a reasonable limitation) to me.
No physical devices need to be dedicated for a aprticular group, as with
virtualization different tenats can occupy a host machine and can be shifted
around.

> Note that I referred to separation of "switch paths" in my previous
> message. In today's average datacenter, this requires hard separation
> of switches (or partitions within the switch). But there are alternative
> switch designs; let me try to imagine what you're describing.  I suppose
> one could build a switch that completely localizes VLAN ID on each
> interface, has a very large internal namespace for tenants, and rewrites
> VLAN ID as frames are forwarded from ingress to egress interface.
> (Sounds a bit like a Frame Relay switch...)  But even with this
> architecture, we're limited to 4k VLANs on a given path - that limitation
> is built into the frame format.
>
> Bringing this back to the ARMD scope:  As I wrote last week, it doesn't
> really matter how we segregate the tenants.  As long as the
> mechanism provides a L2 service, the underlying method of segregation
>  is orthogonal to ARP/ND address resolution.  However, the fact that
> we segregate tenants might allow us to distribute the address
> resolution function more intelligently.  In your approach, do you
> imagine distributing L3 gateways and/or ARP proxy functions
> throughout each topology / partition?
Yes you are right with this design change in ARMD, in my view there is not
much difference as such other than the fact that we can divide the ARP
tables more intelligently and only to members of the group that belong to
the group so a lot easier for sure.

Thanks,
Vishwas

>> Yes, what you describe is possible.  But the issue is that it only works
>> if we "do not have tenants from 2 different groups on the same machine".
>>  Keep in mind that this applies to all "machines" in the network.  In
>> addition to the hypervisor, all of the switch paths and router interfaces
>> must also be segregated.
>>
>> Effectively, we do this today when we build multiple datacenters
>> (including when we build multiple "logical" datacenters inside the same
>> physical building).  We could pursue a more complicated approach where an
>> arbitrary number of topologies are overlaid on a common physical network,
>> constrained by the number of physical paths etc, but that would be an
>> operational nightmare.
>>
>> Cheers,
>> -Benson
>>
>>

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Thu, 6 Oct 2011 09:29:16 +0530

Assumptions about how we can use VLANs are broken down the moment you
think of Hybrid Clouds - i.e. private VLAN stretching out into the
public cloud. In effect, every customer has the full range of 4K VLANs
inside the enterprise and when they span the VLAN into the public cloud,
by implication the entire range is (potentially) spanned into the cloud
as well. There are two ways to deal with this - (a) do VLAN translation
at some boundary or (b) create segmentation orthogonal to VLAN. I prefer
option (b). Option (a) will also work, but will not scale.

Another reason why VLAN as segmentation is not a scalable option because
all inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have
L2 network, with some kind of ECMP, then you don't get ECMP for L3
traffic. That's a huge problem, IMO, especially if you have a VLAN
spanning multiple subnets and the host thinks it is outside the subnet
and the network thinks it is inside the VLAN.

You can design around this and pin the L3 GW at multiple points in the
network - and now you have a different problem - all the L3->L2 bindings
are stored at all the L3 GWs. If you had a million VMs each host talking
to even one host outside the VLAN, then that traffic goes through the
GW. Now, when you apply ECMP, the problem is quickly spread to all L3
GWs. 

In summary, using VLAN for segmentation is just inviting too many
problems, that I would like to side step.

Ashish

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Vishwas Manral
Sent: Thursday, October 06, 2011 5:34 AM
To: Benson Schliesser (bschlies)
Cc: [email-masked]
Subject: Re: [armd] Multi-Tenancy without changes

Hi Benson,

> plane enforces some kind of hard separation.

Yes the concept of group is only in the control plane.A device at one
point of time belongs to one group. Yes there is a limitation of 4k
instances on a particular server (however that sounds like a reasonable
limitation) to me. No physical devices need to be dedicated for a
aprticular group, as with virtualization different tenats can occupy a
host machine and can be shifted around.

> throughout each topology / partition?

Yes you are right with this design change in ARMD, in my view there is
not much difference as such other than the fact that we can divide the
ARP tables more intelligently and only to members of the group that
belong to the group so a lot easier for sure.

Thanks,

Vishwas 

		On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser
<[email-masked]> wrote:

			Hi, Vishwas - 

			Yes, what you describe is possible.  But the
issue is that it only works if we "do not have tenants from 2 different
groups on the same machine".  Keep in mind that this applies to all
"machines" in the network.  In addition to the hypervisor, all of the
switch paths and router interfaces must also be segregated.

			The only way this works is if we partition the
hardware into non-overlapping topologies, such that different tenants
with a conflicting VLAN ID aren't bridged together.  And, as Linda
pointed out, each partition must have a different physical interface to
the L3 gateway. (E.g. by having different routers, or discrete
interfaces on a shared router)

			Effectively, we do this today when we build
multiple datacenters (including when we build multiple "logical"
datacenters inside the same physical building).  We could pursue a more
complicated approach where an arbitrary number of topologies are
overlaid on a common physical network, constrained by the number of
physical paths etc, but that would be an operational nightmare.

			Cheers,

			-Benson

			On Oct 5, 2011, at 3:35 PM, Vishwas Manral
wrote:

				Linda, 

				Thats is a very complicated way of
thinking things, where you have multiple layers. 

				A simpler solution is to just assume
each hypervisor belongs to a particular group. So it will communicate
with only members of the same group.  The mapping layer/ IP has a unique
IP address (not group specific). 

				You have to understand in the cloud you
have an orchesrtator so there is absolute flexibility in this approach,
though there is some additional load on the orchestrator to know groups
+ VLAN's to identify a tenant and ofcourse managing the mappings to best
use the physical infrastructure.

				Thanks,

				Vishwas

				On Wed, Oct 5, 2011 at 1:25 PM, Linda
Dunbar <[email-masked]> wrote:

				I mean "the first hop router". Each
physical port on "the first hop router" can have its own 4095 VLANs.
Each Hypervisor can also have its own 4095 VLANs if Hypervisor does the
IP encapsulation with a key in the header to differentiate clients (e.g.
GRE's KEY field, or VxLan's Client ID field). 

				You can make VLAN locally significant at
the overlay edge node if the Overlay Encapsulation header has a field to
further differentiate the Clients. 

				Linda

				From: Vishwas Manral
[mailto:[email-masked]] 
				Sent: Wednesday, October 05, 2011 3:15
PM
				To: Linda Dunbar
				Cc: [email-masked]
				Subject: Re: [armd] Multi-Tenancy
without changes

				Linda,

				I am unsure what you mean by Gateway
router, but there is always a Hypervisor or the first hop router that
does the encapsulation/ decapsulation.

				Thanks,

				Vishwas

				On Wed, Oct 5, 2011 at 12:06 PM, Linda
Dunbar <[email-masked]> wrote:

				Vishwas, 

				Are you assuming that each group of
tenants are connected to Gateway router via completely different
physical ports? If yes, then you are absolutely correct. 

				Linda

				From: [email-masked]
[mailto:[email-masked]] On Behalf Of Vishwas Manral
				Sent: Wednesday, October 05, 2011 10:10
AM
				To: [email-masked]
				Subject: [armd] Multi-Tenancy without
changes

				Hi,

				I was thinking of multi-tenancy and the
way to achieve > 4094 without any changes in any layer by just using
VLAN's. This is achieved as we already have a mapping layer already in
place.

				So assume there are 8000 tenants. We can
divide them in groups of 4000 each and assign each tenant a particular
id from say 2 to 4001. Now as long as we make sure we do not have
tenants from 2 different groups on the same machine, there are just no
issues. Only the mapping layer needs to be aware of the group a machine
belongs to. There is thus no change in dataplane.

				Any comments?

				Thanks,

				Vishwas

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 6 Oct 2011 12:42:01 -0400

Vishwas:

As always, great comments. 

Thomas Narten is collecting pain points and issues. 

Can you give us details why option (a) VLAN translation at some boundary
will not scale? 

It would help Thomas' pain points. 

Sue

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Ashish Dalela (adalela)
Sent: Wednesday, October 05, 2011 11:59 PM
To: Vishwas Manral; Benson Schliesser (bschlies)
Cc: [email-masked]
Subject: Re: [armd] Multi-Tenancy without changes

Assumptions about how we can use VLANs are broken down the moment you think
of Hybrid Clouds - i.e. private VLAN stretching out into the public cloud.
In effect, every customer has the full range of 4K VLANs inside the
enterprise and when they span the VLAN into the public cloud, by implication
the entire range is (potentially) spanned into the cloud as well. There are
two ways to deal with this - (a) do VLAN translation at some boundary or (b)
create segmentation orthogonal to VLAN. I prefer option (b). Option (a) will
also work, but will not scale.

Another reason why VLAN as segmentation is not a scalable option because all
inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L2
network, with some kind of ECMP, then you don't get ECMP for L3 traffic.
That's a huge problem, IMO, especially if you have a VLAN spanning multiple
subnets and the host thinks it is outside the subnet and the network thinks
it is inside the VLAN.

You can design around this and pin the L3 GW at multiple points in the
network - and now you have a different problem - all the L3->L2 bindings are
stored at all the L3 GWs. If you had a million VMs each host talking to even
one host outside the VLAN, then that traffic goes through the GW. Now, when
you apply ECMP, the problem is quickly spread to all L3 GWs. 

In summary, using VLAN for segmentation is just inviting too many problems,
that I would like to side step.

Ashish

Hi Benson,

Thanks,

Vishwas 

On Wed, Oct 5, 2011 at 2:15 PM, Benson Schliesser <[email-masked]>
wrote:

Hi, Vishwas - 

The only way this works is if we partition the hardware into non-overlapping
topologies, such that different tenants with a conflicting VLAN ID aren't
bridged together.  And, as Linda pointed out, each partition must have a
different physical interface to the L3 gateway. (E.g. by having different
routers, or discrete interfaces on a shared router)

Cheers,

-Benson

Linda, 

Thanks,

Vishwas

On Wed, Oct 5, 2011 at 1:25 PM, Linda Dunbar <[email-masked]>
wrote:

I mean "the first hop router". Each physical port on "the first hop router"
can have its own 4095 VLANs. Each Hypervisor can also have its own 4095
VLANs if Hypervisor does the IP encapsulation with a key in the header to
differentiate clients (e.g. GRE's KEY field, or VxLan's Client ID field). 

Clients. 

Linda

Linda,

Thanks,

Vishwas

Vishwas, 

correct. 

Linda

Hi,

Any comments?

Thanks,

Vishwas

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 6 Oct 2011 21:05:35 -0700

Hi Sue,

Sure I can let you know the details. Like I mentioned and as was
details very well by Ashish, the main issues are:

1. Behind a particular encapsulator/ decapsulator there can be at most 4096
VLAN's and hence 4096 tenants.
2. Also with using the mechanism ther orchestrator has to have a lot of
additional intelligence and will need to optimize the resources of a data
center as the number of tenats grow.

Thanks,
Vishwas
On Thu, Oct 6, 2011 at 9:42 AM, Susan Hares <[email-masked]> wrote:

>  Vishwas:****
>
> ** **
>
> As always, great comments. ****
>
> ** **
>
> Thomas Narten is collecting pain points and issues. ****
>
> will not scale? ****
> It would help Thomas’ pain points. ****
>
> ** **
>
> Sue****
>
> ** **
>
> *Ashish Dalela (adalela)
> *Sent:* Wednesday, October 05, 2011 11:59 PM
> *To:* Vishwas Manral; Benson Schliesser (bschlies)
>
> *Cc:* [email-masked]
> of Hybrid Clouds – i.e. private VLAN stretching out into the public cloud.
> two ways to deal with this – (a) do VLAN translation at some boundary or (b)
> also work, but will not scale.****
>
> ** **
>
> all inter-VLAN traffic gets pinned to some L3 gateway. Now, if you have L2
> network, with some kind of ECMP, then you don’t get ECMP for L3 traffic.
> That’s a huge problem, IMO, especially if you have a VLAN spanning multiple
> it is inside the VLAN.****
>
> ** **
>
> network – and now you have a different problem – all the L3->L2 bindings are
> you apply ECMP, the problem is quickly spread to all L3 GWs. ****
>
> ** **
>
> that I would like to side step.****
>
> ** **
>
> Ashish****
>
> ** **
>
> *Sent:* Thursday, October 06, 2011 5:34 AM
> *To:* Benson Schliesser (bschlies)
> *Cc:* [email-masked]
> Hi Benson,****
> > Your proposal was a design "just using VLAN's" so that there is ****
>
> > "no change in dataplane". Thus I'm not sure what a "group" means,****
>
> > in the context of your response, unless it means that the control ****
>
> > plane enforces some kind of hard separation.****
>
>  ****
>
> around.****
> > Note that I referred to separation of "switch paths" in my previous ****
>
> > message. In today's average datacenter, this requires hard separation***
> *
>
> > one could build a switch that completely localizes VLAN ID on each ****
>
> > VLAN ID as frames are forwarded from ingress to egress interface. ****
>
> > (Sounds a bit like a Frame Relay switch...)  But even with this ****
>
> > is built into the frame format.****
>
> >** **
>
> > really matter how we segregate the tenants.  As long as the ****
>
> > mechanism provides a L2 service, the underlying method of segregation***
> *
>
> >  is orthogonal to ARP/ND address resolution.  However, the fact that ***
> *
>
> > we segregate tenants might allow us to distribute the address ****
>
> > resolution function more intelligently.  In your approach, do you ****
>
> > imagine distributing L3 gateways and/or ARP proxy functions ****
>
> > throughout each topology / partition?****
>
> the group so a lot easier for sure.****
>
>  ****
>
> Thanks,****
>
> Vishwas ****
>
>
>  ****
>
>  Hi, Vishwas - ****
> segregated.****
> different routers, or discrete interfaces on a shared router)****
>
> ** **
>
> nightmare.****
> Cheers,****
> -Benson****
> On Oct 5, 2011, at 3:35 PM, Vishwas Manral wrote:****
>
> ** **
>
>   Linda, ****
>
>  ****
>
> layers. ****
> ofcourse managing the mappings to best use the physical infrastructure.***
> *
>
>  ****
>
> Thanks,****
>
> Vishwas****
>
> I mean “the first hop router”. Each physical port on “the first hop router”
> differentiate clients (e.g. GRE’s KEY field, or VxLan’s Client ID field).
> ****
>
>  ****
>
> https://www.ietf.org/mailman/listinfo/armd****
>
> ** **
>
> ** **
>
> ** **
>
> ** **
>

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 7 Oct 2011 10:36:15 +0530

Specifically on VLAN translation, if you have only 2 sites spanning
VLANs with different id's then you need a table to statically map one
VLAN to another. This doesn't seem too bad if you only use 4096 VLANs.
But, if you have more than 2 sites, spanning the same VLAN as different
VLAN id's then you need to have per MAC entry mapping it to a new VLAN.
Total entries equal total MAC. Obviously, we will design for multiple
sites.

If your DCI already required you to have per host entry at the edge,
then you are not any worse off by adding VLAN translation to it. But, if
not, VLAN translation forces you to have a per-host entry. Either way,
it's not a scalable solution. 

Ashish

From: Vishwas Manral [mailto:[email-masked]] 
Sent: Friday, October 07, 2011 9:36 AM
To: Susan Hares
Cc: Ashish Dalela (adalela); Benson Schliesser (bschlies);
[email-masked]; Thomas Narten
Subject: Re: [armd] Multi-Tenancy without changes

Hi Sue,

Sure I can let you know the details. Like I mentioned and as was details
very well by Ashish, the main issues are:

1. Behind a particular encapsulator/ decapsulator there can be at most
4096 VLAN's and hence 4096 tenants.

2. Also with using the mechanism ther orchestrator has to have a lot of
additional intelligence and will need to optimize the resources of a
data center as the number of tenats grow.

Thanks,

Vishwas

Vishwas:

Sue

Cc: [email-masked]

Ashish

Hi Benson,

Thanks,

Vishwas 

			Hi, Vishwas - 

			Cheers,

			-Benson

				Linda, 

				Thanks,

				Vishwas

				Linda

				From: Vishwas Manral

				Linda,

				Thanks,

				Vishwas

				Vishwas, 

				Linda

				Hi,

				Any comments?

				Thanks,

				Vishwas

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 14 Oct 2011 18:38:24 -0700

Hi Sue,

Sorry I ave been away on some work. If we use MAC-in-MAC the I-SID looks
very similar to the TNI or other multi-tenancy service indicators.

I know multi-tenancy support for TRILL plans to extend to use Q-in-Q and
other such mechanisms. BTW SPB-V uses Q-in-Q too though I do not know if it
is deployed at all.

Thanks,
Vishwas
On Fri, Oct 7, 2011 at 10:59 AM, <[email-masked]> wrote:

> Vishwas:
>
> Thanks.  That clears up a lot.
>
> I'm not a Q-inQ expert so this question maybe off base....
>
> How does Q-in-Q support impact the vlan approac:?  Is there any DC use. Of
> Q-in-Q Vlans?
>
> Sue
>
>
> Sent via BlackBerry by AT&T
>
> -----Original Message-----
> From: Vishwas Manral <[email-masked]>
> Sender: [email-masked]
> Date: Thu, 6 Oct 2011 21:05:35
> To: Susan Hares<[email-masked]>
> Cc: Thomas Narten<[email-masked]>; <[email-masked]>

---

From: "Susan Hares" <[email-masked]>
Date: Sat, 15 Oct 2011 08:21:22 -0400

Vishwas:

Your understanding matches mine on the mac-in-mac  and I-SID, Trills Q-in-Q
planned support, and SPB-V Q-in-Q support.

I think a common ARP solution would be useful for all of these.  

Sue 

From: Vishwas Manral [mailto:[email-masked]] 
Sent: Friday, October 14, 2011 9:38 PM
To: [email-masked]
Cc: Susan Hares; Thomas Narten; [email-masked]
Subject: Re: [armd] Multi-Tenancy without changes

Hi Sue,

Thanks,

Vishwas

Vishwas:

Sue

-----Original Message-----
From: Vishwas Manral <[email-masked]>
Sender: [email-masked]
Date: Thu, 6 Oct 2011 21:05:35
To: Susan Hares<[email-masked]>
Cc: Thomas Narten<[email-masked]>; <[email-masked]>
Subject: Re: [armd] Multi-Tenancy without changes