Subject: [nvo3] New dataplane requirements draft

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Thu, 17 May 2012 10:31:32 +0200

We have just posted a new draft on nvo3 data plane requirements:

http://www.ietf.org/id/draft-bitar-lasserre-nvo3-dp-reqs-00.txt

We welcome your feedback. Our intent is to submit a revised version by Vancouver addressing your comments and/or suggestions.

Thanks,
Marc

---

From: "Adrian Farrel" <[email-masked]>
Date: Thu, 17 May 2012 21:50:50 +0100

Guys,
 
Would you mind reposting this with a name conforming to section 7 of
https://www.ietf.org/id-info/guidelines.html
 
To your benefit, this will mean that the I-D shows up at
https://datatracker.ietf.org/wg/nvo3/
 
To my benefit, this will mean I can find and read your draft in the future. 
 
Thanks,
Adrian
 
PS. When you do this, you might send the secretariat a note saying please mark
draft-foo as replaced by draft-bar.
 
From: [email-masked] [mailto:[email-masked]] On Behalf Of
LASSERRE, MARC (MARC)
Sent: 17 May 2012 09:32
To: [email-masked]
Subject: [nvo3] New dataplane requirements draft

 <http://www.ietf.org/id/draft-bitar-lasserre-nvo3-dp-reqs-00.txt>
http://www.ietf.org/id/draft-bitar-lasserre-nvo3-dp-reqs-00.txt
 
We welcome your feedback. Our intent is to submit a revised version by Vancouver
addressing your comments and/or suggestions.
 
Thanks,
Marc

---

From: AshwoodsmithPeter <[email-masked]>
Date: Fri, 18 May 2012 14:30:17 +0000

Concerning the 'SHOULD' for (*,G) in the underlay:

What percentage of traffic in a DC (cloud or enterprise etc.) is actually non BUM/ARP multicast between VMs?
Who has actual data that does not include BUM/ARP (which has to be solved with distributed control for these scales)?

We heard from some big DC/Cloud folks that they explicitly disable multicast by applications ... so clearly for them its 0.

If it is in general a very very low percentage, then should we not drop (*,G) as a requirement of the underlay?

This work could set the tone for DC/Cloud based applications for years to come and perhaps the applications would be better served by a reliable distributed control mechanism.

Peter

P.S. As a networking person I love the challenges of (*,G) (S,G) trees so I'm biased but objectively I'd like to understand exactly where the requirement is coming from to see if it is not better addressed by distributed control.

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of LASSERRE, MARC (MARC)
Sent: Thursday, May 17, 2012 4:32 AM
To: [email-masked]
Subject: [nvo3] New dataplane requirements draft

Thanks,
Marc

---

From: Dino Farinacci <[email-masked]>
Date: Fri, 18 May 2012 09:35:08 -0700

Someone I respect very much told me this week "if we deploy VxLAN, then my ASM network will need to be converted to Bidir to reduce the amount of (S,G) state the VxLAN control plane is going to create ... per subnet ... sigh ...".

So (*,G) will live on but in Bidir-PIM form.

Dino

On May 18, 2012, at 7:30 AM, AshwoodsmithPeter wrote:

> Concerning the ‘SHOULD’ for (*,G) in the underlay:
>  
> P.S. As a networking person I love the challenges of (*,G) (S,G) trees so I’m biased but objectively I’d like to understand exactly where the requirement is coming from to see if it is not better addressed by distributed control.
>  
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: "Balus, Florin Stelian (Florin)" <[email-masked]>
Date: Mon, 21 May 2012 14:08:52 -0500

Adrian,

We addressed the naming issues and re-submitted the NVO3 data plane requirements draft with a new and compliant name - see at http://www.ietf.org/id/draft-bl-nvo3-dataplane-requirements-00.txt . It shows right now under NVO3 WG URL so it should be easier to track. Will forward separately a request to IETF secretariat per your request.

Florin

From: [email-masked] [mailto:[email-masked]] On Behalf Of Adrian Farrel
Sent: Thursday, May 17, 2012 1:51 PM
To: LASSERRE, MARC (MARC); [email-masked]
Subject: Re: [nvo3] New dataplane requirements draft

Guys,

Would you mind reposting this with a name conforming to section 7 of https://www.ietf.org/id-info/guidelines.html

To your benefit, this will mean that the I-D shows up at https://datatracker.ietf.org/wg/nvo3/

Thanks,
Adrian

PS. When you do this, you might send the secretariat a note saying please mark draft-foo as replaced by draft-bar.

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of LASSERRE, MARC (MARC)
Sent: 17 May 2012 09:32
To: [email-masked]<mailto:[email-masked]>
Subject: [nvo3] New dataplane requirements draft

Thanks,
Marc

---

From: "Adrian Farrel" <[email-masked]>
Date: Mon, 21 May 2012 20:41:15 +0100

Yeah, saw that.
Thanks for humouring me.
Adrian
 
From: Balus, Florin Stelian (Florin) [mailto:[email-masked]] 
Sent: 21 May 2012 20:09
To: [email-masked]; [email-masked]
Cc: LASSERRE, MARC (MARC); Bitar, Nabil N
Subject: Resubmitted with new and compliant name RE: [nvo3] New dataplane
requirements draft
 
Adrian,
 
We addressed the naming issues and re-submitted the NVO3 data plane requirements
draft with a new and compliant name - see at
http://www.ietf.org/id/draft-bl-nvo3-dataplane-requirements-00.txt . It shows
right now under NVO3 WG URL so it should be easier to track. Will forward
separately a request to IETF secretariat per your request.
 
Florin
 
From: [email-masked] [mailto:[email-masked]] On Behalf Of Adrian
Farrel
Sent: Thursday, May 17, 2012 1:51 PM
To: LASSERRE, MARC (MARC); [email-masked]
Subject: Re: [nvo3] New dataplane requirements draft
 
Guys,

 
 
Thanks,
Adrian

 

 
Thanks,
Marc

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Mon, 21 May 2012 20:46:22 +0000

Hi Marc and Nabil,

In section 3.2.1, your draft states:

"It MUST also be possible to select whether the NVE provides optimized multicast trees inside the VNI for individual tenant multicast groups or whether the default VNI flooding tree is used."

Am I correctly interpreting this statement to mean that overlay solution MUST support using multiple multicast trees for each VNI in order to support IGMP Snooping?

If so, I don't see this as a MUST requirement for two reasons. First, many data centers do not support IP multicast services for the applications running within them, and therefore burdening all implementations with a feature that may not be needed seems unnecessary. Second, the amount of multicast routes available in the data center may already be strained to have enough routes for traffic sourced by the NVEs towards the (*, G) just to support the BUM traffic within a VNI. When many VNIs exist within the data center, there may not be enough multicast state for each VNI to have its own single (*, G), let alone multiple for each VNI for this IGMP Snooping function. I think this can be downgraded from a MUST to a MAY.

- Larry

From: <LASSERRE>, "MARC (MARC)" <[email-masked]<mailto:[email-masked]>>
Date: Thursday, May 17, 2012 1:31 AM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: [nvo3] New dataplane requirements draft

Thanks,
Marc

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Mon, 21 May 2012 21:02:34 +0000

Hi Marc and Nabil,

In section 3.2.2 of this draft it states:

"Configuration of the uniform TTL model where the outer tunnel TTL is set equal to the inner TTL on ingress NVE and the inner TTL is set to the outer TTL value on egress should be supported."

Is this "uniform TTL model" explained in some RFC that can look at?  I'm curious to know how it works, particularly what happens with the outer TTL expires and an ICMP is returned to the tunnel source and how traceroute is handled.  I fail to see the benefit of this mode when the underlying network routers are not visible to the tenant end systems inside the VNI.  It seems like this behavior should be a "should not" and that the TTL within the VNI and within the underlying network should be kept separate.

 - Larry

Thanks,
Marc

---

From: Yiqun Cai <[email-masked]>
Date: Mon, 21 May 2012 21:13:40 +0000

The document should not refer to either (*,G) or (S,G).  Instead, it should specify whether an ASM or SSM multicast service model is to be desired from the point of view of an NVE, and what happens if neither multicast service model is available.

--
Yiqun

From: [email-masked] [mailto:[email-masked]] On Behalf Of AshwoodsmithPeter
Sent: Friday, May 18, 2012 7:30 AM
To: LASSERRE, MARC (MARC); [email-masked]
Subject: Re: [nvo3] New dataplane requirements draft

Peter

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> On Behalf Of LASSERRE, MARC (MARC)

Thanks,
Marc

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Mon, 21 May 2012 21:26:07 +0000

Hi Marc and Nabil,

In section 3.2.2, I was not sure what the following statement meant:

"In addition, integrated routing and bridging per tenant must be supported on an NVE."

One could interpret this to mean that NVEs must support both L2 and L3 mode…but then two sentences previous to that it stated "L2 and L3 VNIs can be deployed in isolation or in combination to optimize traffic flows per tenant across the overlay network."

Can you clarify the meaning for the first quoted sentence?  If it means that NVEs must provide both L2 and L3 VNI services, then I don't see why that should be so.  The NVEs may well provide only L2 services and have end systems connected to them provide the L3 services to interconnect the L2 VNIs.

Thanks, Larry

Thanks,
Marc

---

From: Yiqun Cai <[email-masked]>
Date: Mon, 21 May 2012 21:38:39 +0000

I agree with Larry.

Besides, I am also questioning the following statement in the same paragraph,  "the inner TTL field MUST be decremented by 1".  I think a more precise description is that "the inner TTL field MUST be decremented by <at least> 1".  Decrementing by more than 1 should be allowed.

And is there any reason why the overlay network should look like one hop to the sending NVE?

--
Yiqun

From: [email-masked] [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Monday, May 21, 2012 2:26 PM
To: LASSERRE, MARC (MARC); [email-masked]
Subject: [nvo3] nvo3 DP Reqs: integrated bridging and routing

Hi Marc and Nabil,

One could interpret this to mean that NVEs must support both L2 and L3 mode...but then two sentences previous to that it stated "L2 and L3 VNIs can be deployed in isolation or in combination to optimize traffic flows per tenant across the overlay network."

Thanks, Larry

Thanks,
Marc

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Tue, 22 May 2012 09:20:32 +0200

Hi Larry,

Agreed, this sentence should have been a MAY.

Additionally, I'll clarify that either ingress replication or the use of multicast can be used for BUM traffic.

Marc

________________________________
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Monday, May 21, 2012 10:46 PM
To: LASSERRE, MARC (MARC); [email-masked]
Subject: Re: [nvo3] New dataplane requirements draft

Hi Marc and Nabil,

- Larry

Thanks,
Marc

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Tue, 22 May 2012 10:09:19 +0200

Hi Larry,

The idea is whether to treat the nvo3 tunnel as a circuit (pipe model) or the make all the nodes that an nvo3 tunnel traverses visible to nodes outside the tunnel (uniform model) as discussed in rfc3443.
The pipe model should be a MUST while the uniform model should be a MAY. I'd like to hear from SPs/DC operators if they see any value with the uniform model. If not, we will remove it from the draft.

Marc

________________________________
From: Larry Kreeger (kreeger) [mailto:[email-masked]]
Sent: Monday, May 21, 2012 11:03 PM
To: LASSERRE, MARC (MARC); [email-masked]
Subject: nvo3 DP Reqs: TTL handling

Hi Marc and Nabil,

 - Larry

Thanks,
Marc

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Tue, 22 May 2012 10:21:59 +0200

Yiqun,

The draft does not refer specifically to (*,G) or (S,G) except for section 3.2.1 which explains that BUM traffic needs to be handled for L2 NVEs.
When multicast is not enabled, ingress replication is required.

As Peter asked in his mail, let's hear from DC/Cloud operators whether multicast is a MAY or a MUST. So far, it seems to be a soft requirement i.e. a MAY and not a MUST.

Marc
________________________________
From: Yiqun Cai [mailto:[email-masked]]
Sent: Monday, May 21, 2012 11:14 PM
To: AshwoodsmithPeter; LASSERRE, MARC (MARC); [email-masked]
Subject: RE: New dataplane requirements draft

--
Yiqun

Peter

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:%5bmailto:[email-masked]%5d> On Behalf Of LASSERRE, MARC (MARC)

Thanks,
Marc

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Tue, 22 May 2012 10:32:14 +0200

Larry,

Clearly, the intent is not to require that both L2 & L3 services be supported.
I agree that this sentence is confusing. I'll check with the other co-authors and clarify what was meant.

Marc

Sent: Monday, May 21, 2012 11:26 PM

Hi Marc and Nabil,

Thanks, Larry

Thanks,
Marc

---

From: "LASSERRE, MARC (MARC)" <[email-masked]>
Date: Tue, 22 May 2012 10:35:37 +0200

Ok, we'll fix it in the next rev.

Marc

________________________________
From: Yiqun Cai [mailto:[email-masked]]
Sent: Monday, May 21, 2012 11:39 PM
To: Larry Kreeger (kreeger); LASSERRE, MARC (MARC); [email-masked]
Subject: RE: nvo3 DP Reqs: integrated bridging and routing

I agree with Larry.

--
Yiqun

Hi Marc and Nabil,

Thanks, Larry

Thanks,
Marc

---

From: Lucy yong <[email-masked]>
Date: Tue, 22 May 2012 14:22:12 +0000

Larry and Marc,

Integrated routing and bridging per tenant is useful feature for DC operator. It allows operator to design virtual router and bridge functions per tenant for various purposes. I agree that the sentence should be "May be supported" instead of "must be supported".

Lucy

From: [email-masked] [mailto:[email-masked]] On Behalf Of LASSERRE, MARC (MARC)
Sent: Tuesday, May 22, 2012 3:32 AM
To: Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] nvo3 DP Reqs: integrated bridging and routing

Larry,

Marc

Hi Marc and Nabil,

Thanks, Larry

Thanks,
Marc

---

From: Yiqun Cai <[email-masked]>
Date: Wed, 23 May 2012 00:10:14 +0000

Marc,

The following was the sentence in question.

"The flooding tree is equivalent with a multicast (*,G) construct where all the NVEs where the corresponding VNI is instantiated are members."

"(*,G) construct" doesn't deliver any precise meaning. One could interpret it as "ASM" while another could read it as "PIM shared tree" state. Neither would sever well in this document.   My point was that it is better to describe the requirement from the POV of the multicast service model.  This is especially important if an NVE is considered as a host and not part of the L3 data center network.

The volume of IP multicast traffic  in today's data center network is not "0".  Whether it would go towards 0 or away from it is a good question.  I'm not sure our speculation of its moving direction will help to determine the MUST/SHOULD/MAY requirement of the service models.  A detailed analysis is more useful and IETF can certainly help here.

By the way, some portion of my day job is indeed related to operating DC/Cloud.  But I'm commenting as an individual IETF participant, and *NOT* on behalf of my previous or current employers ;)

Thanks.

--
Yiqun

From: LASSERRE, MARC (MARC) [mailto:[email-masked]]
Sent: Tuesday, May 22, 2012 1:22 AM
To: Yiqun Cai; AshwoodsmithPeter; [email-masked]
Subject: RE: New dataplane requirements draft

Yiqun,

Marc
________________________________
From: Yiqun Cai [mailto:[email-masked]]<mailto:[mailto:[email-masked]]>
Sent: Monday, May 21, 2012 11:14 PM
To: AshwoodsmithPeter; LASSERRE, MARC (MARC); [email-masked]<mailto:[email-masked]>
Subject: RE: New dataplane requirements draft

--
Yiqun

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]]<mailto:[mailto:[email-masked]]> On Behalf Of AshwoodsmithPeter
Sent: Friday, May 18, 2012 7:30 AM
To: LASSERRE, MARC (MARC); [email-masked]<mailto:[email-masked]>
Subject: Re: [nvo3] New dataplane requirements draft

Peter

Thanks,
Marc

---

From: AshwoodsmithPeter <[email-masked]>
Date: Wed, 23 May 2012 14:55:32 +0000

> The volume of IP multicast traffic  in today's data center network is not "0".  Whether it would go towards 0 or away from it is a good
> question.  I'm not sure our speculation of its moving direction will help to determine the MUST/SHOULD/MAY requirement of the service
> models.  A detailed analysis is more useful and IETF can certainly help here.
..
> Yiqun

http://tools.ietf.org/pdf/draft-mcbride-armd-mcast-overview-01.pdf

The above draft goes into an overview of multicast uses in the DC. I assume its far from exhaustive but it's a good start.

They describe 3 broad non ARP/ND/BUM cases, but presumably there are many more which would be useful to add to this draft.

(1) - VM sourcing multicast data to a host outside the DC, like IPTV, stock quotes etc.

(2) - VRRP - so two routers etc. deciding who is active/standby

(3) - Cluster maintenance - Ganglia, Windows Server heartbeats. Etc.

Both (2) and (3) are really signalling control functions taking advantage of multicast, while (1) is presumably not sinking any of that traffic within the DC.

There are applications out there using multicast when what they really want is fast reliable distirbuted signalling and/or PUB/SUB and I think we should be taking a forward looking view here in finding a way to provide that in addition to whatever the legacy applications require of IP/Ethernet multicast.

Peter