Subject: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

From: Benson Schliesser <[email-masked]>
Date: Fri, 12 Sep 2014 17:42:15 -0400

Dear NVO3 Contributors -

This message is to initiate a Working Group Last Call for Comments on draft-ietf-nvo3-vm-mobility-issues. The chairs believe there is consensus to submit this draft to the IESG for publication. Please review it and provide feedback on the mailing list by 19-Sep-2014.

As a reminder, this is not an opportunity to vote. Please do not post messages that simply indicate support. Rather, substantial comments and feedback is encouraged.

For your convenient reference, the latest version of the draft can be found at https://tools.ietf.org/html/draft-ietf-nvo3-vm-mobility-issues-03.

Thanks,
-Benson & Matthew

---

From: "Xialiang (Frank)" <[email-masked]>
Date: Tue, 16 Sep 2014 00:38:36 +0000

Hi all,
I support this draft to be adopted as a WG draft. And I think those issues about VM mobility haven't been covered in details by "draft-ietf-nvo3-overlay-problem-statement-04 (RFC queue)".

B.R.
Frank

> -----Original Message-----
> From: nvo3 [mailto:[email-masked]] On Behalf Of Benson Schliesser
> Sent: Saturday, September 13, 2014 5:42 AM
> To: [email-masked]
> Subject: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues
> 
> Dear NVO3 Contributors -
> 
> This message is to initiate a Working Group Last Call for Comments on
> draft-ietf-nvo3-vm-mobility-issues. The chairs believe there is consensus to
> submit this draft to the IESG for publication. Please review it and provide
> feedback on the mailing list by 19-Sep-2014.
> 
> As a reminder, this is not an opportunity to vote. Please do not post messages
> that simply indicate support. Rather, substantial comments and feedback is
> encouraged.
> 
> For your convenient reference, the latest version of the draft can be found at
> https://tools.ietf.org/html/draft-ietf-nvo3-vm-mobility-issues-03.
> 
> Thanks,
> -Benson & Matthew
> 
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: Xuxiaohu <[email-masked]>
Date: Tue, 16 Sep 2014 02:09:47 +0000

This draft describes well the VM mobility specific issues and it's complementary to the NVO3 PS draft. Therefore I support the publication of this draft.

Xiaohu

---

From: Haoweiguo <[email-masked]>
Date: Wed, 17 Sep 2014 01:25:22 +0000

This draft clearly defined the specific issues for VM mobility, i support this draft to be adopted as a WG draft. 
Thanks
weiguo
________________________________________
发件人: Xialiang (Frank) [[email-masked]]
发送时间: 2014年9月16日 8:38
收件人: Benson Schliesser
抄送: [email-masked]
主题: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Hi all,

B.R.
Frank

---

From: "Black, David" <[email-masked]>
Date: Tue, 16 Sep 2014 21:59:32 -0400

Three quick questions:

(1) Why is this draft intended as standards track?  What protocol or standard does it specify?
	Both the problem statement and framework drafts are Informational.

(2) What is the nature of the use of RFC 2119 terms (e.g., "MUST") in this document?

(3) Why are the security considerations "TBD"?  Do the authors really think that's acceptable?

Also, a 1-week WG LC time period is really short - that will not permit me to do a thorough technical review of this draft.

Thanks,
--David

> Sent: Friday, September 12, 2014 5:42 PM
> To: [email-masked]
> This message is to initiate a Working Group Last Call for Comments on draft-
> ietf-nvo3-vm-mobility-issues. The chairs believe there is consensus to submit
> this draft to the IESG for publication. Please review it and provide feedback
> on the mailing list by 19-Sep-2014.
>

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 17 Sep 2014 06:03:20 -0700

David,

> 	Both the problem statement and framework drafts are Informational.

Good point. The draft should be progressed as Informational.

> (2) What is the nature of the use of RFC 2119 terms (e.g., "MUST") in 

To answer this question let's look at few examples of the use of RFC2119
terms in the document:

>From 3.1:

   This document assumes that within a given non-trivial L2 physical
   domain traffic from/to VMs that are in that domain, and belong to the
   same L2-based CUG MUST have the same VLAN-ID.

In the above "MUST" indicates that in the context of this document
the assumption about VLAN-ID can not be violated.

>From 3.5:

                                       In other words, the policies that
   control connectivity between a given VM and its peers MUST NOT change
   as the VM moves from one L2 physical domain to another.

   ....

                                                            Moreover,
   policies, if any, within the L2 physical domain that contain a given
   VM MUST NOT preclude realization of the policies that control
   connectivity between this VM and its peers.

In the above "MUST" and "MUST NOT" indicates that in the context
of this document any policies within the L2 physical domain can not
interfere with the policies that control connectivity between and
given VM and its peers.

> (3) Why are the security considerations "TBD"?  Do the authors really 

The authors hope that once the document is accepted as an NVO3 WG
document, this section will be completed based on the feedback
we'll receive from the NVO3 WG.

> Also, a 1-week WG LC time period is really short - that will not 
> permit me to do a thorough technical review of this draft.
> 
> Thanks,
> --David
>

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Sep 2014 14:52:11 -0400

> we'll receive from the NVO3 WG.

Then the authors are confused, because that draft already is an NVO3 WG
document, and hence has been sent to WG Last Call, even though it's
incomplete.  Perhaps the authors could complete the draft and we could
consider trying again?

Leaving security considerations as "an exercise to the reader" has a
rather poor track record in the IETF.

Thanks,
--David

> From: Yakov Rekhter [mailto:[email-masked]]
> Sent: Wednesday, September 17, 2014 9:03 AM
> To: Black, David
> Cc: Benson Schliesser; [email-masked]
> Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues
> 
> David,
> 
> > Three quick questions:
> >
> > (1) Why is this draft intended as standards track?  What protocol or
> standard
> > does it specify?
> draft
> -
> > > ietf-nvo3-vm-mobility-issues. The chairs believe there is consensus to
> subm
> it
> > > this draft to the IESG for publication. Please review it and provide
> feedba
> ck
> > > As a reminder, this is not an opportunity to vote. Please do not post
> messa
> ges
> > > For your convenient reference, the latest version of the draft can be
> found
>  at

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Sep 2014 16:24:50 -0400

Yaakov,

On 2119 keywords.

> the assumption about VLAN-ID can not be violated.

Lower case "must" is fine for stating an assumption, and I would suggest
doing so.

> interfere with the policies that control connectivity between and

Ok, this seems fine for stating requirements.  When changing the document
to Informational, I suggest adding a sentence or two to Section 1 to 
expand on its title - in essence these keywords are being used to state
protocol design requirements, not protocol implementation/behavior requirements.

Thanks,
--David

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 17 Sep 2014 21:08:29 +0000

David,

> From: Black, David [mailto:[email-masked]]
> Sent: Wednesday, September 17, 2014 2:52 PM
> To: Yakov Rekhter
> Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues
> 
> > document, this section will be completed based on the feedback we'll
> > receive from the NVO3 WG.
> 
> incomplete. 

Sorry, my mistake.

> Perhaps the authors could complete the draft and we could
> consider trying again?
> 

Agreed. With this in mind is there anything specific you would suggest to
put in the Security Considerations section ?

Yakov.

> > To answer this question let's look at few examples of the use of
> > RFC2119 terms in the document:
> >
> > From 3.1:
> >
> > In the above "MUST" indicates that in the context of this document the
> > assumption about VLAN-ID can not be violated.
> >
> > From 3.5:
> >
> > In the above "MUST" and "MUST NOT" indicates that in the context of
> > this document any policies within the L2 physical domain can not
> > document, this section will be completed based on the feedback we'll
> > receive from the NVO3 WG.
> >
> > > > that simply indicate support. Rather, substantial comments and
> > > > feedback is encouraged.
> > > >

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 17 Sep 2014 21:12:17 +0000

David,

> Sent: Wednesday, September 17, 2014 4:25 PM
> To: Yakov Rekhter
> Cc: [email-masked]

Ok.

> protocol design requirements, not protocol implementation/behavior

Would you please propose "a sentence or two" to add to Section 1 ?

Yakov.

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Sep 2014 17:24:21 -0400

> Would you please propose "a sentence or two" to add to Section 1 ?

After the existing text:

This informational document uses these keywords to express requirements for
protocol design in contrast to the more common use of these keywords in
standards track documents to express requirements for protocol implementation,
deployment and/or usage.

Thanks,
--David

> Sent: Wednesday, September 17, 2014 5:12 PM
> To: Black, David
> Cc: [email-masked]

---

From: Yakov Rekhter <[email-masked]>
Date: Wed, 17 Sep 2014 21:35:58 +0000

David,

> Sent: Wednesday, September 17, 2014 5:24 PM
> To: Yakov Rekhter
> Cc: [email-masked]
> > > Ok, this seems fine for stating requirements.  When changing the
> > > document to Informational, I suggest adding a sentence or two to
> > > Section 1 to expand on its title - in essence these keywords are
> > > being used to state protocol design requirements, not protocol
> > > implementation/behavior requirements.
> >
> This informational document uses these keywords to express requirements
> for protocol design in contrast to the more common use of these keywords
> in standards track documents to express requirements for protocol
> implementation, deployment and/or usage.

Thanks ! I'll add this to the revised draft.

Yakov.

> > > >    domain traffic from/to VMs that are in that domain, and belong
> > > > to
> > > the
> > > Lower case "must" is fine for stating an assumption, and I would
> > > suggest doing so.
> >
> > Ok.
> >
> > >
> > > > From 3.5:
> > > >
> > > >                                        In other words, the
> > > > policies
> > > that
> > > > of this document any policies within the L2 physical domain can
> > > > not interfere with the policies that control connectivity between
> > > > and given VM and its peers.
> > >
> > > Ok, this seems fine for stating requirements.  When changing the
> > > document to Informational, I suggest adding a sentence or two to
> > > Section 1 to expand on its title - in essence these keywords are
> > > being used to state protocol design requirements, not protocol
> > > implementation/behavior requirements.
> >
> > > >    domain traffic from/to VMs that are in that domain, and belong
> > > > to
> > > the
> > > >                                        In other words, the
> > > > policies
> > > that
> > > > of this document any policies within the L2 physical domain can
> > > > not interfere with the policies that control connectivity between
> > > > and given VM and its peers.
> > > >
> > > > >

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 17 Sep 2014 23:31:24 +0000

Hi Benson,

Linda was supposed to have a slot during last interim conference call "to
show substantial content in this draft with regard to VM mobility in DC
that are not in the NVO3 problem statement".  Linda didn't do this, so I'm
not sure why this document is being put up for Last Call.

I just read through the document again after a long long time and came
away scratching my head about its value.  I have written down some of my
impressions below as I read it, but in summary I'm not convinced that this
should be published.  It seems to spend a lot of time defining terminology
(e.g. Trivial vs non-trival L2 Physical domain) that in the end doesn't
seem to have anything to do with an overlay virtual network. I would still
like to see Linda's presentation on the "substantial content" that I
missed in my reading.

 - Larry

The document discusses the seamless migration of VMs between two Data
Centers.  Based on the recent charter discussions, the scope of NVO3 seems
to be limited to within a single DC.  In that case should discussion of
inter-DC VM mobility be removed?

The document uses the term CUG (Closed User Group) extensively.  The
problem statement and framework use the term VN (Virtual Network).  The
framework mentions CUG by saying "A VN is also known as a Closed User
Group (CUG)."  Given this, I think this document should use the consistent
term, VN in placed of CUG.

Section 3.1 seems to be equating CUGs to VLANs in an L2 physical domain.
I'm not sure what this has to do with NVO3 Overlay VNs.  It makes "the
assumption that within a given L2 physical domain VLANs are used to
identify individual L2-based CUGs". Why?

Section 3.3 continues this assumption that an L2 CUG is bounded by an L2
physical domain by saying that when a VM moves to a new L2 physical domain
"the new L2 physical domain must become interconnected with the other L2
physical domain".  Since NVO3 is all about creating an overlay over IP, I
don't understand why one would need to extend VLANs from one L2 physical
domain to another providing a "Layer 2 Extension".  The purpose of using
NVO3 overlays is to avoid having to extend L2 physical domains.

Section 3.4 spends most if its time discussing what "Optimal IP Routing"
means, only to go on to say "The ability to deliver optimal routing (as
defined above) in the presence of stateful devices is outside the scope of
this document.".  Why bother spending time to define something which is
out of scope?

Section 3.5 discusses "policies that control connectivity between a given
VM and its peers", which I understand based on the definition of CUG, to
be the VN itself.  It then seems to go on to say that these policies (I.e.
the VN) must not change whether the VM moves between two different L2
physical domains or stays within the same L2 physical domain.  Isn't this
section just a complicated way of stating that when a VM moves, it should
remain connected to the same VN it was originally connected to?  Isn't
that trivial and obvious?

On 9/12/14 2:42 PM, "Benson Schliesser" <[email-masked]> wrote:

>draft-ietf-nvo3-vm-mobility-issues. The chairs believe there is consensus
>to submit this draft to the IESG for publication. Please review it and
>provide feedback on the mailing list by 19-Sep-2014.
>
>messages that simply indicate support. Rather, substantial comments and
>feedback is encouraged.
>
>found at

---

From: Linda Dunbar <[email-masked]>
Date: Thu, 18 Sep 2014 15:43:32 +0000

Larry,

See my replies inserted below:

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Wednesday, September 17, 2014 6:31 PM
To: Benson Schliesser; [email-masked]
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Hi Benson,

Linda was supposed to have a slot during last interim conference call "to show substantial content in this draft with regard to VM mobility in DC that are not in the NVO3 problem statement".  Linda didn't do this, so I'm not sure why this document is being put up for Last Call.

[Linda] It turned out that the NVO3 virtual meeting finished by 11am PST (that is when I was able to join due to my other commitment).  Benson suggested me to send out my points in respond to the WGLC.

      The draft addresses the following part of the charter:
        "Support the placement and migration of VMs anywhere  within the data center, without being limited by DC network  constraints such as the IP subnet boundaries of the underlying DC network."
      It highlights how we should support VM mobility in NVO3 and the issues related to such support

      Those issues haven't been covered in details by "draft-ietf-nvo3-overlay-problem-statement-04 (RFC queue)".

      This draft shouldn't be merged with general NVo3 framework nor problem statement draft due to:
*       The draft emphasizes on detailed issues related to VM mobility beyond the general NVo3 problem statement draft
*       This topic is big enough to deserve its own place.
*       Merging those detailed issues make the general draft not readable.

I just read through the document again after a long long time and came away scratching my head about its value.  I have written down some of my impressions below as I read it, but in summary I'm not convinced that this should be published.  It seems to spend a lot of time defining terminology (e.g. Trivial vs non-trival L2 Physical domain) that in the end doesn't seem to have anything to do with an overlay virtual network. I would still like to see Linda's presentation on the "substantial content" that I missed in my reading.

[Linda] Here are some key points that haven't been addressed by general NVO3 problem statement:
*       Optimal routing of a VM's outbound traffic
*       Optimal routing of VM's inbound traffic
*       Preserving Policies:
            allowing VMs migrating across Data Center will require more stringent security enforcement. The traditional placement of security functions, e.g. firewall, at data center gateways is no longer enough. VM mobility will require security functions to enforce policies among east-west traffic among VMs. When VMs move across Data Center, the associated policies have to be updated dynamically and enforced.
*       Maintaining Connectivity in the Presence of VM Mobility
*       Layer 2 Extension

 - Larry

The document discusses the seamless migration of VMs between two Data Centers.  Based on the recent charter discussions, the scope of NVO3 seems to be limited to within a single DC.  In that case should discussion of inter-DC VM mobility be removed?

[Linda] even within a single DC, you will have problem of VMs migrating across different PODs, which have the same problem of moving across different DCs.

The document uses the term CUG (Closed User Group) extensively.  The problem statement and framework use the term VN (Virtual Network).  The framework mentions CUG by saying "A VN is also known as a Closed User Group (CUG)."  Given this, I think this document should use the consistent term, VN in placed of CUG.

[Linda] We can change CUG to VN.

Section 3.1 seems to be equating CUGs to VLANs in an L2 physical domain.
I'm not sure what this has to do with NVO3 Overlay VNs.  It makes "the assumption that within a given L2 physical domain VLANs are used to identify individual L2-based CUGs". Why?

[Linda] The text is to address issues of today's L2 network being identified by VLANs.

Section 3.3 continues this assumption that an L2 CUG is bounded by an L2 physical domain by saying that when a VM moves to a new L2 physical domain "the new L2 physical domain must become interconnected with the other L2 physical domain".  Since NVO3 is all about creating an overlay over IP, I don't understand why one would need to extend VLANs from one L2 physical domain to another providing a "Layer 2 Extension".  The purpose of using
NVO3 overlays is to avoid having to extend L2 physical domains.

[Linda] The "L2 physical domain" in the text is about the L2 network attached to NVEs. The overlay is among NVEs. This is another area that hasn't been covered by the general problem statement.

Section 3.4 spends most if its time discussing what "Optimal IP Routing"
means, only to go on to say "The ability to deliver optimal routing (as defined above) in the presence of stateful devices is outside the scope of this document.".  Why bother spending time to define something which is out of scope?

[Linda] This section address the issues, whose solutions are out of the scope of this draft. There will be separate IDs to describe the issues.

Section 3.5 discusses "policies that control connectivity between a given VM and its peers", which I understand based on the definition of CUG, to be the VN itself.  It then seems to go on to say that these policies (I.e.
the VN) must not change whether the VM moves between two different L2 physical domains or stays within the same L2 physical domain.  Isn't this section just a complicated way of stating that when a VM moves, it should remain connected to the same VN it was originally connected to?  Isn't that trivial and obvious?

[Linda] No, it is not trivial at all. With 80% traffic being east-west, the security policies, such as FW policies, zone policies, the AntiDoS policies have to be changed on different segments within a single DC. When VMs move, all the relevant policies have to be moving too!

On 9/12/14 2:42 PM, "Benson Schliesser" <[email-masked]<mailto:[email-masked]>> wrote:

>draft-ietf-nvo3-vm-mobility-issues. The chairs believe there is
>consensus to submit this draft to the IESG for publication. Please
>review it and provide feedback on the mailing list by 19-Sep-2014.
>
>[email-masked]<mailto:[email-masked]>

_______________________________________________
nvo3 mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/nvo3

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Thu, 18 Sep 2014 17:39:08 +0000

Hi Linda,

Thanks for following up.  See my replies inline marked with LK>.

 - Larry

From: Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Date: Thursday, September 18, 2014 8:43 AM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, Benson Schliesser <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Larry,

-----Original Message-----
From: nvo3 [mailto:[email-masked]] On Behalf Of Larry Kreeger (kreeger)
Sent: Wednesday, September 17, 2014 6:31 PM
To: Benson Schliesser; [email-masked]<mailto:[email-masked]>
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Hi Benson,

The draft addresses the following part of the charter:
  "Support the placement and migration of VMs anywhere  within the data center, without being limited by DC network  constraints such as the IP subnet boundaries of the underlying DC network.“
It highlights how we should support VM mobility in NVO3 and the issues related to such support

LK> I'm not sure why you think VM mobility needs more highlighting when it is in fact the main motivating use case for NVO3 as described in the problem statement (and as you point out, mentioned in the charter).

LK> I did not read anything in this document that makes clear why the networking issues are specific to VM mobility vs other types of mobility, e.g. containers, or even switchover of a function to a hot standby that takes over the same MAC/IP.

LK> I disagree.

  *

[Linda] Here are some key points that haven’t been addressed by general NVO3 problem statement:

LK> Optimal routing (as David pointed out in his comments) is covered in section 3.7 of the problem statement.  Can you please re-read that section and let me know the substantial content that is missing from it?

  *   Preserving Policies:

LK> First, the draft is unclear about what policies need to be preserved.  The draft seems to equate policy with a L2 CUG (which you seem to agree is the same as an NVO3 VN), so I'm not clear on what policies are meant beyond the implementation of the VN itself.  Also note that section 3.3 of the Framework says "Upon VM mobility, NVE policies that define connectivity among VMs must be maintained."  This seems to cover the gist of section 3.5 in the VM mobility issues draft, with the exception that it uses a paragraph to discuss how this is somehow related to the physical L2 domain (which it shouldn't be for an overlay network).

LK> This seems to be covered in section 3.2 of the problem statement.

  *   Layer 2 Extension

LK>  The whole motivation of creating an overlay network is to avoid the problems of trying to extend the L2 physical domain, which seems to be the focus of the Layer 2 extension section of this draft.  Overcoming the need for L2 extension is covered in section 3.4 of the problem statement.

- Larry

LK> I agree, but a significant portion of this document seems to be covering this topic which is out of scope for our charter.

[Linda] The text is to address issues of today’s L2 network being identified by VLANs.

LK> If so, it wasn't clear to me. In that case, it is already covered in section 3.4 of the problem statement.

[Linda] The “L2 physical domain” in the text is about the L2 network attached to NVEs. The overlay is among NVEs. This is another area that hasn’t been covered by the general problem statement.

LK> So, then if I understand you, this section is related to the split-NVE case where locally significant VLANs are being used to keep the VN traffic segregated as it flows to an external NVE?  If so, that is part of the solution architecture.  It also does not seem specific the VM mobility.

LK> Really? More drafts to cover one niche? I really don’t like the approach of creating a new draft for every niche issue.

LK> This was not clear to me in the draft.  What is currently in the draft does not seem to go beyond the simple statement made in section 3.3 of the Framework.

---

From: Thomas Narten <[email-masked]>
Date: Fri, 19 Sep 2014 13:42:49 -0400

Hi.

I have reviewed the document and I agree with the comments Larry and
David have made and have not been swayed by the responses I've seen so
far. Indeed, if one goes back through the archive to when the document
was originally adopted, some of the same points being raised now were
raised then.

Frankly, I don't think it covers VM mobility issues very well at
all. It doesn't even mention the key issue: moving VMs around is
limited in that VMs can't easily take their IP addresses with them
when they move, unless they are restricted to staying within their IP
subnet and VLAN. That restriction has become problematic and is the
key problem overlays (and NVO3) aim to address. The document doesn't
even mention this as an issue!

It might also be interesting to review some history.

This document was adopted by the WG at the end of 2012, some 21 months
ago. Since that time, there has been essentially no discussion of the
draft and there have been no substantive revisions to the document -
just an update of the date and version number. Hardly an endorsement
of WG interest in the document!

Diff: draft-ietf-nvo3-vm-mobility-issues-00.txt - draft-ietf-nvo3-vm-mobility-issues-03.txt
https://www.ietf.org/rfcdiff?url1=draft-ietf-nvo3-vm-mobility-issues-00&difftype=--html&submit=Go%21&url2=draft-ietf-nvo3-vm-mobility-issues-03

I was opposed to adopting the document in the first place, because I
did not understand the purpose of the document and what content it was
intended to have w.r.t. the charter and what additional content the WG
wanted it to contain and why/how the material wasn't already
adequately covered in the problem statement. And here we are now...

Thomas

---

From: Tapraj Singh <[email-masked]>
Date: Fri, 19 Sep 2014 20:45:12 +0000

Hi,

I have reviewed the draft. It describes the VM mobility issues very well.

I going through this document. This document describes the VM mobility
deployment and 
Issues very well. Based on the contents of the document I fully agree that
it is ready to move to IESG stage.

Tapraj Singh.

---

From: Linda Dunbar <[email-masked]>
Date: Fri, 19 Sep 2014 22:53:34 +0000

Larry,

Thanks for the feedback. Please let us know if the updated text will address your concerns (marked by [Linda-2]):

From: Larry Kreeger (kreeger) [mailto:[email-masked]]

[Linda-2] When VMs in a single VN are spread across many different DCBRs, all DCBRs have to announce their directly VMs in order to achieve optimal routing. This approach can dramatically increase the number of routes on routers in the wide area network.

*       Preserving Policies:

[Linda-2] There could be many policies guarding VMs across different VNs, with some being enforced by Firewall, some enforced by NAT/AntiDDOS/IPS/IDS, etc.  It is less about NVE polices to be maintained when VMs move, it is more along the line of dynamically changing policies associated with the "middleware" boxes attached to NVEs (if those middle boxes are distributed).
*       Maintaining Connectivity in the Presence of VM Mobility

*       Layer 2 Extension

- Larry

[Linda-2] Most large DCs have regular IP interconnecting different PODs. Moving VMs among DCs has same issue of moving VMs among PODs.

[Linda-2] The Section 3.4 of the problem statement didn't address scenario when there is a bridged network between NVE and Guest OS (either VM or server) and  tenant systems use their own VID (with policies on tenant's VIDs).

[Linda- 2] The "split-NVE" is assuming that NVE is directly attached to VMs (Guest OS). When there is a bridged network (virtual switches, blade switch attached to NVE on TOR), additional mechanism is needed for the first hop switch to do the translation (before NVE).

Linda

---

From: "Black, David" <[email-masked]>
Date: Fri, 19 Sep 2014 20:02:25 -0400

Butting in w/one comment ...

> [Linda-2] When VMs in a single VN are spread across many different DCBRs, all DCBRs have to announce their directly

> VMs in order to achieve optimal routing. This approach can dramatically increase the number of routes on routers in the wide area network.

I think that comment is in solution-space, not problem-space or issue-space (where this draft is supposed to be), as the comment appears to be based on specific assumptions about DCBR existence and functionality that aren't true in general.

For example, I'm sure Yakov can explain how BGP can do better than that ;-).

Thanks,
--David

From: nvo3 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Friday, September 19, 2014 6:54 PM
To: Larry Kreeger (kreeger); Benson Schliesser; [email-masked]
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Larry,

*         Preserving Policies:

*         Layer 2 Extension

- Larry

Linda

---

From: Benson Schliesser <[email-masked]>
Date: Fri, 19 Sep 2014 18:20:04 -0700

Dear NVO3 Contributors -

During the WG Last Call period ending today, list discussion and feedback
has made it clear to the chairs that there is currently not consensus to
publish this draft. The authors and other contributors are encouraged to
continue discussion to help the chairs determine the most appropriate
disposition for this document.

Cheers,
-Benson & Matthew
On Sep 12, 2014 5:41 PM, "Benson Schliesser" <[email-masked]> wrote:

> found at https://tools.ietf.org/html/draft-ietf-nvo3-vm-mobility-issues-03
> .
>
> Thanks,
> -Benson & Matthew
>
>

---

From: "Lizhong Jin" <[email-masked]>
Date: Sun, 21 Sep 2014 23:51:12 +0800

Hi Chairs,
I thought the last call would be ended in two weeks. Sorry for the delay reply.
I review the draft, and I also see many overlaps with the problem-statement draft, which have been already discussed in the list.
1. Section 2, it emphasizes "seamless", but the definition is missed. I thought it at least included "no traffic loss", but it is not true when I read section 3.2.
2. Section 3.1, I believe the point described in the draft is correct. But I think the description in problem-draft as below has already contain VLAN.
A key requirement for live migration
   is that a VM retain critical network state at its new location,
   including its IP and MAC address(es)
As we know, linux stack could support other Layer2 header, e.g., SNAP, PPPoE, then should we describe all of them explicitly? I think a general description is enough.
3. Section 3.4 is overlap with section 3.7 of problem-draft.
4. Section 3.5, I went through the list discussion, the author refers to FW policy. I am curious, does the middle-box problem of VM mobility is in the scope of NVO3 WG? I did not see that in charter.

Regards
Lizhong Jin
 
From: Benson Schliesser
Date: 2014-09-20 09:20
To: nvo3
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues
Dear NVO3 Contributors -
During the WG Last Call period ending today, list discussion and feedback has made it clear to the chairs that there is currently not consensus to publish this draft. The authors and other contributors are encouraged to continue discussion to help the chairs determine the most appropriate disposition for this document. 
Cheers,
-Benson & Matthew 
On Sep 12, 2014 5:41 PM, "Benson Schliesser" <[email-masked]> wrote:
Dear NVO3 Contributors -

Thanks,
-Benson & Matthew

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 22 Sep 2014 15:32:11 +0000

David,

See comment in line below

From: Black, David [mailto:[email-masked]]
Sent: Friday, September 19, 2014 7:02 PM
To: Linda Dunbar; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

[Linda-3] "How BGP can do better" is actually a solution. What described is a problem.

Linda

Thanks,
--David

From: nvo3 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Friday, September 19, 2014 6:54 PM
To: Larry Kreeger (kreeger); Benson Schliesser; [email-masked]<mailto:[email-masked]>
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Larry,

*       Preserving Policies:

*       Layer 2 Extension

- Larry

Linda

---

From: "Black, David" <[email-masked]>
Date: Mon, 22 Sep 2014 11:46:05 -0400

Linda,

I used BGP as an example to indicate that the "have to announce" statement is not correct.

There's no need to describe what BGP does, but stating that something cannot currently be done (i.e., needs WG attention) when BGP already does that something is not a good idea.

Thanks,
--David

From: Linda Dunbar [mailto:[email-masked]]
Sent: Monday, September 22, 2014 11:32 AM
To: Black, David; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

From: Black, David [mailto:[email-masked]]
Sent: Friday, September 19, 2014 7:02 PM
To: Linda Dunbar; Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Linda

Thanks,
--David

Larry,

*         Preserving Policies:

*         Layer 2 Extension

- Larry

Linda

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 22 Sep 2014 15:59:26 +0000

David,

I see what you mean now. How about this updated wording:

In order to achieve the optimal routing, routers in the Wide Area Network have to be  aware which DCBRs can reach the designated VMs. When VMs in a single VN are spread across many different DCBRs, all individual VMs' addresses have to be visible to those routers, which can dramatically increase the number of routes in those routers.

Linda

From: Black, David [mailto:[email-masked]]
Sent: Monday, September 22, 2014 10:46 AM
To: Linda Dunbar; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Linda,

Thanks,
--David

From: Linda Dunbar [mailto:[email-masked]]
Sent: Monday, September 22, 2014 11:32 AM
To: Black, David; Larry Kreeger (kreeger); [email-masked]<mailto:[email-masked]>
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

Linda

Thanks,
--David

Larry,

*       Preserving Policies:

*       Layer 2 Extension

- Larry

Linda

---

From: "Black, David" <[email-masked]>
Date: Mon, 22 Sep 2014 12:23:26 -0400

I don't think it's that simple, as I think this makes an assumption about routing structure and what "optimal" means, specifically that it's not "optimal" for inbound external traffic to a VN at site A to show up at the DCBR at site B.

If so, I disagree, as this ignores the possibility of multi-homing the VNs entire IP address block.  It's quite reasonable for DCBRs at two different data centers to both announce the entire IP address block for the VN and forward inbound traffic that arrives at the wrong place to the other data center.  Under that structure, the external WAN routers still see a single IP address block.

This can result in asymmetric "hot potato" routes where inbound traffic shows up at the site B DCBR, but outbound traffic leaves via the site A DCBR, which may still be "optimal".

Thanks,
--David

From: Linda Dunbar [mailto:[email-masked]]
Sent: Monday, September 22, 2014 11:59 AM
To: Black, David; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

Linda

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

Larry,

*         Preserving Policies:

*         Layer 2 Extension

- Larry

Linda

---

From: Linda Dunbar <[email-masked]>
Date: Mon, 22 Sep 2014 21:40:20 +0000

David,

Indeed we have different interpretations of "Optimal routing".
In "vm-mobility" draft, the "Optimal routing" is to avoid "triangular routing" (or what you stated "hot potato" route).

If a VN is spread across multiple DCBRs and all those DCBRs announce the same IP prefix for the VN, there could be many issues, including:

-          Traffic could go to DCBR A where target is in DCBR B. and DCBR "A" is connected to DCBR "B" via WAN

-           If majority of one VN members are under DCBR "A" and rest are spread across X number of DCBRs. Will DCBR "A" have same weight as DCBR "B", "C", etc?

If all those DCBRs announce individual IPs that are directly attached and those IPs are not segmented well, then all the VMs IP addresses have to be exposed to the WAN. So overlay hides the VMs IP from the core switches in one DC or one  POD, but exposes them to the WAN. There are more routers in the WAN than the number of core switches in one DC/POD.

All those issues haven't been properly addressed by NVO3 problem statement.

Linda

From: Black, David [mailto:[email-masked]]
Sent: Monday, September 22, 2014 11:23 AM
To: Linda Dunbar; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Thanks,
--David

David,

Linda

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

Larry,

*       Preserving Policies:

*       Layer 2 Extension

- Larry

Linda

---

From: "Black, David" <[email-masked]>
Date: Mon, 22 Sep 2014 20:06:25 -0400

Linda,

> In "vm-mobility" draft, the "Optimal routing" is to avoid "triangular routing" (or what you stated "hot potato" route).

I agree on avoiding "triangular routing", but "hot potato" routing is different from "triangular routing" and may be "optimal" depending on how "optimal" is defined.

In general, "hot potato" routing refers to connectivity between two networks in which the upstream network hands off traffic to the downstream network as quickly as it can in both directions, resulting in asymmetric routes.  This can and does work in practice when used and it would be incorrect to characterize it as something that should be avoided in all cases.

> If a VN is spread across multiple DCBRs and all those DCBRs announce the same IP prefix for the VN, there could be many issues, including:

-          > Traffic could go to DCBR A where target is in DCBR B. and DCBR "A" is connected to DCBR "B" via WAN

Note that "via WAN" makes a specific solution assumption. There are cases in which forwarding traffic among the DCBRs is a reasonable thing to do; to the extent that I see an implication that this is never "optimal", I disagree.

-          >  If majority of one VN members are under DCBR "A" and rest are spread across X number of DCBRs. Will DCBR "A" have same weight as DCBR "B", "C", etc?

That depends on routing protocol (e.g., BGP) configuration.

> If all those DCBRs announce individual IPs that are directly attached and those IPs are not segmented well, then all the VMs IP
> addresses have to be exposed to the WAN.

Sure, that may happen, but that description assumes a) that DCBRs exist and b) that they advertise individual IP addresses, both of which are specific solution assumptions.

> All those issues haven't been properly addressed by NVO3 problem statement.

Those issues appear to be solution-specific issues with alternative routing structures for external traffic.  IMHO it was not the job of the problem statement to survey solution approaches and discuss issues with each of them, so this material does not belong there.  As solution material, it would belong in solution drafts, but the vm-mobility-issues draft is supposed to be a general draft that isn't solution-specific.

As long as the draft assumes the existence of a DCBR, it's probably in solution-space.  If the current VM mobility draft is evolved into a design draft that assumes DCBRs, no multi-homing on the WAN and no software network switches in hypervisors (all of which the current draft assumes as I understand it), that could be considered as a solution draft, but it would not be a general (solution-independent) VM issues draft.

Thanks,
--David

From: nvo3 [mailto:[email-masked]] On Behalf Of Linda Dunbar
Sent: Monday, September 22, 2014 5:40 PM
To: Black, David; Larry Kreeger (kreeger); [email-masked]
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

Linda

Thanks,
--David

David,

Linda

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

Larry,

*         Preserving Policies:

*         Layer 2 Extension

- Larry

Linda

---

From: "Black, David" <[email-masked]>
Date: Mon, 22 Sep 2014 20:08:19 -0400

Elaboration on this:

> that could be considered as a solution draft, but it would not be a general (solution-independent) VM issues draft.

"that could be considered" is poorly phrased - "it may be a reasonable" would have been better - sorry :-(.

Thanks,
--David

From: Black, David
Sent: Monday, September 22, 2014 8:06 PM
To: 'Linda Dunbar'; 'Larry Kreeger (kreeger)'; [email-masked]'
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

David,

Linda

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

Larry,

*         Preserving Policies:

*         Layer 2 Extension

- Larry

Linda

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 23 Sep 2014 22:37:13 +0000

David,

Is your "hot potato" routing referring to outbound traffic going through a different DC Gateway than the inbound traffic? Then there will be issues with some middleware boxes that monitor both directions of traffic.

Linda

From: Black, David [mailto:[email-masked]]
Sent: Monday, September 22, 2014 7:06 PM
To: Linda Dunbar; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Linda,

---

From: "Black, David" <[email-masked]>
Date: Tue, 23 Sep 2014 18:49:23 -0400

And since all of this discussion is in solution space, the answer is not to use such middleboxes on the WAN side of the DCBR when asymmetric routing is possible on the WAN side - they work fine on the data center side.  The meta-point is that this approach is viable and excluded by the language you're proposing to add to this draft - that's not appropriate for a general issues draft, but could be appropriate for a solution architecture draft for the specific sort of solution that the vm-issues draft appears to envision.

Thanks,
--David

From: Linda Dunbar [mailto:[email-masked]]
Sent: Tuesday, September 23, 2014 6:37 PM
To: Black, David; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

Linda

Linda,

---

From: Linda Dunbar <[email-masked]>
Date: Tue, 23 Sep 2014 22:53:49 +0000

David,

Questions inserted below:

Linda,

Note that "via WAN" makes a specific solution assumption.
[Linda] it is only to describe the issues associated with  the two DCBRs not directly connected by one single link.  It is much less a "solution" than NVO3 general problem statement describing "Overlay" to hide VMs addresses.

[Linda] The draft is only to describe the issues associated with common IP routing practices (i.e. with or without advertising individual IP addresses ).

> All those issues haven't been properly addressed by NVO3 problem statement.

Those issues appear to be solution-specific issues with alternative routing structures for external traffic.  IMHO it was not the job of the problem statement to survey solution approaches and discuss issues with each of them, so this material does not belong there.

[Linda] Based on your definition, the NVO3 general problem statement draft describing using "overlay" to hide individual VM addresses should also be considered as solution, correct?

As solution material, it would belong in solution drafts, but the vm-mobility-issues draft is supposed to be a general draft that isn't solution-specific.

Thanks,
--David

David,

Linda

Thanks,
--David

David,

Linda

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

Larry,

*       Preserving Policies:

*       Layer 2 Extension

- Larry

Linda

---

From: "Ken Gray (kegray)" <[email-masked]>
Date: Tue, 23 Sep 2014 23:01:44 +0000

IMO, the middle box case is orthogonal to the general purpose of the draft.  The requirements for a service overlay (implied by the middle box) belong in a different workgroup.

From: Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, September 23, 2014 6:37 PM
To: "Black, David" <[email-masked]<mailto:[email-masked]>>, "Larry Kreeger (kreeger)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

Is your “hot potato” routing referring to outbound traffic going through a different DC Gateway than the inbound traffic? Then there will be issues with some middleware boxes that monitor both directions of traffic.

Linda

Linda,

> In “vm-mobility” draft, the “Optimal routing” is to avoid “triangular routing” (or what you stated “hot potato” route).

I agree on avoiding “triangular routing”, but “hot potato” routing is different from “triangular routing” and may be “optimal” depending on how “optimal” is defined.

In general, “hot potato” routing refers to connectivity between two networks in which the upstream network hands off traffic to the downstream network as quickly as it can in both directions, resulting in asymmetric routes.  This can and does work in practice when used and it would be incorrect to characterize it as something that should be avoided in all cases.

---

From: "Black, David" <[email-masked]>
Date: Tue, 23 Sep 2014 19:19:10 -0400

> Those issues appear to be solution-specific issues with alternative routing structures for external traffic.
> IMHO it was not the job of the problem statement to survey solution approaches and discuss issues with each
> of them, so this material does not belong there.

> [Linda] Based on your definition, the NVO3 general problem statement draft describing using "overlay"
> to hide individual VM addresses should also be considered as solution, correct?

Nope - the problem statement draft describes the overall problem and there is both WG and IETF (rough) consensus that it does so.

To the extent that the vm-issues draft makes limiting assumptions that constrain it to not cover significant areas of possible solutions to the problem as described in the problem statement draft, the vm-issues draft looks like a solutions draft.

Thanks,
--David

From: Linda Dunbar [mailto:[email-masked]]
Sent: Tuesday, September 23, 2014 6:54 PM
To: Black, David; Larry Kreeger (kreeger); [email-masked]
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

David,

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

David,

Linda

Linda,

Thanks,
--David

David,

Linda

Thanks,
--David

Larry,

*         Preserving Policies:

*         Layer 2 Extension

- Larry

Linda

---

From: "Larry Kreeger (kreeger)" <[email-masked]>
Date: Wed, 24 Sep 2014 23:56:17 +0000

Hi Linda,

I don't want to respond to proposals for rewording individual sentences at this time because I think it misses the major points of my feedback on the draft which are:

  1.  There is major overlap with the NVO3 Problem Statement. VM mobility is a main motivating use case in the problem statement.
  2.  Content related to inter-datacenter issues are not within scope for NVO3.
  3.  Most if the issues discussed are not specific to VM mobility (they could apply to other types of mobility, such as containers or hot standby switchover of virtual network functions).
  4.  I don't believe the small amount of remaining non-overlapping and in-charter content will effect the framework, architecture, or intra-datacenter solutions.

 - Larry

From: Linda Dunbar <[email-masked]<mailto:[email-masked]>>
Date: Friday, September 19, 2014 3:53 PM
To: Larry Kreeger <[email-masked]<mailto:[email-masked]>>, Benson Schliesser <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: RE: [nvo3] WG Last Call on draft-ietf-nvo3-vm-mobility-issues

Larry,

[Linda] Here are some key points that haven’t been addressed by general NVO3 problem statement:
·       Optimal routing of a VM's outbound traffic
·       Optimal routing of VM's inbound traffic

·       Preserving Policies:

[Linda-2] There could be many policies guarding VMs across different VNs, with some being enforced by Firewall, some enforced by NAT/AntiDDOS/IPS/IDS, etc.  It is less about NVE polices to be maintained when VMs move, it is more along the line of dynamically changing policies associated with the “middleware” boxes attached to NVEs (if those middle boxes are distributed).
·       Maintaining Connectivity in the Presence of VM Mobility

·       Layer 2 Extension

- Larry

[Linda-2] The Section 3.4 of the problem statement didn’t address scenario when there is a bridged network between NVE and Guest OS (either VM or server) and  tenant systems use their own VID (with policies on tenant’s VIDs).

[Linda- 2] The “split-NVE” is assuming that NVE is directly attached to VMs (Guest OS). When there is a bridged network (virtual switches, blade switch attached to NVE on TOR), additional mechanism is needed for the first hop switch to do the translation (before NVE).

Linda