Subject: Re: [nvo3] Fwd: New Version Notification for draft-gu-statemigration-framework-02.txt

From: "Ivan Pepelnjak" <[email-masked]>
Date: Sat, 1 Sep 2012 16:36:32 +0200

In general, I find the draft way too vague (particularly sections 6 & 7, so I made no comments on them). I know state migration is a very difficult problem, so maybe you should limit yourself to a smaller and manageable subset, for example VM migration.

Major remarks
=============
4.1 - What is state
===================
* The flow 5-tuple (section 4.1) is the _flow classifier_ (and even the 5-tuple would often include context like VRF or VLAN). There are usually _actions_ associated with the flow (FW: permit, log ..., LB,NAT: source or destination rewrite). It might make sense to import ideas from OpenFlow documents.

* Middlebox state could be associated with interface, but also with VLAN, VRF ... Interface is just an example of the context I've mentioned in the previous bullet.

* I would love to see an example of middleboxes preserving flow state across HTTP-in-TCP sessions as an efficiency mechanism. Middleboxes might retain the flow state after TCP session termination to process residual packets (in case RST or FIN got lost after traversing the middlebox).

4.2 - State versus Policy
=========================
I found the whole section confusing. I always thought the policies were (in general) hierarchical set of classifiers and actions, and when a flow state is created, a list of policy actions gets attached to the flow.

4.3 - Instantiating state
=========================
I don't understand what the mechanisms for policy creation (and consumer equipment in particular) have to do with this draft. Keep it simple and state that the mechanisms used to create policies are out-of-scope.

Also, there's no "hybrid mechanism". Either the middlebox supports a state signaling protocol (ex: RSVP) or not. STUN is just a necessary kludge trying to guess properties of adjacent NAT devices and using them to its advantage.

5.1 - Addresses of moving endpoints
===================================
There are only two major scenarios: either the network address stays unchanged or it doesn't. If it does, we're done, if it doesn't then either the endpoint support IP(v6)? Mobility or not. If not, all the sessions are lost and we're done. Please keep the wording precise.

5.2.1 - VM migration
====================
Please remove all the marketing & hype. Running VM is moved, its IP addresses and sessions are preserved. End of story.

5.2.3 - Failover & HA
=====================
Every single VM failover/HA solution I've seen (apart from VMware's FT) restarts the service or VM after the failure, losing all sessions (and all the state this draft is discussing). Maybe you should mention that instead of more marketing fluff.

7.1 - Recognizing when an endpoint has moved
============================================
There are no "unplanned" moves. Even after a failure, a software component (cluster manager, HA manager ...) must restart the failed services/VMs.

Minor details:
==============

1 - Introduction
================
"When an endpoint changes its point of attachment to a network, it retains its IP address, and the standard 5-tuple used to describe a flow ..."

That is true in VM mobility scenario, but not in a generic case. How about

"When an endpoint changes its point of attachment to a network while retaining its IP address, the standard 5-tuple used to describe a flow ..."

You might also consider slightly rewording the definition of "move" in Terminology

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Melinda Shore
> Sent: Friday, August 31, 2012 7:58 PM
> To: [email-masked]; Yingjie Gu(yingjie)
> Subject: [nvo3] Fwd: New Version Notification for draft-gu-statemigration-
> framework-02.txt
> 
> We've published a revision of our draft providing a framework for
> migrating flow-associated state between middleboxes.  One use case is VM
> migration.
> 
> Melinda
> 
> 
> -------- Original Message --------
> Subject: New Version Notification for
> draft-gu-statemigration-framework-02.txt
> Date: Thu, 30 Aug 2012 21:37:41 -0700
> From: [email-masked]
> To: [email-masked]
> CC: [email-masked], [email-masked]
> 
> 
> A new version of I-D, draft-gu-statemigration-framework-02.txt
> has been successfully submitted by Melinda Shore and posted to the
> IETF repository.
> 
> Filename:	 draft-gu-statemigration-framework
> Revision:	 02
> Title:		 A Framework and Problem Statement for Flow-associated
> Middlebox
> State Migration
> Creation date:	 2012-08-27
> WG ID:		 Individual Submission
> Number of pages: 22
> URL:
> http://www.ietf.org/internet-drafts/draft-gu-statemigration-framework-
> 02.txt
> Status:
> http://datatracker.ietf.org/doc/draft-gu-statemigration-framework
> Htmlized:
> http://tools.ietf.org/html/draft-gu-statemigration-framework-02
> Diff:
> http://www.ietf.org/rfcdiff?url2=draft-gu-statemigration-framework-02
> 
> Abstract:
>     This document presents an initial framework and discussion of the
>     problem of transferring middlebox (for example, firewall or NAT)
>     flow-coupled state from one middlebox to another while the flow is
>     still active.  This has most recently come up in the context of
>     virtual machine (VM) migration between hypervisors, but it is a
>     problem that has appeared in other situations, as well.  We present
>     some of the parameters of the problem, define some language for
>     discussing the problem, and begin to identify a path forward for
>     addressing it.
> 
> 
> 
> 
> 
> The IETF Secretariat
> 
> 
> 
> _______________________________________________
> nvo3 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nvo3

---

From: Melinda Shore <[email-masked]>
Date: Mon, 03 Sep 2012 16:49:29 -0800

Hi, Ivan:

Thanks for your thoughtful review.  We'll be incorporating
many of your suggestions into the next revision.

On 9/1/12 6:36 AM, Ivan Pepelnjak wrote:
> In general, I find the draft way too vague (particularly sections 6 &
> 7, so I made no comments on them). I know state migration is a very
> difficult problem, so maybe you should limit yourself to a smaller
> and manageable subset, for example VM migration.

Scoping this (and most IETF problems) is difficult, and it's
something we've gone around and around on.  I expect that if
the SOCKS authors hadn't been so intransigent about NAT, much
of the firewall discussions we've been having for the last, oh,
15 years or so would not have been necessary, but scoping is
hard and they didn't want to bite off too much.  Anyway, I
think your point is valid but I'm not sure yet that we know
exactly how to scope the problem in a way that will be small
enough to solve on the one hand and not so narrow as not to
be useful on the other.  I'm hoping that we can have a BOF
in Atlanta to be able to have this discussion in real-time.

> Major remarks ============= 4.1 - What is state =================== *
> The flow 5-tuple (section 4.1) is the _flow classifier_ (and even the
> 5-tuple would often include context like VRF or VLAN). There are
> usually _actions_ associated with the flow (FW: permit, log ...,
> LB,NAT: source or destination rewrite). It might make sense to import
> ideas from OpenFlow documents.

I think at this point we're trying to focus on abstractions
that will be applicable across a range of firewalls.  I had
hoped that the notion of the *action* being migrated along
with the 5-tuple (or other descriptive information) would be
implicit in the discussion, but you're right that it should
be explicit.

> 4.2 - State versus Policy ========================= I found the whole
> section confusing. I always thought the policies were (in general)
> hierarchical set of classifiers and actions, and when a flow state is
> created, a list of policy actions gets attached to the flow.

It certainly can be, but this discussion was at a higher level.
You're describing an instance of "policy."  Unfortunately there's
no clear definition of state that I could find in existing IETF
documents, and "policy" tends to be associated with specific
policies in particular protocols.

> 4.3 - Instantiating state ========================= I don't
> understand what the mechanisms for policy creation (and consumer
> equipment in particular) have to do with this draft. Keep it simple
> and state that the mechanisms used to create policies are
> out-of-scope.

Well, they're not, really, in that in some cases participants
in a session/flow may not know that there's middlebox state
associated with them.  If they use some mechanism to explicitly
instantiate the state, they will.  This has considerable
impact on where to place an agent that will be responsible
for managing (for lack of a better term) a state migration.

> Also, there's no "hybrid mechanism". Either the middlebox supports a
> state signaling protocol (ex: RSVP) or not. STUN is just a necessary
> kludge trying to guess properties of adjacent NAT devices and using
> them to its advantage.

No, this is not correct.  STUN may be used to create the NAT table
mapping, since in a typical case (VoIP, say) the endpoint or its
call control server cannot know what media addresses to embed in its
signaling messages until it's somehow gotten that address.  No actual
media traffic have flowed yet.  Even in this case STUN does not
communicate directly with a NAT, but works by side-effect.

We'll be getting a revised draft out prior to the next meeting, and
as I mentioned I hope that we'll be able to have a (short) BOF to
try to get a handle on some of these issues.

Melinda

---

From: Ivan Pepelnjak <[email-masked]>
Date: Tue, 04 Sep 2012 19:01:00 +0200

BTW, Midokura has solved the state migration problem in their MidoNet 
solution with a central flow state database that devices refer to when 
verifying return traffic flows.

Ivan