Subject: Overlays and encapsulations (was Re: Engineering discussions )

From: Alia Atlas <[email-masked]>
Date: Sun, 9 Mar 2014 17:33:43 -0400

In the last few years, there seems to be a drive towards overlays and
additional
packet encapsulations.  What problems do you see these as solving?  Is
there a
more focused way to consider the drivers and downsides?

Thoughts?

Alia

On Sun, Mar 9, 2014 at 5:29 PM, heasley <[email-masked]> wrote:

> Sun, Mar 09, 2014 at 11:10:27AM +0000, Dave Crocker:
> > The phrasing of your suggestion presumes that you are currently
> > prevented from having those discussions.  But of course you aren't.
>
> I believe the point is to separate general technical discussion from the
> general everything else discussion, such as the draft-how-not-to-be-a-
> wanker discussion, so that those here just for the technical aspects of
> IETF need not wade through it.  Which I support.
>
>

---

From: Eric Gray <[email-masked]>
Date: Sun, 9 Mar 2014 22:02:20 +0000

Alia,

                I assume that this is a change of subject in part to demonstrate that we
can in fact have a technical discussion on this list.  :)

                But the question is a reasonable example whether that is the case or not.

                IMO, two of the biggest drivers for this work in the IETF are location and
identity separation, and converged networking.

                Just as examples, the work being looked into in NVo3 is one example of
one aspect of the first case (where end-user or server application locations are
being separated from specific network entry points or physical servers accessible
using IP addresses, for example) and both PWE3 and L2VPN are two currently
active examples of the second case (where - for instance - Ethernet traffic is to
be carried over an IP network).

                I think the problems these examples are solving are self-evident.  That
may not be true for other cases.

--
Eric

From: ietf [mailto:[email-masked]] On Behalf Of Alia Atlas
Sent: Sunday, March 09, 2014 5:34 PM
To: heasley
Cc: Dave Crocker; IETF Disgust
Subject: Overlays and encapsulations (was Re: Engineering discussions )

In the last few years, there seems to be a drive towards overlays and additional
packet encapsulations.  What problems do you see these as solving?  Is there a
more focused way to consider the drivers and downsides?

Thoughts?

Alia

On Sun, Mar 9, 2014 at 5:29 PM, heasley <[email-masked]<mailto:[email-masked]>> wrote:

---

From: Brian Trammell <[email-masked]>
Date: Sun, 9 Mar 2014 23:08:02 +0100

> more focused way to consider the drivers and downsides?

Oh. Could we have an easier one to start with? Especially on the Sunday after an IETF? Because my response to this one right now would be to say lots of terribly unkind things about dodgy middleboxes, which is not a very balanced approach to the issue and unlikely to be very helpful in steering the list toward more productive conversation. But let me try.

It's hard to say anything in general about what "overlay x on y" solves in detail (I'll consider encapsulation an implementation detail of overlay for now) for all x and y, other than (1) somebody thought that it would provide a service that y doesn't on its own that (2) they locally thought they needed at the time, and (3) they might actually have been right about (1) and (2). 

The upside is that overlaying allows us to use the abstractions from the higher layer of the overlay when all we have is the lower layer.  

The downside is that x over y might not really be the same as x, because you miss some assumption inherent in either x or y about the corresponding y or x, and you end up breaking something end-to-end. This gets more complicated when years later, after all the bugs are worked out, somebody notices the shiny new x and decides to overlay z on top of it.

I don't see a more focused way to consider this at this (ridiculously high) level of abstraction. A good general considering-the-downsides question might be "how well can I run NNTP, SSH, DNS, World of Warcraft, BitTorrent, Windows file sharing, and WebRTC over it at the same time in an airplane at rush hour on Mars next to my microwave while randomly unplugging stuff?" 

Admittedly this is a rather higher-layer viewpoint. On preview, I suspect you might get a more useful (and more concrete) thread out of Eric's response.

Cheers,

Brian

---

From: Alia Atlas <[email-masked]>
Date: Sun, 9 Mar 2014 18:14:29 -0400

Eric,

On Sun, Mar 9, 2014 at 6:02 PM, Eric Gray <[email-masked]> wrote:

> can in fact have a technical discussion on this list.  J
>
>
>
It's something that struck me as being interesting to discuss and having
general appeal and
benefiting from the perspective of many areas.  :-)

> be carried over an IP network).

Yes, I agree that identity separation is a driver.   I'm not as persuaded
that converged
networking is driving new overlays and encapsulations - just b/c
pseudo-wires have been
around for about 10 years.

> may not be true for other cases.
I'm, of course, more interested in whether there are common drivers and
whether all the
solutions need to be point solutions or are more generalizable.

Alia

>
>
> --
>
> Eric
>
>
>
> *From:* ietf [mailto:[email-masked]] *On Behalf Of *Alia Atlas
> *Sent:* Sunday, March 09, 2014 5:34 PM
> *To:* heasley
> *Cc:* Dave Crocker; IETF Disgust
> *Subject:* Overlays and encapsulations (was Re: Engineering discussions )
>
>
>

---

From: Alia Atlas <[email-masked]>
Date: Sun, 9 Mar 2014 18:32:29 -0400

Hi Brian,

On Sun, Mar 9, 2014 at 6:08 PM, Brian Trammell <[email-masked]> wrote:

> Oh. Could we have an easier one to start with? Especially on the Sunday
> after an IETF? Because my response to this one right now would be to say
> lots of terribly unkind things about dodgy middleboxes, which is not a very
> balanced approach to the issue and unlikely to be very helpful in steering
> the list toward more productive conversation. But let me try.
>

Oh, but this is exactly the type of Sunday-after-IETF discussion when one
can think about big picture wide-ranging ideas without needing to think
about the practical implications for a while!

So - to your point about middleboxes (minus the unkind things), would it be
fair to say:
   a) middleboxes serve to restrict what kind of traffic is perceived to
flow freely to TCP/UDP?
   b) middleboxes impact security by acting as an unknown MITM

we also need to add:
   In most modern networks, it is important to have microflow diversity.
 When IPv6 and flow-labels aren't an option, this frequently defaults to
UDP or TCP 5-tuples (src/dest IP, protocol, src/dest port)

For instance, how many issues would be solved if there were a well-known
meta-data header that an application could use to describe itself to the
network and middle boxes?

It's hard to say anything in general about what "overlay x on y" solves in
> detail (I'll consider encapsulation an implementation detail of overlay for
> now) for all x and y, other than (1) somebody thought that it would provide
> a service that y doesn't on its own that (2) they locally thought they
> needed at the time, and (3) they might actually have been right about (1)
> and (2).
>

Ah - I don't think that encapsulation is an implementation detail of
overlay.   But I'm willing to be persuaded - I think it is more about
carrying additional information; frequently that is done as an overlay
because it isn't possible to carry it otherwise.

> The upside is that overlaying allows us to use the abstractions from the
> higher layer of the overlay when all we have is the lower layer.
>
> The downside is that x over y might not really be the same as x, because
> you miss some assumption inherent in either x or y about the corresponding
> y or x, and you end up breaking something end-to-end. This gets more
> complicated when years later, after all the bugs are worked out, somebody
> notices the shiny new x and decides to overlay z on top of it.
>

Are you thinking of this as something more than foo over IP or over MPLS?
 I, of course, tend to think of it at that level and find it interesting to
learn whether similar issues are appearing at other layers?

>
> I don't see a more focused way to consider this at this (ridiculously
> high) level of abstraction. A good general considering-the-downsides
> question might be "how well can I run NNTP, SSH, DNS, World of Warcraft,
> BitTorrent, Windows file sharing, and WebRTC over it at the same time in an
> airplane at rush hour on Mars next to my microwave while randomly
> unplugging stuff?"
>

Hmm, I was thinking more of things like MTU, implications on increased DPI,
etc.

>
> Admittedly this is a rather higher-layer viewpoint. On preview, I suspect
> you might get a more useful (and more concrete) thread out of Eric's
> response.
>

The point of a general discussion is to get perspectives other than just
those who play at the same layer.  That said, of course, Eric is very
familiar with some of the drivers behind NVO3.

Regards,
Alia

---

From: Spencer Dawkins <[email-masked]>
Date: Sun, 09 Mar 2014 18:29:37 -0500

On 03/09/2014 05:32 PM, Alia Atlas wrote:
> Hi Brian,
>
> On Sun, Mar 9, 2014 at 6:08 PM, Brian Trammell <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     On 09 Mar 2014, at 22:33, Alia Atlas <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>     Oh. Could we have an easier one to start with? Especially on the
>     Sunday after an IETF? Because my response to this one right now
>     would be to say lots of terribly unkind things about dodgy
>     middleboxes, which is not a very balanced approach to the issue
>     and unlikely to be very helpful in steering the list toward more
>     productive conversation. But let me try.
>
>
> Oh, but this is exactly the type of Sunday-after-IETF discussion when 
> one can think about big picture wide-ranging ideas without needing to 
> think about the practical implications for a while!

Speaking as someone who hasn't been at home for the airport 24 hours 
yet, I might have characterized this as "without remembering all the 
reasons why this is hard, because we wouldn't be able to remember much 
of anything before Wednesday-ish" :p

Spencer

---

From: Eric Gray <[email-masked]>
Date: Sun, 9 Mar 2014 23:41:36 +0000

Alia,

                Hmmm.  10 years ago is not (relatively) new?  :)

                I believe that - if we bring in some other examples - converged
networking is still an ongoing process.

                And PWE3 has been affected (in minor ways) by this process.

                See additional responses below...

From: Alia Atlas [mailto:[email-masked]]
Sent: Sunday, March 09, 2014 6:14 PM
To: Eric Gray
Cc: 'IETF' ([email-masked])
Subject: Re: Overlays and encapsulations (was Re: Engineering discussions )
Importance: High

Eric,

On Sun, Mar 9, 2014 at 6:02 PM, Eric Gray <[email-masked]<mailto:[email-masked]>> wrote:
Alia,

It's something that struck me as being interesting to discuss and having general appeal and

Yes, I agree that identity separation is a driver.   I'm not as persuaded that converged
networking is driving new overlays and encapsulations - just b/c pseudo-wires have been

Network convergence is an ongoing process, even now.  Admittedly, the changes
driven by this are mostly small - though arguably that is an artifact of the fact
that changes required to deployed equipment MUST be "mostly small" and this
means that we may choose a solution that involves small changes over one that
might have been a better long-term choice.

                I think the problems these examples are solving are self-evident.  That
may not be true for other cases.
I'm, of course, more interested in whether there are common drivers and whether all the
solutions need to be point solutions or are more generalizable.

The answer to this only seems obvious.

Ideal solutions are completely generalizable for all applications, both those that
exist now, and those that may come.

Then along came reality.

Point solutions have the advantages of being simpler to implement (taken one at
a time, of course) - i.e. - lower-cost, simpler testing associated with incremental
changes and earlier delivery.

>From a strictly business perspective, point-solutions are also easier to justify as
each such solution can be easily tied to a specific customer need (hence revenue
opportunity) and early delivery means a faster ROI - but here I stray from purely
technical discussion.

In addition, each point solution deployed becomes part of the "field" against
which future changes MUST be compared for compatibility reasons - because
(for additional non-technical reasons) we cannot simply wish them away.

Hence generalization - while a goal - is not something we can easily apply to
a problem space (that has been partially solved in deployments) after the fact.

This is the reason why we currently have so many RFCs.  Many (possibly most?)
of them could be considered point solutions.  We could probably come up with
a single generalized RFC to replace them all.  Chances are good it would be lots
simpler - especially if it included function sub-setting, appropriate capability
negotiation, etc. AND assumed compatibility with existing solutions was not a
requirement.

It WOULD be interesting to see how the RFC Editor would deal with the list of
obsoleted RFCs.  Possibly by saying "see Appendix X."

We would then have a generalized solution to all networking problems and no
hope that anyone would use it in our collective lifetimes.

So it is likely that the best approach is to work toward a "point-solution" that
is generalized for the uses about which we are aware and compatible with the
cumulative "point-solutions" we know about.  I think we all agree that for this
to be useful, it MUST also be timely - which means we have to decide at some
point that we "have enough information" and get started.

And then we can look forward to a series of point solutions to address what
was out there, but we were not aware of.  This can be particularly difficult
when nobody is very willing to talk about what is "out there."

Point solutions - depressing as it may seem - are what us less-than-perfect
people have to look forward to.

--
Eric

--
Eric

From: ietf [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Alia Atlas
Sent: Sunday, March 09, 2014 5:34 PM
To: heasley
Cc: Dave Crocker; IETF Disgust
Subject: Overlays and encapsulations (was Re: Engineering discussions )

Thoughts?

Alia

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 09 Mar 2014 19:45:25 -0400

(commenting on two of the points from this exchange)

On 3/9/14, 6:32 PM, Alia Atlas wrote (in response to Brian Trammell:
...
>
> So - to your point about middleboxes (minus the unkind things), would it
> be fair to say:
>     a) middleboxes serve to restrict what kind of traffic is perceived
> to flow freely to TCP/UDP?
>     In most modern networks, it is important to have microflow
> diversity.  When IPv6 and flow-labels aren't an option, this frequently
> defaults to UDP or TCP 5-tuples (src/dest IP, protocol, src/dest port)

Agreed.  I would be very happy if we could even get as far as reliably 
being able to use the Ipv6 flow label for this.  It would allow us to 
finesse a lot of the current disagreement between routing and transport.

>     It's hard to say anything in general about what "overlay x on y"
>     solves in detail (I'll consider encapsulation an implementation
>     detail of overlay for now) for all x and y, other than (1) somebody
>     thought that it would provide a service that y doesn't on its own
>     that (2) they locally thought they needed at the time, and (3) they
>     might actually have been right about (1) and (2).
>
>

While their are encapsulations whose primary purpose is adding 
additional information, most of the cases I can think of (MPLS, LISP, 
GRE, Mobile-IP) are cases where the primary purpose of the encapsualtion 
is to direct the traffic over a path that it might not otherwise chose 
or otherwise be able to traverse.

Yours,
Joel

---

From: Alia Atlas <[email-masked]>
Date: Sun, 9 Mar 2014 20:07:10 -0400

Joel,

> finesse a lot of the current disagreement between routing and transport.

Yes - that did come to mind.  The IPv6 Flow Label does have a Proposed
Standard RFC describing this use.  I (very very optimistically) would view
the use of UDP as a transition mechanism.

> While their are encapsulations whose primary purpose is adding additional
> information, most of the cases I can think of (MPLS, LISP, GRE, Mobile-IP)
> are cases where the primary purpose of the encapsualtion is to direct the
> traffic over a path that it might not otherwise chose or otherwise be able
> to traverse.
>

For MPLS, I think that the services, such as L3VPN, VPLS, pseudo-wires, etc
are hiding data from the underlay and conveying extra info.  Arguably, so
is LISP.

Regards,
Alia

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 09 Mar 2014 20:13:40 -0400

On 3/9/14, 8:07 PM, Alia Atlas wrote:
> Joel,
>
> On Sun, Mar 9, 2014 at 7:45 PM, Joel M. Halpern <[email-masked]
> <mailto:[email-masked]>> wrote:
...
>
>     While there are encapsulations whose primary purpose is adding
>     additional information, most of the cases I can think of (MPLS,
>     LISP, GRE, Mobile-IP) are cases where the primary purpose of the
>     encapsualtion is to direct the traffic over a path that it might not
>     otherwise chose or otherwise be able to traverse.
>
>
> For MPLS, I think that the services, such as L3VPN, VPLS, pseudo-wires,
> etc are hiding data from the underlay and conveying extra info.
>   Arguably, so is LISP.

If you consider "frame format" extra information, then I guess 
Pseudowires carry extra information.  But the primary point is to get 
the packet across the net.

At that point, you end up calling the tunnel endpoint identifier 
metadata instead of delivery information.  Which seems to reduce the 
value of the distinction.

In the case of LISP, as specified, I don;t see that it carries extra 
packet information.  It carries some tunnel maintenance information.
When we add the LCAF in the resolution, we can get an implicit packet 
type.  But even then the primary purpose of the encapsulation is to get 
the packet to the right place on the net.  That is why it separates 
location (the outer destination) from identity (the inner destination).

Yours,
Joel

---

From: Alia Atlas <[email-masked]>
Date: Sun, 9 Mar 2014 20:51:07 -0400

Joel,

(top-posting the response)
I think the meta-data, for lack of a better term, is really the inner
recipient or service.   If we want to think of that as the identity, we
can, but I don't think it's just a locator/identity split.  It feels like
it's also being used for scaling, privacy, and even technology convergence.

Alia

On Sun, Mar 9, 2014 at 8:13 PM, Joel M. Halpern <[email-masked]> wrote:

> If you consider "frame format" extra information, then I guess Pseudowires
> carry extra information.  But the primary point is to get the packet across
> the net.
>
> At that point, you end up calling the tunnel endpoint identifier metadata
> instead of delivery information.  Which seems to reduce the value of the
> distinction.
>
> type.  But even then the primary purpose of the encapsulation is to get the
> packet to the right place on the net.  That is why it separates location
> (the outer destination) from identity (the inner destination).
>
> Yours,
> Joel
>
>
>> Regards,
>> Alia
>>
>>
>>     Yours,
>>     Joel
>>
>>
>>
>

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 09 Mar 2014 21:50:05 -0400

I am having trouble following you.
We have an original (inner) packet I, with IS and ID the source and 
destination information in that packet.  In that packet, that is clearly 
not metadata, either on the original side before tunnel ingress, or on 
the far side after tunnel egress.
We wrap that up in a tunnel wrapper.   There may be some shim 
information in between.
The outer packet O has destination OD (and for some tunnels outer source 
OS).  When the packet gets to OD, OD may help sort out the packet format 
or context (VPN / VRF, LISP{ operating space, ...
But that does not seem to be metadata.

Hence I am having trouble mapping either your comment here that the 
inner destination is metadata or the original comment that the point of 
the tunnel was metadata carriage to the semantic behavior.

Quite possibly I am missing the point of the distinction you are trying 
to draw.

Yours,
Joel

On 3/9/14, 8:51 PM, Alia Atlas wrote:
> can, but I don't think it's just a locator/identity split.  It feels
> like it's also being used for scaling, privacy, and even technology
> convergence.
>
> Alia
>
>
> On Sun, Mar 9, 2014 at 8:13 PM, Joel M. Halpern <[email-masked]
>         On Sun, Mar 9, 2014 at 7:45 PM, Joel M. Halpern
>         <[email-masked] <mailto:[email-masked]>
>         <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>
>     ...
>
>
>     Pseudowires carry extra information.  But the primary point is to
>     get the packet across the net.
>
>     When we add the LCAF in the resolution, we can get an implicit
>     packet type.  But even then the primary purpose of the encapsulation
>     is to get the packet to the right place on the net.  That is why it
>     separates location (the outer destination) from identity (the inner
>     destination).
>
>     Yours,
>     Joel
>
>
>         Regards,
>         Alia
>
>
>              Yours,
>              Joel
>
>
>
>

---

From: Måns Nilsson <[email-masked]>
Date: Mon, 10 Mar 2014 10:15:37 +0100

>    b) middleboxes impact security by acting as an unknown MITM

For 'a' I'd argue that the bar is at "can it be squeezed into
HTTP/1.1?" -- UDP is not to be trusted.

Having said that, yes, both a and b are true. 
 
> network and middle boxes?

None. Yeah, I'm naïve. More to the point is that we should not tamper with the 
success factors in IP. Not knowing what the packets are doing is good in a 
big picture sense. Not all in-transit optimisations are beneficial to
your packets, especially if throwing them on the to-do pile would free
up resources for routing better paying packets.

The nature of coöperating (albeit barely) networks sooner or later leads
to your packets traversing a network with which you have no relations.

-- 
Måns Nilsson     primary/secondary/besserwisser/machina
MN-1334-RIPE                             +46 705 989668
While my BRAINPAN is being refused service in BURGER KING, Jesuit
priests are DATING CAREER DIPLOMATS!!

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Mar 2014 08:46:08 -0700

On 3/9/2014 2:33 PM, Alia Atlas wrote:
> packet encapsulations.

If by "few" you mean over 15, agreed: http://www.isi.edu/xbone

E.g., it was nearly 10 years ago that I gave a tutorial at Infocom on 
virtual and overlays.

 > What problems do you see these as solving?

These have been addressed in a multitude of IETF WGs and RGs over the 
past years - currently L2VPN, L3VPN, NVO3, TRILL, LISP, and the past 
VNRG RG.

The goals are already understood as virtualizing resources (routers, 
subnets, links, etc., for the same benefit an OS adds to computation:

	- abstraction
	- resource sharing
	- shared resource isolation
	- software/service reuse

> Is

Although network virtualization has been around for a long time, even 
actively being developed in the IETF, stepping back to look at the long 
view is starting to gain more traction. I agree that the wheel is being 
reinvented, esp. with a lot of recent "X over UDP" proposals.

FWIW, INTAREA has a tunnels document that Mark Townsley and I started a 
few years ago on which work is resuming. A revision will be out soon. 
That document focuses on IP over encapsulation X, esp where X is also 
(or ultimately) IP, but a lot of the principles may be of more general 
applicability.

Joe

---

From: Joe Hildebrand <[email-masked]>
Date: Wed, 12 Mar 2014 00:15:38 -0600

>network and middle boxes?

No issues would be solved.  Applications would all tag things the same as
the HTTPS traffic to Google to ensure that the network did not prevent the
application developer from making money.

-- 
Joe Hildebrand

---

From: Alia Atlas <[email-masked]>
Date: Wed, 12 Mar 2014 10:37:07 -0400

> application developer from making money.

Sadly, you are probably right.  Some of it might be independent of that,
but this
is why the network tends to not trust what applications say.  It'll be
exciting if the
information that networks do depend on starts being primarily encrypted.

Alia

---

From: [email-masked] (Dale R. Worley)
Date: Wed, 19 Mar 2014 19:19:48 -0400

> network and middle boxes?

Let me argue that this would be a *bad idea*.

One of the characteristics of CCITT networking was that the endpoint
had to declare the nature of each data stream to the network, which
then allowed the network to charge "what it is worth to you" for every
data stream.

One of the characteristics of IETF networking is that it takes a great
deal of work for the network to discern the nature (or even the
identity) of each data stream, resulting in rather spastic network
performance, and network capacity being purchased inexpensively, in
bulk.  In response, network applications have had to go through a
great deal of development to allow them to work despite spastic
network service.  (I work in VoIP, where this is a practical problem.)

Because the endpoints get to choose which sort of networking to
purchase, the economic driver of Internet adoption was the absence of
clear flow labeling...

Dale

---

From: Alia Atlas <[email-masked]>
Date: Fri, 21 Mar 2014 11:34:03 -0400

Dale,

I certainly wouldn't want to go to the "identify the application" stage
with the
issues that you've mentioned.  I'll try to ignore the DPI and encapsulating
of
applications in UDP or HTTP to get through firewalls and so on.

What I was thinking of is more aspects like identifying a micro-flow that
shouldn't
be reordered; if an application doesn't need this treatment, that's
different.

Regards,
Alia

---

From: Alia Atlas <[email-masked]>
Date: Fri, 21 Mar 2014 11:36:16 -0400

Joel,

Metadata wasn't the best term.   Sorry about that.

The question is whether we can categorize the inner information that needs
to be carried
and see if there are similarities.

This conversation does seem to have tapered off, which is fine.   Maybe some
folks are thinking about the various encaps that we are seeing.

Regards,
Alia

On Sun, Mar 9, 2014 at 9:50 PM, Joel M. Halpern <[email-masked]> wrote:

> not metadata, either on the original side before tunnel ingress, or on the
> far side after tunnel egress.
> We wrap that up in a tunnel wrapper.   There may be some shim information
> in between.
> OS).  When the packet gets to OD, OD may help sort out the packet format or
> context (VPN / VRF, LISP{ operating space, ...
> Hence I am having trouble mapping either your comment here that the inner
> destination is metadata or the original comment that the point of the
> tunnel was metadata carriage to the semantic behavior.
>
> Quite possibly I am missing the point of the distinction you are trying to
> draw.
>
> Yours,
> Joel
>
>