Subject: [Int-area] Some thoughts on draft-yong-intarea-inter-sites-over-tunnels

From: Brian E Carpenter <[email-masked]>
Date: Tue, 29 Nov 2016 16:59:49 +1300

Hi,

My first question is not whether it's a good idea to build an IP VPN over
IP tunnels, because I'm sure it is. It is more whether we actually need
a BCP describing how to do it, rather than just, say, open-source code
for a VRF instance that does this.

I think that question is definitely worth exploring, and is probably a big
enough question to deserve a BOF (not necessarily a WG-forming BOF). But
that needs to be based on a more problem-oriented and analytic draft, I think.
It definitely needs expertise from the Transport Area as well as the Internet
Area, to get the congestion management right.

For the moment, I am quite unable to judge whether the proposal in this draft
to use GRE-in-UDP or GUE is the best answer. I also don't really understand
the security model. There is some discussion of IPsec tunnels and RFC3884.
If we use IPsec tunnels, why would we need DTLS? For that matter, if we use
TLS tunnels, why would we need DTLS?

I'm also quite unable to know how to position this proposal compared to
https://tools.ietf.org/html/draft-templin-aerolink which has been
in development for several years. They seem to tackle some of the same
problems.

Regards
   Brian Carpenter

---

From: Joe Touch <[email-masked]>
Date: Mon, 28 Nov 2016 20:38:26 -0800

Hi, Brian,

> for a VRF instance that does this.
+1
> Area, to get the congestion management right.
-1

We already have RFC6040. This isn't a transport problem (if it is, it
has been done incorrectly - see below).
> to use GRE-in-UDP or GUE is the best answer.
There can be no single answer to that question. Like regular links,
tunnels (virtual links) vary with their environment, and should.
>  I also don't really understand
TLS is a very bad idea. We should never try to tunnel IP over TCP.

DTLS might be available where IPsec isn't.

> in development for several years. They seem to tackle some of the same
+1
>
> Regards
>    Brian Carpenter
>
>
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 30 Nov 2016 08:43:02 +1300

Joe,

On 29/11/2016 17:38, Joe Touch wrote:
> We already have RFC6040.

Doesn't that only apply with ECN-capable end points?

> This isn't a transport problem (if it is, it

No, but it might induce interesting transport problems if the tunnel behaves
other than as a piece of wire. RFC6077 seems to identify a number of open
problems in this area, but you certainly know more about this than I do.

> TLS is a very bad idea. We should never try to tunnel IP over TCP.

I agree it's a terrible idea, but pragmatically situations can arrive where it's
the only real option.

> DTLS might be available where IPsec isn't.

If that is the case, it needs to be explained in more detail in the draft.
Anyway it needs to be a clear choice: IPsec or DTLS, but not both.
(This is a point that has also come up in the Anima WG, as it happens.)

   Brian

---

From: Joe Touch <[email-masked]>
Date: Tue, 29 Nov 2016 12:04:39 -0800

Hi Brian,

> Doesn't that only apply with ECN-capable end points?

Yes. Where those aren't available, IP over IP tunnels (or IP over X over
IP) provide the same "congestion control" that all links do (i.e., none).

> problems in this area, but you certainly know more about this than I do.

The place to go here if you want bandwidth guarantees or rate limits
within a tunnel is PWE, but again that's not transport. I'm not saying
there should be no congestion control, only that it's a different animal
than at the transport layer.

> I agree it's a terrible idea, but pragmatically situations can arrive where it's

Sure, but it's bad enough to try to avoid if there are other alternatives.

> If that is the case, it needs to be explained in more detail in the draft.

Agreed.

> (This is a point that has also come up in the Anima WG, as it happens.)

Agreed as well.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 29 Nov 2016 12:56:22 -0800

> Sure, but it's bad enough to try to avoid if there are other alternatives.
This is why we like DTLS as opposed to IPsec, to the network this just
looks like UDP which presumably is more palatable to some network
devices. DTLS can also be implemented by an application (e.g. in
userspace). GUE with DTLS is nice because we can have both an
extensible encapsulation header that might read readable by the
network as well as a secured payload.

>> (This is a point that has also come up in the Anima WG, as it happens.)
I'm not sure I understand why this is a binary choice. The draft isn't
requiring any particular tunnel encapsulation and so security isn't be
required (might depend on encapsulation). Or does this mean that we
want to avoid using DTLS or TLS simultaneously on the same packets?
(which does some like a bad idea...)

Thanks,
Tom

---

From: Lucy yong <[email-masked]>
Date: Tue, 29 Nov 2016 21:59:45 +0000

Hi Brian, Joe, and et al,

Thank you for the questions and comments. Please see inline below.

-----Original Message-----
From: Int-area [mailto:[email-masked]] On Behalf Of Brian E Carpenter
Sent: Monday, November 28, 2016 10:00 PM
To: [email-masked]
Subject: [Int-area] Some thoughts on draft-yong-intarea-inter-sites-over-tunnels

Hi,

My first question is not whether it's a good idea to build an IP VPN over IP tunnels, because I'm sure it is. It is more whether we actually need a BCP describing how to do it, rather than just, say, open-source code for a VRF instance that does this.
[Lucy] We certainly like to discuss what IETF needs to do here. Now more and more vendors make such products to enable site-interconnection over IP tunnels (note: site traffic may be non-IP), and more and more enterprise companies use such product for site-interconnection. Some products may not even meet Internet requirement or UDP application requirement, which is not good, having a standard way to implement is valuable for the Internet. In addition, in future, such products can be commoditized and be used in very large scale and/or between two trusted parties, then standard solution is necessary in this space to avoid interworking issues.

I think that question is definitely worth exploring, and is probably a big enough question to deserve a BOF (not necessarily a WG-forming BOF). But that needs to be based on a more problem-oriented and analytic draft, I think.
It definitely needs expertise from the Transport Area as well as the Internet Area, to get the congestion management right.
[Lucy] Thank you for the suggestion. We like to solicit interest and suggestion on what is the proper way to develop this application in IETF. Before IETF98, should we discuss this in intarea mailing list or have a new mailing list to discuss this? FYI: here is another draft related to this application.
 https://datatracker.ietf.org/doc/draft-dunbar-opsawg-private-networks-over-thin-cpe/

For the moment, I am quite unable to judge whether the proposal in this draft to use GRE-in-UDP or GUE is the best answer. I also don't really understand the security model. There is some discussion of IPsec tunnels and RFC3884.
If we use IPsec tunnels, why would we need DTLS? For that matter, if we use TLS tunnels, why would we need DTLS?
[Lucy] I agree Joe and Tom's answer.

I'm also quite unable to know how to position this proposal compared to https://tools.ietf.org/html/draft-templin-aerolink which has been in development for several years. They seem to tackle some of the same problems.
[Lucy] In fact, Fred informed us about draft-templin-aerolink before Seoul meeting and we had some offline discussions about the two. We agreed that both Fred's draft and Yong's draft specify a solution for site-interconnection by IP tunnels; however Fred's draft solves more complex case than Yong's draft. Yong's draft assumes that each enterprise site is at a location and not often moves; Fred's draft assume that each network site can dynamically moves (e.g. airplane is a site), so site location address, i.e. tunnel EP, may change frequently on Internet; thus Fred's solution uses AERO protocol [RFC6706] to dynamically get tunnel EP address and establish tunnels. The tunnel EP in Yong's case is static. We kindly agreed to work together in this subject, Fred also want to submit Fred's draft to intarea WG draft. We certainly like to hear the intarea expert suggestion on the work development.

Thanks,
Lucy   

Regards
   Brian Carpenter

---

From: Joe Touch <[email-masked]>
Date: Tue, 29 Nov 2016 17:31:47 -0800

Hi, Tom,

On 11/29/2016 12:56 PM, Tom Herbert wrote:
> devices. 
Those devices might support "network connectivity", but they are NOT the
Internet. The Internet is supposed to be agnostic to IP packet contents.

Further, IPsec over UDP also exists (RFC3948).

> DTLS can also be implemented by an application (e.g. in
> userspace). 

So can IPsec over UDP.

> GUE with DTLS is nice because we can have both an
There are many tunnels that have extensible headers. Additionally, IPsec
can protect the tunnel header whereas DTLS cannot.

However, I agree that there are many solutions.

> (which does some like a bad idea...)
I think Brian was saying there might be little benefit to using both
IPsec and DTLS on the same packet, but I also think we're all in
agreement here.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 29 Nov 2016 17:40:05 -0800

Lucy,

I think the key question is "what is this draft contributing".

IP tunnel specs exist. Creating a network out of IP tunnels is no more a
mystery than creating an IP network out of links. Using IP networking to
interconnect sites is also not new or difficult.

I don't think we need a "standard way" to create a network out of IP
tunnels any more than we need a standard way of creating IP networks
from links. The whole point of the Internet is the ability to "mix and
match" links (tunnels being just links).

The only missing piece seems to be network management - i.e., a way to
configure these tunnels as part of a network. But that ultimately is
just a YANG model for tunnels (i.e., the equivalent of an SNMP MIB). But
that's not all that difficult nor does it require a new WG.

The rest, AFAICT, is in the hands of network management application
designers. We built such an application (the X-Bone Overlay Manager),
using our own version of a YANG model/language to configure the
components - which, FWIW, are more complex than just tunnels (you need
to configure virtual routers too; tunnels are just interfaces on virtual
hosts/routers).

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 30 Nov 2016 08:08:16 -0800

Joe,

Sadly, what the Internet is supposed to be and what it is have
diverged. As much as we wish otherwise, there are still many devices
and networks on the Internet that are not agnostic to IP packet
contents. See draft-gont-v6ops-ipv6-ehs-in-real-world-02,
draft-byrne-opsec-udp-advisory-00, the difficulties of deploying SCTP,
or for that matter how any stateful firewall or stateful NAT works.
Actually matters have been even worse, there are devices that are not
even agnostic to _transport_ layer payloads-- when we turned up TLS
several radio network operators complained that they could no longer
parse HTTP which they deemed was important to their operations. The
net result of all this is protocol ossification on the Internet. When
we deploy protocols on the Internet we have to consider what the real
effects are.

>> DTLS can also be implemented by an application (e.g. in
Yes, but there is no analogue for IPsec over TCP (and should not be!).

> IPsec and DTLS on the same packet, but I also think we're all in
One open question we have concerning tunnels in this area is how to
know when we need to apply security at lower layers. For instance, if
an application has already encrypted payload using TLS then we
shouldn't need to encrypt over a tunnel, doing would be mostly
redundant and result in wasted resources and increased latency. If no
higher layer has encrypted the packet then we may want to do it for
tunneling over Internet to secure user's data. We expect the trend to
be more applications will implement their own security, however there
are still a significant number that benefit from tunnels with
security. The problem is there's no obvious way to detect that a
packet is already encrypted at tunnel ingress. Trying to deduce by
port number is too weak of a signal for security. TCP ENO has promise
but that would require connection tracking and more of intermediate
devices parsing TCP options...

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Nov 2016 08:25:39 -0800

Hi, Tom,

On 11/30/2016 8:08 AM, Tom Herbert wrote:
> contents. 

And those issued need to be addressed and called out.

FWIW, this is my definition of "being on the Internet":
http://www.isi.edu/touch/internet-rights/

> See draft-gont-v6ops-ipv6-ehs-in-real-world-02,
We really should be pushing back on the notion that "what vendors deploy
is what the Internet is". Vendors' interests focus on increased profit
for decreased capability, which drives us towards the lowest common
denominator.

> we deploy protocols on the Internet we have to consider what the real
The ossification is created by those who restrict protocols. It is
pointless to try to endorse that ossification; all we end up with is the
same problem at another layer.

> Yes, but there is no analogue for IPsec over TCP (and should not be!).
This contradicts your argument above for two reasons:
1) it exists:
http://www.cisco.com/c/en/us/support/docs/security/vpn-3000-series-concentrators/14370-vpn3k-ipsec-tcp.html

2) it is an example of "we had to do it this way to traverse TCP-only NAT"

So if you agree this is incorrect, then so should other methods that do
not provide a *real* Internet.
> ...
> know when we need to apply security at lower layers.
I agree, but I would note that you can remove "tunnels in this area".
But that's because the role of security at these layers is very poorly
understood.

>  For instance, if
> redundant and result in wasted resources and increased latency. 
TLS protects the application data, but not the headers (i.e., the
"protocol control plane").

I.e., you might still want to use IPsec or TCP-AO to protect TCP from
attacks.

> If no
> tunneling over Internet to secure user's data.
The only way a user can know their data is protected is to setup the
protection. Otherwise, it has to assume the data is not protected. This
is the whole point behind the E2E argument - you cannot delegate some
responsibilities.

>  We expect the trend to
And you should NEVER do any of these things anyway. You protect the
tunnel because you want the tunnel and its control to be protected.

I agree - it's not useful to try to determine whether security is active
at other layers, but it shouldn't be the deciding factor IMO either.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 30 Nov 2016 17:35:23 +0000

> I'm also quite unable to know how to position this proposal compared to https://tools.ietf.org/html/draft-templin-aerolink which has
> been in development for several years. They seem to tackle some of the same problems.
> [Lucy] In fact, Fred informed us about draft-templin-aerolink before Seoul meeting and we had some offline discussions about the
> two. We agreed that both Fred's draft and Yong's draft specify a solution for site-interconnection by IP tunnels; however Fred's draft
> solves more complex case than Yong's draft. Yong's draft assumes that each enterprise site is at a location and not often moves; Fred's
> draft assume that each network site can dynamically moves (e.g. airplane is a site), so site location address, i.e. tunnel EP, may change
> frequently on Internet; thus Fred's solution uses AERO protocol [RFC6706] to dynamically get tunnel EP address and establish tunnels.
> The tunnel EP in Yong's case is static. We kindly agreed to work together in this subject, Fred also want to submit Fred's draft to intarea
> WG draft. We certainly like to hear the intarea expert suggestion on the work development.

How would people feel about seeing draft-templin-aerolink become an intarea
document?

Thanks - Fred
[email-masked]

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Nov 2016 10:49:16 -0800

Hi, Fred, et al.,

> How would people feel about seeing draft-templin-aerolink become an intarea

In one sense, every IP over X tunnel proposal could be an INTAREA doc,
but the bigger question is "what problem is being solved" that is
specific to INTAREA.

Absent that goal statement, I'm not sure we should be adopting anything...

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 30 Nov 2016 19:25:13 +0000

Hi Joe,

Very basically, the problem being solved is an overlay network over the
Internet for inter-site tunneling between sites that may be either fixed
or mobile.

Is this what you are looking for, or do you want specific use cases?

Thanks - Fred

> From: Joe Touch [mailto:[email-masked]]
> Sent: Wednesday, November 30, 2016 10:49 AM
> To: Templin, Fred L <[email-masked]>; Lucy yong <[email-masked]>; Brian E Carpenter
> <[email-masked]>; [email-masked]
> Subject: Re: [Int-area] Some thoughts on draft-yong-intarea-inter-sites-over-tunnels
> 
> Hi, Fred, et al.,
> 
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Nov 2016 11:32:38 -0800

> Internet for inter-site tunneling between sites that may be either fixed

That's the same problem solved by every tunnel, isn't it?

There is nothing unique about "inter-site" tunnels - all tunnels that
carry IP qualify.

Endpoint mobility might be the new part, but in that case I'm not seeing
how this is a new problem (vs. any other sort of endpoint mobility,
e.g., use HIP, mobile IP, etc.). If that's the new part, it seems like
it belongs in a mobility WG, not here....

> Is this what you are looking for, or do you want specific use cases?

That is close to what I was asking, but I'm not sure I see the answer yet...
Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 30 Nov 2016 19:41:17 +0000

Hi Joe,

> Sent: Wednesday, November 30, 2016 11:33 AM

It is about more than just mobility - security, traffic engineering, routing
control, VPN, and other aspects. What really makes AERO intarea-like
is the fact that it is modeled as an NBMA link-layer for IP and can be
considered an "IP-over-foo" document.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Nov 2016 12:03:38 -0800

> considered an "IP-over-foo" document.
You're talking about the features, not the problem. Every tunnel that
includes encryption supports security and can be used for a VPN. Routing
isn't part of a tunnel (a tunnel ends in interfaces); routing happens in
the overlay *to* that tunnel interface. And traffic engineering is easy
in a tunnel *if* it's supported in the base network over which the
tunnel operates, and impossible otherwise.

I'm not claiming this wouldn't be useful. I'm saying that we need to
know what problem it solves to know where to home it.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 30 Nov 2016 20:51:01 +0000

Hi Joe,

> Sent: Wednesday, November 30, 2016 12:04 PM
> the overlay *to* that tunnel interface.

I didn't mention route optimization. With AERO, route optimization is what happens
when the tunnel ingress switches from an egress that is on a suboptimal path to a
different egress that is on a better path. Due to the link nature of the NBMA overlay,
that switching is accomplished through the use of IPv6 ND Redirect messages the
same as would occur on a physical link (and in the same spirit as published in RFC6706).
That is why I ended up agreeing with you that fully embracing fragmentation is the
only way to truly handle tunnel MTU, because without fragmentation an MTU that
worked over the suboptimal path might fail over the new path once route
optimization is employed.

> And traffic engineering is easy

Traffic engineering as in allowing the Client to select both the outbound underlying
interface for outbound traffic and the inbound underlying interface for inbound
traffic. So, a device that has both cellular and WiFi can send and receive packets
with different TOS markings over both interfaces simultaneously (e.g., TOS '1' goes
out over cellular, TOS '2' goes out over WiFI, etc.) and respectively for the
inbound direction.

> know what problem it solves to know where to home it.

I have identified two very important use cases relating to aviation. So, the fortuitous
selection of the AERO acronym now seems quite appropriate. We are also using it for
mobile VPN management for corporate enterprise mobile device users (cellphones
and tablets), and we are planning to release source code soon.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Nov 2016 14:31:33 -0800

Hi, Fred (et al.),

On 11/30/2016 12:51 PM, Templin, Fred L wrote:
> different egress that is on a better path. 
That's internal to the tunnel itself, related to the fact that your
tunnel is multipoint and treats all egresses as valid.

That's another feature, but again it's hard to see where that would be
discussed. It's closest in spirit to egress determination in BGP or
LISP, and has no clear relation (AFAICT) to INTAREA (expertise or
purview). However...

> Due to the link nature of the NBMA overlay,
I don't think I would call that routing either when it happens in a
regular network or inside an NMBA tunnel.

> traffic. 
A tunnel ends at interfaces. Client determination of interfaces happens
separate from a tunnel, IMO.

> So, a device that has both cellular and WiFi can send and receive packets

That's the job of interface selection in the host, which is already
specified in RFC1122 and extensions. Are you suggesting a change to
that? If so, it seems like it's completely orthogonal to the use of
tunnels as interfaces.

> I have identified two very important use cases relating to aviation. 
You've defined a customer, IMO - which is great, but short of taking
this to an aerospace forum, it doesn't really help me understand where
in the IETF it should be discussed.

> So, the fortuitous

That's all good news, but I'm still left not quite understanding where
in the IETF this should (or could) be handled. We don't really have a WG
for creating tunnels and I'm not sure INTAREA should serve as  a default.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 30 Nov 2016 23:17:50 +0000

Hi Joe,

> Sent: Wednesday, November 30, 2016 2:32 PM

It is the same as the way the Redirect function works on physical links, except
that the Redirect applies to an entire network prefix and not just a singleton
address. This has already been published in RFC6706, and what we are now
specifying is essentially RFC6706(bis).

> regular network or inside an NMBA tunnel.

It is the moral equivalent of the Redirect function on physical links.

> that? If so, it seems like it's completely orthogonal to the use of

The AERO interface is an NBMA interface where the ingress sees all potential
egresses as potential "neighbors". An AERO interface that is configured over
multiple underlying interfaces would appear to have multiple link-layer
addresses. Each link-layer address can have a different set of TOS-based
priorities so that the AERO interface can specify to its neighbors as to
which link-layer address to use for which class of traffic.

>From an IPv6 neighbor discovery standpoint, this is realized by allowing IPv6 ND
messages to include multiple S/TLLAOs - one for each underlying interface. Each
S/TLLAO has a set of TOS-based priorities allowing neighbors to select the best
underlying interface for the given packet TOS. This is described in Section 3.5
of the document. I can demonstrate how this works in actual running code
over a webex if you would like.

> in the IETF it should be discussed.

It is being proposed to the International Civil Aviation Organization (ICAO)
where standards are being established for the future civil aviation data
communications architecture.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 30 Nov 2016 16:51:48 -0800

> specifying is essentially RFC6706(bis).

OK, then the bis doc might fall within INTAREA (though I would think it
could be very hazardous to allow a redirect to apply to an entire
prefix! - isn't that an attack vector?)

> It is the moral equivalent of the Redirect function on physical links.
It's the exact same thing. I agree it's useful, just not that it's routing.

> ....
> egresses as potential "neighbors".
Isn't that true for all multipoint links?

> An AERO interface that is configured over
That just means you have a tunnel model that allows one instance to
represent multiple tunnels (or tunnel interfaces). That's fine, but
seems local to the definition of the tunnel.

> over a webex if you would like.
I understand the principle, but you're describing a feature of the
tunnel mechanism that seems based largely on applying existing IP
capability inside the tunnel.

> communications architecture.

Sure, so it sounds like a standard there and a de-facto one within the
IETF (e.g., individual informational). I haven't seen a case yet as to
why INTAREA should adopt it per se...

Joe