Subject: Re: I-D Action: draft-smith-6man-in-flight-eh-insertion-harmful-00.txt

From: Brian E Carpenter <[email-masked]>
Date: Sun, 13 Oct 2019 11:15:08 +1300

Hi Mark,

As you have no doubt seen, the active proposal for EH insertion (draft-voyer-) states that the source and destination addresses of packets subject to insertion both lie within a controlled domain. Now while draft-carpenter-limited-domains calls for secure mechansims for determining whether a node is inside or on the border of a domain, it seems to me that address scope (and therefore routing scope) is today the state of the art for that. Of course you're correct that configured scope is subject to human error, but that already has very serious consequences, especially BGP misconfigurations. I think that draft-voyer-6man-extension-header-insertion-07 (not the earlier versions) is resistant to your arguments. It certainly does not describe the use case you give at https://tools.ietf.org/html/draft-smith-6man-in-flight-eh-insertion-harmful-00#section-3 .

EH insertion for packets that traverse the open Internet is certainly harmful and violates RFC 8200. draft-voyer-6man-extension-header-insertion-07 does not propose that.

Regards
   Brian Carpenter

On 09-Oct-19 18:30, [email-masked] wrote:
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
>         Title           : In-Flight IPv6 Extension Header Insertion Considered Harmful
>         Author          : Mark Smith
> 	Filename        : draft-smith-6man-in-flight-eh-insertion-harmful-00.txt
> 	Pages           : 10
> 	Date            : 2019-10-08
> 
> Abstract:
>    In the past few years, as well as currently, there have and are a
>    number of proposals to insert IPv6 Extension Headers into existing
>    IPv6 packets while in flight.  This contradicts explicit prohibition
>    of this type of IPv6 packet proccessing in the IPv6 standard.  This
>    memo describes the possible failures that can occur with EH
>    insertion, the harm they can cause, and the existing model that is
>    and should continue to be used to add new information to an existing
>    IPv6 and other packets.
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-smith-6man-in-flight-eh-insertion-harmful/
> 
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-smith-6man-in-flight-eh-insertion-harmful-00
> https://datatracker.ietf.org/doc/html/draft-smith-6man-in-flight-eh-insertion-harmful-00
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html
> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 13 Oct 2019 01:08:51 +0100

Brian E Carpenter wrote on 12/10/2019 23:15:
> EH insertion for packets that traverse the open Internet is certainly harmful and violates RFC 8200

How does the packet know whether it's the open internet or a controlled 
domain?

Nick

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 13 Oct 2019 13:36:31 +1300

> How does the packet know whether it's the open internet or a controlled 

The packet is too dumb to know anything ;-). My question is how each node it traverses knows. Indeed, Mark's draft describes a scenario where the controlled domain argument breaks, because the exit node might not know that the packet had suffered EH insertion. The draft-voyer- scenario is not like that, because the affected IPv6 headers are created locally and identifiable as such.

   Brian

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sat, 12 Oct 2019 20:53:42 -0400

It seems to me, given the problems with the open internet that
Because I want to
and I think I have made everything safe for it

does not constitute a good reason for violating an RFC.  particularly 
one that was extensively discussed.
If the draft actually provided a significant reason for wanting the 
exception, we could debate the risks versus benefits.  But it does not 
do so.

Hence, Brian, I think you are cutting them more slack than they have earned.

Yours,
Joel

On 10/12/2019 8:08 PM, Nick Hilliard wrote:
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: Mark Smith <[email-masked]>
Date: Sun, 13 Oct 2019 12:49:39 +1100

Hi Brian,

On Sun, 13 Oct 2019 at 09:15, Brian E Carpenter
<[email-masked]> wrote:
>
> Hi Mark,
>
> As you have no doubt seen, the active proposal for EH insertion (draft-voyer-) states that the source and destination addresses of packets subject to insertion both lie within a controlled domain.
> Now while draft-carpenter-limited-domains calls for secure mechansims for determining whether a node is inside or on the border of a domain, it seems to me that address scope (and therefore routing scope) is today the state of the art for that. Of course you're correct that configured scope is subject to human error, but that already has very serious consequences, especially BGP misconfigurations. I think that draft-voyer-6man-extension-header-insertion-07 (not the earlier versions) is resistant to your arguments. It certainly does not describe the use case you give at https://tools.ietf.org/html/draft-smith-6man-in-flight-eh-insertion-harmful-00#section-3 .
>

Yes, they've mitigated leaking by enclosing in a tunnelling header
with local network addresses. However, that's now being used to ignore
problems with EH insertion when used with an outer IPv6 tunnel packet.
The processing of the IPv6 tunnel packet is now not compliant with RFC
8200. The Source Address field is a lie after the first mid-tunnel SRH
insertion.

Many of the the problems in the EH insertion considered harmful draft
occur whether or not the IPv6 packet happens to have a payload that is
also another IPv6 packet.

Regards,
Mark.

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 13 Oct 2019 11:25:02 +0100

Brian E Carpenter wrote on 13/10/2019 01:36:
> The packet is too dumb to know anything ;-). My question is how each
> node it traverses knows. Indeed, Mark's draft describes a scenario
> where the controlled domain argument breaks, because the exit node
> might not know that the packet had suffered EH insertion. The
> draft-voyer- scenario is not like that, because the affected IPv6
> headers are created locally and identifiable as such.

the world isn't nearly this pigeonholed though.  The edge around this 
"controlled domain" that you're implicitly suggesting is sharply-defined 
is in reality more of a blurry smear.  Think tunnels, leaks, back-doors, 
"SD-WAN" (whatever that means), policy routing, routed VPNs, etc.

If the ietf wants to define a new ipv6-like protocol which is not 
guaranteed to be interoperable with 8200, there's still space in 
www.iana.org/assignments/version-numbers to accommodate this :-)

Nick

---

From: Gyan Mishra <[email-masked]>
Date: Sun, 13 Oct 2019 11:59:32 -0400

Mark

Few comments on the draft in-line and also comparison of MPLS label insertion to SRV6 EH insertion.

Sent from my iPhone

> On Oct 13, 2019, at 6:25 AM, Nick Hilliard <[email-masked]> wrote:
> 
> the world isn't nearly this pigeonholed though.  The edge around this "controlled domain" that you're implicitly suggesting is sharply-defined is in reality more of a blurry smear.  Think tunnels, leaks, back-doors, "SD-WAN" (whatever that means), policy routing, routed VPNs, etc.
> 
> If the ietf wants to define a new ipv6-like protocol which is not guaranteed to be interoperable with 8200, there's still space in www.iana.org/assignments/version-numbers to accommodate this :-)
> 
> Nick
> 

[Gyan]

Comparing MPLS in flight label insertion to SRv6 in flight EH insertion below:

We will use the Service Provider scenario for SRv6 to compare apples to apples use case since SRv6 has a much wider scope as it pertains to IPv6 data plane traffic engineering capabilities.

Typical SP mpls core / SRv6 core with L3 VPN services CEs attached.

So for both scenarios the the packet originates from the customer network.

CE - customer edge originating packet 
PE1 - ingress PE device performing label imposition or EH insertion for SRH header 
P2 - LDP Remote LFA PLR node/path protection 
Performs LDP tunneling to PQ node by inserting MPLS label for FRR link and node protection. For SRv6 Ti-LFA PLR node performs EH SRH header insertion for the “LFA” backup protected path during a link or node failure 
P3 - PQ node tunnel for R-LFA 
P4 - egress P performing MPLS PHP pop implicit null value 3 or SRv6 PSP SRH segment pop and removal of SRH EH header type 4
PE2 - egress PE label disposition of bottom of stack L3 VPN  labels for MPLS use case

CE - PE1 - P1 - P2 - P3 - P4 - PE2 - CE

So in both cases the MPLS LDP or SRv6 domain the MPLS PE does in flight label imposition similar to SRv6 on the same ingress PE SRv6 does in flight SRv6 EH SRH Routing header type 4 insertion.

So in both cases the MPLS LDP or SRv6 domain the MPLS P2 “IP FRR PLR node”  does in flight label imposition similar to SRv6 on the same  P2 node SRv6 does in flight SRv6 EH SRH Routing header type 4 insertion as well for Ti-LFA 50ms FRR link and node protection.

So basically both MPLS and SRv6 perform in flight insertion of a header mpls 4 byte shim for mpls scenario and SRv6 EH SRH Routing header type 4 influencing the packet.

The major difference with mpls and why that in flight label insertion has never been an issue is that MPLS is considered “Layer 2 1/2” due to the label stack being placed between the L2 link layer header and the L3 header.  So impact to routing related to mpls shim not being removed or security implications don’t exist since in the mpls core you are not IP routing you are “label switching” so forwarding plane is at that “layer 2 1/2” swapping the local label with remote label along the LSP path to FEC destination.

With SRv6 since the EH headers are all inserted after the IPv6 header and before the transport header their are MAJOR security implications with EH headers not being removed as the EH headers are part of the L3 routing functionality.

One other major impact to EH insertion in flight with IPv6 is that SRv6 can be used ubiquitously in almost any scenario where traffic engineering is required.

With mpls the downside but from a security perspective upside as why mpls label imposition insertion in flight is not impacting is that with mpls state has to be maintained hop by hop for mpls data plane forwarding plane with either LDP or TE topmost label so mpls forwarding plane due to this “mpls state” LFIB dependency cannot exist outside of the mpls domain the egress P must to a PHP or if Default implicit null mpls label value type 3 is used or explicit null where the label is preserved on the egress P and popped on the egress PE along with the entire label stack is popped “label disposition” so the egress PE on the PE-CE link can forward as native IP packet.

RFC R-LFA used with MPLS LDP IP FRR

https://datatracker.ietf.org/doc/rfc7490/

What I think needs to change and we need to discuss with Spring folks is on the PLR node doing the 2nd EH insertion for Ti-LFA  to achieve link and node protection 50ms failover they need to do another IPV6 header encapsulation.

So the ingress PE in SRv6 domain would add the 1st outer layer IPv6 header but then on the Ti-LFA node now we would require from a 6MAN RFC 8200 a 2nd IPv6 encapsulation and the FRR steering would be done through the PQ node via the 2nd encapsulation and and the PQ node would remove the 2nd encapsulation which would contain the SRH Ti-LFA SID list for the traffic engineered path.

In the Voyer draft addressing EH insertion we should also have them make a bore that Ti-LFA would not exist in every scenario using SRV6 due to the ubiquitous use cases of SRv6 so that 2nd encapsulation would only be required wherever the 2nd or I guess even 3rd or every time an EH SRH type 4 routing header is added for that matter a IPv6 encapsulation is required to satisfy IPv6.

Regards 

Gyan

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 14 Oct 2019 08:34:33 +1300

> "SD-WAN" (whatever that means), policy routing, routed VPNs, etc.

Of course. In the limited-domains draft we're advocating an explicit
mechanism for defining the boundary; which of course wouldn't automatically
make it watertight, but would offer some hope of making it watertight.
But that being hand-waving, I'll stick to what I said: defining the boundary
by means of adddress scope and router config is the state of the art.
 
> www.iana.org/assignments/version-numbers to accommodate this :-)

No need. People who want to deploy it will just do so.

    Brian

---

From: Nick Hilliard <[email-masked]>
Date: Sun, 13 Oct 2019 20:53:39 +0100

> by means of adddress scope and router config is the state of the art.

you can define this as carefully you want, but people are going to 
ignore the scoping references and will expect that if the underlying 
protocol is notionally based on "ipv6", that it will operate correctly 
in general purpose ipv6 networks, e.g. the wider internet.

If your suggestion is to produce a protocol which is only expected to 
work within carefully defined walled gardens, that's fine and I think 
you should go ahead and do that.

But please don't say it's ipv6 or that it's compatible with ipv6 because 
it's not; it's something else.

Nick

---

From: Mark Smith <[email-masked]>
Date: Mon, 14 Oct 2019 09:48:47 +1100

> www.iana.org/assignments/version-numbers to accommodate this :-)

This is one of the fundamental observations.

 The trouble is calling this sort of closed/limited domain variant
that doesn't obey RFC 8200 "IPv6".

Protocols are more than just packet structures and reserved values,
they also specify field value rules, interpretations and permitted and
non-permitted behaviours. All of these latter properties are relied on
and expected by other implementations.

I think there are parallels with how human languages work.

For example, If I define the word "dog" (noun) to mean "wash" (verb)
within the domain of my house, can I still claim English is being
spoken in my house? The spelling and pronunciation of "dog" is the
same, but the meaning of "dog" and where it can be used in sentences
is different.

So it isn't English being spoken in my house anymore, even though it
resembles it, and there is quite a lot of commonality. English
dictionaries can't be reliably used for any word definitions in my
house once just one word is redefined. The more words I redefine the
less applicable the conventional English dictionary is in my house,
even though the word spellings and soundings continue to be the same.

This could be solved by calling it "Mark's English" or "Mark's Version
Of English", which clearly states it is different to conventional
English dictionary English. There needs to be a "Mark's English"
dictionary.

So as Nick said, if the IETF want to have non-interoperable versions
or variants of "IPv6", then they need different version numbers
somewhere or somehow - either minor e.g. 6.1, 6.2, or major e.g.
IPv10.

Regards,
Mark.

Regards,
Mark.

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 14 Oct 2019 13:19:47 +1300

One comment at the end:

> somewhere or somehow - either minor e.g. 6.1, 6.2, or major e.g.

Be careful what you wish for. It would be easy enough to define a
(probably hop-by-hop) option for this:
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   110xxxxx    |   00000001    | IPv6 version  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    Brian

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 14 Oct 2019 13:55:36 +0200

On 13/10/2019 02:08, Nick Hilliard wrote:
> How does the packet know whether it's the open internet or a 
> controlled domain? 
The packet does not, but the FIB could know. If the DA is not in the FIB 
the packet is not going anywhere.

Much of the problem of packet scoping can be dealt with by address scoping.

- Stewart

---

From: Mark Smith <[email-masked]>
Date: Tue, 15 Oct 2019 08:48:00 +1100

> >> Brian E Carpenter wrote on 13/10/2019 01:36:

<snip>

> (probably hop-by-hop) option for this:

While that should be a last resort compared to trying hard to fit
within existing RFC8200 mechanisms, I'd much rather that explicit
variant option than some implicitly different from RFC8200 behaviour
within a domain.

I like to imaging I'm troubleshooting by looking at a packet in a
packet capture tool like Wireshark. If I can look at the packet by
itself and tell how it should be to be processed without any
ambiguity, I'm much closer to solving the fault. If there is some
other processing context that isn't recorded in the packet itself,
that makes troubleshooting harder and take longer.

I think the more obvious it is how something works, the more obvious
it is where it may have broken.

Regards,
Mark.

---

From: Fernando Gont <[email-masked]>
Date: Mon, 14 Oct 2019 16:57:17 -0500

> (probably hop-by-hop) option for this:

It's easier than that. There's an in IP version field in the IPv{4,6}
header.

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Mark Smith <[email-masked]>
Date: Tue, 15 Oct 2019 09:16:13 +1100

<snip>

> It's easier than that. There's an in IP version field in the IPv{4,6}

While I still think this should be avoided, this would be an "accent"
or "dialect" of IPv6 rather than a different version.

Possibly relevant as an example of something to avoid in terms of
versions and needing sub-versions. English is my first language, and
apparently so is the older gentleman's in this video - not that I can
tell. I can easily understand the interviewer, and he apparently can
understand the older gentleman!

https://twitter.com/alreadytaken74/status/1151597731187777541?s=20

Regards,
Mark.

---

From: Fernando Gont <[email-masked]>
Date: Tue, 15 Oct 2019 10:09:11 -0500

> or "dialect" of IPv6 rather than a different version.

I was being ironic *a bit*. Something that inserts EHs in the middle is
not IPv6. So rather than calling it a version of IPv6, they might as
well call it a different version of IP. Say, IPv11 (as all the others
have been taken, IIRC).

Thanks,