Subject: Re: [ippm] WG Last Call for draft-ietf-ippm-ioam-ipv6-options

From: Erik Kline <[email-masked]>
Date: Fri, 17 Dec 2021 13:37:23 -0800

+6MAN WG, just as an FYI

On Fri, Dec 17, 2021 at 7:39 AM Marcus Ihlar
<marcus.ihlar=[email-masked]> wrote:
>
> Hello IPPM,
>
> This email starts a Working Group Last Call on draft-ietf-ippm-ioam-ipv6-options.
>
>
>
> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-ipv6-options/
>
> https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-ipv6-options-06
>
>
>
> Please provide feedback by replying to this email with your reviews and if you think this document is ready to progress, by Friday, January 14.
>
>
>
> Best Regards,
>
> Marcus & Tommy
>
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: Justin Iurman <[email-masked]>
Date: Wed, 29 Dec 2021 15:08:55 +0100 (CET)

Hi ippm, 6man,

I do believe this document is almost ready, though it probably needs a
new version to address the following review comments.

1) In section 4 ("In-situ OAM Metadata Transport in IPv6"), third
paragraph, it says:

  "In order for IOAM to work in IPv6 networks, IOAM MUST be explicitly
   enabled per interface on every node within the IOAM domain.  Unless a
   particular interface is explicitly enabled (i.e., explicitly
   configured) for IOAM, a router MUST drop packets that contain
   extension headers carrying IOAM data-fields.  This is the default
   behavior and is independent of whether the Hop-by-Hop options or
   Destination options are used to encode the IOAM data.  This ensures
   that IOAM data does not unintentionally get forwarded outside the
   IOAM domain."

I don't think "a router MUST drop packets" is correct. IOAM options are
defined with type "00", i.e., skip over this option and continue
processing the header. Instead, I would rather have something like "a
router MUST ignore IOAM options". Indeed, you don't want a node to drop
packets with IOAM options, when not configured, to keep consistency with
the behavior of type "00". This is the role of a firewall to drop
packets if it detects IOAM injections or IOAM leaks.

New text suggestion:

  "In order for IOAM to work in IPv6 networks, IOAM MUST be explicitly
   enabled per interface on every node within the IOAM domain.  Unless a
   particular interface is explicitly enabled (i.e., explicitly
   configured) for IOAM, a router MUST ignore IOAM Options.  As
   additional security, IOAM domains SHOULD provide a mechanism to
   prevent injections at ingress or leaks at egress."

Note that the last sentence could be omitted, as consideration #4 (see
section 5.1) already talks about it.

2) Section 5.4.1 ("IPv6-in-IPv6 encapsulation") describes a hacky way to
encapsulate packets. Instead of using the egress as the outer
destination address, it uses the same destination address as the inner
one. I understand the pros of such a technique (i.e., same forwarding
path), but it goes against RFC2473 and, as a consequence, also against
RFC8200. So, I don't think it should be part of a standard. Since the
"normal" IPv6-in-IPv6 use case is implicitly included in the next
section (5.4.2, "IP-in-IPv6 encapsulation with ULA"), I think section
5.4.1 could/should be dropped.

Thanks,
Justin

On Dec 17, 2021, at 10:37 PM, Erik Kline [email-masked] wrote:
>> Please provide feedback by replying to this email with your reviews and if you
>> think this document is ready to progress, by Friday, January 14.
>>
>>
>>
>> Best Regards,
>>
>> Marcus & Tommy
>>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Mark Smith <[email-masked]>
Date: Thu, 30 Dec 2021 06:14:23 +1100

Hi,

Hmm, not sure how I missed this, I thought I was subscribed to the IOAM ML.

I've been thinking for a while that there might be an opportunity to
simplify IOAM over IPv6 significantly, based on the following:

- that the IOAM domain is contiguous, which I think is really saying
that all IOAM nodes in the IOAM paths/IOAM domain are directly
adjacent at layer 2, meaning directly connected to each other via
layer 2 links (or tunnels, which are virtual layer 2 links.)

- IOAM over IPv6 IOAM nodes are RFC8200 IPv6 Nodes.

- per RFC4291, all IPv6 enabled interfaces on IPv6 nodes are required
to have Link-Local addresses.

Since the IOAM domain is a contiguous path of IPv6 nodes, it means
that the DA of the outer IOAM encapsulation IPv6 packet could be the
next node's IPv6 Link-Local Address. For each IOAM node (IPv6
router/host) these LLAs would be the same LLAs as the next hop router
used by routes/routing protocols e.g. OSPF route next hop LLAs or
default router LLAs advertised to hosts via RAs.

This idea is really just a simpler version of the IPv6-in-IPv6 using
ULA method described. The ULA addressing was partly necessary to skip
past/over non-IOAM enabled nodes within an IOAM domain, and was a way
to prevent or mitigate the effects of IOAM encapsulated packets
leaking. Using LLAs for IOAM encap DAs solves that problem too, and is
also simpler because there is no need to deploy a ULA address space.
The automatic LLAs can be used instead.

That being said, I've also wondered about a "donut" IOAM deployment
model. IOAM processing is performed at a contiguous IOAM edge of the
network (e.g. IOAM enabled access, distribution layers) using LLAs.
IOAM is not normally performed in the core of the network for
performance reasons, and ULA addressing is used for the encapsulation
DAs to send those IOAM packets from one side of the network core
"donut hole" to the other.

Regards,
Mark.

---

From: Haoyu Song <[email-masked]>
Date: Wed, 29 Dec 2021 21:29:46 +0000

Hi IPPM and 6MAN,

After reading the draft, I have one question. My understanding is that the proposed approach basically uses IP-in-IP encapsulation and encapsulates the IOAM option in the outer IPv6 header. While the in-band measurement is intended for the inner IPv6 packet, I'm concerned on how the requirement C1 can be met. For example, I'm not sure how the forwarding and ECMP can behave identically with and without the outer IPv6 header (i.e., for the overlay and underlay networks).

Thanks,
Haoyu

From: ippm <[email-masked]> On Behalf Of Marcus Ihlar
Sent: Friday, December 17, 2021 7:39 AM
To: [email-masked]
Subject: [ippm] WG Last Call for draft-ietf-ippm-ioam-ipv6-options

Hello IPPM,

https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-ipv6-options/<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-ippm-ioam-ipv6-options%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7C8895882824fd43c501e508d9c1736d28%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637753523788355875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WJ1EZAPGbexogIV4J4pQR69%2F7Gh0%2FiVMn3lyWORJAic%3D&reserved=0>
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-ipv6-options-06<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-ippm-ioam-ipv6-options-06&data=04%7C01%7Chaoyu.song%40futurewei.com%7C8895882824fd43c501e508d9c1736d28%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637753523788355875%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YrK0mXHkK2thhO29ogOryETZZifwDR%2BE8VS%2FXV3%2FmA4%3D&reserved=0>

Best Regards,
Marcus & Tommy

---

From: Mark Smith <[email-masked]>
Date: Thu, 30 Dec 2021 12:26:27 +1100

On Thu, 30 Dec 2021 at 08:30, Haoyu Song <[email-masked]> wrote:
>
> Hi IPPM and 6MAN,
>
>
>
> After reading the draft, I have one question. My understanding is that the proposed approach basically uses IP-in-IP encapsulation and encapsulates the IOAM option in the outer IPv6 header. While the in-band measurement is intended for the inner IPv6 packet, I’m concerned on how the requirement C1 can be met. For example, I’m not sure how the forwarding and ECMP can behave identically with and without the outer IPv6 header (i.e., for the overlay and underlay networks).
>

I think this is another example of where thinking IP-in-IP is
something special and different makes things confusing. "IP
tunnelling" is no more than a link-layer encapsulation. An "IP tunnel"
is a virtual layer 2 point-to-point link.

Original IPv6 packet forwarding through the IOAM domain would still
use information in the original packet (DA minimum, and whatever else
is considered for ECMP). The addition of the encapsulated updated IOAM
information and the outer IP header to get it to the next hop occurs
after that, as the packet goes down to the link-layer when link-layer
encapsulation occurs. IOAM conceptually occurs at "layer 2.5".

Since the IOAM domain is contiguous IPv6 nodes, meaning they're all
link-layer adjacent, imagine instead using bare Ethernet to carry the
IOAM information to the next IOAM hop, with the IOAM information added
when the original IPv6 packet is passed down for link-layer
encapsulation:

Packet With IOAM:

[Ethernet] - [IOAM] - [original IPv6 packet A]

Packet Without IOAM:

[Ethernet] - [original IPv6 packet A]

Now substitute [Outer IPv6] for [Ethernet] above. Except, don't do
that often, it is far less confusing with "IP tunnels" if you imagine
the link-layer encapsulation format is a typical layer 2 frame
different from the original link-layer payload IPv6 packet.

(I've started to think using native Ethernet as above would be better
when possible for IOAM instead of IPv6-in-IPv6 -
draft-weis-ippm-ioam-eth-04 defines an Ethertype. IPv6-in-IPv6 could
be used when there is no native link-layer type field defined to carry
IOAM e.g. [Bluetooth]-[IPv6]-[IOAM]-[original IPv6 packet A]).

Regards,
Mark.

---

From: Haoyu Song <[email-masked]>
Date: Thu, 30 Dec 2021 17:33:04 +0000

Hi Mark,

Please see inline [HS].

Thanks,
Haoyu
-----Original Message-----
From: Mark Smith <[email-masked]> 
Sent: Wednesday, December 29, 2021 5:26 PM
To: Haoyu Song <[email-masked]>
Cc: Marcus Ihlar <marcus.ihlar=[email-masked]>; [email-masked]; 6man <[email-masked]>
Subject: Re: WG Last Call for draft-ietf-ippm-ioam-ipv6-options

> After reading the draft, I have one question. My understanding is that the proposed approach basically uses IP-in-IP encapsulation and encapsulates the IOAM option in the outer IPv6 header. While t…

I think this is another example of where thinking IP-in-IP is something special and different makes things confusing. "IP tunnelling" is no more than a link-layer encapsulation. An "IP tunnel"
is a virtual layer 2 point-to-point link.

Original IPv6 packet forwarding through the IOAM domain would still use information in the original packet (DA minimum, and whatever else is considered for ECMP). The addition of the encapsulated updated IOAM information and the outer IP header to get it to the next hop occurs after that, as the packet goes down to the link-layer when link-layer encapsulation occurs. IOAM conceptually occurs at "layer 2.5".

[HS]  I'm still confused. The draft says "The source address of the outer IPv6 header is that of the IOAM encapsulating node.  The destination address of the outer IPv6 header is the same as the inner IPv6 destination  address". So to me it's not a link-layer encapsulation. It's a new packet different from the original one. If a router forwards packets only based on DA, that's fine. Otherwise, the forwarding behavior can be different. Do I miss something here?

Since the IOAM domain is contiguous IPv6 nodes, meaning they're all link-layer adjacent, imagine instead using bare Ethernet to carry the IOAM information to the next IOAM hop, with the IOAM information added when the original IPv6 packet is passed down for link-layer
encapsulation:

Packet With IOAM:

Packet Without IOAM:

Now substitute [Outer IPv6] for [Ethernet] above. Except, don't do that often, it is far less confusing with "IP tunnels" if you imagine the link-layer encapsulation format is a typical layer 2 frame different from the original link-layer payload IPv6 packet.

(I've started to think using native Ethernet as above would be better when possible for IOAM instead of IPv6-in-IPv6 -
draft-weis-ippm-ioam-eth-04 defines an Ethertype. IPv6-in-IPv6 could be used when there is no native link-layer type field defined to carry IOAM e.g. [Bluetooth]-[IPv6]-[IOAM]-[original IPv6 packet A]).

Regards,
Mark.

> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata
> tracker.ietf.org%2Fdoc%2Fdraft-ietf-ippm-ioam-ipv6-options%2F&amp;data
> =04%7C01%7Chaoyu.song%40futurewei.com%7C095ac60a5eea46c2251608d9cb3376
> 85%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637764244169572611%7CU
> nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> WwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=dEMKFMleDA31SVX9TxkCsGG6R9vmz%2FEPj
> RrHAzvwecY%3D&amp;reserved=0
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdata
> tracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-ippm-ioam-ipv6-options-06&a
> mp;data=04%7C01%7Chaoyu.song%40futurewei.com%7C095ac60a5eea46c2251608d
> 9cb337685%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637764244169572
> 611%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=wITIFHgBev%2BnPyfs73jKKjRaFG
> 5alDMtv1gsqtantxM%3D&amp;reserved=0
>
>
>
> Administrative Requests: 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40f
> uturewei.com%7C095ac60a5eea46c2251608d9cb337685%7C0fee8ff2a3b240189c75
> 3a1d5591fedc%7C1%7C1%7C637764244169572611%7CUnknown%7CTWFpbGZsb3d8eyJW
> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&
> amp;sdata=r1766WzhOMQ7SE4q%2F6EtJkJexJ18xKLx4pCXsFoY8%2F8%3D&amp;reser
> ved=0