Subject: [ippm] Is loopback/direct export amplification worse than linear?

From: Martin Duke <[email-masked]>
Date: Sun, 15 Nov 2020 21:45:04 -0800

I recognize that the loopback and direct export drafts discuss potential
amplification results, as a path of length N will generate N packets for
each IOAM packet -- a linear relationship. As these are discussed in the
draft with proposed mitigation, in an editorial sense the job is done.
Whether giving people this kind of foot gun is a good idea is a separate
question, but reasonable people can disagree.

However, if I understand correctly, certain pathological combinations of
IOAM namespaces could result in amplification far worse than linear.

1) Loopback loops
Imagine there is an IOAM namespace that covers the path from Hop A to Hop
D. There is a separate namespace entirely contained in A-D, from hop B from
hop C. Both namespaces enable loopback.
                           A
--------------------------------------------------------D

 B-------------------------------C

So a user packet is traveling from A to D. There will be (D-A) loopback
packets headed towards A. The (D - C) loopback packets that generated
between C and D will travel through the encapsulating node at C and then
trigger further loopbacks to C. Thus the total number of packets generated
by the single user packet is
(D - A) + (D - C)(C - B)
and obviousy there could be several of these smaller namespaces, or nested
namespaces, that aggravate the amplification further.

2) Direct Export
Direct Export potentially has even worse properties. Suppose namespace A,
N_A hops across, direct exports to a node that is reachable across
namespace B. Namespace B, N_B hops across, direct exports to a node
reachable across namespace A.

Thus a single user packet sent over namespace A will generate N_A packets
that traverse Namespace B. This will, in turn, generate N_A * N_B packets
that traverse namespace A, which in turn triggers N_A ^ 2 * N_B packets,
and so on.

In fact, if the namespaces direct export with probability exactly,1/N_A and
1/N_B, the same level of traffic will ping-pong infinitely. Any more than
that, and the traffic will steadily increase to infinity. If the
probability is 1, the growth is exponential.

****************

This analysis, if correct, raises three questions that I can think of:
- Are these corner cases plausible?
- If so, and they occurred, would the namespace administrators necessarily
be aware of each other? If not, I'm concerned that this is unsafe to deploy.
- Do phenomena like this indicate that the design is brittle and putting in
some "considerations" doesn't really mitigate the dangers here?

Martin

---

From: Tal Mizrahi <[email-masked]>
Date: Wed, 25 Nov 2020 11:33:50 +0200

Hi Martin,

Thanks for raising these points.
We have discussed these points today at the IOAM design team meeting,
and we suggest to mention these scenarios in the drafts.
We believe the reader should be aware of these scenarios, but that the
mitigations that are already described in the draft are sufficient at
this point.

Regarding issue (1) the following  text is proposed to be added to the
security considerations section of the flag draft:

Some of the security threats that were discussed in this document may
be worse in a wide area network in which there are nested IOAM
domains. For example, if there are two nested IOAM domains that use
loopback, then a looped-back copy in the outer IOAM domain may be
forwarded through another (inner) IOAM domain and may be subject to
loopback in that (inner) IOAM domain, causing the amplification to be
worse than in the conventional case.

Regarding issue (2), the following text is proposed for the DEX draft:

The amplification effect of DEX may be worse in wide area networks in
which there are multiple IOAM domains. For example, if DEX is used in
IOAM domain 1 for exporting IOAM data to a receiving entity, then the
exported packets of domain 1 can be forwarded through IOAM domain 2,
in which they are subject to DEX. The exported packets of domain 2 may
in turn be forwarded through another IOAM domain (or through domain
1), and theoretically this recursive amplification may continue
infinitely.

Thanks,
Tal.

On Mon, Nov 16, 2020 at 7:45 AM Martin Duke <[email-masked]> wrote:
>
> I recognize that the loopback and direct export drafts discuss potential amplification results, as a path of length N will generate N packets for each IOAM packet -- a linear relationship. As these are discussed in the draft with proposed mitigation, in an editorial sense the job is done. Whether giving people this kind of foot gun is a good idea is a separate question, but reasonable people can disagree.
>
> However, if I understand correctly, certain pathological combinations of IOAM namespaces could result in amplification far worse than linear.
>
> 1) Loopback loops
> Imagine there is an IOAM namespace that covers the path from Hop A to Hop D. There is a separate namespace entirely contained in A-D, from hop B from hop C. Both namespaces enable loopback.
>                            A --------------------------------------------------------D
> So a user packet is traveling from A to D. There will be (D-A) loopback packets headed towards A. The (D - C) loopback packets that generated between C and D will travel through the encapsulating node at C and then trigger further loopbacks to C. Thus the total number of packets generated by the single user packet is
> (D - A) + (D - C)(C - B)
> and obviousy there could be several of these smaller namespaces, or nested namespaces, that aggravate the amplification further.
>
> 2) Direct Export
> Direct Export potentially has even worse properties. Suppose namespace A, N_A hops across, direct exports to a node that is reachable across namespace B. Namespace B, N_B hops across, direct exports to a node reachable across namespace A.
>
> Thus a single user packet sent over namespace A will generate N_A packets that traverse Namespace B. This will, in turn, generate N_A * N_B packets that traverse namespace A, which in turn triggers N_A ^ 2 * N_B packets, and so on.
>
> In fact, if the namespaces direct export with probability exactly,1/N_A and 1/N_B, the same level of traffic will ping-pong infinitely. Any more than that, and the traffic will steadily increase to infinity. If the probability is 1, the growth is exponential.
>
> ****************
>
> - If so, and they occurred, would the namespace administrators necessarily be aware of each other? If not, I'm concerned that this is unsafe to deploy.
> - Do phenomena like this indicate that the design is brittle and putting in some "considerations" doesn't really mitigate the dangers here?
>
> Martin
>
>
>
>
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm

---

From: Martin Duke <[email-masked]>
Date: Wed, 25 Nov 2020 10:22:14 -0800

Hi Tal,

I believe this text adequately describes my concerns, so you've done what
you can in terms of updating the draft. Thanks.

I am increasingly skeptical that protocols with these dynamics are a good
thing to deploy in the internet, regardless of what we put in the Security
Considerations. I would certainly like to hear the opinions of other people
in the WG on this topic.

Martin

On Wed, Nov 25, 2020 at 1:34 AM Tal Mizrahi <[email-masked]>
wrote:

> > Imagine there is an IOAM namespace that covers the path from Hop A to
> Hop D. There is a separate namespace entirely contained in A-D, from hop B
> from hop C. Both namespaces enable loopback.
> >                            A
> > and obviousy there could be several of these smaller namespaces, or
> nested namespaces, that aggravate the amplification further.
> >
> > 2) Direct Export
> > Direct Export potentially has even worse properties. Suppose namespace
> A, N_A hops across, direct exports to a node that is reachable across
> > Thus a single user packet sent over namespace A will generate N_A
> packets that traverse Namespace B. This will, in turn, generate N_A * N_B
> packets that traverse namespace A, which in turn triggers N_A ^ 2 * N_B
> packets, and so on.
> >
> > In fact, if the namespaces direct export with probability exactly,1/N_A
> and 1/N_B, the same level of traffic will ping-pong infinitely. Any more
> than that, and the traffic will steadily increase to infinity. If the
> > - If so, and they occurred, would the namespace administrators
> necessarily be aware of each other? If not, I'm concerned that this is
> unsafe to deploy.
> > - Do phenomena like this indicate that the design is brittle and putting
> in some "considerations" doesn't really mitigate the dangers here?
> >
> > Martin
> >
> >
> >
> >

---

From: Haoyu Song <[email-masked]>
Date: Wed, 25 Nov 2020 21:00:35 +0000

Hi Martin,

I don’t understand the DEX amplification scenario. We don’t specify the export packet format yet but conceivably those packets are not like the original packets and just for the purpose of data export, so they shouldn’t trigger any data collection and export anymore. If I misunderstood something, could you please clarify your concerns? Thanks!

Best regards,
Haoyu

From: ippm <[email-masked]> On Behalf Of Martin Duke
Sent: Wednesday, November 25, 2020 10:22 AM
To: Tal Mizrahi <[email-masked]>
Cc: IETF IPPM WG <[email-masked]>
Subject: Re: [ippm] Is loopback/direct export amplification worse than linear?

Hi Tal,

I believe this text adequately describes my concerns, so you've done what you can in terms of updating the draft. Thanks.

I am increasingly skeptical that protocols with these dynamics are a good thing to deploy in the internet, regardless of what we put in the Security Considerations. I would certainly like to hear the opinions of other people in the WG on this topic.

Martin

On Wed, Nov 25, 2020 at 1:34 AM Tal Mizrahi <[email-masked]<mailto:[email-masked]>> wrote:
Hi Martin,

Thanks,
Tal.

On Mon, Nov 16, 2020 at 7:45 AM Martin Duke <[email-masked]<mailto:[email-masked]>> wrote:
> [email-masked]<mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/ippm<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fippm&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cb41ad88d7d6e4241b3af08d8916f275e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637419253886492031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iBoKLaGtRDisqs2lhtByByAW%2FVMPpTOeQL%2BZ307%2BTAk%3D&reserved=0>

---

From: Martin Duke <[email-masked]>
Date: Wed, 25 Nov 2020 13:49:23 -0800

Hello Haoyu,

Let's assume that Namespace A and B are completely independent entities.
Further, assume that the ingress points to A and B are both pretty
liberally attached DEX flags to traffic.

So if a single user packet traverses A, it will generate N_A DEX packets
that will be routed through B. If these meet the criteria for B to mark for
DEX, there will be N_A * N_B packets that then traverse A. If these meet
the A criteria, then N_A^2 * N_B packets will go through B, and so on.

I certainly agree that this is a corner case. And yet, it appears that two
totally independent entities could create this catastrophic loop. It seems
like a design flaw! Now maybe the DEX packets are in a format that MUST NOT
be DEXed. That would mostly solve the problem, except that those packets
might be tunneled over something that once again triggers DEX.

On Wed, Nov 25, 2020 at 1:00 PM Haoyu Song <[email-masked]> wrote:

> Hi Martin,
>
>
>
> I don’t understand the DEX amplification scenario. We don’t specify the
> export packet format yet but conceivably those packets are not like the
> original packets and just for the purpose of data export, so they shouldn’t
> trigger any data collection and export anymore. If I misunderstood
> something, could you please clarify your concerns? Thanks!
>
>
>
> Best regards,
>
> Haoyu
>
>
>
> *From:* ippm <[email-masked]> *On Behalf Of * Martin Duke
> *Sent:* Wednesday, November 25, 2020 10:22 AM
> *To:* Tal Mizrahi <[email-masked]>
> *Cc:* IETF IPPM WG <[email-masked]>
> *Subject:* Re: [ippm] Is loopback/direct export amplification worse than
> linear?
>
>
>
> Hi Tal,
>
>
>
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fippm&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cb41ad88d7d6e4241b3af08d8916f275e%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637419253886492031%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=iBoKLaGtRDisqs2lhtByByAW%2FVMPpTOeQL%2BZ307%2BTAk%3D&reserved=0>
>
>