Subject: [v6ops] Operational Implications of IPv6 Packets with Extension Headers - implications from new development for EHs

From: Vasilenko Eduard <[email-masked]>
Date: Wed, 29 Jul 2020 08:26:34 +0000

Hi Fernando,
IETF has many WGs that developing now many new EHs. iOAM, iFit, SRv6, AltMarking, APN6, and many others.
Do you believe that it would increase IPv6 drop rate in midterm?
I do assume that you do not want to make any comment in the draft, because it is "speculation" (or "prediction"?).
I am asking, because may be the question: what to do to mitigate anticipated problem?
Eduard
-----Original Message-----
From: v6ops [mailto:[email-masked]] On Behalf Of Fernando Gont
Sent: 26 июля 2020 г. 8:46
To: IPv6 Operations <[email-masked]>
Cc: [email-masked]
Subject: [v6ops] Operational Implications of IPv6 Packets with Extension Headers (Fwd: New Version Notification for draft-gont-v6ops-ipv6-ehs-packet-drops-04.txt)

Folks,

We have posted a rev of our IETF I-D "Operational Implications of IPv6 Packets with Extension Headers".

The I-D is available at: 
https://www.ietf.org/internet-drafts/draft-gont-v6ops-ipv6-ehs-packet-drops-04.txt

Your feedback will be appreciated.

Thanks!

Cheers,
Fernando

-------- Forwarded Message --------
Subject: New Version Notification for
draft-gont-v6ops-ipv6-ehs-packet-drops-04.txt
Date: Sat, 25 Jul 2020 22:28:50 -0700
From: [email-masked]
To: Fernando Gont <[email-masked]>, Gert Doering <[email-masked]>, Geoff Huston <[email-masked]>, Warren Kumari <[email-masked]>, Nick Hilliard <[email-masked]>

A new version of I-D, draft-gont-v6ops-ipv6-ehs-packet-drops-04.txt
has been successfully submitted by Fernando Gont and posted to the IETF repository.

Name:		draft-gont-v6ops-ipv6-ehs-packet-drops
Revision:	04
Title:		Operational Implications of IPv6 Packets with Extension Headers
Document date:	2020-07-25
Group:		Individual Submission
Pages:		15
URL: 
https://www.ietf.org/internet-drafts/draft-gont-v6ops-ipv6-ehs-packet-drops-04.txt
Status: 
https://datatracker.ietf.org/doc/draft-gont-v6ops-ipv6-ehs-packet-drops/
Htmlized: 
https://tools.ietf.org/html/draft-gont-v6ops-ipv6-ehs-packet-drops-04
Htmlized: 
https://datatracker.ietf.org/doc/html/draft-gont-v6ops-ipv6-ehs-packet-drops
Diff: 
https://www.ietf.org/rfcdiff?url2=draft-gont-v6ops-ipv6-ehs-packet-drops-04

Abstract:
    This document summarizes the security and operational implications of
    IPv6 extension headers, and attempts to analyze reasons why packets
    with IPv6 extension headers may be dropped in the public Internet.

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

_______________________________________________
v6ops mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/v6ops

---

From: Gert Doering <[email-masked]>
Date: Wed, 29 Jul 2020 10:43:51 +0200

Hi,

> I am asking, because may be the question: what to do to mitigate anticipated problem?

Do not assume EHs will work outside a very well-defined scope (SRv6 
is an example of "will work, because won't leave the network where
needed").  If you start work with the goal "must work across the 
wild Internet", abandon the idea to do this with EH.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                      Vorstand: Sebastian v. Bomhard, Michael Emmer
Joseph-Dollinger-Bogen 14        Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                 HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444         USt-IdNr.: DE813185279

---

From: Vasilenko Eduard <[email-masked]>
Date: Wed, 29 Jul 2020 09:10:28 +0000

Hi Gert,
This approach is not universal. We need something better.
Some EHs (like APN6 or Network Token) do make sense only cross-border. Because source to attach EH is assumed outside of administrative domain. Just today was discussion about "how to make Alternative Marking cross-border".
Moreover, "business" by definition is between different legal entities. If all these new fancy features are not related to business - many people would be very sorry.
Eduard
-----Original Message-----
From: Gert Doering [mailto:[email-masked]] 
Sent: 29 июля 2020 г. 11:44
To: Vasilenko Eduard <[email-masked]>
Cc: Fernando Gont <[email-masked]>; IPv6 Operations <[email-masked]>
Subject: Re: [v6ops] Operational Implications of IPv6 Packets with Extension Headers - implications from new development for EHs

Hi,

> I am asking, because may be the question: what to do to mitigate anticipated problem?

Do not assume EHs will work outside a very well-defined scope (SRv6 is an example of "will work, because won't leave the network where needed").  If you start work with the goal "must work across the wild Internet", abandon the idea to do this with EH.

Gert Doering
        -- NetMaster

---

From: Gert Doering <[email-masked]>
Date: Wed, 29 Jul 2020 11:21:11 +0200

Hi,

On Wed, Jul 29, 2020 at 09:10:28AM +0000, Vasilenko Eduard wrote:
> Some EHs (like APN6 or Network Token) do make sense only cross-border. 

It might be possible to make this work between cooperating networks.

For generic "across the wide internet, without underlying commercial
relationships that ensure that a particular EH will work" EHs do not
work well enough to be a suitable tool.

Not my decision.  

Just observation after 20 years of living with some of the ivory tower
silliness that went into IPv6 protocol design.  Like "it should be hard
to handle EHs in middleboxes" and "make this an unbounded chain! 
computers love unbounded work!"

> Because source to attach EH is assumed outside of administrative domain. Just today was discussion about "how to make Alternative Marking cross-border".

The Internet is, generally speaking, in a very sorry state.

Nothing new here.

Gert Doering
        -- NetMaster

---

From: [email-masked]
Date: Wed, 29 Jul 2020 12:13:03 +0200

> wild Internet", abandon the idea to do this with EH.

Is that significantly different from:

"Do not assume ICMP, IP fragments, or any other transport protocol than TCP port 80,443 or UDP port 53 will work. Do not assume anything but IPv4 will work."

I think the EHs suffer a bit from a chicken and egg problem.
That is, if a cross the Internet EH ever got defined, then networks would adopt to carry it transparently.
I didn't see if this draft got into localising where the breakage happens; at least previous wisdom was that it happened at the edges not in the core.

Anyway I don't quite understand what this document adds to work describing ossification in general and the previous work on operational considerations for EHs.

Best regards,
Ole

---

From: Fernando Gont <[email-masked]>
Date: Wed, 29 Jul 2020 07:12:13 -0300

> Do you believe that it would increase IPv6 drop rate in midterm?

All the measures that we have so far done seem to indicate that the use 
of IPv6 EHs greatly increase the packet drop rate when traversing the 
public Internet.

You may be able to employ EHs within the so called "controlled domains", 
though.

> I do assume that you do not want to make any comment in the draft, because it is "speculation" (or "prediction"?).

What we are trying to do with this document is to report and elaborate 
on the current situation, more than trying to guess what might happen in 
the future.

> I am asking, because may be the question: what to do to mitigate anticipated problem?

My understanding is that technologies such as SRv6 are to be employed in 
controlled domains. In such cases, either you control your network or 
you cooperate with the folks controlling the network -- so you shoudl be 
fine (or should be able to be fine).

OTOH, you are not that lucky when sending packets on the public Internet.

To a large extent, the drops have to do with boxes not being able to do 
their work when packets contain EHs. So most likely the problem would 
how away if/when such boxes are replaced and if they are replaced that 
are able to handle packets with EHs more gracefully. So, that's 
functionality that vendors are not willing to provide for free, and 
operators are not willing to pay extra for. So the future doesn't look 
very promising in this respect... -- but you never know.

But certainly, if there's any improvement in this respect, it will grow 
out of widespread awareness about the problem is question.

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Fernando Gont <[email-masked]>
Date: Wed, 29 Jul 2020 07:28:06 -0300

> "Do not assume ICMP, IP fragments, or any other transport protocol than TCP port 80,443 or UDP port 53 will work. Do not assume anything but IPv4 will work."

The difference is that modulo fragments, non of those prevent boxes from 
doing their work. -- they may still be filtered, but folks don't 
necessarily have to resort to that.

> That is, if a cross the Internet EH ever got defined, then networks would adopt to carry it transparently.

There is indeed quite a bit of a "chicken and egg" problem here (*). But 
it's not simply an arbirtrary filtering policy, but rather the result of 
packet handling constraints.

(*) There *are* cross-Internet EHs: ESP. But it suffers from the same 
problem -- so, ironically, you may need to encapsulate or do something else.

Quite recently, I migrated a few IPv6-over-IPv4 tunnels I had to 
IPv6-over-IPv6. BUt things wouldn't work. After some debugging, it 
turned out that the "tunnel encapsulation limit" option was being 
automatically employs for v6 tunnels, which caused the resulting packets 
to be dropped. Of course, the workaround was to disable it.

> I didn't see if this draft got into localising where the breakage happens; at least previous wisdom was that it happened at the edges not in the core.

RFC7872 suggests that it happens both at the edge and at the core.

> Anyway I don't quite understand what this document adds to work describing ossification in general

Avoiding rehashing the same discussion everytime the issue of EHs comes up?

> and the previous work on operational considerations for EHs.

What's the "previous work on operational considerations for EHs"?

Thanks,

---

From: Gert Doering <[email-masked]>
Date: Wed, 29 Jul 2020 12:40:34 +0200

Hi,

> "Do not assume ICMP, IP fragments, or any other transport protocol than TCP port 80,443 or UDP port 53 will work. Do not assume anything but IPv4 will work."

I'm not *that* depressed yet.

But yes, this seems to be reasonable advice to application programmers today.

Gert Doering
        -- NetMaster

---

From: [email-masked]
Date: Wed, 29 Jul 2020 12:43:23 +0200

> Avoiding rehashing the same discussion everytime the issue of EHs comes up?

So you are proposing to fully embrace ossification and deprecate all EHs?

Ole

---

From: Fernando Gont <[email-masked]>
Date: Wed, 29 Jul 2020 07:50:56 -0300

> So you are proposing to fully embrace ossification and deprecate all EHs?

I'm proposing to document and acknowledge reality.

Are you proposing to continue in denial?

Thanks,

---

From: Philip Homburg <[email-masked]>
Date: Wed, 29 Jul 2020 13:06:39 +0200

>So you are proposing to fully embrace ossification and deprecate all EHs?

In my experience, any area where IPv6 is different from IPv4 is 
extremely annoying. As long as IPv6 is just IPv4 with bigger addresses,
you write the code once, taking into account the difference in 
addresses and that's it.

For the most part, todays internet is IPv4 with a bit of IPv6. So you have
do the IPv4 part anyhow, and it needs to work well.

There may be some isolated parts of the internet that can benefit from
features that only exist in IPv6, but in general, I think those features
are just going to be igored.

Of course this may change when IPv6 becomes the default, dominant protocol
and IPv4 is left to rot, but that may take a few years.

---

From: Gert Doering <[email-masked]>
Date: Wed, 29 Jul 2020 14:26:22 +0200

Hi,

> So you are proposing to fully embrace ossification and deprecate all EHs?

Can you see a way to make them work?

Gert Doering
        -- NetMaster

---

From: Tom Herbert <[email-masked]>
Date: Wed, 29 Jul 2020 07:34:26 -0700

> Can you see a way to make them work?

Gert,

Yes. We can apply the same principles that helped IPv6 get any
traction (even if it only took twenty years :-) ): Assume deployment
of any new feature on the Internet will be incremental, Provide an
automatic fallback to a lesser protocol when the feature doesn't work
over some path at the possible cost of degraded service to the user,
Collect real time data on status of feature support in different parts
of the Internet and make it public, Ensure there is economic value in
supporting the feature so it's not just an academic exercise, Be
flexible in use of the feature (e.g. extension header
insertion/removal).

IETF can address issues in the protocols themselves that inhibit
practical implementation. For instance, in retrospect wrt extension
headers it's clear that requiring every node to process HBH options
and requiring these devices to potentially process hundreds of TLVs in
the fast path wasn't feasible-- we can provide practical guidance to
mitigate these (which RFC8200 and RFC8504 does in these cases).
There's still work to be done, for instance hardware limitations
around how many bytes of headers a device can process is regularly
raised (including by this drat) but what exactly is "too big" has yet
to be quantified; it would be nice to provide guidance for host
implementations so they can know when they're sending a packet with
too many bytes of headers (draft-ietf-6man-icmp-limits-08 is a good
direction but only to the extent that ICMP is usable).

Tom

---

From: Gert Doering <[email-masked]>
Date: Wed, 29 Jul 2020 16:58:28 +0200

Hi,

> flexible in use of the feature (e.g. extension header

This looks like tremendous success story in the making.  Just like 
IETF succeeded with getting IPv6 and all the cool bits of it (look,
ma, it has EH!!!  AND IPSEC!!) out into the fields.

There is no incentive for network operators to make sure their network
can transport EHs if it works "without".

There is no incentive for application developers to code support for
"anything with EH" if they have to assume "it will need to work across
networks without".

Now, if IETF could find big bags of money somewhere to hand this over
to network gear vendors to build EH-friendly equipment without extra
costs, and give some more money to network operators operating in an
increasingly low-margin market to replace all their gear with "new and
full of EH support!  same power consumption, same everything, same
price, just more EH!", then there might be some hope.

But even then people will happily use the no-longer-in-use non-EH
compliant gear to build new networks with lower setup costs.

Gert Doering
        -- NetMaster

---

From: Tom Herbert <[email-masked]>
Date: Wed, 29 Jul 2020 08:18:28 -0700

Gert,

Then they won't be able to use segment routing, IOAM, or new QoS
signaling like Network Tokens that are now being developed and being
deployed. That's fine, it's their prerogative. Some operators will
certainly make this decision to not invest in new technologies, like
when they decided to defer on IPv6, but I don't believe all operators
will come to that same conclusion. Even the data bears this out, the
oft quoted RFC7872 still shows that even in the worst case of HBH
options the majority of flows still made it to the destination. It's a
given that we can't rely on much to always work 100% on the Internet,
but IMO if we apply the right techniques we can opportunistically
provide something that works and provides value to users.

Tom

Tom

---

From: Gert Doering <[email-masked]>
Date: Wed, 29 Jul 2020 17:35:00 +0200

Hi,

On Wed, Jul 29, 2020 at 08:18:28AM -0700, Tom Herbert wrote:
> deployed. That's fine, it's their prerogative. 

And by that, you acknowledge that using "EH across the wide Internet"
is not going to work.

Nobody challenged that the use of EH in a closely controlled environment
("my network and all the partner networks that I have a contract that
governs that we'll cooperate regarding these EHs") will work.

[..]
> provide something that works and provides value to users.

EH has turned out to be costly to people that have no way to recover
these costs as their customers have no interst.

Now, if Google would show up and say "all web sites that demonstrate
use of the TurboEH are ranked up by 20%", you can bet that SEOs would
be the first to demand working and fully transparent EH support, no
matter the cost...

Gert Doering
        -- NetMaster

---

From: Tom Herbert <[email-masked]>
Date: Wed, 29 Jul 2020 08:59:37 -0700

Gert,

Well, if by "works across the wide Internet" you mean that there is a
100% probability that packets with extension headers will reach every
possible destination over every possible path then, yes, "EH across
the wide Internet" is not going to work. However, by this same
definition IPv6, UDP, SCTP, IP fragmentation, ICMP, and pretty much
anything except possibly the most rudimentary plain TCP/IPv4 packet
don't work "across the wide Internet" either. We know ubiquitous
support for protocols across the whole Internet is not feasible, it's
pointless to assume a requirement for a protocol that it must be
universally supported everywhere before it can be useful.

> these costs as their customers have no interst.

I don't expect customers to have any interest in extensions headers, I
do expect they'll have interest in services they help provide.

> be the first to demand working and fully transparent EH support, no

Yep. This was clearly demonstrated by Google's success in pushing
operators to support QUIC. If I was endeavouring to bring up a new
protocol or extension header on the Internet I'd certainly start with
getting a major content provider on board even before service
providers.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Wed, 29 Jul 2020 09:04:10 -0700

> On Jul 29, 2020, at 3:50 AM, Fernando Gont <[email-masked]> wrote:
> 

We’ve spoken about this before, but for the group:

There is a tension between “document and acknowledge reality” and “adhere to standards to ensure interoperation”.

Not every “reality” represents something that should change a standard. Some can - and should - be declared as “bugs” or “errors” and called out for doing so.

If we are merely documenting what happens to be implemented, we cease to be a standards body and become merely reporters.

Joe

---

From: Owen DeLong <[email-masked]>
Date: Wed, 29 Jul 2020 09:32:04 -0700

> On Jul 29, 2020, at 09:04 , Joseph Touch <[email-masked]> wrote:
> 
> 
> 
>> On Jul 29, 2020, at 3:50 AM, Fernando Gont <[email-masked] <mailto:[email-masked]>> wrote:
>> 

Yes, so let’s do that… Let’s have this document represent the current situation and then call out which parts of that situation are considered to be bugs/errors
and which parts we believe represent “opportunities” for improvements in the standards.

> If we are merely documenting what happens to be implemented, we cease to be a standards body and become merely reporters.

If we avoid any introspection or consideration of operational reality, the cease to be a relevant standards body and become an ivory tower.

Owen

---

From: Joseph Touch <[email-masked]>
Date: Wed, 29 Jul 2020 13:16:47 -0700

> On Jul 29, 2020, at 9:32 AM, Owen DeLong <[email-masked]> wrote:
> 
> ...Let’s have this document represent the current situation and then call out which parts of that situation are considered to be bugs/errors

Agreed.

> If we avoid any introspection or consideration of operational reality, the cease to be a relevant standards body and become an ivory tower.\

That’s why I said “merely”. Doing both and appreciating the balance is fine - the point is that “what is implemented/able TODAY” is NOT the only consideration.

Joe

---

From: Fernando Gont <[email-masked]>
Date: Wed, 29 Jul 2020 23:25:37 -0300

Hello, Joe,

On 29/7/20 17:16, Joseph Touch wrote:
[...]
> That’s why I said “merely”. Doing both and appreciating the balance is fine - the point is that “what is implemented/able TODAY” is NOT the only consideration.

FWIW, I don't think we should limit ourselves to documenting the 
problem.Indeed, documenting the problem can certainly be a starting 
point to consider possible ways to mitigate it.

For a long time, the status quo was assuming that EHs work. More 
recently, thanks to a number of efforts (Geoff's measurements, what we 
did in RFC7872, and others), there has been increased awareness about 
the packet drops.

I would expect that a common understanding that there are underlying 
issues that lead to the packet drops (and it's not just folks playing 
with "firewall" rules at random) can serve as a starting point to 
consider what can be done to make things better, closing the gap (to the 
extent that is possible) between what the IETF says IPv6 is, and the 
operational reality of it.

Thanks,

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 30 Jul 2020 15:52:00 +1200

And it may be another unpopular fact of life, but I believe that RFC8799 is relevant. Certainly some basic rethinking about interoperability is needed.

Regards
   Brian

---

From: Fernando Gont <[email-masked]>
Date: Thu, 30 Jul 2020 01:59:40 -0300

> And it may be another unpopular fact of life, but I believe that RFC8799 is relevant. Certainly some basic rethinking about interoperability is needed.

Indeed, of all the possible outcomes, the worst one is bury one's head 
into sand.

I'm not sure what would be the best part for this particular case 
(whether to embrace the idea of limited domains, and spell out what's 
expected to work where, to constrain the IPv6 spec a bit such that we 
can somewhat close the gap between what 6man specifies and the real 
world, or what else).

Of the possible options, the RFC8799 seems to be among the most 
conservative: acknowledges operational reality, while it doesn't close 
the door.

---

From: Fred Baker <[email-masked]>
Date: Fri, 31 Jul 2020 07:01:31 -0700

If I may make a minor suggestion...

If you're going to throw brickbats at each other, that seems to be more appropriate to private exchanges. 

In 6man, there is a discussion regarding draft-ietf-6man-mtu-option, which looks at the issue of extension headers, and - based on data - makes suggestions of how one might use them in a manner that is not filtered by the network. Their approach seems to not start from "let's deprecate everything" or "any opinion that disagrees with mine is idiocy". Frankly, I think there might be value in it.