Subject: Some stats on IPv6 fragments and EH filtering on the Internet

From: Fernando Gont <[email-masked]>
Date: Mon, 04 Nov 2013 15:01:48 -0800

Folks,

I did a presentation on the topic at the IEPG meeting earlier this week.
It provides some concrete data regarding IPv6 fragmentation and
Extension Header filtering on the Internet.

The slideware is available at:
<http://www.iepg.org/2013-11-ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf>

Certainly there's *much* more work to be done in this area, but I
thought that this could be good food sfor some of the discussions that
we were having on the topic.

Thanks,
-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: Tim Chown <[email-masked]>
Date: Mon, 4 Nov 2013 23:27:12 +0000

Hi,

Also as per the IEPG discussion, the results I had in conjunction with a summer MSc project student can be summarised as follows. 

The headline is that he saw a 37.7% failure rate for the Fragmentation Header (alone), a bit better than Fernando’s results, but still not good.

He tested the top 1,000 IPv6-enabled Alexa sites.
He used the scapy toolkit which supports the four main extension header types (routing, fragmentation, destination and hop-by-hop)
He tested
a) valid combinations of those 4 extension headers as per RFC 2460
b) other non-valid combinations
c) duplicated extension headers
d) fragmentation header
Primarily TCP tests, doing HTTP GET requests.

For single extension headers, acceptance was
Routing header 63.9%
Frag header 62.3%
Hop by hop header 60%
Destination option header 15.8% 
When using no extension headers, success rate was 100%
When using multiple headers, the rates fall markedly, not dissimilar with Fernando’s numbers for longer headers.

About 120 sites accept all four types of extension headers. 

A small number of sites accepted illegal combinations/ordering of extension headers.

A more detailed set of results is being pushed to a conference paper.

I now have another student taking this further, and validating the above results, so feel free to contact me off-list if you’re interested.

Tim

On 4 Nov 2013, at 23:01, Fernando Gont <[email-masked]> wrote:

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 5 Nov 2013 00:32:00 +0100 (CET)

> we were having on the topic.

"Operators are known to filter IPv6 fragments". This kind of writing 
implies intent, right? Most of the "filtering" I would say is due to 
default device behavior, where no intent exists on the operator. So I'd 
rather see this changes to "IPv6 fragment header packets are having a 
severely degraded network experience" or something like that.

Otherwise, great work. I would like to see traceroute or alike done with 
these kinds of packets headers also, to see where packets are dropped.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Fernando Gont <[email-masked]>
Date: Mon, 04 Nov 2013 16:00:53 -0800

> "Operators are known to filter IPv6 fragments". This kind of writing

Isn't there a v6ops I-D with similar wording?

> implies intent, right? 

mm... as far as discussion on a number of forums went, apparently
there's intent to do so.

> Most of the "filtering" I would say is due to
> default device behavior, where no intent exists on the operator.

Data? References?

> So I'd

Well, the preso is out there, already. :-(

> these kinds of packets headers also, to see where packets are dropped.

Yep. This is in my TODO list. Hopefully we will cooperate with Tim
(Chown) et al to take these measurements further.

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 5 Nov 2013 01:08:18 +0100 (CET)

> Isn't there a v6ops I-D with similar wording?

I opposed to this wording in there as well.

If one deploys a 6500/7600/SUP720 and configures IPv6 on it today, it'll 
slow-path every IPv6 packet with a fragmentation header. Default 
behaviour.

I have no idea how much of the behaviour your're seeing is caused by this 
default behaviour, but I'd venture to guess a lot.

> mm... as far as discussion on a number of forums went, apparently 

Oki, I haven't seen these.

> Data? References?

http://seclists.org/nanog/2011/Sep/1076

---

From: Mark Andrews <[email-masked]>
Date: Tue, 05 Nov 2013 11:12:43 +1100

> mm... as far as discussion on a number of forums went, apparently

There is also often no intent.  The defaults drop fragmented packets
any you have to explicitly pass them.

> > default device behavior, where no intent exists on the operator.

*BSD firewall drop by default.  For all flavours of firewall supported.
e.g. ipfw, ipf.
 
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Pedro Torres <[email-masked]>
Date: Mon, 4 Nov 2013 23:53:38 -0200

Tim/Fernando,

Wow! I'm scared of these results!
(If that was the intention, it worked!)

--
Pedro

---

From: Fernando Gont <[email-masked]>
Date: Mon, 04 Nov 2013 18:45:29 -0800

Hi, Pedro,

> (If that was the intention, it worked!)

The intention certainly wasn't to scare anyone -- for instance, I didn't
expect these results ot be that bad.

The goal of this measurements was to get some data regarding on where
we're standing with respect to IPv6 fragmentation and extension headers.

This is just data that is useful and should keep in mind when developing
protocols and/or extensions.

Thanks,

---

From: Tim Chown <[email-masked]>
Date: Tue, 5 Nov 2013 02:50:41 +0000

> we're standing with respect to IPv6 fragmentation and extension headers.

And to encourage further measurements, which I think Fernando will do via his scripts (which I think are part of his wider set of IPv6 tools) and which a student here will do via use of and/or extending scapy.

We can then get some idea of where the problems exist, and whether, for example, there are ICMPv6 messages returned or the packets are simply being dropped.

> protocols and/or extensions.

Indeed, or when writing docs like the one Ron is authoring on extension header handling.

Tim

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 5 Nov 2013 02:57:34 +0000

Hi Tim and Fernando,

> > protocols and/or extensions.

We already have at least one protocol that requires IPv6 fragmentation
(RFC2473) and I'm sure there are others.

> Indeed, or when writing docs like the one Ron is authoring on extension header handling.

Or SEAL also. But, SEAL simply takes the existing precedent set by RFC2473.

Thanks - Fred
[email-masked]

---

From: Hannes Frederic Sowa <[email-masked]>
Date: Wed, 6 Nov 2013 14:01:40 +0100

Hi Fernando,

> we were having on the topic.

We had some some bugs in path MTU handling in the linux IPv6 stack which
will get resolved with the next stable kernels.

If such an experiment is repeated I would suggest keeping a tcpdump of
the traffic and an ip -6 monitor route log while doing this experiment.

I currently don't know if we report path MTU events back up to user space,
and if not, it will be hard to do so because that would confuse user space
routing daemons which already listen for route change notifications. But
for such an experiment I could provide such a patch. It would be nice
that we don't drop path mtu discovery on such busy boxes because of
regular routing garbage collector pressure or just because of bugs.

Greetings,

  Hannes

---

From: Fernando Gont <[email-masked]>
Date: Wed, 06 Nov 2013 18:53:12 -0800

Hi, Michael,

> I opposed to this wording in there as well.

Then I apologize -- I kind of assumed some sort of intention based on
that I-D. So.. point taken.

That said, the meat is in stats. :-)

> slow-path every IPv6 packet with a fragmentation header. Default behaviour.

But slow path != drop.

> I have no idea how much of the behaviour your're seeing is caused by
> this default behaviour, but I'd venture to guess a lot.
> 

Quickly skimming through the discussion it looks like a decision is made
to drop -- albeit for reasonable reasons. Am I missing something?

Thanks,

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 7 Nov 2013 15:05:00 +0100 (CET)

>> slow-path every IPv6 packet with a fragmentation header. Default behaviour.

Since this platform does few megabits/s in slowpath, it's pretty close. 
Most reasonable people will deploy CoPP here and set low policer limits 
for this kind of traffic.

> to drop -- albeit for reasonable reasons. Am I missing something?

I'm sure there are some who set it to drop, and some set it to forward 
(both for good reasons), and some don't know and leave the default.

---

From: Ronald Bonica <[email-masked]>
Date: Mon, 18 Nov 2013 21:50:39 +0000

Folks,

Fernando presents two studies in <http://www.iepg.org/2013-11-ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf>. The second study is more interesting to me, because duplicate addresses are removed.

The following are a few questions regarding Fernando's second study:

1) Fernando observes that 41% of sites discard fragmented packets. However, in a similar study (http://www.nlnetlabs.nl/downloads/publications/pmtu-black-holes-msc-thesis.pdf) only 10% of sites discarded fragmented packets. I wonder why the two studies yield such divergent results.

2) Fernando observes that 44% of sites discard packets containing an 8 byte destination header, while 89% of sites discard packet containing 1 kilobyte of extension headers. Because the first number (44%) is so high, can I conclude that the second (89%) is insignificant.

Could it be that extension header length is a non-issue, because so many sites filter packets containing extension headers, regardless of their length?

                                                          Ron

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Fernando Gont
> Sent: Monday, November 04, 2013 6:02 PM
> To: [email-masked]; IPv6 Operations
> Subject: Some stats on IPv6 fragments and EH filtering on the Internet
> 
> Folks,
> 
> e-mail: [email-masked] || [email-masked] PGP Fingerprint:
> 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
> 
> 
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 19 Nov 2013 17:35:25 +1300

> 1) Fernando observes that 41% of sites discard fragmented packets. However, in a similar study (http://www.nlnetlabs.nl/downloads/publications/pmtu-black-holes-msc-thesis.pdf) only 10% of sites dis…

Well, that depends on whether there are significant differences between the
observation points and/or the set of target sites for the two studies. There's
certainly no law of physics that prevents both measurements being correct.

> Could it be that extension header length is a non-issue, because so many sites filter packets containing extension headers, regardless of their length?

I don't think so. We've just agreed on an update to RFC 2460 that, if it is
adopted by middleboxes, will fix or at least clarify the issue of what
happens to short extension headers. We need to wait a few years to see
if that happens, of course. That seems to me to be quite disjoint from
the issue of whether boxes that inspect extension headers have big
enough buffers, and distinct again from whether they handle fragmented
headers. I think there are three problems, needing three solutions.

   Brian
                                                          Ron

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 19 Nov 2013 15:29:00 +0000

Brian,

I see your point. If we fix the problems associated with extension headers, people will configure their middle-boxes to allow them. So, if we repeat Fernando's experiment in a few years, we may see very different results. 

You say that there are three distinct problems. I am aware of the following:

- the tiny fragment problem (addressed by draft-ietf-6man-oversized-header chain)
- the long header chain problem (addressed by draft-kumari-6man-long-headers)

What is the third problem?

                                              Ron

> From: Brian E Carpenter [mailto:[email-masked]]
> Sent: Monday, November 18, 2013 11:35 PM
> To: Ronald Bonica
> Cc: Fernando Gont; [email-masked]; IPv6 Operations
> Subject: Re: [v6ops] Some stats on IPv6 fragments and EH filtering on
> the Internet
> 
> > Fernando presents two studies in <http://www.iepg.org/2013-11-
> ietf88/fgont-iepg-ietf88-ipv6-frag-and-eh.pdf>. The second study is
> more interesting to me, because duplicate addresses are removed.
> >
> > 1) Fernando observes that 41% of sites discard fragmented packets.
> However, in a similar study
> (http://www.nlnetlabs.nl/downloads/publications/pmtu-black-holes-msc-
> thesis.pdf) only 10% of sites discarded fragmented packets. I wonder
> why the two studies yield such divergent results.
> 
> Well, that depends on whether there are significant differences between
> the observation points and/or the set of target sites for the two
> studies. There's certainly no law of physics that prevents both
> measurements being correct.
> 
> > 2) Fernando observes that 44% of sites discard packets containing an
> 8 byte destination header, while 89% of sites discard packet containing
> 1 kilobyte of extension headers. Because the first number (44%) is so
> high, can I conclude that the second (89%) is insignificant.
> 
> > Could it be that extension header length is a non-issue, because so
> many sites filter packets containing extension headers, regardless of
> their length?
> 
> I don't think so. We've just agreed on an update to RFC 2460 that, if
> it is adopted by middleboxes, will fix or at least clarify the issue of
> what happens to short extension headers. We need to wait a few years to
> see if that happens, of course. That seems to me to be quite disjoint
> from the issue of whether boxes that inspect extension headers have big
> >> From: [email-masked] [mailto:[email-masked]] On Behalf
> >> Of Fernando Gont
> >> thought that this could be good food sfor some of the discussions
> >> that we were having on the topic.
> >>
> >> Thanks,
> >> --
> >> Fernando Gont

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Nov 2013 08:26:56 +1300

> What is the third problem?

 - middleboxes that don't deal with all header types (addressed by draft-ietf-6man-ext-transmit)

   Brian