Subject: PMTU blackhole detection

From: Philip Homburg <[email-masked]>
Date: Sat, 16 Apr 2011 11:17:02 +0200

PMTU blackhole detection seemed so obvious to me, that I never bothered to
find out if there was an RFC specifying that it should be done. That is, until 
I encountered an admin said that he didn't do PMTU blackhole detection because
end-users should just fix their systems (the irony is that this came up when
said admin's systems where broken).

So now I wonder, is this something that should be discussed in 
draft-ietf-6man-node-req-bis-08.txt?

---

From: Philip Homburg <[email-masked]>
Date: Sat, 16 Apr 2011 12:44:17 +0200

In your letter dated Sat, 16 Apr 2011 11:28:53 +0100 you wrote:
>On Sat, 2011-04-16 at 11:17 +0200, Philip Homburg wrote:
>> 
>> find out if there was an RFC specifying that it should be done.
>
>Conversely...
>
>It's always been *so* obvious to me that firewalling ICMP is broken,
>that I never bothered to find out of there was an RFC specifying that it
>shouldn't be done.
>
>Perhaps *that* is what should be discussed, instead?

I think that's fine. I can understand the argument that breaking things causes
bugs to be resolved.

On the other hand, the difference between 1500 and 1280 is so small, I wonder
if breaking things just because you want to send packets at 1500 bytes makes
a lot of sense.

One other thing, if this makes the IPv6 experience worse than industry
standard for IPv4, then maybe it is also not a good idea.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 16 Apr 2011 15:28:09 +0200 (CEST)

> find out if there was an RFC specifying that it should be done.

<http://tools.ietf.org/html/rfc4821> is what you're looking for I guess?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Philip Homburg <[email-masked]>
Date: Mon, 18 Apr 2011 13:00:30 +0200

><http://tools.ietf.org/html/rfc4821> is what you're looking for I guess?

I guess so. I wonder why google couldn't find it. 

I'm not sure I like their approach. I'd prefer to treat PMTU blackholes as
an exception but they treat it as the normal case.

But I guess that a node requirements RFC should refer to it.

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 18 Apr 2011 11:45:45 -0700

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Philip Homburg
> Sent: Saturday, April 16, 2011 3:44 AM
> To: David Woodhouse
> Cc: [email-masked]
> Subject: Re: PMTU blackhole detection
> 
> On the other hand, the difference between 1500 and 1280 is so small, I
> wonder if breaking things just because you want to send packets 
> at 1500 bytes makes a lot of sense.
>

The IPv6 headers are 20 bytes bigger than IPv4 headers, so a fairer
comparison is 1500 against 1260 (1260=1280-20).  That is, with a 1500
byte MTU with IPv4, the effective data payload is 1480 bytes (assuming
no IP options, which is a reasonable assumption with IPv4); with a
1280 byte MTU with IPv6, the effective data payload is 1240 bytes 
(assuming no IPv6 extensions).  That's a 16.2% reduction in data 
payload size from IPv4 to IPv6, with a commensurate increase in 
number of packets to send the same data (assuming MTU-sized packets).

This isn't quite "packets per second will increase by 16.2%", though,
as of course not all packets are 'full'.  But there will be a pps
increase.

-d

---

From: Philip Homburg <[email-masked]>
Date: Tue, 19 Apr 2011 10:28:35 +0200

>as of course not all packets are 'full'.  But there will be a pps

Yes, but is avoiding that small increase worth the complexity that things
break whenever an ICMP doesn't get delivered?

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 19 Apr 2011 08:47:23 -0700

> From: [email-masked] [mailto:pch-b6B5344D9@u-
> 1.phicoh.com] On Behalf Of Philip Homburg
> Sent: Tuesday, April 19, 2011 1:29 AM
> To: Dan Wing
> Cc: [email-masked]
> Yes, but is avoiding that small increase worth the complexity that
> things break whenever an ICMP doesn't get delivered?

No, not yet.

But it is a problem worthy of getting fixed.  We have slid backwards
on the PMTUD problem with IPv6.  We need to move the other direction
(forwards).

-d

---

From: Bob Hinden <[email-masked]>
Date: Tue, 19 Apr 2011 16:19:56 -0700

Dan,

> number of packets to send the same data (assuming MTU-sized packets).

I am a little confused by the comparisons being made in this thread.

There is no guarantee that an 1500 IPv4 packet won't be fragmented, so a path that drops ICMPv4 packet too big messages will cause PMTU to fail.  The 1280 number is the size of an MTU that IPv6 traffic can go on with out a need to be fragmented (that is, no PMTU issue).

If a path can deliver 1500 byte IPv4 packets, it can also deliver 1500 byte IPv6 packets.  The resulting payloads will be 20 byte less for IPv6, but that less than a 2% difference in payload size.

I doubt middle boxes are going to let ICMPv4 packet too big messages through, and drop ICMPv6 packet too big messages.

Am I missing something here?

Bob

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: "Dan Wing" <[email-masked]>
Date: Tue, 19 Apr 2011 18:05:35 -0700

> From: Bob Hinden [mailto:[email-masked]]
> Sent: Tuesday, April 19, 2011 4:20 PM
> To: Dan Wing
> Cc: Bob Hinden; 'Philip Homburg'; 'David Woodhouse'; [email-masked]
> There is no guarantee that an 1500 IPv4 packet won't be fragmented, so
> a path that drops ICMPv4 packet too big messages will cause PMTU to
> fail.  The 1280 number is the size of an MTU that IPv6 traffic can go
> on with out a need to be fragmented (that is, no PMTU issue).
> 
> If a path can deliver 1500 byte IPv4 packets, it can also deliver 1500
> byte IPv6 packets.  The resulting payloads will be 20 byte less for
> IPv6, but that less than a 2% difference in payload size.
> 
> I doubt middle boxes are going to let ICMPv4 packet too big messages
> through, and drop ICMPv6 packet too big messages.
> 

Yes.  Over the years, the IETF's own website has suffered two outages
(and perhaps three) that were attributed to IPv6 PMTUD failures.  To
my knowledge, it has suffered no outages attributed to IPv4 PMTUD
failures.

Google runs its IPv6-facing properties using a 1280 byte MTU, likely
to avoid adding IPv6 PMTUD failures to the reasons that IPv6 might
provide a worse user experience than IPv4.  Afterall, there is no
need to try to resolve all problems at once while we're getting IPv6
off the ground.

So, while I agree that 1500 should work equally well for both 
IPv4 and IPv6, it seems that for whatever combination of reasons, 
it works less well on IPv6 than IPv4.  This is perhaps user error 
(configuration mistakes on firewalls or ACLs), more common use 
of IPv6-over-IPv4 tunnels than IPv4-over-IPv4 tunnels, bugs 
in vendor equipment, or something else.

-d

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 20 Apr 2011 15:01:05 +1200

> in vendor equipment, or something else.

Specifically, I discovered while I was a 6to4 user that some
sites, but not others, could successfully complete a SYN/ACK
exchange (where all the packets were well below 1280 bytes) but
never replied to the first HTTP GET command. iirc, it seemed
that the server would set its TCP MSS to 1440, regardless of the
client setting 1220, and presumably the server sent 1500 byte
IPv6 packets that never arrived. I never could debug it.

In this situation a 6to4 relay (like any other tunnel end point)
should behave according to section 3.2 of RFC 4213. That's quite
a complicated section and I suppose there may be buggy
implementations, even in the absence of ICMP filtering.

   Brian

---

From: Mohacsi Janos <[email-masked]>
Date: Wed, 20 Apr 2011 09:43:35 +0200 (CEST)

> I doubt middle boxes are going to let ICMPv4 packet too big messages through, and drop ICMPv6 packet too big messages.

Then middleboxes does not follow RFC 4890:
Recommendations for Filtering ICMPv6 Messages in Firewalls

Best Regards,
 		Janos Mohacsi

---

From: Philip Homburg <[email-masked]>
Date: Wed, 20 Apr 2011 10:56:11 +0200

>implementations, even in the absence of ICMP filtering.

That illustrates what I'm trying to say. Sending packets with an MTU of 1280
is always safe. So to be able to send packets with an MTU of around 1500,
we have a lot of complated machinery that has been shown to be very fragile,
both in IPv4 and IPv6.

And all we get from that is a rather modest efficiency gain. At the cost of
breaking things completely when it goes wrong.

So I would suggest that we strongly recommend to host operating system
implementors and content providers that they make sure that a sending system's
MTU is quickly reduced in the presence of an MTU blackhole.

---

From: "Dan Wing" <[email-masked]>
Date: Wed, 20 Apr 2011 08:42:41 -0700

> From: Brian E Carpenter [mailto:[email-masked]]
> Sent: Tuesday, April 19, 2011 8:01 PM
> To: Dan Wing
> Cc: 'Bob Hinden'; [email-masked]
> >>>> On the other hand, the difference between 1500 and 1280 is so
> small,
> >> I

Ben Stasiewicz has done some research on IPv6 MTU problems,
http://ripe60.ripe.net/presentations/Stasiewicz-Measurements_of_IPv6_Path_MTU_Discovery_Behaviour.pdf

-d

---

From: Bob Hinden <[email-masked]>
Date: Wed, 20 Apr 2011 14:24:58 -0700

Dan,

> in vendor equipment, or something else.

Thanks, this was helpful.  I suspect that native (dual stack) end-to-end probably works as well as IPv4, but see that the various tunneling solutions don't.

Bob

---

From: Bob Hinden <[email-masked]>
Date: Wed, 20 Apr 2011 14:26:08 -0700

Dan,

> http://ripe60.ripe.net/presentations/Stasiewicz-Measurements_of_IPv6_Path_MTU_Discovery_Behaviour.pdf

Very interesting.  A good next step would be to figure out what caused each type of failure.

Bob

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Apr 2011 10:17:16 -0700

IMHO, IPv6 links that occur in the network (even tunnels)
should try to do better than 1280 if possible. Otherwise,
VPNs/tunnels that ride over the links may see a too-small
MTU resulting in IPv6 fragmentation. Granted, doing better
than 1280 may be difficult for some links.

Thanks - Fred
[email-masked]

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Bob Hinden
> Sent: Wednesday, April 20, 2011 2:26 PM
> To: Dan Wing
> Cc: [email-masked]; Bob Hinden; 'Brian E Carpenter'